Adding one important fact - computing color distance between two colors in order to identify similar colors doesn't work well in any color model (often matching colors with reduced vibrance). What you need to do is to define overlapping color regions (e.g. this is yellowish, that is greenish etc.) and treat this information as more important than the computed distance for similarity matching.
Yes, that's an attempt to solve this problem however if you try to use it for color matching you'd find problematic areas there as well, hence you still need to use color regions.
