How do I reliably determine the “mostly likely” color of a pixel given an RGB tuple?

I’m trying to find a way to interpret Analyst Recommendation Information available on nasdaq.com.
I think the right way to approach this using PIL and count pixels of different colors within the image.
(White, Black Red, Yellow, Green).
The issue (I think) is all the slight variation in the RGB values of pixels.
How do I reliably determine the “mostly likely” color of a pixel given an RGB tuple.