I know that Apple Color is now in Final Cut Pro X, but I don't want to buy that whole thing. Where can I download Apple Color by itself?
You can't. Color as a stand alone app was only part of the Final Cut Studio collection of applications. It was never sold independently.
If you want to use it, best bet is to find a reseller with FCS 2009 (aka FCS3) for sale on ebay or other similar sites. Caveat Emptor applies.
FWIW -Color is not part of FPX. Maybe some aspects were incorporated but nothing near the power of the program.
If you don't want to find FCS3 to work with Color, your next best bet is to get involved with DaVinci Resolve.