How to sort custom entries in the color picker app from macOS

There is no inherent sort feature in the color picker, or its color palettes panel. However, palettes that you have created are stored in your /Users/yourname/Library/Colors folder. To force a (confirmed) ordering of these .clr palettes in the Color Picker's palette selector menu, you prepend each existing name with a "nn_" string where single digits are zero padded.

Before I forced the prefix on those three menu items, they appeared in a different order. You may want to use a 3-digit prefix so you can "insert" future palettes into the sort order without going nuts resequencing the existing 2-digit prefixes.

Click on your Desktop, and then press shift+command+G to open the Go to Folder dialog. Copy/paste the following into that dialog:

~/Library/Colors

and a Finder Window will open displaying your palettes, which you can selectively rename to your order preference.

You may wish to send Apple direct feedback about a color picker enhancement as you need, and the reason why it would add value to the color picker.

Now, that is frustrating.

I looked at the palette order on another Mac running Mojave 10.14.6, and the (unrenamed) palette names are organized in perfect alphabetical order. It is not attempting to order by creation/modification date, or size.

I am located in the U.S. with a U.S. English installation of macOS, and this may be playing a role in the differences we observe for the sorting behavior in the palette menu. Based on what you have already done, I see no solution for you in achieving the desired menu organization.

I suspect it is the use of white-space, Punkt, and different capital letter locations in the palette names that is confusing the expected sort results.

My example was on Mojave 10.14.6 (18G4032). Here are the renamed custom palettes in my local Library/Colors folder:

and again from my color palette (third from left) panel. The right-most three are third-party tools.

I don't understand the individual palette name truncation that you are experiencing, if they appeared on this menu correctly before, unless their length plus the prefix is exceeding some internal maximum string length setting that is not user changeable.

You are welcome.