Yeah... AND consider how it got there (from INSIDE your Shuffle).
This is a good point to bring up. Now there is a chance that the corrosion was simply sweat reacting to the metal then once it's clean and you work to keep the sweat off of it than you shouldn't see it return. This would be like how some peoples skin reacts to gold plated jewelry and causes the piece to tarnish and discolor the skin green.
However if by chance it continues to appear there might be a chance that the battery has been damaged and it is leaking. In my experiences I often see a green, white(ish), or brown (dirty yellow) when I come across leaking batteries.
I would suggest against CLR because of the remote fact that this problem is caused by a leaking battery. The solutions I listed earlier leave less of a chance for a chemical reaction. Battery Terminal Cleaner is made to clean a car battery.
CLR is used on lime scale and rust usually on household items (coffee pots, silverware, and sinks) or the side of a house. The chemical contents in CLR may have a bad reaction to a leaking battery and could cause a release of toxic fumes.
I don't know if that would be the result but the risk is there and as such you should take precautions to ensure your safety. Your life is worth more than a consumer electronic device.
I am not a chemist but I would try erasers and baking soda before moving up to the more powerful cleaning agents. For one it's easier on the device as well as being cheaper.