Your PC may have an "energy-saver" feature that turns OFF full power to the USB port, when the connected device is unmounted by the system. Unfortunately, when this happens, the USB port is not able to charge the iPod's battery (and the light is OFF too). You may have settings in your computer's Control Panels to disable this feature, so that the USB port maintain full power.
Otherwise, try this workaround. Select the shuffle in iTunes (click its device button) to show its Summary settings screen in iTunes window. Find the checkbox for Enable disk use, and checkmark it. Click Apply. This setting tells iTunes to keep the iPod's disk mounted continuously, even when not syncing. If the shuffle's disk is mounted, it should have full power and its battery should charge. And the light should stay ON, although it will be blinking orange continuously. With disk use enabled, you'll need to eject the shuffle in iTunes before disconnecting.
There is an eject button next to the iPod's name in iTunes. After a few hours, the battery should be fully charged. Click the eject button, and disconnect the shuffle. When playing the shuffle, press the shuffle's battery light button on the back. When you see green, that mean the battery has a good charge - more than 25%. When you see orange, that means less than 25%. When you see red, that means very low charge.
NOTE: You can also charge the shuffle using a USB power adapter, like the one that comes with iPhone or other device. The shuffle's light is always solid when charging with a power adapter.