The shuffle connects to the computer with a long USB cable that the device sits in.
That sounds like a 2nd gen shuffle. You need to use that USB docking cable, with a cradle for the shuffle and long cord. The ultra-short cable that comes with 3rd and 4th gen shuffle does not work with 2nd gen shuffle (connector is different).
NOTE: 1st gen shuffle is white, does not have a built-in clip, and looks like a USB flash drive (with a built-in USB plug).
Your other newer shuffle currently connects to iTunes without causing a similar problem?
The iPod appears in Windows as a USB storage device but I keep getting the message that the device is in use when I try to disconnect it.
That sounds normal, if iTunes is not running and the shuffle is acting like a USB flash drive. You can try reformatting the shuffle using Windows, like it's a USB flash drive. This will obviously erase the shuffle and its currently loaded songs. You can look up how to format drives using Windows, if you have not before. iTunes should not be running. For the format type (or file system), select FAT32 (not NTFS or exFAT). Do not use the option to do a quick format. The name you use for the volume does not matter, you can just call it IPOD.
After the shuffle is reformatted, it is completely blank. It does not have its onboard software, so you'll need to do a Restore on it using iTunes. Run iTunes. Hopefully, it no longer causes the "freeze" issue, and the shuffle appears in iTunes. Even if iTunes does not prompt you, do a Restore on it
Use iTunes on your Mac or PC to restore your iPhone, iPad, or iPod to factory settings - Apple Support
NOTE: If the shuffle has a hardware problem, but it currently still plays music, it will no longer operate as an iPod after reformatting with Windows. If you are not able to use iTunes to Restore it, and then reload it with songs, you will have a non-functional iPod shuffle.