Q: How can I copy my entire library to a new computer?

Correct. Only new purchases made directly on the iPod will transfer.  Anything that was synced to the iPod from a computer will not.

If you still have your old computer:

How to move your iTunes library to a new computer - Apple Support

If you no longer have the old computer to access your library from and transfer it the new one, you will have to use a 3rd party App to extract the contents of your iPod to the new computer.  One such as iexplorer can help you do that.

http://www.macroplant.com/iexplorer/

See also Recover your iTunes library from your iPod or iOS device.

tt2