There are actually three "Library" folders:
Although there appear to be duplicate named items in them, these are not duplicates. Furthermore, except for the /Home/Library/ folder you should not be doing anything in the others. They are reserved for system usage only. If you start removing stuff you don't know anything about soon you will be erasing your drive and starting over from scratch. A word to the wise.
The /Home/Library/ folder contains files and folders essential to the proper use of your user account and applications' support and preferences files. Except to fix a problem or uninstall some software there's no reason for you to muck around in this folder either.
So, leave well enough alone.