Can I delete music from iTunes and my "Download File" at the same time?

Merry Christmas everyone! I'm sitting on the couch trying to clean up my iTunes library and I can't figure out an easy way to delete my music. It seems like it's in two places..... It's in my downloads folder and in my iTunes library.

For example, I've got a Grateful Dead concert that I downloaded (legally) and it shows up in my iTunes library and in my downloads. If I erase the file in my downloads it still shows up in Itunes and vice versa....

What is the best way to clean up my music? It seems like a pain to have to delete a song in downloads and then erase it again in itunes. Any suggestions?

Thanks.

Tom