Q: How do I permanently delete songs from my computer

I have an HP computer with Windows 7 and the latest version of iTunes.  My computer will barely operate because I have filled the storage space full of music.  I now wish to go in an purge about half of this old music out of iTunes and off of this computer so that it frees up memory space.  I have Googled "How to delete songs from iTunes" and those suggestions aren't working for me.  Once I hit delete it isn't asking me the second question of "keep file or send to recycle bin."  Last night I deleted about 5500 songs, exited out of iTunes, reopened iTunes and they were all right back in my library.  All of that time was a complete waste. How do I permanently delete these songs from iTunes and my computer at the same time?

Thanks,

Wayne