Majority of my library deleted

Whilst trying to sort out my library of some songs I no longer want, I deleted 2 albums which didn't properly delete, after 5 minutes or so they went. But also half my library.

I went to play an album and it could not locate it so I went in to Finder and located that specific album, all it had was the album art etc. No mp3's.

I didn't freak out too much because I use Time Machine and its all backed up. What has made me angry is the fact that it won't let me restore my library. Comes up with an error message saying there isn't enough space.

So to save time and hassle I've deleted my entire iTunes folder and I've made sure there is enough space. My iTunes library is 105GB and I have 200GB space left on my MacBook Pro, so I see no reason as to why it won't simply restore the folder back.

Any ideas?