Interesting. Thanks for the quick reply!
So knowing that I currently have mostly these same files in 2 places - 69,000 or so transfered into the external drive when I copied the desk folder into iTunes and then 70,000 or so still sitting in the desktop folder - when I consolidate the library like that will I suddenly have 159,000 songs in my library with most of them being duplicates??
I can only work by deconstructing your process.
Copied my external drive to my iTunes.
Okay, assuming you use iTunes' default settings to copy items to your iTunes folder when adding, and to organize them, and to have your media folder on Home > Music > iTunes > Media, this put copies of all the files in your internal drive.
Deleted all those files - they ended up in the trash. The external drive was now empty.
Emptied your external as you say, but you didn't really - see later.
Copied the files in the trash to my desktop.
Ouch. Your trash as you see it is really a combination of all the individual trash folders on each drive attached at the time of viewing the trash. So if you dragged items to the trash on the external, didn't actually empty the trash, all the items are still in it. If you then drag them to your desktop it copies them into your user desktop folder on your internal drive.
Copied the files on the desktop into iTunes.
By doing this you then added the second copy of the files to iTunes, likely ending up with many duplicates.
The external drive is now filled up with most of the songs on my library.
I thought you had added all these to your internal drive. If you're not using iTunes' default settings you need to tell us at the start.
some of those 2000 files are on the external drive, but some files for some reason did not make the transfer.
I thought your external drive was empty.
Thanks for responding - let me try to answer some of those questions in detail. I think the confusion was me trying not to give too much detail.
So my initial set up was iTunes with all files in the external drive. When the hard drive crashed, I had to reconstitute that set up. Enter Apple Care.
What really happened with AppleCare was I had to work with 3 different reps - the rep I got kept having the next day off. (They were all pretty helpful btw - they just had different opinions about how to get this done.)
So the first rep dragged and dropped everything from my external hard drive back into iTunes (which had obviously been empty since the hard drive had been reinstalled).
The second rep didn't think we needed to copy all those songs into iTunes, so asked me to delete all 70,000; when I asked her if I should send those files to the trash, she said yes and was hurrying off to another call but was calling me back to check in, in 30 minutes. When she called back, she was lightly panicked and asked if I had emptied the trash. I had considered it, but hadn't done it. Then she asked me to check my external drive and lo and behold it was mostly empty. (That's when I started to panic.)
We had to take the files from the trash and put them on the desktop. We tried to drag and drop them directly back into iTunes but that didn't work for some reason.
So we created a file on the desktop that now contains all the files that were taken out of the trash. We started dropping files from the desktop file into iTunes 5-10,000 at a time. Doing that added copies to the external drive and filled it back up.
Rep #3 came next. The library was now full but there was no ratings, playlists, etc. He found the library .xml file (thankfully!). We restarted using that .xml which repaired most of the problems.
Still, there were these 2000 exclamation points. He told me that I had to go back one-by-one and fix each.
I found an article on here that suggested I create one playlist by dragging and dropping everything in my library into a playlist - called it "Not Missing." Then I created a smart playlist of all songs not in the Not Missing file - I called it "Exclamation Points." So all the songs with broken links are in this playlist. And for the last week, I've been going song-by-song trying to repair them.
When I click on the exclamation point, one of two things happen:
- Sometimes a copy is found on the external drive, but iTunes isn't finding it for some reason. Whenever this happens, there is always also a copy on the Desktop file. I have usually been linking iTunes to the copy on the external drive, but NOT ALWAYS. Because for some reason, when I choose the desktop file, iTunes asks me if I want to look for other missed files at that location - and, usually not, but sometimes it will find a bunch of other related files automatically, thus saving me having to find those files myself manually.
- Sometimes a copy is ONLY found in the Desktop file. So I link to that in that case.
What I've discovered, now that I've done about 700 of these "repairs" is that the group of songs that I linked from the desktop file are not copying into the external drive when I click "open" during the "location" process.
My current solution is that whenever I find a song that is only in the desktop file, I cancel out of the "location" process through iTunes and go into Finder, locate the file in the Desktop file, and then drag and drop it into iTunes. That way a copy goes to the external drive.
So my problem is how do I go back through those 700 other files that I've connected to iTunes and figure out which of them are linking to the desktop file (and thus have not created a copy in the external drive). Because my ultimate idea is to delete the desktop file since it's mostly copies.
Sorry to go on and on, but I hope that helps you help me.
Any ideas how I can either do this whole thing more efficiently or find which songs I've just linked to the desktop file and copy those songs into the external drive?
I have to admit I am bewildered and a bit lost in all the initial file moving.
A few factors:
- iTunes works through a database file, library.itl, only readable by iTunes. It has a partner file called library.xml which is readable by other applications. The .xml file contains most but not all the information in the .itl file. Still, if something happens to .itl the .xml can be used to rescue most of the library.
What are the iTunes library files? - http://support.apple.com/kb/HT1660
More on iTunes library files and what they do - http://en.wikipedia.org/wiki/ITunes#Media_management
What are all those iTunes files? - http://www.macworld.com/article/139974/2009/04/itunes_files.html
- Since iTunes works through the database file it does not look directly at your files until you try to do something with them (e.g play them). Nor does it look for files (usually). If you click on a track, iTunes looks up where that track is located in the database files and goes there and plays that file. If the file is not where iTunes thinks it should be you get a broken link. Something happened with all your file moving that some files did not end up back where they should be.
- iTunes does not like you moving files. It will only tolerate this if you have iTunes disabled for managing your music ("copy to" and "organize" turned off in preferences). Even then it will not track a file being moved from drive to drive since moving between drives is really copying and deleting. For most people who have iTunes managing their music, iTunes has no tolerance for you manually moving files and you can easily break a link. With all the moving you did I am surprised you don't have 100% broken links!
- There are various scripts and utilities for working with broken links and duplicates, but from reports I have seen none of them are perfect. iTunes itself will find files if they are still in the folder structure you set up and you just happened to move the top level folder somewhere else. This is when you fix one link and iTunes asks if you want to find others based on the same location criteria.
- One thought, not fully worked out though. If these files have lost copies on the external you can drag the media folder for the whole external drive to iTunes. It will add anything not already in the library and ignore files that are already in the library. This will not re-link the files. That is pretty much impossible except doing it the way you are doing it. However, if in the end it's a matter of getting rid of broken links that is easy with a script from the Dougscripts web site. You may have to visually confirm all your broken links do now have a valid entry equivalent, but that's a matter of sitting down for 10 minutes.
- If you have files only on the internal, if you can isolate which ones those are you can add them to the library and presumably you have iTunes configured to store your media on the external drive (you know, I think I still haven't seen you tell me what your preferences settings are for your media folder location, copy to, and organize, as well as if your library files are on yoru internal or external).
- The bottom line is, there isn't any automated way to do this. Well, you can check out tools but I don't know I would trust them with my collection. I have some truly unique files and I'd rather spend 30 minutes manually checking files than risk losing some of them.
Yeah believe me - all that file moving had me pretty confused too! Not to mention terrified - for a few seconds, we thought we had lost all my music. I think it was kind of a screw up on the part of the AppleCare reps.
I was surprised there weren't more broken links too. At first, I was relieved there were only 2200 but now it's feeling like a pretty big mountain to climb.
And to your question on the organization issue - (you know, I think I still haven't seen you tell me what your preferences settings are for your media folder location, copy to, and organize, as well as if your library files are on yoru internal or external).
I'm set up so my itunes library is automatically copying to the external drive. I think the media folder location and copy to/organize files are all set to the external drive too. Same with the library files. Does that answer the question?
So let me ask you a much smaller question - is there a way (a smart playlist or some search function) to see which files are housed on the external drive and which still on the desktop folder?
It still would help to know where you library files are located, internal or external. The only way I know of doing this without actually setting it up when starting iTunes is to look in any iTunes folders you have and see which one has a library.itl file and if there are multiple folders, see which one has the most recently modified.
Gosh, I think it was only 2 says ago somebody was asking about sorting location, internal/external. No easy way. If you have a spare tag field with no other contents there's:
The only other way I can think of would depend upon where you keep your library files and would involve disconnecting your external drive, starting iTunes, then see which links are broken. Any not broken would have to be on the internal. Make a playlist of that. Not easy, and only works if you keep your library files on the internal.
As you can see, some of this is really a question of scale. I bet I could have hand sorted 500 of your broken links in the time I have devoted to typing replies. I remember somebody wanting a script to rename a dozen files and spent an hour writing it -- could have renamed them by hand in 3 minutes.