Making iTunes library find missing tracks in multiple, not one by one

Adding to my previous info TT2,

When the item is missing the path is ALMOST exactly the same. For some reason, iTunes has added file://localhost in front of what has been the path for a while now. See the missing and found images of 'get info / file' and image below.

Missing

Found

I hope this makes it as simple as possible for you to advise me of a fix. I would really appreciate your assistance. Many thanks in advance :-D

Hi again TT2,

Strange things are afoot.

Every time I open iTunes something seems to change.

Now, it seems to have created a load of duplicate files with different c=variations depending on some or all of the letters being capital letters, this is what I mean.

More disturbingly, these files show they are empty. Various Artists should be over 50gb. When I search for items I know should be in there, it shows the file but without a path or preview pic, eg..

The ones with a picture are on my internal drive (iMac) and the ones with a dashed outline are on the XHDD but cannot be located through iTunes. I started to really panic but it seems they are still there. When I view them as a tile, a play button appears and I can play them.

I've turned on (within Finder) to show hidden files but that didn't do any good in regards to the files howling as "there". It's extra frustrating because a load of the files I located before were in the various artists folder. Could this be anything to do with the fact Ichanged the 3tb XHDD which was partitioned into 2 parts (roughly 2.3tb & 700gb)?

It's totally doing my head in now! I'm already 8 days in and seemingly no light at the end of the tunnel; only more annoying inconsistencies and weird /unexplained occurrences.

What do you think about if I plugged in the XHDD to my new lappy and started from scratch on Windows iTunes? I'm gonna hold off for your response cos I wanna do this correctly and as painlessly / labour-unintensively as poss! So, I guess it would entail

1, ticking the 'keep iTunes Media folder organised' box

2. unticking the 'copy files to iTunes media folder when adding to the library' box

3. Make sure the XHDD is plugged in

4. Change the iTunes media folder location to where I want it on the XHDD.

Either

5. Then File -> Add File to Library

6. Select the Music folder within the media folder

Or

5. Copy across the iTunes Library.itl file from my iMac and other files include in Album Artwork

6. Turn it on and It should be pretty much the same as what I have on the Mac except hopefully some of the quirks might be fixed?

SO, it is clear I am pretty confused at to what to do now.

Sorry to be an idiot but I can't figure this out. Bloody typical as I should be practicing for Glastonbury not ******* about with this! Please can you help or suggest the best course of action.

Talk soon, thanks...

Russ.

Hi again tt2,

OMG. :O WTAF?!

In the lost folder there are nearly 23000 items.

I can no longer search & find files on the drive. The only (remotely) positive thing is that the drive shows probably the correct amount of memory remaining (650gb'ish), so I'm hoping the files are there somewhere???

There are some other peculiarities I have spotted...

Created and modified dates are in 1979, weird and unexplainable. The date on my laptop is fine and has been fine & up to date. I copied from 1 drive to another after reformatting to ExFat. Previously it was Fat32. Obviously, I did that on the new machine cos it's so much faster plus it has usb 3, which I don't have on the iMac.

I know I've sent you 3 messages, with a fair amount of info within.

If you could shed some light on this and suggest my next steps with (hopefully) the path of least resistance.

I'm so stressed now, I am totally befuddled. Many thanks, and I hope to hear back soon...

Cheers,

Russ.

Hey tt2,

I'm not interested in play counts, ratings and associated data etc as I lost it in a blip when changing drive before.

I have a few playlists that I would like to keep but that's really it. It's possible to export a playlist isn't it? Could I do that before whatever change you're thinking of and then import it again afterwards?

I don't keep wav, FLAC or ALAC, and even try to convert all my AAC's so theyre all mp3, however my housekeeping has been not great so I know I have around 500 aac but the rest are mp3s. There is a fair chunk in an unknown artist folder because I have to organise that. After an import, if I have coverted from FLAC to mp3, there's normally no data apart from track name so I have to label all that afterwards. That was what I was doing when I noticed all this stuff going on and my attention has been diverted ever since.

On a positive note. I plugged the drive into my laptop (this is my machine which is on windows 10pro) to see if I could find the missing 22000 tracks on that machine. I couldn't. So, I took the plunge and clicked to check drive for problems, this showed up none. So I clicked the button for scan and fix errors. This took a while but when it finished it had found all the missing files. However, it has put them in two new folders (one called Various 1 - which was the previously named 'Various' & 'various' and another called 'Various A - 1' - which was previously called 'Various Artists' & 'various artists') and a few others I am certain.

I'm not sure how iTunes will deal with these changes when I plug the XHDD back into the iMac, so I haven't as yet. I wanted to see what you'd said before I did so.

When looking at the file system on my iMac, it is Mac OS Extended (Journaled), not the case sensitive version, which is surprising. I know the file system on the 'problem' / newest drive I set to ExFat and the one on the previous working XHDD was MS DOS Fat32. So, not sure what this case issue is about.

In regards to permissions. I looked at the XHDD and, in Properties there was no Security tab. After some investigation, apparently, on windows 10pro, and for FAT file systems - there will be no security tab. However, it does show in the Attributes section a box labelled 'Read only (Only applies to files in folder)'. So I unchecked this box and also, in the 'Customise' tab, I clicked to apply this to all subfolders within the structure. I've actually done this twice because after it completes, if you look again, the box looks like this, with a black box inside the other box?

Goodness knows why it's not just a white box like next to the Hidden option?? I assume it's done as it took a few minutes seemingly going through the whole structure making changes. Plus, I clicked in the box and 1 click clears it like the empty Hidden box; click again and you get an actual tick, click once more and it goes back to the aboce box in a box and those are the 3 states it can be in. It was an empty box when I applied it to the whole structure so I'm hoping it's done, even though its reverted to this box within a box look now. I'm logged in as an administrator, as I do on all my computers.

I think I've covered everything in your message. With the above info in mind, what do YOU think I should do now?

I want to plug into iTunes again to see if things are better with the permission changes and now windows has repaired/ slightly altered the structure, but conversely, I am also concerned now that some folders are changed, iTunes might start creating more folders again. I know it's best to make changes within iTunes but I'm not sure which way to do it or at what stage. The new Windows laptop is much faster and I am fine with only using iTunes on here for now on, if that would be easier & faster to fix everything.

wadayafink??

Thanks, Russ.

Hi Russ,

I presume you're referring to this thread: why has itunes moved my music files out of the Music folder?

The Make a split library portable user tip is a more general article on the topic of library layout and what steps might be used to safely manipulate it. The old school layout put artist folders inside the media folder, the newer layout creates the extra music folder for music so that each media kind gets its own subfolder in the media folder.

I've seen a couple of threads where users have commented that supposedly missing files are, in fact, exactly where they are supposed to be. The extra file://localhost/ is expected for the missing tracks. This can happen if the drive they are on is offline when iTunes is first fired up but otherwise I don't know why this might happen. For Windows I have a script that can attempt to search and repair but I don't have a Mac equivalent. As long as iTunes shows a bunch of broken links you can try alternately attempting to play or use Get Info on one item, locate the missing file when prompted, and then iTunes should offer to attempt to repair others by itself.

tt2

What you describe is the classic split library, where a new media folder has been created on an external drive, leaving the library files in the usual location on the internal. In an ideal world you would have the library files inside <External>/iTunes, the media folder set to <External>/iTunes/iTunes Media, and all media content somewhere inside iTunes Media. Then a straight clone of the iTunes folder on another drive will be a fully functioning library. Either copy of the library can be connected to any computer running the same build of iTunes. You can even flip from Mac to Windows, however you'd need to add in an HFS+ driver for Windows, or NTFS driver for OS X as there isn't a disk format that is natively compatible with both and suitable for large libraries. What you can also do is clone a library from an OS X shared folder to a Windows shared folder. There is a little processing overhead each time the library wakes up in the other filing system, but otherwise this works perfectly. I have around 4,000 tracks in an iTunes Match library that I use this way in both OS X and Windows, and a main library of around 50k that exists as three parallel clones, one at home, one at work, and one on a portable 3TB external that I use to keep the other two up to date with SyncToy.

To start with I think you should focus on those broken links.

Lost & Found Playlists

Create a playlist called Found, select everything in Music and drag it into the Found playlist. Create a smart playlist called Lost matching only Music items and the rule Playlist is not Found. Your lost tracks will be in this playlist.

Once you can see which tracks iTunes cannot locate you may have more luck using its locate and repair mechanism.

When you've fixed the links we can talk about fixing the shape of the library to make it easier to move around. How big is the media folder? How big is the drive? It is somewhat easier to consolidate the media to the final location, but that does mean copying a lot of files, and can only work if there is enough room to temporarily duplicate all the media. The other approach has more steps, which need to be followed carefully, but avoids the temporary duplication.

The XML file is used by third party software to quiz the iTunes database without having to launch iTunes. The core of the library is the ITL file. You need both database and media for the library to work properly.

tt2

Hi,

The first images indicate that you have a case sensitive file system. Seeing the folders with alternate cases is expected if you have any inconsistency in the metadata. Windows is usually case insensitive so there could be issues switching between the two.

Take a look at Repair security permissions for iTunes for Windows. Although written for Windows the permissions requirements should be similar on a Mac. Sometimes there can be problems if permissions are set inconsistently. Both the user account and system should have full access to the entire iTunes folder structure. Probably not your issue, but I'm including it as a possibility.

Your file creation dates look way out of whack. I've no idea what that is about.

At this stage your best bet is probably to pick up another external drive to rebuild the library on in a portable shape. Do you have access to Windows at the moment? If so then look at Re: New computer and itunes not working. Does your library contain ratings, play counts, playlists, date added data, etc. that matters to you, or is it sufficient to have all the media files cataloged and accessible on demand? Are there any tracks in .wav format? These don't carry a tag which complicates things when they are imported into a new library.

tt2

See Re: iTunes not able to local original smogs for one of more recent posts on this topic.

See Make a split library portable in particular. I suspect the best approach might be to manually reorganize the drive so that your media folder is set as H:\iTunes\iTunes Media and your artist folders are inside H:\iTunes\iTunes Media\Music. After changing the media folder location use File > Library > Organize Library and tick the box Reorganize files if it isn't greyed out. iTunes should then be able to reconnect all of the missing files if you can get it to prompt you to repair one, otherwise my script should help you out. Once you have fixed the broken links you will also be able to place the library files into H:\iTunes which will make the library portable, and make it easier to back up and move around in future.

tt2

No problem. Backup what you have first if at all possible. If it isn't maybe now is the time to invest in that extra backup drive. Then take it slowly and by all means ask me to explain anything that isn't clear.

tt2

RustyC4$ wrote:

OK, that sounds like a decent plan. The only thing I would say is, if I move the media folder (renamed as import) into the media folder, that will take up so much of the 3tb drive that I won't have enough room for the 'copy files to media folder location' ticked box to fit it all on there.

Moving (not copying) the current media folder inside the new one you've made ensures that everything you're going to add to the new library is already inside the designated media folder. The Copy files... option only applies to items that are outside the media folder. It is best left enabled so that you don't accidentally add content from a memory stick, for example, that then won't play later because the stick is no longer there.

1. I was imagining you working with a disc that already had the media files copied to it. If you want to start with an empty drive then Create the new library at the root of the drive, make sure Keep... and Copy are enabled, and that the media folder is pointing to /iTunes/iTunes Media on the same drive, then use File > Add Folder to Library and point it at the media folder on the other drive. There is no need to rename the folder you're going to import. Everything will be copied over into the new media folder and arranged as it should be. You can only do this type of import once without creating duplicates.
   
2. Yes, in the general form of \iTunes\iTunes Media\Music\<Album Artist>\<Album>\## <Name>.<Ext>.
   
3. The folders Unknown Artist or Unknown Album are used when either Artist & Album Artist or Album are blank.
   
4. If a track has Part of compilation is true then it ends up in <Compilations>\<Album> otherwise it ends up in <Album Artist>\<Album> where Artist is used if Album Artist is blank.
   
5. Importing media won't change any embedded metadata. Ratings, play counts, etc. that aren't stored in tags are set to default values.
   
6. I don't honestly know, or at least not if you're using the drive formatted in case sensitive mode. The only way to know for sure is to experiment.
   
7. Yes... but it isn't built into iTunes.
   
8. I have a Windows only script called TitleCase that I use to bring consistency to my library. It has a bunch of exception lists to set or preserve specific words that shouldn't be converted to initial capital with everything else lower case, e.g. CiSTM K0nFLiqT. I run it manually to check every proposed conversion, and update the rules as needed to prevent unwanted changes. It's not perfect but it does what I want. One of its features, that can be optionally turned off, is that descriptive terms like (Acoutstic), (Live), or (Feat. XYZ), which are not really part of the song name, are put in square brackets to distinguish them from the title itself.

See also Grouping tracks into albums for more on tidying up in iTunes.

tt2

Hi TT2,

I've seen many responses from you and it seems that you are one of the most well informed people on here in regards to iTunes.

With that in mind, I have a query which I can't find within this thread but this is the closest one to my issue.

I am on Mac 10.7.5. My iTunes library contains more than 200,000 tracks. I've been on an XHDD for some time and swapped twice before and changed to a new iMac once. All have created their own unique problems.

The most recent one is I have just changed from my original 2tb XHDD to the (what was back up) new 3TB XHDD. After a few new uploads I looked at the structure and it had started putting the music in new iTunes media folder rather than the music folder within that. So, following your instructions I found on the thread 'Why has iTunes moved my music out of the music folder?', I proceeded to do step 1. reorganise the folder - this finished yesterday (after 5 days!!). I haven't done the next step 'consolidate files' yet because I'm expecting it will take ages again. Plus I have noticed another issue and after a lot of head scratching and a fair bit of (probably unnecessary) work, I thought contacting you would be easier. Now, I have the dreaded missing track exclamation mark on over 11000 files. After the aforementioned 'work' I'm down to just over 7000. The really confusing and frankly annoying thing is that when you look at where iTuned think the file is - that is exactly where the file is located. I can't understand why I'm having to locate tracks which are where iTunes thinks it is (according to 'get info' / 'file' tab).

Can you shed any light on this please? I'm at the point of putting my head through the screen!

Thanks...

Russ

Hi TT2,

Thanks for getting back so quick! I have looked at the split library and I don't think that will help me.

I have only one XHDD that is in use for adding and storing my tracks using iTunes and another mirror of that as a backup. Whenever I DJ, I copy (or actually have most of my commonly played) tracks on a smaller & more mobile XHDD. Then I use that with Traktor to do the night of whatever it is (from club - wedding, etc)

When I was forced to migrate to an XHDD because I outgrew my iMac's memory 2 iMacs ago, I noticed that, although \i had changed the location of my iTunes library, all the associated files, Album Artwork, iTunes Library Extras.itdb, iTunes Library Genius.idtb, iTunes Library.itl, iTunes Library.xml are in the original iTunes folder on the hard drive Mac HD/Users/Russell/Music/iTunes. I couldn'f ind a way to point iTunes at this info in an alternative location. Might this be causing issues? If so, is it possible to have ALL files on the XHDD?

As I mentioned before, I am missing around 4000 tracks now. And, as it 'appens, I have just bought a shiny new windows lappy. It has an i7 processor and 2tb hybrid and 500gb ssd HDDs inside. So, when I'm doing a gig, I will just make sure I have what I need for my set on these internal drives. I just want to make sure that iTunes has everything sorted after I've just changed to a new XHDD. Am I correct in thinking that one of the above files (I think iTunes Library.xml) that i can use to use on my windows laptop so that iTunes will look the same on both platforms?

With all the above info in mind, what would you suggest be the order of my approach to solve my issues please?

Many thanks...

Russ.

Hi tt2,

You replied while I was sending the next update.

I had already done on many occasions the lost and found thing. It went from 11000 down to 4000 after a load of work. Then, I tried one of the suggestions to unstick the keep my folder organised, turn off, then turn on, click a missing file, etc. But that did nothing. The, when I set it back how it was, I was missing 8000 tracks again.

Subsequent to that, I've had the most recent duplicate (capital/non-capital letters) filename & hidden files problem I explained in the one while you were replying to my last.

I turned off iTunes and ejected the drive cos it is stressing me out but I will turn back on and do an up to date count to see how many are missing at the moment. However, after checking few that I had previously fixed, the **** thing seems to have added the file://localhost in again. So I am almost loathed to do any more locate/find things as I've wasted so much time on this already. Of course I will do it… I HAVE to, but I want to do the path of least resistance now.

I'll do the count which takes a couple of hours'ish. My media folder is 1.98tb and the drive is a 3tb (only shows 2.72 on windows for some reason) formatted in ExFat, which was partitioned but I removed that (see prev msg).

Cheers

Russ.

In Windows folders always show as read-only after the read-only flag has been cleared. You can ignore that.

Exported playlists only reimport properly if the files are still in the same locations. A workaround for this is to tag the files using one of the fields that can be used in smart playlists. I've used the grouping field to mark tracks for my family so that each gets the right content on their devices even if I have to rebuild the library. E.g. Alice/Bob would match both Grouping contains Alice and Grouping contains Bob, so ends up in two lists.

I'd be tempted to shift-start iTunes and create a new empty library in a folder called iTunes in the root of whichever drive you want to work with. Under Edit > Preferences > Advanced make sure Keep... & Copy... are both ticked. You could then rename the current media folder as Import and move it into /iTunes/iTunes Media. Then use File > Add Folder to Library to import the folder. iTunes should add all of the content and reorganize it. Anything left inside the Import folder at the end of the import is something that iTunes couldn't understand. Everything else should get organized properly according to its metadata.

tt2

OK, that sounds like a decent plan. The only thing I would say is, if I move the media folder (renamed as import) into the media folder, that will take up so much of the 3tb drive that I won't have enough room for the 'copy files to media folder location' ticked box to fit it all on there.

1. Could I just rename the folder on the drive where it is 'F', plug a new, empty & formatted drive in as well 'G', create an itunes folder on that; Then open itunes + shift key, point it at the itunes directory on 'G'; Then file > Add folder to library and point that to grab from G:"whatever/the/path/is"?

2. Am I also correct in thinking, if I do this, it will pull and place the files in accordance with the metadata, wherever they are?

3. If so, what will it do with songs which are in an unknown album by an unknown artist?

4. Also, in regards to the case issues happening on the Mac - in the metadata it will say that the file is by an artist called 'Various Artists', 'VARIOUS ARTISTS' or 'various artists', on Windows, not being case sensitive, will it put it all in one folder 'various artists'?

5. Will it leave the metadata the same?

6. Will doing this create problems if I plug the drive back in on the Mac & use itunes?

7 Is there a way to clean all these small variants up?

8. if so, is there a piece of software or code that is best for these kinds of issues?

I think we're nearly there... RSVP. Many thanks,

RC