Songs Disappearing from iTunes

Here is what I think is going on (but I could be wrong). As I see it every action in iTunes triggers updates to the database files, in particular both iTunes Library.itl and iTunes Music Library.xml. The update process involves writing out a new temporary copy of the database file, deleting the original, then renaming the temporary file to take its place. I think AVG in particular, of late, has been stalling one part of the process, presumably the renaming of the temporary files since people have successfully recovered their libraries by renaming such files manually. I don't think a virus is detected, seemingly no files are actually removed by AVG, but iTunes doesn't check to confirm that the .tmp file has been renamed after the original is deleted so the library vanishes. I'm sure iTunes could be better at detecting when this has happened and recover gracefully, but there it is. It's annoying when it happens to you but I suspect in the great scheme of things it is not happening often enough to register with the behemoth that is Apple, and even if it does there is no reason not to act rationally and unilaterally to protect your data now.

One noble soul in another thread who had moved to Microsoft Security Essentials reinstalled AVG to test things since their patch was released, but I've just seen a report where his library has vanished again and he plans to switch back to MSE. There may be other factors involved but if the problem were purely of iTunes making then more people ought to be affected, and switching AV engines wouldn't be as successful a resolution as it appears to be. Several users have reported that adding exceptions in AVG hasn't helped which made me wonder if AVG isn't (or at least wasn't) obeying its exceptions. A folder level exception will be more useful since you want to allow the uninterrupted creation and renaming of as yet unarmed and non-existent temporary files. The iTunes library folder holds no executable files so unless you were to go poking around at random there is no reason for anyone to copy a virus to the iTunes folders, or for you to execute it.

Note that in iTunes 11 an "emprty" library shows cloud links to past purchases if you are "signed in". It is still essentially empty, even though your data is on the drive.

tt2

I don't have AVG, so that can't be the problem. I appreciate the advice you and others have provided, but having spent hours on the phone to Apple support, been into the store, had my IT manager on the case etc, it's just too much like hard work!

In the days before I chose Itunes as my music playing / storing solution, I just went to the cabinet, selected a CD, put it in the machine and pressed play. I'm tempted to shift to Sonos now, because I'm not sure how we can be confident of this being sorted out?

Let me explain - Apple MUST know this is a problem. A 'behemoth' the company may be as you describe, but there's just too much on the web for that not to be the case. But as they won't admit there's a problem, then how can they let everyone know when it's been sorted?

If anyone can answer that, then I might hold out a bit longer before taking the Sonos plunge

I can't give you a definitive answer as to why things go missing in your case, although other software is perfectly capable of interfering with iTunes in a similar manner to AVGs recent behavior, as users with Sony VIAOs well know. To some extent I wish it was happening to me because at least I'd get some data to work with. Up until recently I've tended to see occurences of this problem pick up immediately after a new iTunes release and then die away again, so I've treated it as an iTunes "thing". The fact that it was happening "out of season" this time led me think about other external causes and, after asking a few sufferers which AV there were using it became apparent that AVG was responsible for at least part of the current rash of cases.

Do you have lots of .tmp files in your iTunes folders? What AV package do you use? Do you backup everything?

I'm conscious this thread started life with mystery disappearances of random songs from the library rather than the entire database, again an issue I've not personally experienced, nor one I've so far been able to think of a likely explanation for. 😕

tt2

Yup, all of my IT library was eliminated from Itunes again today. This time it happened (several times, as it happens) while I was checking through Windows Explorer to see what files were there...or not. And yes, at some point I had over 70 temporary files in the folder, all just numbered. I tried deleting them, until I realized that the main Library folder wasn't even there. Somehow, it's being eliminated even if all the music files are all still intact under the Itunes Media folder. At one point during my troubleshooting, I actually saw the Library folder (the one with the icon of little musical notes) vanish in front of my eyes.

After wasting another day with the Itunes folder/program that cannot maintain my music files long enough for me to play them, I'm just deleting the whole program once and for all. I can live without my little nano ipod. Lucky me, I can still pull out all my CDs and play them--since I never bought into this "cloud" business. I knew there would be problems, and I have the most reliable back-up there is!

I thank TT2 for the input, too, but I still don't believe the problem lies with AVG, even though they have issued an ineffective patch and are indicating they're aware of the problem (unlike Apple) in their forums.

Interestingly, both AVG and Itunes put out their new versions around the same time (or at least I downloaded them almost simultaneously), so it's been a challenge to figure out where the real problem lies. If these two big companies can't figure it out, though, I'm not bothering anymore. I don't feel like doing their work for them.

I assume they'll eventually figure it out. But as Petederby worried, I don't see how I'm going to know if they do--I'm really tired of crawling through these chats to get word of the long overdue fix.

Thanks for the help, y'all.

If you have AVG I really would recommend switching to another AV solution to see if that resolves the issue for you. If you use another AV package I would try excluding the iTunes folder from all real time scanning activity. I've been reminded in another thread that another possible cause could be any automatic backup tool, this might include Windows' Shadow Copy system.

If you read this post you should see that it may be possible to restore your library quickly by renaming a copy of relevant .tmp file. Once you have a known good library you should back that up, and be prepared to take periodic backups. Once you have it working you can clear out the rest of the .tmp files. The fact that they remain after iTunes has finished with them is proof that the normal process of updating library files is being affected.

tt2

I just want to add that I have the same problem. I have a rather large library and only discovered it when I decided to change the playlist synched to my phone. I see now that I have many albums deleted. Not just removed from my index file, deleted entirely from my hard drive. I use Carbonite for backup and it removes deleted files from my backup after 30 days--the files that have been deleted from my library are no longer on my backup, so longer than 30 days ago. One of the iTunes upgrades evidently did something pretty f#&*'d-up. I have all of the missing albums on CD and will have to re-rip. This really *****. My plan is to set up a folder where I put all music files then mirror that to an iTunes library directory--and never let iTunes touch my "music archive" directory. What a pain in the ***. Oh, and I don't have AVG.

And the fact that this thread started on the Mac side of things is another reason I've stayed away previously since I don't have one to test with. I use a folder cloning tool to backup the library on Windows, and preview the proposed updates so I can make sure nothing untoward will be deleted or overwritten. Again, I have never experienced similar symptoms, not that it is any comfort to anyone else.

tt2

Low and behold, i go to my Itunes library again today and all music is missing again apart from items in 'iCloud'.

APPLE - please listen as your company is slowly descending in to a FARCE.

People do no want to be constantly backing up their music because Itunes libraries are not doing the job they are designed for - the safe store of music - music we have at times heavily invested in whether that is in the form of purchases from iTunes, music CD's or DVD's

People have invested in your software as it was a world leader in its field, knowing full well if any problems were to occur, Apple had the staff, ability and general know how to deal and solve almost all issues no matter how small or large they were.

This problem has been going on for so long now, yet nobody at Apple acknowledges the problem, can identify any fix, update the program - probably too scared of any potential share fall consequence.

Apple as a company seem leaderless, rudderless and seems heading for an almighty fall if something as simple as storing of music within one programme cannot be remedied - sinking ship!

I for one will never invest in another Apple product due to complete lack of faith in Apple, its staff and its service.

I use iTunes, own 5 iPods, an iPhone, and an iPad Mini. I've decided to abandon iTunes and get rid of my iPods. I just placed an Amazon order for an 80GB Zune media player. When my contract runs out, I'll get rid of my iPhone. I have a Kindle Fire already and plan to sell my iPad Mini.

<Edited By Host>

Took a while since my last post, but today I finally had a broken link to one song. I know it already existed because a copy of the mp3 was in the backup folder I made. I know this is nothing to what others are experiencing, but I neither have AVG nor do I buy songs - I make my own.

QuoteNotes wrote:

I use iTunes, own 5 iPods, an iPhone, and an iPad Mini. I've decided to abandon iTunes and get rid of my iPods. I just placed an Amazon order for an 80GB Zune media player.

That's hilarious!

Thanks for the laugh!

I wasn't laughing when I wrote it. There are better alternatives to the iPhone. I like my Kindle Fire HD better than the iPad which I rarely use. Abandoning the iPods won't be difficult.

QuoteNotes wrote:

I wasn't laughing when I wrote it.

I guessed you weren't and that's what makes it so funny!

I had no idea the problem was this old.

I've moved from desktop to titanium to MacBook Air; from iPhone3 to iPhone4; from iPod Classic to Nano and been on mac's forever and have always assumed they were lost by transferring.

Reading the unresolved litany above it is clear that when Apple revises it's "national" libraries and is more inclined to eliminate than to add. Purchases of international music that once qualified as world music is now eliminated simply because Apple has no rights to the music. And thereby, neither should I.

Nearly 100% of my music is international. I am compelled to pay extra for imported CDs and to shop at Amazon where international music is legal. Approximately 50% of my music has been lost by Apple. The Apple purchase history says I have only purchased 550 songs. All of them since I purchased my MacBook Air. Not cool. What I purchase legally can not be revoked or overruled by the small minds of programmers governed by Apple's inability to achieve discount rights.

Like others have posted here, I own the CDs. I have uploaded them many times thinking I was the fool. Now I know it is Apple who refuses to grant me ownership rights.

The basis of this assessment is really simple to observe. Apple sells next to no French music. I can track Apple policy simply by watching the songs disappear. Those albums once purchased from Apple are truly gone for ever. I've watched as label artwork of purchased downloads, now unavailable for sale in the US, have been altered to some absurd label guaranteed to fool the visual thinkers.

Myself as well. I opened up iTunes and there was nothing but the music I bought from iTumes left. All the playlists, all the CD's I own that I ripped into my own library are gone. What the heck has happened here? I didn't want any iCloud nonsense. I just wanted to be able to put my music together in one place, put some on my iPhone. What' the heck is going on?