Very large collection, slow iTunes

Everyone worries about "bloatware" and IMHO, this
would be a perfect example.

So iTunes 6 was bloatware?... It worked.

However, if you want to create playlists for every
album though, have at it.

Not a very helpful comment, but thanks :/

As you have noticed, it
slows down iTunes.

It does now, you are right and that is why I am and it seems others are here.

You can send your suggestions to Apple here ->
http://www.apple.com/feedback/itunesapp.html

Already kindly suggested by Terrence some posts above yours and it has already been done 😉



@ David I have added a lot of music after I "upgraded" to 7, but it slowed down for me right after I installed 7. So I am sure it was 7 that did it. And I have also seen many other posts over the net for this same problem.

And thanks for letting me know you did not get any speed bump after deleting all those playlist...what a drag :/

So iTunes 6 was bloatware?... It worked
No, it wasn't. As stated, one reason for slowdowns is possibly because of the additional artwork features in iTunes 7, which were not in iTunes 6.

As programs get tweaked and more features added, something's gotta give.

As an aside, I'm curious as to how many playlists you actually have?

So how many playlists do you have?

First of all, I remember the days when iTunes was limited to something like 32,000 songs. Apple listened to complains about that and removed those limitations, which no doubt involved some major database redesign. It is quite clear that over time more and more people will have huge libraries, and Apple will have to deal with that.

Besides all that has been mentioned here, one thing to be very aware of is what 3rd party applications you may have running that interface with iTunes (widgets, etc.) Many of them are very poorly designed for large databases: they might, for instance, tell iTunes to get a list of every track of your library, guaranteed to lock up iTunes. In addition, even good ones, if there are too many running, may crowd iTunes for info.

I would also reiterate that it is pointless to have a playlist for every CD! That's what the browse function is for.

And I thought I had a large library! (78GB, with 57 Smart Playlists and 42 regular Playlists)

I have mine stored on a 120GB Firewire 400 HD ( and a backup mirror). The only time I see slow performance is when loading iTunes.

I have been thinking forward to the time when my libarary exceeds my hard drive space. My LP collection is only minimally transferred at this date.

I would hope that Apple would notice the use of large collections.

If Apple doesn't think iTunes needs to be ramped up perhaps they would consider the "ProTunes". Sort of like iMovie, Final Cut Express, and Final Cut Pro.

And I thought I had a large library! (78GB, with 57 Smart Playlists and 42 regular Playlists)...The only time I see slow performance is when loading iTunes.

I don't know the magic number, but iTunes doesn't really start to bog down until well over 100GB (in my experience). Depends on the muscle of the hardware setup you have as well, but you're rocking a G5 so you can probably fill up that HD before you really start to feel the lag.


The iTunes 7.1 update is considerably faster writing to the database. The software is not too painful to use now (98,000+ songs).

Thanks for the report on that...I've been holding off on installing until I hear how it works. Has anyone else installed the update yet? Any other reports on speed improvements?

One more thing...sorry to ask again, but I haven't found an answer to this yet:

* Here's my question: If someone wanted to set up for a 400 GB library (knowing and accepting all the iTunes software performance issues) would there be any difference between buying one good 400 GB HD and buying, let's say, 4 100 GB HD's and doing a hardware RAID array?

Hi David,

I'm by no means an iTunes boffin, (and my own library is a piddling 50 gig or so) but one thing I can say is that putting a 400 Gig library on 400 Gigs of drive space is not a good idea. Full drives are inherently slow drives. Firstly the inner sectors of the drive have inherently slower access times than the outer ones. When a drive gets close to full this has a significant affect on speed.

Secondly full drives, especially if you are regularly editing files, tend to develop a lot of free space fragmentation. A very large library db is itself also going to become severely fragmented (My own little 50 gig iTunes collection has a library db file that is around 20MB in size. THis is the maximum size that OSX itself automatically defrags. Yours will be much larger, and therefore not defragged by OSX). As this file is very regularly being modified when iTunes is running, fragmentation is likely to be substantial and to cause significant slowdowns.

If you have a 400GB music library my suggestion would be that, regardless of the other issues, you keep it on a 500Gig drive, to avoid db fragmentation issues and to minimise use of "slow" inner sectors of the drive. Another alternative might be to keep the music and the database files on seperate drives, I guess.

(This doesn't answer your question, I know, but it is important to bear in mind, regardless)

Cheers

Rod

Thanks for the input there, even though you are not a "boffin".

I didn't make something clear above (I mentioned it in the earlier post that I quoted from)...I have no intention of filling up the HD's and know to leave around 15% free for good performance. And I have always kept my iT Music folder on a different drive than the startup disk with the application and iT Library file. But I have some questions for you on the other points:

- I didn't know that the db file itself could get fragmented. I assumed that the db would build itself in the same way that the <Keep iTunes Library Organized> preference keeps the "iTunes Music" folder organized.

- I also wasn't aware that the OS automatically defrags an actual file, or that a file needed to be defragged at all.

- I thought that journaling took care of this kind of stuff.

- Is there anyway to defrag a file manually...or rebuild the iTunes Library? I'm sure if this was possible then someone would have mentioned that by now.

- Do I need to defrag all of my HD's if they are formated for journaling?

Wow, your point really brings up a lot of questions for me. I'd love it if you had time to elaborate. Aside from that, my question about the "400 GB" HD for my library was about whether setting up a RAID of 4 smaller drives would provide faster access for iTunes than having one HD of the same size. I just picked 400 GB as an example.

- I didn't know that the db file itself could get fragmented. I assumed that the db would build itself in the same way that the <Keep iTunes Library Organized> preference keeps the "iTunes Music" folder organized

Just about any file can get fragmented , David. Files that are constantly / very regularly updated tend to do so more than most others. As long as you have plenty of free space it is not likely to become a major issue, as the number of fragments will be low. OSX deals with such things well when there is plenty of room to move. When the disk gets full , though, the level of file fragmentation (and a consequent deterioration in performance when the file has to be accessed or written to), will increase because there aren't any nice large contiguous blocks of space for it to be written to.

Users of large database files and people who edit video are the ones who are usually most conscious of the problems of fragmentation. I guess with a very large iTunes Library (and consequently large iTunes Library databse files), you may well be suffering some of the problems that large database users find they have to deal with.

You can get a free utility called "ShowVolumeFrag" which allows you to see just how fragmented particular files are. Might well be worth taking a look. I find regularly that the most heavily fragmented files on my own drives are the Spotlight databases and the Microsoft user database. In my case, as the Music library database is below 20 megabytes it gets the benefit of OSX's own defrag routines, and stays reasonably whole (though I see it currently it is split into 6 seperate fragments.)

- I also wasn't aware that the OS automatically defrags an actual file, or that a file needed to be defragged at all.

The "official" view on defragmentation from Apple can be found at http://docs.info.apple.com/article.html?artnum=25668 . There is also a well known "classic" article on fragmentation issues with the mac at http://www.kernelthread.com/mac/apme/fragmentation/ which explains the OSX routines well and which argues against defragmentation in most circumstances.

There are, however, many who disagree. Arguments about the matter inevitably become heated unfortunately. http://www.macfixit.com/article.php?story=20070301091515843 (not sure if this is available if you are not a MacFixit member) provides a useful summary (followed by some shouting by those who comment on the article). Charles Moore, at http://www.applelinks.com/index.php/more/isdisk_defragmenting_worth_thetrouble/ , provides a well argued counter view and some more links. The post from a Micromat technician HERE is also worth a read.

Personally my own experience has been that when, over time, fragmentation of free space becomes substantial actual file fragmentation increases rapidly, and defragmenting can produce a substantial improvement in performance for some tasks. Spotlight, for example, is much zippier when its .db files are not fragmented.

I guess its the fact that iTunes libraries of the size being discussed here are somewhat "out of the ordinary" that makes me suspect that defragmentation may be of value with them. I certainly don't know for sure but it would be interesting to find out.

I thought that journaling took care of this kind of stuff.

Journaling doesn't really have much to do with "fragmentation" issues like this. Journaling " is a system which maintains a secondary record of file locational information. It adds security by reduing the risk of data loss, but it doesn't have any effect on fragmentation. Its worth noting, by the way, that journaling actually exacts a bit of a speed hit of its own. see http://docs.info.apple.com/article.html?artnum=107249 for an explanation of journaling and its pros and cons.

- Is there anyway to defrag a file manually...or rebuild the iTunes Library? I'm sure if this was possible then someone would have mentioned that by now.

Simply rebuilding the library probably wouldn't generally do the job here, David, as it would most likely be being written to free space which was already somewhat fragmented, unless the disc was reasonably empty. You can defragment a complete drive by simply cloning it to an empty one, of course, using a utility like SuperDuper, wiping the original and then cloning back.

I personally use iDefrag .


If you have plenty of free space on your drives you probably don't need to defrag most of them very often, if at all, David. Its the drive with the library file on in particular that sounds like it might be worth defragging . It would be interesting to see what the fragmentation level of your iTunes Library file is like first. Maybe you could run "ShowVolumeFrag" and let us know.

Cheers

Rod

Wow! Tons of good info there. Thanks for taking the time to elaborate Rod.

To keep the discussion going...

- anyone have any feedback on the 7.1 update?

- anyone have an opionion on the speed/performance results RAID array of HD's versus a single HD of the same size?

Wow! Tons of good info there. Thanks for taking the
time to elaborate Rod.

Very good info indeed! thank you Rod 🙂

- anyone have any feedback on the 7.1 update?

Same as 7.0 I am afraid to say...slow when trying to work with titles, or when adding music :/

Just ran "Show Volume Fragmentation" and here's my report:

- First of all, I got involved in this discussion b/c my "iTunes Music" folder(all of my music files) was reaching a critical size and I need to migrate to a new, larger HD. So, I am knowingly operating with my "iT Music" folder too full (6 GB available on a 300 GB HD dedicated to iT Music).

- My startup disk was getting full as well. After running "Show Vol. Frag" I finally stopped procrastinating and cleared out some space on the startup disk. I'm now running my 160 GB startup disk with about 40 GB free space. (The pie chart is already looking better!!!)

- I used to defrag quite often, but I was under the mistaken impression that the new OS's and the journaling meant that I didn't need to do that anymore. After reading your replies, I now get it. Knowing that the newer OS's help with fragmentation issues, but don't solve them is a GREAT revelation to me. Also, knowing that journaling doesn't have anything to do with fragmentation is great to understand.

- ** So here's the results of the iTunes Library files:

iTunes Library File: 57 Frags / 82.26 MB file
iTunes Library File (XML file): 43 Frags / 78.45 MB file

So that doesn't look good to me. I'm going to run iTunes for a bit now that I have cleared some space on my startup disk. After I see how that affects my speed, then I'm going to defrag those iTunes library files and check it out. After that, I'll probably use Tech Tools (which I got with my extended applecare warranty) to defrag my whole drive.

I'll report back. And once again Rod, thanks for the info. I had issues with iTunes back when I had plenty of space on both the startup disk and the dedicated HD for the iT Music folder, but since I lose performance in increments here, it's hard to know just how much all these little things are slowing me down. Your post was a good reminder to always remember to deal with some of the basics first, then start looking into all the other tweaks.

The library.xml file is not used by iTunes.
It is only written by iTunes for use by other apps.

I'll probably use Tech Tools (which I got with my extended applecare warranty) to defrag my whole drive.
Techtools Deluxe (which comes with Applecare) does not include defrag. It'll repair directories and run diags but that's about it.

http://www.micromat.com/index.php?option=com_content&task=view&id=37&Itemid=51
Repairs Directories by Rebuilding
Volume Repair Preview Report
Bad Block Detection on Drives
Checks the Drive Mechanism
Checks USB Functionality
Checks Memory
Checks Video RAM
Checks CPU

Hi,

Don't know if anyone has mentioned this..

My library is on the large side (2.3 Tb). I have all the files stored on multiple firewire drives connected to a G4. I run iTunes 6 on that machine and share it with anyone in the house that wants it. I have iTunes 7.1 on my iBook (which is my primary stereo controller) I connect to the shared main library from the G4 and wirelessly pump that through a couple of Airport Extremes. I never really have any problems with speed or interuptions of any kind. Life is good. Oh, I don't have the option of album art while using the shared library, but that is a small price to pay.

Hope this might help.

Jeff

iBok G4 1.2GHz Mac OS X (10.4.7) 768mb RAM