How to make iTunes add a folder without creating duplicates?

iTunes only creates duplicates if you ask it to, for example by adding the same folder of content twice when that folder is outside the iTunes Media folder and iTunes is set up to copy added music into the media folder, or if the folder you are adding already contains duplicates. A have a script called DeDuper for clearing duplicates from the iTunes library. See this thread for background.

If you want to monitor a set of folders in the WMP fashion use iTunes Folder Watch.

See also Grouping tracks into Albums and Getting iTunes & Windows Media Player to play nicely for some more background reading...

tt2

If iTunes starts up when your media folder is offline then it may flag up all your files as missing. If you subsequently connect the drive and import the media folder to add files then it can add new entries because it doesn't "see" that the the files have already been imported. Shut down and restart iTunes and now both copies work. These are what I call logical duplicates. They need to be tackled by deleting all but one of each set of repeated entries from the libray only, leaving the file where it is.

Another potential source of problems is when the same set of media can be accessed via multiple paths, e.g. as \\<device>\<path 1> or X:\<path 2> where X is a mapped drive letter, or if junctions/folder redirects have been used to make folders appear on different paths. C.f. the virtual remap of requests to Documents and Settings through to Users that can happen on Vista/7. Currently this type of duplicate appears to my script as physical dupes and the script tries to delete all but one, sending to the recycle bin. This is undesirable because the files are really logical dupes and there is no recycling on remote storage so the files are just deleted.

At present I haven't worked out a way to automatically detect if multiple paths to a given file exist. I have however been toying with the idea of moving the dupes rather than deleting them so that they could be restored if deleting turns out to be an error. If all your duplicates show as either on the same path, or two distinct files on two independent paths, then the script is safe to use. If you're not certain (and don't have a backup) then I would not recommend you use it in the current form.

tt2

You've given me some food for thought. I think I've got a plan but I haven't got time to edit & test a rewrite just now. With luck I can get to it later today or tomorrow. Instead of the current Yes/No/Cancel question after the files have been scanned I'll put in something regarding the max. level of duplicate to clean from the following, ideally before scanning takes place.

Playlist

Logical

Physical

Alternate

There are also two versions at the moment, the main one keeps the largest alternate dupe, the other keeps the smallest. Hopefully I can wrap up other options into the prompt so it becomes a choice from something like this.

1. Report
2. Playlist (& 1)
3. Logical (& 1-2)
4. Physical (& 1-3)
5. Alternate (Keep Largest) (& 1-4)
6. Alternate (Keep Smallest) (& 1-4)
7. Alternate (Keep Oldest ) (& 1-4)
8. Alternate (Keep Newest) (& 1-4)
9. Alternate (Choose) (& 1-4)

This would all be much easier to do in some respects as a full blown app with proper dialog boxes, but keeping it as a script makes it easy for anyone else to inspect the code and adapt it if they choose.

tt2

The report level would be just as you say. Currently the script performs playlist & play count merges whether or not that class of dupe goes on to be removed. I'll still add in a confirmation step after the report but I can probably omit the merges for files that aren't going to be cleaned. There is probably a really good reason why I did the merges in the scanning phase (concerning what happens if you interupt the process) but it would be more elegant if it can be done later. If I'm going to do that as well that will add to the complexity of the rewrite.

Playlist dupes are the same tracks from the iTunes database added to a regular playlist. This is a perfectly valid action, but can cause confusion when trying to clean up... If someone has chosen to gather together a playlist of "things that need fixing", rather than a playlist where playback order is relevant and repeats might be intentional, then it is probably worth cleaning any "playlist" only dupes out of the list while the script is tidying up, if only to provide visual confirmation that all the dupes have been removed, or perhaps more importanly, not leave any mystery ones...

I hadn't even thought about the possibility of them until trying to debug the script for someone else where such duplicates caused an earlier version to bomb after removal as the list shrunk in size by two or three instead of the one that was expected. 😕

tt2

Reworking the script is taking longer than I thought, partly because I've been busy at work and also because the new iOS release has taken some of my attention.

Because of the way iTunes provides access to the tracks for scripting purposes it is common to process lists of tracks in reverse order. That way if any update causes the item you've just processed to evaporate from the list you can still work with the rest. Playlists duplicates, where the same iTunes entity appears twice or more in a user playlist, need considering simply because not considering them can lead to errors. Previously such items would have been read by my script as logical duplicates since both connect to the same file on disc, however removing one entry to "dedupe" would have removed the track from the library, thus also removing all copies from the playlist being processed. This also causes the script to crash out because the length of the list shrinks by two (or more) while the index into the list only reduces by one.

The features that I've put into the current version that merge playlist membership, ratings, play & skip counts, last played & skipped are all currently connected to the scanning phases rather than the delete phase, so I'm going to need to rework a few sections of code to make it work properly, ideally without making the script take twice as long to run...

An added complication I've proposed is to decide which type of alternate dupe to keep after they have been counted (or possibly on a track by track basis). Currently the script works on the basis that logical dupes are detected from a list cleaned of playlist dupes, physical dupes are detected from a list cleaned of playlist and logical dupes, and alternate dupes detected from a list cleaned of playlist, logical & physical dupes. I'm still wrapping my head around the best way to make sure that the metadata that is being kept (ratings, playlists etc.) is applied to the correct file when I move that code into a different section. 😕

tt2

The feature we want is "Look in this folder and add to iTunes any song you don't have already / haven't imported previously."

Giving iTunes this feature would make it much easier for people to use content purchased from other stores, use devices from other vendors, and use different music players. Each of these works against Apple's interests. It's never going to happen. End of story.

iTunes already does that just fine as long as either the folder you want it to scan is the iTunes Media folder or is somewhere inside it, OR the folder you want it to scan is anywhere, but the option to Copy files to the media folder when adding to library is not enabled...

tt2