how to delete duplicated songs all at one time?

I'm guessing your library is full of broken links. The script is designed to tackle the situation where you either have multiple links to the same songs or have physically imorted multiple copies to the media folder. The script FindTracks ought to be able to repair the broken links, after which DeDuper should be able to do its thing properly.

An alternative approach would be to use iTunes Folder Watch with its option to check for dead tracks on startup however this would likely remove the entries holding your ratings, playcounts, playlists membership etc. which is what DeDuper aims to preserve.

tt2

Im running this script now, through a playlist of 96k songs. Its really about 46k, but there are some duplicates, and triplicates. Question is, how do I know the script is running, and how will I know when it is finished?

I'm hoping this will work. I don't see a progress bar.

I have a message at the bottom of the prompt that says - NB Disable user access control to allow progress bar to operate... I'm not sure what it's asking of me.

The script will do its job quite effectively with or without a progress bar but when processing large numbers of files it is nice to be able to see that it is still working and get an idea of how long it will be before it finishes. Unfortunately the code to create and contol the progress bar uses a feature of Windows where one program (my script) is allowed to control another (an instance of Internet Explorer). Security features introduced in Windows Vista break this method. I don't have the resources to develop or purchase an alternative progress bar control that is compatible with UAC so the easiest approach, at least from my point of view, is to detect if UAC is active and suggest that you disable UAC if you want to see a progress bar while running the script. You can reset UAC later.

How to disable User Account Control

tt2

I've just learned the term "deduplication," and a program like DeDuper certainly holds great promise for me. I'm not interested in writing scripts, and the process of going through the library manually, using Apple's "exact" duplicate-sniffing system, is tedious, requiring going one alphabetical letter at a time and zapping every other line.

I was so frustrated I impulsively took up Amazon on their offer to give me unlimited storage of all my music without charge. However, it appears not to screen out any duplicates, and the process of uploading 70 GB is now in its 2nd week.

Theoretically, it should work to re-download all of the Amazon uploads, put them through the DeDuping program, then re-upload them (if Amazon's offer still holds). Apple's Cloud and Match service still leave me somewhat mystified about how much is being matched when and where. It may be far superior to Amazon's Cloud, but Amazon's system is straightforward enough to give me a sense of what's actually backed up. The major drawback seems to be the cumbersome Amazon player. If you think that manually deleting duplicates is a chore in iTunes, wait til you try to do the same in Amazon's player!

Cap

(I should also mention that Amazon's service returned a truckload of "protected" files (DDRM?), which is another problem. I don't know how involved it is to remove the copyright protection. As I understand it, Apple has no such restriction.)

turingtest2,

Thanks for the scripts. I used FindTracks on my library which seems to have worked great. Now I'm trying to use DeDuper but ran into a issue...

I'm not sure if it's because I'm using iCloud/MusicMatch, but after DeDuper analyzes my songs, I get prompts for each song it wants to delete with an extra option to remove it from iCloud. I have a ton of duplicates and was hoping I didn't have address each dialog box. Does your script provide a way around this that I missed? Can i change this behavior in itunes?

mnythx,

~m

Sorry, I don't have access to the iTunes Match service so I'm not able to explore and test its behaviour. I would think it best, if possible, to turn off the service, tidy the library, then turn it back on again. But again I really don't have any practical experience with how it works.

tt2

Thanks much for the quick and helpful reply. (My thought in redownloading my files from Amazon prior to deduping was that Amazon might eventually attach a fee to the service, in which case the smaller my library in the Cloud, the better.)

"Respawning" is a good word for the problems I'm currently observing. When I had more space than music files on my iMac, it didn't matter. Now that I've begun to receive continual warnings about a maxed-out hard drive, I'm paying the price. So besides freeing up space by eliminating dupes, I'm hoping to get my iTunes Music folders down to a single one.

I have the same question, but it appears to be a manual process. I somehow got all 13,000 songs entered twice into my library and don't want to have to manually select every duplicate to delete. Anyone have any ideas?

turingtest2:

Thank you so much for that script! I had an overwhelming amount of duplicates in my library and was at a total loss of what to do besides selecting them all individually and deleting them. I didn't even understand how they all got there in the first place.

Your script nabbed and deleted the majority of them (about 1200 duplicate songs) in less than 5 minutes, allowing me to deal with the ones that were duplicates because of misnaming or actual physical duplicates in the files. Your script worked quickly and smoothly and I am absolutely delighted, so thank you so much for putting time into scripts that you let other people use for free. This issue has been bothering me for AGES and I am so happy that I have been able to finally deal with it. I do wonder if you have any advice for doing the same thing in Windows Media Player?

My itunes is clean and beautiful and "I Feel Good" just started playing which describes my mood right now perfectly!

I used DeDuper and it came back with no duplicates found to delete. I have a siutaiton where the filename is different by a number at the end of the file name and the dates are different. Does DeDuper not handle this situation?

It uses the same matching criteria as the iTunes Display Exact Duplicates feature and works where the information in iTunes in the following fields is identical:

Artist, Album, Disc Number, Track Number, Name

Oddly enough Album Artist & Part of a Compilation are ignored, along with a few other details such as location and size that I use to distinguish the different kinds of duplicates. Another weird discovery I've recently made is that is possible for the same file to be logically duplicated so that two entries are point to the same file, and in principle are connected to the same tag, yet one entry can have Disc 1 of 1 and the other Disc <none> of <none>.

The script, in its current form, will only process files that can be found so it isn't any use for deduping if half the library is also missing in action. I'm considering a modification that would change that as long as at least one file could be located.

The type of file you describe is what I have called physical or alternate duplicates depending on whether or not the related files have the same size. Is the library connected to both files and do they show up together when you get iTunes to display exact duplicates?

tt2

Yes the library is connected to both files. They exist in the same folder and sometimes there are up to four of each mp3. No, when you do Display Duplicates or Exact Duplicates, they do not show up.

In that case there will be subtle differences in the tag information. If you want to post a screenshot showing Artist, Album, Disc Number, Track Number, Name for a few of your dupes so I can see what is going on I might be able to tweak the script to make it effective.

tt2

tt2: Thanks for the all the hard work! The DeDuper script is great, and almost what I'm looking for. If you're still accepting suggestions, I was wondering if you might be able to modify the script a little further.

My problem: My father has somehow ended up duplicating virtually his entire music library. Each song has a duplicate song in the same folder, only with an extra space and digit appended as a suffix. For example:

"01 I've Been Loving You Too Long.m4a"

"01 I've Been Loving You Too Long 1.m4a"

My preference is to delete all the duplicates with the suffix. Unfortunately, the DeDuper script doesn't consistently pick those duplicates for deletion, even with all other things being equal (e.g. Plays, Date Added). Might it be possible to expand the script to treat obviously-suffixed files as the duplicate?

Thanks!