Q: how to delete duplicated songs all at one time?

I used DeDuper and it came back with no duplicates found to delete.  I have a siutaiton where the filename is different by a number at the end of the file name and the dates are different.  Does DeDuper not handle this situation?

It uses the same matching criteria as the iTunes Display Exact Duplicates feature and works where the information in iTunes in the following fields is identical:

Artist, Album, Disc Number, Track Number, Name

Oddly enough Album Artist & Part of a Compilation are ignored, along with a few other details such as location and size that I use to distinguish the different kinds of duplicates. Another weird discovery I've recently made is that is possible for the same file to be logically duplicated so that two entries are point to the same file, and in principle are connected to the same tag, yet one entry can have Disc 1 of 1 and the other Disc <none> of <none>.

The script, in its current form, will only process files that can be found so it isn't any use for deduping if half the library is also missing in action. I'm considering a modification that would change that as long as at least one file could be located.

The type of file you describe is what I have called physical or alternate duplicates depending on whether or not the related files have the same size. Is the library connected to both files and do they show up together when you get iTunes to display exact duplicates?

tt2

Yes the library is connected to both files.  They exist in the same folder and sometimes there are up to four of each mp3.  No, when you do Display Duplicates or Exact Duplicates, they do not show up.

In that case there will be subtle differences in the tag information. If you want to post a screenshot showing Artist, Album, Disc Number, Track Number, Name for a few of your dupes so I can see what is going on I might be able to tweak the script to make it effective.

tt2

tt2: Thanks for the all the hard work! The DeDuper script is great, and almost what I'm looking for. If you're still accepting suggestions, I was wondering if you might be able to modify the script a little further.

My problem: My father has somehow ended up duplicating virtually his entire music library. Each song has a duplicate song in the same folder, only with an extra space and digit appended as a suffix. For example:

"01 I've Been Loving You Too Long.m4a"

"01 I've Been Loving You Too Long 1.m4a"

My preference is to delete all the duplicates with the suffix. Unfortunately, the DeDuper script doesn't consistently pick those duplicates for deletion, even with all other things being equal (e.g. Plays, Date Added). Might it be possible to expand the script to treat obviously-suffixed files as the duplicate?

Thanks!

My initial aim was to keep iTunes connected to the first copy of a track that was added to the library. This will normally be the one without the trailing digit, though if you subsequently rename the artist or the album, trigger the restructuring of the library, or consolidate files that were previously outside the media folder, then this will become indeterminate. If both files have the same date added value that would suggest the files were already duplicated before being added to a new library making the choice somewhat arbitrary. I assume that in this case both files have the same size so we're not looking at preserving the larger, better quality file?

Although I probably could add some extra steps to the script it would be much easier to uncheck Keep iTunes Media folder organized, click OK, then turn it on again, assuming of course that you normally let iTunes organize your filenames.

tt2

Wow, your work and responsiveness is amazing.  Thanks! 

I'm running into a problem using DeDuper that you'll probably undestand in a few seconds, but I now have a flat spot on the front of my head from beating it against the wall:

---------------------------

Windows Script Host

---------------------------

Script:          C:\Users\Gerry\Desktop\DeDuper.vbs

Line:          421

Char:          3

Error:          This key is already associated with an element of this collection

Code:          800A01C9

Source:           Microsoft VBScript runtime error

---------------------------

OK  

---------------------------

Any help would be greatly appreciated.

Have you used the Show Exact Duplicates feature on the main Music playlist or have you used a regular playlist which could have had the same tracks added to it twice?

Each entry in the iTunes database has a unique identity, its LibraryPersistentID. In the line with the error it would seem that that very same object is being scheduled for deletion a second time. When designing and testing the script I hadn't considered that could be a possibility but when using a regular playlist as the source for the script it might be. Media sources and smart playlists should contain at most one instance of each LibraryPersistentID, but regular playlists can have the same song appear more than once if you wish.

Right now I can't think of another reason the error might occur. Funnily enough I've been thinking about changing the way my scripts work so that the first pass reads in all the LibraryPersistentIDs into an array (which I can see now needs to be deduped also for use with regular lists). That way if the script is processing a smart playlist in such a way the update causes the item to fall out of the list the script still has access to it for any other changes. So far it hasn't happened to me often enough to consider it really worth the effort, and then it has only been in a personal script where I've merged a few different fixes for tidying up podcasts into one script. Perhaps this is a hint I should investigate the idea further.

tt2

You are spot on!  It is a regular playlist.  And your answer gave me a route to fix it:

I made a Smart Playlist that contained all the contents of the regular playlist.  But as you pointed out the smart playlist will not allow duplicates.  Then I copied the new smart playlist to a new regular playlist that contained only one copy of the music.  Deleted the old regular playlist, deleted the new Smart Playlist, renamed the new playlist to the old playlists name and voila!  Same list, no duplicates.

I really appreciated your effort and insight!

Cheers!

3Orange1Blue

tt2

Hi Turingtest2

It looks like you've put a lot of work into this and in theory I want to say a huge thank you for what you're offering to people. But so far it's not working for me. I have over 12,000 duplicates, but I have been testing it on a smaller number and it's deleting the only copy of the file. I.e. I select two entries in iTunes that are pointing towards the same file on the local drive, and the script removes the source file instead of just removing the duplicate entry in iTunes.

Do you have any suggestions?

Michael Cawood wrote:

 

...I have been testing it on a smaller number and it's deleting the only copy of the file. I.e. I select two entries in iTunes that are pointing towards the same file on the local drive, and the script removes the source file instead of just removing the duplicate entry in iTunes.

Eeek! I hope you read my warning about backing up first.  At a guess you've got a "multiple paths to the same file" issue. I haven't yet worked out how to resolve that issue in a generic way but I should be able to cook up a workaround for a specific case. Could you post some sample paths? Are there just two?

tt2

Yes, and the recycle bin works just fine so nothing lost. This is basically what allmost all my track locations look like. This is on Windows, of course.

D:\Users\Name\Music\Album\Track_Name.mp3

OK, I wasn't specific enough. Could you post the exact paths as shown on the iTunes  Get Info. > Summary tab to two individal files that are treated as dupes so I can see why my script might treat them as "physical" dupes when you say they are really "logical" dupes?

tt2

Okay, I've read the first (2) pages plus the last page, so I apologize if I missed it. I do have a "backup" so I'm not concerned about having lost files at this point either....

Anyway, I've really hosed up my library after having a hard drive failure. I had copied my music from my iPod to more than one PC and when attempting to get ALL of my music back, in case all libraries weren't identical, I ended up copying multiple versions/copies. My iPod had (4) copies of the same song in some cases !!

At this point, I've copied my library to my wife's laptop and added them to her iTunes libary, then did Shift, File, Display Duplicates. Since I have thousands of songs x 2 or x 3 or x 4, selecting all but one of each is very impractical. I didn't realize that was what I was supposed to do as the first error message I rec'd was "Selecting at least two songs...". So, I did "Crtl+A" and then ran your script. I fear I am going to delete ALL of the songs at this point.

Did I click the 2nd dialog too quicly as I thought it had three options for cleaning up, with one labeled "safer" ? I clicked "Yes" on that dialog though.