Skip navigation

how to delete duplicated songs all at one time?

273626 Views 308 Replies Latest reply: Apr 13, 2014 2:50 PM by turingtest2 RSS Branched to a new discussion.
1 2 3 ... 21 Previous Next
Suavecito69 Level 1 Level 1 (0 points)
Currently Being Moderated
Aug 28, 2011 10:29 AM

I would like to get help deleting duplicated songs, I do not know how they got their in the first place.

Windows 7
  • Allan Sampson Level 10 Level 10 (122,145 points)
    Currently Being Moderated
    Aug 28, 2011 10:31 AM (in response to Suavecito69)

    With Music selected under Library in the iTunes source list, at the iTunes menu bar go to File and select Display Duplicates. Select and delete the duplicates there.

  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Aug 28, 2011 8:22 PM (in response to Suavecito69)

    iTunes has two modes for displaying duplicates. The first is accessed via File > Display Duplicates and lists all tracks in the current playlist where the same track title occurs more than once. This is, more often than not, a rather loose definition of the term and can bring back obviously different songs, e.g. studio and live versions of the same song, the same song recorded by different artists, or by the same artist but on different albums, or completely different songs that have nothing in common but their title.

     

    A more useful feature can be found if you hold down the SHIFT key and then select File > Display Exact Duplicates. This gives a list of all tracks where more than one track has essentially the same metadata, name, track no., artist, album, etc. (I'm not sure what the exact criteria are for an exact duplicate as far as iTunes is concerned, but it's good enough for most needs).

     

    There are three types of files that can show up as exact duplicates:

    1. Multiple library entries for the same physical file
      These can occur if your media location is temporarily unavailable, e.g. due to a disconnected external drive, iTunes becomes aware and marks the tracks as missing, you then reconnect the drive and add the media again. iTunes won't recognize that the files are already in the library and will create duplicate entries.
    2. Multiple copies of the same physical file
      This can happen if you have "Copy to iTunes media folder when adding to library" enabled and accidentally add the same folders from outside of the iTunes Media folder into your library.
    3. Different copies of the same track, in different formats or bit rates
      This can happen if you decide to rip a CD that is already in your library on a later occasion or, for example, decide to create MP3 versions of some existing AAC files and then don't remove the originals.

    Note you could also have files which have the same audio data but different metadata, e.g. the same album ripped twice but with a slightly altered album title, however there isn't an obvious way to highlight these within iTunes.

     

    Once you have used the Show Exact Duplicates feature you can set about deleting all but one copy of each duplicate file. If all the duplicates have recently been added into your library then you could sort the list on the date added column, select a range of tracks with SHIFT-click and delete all the recent entries. If the duplicates have been added into the library at various times then sort the list by album or track name, select all but one of each group of matching tracks with CTRL-click and delete the selection. Probably best to work one screenful at a time.

     

    Of course the issue isn't quite as simple as it seems. If the duplicates are of type 1 then you want to delete the tracks from the library, but not delete or send the underlying files to the recycle bin as there is only one copy of each file referenced by the duplicate entries. For tracks of type 2 you do want to delete the duplicate physical copies leaving one remaining version. For files of type 3 you probably want to decide which of the two or more copies you want to keep, the smallest to take up less room, the largest because it is the best quality, the MP3 version because it is the most portable or more than one because each has their uses for you.

     

    By now you should have some idea of why deduping is not a trivial issue. Of the three types I have described it turns out that cleaning up type 1 duplicates is quite easy to automate. I've written a script called DeDuper which can go through a selection of tracks, spot those with the same path and remove the redundant entries. As a bonus it retains the entry with the earliest date added value, adds in the play & skip counts from the deleted entries and sets the last played/skipped dates to their most recent values. Although the script would work on the whole library it will be much more efficient to use the iTunes Show Exact Duplicates tool before running the script. You should also backup your library or, at the very least, the iTunes database iTunes Library.itl before running the script. Chances are you have type 2 duplicates and the script is of no use to you in its present form...

     

    I hope to extend the script over the next few days to cope with type 2 and type 3 duplicates, ideally making the distinction between the two types, handling type 2 automatically (hopefully moving the removed files to somewhere they could still be restored from if required) and prompting for the user to choose which file to keep in the case of type 3. Again metadata would be merged. Hmm, just realized I need to do something about rating too.

     

    tt2

    Various PCs, iTunes 10.4.1.10, Windows XP Pro, iPhone3GS,iPad 2,iPod Classic+Nano
  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Aug 29, 2011 10:23 AM (in response to turingtest2)

    Spotted a race condition error and have uploaded a new build which also catches track ratings. Working on the next phase.

     

    tt2

  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Aug 29, 2011 1:42 PM (in response to turingtest2)

    I've completed a new version of the DeDuper script that can now automatically delete redundant copies of both type 1 and type 2 duplicates where type 1 are identified by having the same file path, and type 2 by having the same artist, album, disc no., track no., track name and file size. As before the earliest imported copy of each file is preserved, play & skip counts are combined, the latest played or skipped dates are set and, in addition, the highest track rating is preserved. Type 2 duplicates which are deleted are sent to the recycle bin, but given the tests of both metadata and file size it is extremely unlikely for these files not to pure byte by byte duplicates that we are happy to delete. I'll leave type 3 for cleaning up by hand.

     

    The script isn't very fast, at least not for me, but that is probably down to old hardware and a largish library. iTunes rewrites the entire database after every minor change so the larger the library, the slower a script will run. Test it on a few hundred tracks and, if you are happy it performs as described, leave it to run while you do something else. The program runs in two separate stages and after each you can decide whether or not to proceed with the removal of redundant copies.

     

    tt2

  • tcminyard Level 1 Level 1 (0 points)
    Currently Being Moderated
    Sep 27, 2011 6:09 AM (in response to turingtest2)

    I'm having trouble with the DeDuper script.  It fails with a VBScript error (Code 800A01A8, if that helps) on Line 443 Char 3 with the following error: Object required: 'objFolder'

     

    It seems to occur when there are 4 duplicates.  Most of the songs being processed have 3 duplicates, but when it hits one with 4, it deletes two of them then fails with that error.

  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Sep 27, 2011 7:24 AM (in response to tcminyard)

    Thanks for the feedback. I'll have a look into that and get back as soon as I can.

     

    tt2

  • tcminyard Level 1 Level 1 (0 points)
    Currently Being Moderated
    Sep 27, 2011 7:38 AM (in response to turingtest2)

    I went ahead and debugged the problem.  It turns out there were '/' characters in some of the names pulled from playlists, because they were done on a Linux system.  That's what was confusing things.  I'd suggest a global replace of '/' with '\' before

     

    Set objFolder=objShell.Namespace(strFolderParent)

     

    to avoid that problem.

  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Sep 27, 2011 7:56 AM (in response to tcminyard)

    Great, thanks for that. I'll post a revision up later. I'm on my iPad rather than at a PC just now...

     

    tt2

  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Sep 27, 2011 9:56 AM (in response to turingtest2)

    Hmm, so in your case iTunes has a record of the location as say:

    <Some Path>/iTunes/iTunes Media/Music/<Artist>/<Album>/## <Name>.<Ext>

    instead of :

    <Some Path>\iTunes\iTunes Media\Music\<Artist>\<Album>\## <Name>.<Ext>

    does it? I guess this could potenentially happen with libraries/playlists migrated from Macs too. I should probably correct for that error earlier in the processing as it might be missing some logical dupes (two entires to same file) with the current code.

     

    tt2

    Various PCs, iTunes 10.4.1.10, Windows XP Pro, iPhone3GS,iPad 2,iPod Classic+Nano
  • tcminyard Level 1 Level 1 (0 points)
    Currently Being Moderated
    Sep 27, 2011 11:38 AM (in response to turingtest2)

    Yes, I believe that's the problem.  You are probably right, it could happen getting playlists from Macs, too.  I made a quick hack and it fixed my problem, but I don't know anything about VB, and I'm sure my hack isn't complete.

     

    This is all kind of silly, anyway.  iTunes should ignore duplicates if they are in exactly the same place in the filesystem.  Oh well.

  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Sep 27, 2011 1:42 PM (in response to tcminyard)

    A second attempt to add the same file will normally be ignored. However if an existing file isn't where it is supposed to be (e.g. external drive offline), and iTunes gives it the exclamation mark, then you can restore the file (e.g. connect the drive) and add it without iTunes detecting that the file is already in the library. Next time you attempt to access the "missing" file, iTunes will find it again, but now you have two files with the same file path aka "logical dupes" or "type 1" as I called them earlier.

     

    I've posted an updated version of the script which will replace "\" for "/" in the paths for the logical dupe detection cycle and when attempting to send a file to the recycle bin so that should cover that issue.

     

    tt2

  • dbois Level 1 Level 1 (0 points)

    probably my lack of tecnical expertise, but when I hit delete I don't think the underlying file gets deleted?

  • turingtest2 Level 8 Level 8 (43,845 points)
    Currently Being Moderated
    Nov 8, 2011 12:17 PM (in response to dbois)

    dbois wrote:

     

    probably my lack of tecnical expertise, but when I hit delete I don't think the underlying file gets deleted?

    If the file is inside the iTunes Media folder and you delete it from say the Music source (rather than a playlist) then you will get an option to delete the underlying file. If deleteing from a playlist Shift-Delete will offer to remove it from the library, and then (if inside the media folder) ask whether it should move the file to the recycle bin. Ctrl_Shift-Delete will just ask the last question. If a file is outside the media folder iTunes will never delete it should you remove it from the library

     

    My DeDuper script recycles all files it sees as physical duplicates (triplicates etc.) regardless of their location.

     

    tt2

  • viaTech Level 1 Level 1 (0 points)
    A more useful feature can be found if you hold down the SHIFT key and then select File > Display Exact Duplicates.

     

     

    uringtest2 is spot-on. I'd only add that the correct Key-Click combination for "Display Exact Duplicates" under iTunes v10.5.2 (11) is to hold down the OPTION key and then click the File menu.

1 2 3 ... 21 Previous Next

Actions

More Like This

  • Retrieving data ...

Bookmarked By (7)

Legend

  • This solved my question - 10 points
  • This helped me - 5 points
This site contains user submitted content, comments and opinions and is for informational purposes only. Apple disclaims any and all liability for the acts, omissions and conduct of any third parties in connection with or related to your use of the site. All postings and use of the content on this site are subject to the Apple Support Communities Terms of Use.