Transferring, converting and duplicate problems.

Apple's official advice on duplicates is here... HT2905: How to find and remove duplicate items in your iTunes library. It is a manual process and the article fails to explain some of the potential pitfalls such as lost ratings and playlist membership.

Use Shift > View > Show Exact Duplicate Items to display duplicates as this is normally a more useful selection. You need to manually select all but one of each group to remove. Sorting the list by Date Added may make it easier to select the appropriate tracks, however this works best when performed immediately after the dupes have been created. If you have multiple entries in iTunes connected to the same file on the hard drive then don't send to the recycle bin.

Use my DeDuper script if you're not sure, don't want to do it by hand, or want to preserve ratings, play counts and playlist membership. See this thread for background, this post for detailed instructions, and please take note of the warning to backup your library before deduping.

(If you don't see the menu bar press ALT to show it temporarily or CTRL+B to keep it displayed.)

The most recent version of the script can tidy dead links as long as there is at least one live duplicate to merge stats and playlist membership to and should cope sensibly when the same file has been added via multiple paths.

See also Make a split library portable.

tt2

I doubt you have had time to read the background thread to the deduper script yet. Enabling the show duplicates feature doesn't by itself do anything to clean up the library. Broken tracks (that aren't where iTunes thinks they should be, and multiple connections to the same file can cause problems when cleaning up. I suspect that assumptions you've made here in the past are part of your problem.

Unchecked tracks will be skipped during regular track to track playback, are often excluded from smart playlists, and nor normally synced to any device. I use them for things like bonus interview tracks which I keep in my library but will rarely want to play. Unless you have a particular reason for unchecking tracks I would leave them all ticked. Check marks (in the music list) are not the best way of controlling what syncs to a device.

tt2

Wasn't meant to be any undertone. 🙂

I obviously don't know what you have and haven't read before you posted, but I may have assumed from your second post that you hadn't really looked at my first one. Apologies for that.

Anyway, the checkmarks don't have any other purpose in the show duplicates display from normal. though I guess one could use them to mark things out in that display if one wanted to, then delete later using a smart playlist of unchecked tracks.

The deduper script arose from a conversation about the difficulties of mechanically deciding which tracks should and should not be removed by any automatic system. It isn't completely perfect because it can only detect tracks with the same metadata, not the same audio data, but I think it beats working by hand.

One of the issues with duplicates is when two entries in the library are connected to the same physical file. In the background thread I have termed these logical duplicates. Deleting either entry and the underlying file breaks the remaining entry in the library because there was only really one file to start with.

On the other hand if you are trying to recover wasted space taken up by physical duplicates on your drive then it isn't enough to simply remove them from iTunes without trashing the files related to the copy you remove. You will tidy up the interface in iTunes, but won't recover any space.

There is a tool called iTunes Folder Watch which can be used to scan the media folder and can tell you which files in the media folder are not connected to the library and (with the option check for dead tracks on start up enabled) which entries in iTunes are not connected to any file. If you reconnect everything in your media folder to iTunes (you could also just use File > Add Folder to Library for this) then my script should be able to safely dedupe and recycle where appropriate.

I cannot understate the usefulness of a complete backup before you start. I recommend you purchase a second external hard drive and backup using the method suggested in Backup your iTunes for Windows library with SyncToy.

When you have cleaned the duplicates from your library there is more advice at Grouping tracks into albums on organizing it to the best effect for an iPod classic. In particular it doesn't group on album artist, so any albums that have guest/featured artists may break up or repeat on the iPod even when all looks well in iTunes.

Regarding the files with exclamation marks. The "missing file" error happens if the file is no longer where iTunes expects to find it. Possible causes are that you or some third party tool has moved, renamed or deleted the file, one of its parent folders, or the drive it lives on has had a change of drive letter. It is also possible that iTunes has changed from expecting the files to be in the pre-iTunes 9 layout to post-iTunes 9 layout, or vice-versa, and so is looking in slightly the wrong place.

Select a track with an exclamation mark, use Ctrl-I to Get Info, then click No when asked to try to locate the track. (Due to a bug in iTunes 12 you currently have to say No twice!) Look on the summary tab for the location that iTunes thinks the file should be. Now take a look around your hard drive(s). Hopefully you can locate the track in question. If a section of your library has simply been moved, a folder renamed, or a drive letter has changed, it should be possible to reverse the actions. If the difference between the two paths is an additional Music folder in one path then this is a layout issue. I can explain further if that is the case.

In some cases iTunes may be able to repair itself if you go through the same steps with Get Info but this time click Locate and browse to the lost track. It may then offer to attempt to automatically fix other broken links.

If another application like Windows Media Player has moved/renamed the files then the chances are that subtle differences in naming strategies will make it hard to restore the media to the precise path that iTunes is expecting. In such cases, as long as the missing files can be found somewhere, you should be able to use my FindTracks script to reconnect them to iTunes. See this post for an explanation of how it works.

If the files have been deleted then you will need to rerip or redownload from source.

tt2

You're welcome. 🙂

tt2

You can choose which format the WMA files are converted to from Edit > Preferences > General > Import Settings. You probably want to use a matching bit rate to the source files as anything higher will give you bigger files of the same quality, lower and you would lose some. Personally I switched to mp3 for wider comparability, but AAC may well be the best choice.

My recommendation would be that you set your media folder to E:\iTunes\iTunes Media and use the options File > Library > Organize Library > Consolidate files and Rearrange files in the folder iTunes Media to copy all of your iTunes media files into that location, and organize them in the modern layout.. You would then be able to copy the library files into E:\iTunes to make the library portable. If you are limited for space on the E: drive however you could use a number of steps as outlined in Make a split library portable to transform the existing folder at E:\Music HD into the portable layout without having to temporarily duplicate files.

I suspect that you have yet to consolidate after changing your media folder which is why you have yet to have the Books folder appear on the external drive. If the artist folders are in E:\Music HD rather than E:\Music HD\Music that suggest the media folder is currently using the old layout. See the split library tip for details. Consolidating makes copies, you have to manually go in to reclaim the space taken my the originals by deleting and then emptying the recycle bin.

If you're adding content from the same drive that the media folder is on, and don't want to duplicate, either move it into <Media Folder>\Automatically Add to iTunes if you are happy leaving iTunes in charge of organizing, or move it where you want it inside the media folder before dragging & dropping into iTunes if you want things in a personal layout. The automatically add folder has an advantage that it will show which, if any, files iTunes doesn't want to import by putting them into a Not Added subfolder. Problem mp3 files can be checked with Mp3val.

I would normally suggest you enable both the Keep... & Copy... options. Keep... only affects content inside the media folder, and makes the folder layout for the media files update to reflect any tag changes. Copy... tells iTunes to make local copies inside the media folder of any tracks added from outside the media folder. This is particularly useful when copying from removable storage. Anything ripped, purchased, or converted by iTunes is stored in the media folder in the standard iTunes structure regardless of the settings of those two options. I tell Amazon to download to <Media Folder>\Automatically Add to iTunes and (IIRC) not to add to iTunes, so that iTunes can manage adding the files and moving them where it wants to.

I keep my originals of my converted files, both the WMAs and the protected AACs that I burned & ripped so that I could play on my non-Apple player in days gone past. They are in <Media Folder>\Originals & Dupes\<Artist>\<Album> (but not connected to iTunes) so they are backed up with the rest of my library. An advantage of putting the WMAs here before trying to add to iTunes is that the converted files don't end up in the same folder as the originals.

tt2