Matching digitized vinyl iTunes imports

This is just a wild guess but my theory is all CDs have a digital "fingerprint". Thats how you can get track names from Gracenote when you import a CD into iTunes. Digitizing a vinyl record won't be an exact match, even though the ID tags match the copies Apple has. I'm guessing they are erring on the side that what you have is different from what they have. What would be cool (although I don't see it happening) is the ability to force iTunes to match those kind of tracks.

It may take some time to ferret out just how iTunes match works (or in some cases, doesn't) but I can assure you that at least in some instances, it DOES match vinyl rips. I know, because I've ripped quite a few myself and while not all songs end up matching, many of them do. Some records won't make it at all, but if they do, generally, 40%-60% of the tracks make it. There are a select few albums that have made it through 100% (Nautilus Superdisks), something that's not true even for some CDs. I'm in the process of comparing my vinyl rips to what is on the iTunes store in order to identify any discrepancies in track name, length, order (often vinyl track orders are different) that might in their totality keep a match from being made, but so far what I have are just a few clues to the mystery.

Given that I haven't yet heard of anyone else having success with the matching process when it comes to vinyl, finding out what it is that I do "right" is, I know, kind of major for anyone else wanting to repeat my success. It's important to me too, and I promise I'll keep working on it. You may have part of the answer: wow and flutter, clicks and pops, silence duration between tracks, tracks in different order, all could be factors. My guess is that there are many things that come into play, that Apple and Gracenote are using weighted criteria to determine what matches, what doesn't. If so, the key is to figure out which factors are the most important and to focus on those.

I won't put every detail of my recording process here, but assuming there's something in how I do it that affects the matching proceess, I will give you the basics. I use a Dual 1219, feeding a Yamaha CX-600U preamp, set in Direct mode. The Yamaha's output is fed directly into a MacBook Pro, so I'm at the mercy of its sound card. The Dual 1219 is a good table, but even using a strobe, I'm not about to say it's so speed accurate that I'm not losing or gaining appreciable time in my rips, compared to Apple's database. Even so, I've got songs matched that were off by a few seconds, so clearly that's not a deal-breaker. Conversely, I've seen songs with identical durations not match, so go figure.

I record using Audacity and only use two filters for posting in it. (1) DC offset remover because of the sound card (2) amplify to get the level of the entire album maxed (if I weren't lazy, I'd max each side separately, which I suspect is how the master cutter did it). I record both sides of an album at once and shoot for a -6 Db so I don't clip. Depending on the condition of the record, I may elect to put fade in/outs, but generally only on the first and last tracks of each side. I've recently started using ClickRepair which gets done after the above in Audacity, but I use it sparingly, sometimes not at all if the record's in good enough shape. I import back into Audacity to name my tracks and to output them. I leave the silence duration after each song intact (I don't replace it with true silence), though I do kill dead air at the beginning and end of each side.

(NOTE: I'm looking into whether first and last songs are less likely to get matched, I don't have that answer now).

Finally, while I record and process at 96000 Hz, 32-bit float, I output individual tracks at 44100 Hz in AAC format. These are the files that get imported into iTunes. Those files have some metadata in them, such as album and track names, genre. For some reason, artist name added in Audacity gets wiped before it hits iTunes, so I end up re-entering that, plus adding total number of tracks, something Audacity doesn't have a field for.

Below are the highlights of what I've discovered about the matching process in general and vinyl in specific:

(1) If you think that having a really good recording from pristine vinyl is going to improve the match, maybe so, but consider this. I have a beat up copy of Billie Holiday's Lady in Satin and some of those tracks matched. That recording was cleaned up using the click removal in Audacity, so it's still pretty noisy and probably more dull than it should be. My copy of Janis Joplin's Pearl is in better shape than Lady in Satin, but it's seen better days too. That one got cleaned up in ClickRepair. Several of those tracks matched too. Some tracks, like Mercedes Benz did not, but given the amount of silence, the noise on that one could have kicked it out, since if I think the music is going to suffer, I kick back the cleanup.

(2) The matching process likely puts weight on a scan of portions of the file that actually consist of music, which only makes sense, really. I point this out only to caution that just fixing track names, metadata etc. probably isn't going to do it. Songs in my library (from CD) with corrupted metadata, actually made it through. According to iTunes, this handful of songs had no metadata other than track number and total tracks on the disk. After the matching process, iTunes still showed them as being "track 04," but using iTunes to query Gracenote resulted in them quickly being matched and metadata restored. I suppose it's possible Gracenote was able to read metadata that was corrupted but that seems a bit of a strech to me.

(3) Getting something to "match" and getting it past "ineligible" are two different animals entirely. While it won't be the answer for everyone, all I had to do to get every "ineligible" track into the system was to make an AAC copy within iTunes. I would sort to show me ineligible tracks, batch process them, immediately delete the originals and have iTunes rescan for matches. They all made it through second time, though as usual, some matched, some uploaded.

(4) Deleting songs from your library and from the cloud (you get that option when you opt to delete a track) does NOT immediately delete it from the cloud. Trashing your local copies doesn't do it either. The tracks will be deleted from the directory you see in iTunes on the computer where you're working, but trust me, they stay resident on Apple's servers for some time. I know this because (1) enabling iCloud on an iPad after deleting albums from the MacBook Pro showed them still being there; (2) re-uploading those "deleted" tracks resulted in their status moving from "waiting" to "uploaded" in a heartbeat and never even showing in the status bar as being uploaded. I waited over 24 hours, tried it again, and same result. This is a real pain, as it makes experimenting with changes pretty much impossible.

Sorry that I don't have a magic bullet that will get your vinyl rips through. After all, my own success has been mixed. I'd say take heart, because it can be done, but until we determine just what are the most important factors to getting tracks to match, there's no way to know what you'll have to do to get your stuff to pass muster with Apple. Meanwhile, enjoy the music.

Message was edited by: JiminMissouri, fixed typo

I too have been disappointed by iTunes Match not matching alot of my digitized vinyl albums. The one in particular is Led Zeppelin (the first album) which clearly is available from the iTunes store. Yet none of the songs were matched. I was really hoping that the $25 subscription would have provided me with clean versions of my vinyl recordings.

Am in the same boat. Virtually none of my vinyl rips have matched, and in fact a lot of my CD rips haven't either. Compared to e.g. Shazam the algorithm seems very finickety. I expect this is to avoid false positives that would be acceptable to Shazam... Match has to be absolutely 100% to match a track, Shazam can just be 90%. What I suspect is that the algorithm is matching our songs, just not to the right confidence threshold...

I also wouldn't be surprised if a condition of record company permission to launch match wasn't to make that confidence threshold very high. It's not in their interest to launder terrible quality Napster era 128k rips into pristine 256k AACs. And unfortunately those of us with vinyl fall into that same boat.

What I'd like to see in future is something more like the way TuneUp handles artwork - i.e. the software gives you a list of possible songs and you choose the one that's right. I would be amazed if the record companies ever allow this though as essentially it would let you choose other songs you don't own and get them for free.

I do think that the matching algorithm will get some fine-tuning in future, though, so hopefully we'll see accuracy improve.

Track lengths are definitely one factor and in the case of LP rips they aren't going to match in many cases.

So if I match the length of the same song found in the iTunes Store by adding or deleting from my vinyl ripped song, you are saying that the iTunes Match might work?

It would probably get you more matches although it might not be that many more.

I have a lot of needledrops and actually prefer them over the CD / iTunes versions - so I'm actually on the other end of this discussion believe it or not. I am usually on the Steve Hoffman forums where other persons like myself frequent, btw (we are all vinyl nuts). Ideally I woudn't want any of my vinyl needledrops to Match.

Here's my setup, first of all: AT-LP120-USB turntable, AT120e or AT440MLa cart, Sherwood RX-4109 receiver into a DELL business laptop line-in. The laptop is running Audacity on Ubuntu as the Windows version doesn't support anything higher than 16 bit input (this is a known bug with Audacity for Windows). This isn't the best setup, but for my needs it works well. I usually record in 24/48 (as 96khz is, with my setup, overkill), then save as a 24-bit WAV file. I then run a tool called "ClickRepair" ($40 but definitely worth the money for my needledrops) to reduce the pops and clicks - it works much better than what Audacity gives you. I then take that cleaned up WAV file and import back into Audacity to break up the tracks and Amplify them, usually to -.5db. I usually keep the silence at the end of the track btw, I don't fade in/out and I don't cut it out.

With all this in mind, I just did two match attempts today and here's my results. I have FLAC sources and transcoded them to V2 MP3's first, for the import. I used V2's as I expected a lot of uploads. Will use my AAC transcodes for subsequent attempts:

- Beatles "Abbey Road" (US LP, 1970's pressing). 9 of 17 tracks matched (2,4,6,7,9,10,11,12,13,14,17)

- U2 "The Unforgettable Fire" (2008 LP remaster). 3 of 10 tracks matched (1,2,9)

I would have done more, but Match is very slow right now. 😟 But I'm going to take a big guess here as to the reasons why there are uploads vs matches:

- Masterings. Given the two different formats themselves (a vinyl mastering, sometimes decades old versus whatever digital mastering Apple uses), the waveform analysis they use will likely have a lot of misses. I'm fairly certain they are not using tagging, so I don't think it factors in the equation.

- Pops / clicks. With the given that they are using waveforms, I suspect that pops/clicks will be an issue. And if you use Audacity to remove them, the waveform will still be altered (it's why I use ClickRepair - it just does a better job with keeping the actual audio intact and not introducing distortion).

- Your gear. I know that, in my case, the line-in I'm using on my laptop introduces noise to my setup (I can see it in tools like Spek, when recording silence and then checking the results). I also had a grounding issue for awhile that I eventually corrected, but some of my older needledrops probably still have a very faint buzzing sound which would likely cause problems with my source recording.

- Track length. I've seen this crop up with CD rips and I bet it's a factor here as well. I don't keep track of whether my vinyl track is the same length as a CD copy, but it's likely to be a few seconds off in many cases. And remember that I keep the silence at the end of the track whenever possible.

Here's the ujltimate issue - Apple isn't targeting people like you or I. Their target audience are people who have either CD rips, iTunes purchased tracks, eMusic / Amazon tracks or "other". Needledrops or cassette dubs probably aren't a factor in the equation. To be honest, I was suprised that I had any tracks matched at all - I was expecting a very low % of tracks given all the potential pitfalls with a needledrop. Abbey Road, in particular, was over 50% matched which was a complete shock to me.

When Match wakes up, I'll try some more needledrops to see my results.

More results:

- PJ Harvey "Let England Shake" (2011 LP) - 9 of 12 tracks matched (1,2,4,5,6,7,8,9,11,12)

- Neil Young S/T (1970's LP) - 3 of 10 tracks matched (2,3,9)

- Fleet Foxes "Helplessness Blues" (2011 LP) - 7 of 12 tracks matched (1,2,3,4,5,8,9)

I picked PJ Harvey specifically because it's a new release, but also because I've heard the CD version and the sound differences between the two are hard to distinguish assuming the few pop/clicks are removed. And this is of interest because the match % on that one was very high for me. I still think masterings are a critical component to the matching process.

Also, another factor to keep in mind as far as your TT - pitch control.. On my AT TT, it does a pretty good job of keeping the pitch correct. But some turntables do a terrible job at this and there may not be any way of fixing it. I also have an old vintage Philips turntable and I had to adjust the pitch manually until it was correct.

I have a lot of needledrops and actually prefer them over the CD / iTunes versions - so I'm actually on the other end of this discussion believe it or not. I am usually on the Steve Hoffman forums where other persons like myself frequent, btw (we are all vinyl nuts). Ideally I woudn't want any of my vinyl needledrops to Match.

I've checked out the Steve Hoffman forums, but not for quite awhile. I have a Discogs account and the forums over there have been of some help. Most of what I've learned has been tied directly to the Audacity help resources.

http://www.discogs.com/user/jamesneal241

I then run a tool called "ClickRepair" ($40 but definitely worth the money for my needledrops) to reduce the pops and clicks - it works much better than what Audacity gives you. I then take that cleaned up WAV file and import back into Audacity to break up the tracks and Amplify them, usually to -.5db. I usually keep the silence at the end of the track btw, I don't fade in/out and I don't cut it out.

Aha! The previsoulsy discussed issue of clipping. I've left Amplify at the default. -.5db could be a significant factor, given as you pointed out in another posting, there is a lot of stuff out there that's clipped and people just don't know it. I'll be sure to make that change to my process.

I recently began using ClickRepair and agree with your assessment - certainly worth the $40, much better than the Audacity solution, which I stopped using very quickly. Until I got ClickRepair I was manually removing the worst damage and just leaving the rest. Interestingly, I put a manually cleaned copy of Pearl to iTunes Match, later a version run through ClickRepair. The results were the same (see above). Honestly, my experience is that iTunes Match is a lot more forgiving, at least in some cases, of what I'd call marginal source material.

The way you handle silence is pretty much how I do it. When I've done manual cleanup, I have found myself putting a great deal of time into fade in/outs and silence between tracks, simply because that's where damage is more likely to be an issue for my ear.

- Your gear. I know that, in my case, the line-in I'm using on my laptop introduces noise to my setup (I can see it in tools like Spek, when recording silence and then checking the results). I also had a grounding issue for awhile that I eventually corrected, but some of my older needledrops probably still have a very faint buzzing sound which would likely cause problems with my source recording.

I did have a grounding issue when I first got set up, but worked it out before I started recording. One problem I had was a buzzing from a light on the same circuit. It was coming from a dimmer switch - something I'd learned long ago to avoid when recording, but also forgot long ago. I do have a question as to whether Audacity's Bias Offset Removal, while clearly straightening things out on the waveform for me, is helping or hindering when it comes to matching.

- Track length. I've seen this crop up with CD rips and I bet it's a factor here as well. I don't keep track of whether my vinyl track is the same length as a CD copy, but it's likely to be a few seconds off in many cases. And remember that I keep the silence at the end of the track whenever possible.

I'm unsure how much weight this is being given, particularly when there are so many reports of people getting things "matched" only to find what they got was a shorter version than what they actually have. Also, my own experience has been that songs + - several seconds are often matching.

I do wonder though, if nailing the opening of a track, getting it as close to what Apple's version is, would make a difference. It would be interesting to take a song that matched, move the start point so there's a full second before the song actually starts, and see if it would still get through.

My guess is silence at the end of a track can't be very relevant to what matches. Nothing is more likely to be different between a CD and vinyl than silence duration and waveform. The day I see a clean, flat, uninterrupted line on a waveform of the "silent" section of one of my vinyl rips, well that's not a day I expect to ever see!

Here's the ujltimate issue - Apple isn't targeting people like you or I. Their target audience are people who have either CD rips, iTunes purchased tracks, eMusic / Amazon tracks or "other". Needledrops or cassette dubs probably aren't a factor in the equation. To be honest, I was suprised that I had any tracks matched at all - I was expecting a very low % of tracks given all the potential pitfalls with a needledrop. Abbey Road, in particular, was over 50% matched which was a complete shock to me.

Agreed that we don't fit the domographic, but for those of us who have more vinyl than CD source material (I think I'm past 4,000 albums now) and are interested in being able to use iTunes Match, the fact that it is possible to do so is tantalizing. My guess is that the need to implement a matching process that would allow for some pretty crappy MP3 files to get through is the very reason we're having any luck at all getting vinyl (and in your case cassette as well) to match. Before I tried it myself, all I'd read led me to believe it wouldn't work at all, so I've been as surprised as you at my results.

Thanks for all the info. on your own process. I'll dig through it and see what I might do to alter my own.

Message was edited by: JiminMissouri - Added note on -.5 Db on Amplify

I picked PJ Harvey specifically because it's a new release, but also because I've heard the CD version and the sound differences between the two are hard to distinguish assuming the few pop/clicks are removed. And this is of interest because the match % on that one was very high for me. I still think masterings are a critical component to the matching process.

My best match, all tracks but one, is Jaco Pastorius' self-titled. Only Opus Pocus, track one on side two, didn't make it through. I've noticed it seems as though songs that make it through are in groups, then one doesn't make it, then groups again. I'm wondering if in some cases being the first or last track on a side is significant. Outer and inner tracks are more susceptible to certain kinds of wear and damage.

Also, another factor to keep in mind as far as your TT - pitch control..

True. My Dual is an idler wheel. I use a strobe disc to dial it in, but admittedly I haven't been as religious about checking it as I should. I do think getting pitch as close to what Apple's using as a baseline is probably a significant factor in getting a match. I've read of people who knew the vinyl they had was mastered at the wrong speed, something that was corrected when it went to CD. In those cases, a match might require being able to adjust your playback pitch to approximate the CD correction - something I can't imagine having the patience for.

Here are the results of some of my vinyl matches.

Art Tatum - The Essential Art Tatum 7, 9 (this disc is in bad shape so I was surprised even two made it)

Billy Holiday - Lady In Satin 11, 12 (also in bad shape, so bad I haven't recorded the A side, so two out of six I submitted made it)

Bob James - Bob James Two 2, 4, 6 (50% match)

Buckingham/Nicks Nothing matched, but I'm not sure it's in the iTunes store

Chick Corea - Eye Of The Beholder 1, 3, 4, 5, 9 (there are variations in track order between vinyl and CD, songs on CD not on vinyl (or vice-versa, forget which)

Classics IV - Spooky 11 - so it matched the title track, which is probably somewhere on the store, but I don't think the actual album's available.

Creedence Clearwater Revival - no matches. Fine, with my luck I'd get the short version of SuzyQ which really isn't SuzyQ at all.

Dave Brubeck - Greatest Hits 5, 10 Again, most likely songs available on other albums and in the store.

Dave Brubeck - Time Out 1, 3, 4, 7 I've got three copies on vinyl of this one, so it's probably a good one to experiment with.

David Crosby - Wind On The Water 1, 2, 3, 5, 6, 7, 8, 9, 10, 11. Didn't realize this matched all but one until I looked. My copy is M-, but that in and of itself hasn't been enough to get other albums to match so well.

DIane Schuur - And the Count Basie Orchestra 1, 2, 3, 6, 7, 8, 9. This is a digitally mastered album I got still sealed. My guess is others she did around that time would match too - if they were in the iTunes store that is.

Donovan - The Hurdy Gurdy Man 1, 5

Earl Klugh - Heartstring 100% match

Issac Hayes - . . . To Be Continued 0% match. Tuneup has subsequently pointed out that the title on my copy was wrong (missing three periods). It's one on the list to fix and resubmit.

Madonna - True Blue 100% match

Miles Davis - Amandla 1, 2, 3, 4, 8 Recorded from a pretty clean white label promo

Peter Gabriel - Peter Gabriel 2, 5

Peter Gabriel - Security 3, 8

Peter Gabriel - Sledgehammer (maxi-single) 3, 4

Ray Lynch - Deep Breakfast 0% match

Rush - Hold Your Fire 100% match. One of the few albums I captured at 44100. Most are done at 96, but downsampled to 44100 when exported from Audacity.

Spencer Brewer - Emerald 7

Steely Dan - Aja 3, 6

Van Morrison - Moondance - matched all but last track on Side B

Van Morrisson - A Sense Of Wonder - Matched only track 1 Both Morrison albums were recorded in the same way and the one that matched was in worse shape than the one that didn't.

Vangelis - Direct 1, 2, 3, 6, 7, 9

Newport Jazz Festival All- Stars 0% match - may not be in the iTunes store.

William Ackerman - Conferring With The Moon 5, 8

I also have converted from vinyl a multi-disk compilation "solid gold party rock" Not in the store, but a good example of how iTunes may or may not match something if it's found on another album. I'd say I got a 40%-%50 match on that one

You might be right on the end silence versus the beginning silence - I did have one instance with a digital track (not vinyl) where it was uploading until I added a few seconds of silence, but that was just a one-off event. But most of my vinyl needledrops probably have more silence than their CD counterparts and I'm still getting matches.

I haven't heavily tested clipping, but I have to think it's a factor. Like the pop/click repair in Audacity, clipping alters the waveform (in this case, it chops off those peaks) and if they are using these waveforms for matching it makes sense that it would be affected if they set their threshold too high. I use -.5db just so it's slightly below peak and then my AAC transcodes usually do not clip (or if they do, it's very light) - when I use 0db and then transcode to AAC or MP3, it usually introduces clipping. I haven't used Audacity's offset (that's in the normalization tool, as I recall) - I prefer to keep the balance between both channels the same as what's coming out of my receiver. It's why I use Amplify vs. Normalize.

4,000 LPs? Yikes! Not even close - I think I have ~500 right now. Not sure if I'd have the room for 4,000. 🙂

EDIT: Reading your other post, now. I actually have Madonna's True Blue on vinyl, and have a needledrop of it. I'll test that next, once Match is fixed (it's very slow, today)

I haven't used Audacity's offset (that's in the normalization tool, as I recall)

No, I was referring to the filter DC Offset removal. Supposedly, some sub-par sound cards (and I'm putting the one in my Macbook into that category) somehow record stereo tracks out of phase of each other. You can see this in Audacity if you take a look at an area of silence. For me on one track I can see the waveform positive, negative at the same point on the other. When I run DC Offset, this gets fixed and what I get is a much flatter line where you'd expect it to be nearly flat. I've watched it run during actual music passages and it's clear it's making significant changes to the wafeform. I'm just not entirely convinced that what it's doing is improving things. I like the resulting sound, but there's always room for improvement.

4,000 LPs? Yikes! Not even close - I think I have ~500 right now. Not sure if I'd have the room for 4,000.

Sounds like the library a local friend I trade with has and he has the same space concern you do. 4,000 Lps - didn't count the 45's or the shellacs, which I don't have any way of playing. Contrast that with maybe a couple of hundred CDs and you can see why getting iTunes Match to work with my vinyl holds some interest for me. I don't have room for 4,000 other. Three level house - 2 walls in the basement are album storage, first and second floor have them in cabinets on shelves, and have to admit, leaned against the fireplace mantel - something I guess I'd better fix before we start using it this winter! It's an obsession, but at least it's a legal one!

then save as a 24-bit WAV file.

Could you give me some detail on how you're getting 24-bit WAV file out of Audacity? I see a custom option that might get me there, but I'd like to try and duplicate what you're using. I'm in the process of recording an album, one side at 96, the other at 44 to see if that makes any difference.