Matching digitized vinyl iTunes imports

Roebeet, been a long time since I checked the forums. A quick scan didn't bring up much new on the subject, so below is my latest. Forgive me if it's old news to some.

To date, I have submitted to iTunes Match approximately 325 albums recorded digitally from vinyl, all recorded with iTunes Match in mind. That's about 2,900 tracks, 1,442 of which matched.

At first glance, that doesn't sound all that great, but a few factors have contributed to my overall track record being significantly lower than what I've achieved of late, most of which we've covered before: (1) songs, entire albums just not in the store, (2) experiments that simply made things worse for a time, etc.

The short story is I believe the better matches I'm getting now (95%-100%) come down my shifting focus to removing noise from the tail of the track, including the silence left on it, although other factors we all covered some time back certainly matter.

I believe the matching process needs to be able to sync to the track tail and anything that prevents that from happening, prevents a match. Further, the actual sample matched seems likely to be in or near the tail, as since I've shifted focus to the tail, I've been able to get matches for some tracks that had a fair amount of unrepaired damage early on (which I either repaired later or decided it best to trash my file once I had a good match).

To accomplish this I am still using ClickRepair, but applying break points to allow me to adjust settings differently for the main part of the track, for the tail, and finally for the silence.

After having reviewed "damage" flagged by ClickRepair down to an auto setting of 6, I've concluded that for far too many well recorded albums in good condition (true vintage albums with poor dynamic range & more narrow frequency response are a different matter), more music is being lost than damage repaired. In some cases, having the repair setting at 1 and still watching any proposed change over 6, I still see far too much music being removed.

Therefore, I often turn off processing for the body of the track, run in manual for the remainder, until it's clear noise amplitude is well above the music and ClickRepair is making good decisions on its own. Once I'm there I, put ClickRepair to 100 and full Auto. It then does its thing quickly and stops at the beginning of of the next track

With respect to the concept of iTunes Match needing a clean tail in order to sync, consider what happens in Audacity if you try to use the "Silence Finder" as a means of placing label markers at the beginning of each track. It works - so long as there isn't something in the silence that rises above the threshold amplitude level. One little nit above that threshold and the label is either mis-placed or an additional label is put right before the nit. As for why the tail and not the head, the fact that the tail of a track is usually only available to someone who has the full track (previews I'm aware of don't include the tail), plus the need to obtain and verify track length through means other than metadata are relevant.

Putting your own fade out over the original, followed by inserting true silence is a mixed bag. Putting aside the fact it ruins the continuity, the feel of the vinyl recording, you run the risk of altering the tail enough to cause matching problems. It can help if one wants to go that route, particularly on tracks that are so noisy ClickRepair just can't quiet things down below whatever threshold iTunes Match requires. In my experience it will improve your matching percentage, but I didn't find it to be as effective as what I'm doing now, and as I said, it's not something most of us want to do anyway.

With respect to the other side of the discussion, working to force uploads, all you've said still seems on target. My guess is if someone really wanted to force an upload, based on my current understanding, placing a simple, soft click or any more subtle, pleasant noise shortly after the tail ought to do it. My guess is if it's loud enough to trip up Audacity's silence finder, it will trip up iTunes Match. Not optimal, but then short of the magical force upload button suddenly appearing, nothing really is.

There's a lot more to my current process, which I'll be glad to explain if anyone is still interested in the subject.

Hi Robeet,

I agree that a reduction in rumble or anything else that just wouldn't be in the iTunes version could improve matching. To that end for a time I was using the Apple Low Shelf filter in addition to the rumble filter on my preamp to knock down everything in the 10-20hz range. I was never able to verify that it mattered for matching (tried some problem tracks both ways and nothing changed), but felt it was the right thing to do for my rips anyway. However I also believe that it's one thing to keep low frequency from ever hitting the sylus, quite another to remove it after the fact. I suspect once its hit the stylus, the low frequencies alter the music in a way that filters simply can't counter. Wouldn't surprise me if this was heavily documented by audio engineers in the middle of the last century.

While I've not moved from my Dual 1219, I have made some simple changes that have knocked down rumble to the point where using the Low Shelf Filter is no longer necessary. One day perhaps the rumble filter can be removed from the equation as well. In any event, placing some damping material inside the plinth helped quite a bit, as has using a center weight. It's certainly possible that if I had not made those changes, I never would have discovered that processing tails and silence could make or break a match.

I recall there was some discussion regarding whether it was possible to match vinyl to albums only available as remasters in iTunes, though the discussion focused on Beatles tracks.I can confirm that in some cases vinyl will match with remasters (other than Beatles). I got a 100% match last week to Johnny Hammond's Wild Horses Rock Steady which is onlly available as an anniversary issued remaster. Apparently the original KUDU holds up pretty well after all these years.

Hi,

Like you I have had good success matching vinyl rips.

I use project turntable with built in analog to digital converter and connect by USB to my mac. Audio hijack pro is used to capture record as 24 bit AIFF files.

I use Denoise LF at default settings, then Click repair to decracle, then again to de click. I then use Denoise at default levels.

I then edit track in Audacity to get track to be the same length as store version and use normalise function. Sometimes use click removal or repair to fix tracks that Click repair missed.

File is exported as Aiff and added to iTunes. This format along with lossless appear to be causing problems in latest version of iTunes but that is another story.

Do you have any suggestions to help me improve match process, specifically the settings on Denoise and Click Repair.

Jim

Jim, I'm 3 hours north of you, in Des Moines, IA. I haven't done much more matching since last Spring, but that stack of unripped vinyl has been calling to me, of late. Although I've had reasonable Match success, I know that it could be much better. I'm not an audio guy. And, while I get a lot of what's being talked about, there's more that I don't understand, and there are methods being talked about that I don't understand, and software that I don't have/have never used. Is it possible (and would you be interested in) to take a discussion offline to real time? Like hauling my setup down and doing a rip session together? B lehn ertz at me

Jimzgoldfinch - My overall opinion of your process is that for any record in good condition, you're doing far too much alteration of the master music file, regardless of whether your primary objective is to get more matches or to retain as much of the quality of the source material (vinyl) as possible. Offering specific settings recommendations is something I generally don't do.

Everyone has their own set of ears, their own objectives, not to mention differences in equipment. While I have little desire to remove every nit from a recording made from vinyl, someone who didn't (as I did) grow up listening to the medium, someone who listens primarily to digital source material via earbuds may well consider any noise during quiet passages and between tracks unacceptable and do everything possible to wipe it out. For me, the only reason to remove as much noise as possible between tracks is because I believe it's an impediment to the matching process. A nit here or there within the body of a track just reminds me of its origin and I derive some pleasure from that.

Because of that, instead of a "try these settings" approach, I try to show people how they can better educate themselves, so they can have the measure of control over their process necessary to tailor it to their own needs. Apologies if what follows is long-winded. Shorter responses normally come after the first one, I promise. ;-). Just keep in mind as long as it may take to read what I write, I assure you it took me much longer to write it!

That said, less is more when it comes to capturing vinyl. There are many apps intended to "clean up" vinyl in one way or another, but I'm not aware of any that don't in the fine print make it clear that the music is, to some extent, affected by the process. Steps that involve import/export, downsampling/upsampling, etc. should also be kept to a minimum, the theory being that each unnecessary step brings with it the risk of changing the music in some way, however imperceptible.

Besides, every step takes more time - time better spent listening to music.

So if a "cleanup" process isn't absolutely necessary, I say don't use it. If you can't hear it (or if you can it doesn't really bother you), then if it can't damage your equipment, or doesn't adversely affect the matching process, forget it. If it is something that needs to be addressed, use the lowest settings possible. Default to me is a dirty word. There is no "default" when it comes to processing vinyl. Every disk is different and much is lost by allowing any process that alters the file to do so unattended. Thus, I'm not a fan of "auto" anything either.

The human ear, coupled with the human eye can do amazing things. Train them. Use them.

With that, I'll try to address each step in your process in the order you presented it.

Denoise LF -

I tried a demo copy of Denoise LF, but wasn't happy with the results. It reminded me of the results I got by using Audacity's Denoise filter, which I quickly abandoned. I understand the concept though and as I said earlier, the idea of de-emphasizing LF does make sense if it is significant enough to cause speaker damage. I believe a better way to go is to either remove the source of the LF entirely, or to prevent it from getting to the stylus. A good turntable mat, coupled with a center weight/clamp can do wonders in that regard.

Persons who have studied the waveform of a purely digital source are used to seeing true silence, a waveform devoid of all signs of the tell-tale "snake" of LF that runs throughout the music, a signature one can really obsess on if you work in Audacity a lot and work in ways that exaggerate amplitude to the max. I suspect they are more likely than I am to consider the goal making the waveform perfectly flat. I think that's where many people get into trouble when working with files sourced from vinyl.

But if you are intent on using Denoise LF, I would strongly suggest using Audacity to examine what it's actually doing by examining "before and after" spectrograms. I normally look at the first minute, a 60-second sample three minutes in, and a third one around 15 minutes in. There are reasons why I sample different sections of the record. LF noise is generally worst at the edge, but unless it's horrible (an edge warp), since it's not a representative sample, I don't use as a guide to my settings. But understanding the overall signature in the 10hz-20 Hz range is critical in those few instances where I opt to apply the Apple Low Shelf Filter (an alternative to Denoise LF).

To do this, take a good look at the plateau or hump in the spectrogram in the 10-20 Hz range. If it is well below the peak amplitude, in the -50 Db range or lower, then I wouldn't do anything to that particular album. For me, well done recordings, like those on KUDU, CTI, Sire and many others, there's simply no need to do anything other than apply the rumble filter on my preamp. If I weren't using an idler wheel drive TT, I doubt I'd need even that, as it's the drive of my TT that is the limiting factor.

Decrackle -

With respect to "Decrackle" in ClickRepair, it's intended to be used only on 78's, which have a different noise profile than Lps. Use of it on an LP may result in some noise removal, sure, but if the LP is in good condition, there's nothing it should be removing that the "DeClick" process won't. Personally, unless you are processing 78's I would leave this step out. Not only is it unnecessary, it is likely destructive when applied to Lps.

Declick -

The "Declick" in ClickRepair does not simply remove clicks and pops. The name is really quite misleading, because what the app is doing is trying to identify anything and everything that is not music and "fix" it. That means even little, soft nits that may be present throughout the recording, but are audible only in quiet passages, will often get picked up by ClickRepair.

As I said in an earlier post, if you listen to the record, or to the unprocessed sound file and only hear a few clicks, you are better off using Audacity's repair tool to manually remove them. ClickRepair will remove them too, but it removes a lot more, some of which unfortunately will be music if you're not doing a lot of work manually.

As for specific settings, there are plenty of recommendations out there and early on I tried many of them. I've read accounts that suggest that as people learn more about ClickRepair, the refinements they make to their settings generally follow the same path. They learn that ClickRepair actually has several default settings, some intended to work better on music with brass instruments, others with lots of percussion. From there they normally end up reducing the "Declick" setting further, but often don't make a similar adjustment to the "Auto" setting.

If one doesn't drop the Auto setting down significantly, they're still letting the app work largely on its own. It's easy to conclude a reduction in the Declick setting is all that's necessary, because you'll see the file process more quickly and will see fewer spots highlighted as having been "fixed" in the zoomed out waveform at the top of the window.

If you set Auto at 50 as many people do and reduce the sensitivity a little at a time, you'll certainly be rewarded by seeing the app running faster, stopping less often. For many people, that may be all they need to do, all they've got patience for. Honestly though, good as ClickRepair can be, its author is the first one to tell you it's not as good at making decisions as you are. I believe the strongest feature of the app is its ability to cue up spots that may be noise or damage and let you make your own decisions. Even in cases where you are certain a repair is warranted, you will occasionally decide you can do a better job manually than what ClickRepair proposes to do.

Reducing ClickRepair's Auto setting, so you can review practically all proposed changes is highly educational, albeit a tad frightening once you see what you've been letting the app do to your music in default mode. Try using an auto setting of six and DeClick at 50 on the silence between passages and you'll learn a lot about what true noise and damage looks like, as well as how large some of the damaged segments can be.

But do the same experiment on the very beginning of a song where the first note is solid and strong and chances are you won't like what you see. Do the same thing on a passage by your favorite sax player using the default "brass" settings, but again set Auto to 6. Check out what it's still trying to do to your favorite sax player.

You'll learn even more if you first open an unprocessed sound file of an entire album in Audacity and study the peaks, then run it through ClickRepair and study it again. If while studying the unprocessed sound file, you identified music peaks that were significantly higher than the rest (one or two hard struck individual keys on a piano, or those few times the drummer really came down hard), you may discover that during processing, ClickRepair grabbed those notes and dropped their amplitude, removed what made them stand out above all else in the music.

Once you've learned what ClickRepair, if left unattended can do to your music, obviously the next step is to figure out how best to keep it from making as many unwarranted changes as possible. To this end, I recommend using the app's "Abort" feature.

If you use breakpoints as I do, you can safely process the fraction of a second of silence before the first track in with Declick at 100 and full Auto mode. If you then drop Auto to six and click through a dozen or so proposed changes, you will see fairly quickly whether the app is going to alter more of the music than is acceptable to you. If that happens, simply hit abort, choosing the option that lets you simply start over, make adjustments to your settings and try again. Eventually you will either hit on settings you are comfortable with, or you will, as I often do, opt not to let Clickrepair process the body of the track.

A caution though. You're still only sampling the first track. While this should get you in the ballpark, every track is different and since Abort makes you start the entire process over, it's best to start every track with DeClick set very low, auto set low as well. If as you start processing you think it's safe, slowly increase your settings.

I also recommend using another feature of ClickRepair to help verify your decisions. On the upper graph, at the far right, there is a little marker that corresponds to the point ClickRepair has stopped at. Drag across that marker, then hit play and you are hearing the area in question. Assuming you're using a good set of headphones, if the proposed fix is actually noise (noise that matters) you should hear it. If you're not certain, hit the "fix" button while the segment is playing. You can toggle between "fix" and "undo" while it plays. As much as I've worked with ClickRepair, I'm still often tricked into thinking music is damage, so listening to proposed changes remains quite important.

Audacity's "Click Removal" vs. "repair" -

I don't use "click removal" ever. I tried it, but after examining the file more closely, decided the time necessary to use the "repair" function to manually make corrections was time well spent.

Audacity's "Normalize" function -

I don't use it, but I know many people have no other means of correcting channel imbalance. Instead I audition enough of each album prior to recording to know if there is an imbalance worth correcting. if it's quite obvious that uneven groove wear has caused a problem, I make adjustments using my preamp's balance control. If there isn't a readily apparent imbalance, I leave well enough alone.

I hope this helps.

Hi Jim.

Thanks for your extensive response and will consider some of your suggestions.

I have been fortunate to have reasonablely good record decks over the years and in general my LP's are in good condition but some are now 45 years old, the last LP I bought would have been around 1990.

I am pleased with the results I have achieved over the years but long gone are the days when I looked a good performance from my Hifi equipment. I suppose I'm less critical these days and probably enjoy my music more these days without seeking ultimate sound quality.

I think that I can get more LP's to match but I am looking to get the best out of my LP's that not available as digital downloads.

Thanks again.

Jim

MacDiverIA - Sounds like getting together in the Spring might be a good idea. At the moment I'm sure you know we're kind of snowed under down here. Net was down a good part of the day too and I'm just getting back up. A few short things for you to consider. Audacity is free and while the interface is very non-Mac like, once you get used to it, the app works well. ClickRepair and DeNoise LF are very cheap and are the creation of the same fellow.

Another reason to get together once daffodils start showing up is I want to get my preamp and an integrated amp downstairs in for some service/repair. The preamp is a Yamaha CX-600U picked up on Craigslist. The integrated amp is a Sansui AU-719, picked up at a local Salvation Army. I bring them up along with the software I use to show that with patience, good things can be had fairly inexpensively. Even my Dual 1219, which came with a Shure V15 MKIII cartridge was a very lucky $10 CL buy. I think my most expensive purchase to date has been a JICO SAS stylus for the cartridge - money well spent. - formal version of jim dottt neal att same place as you. Have fun figuring that one out!

As for not being an audio guy, while I worked in radio & tv production in my 20's (I'm 60 now), and spun records for pocket change and free drinks, what I know is still pretty much the result of toying with things. I'm sure true sound engineers just laugh and shake their heads at me.

Hi Jim,

My lengthy response to your questions was done keeping in mind that others interested in vinyl might happen upon it. My own record collection, some 6,000 Lps, uncounted 45's and a handful of 78's, are probably as old as what you have and much of the collection could well be in worse condition than what you have, since I gather you purchased them new. I admit I've bought much more than I've had time to clean and listen to, let alone record. Most of my collection was purchased second-hand from various sources in the last 5 years, although there is that one box that back in the 70's got put in the back of the car every time I moved. Admittedly those albums are the worst of the lot!

Anyway, while I do evaluate every disc before purchase, not everything is mint by any means and ClickRepair does come in very handy for me. I've just come to understand how much I lose if I don't ride herd on the app. For me it is important to retain as much of the music as I can within reason of course, particularly because I found I can tell the difference between the record and digital if I just let ClickRepair have its way. I admit I also enjoy experimenting and trying to improve my recording process. We all need a hobby, right?

Besides, while I like to listen to the originals when I can, often it's not possible, or simply not convenient. Firing up the AppleTV while I work around the house works pretty well for me. I get a lot of satisfaction also out of knowing my wife, who is 10 years younger than I am, is coming to enjoy music she never heard before, thanks to the vinyl rips she's loading up on her iPod.

JIm

Track lengths are definitely one factor and in the case of LP rips they aren't going to match in many cases.

So if I match the length of the same song found in the iTunes Store by adding or deleting from my vinyl ripped song, you are saying that the iTunes Match might work?

It would probably get you more matches although it might not be that many more.

I have a lot of needledrops and actually prefer them over the CD / iTunes versions - so I'm actually on the other end of this discussion believe it or not. I am usually on the Steve Hoffman forums where other persons like myself frequent, btw (we are all vinyl nuts). Ideally I woudn't want any of my vinyl needledrops to Match.

Here's my setup, first of all: AT-LP120-USB turntable, AT120e or AT440MLa cart, Sherwood RX-4109 receiver into a DELL business laptop line-in. The laptop is running Audacity on Ubuntu as the Windows version doesn't support anything higher than 16 bit input (this is a known bug with Audacity for Windows). This isn't the best setup, but for my needs it works well. I usually record in 24/48 (as 96khz is, with my setup, overkill), then save as a 24-bit WAV file. I then run a tool called "ClickRepair" ($40 but definitely worth the money for my needledrops) to reduce the pops and clicks - it works much better than what Audacity gives you. I then take that cleaned up WAV file and import back into Audacity to break up the tracks and Amplify them, usually to -.5db. I usually keep the silence at the end of the track btw, I don't fade in/out and I don't cut it out.

With all this in mind, I just did two match attempts today and here's my results. I have FLAC sources and transcoded them to V2 MP3's first, for the import. I used V2's as I expected a lot of uploads. Will use my AAC transcodes for subsequent attempts:

- Beatles "Abbey Road" (US LP, 1970's pressing). 9 of 17 tracks matched (2,4,6,7,9,10,11,12,13,14,17)

- U2 "The Unforgettable Fire" (2008 LP remaster). 3 of 10 tracks matched (1,2,9)

I would have done more, but Match is very slow right now. 😟 But I'm going to take a big guess here as to the reasons why there are uploads vs matches:

- Masterings. Given the two different formats themselves (a vinyl mastering, sometimes decades old versus whatever digital mastering Apple uses), the waveform analysis they use will likely have a lot of misses. I'm fairly certain they are not using tagging, so I don't think it factors in the equation.

- Pops / clicks. With the given that they are using waveforms, I suspect that pops/clicks will be an issue. And if you use Audacity to remove them, the waveform will still be altered (it's why I use ClickRepair - it just does a better job with keeping the actual audio intact and not introducing distortion).

- Your gear. I know that, in my case, the line-in I'm using on my laptop introduces noise to my setup (I can see it in tools like Spek, when recording silence and then checking the results). I also had a grounding issue for awhile that I eventually corrected, but some of my older needledrops probably still have a very faint buzzing sound which would likely cause problems with my source recording.

- Track length. I've seen this crop up with CD rips and I bet it's a factor here as well. I don't keep track of whether my vinyl track is the same length as a CD copy, but it's likely to be a few seconds off in many cases. And remember that I keep the silence at the end of the track whenever possible.

Here's the ujltimate issue - Apple isn't targeting people like you or I. Their target audience are people who have either CD rips, iTunes purchased tracks, eMusic / Amazon tracks or "other". Needledrops or cassette dubs probably aren't a factor in the equation. To be honest, I was suprised that I had any tracks matched at all - I was expecting a very low % of tracks given all the potential pitfalls with a needledrop. Abbey Road, in particular, was over 50% matched which was a complete shock to me.

When Match wakes up, I'll try some more needledrops to see my results.

More results:

- PJ Harvey "Let England Shake" (2011 LP) - 9 of 12 tracks matched (1,2,4,5,6,7,8,9,11,12)

- Neil Young S/T (1970's LP) - 3 of 10 tracks matched (2,3,9)

- Fleet Foxes "Helplessness Blues" (2011 LP) - 7 of 12 tracks matched (1,2,3,4,5,8,9)

I picked PJ Harvey specifically because it's a new release, but also because I've heard the CD version and the sound differences between the two are hard to distinguish assuming the few pop/clicks are removed. And this is of interest because the match % on that one was very high for me. I still think masterings are a critical component to the matching process.

Also, another factor to keep in mind as far as your TT - pitch control.. On my AT TT, it does a pretty good job of keeping the pitch correct. But some turntables do a terrible job at this and there may not be any way of fixing it. I also have an old vintage Philips turntable and I had to adjust the pitch manually until it was correct.

You might be right on the end silence versus the beginning silence - I did have one instance with a digital track (not vinyl) where it was uploading until I added a few seconds of silence, but that was just a one-off event. But most of my vinyl needledrops probably have more silence than their CD counterparts and I'm still getting matches.

I haven't heavily tested clipping, but I have to think it's a factor. Like the pop/click repair in Audacity, clipping alters the waveform (in this case, it chops off those peaks) and if they are using these waveforms for matching it makes sense that it would be affected if they set their threshold too high. I use -.5db just so it's slightly below peak and then my AAC transcodes usually do not clip (or if they do, it's very light) - when I use 0db and then transcode to AAC or MP3, it usually introduces clipping. I haven't used Audacity's offset (that's in the normalization tool, as I recall) - I prefer to keep the balance between both channels the same as what's coming out of my receiver. It's why I use Amplify vs. Normalize.

4,000 LPs? Yikes! Not even close - I think I have ~500 right now. Not sure if I'd have the room for 4,000. 🙂

EDIT: Reading your other post, now. I actually have Madonna's True Blue on vinyl, and have a needledrop of it. I'll test that next, once Match is fixed (it's very slow, today)

I haven't used that filter, although it sounds like it does a similar change to what the Normalization effect does. Normalization is basically Amplify + DC offet combined.

Make sense though, if your line-in is causing some problems that you can visibly see in Audacity, as well as hear audibly.

JiminMissouri wrote:

then save as a 24-bit WAV file.

Could you give me some detail on how you're getting 24-bit WAV file out of Audacity? I see a custom option that might get me there,

but I'd like to try and duplicate what you're using. I'm in the process of recording an album, one side at 96, the other at 44 to see

if that makes any difference.

I use 32-bit float in, as you do - but at a sample rate 48khz and not 96khz. I don't use 96khz because I know that my line-in is, at best, 24/48. I'm not even convinced that it's doing true 24/48, but since 24/48 isn't much more storage than 16/44, I've kept it at 48. For a 24-bit wave file output, use "Other Uncompressed Files" and then in Options choose WAV and then Signed 24-bit PCM.

96khz is deceiving - you might think your sound card supports 96khz because Audacity records it as such, but more likely than not it's probably mostly dead space. I've used many tools to try and verify my own setup, and Spek (a Windows tool) was enlightening it's basically a better version of Audacity's spectrogram and gives me a good visual queue if I have data informaton beyond 22khz. Sometimes I do, but I'm convinced that it's just noise and not actual analog waveforms.

When I save up enough money, I plan on buying a better DAC for my needledrops - specifically one that is designed to handle 24/96 in.