iTunes Match explicit/clean music switch

Not sure if this means you may be in luck, or it may not solve the issue... but the wording looks semi-promising... for the new iTunes 10.6 released yesterday or the day before... Here is the change log:

iTunes 10.6 adds the ability to play 1080p HD movies and TV shows from the iTunes Store.

This release also includes many improvements for iTunes Match, including:

• Improved song matching
• Improved album artwork handling, downloading, and display
• Addresses an issue where songs may skip when playing from iCloud
  
  I know many had an issue about album art messing up or not downloading correctly, so that looks like the second one might help out... #1 is kind of vague and ambiguous, but COULD mean it can tell the difference now? I hope. Can anyonoe report on it after upgrading to iTunes 10.6 (and preferrably iOS 5.1, which finally fixed my battery issues, YES! Apple scores a point, finally!)

I have one Jay-Z album that's affected, and I put in a case last month on. Upgraded to 10.6, did NOT fix the problem. 😟

So I loss all of my Matched Explicit songs for good?

If you have actually deleted your expliicit tracks from your iTunes library and downloaded new "clean" versions from iTunes Mach, then yes you lost them, assuming you don't have some other means of getting new copies from the source.

However, if you haven't deleted the explicit songs from your iTunes library,you should be fine. You would hear clean versions on other devices that are accesing the iTunes match versions though.

If you are not certain, then try playing one from the iTunes library on the machine that had the explicit tracks to begin with.

All, I have acheived 100% accuracy on this. Follow this process and share your results:

This process assumes that you have not deleted the Original Music files on your local harddrive that still contain explicit lyrics. If you have Deleted the Local Versions of your Explicit Audio Files, Delete the Matched and Local Versions of the Audio Files from iCloud and the Local Library following this article: http://support.apple.com/kb/HT4915, click Move to Recycle Bin, this will remove the incorrectly Matched songs from your library folder.

*Note: Allow several hours for the Delete to be completed. Leave your iTunes open during this time.

After this Deletion Process has completed, restore the original Versions of the Audio Files containing the Explicit Audio by importing your original backups of these files into your iTunes Library.

*Note: If you don't have backups, you cannot go back to your original songs that had Explicit Audio.

1. Make sure the following settings are set for your iTunes Library:

2. Convert All MP3s to AAC 256K CBR.

3. Create and Export(M3U) a Static Playlist of Explict Albums, including Explicit Songs.

4. Create and Export(M3U) a Static Playlist of Explicit Songs, only Explicit Songs.

5. Download and Install MP3Tag (Freeware/Donations Accepted). Make sure that Automatic renaming of files is not turned on.

6. Set up a Action in MP3 Tag to add the following Value to your Comments Tag of your Explicit Albums "[Explicit_Album] %COMMENT%". This allows you to track your Explicit Albums with a Smart Playlist.

7. Set up a Action in MP3 Tag to add the following Value to your Comments Tag of your Explicit Songs [Explicit] [Explicit_Song] %COMMENT%. This allows you to track your Explicit Songs with a Smart Playlist.

8. Set up a Action within MP3Tag to Add the Tag ITUNESADVISORY and set it equal to 1. This will make the songs show up in iTunes with the Explicit Icon.*Note: This Metatag feature only works for AAC(M4a) Versions of Audio Files.

9. Set up a Action within MP3Tag to Strip the " [Explicit]" Text out of your Album Names and Song Names. This will assist ITM in matching the songs by name, in addition to other criteria that it uses to Match Songs.

10. Load the Explicit Album Playlist in MP3Tag and Process the MP3s with the Action listed in Step 6.***

11. Load the Explicit Song Playlist in MP3Tag and Process the MP3s with the Actions listed in Step 7, 8 & 9.

***Note for Steps 10 & 11: If any songs are not found due to special characters, you can copy the text that they were not found into a Text Editor, like Textpad, Notepad++ or TextWrangler and add each Directory containing the missing songs manually to make sure you process everything in Batch Mode(Less Cumbersome/Time Consuming)

12. Within iTunes, open your Explicit Songs playlist and right-click the first song and select Get Info

13. Delete all incorrectly matched non-explicit versions of Music from iTunes and iCloud per this link: http://support.apple.com/kb/HT4915 Option+Delete(Windows: Shift+Delete) will allow you to delete the songs from a Smart Playlist that you can use to Track your Explicit Songs, based on the Metatag work from Steps 10 & 11.

I.E. Your Explicit Songs Smart Playlist would have the criteria: Kind->Music, Purchased->False & Comments->contains->"[Explicit]" or '[Explicit_Song]'. Do not click Move to Recycle Bin, this will leave your songs in your library folder.

****Note: Allow several hours for the Delete to be completed. Leave your iTunes open during this time.

14.Turn off iTunes Match.

15. Re-import the Deleted Songs into iTunes.

16. Turn on iTunes Match and after the process is finished, Explicit songs should be Matched or Uploaded 100%.

I have one album affected so this will be easy for me to test, but one question -- not all explicit albums are affected. Did you have an explicit album with a problem, that this procedure corrected?

I'll report back shortly if this works for me. Thanks,

Holy sh**, dude! I have to give you mad respect for fleshing out that whole process for everyone. Everyone appreciates it when someone puts a lot of effort and hard work into helping others with problems. Having read your whole process, I am fairly confident that the key steps here are the AAC file format (which is actually an MPEG-4 file)-- and you are correct, AAC files have different tags because they do not use the IDv2 tag standard that MP3's do.

I was going to post some interesting technical info that goes hand in hand with your info. But the #1 important fact of the whole thing is that AAC files do indeed have an "explicit field". I forget the actual name but it is capable of 3 values. These are "remove" (?), "clean", "explicit". I have a feeling that tripping this tag to "explicit" on all files will result in the same outcome-- the correct version. I believe it is the LACK of this identifying field that causes clean versions to be downloaded, and since mp3's don't have it and albums ripped to AAC may not have it set either, then this is likely the root cause.

I'm curious if you set a field in the ID3 tag to be named the same value (you can make up any tag name you want) and set it to explicit, then could the conversion step be bypassed? Luckily I have ripped my whole CD collection to iTunes plus AAC files, with plenty of explicit CDs. I could simply set the explicit flag on ALL files regardless, and see what happens. That would save hours and hours.

Let me do some quick research and I'll follow up with my findings and the original tech info that supplements the whole theory (I have it typed, I just couldn't finish and saved it to a text file until I could finish it.

You'll hear back from me soon. Thanks again for the effort you put into this.

Didn't work. 😟 Admittedly I did bypass a few steps, but only because my library is clean and I only have one problematic album I'm working with. Some of your steps assume a larger library where you need to setup a batch process.

The key item you have is that you're setting up the [Explicit] tag, and waiting a few hours between deletes / re-adds and also removing / re-adding back iTunes Match. In my case, the album in question was already gone for days, and I did do the re-add as you had written. And all the tracks are now [Explicit] - but, to no avail.

Keep in mind that I did add the tags during an earlier test, as well.

Remember, not all Explicit albums are affected. I have a ton of hip-hop, punk and metal that are Explicit, and so far this Jay-Z album is the only one I've had an issues with Tmk, if you have an affected album there is no way to fix it - I've even called Apple and gone up to a L2 tech, and they have no solution yet either.

A couple of notes:

- The Explicit Metatag on the MP4 Audio files is ITUNESADVISORY and it is proprietary unique to the AAC Format.

- I did try the ITUNESADVISORY Metatag on MP3s, but it did not work.

- AAC is the only way I have been able to restrict my kids from listening to Explicit Music on my wife's iPhone. Kind of makes sense that Apple wants you to use their proprietary format to obtain this "feature".

- I had to Delete my Explicit songs several (2 to 3) times before they reflected the Explicit Metatag on iTunes Match.

- Make the Metatags match the iTunes Store exactly(I see in your screenshot that you still have the " [Explicit]" text at the end of the Song Name). Remove this from both the Song Name and the Album Name. I relocated these Pseudotags to the Comments as mentioned above. I used Screenshots of the iTunes Store or opened the iTunes Store links in a Web Browser in my second monitor to manually retag the songs (All 2000 of them.) This process took a long time, but I believe I got a very succesful Match due to this process.

- I have found that if you wait over 24 hours for the Delete to occur, it is the most successful.

Hopes this helps.

FYI, the AAC format is not "proprietary" to Apple.

http://en.wikipedia.org/wiki/MPEG-4_Audio

My mistake, I was under the impression it was the Apple Audio Codec, as I have heard it called. The Apple Audio Codec wiki redirects to the page you listed. 5 points to Gryffindor.

No worries. It's an easy mistake to make because when Apple introduced the iTunes Store and said they would be using AAC (which really can be called .mp4) wrapped in their FairPlay DRM folks that had an anti-Apple agenda started saying that AAC was proprietary to the company. So, it is a bit of FUD that isn't quite dead yet. 😁

Good info, thank you very much. I was manually trying to find this info out. Much harder, but now I trust that I could code a program to pull the metadata fields from an AAC in an MPEG4 container!

Do you happen to know what type of tag that is? I don't see defined as any of the existing atoms, only that I found the atom where it appears to rest in... I need to look further into it but this is literally at the bit level with a hex editor using atom positions, etc...

First, I got the mapping of the metadata/atoms:

Atom ftyp @ 0 of size: 32, ends @ 32

Atom moov @ 32 of size: 57065, ends @ 57097

Atom mvhd @ 40 of size: 108, ends @ 148

Atom trak @ 148 of size: 53565, ends @ 53713

Atom tkhd @ 156 of size: 92, ends @ 248

Atom mdia @ 248 of size: 53465, ends @ 53713

(..omitted to cut down the size......)

Atom stsz @ 557 of size: 50724, ends @ 51281

Atom stco @ 51281 of size: 2432, ends @ 53713

Atom udta @ 53713 of size: 3384, ends @ 57097

Atom meta @ 53721 of size: 3315, ends @ 57036

Atom hdlr @ 53733 of size: 34, ends @ 53767

Atom ilst @ 53767 of size: 1282, ends @ 55049

Atom ©nam @ 53775 of size: 32, ends @ 53807

So we hit here, this is typical, using the naming convention, we've drilled down to ftyp.moov.udata.meta.ilst.©nam.data (.data is implied, and prepends all meta fields after the 4 character atom name)

I find at the end: Atom Xtra @ 57036 of size: 61, ends @ 57097

Movie duration: 294.336 seconds (04:54.34) - 282.72* kbp/sec bitrate (*=approximate)

Low-level details. Total tracks: 1

Trk Type Handler Kind Lang Bytes

1 soun [none listed] mp4a und 10401922

256.00 kbp/s MPEG-4 AAC Low Complexity/LC Profile channels: [2]

Ok, so there's our AAC profile info. Now here's looking at that Xtra atom in a hex editor:

I hope that's big enough for all to see, it shrunk it down. I highlighted/marked the position @ 57036 for 61 bytes, which ends at 53807. As you'll see, there's this data "ParentalRating" and followed a few bytes later by "Explicit". If you say the tag actually has a defined name (I believe you) -- then *** is this here, and why is it in a generic Xtra tag at the very end?

Neevermind. Figured it out. I had a hunch. It's freakin' Windows that put it there. God knows what kind of tag it is... ID3? Windows isn't gonna handle it properly. Gah. Red herring!

Back on the trail. Here's some documents those who are technically inclined may find interesting:

Here is the full PDF file defining Apple's format of MPEG-4 media (audio, movies) -- pretty awesome, detailed info. Definitely keeping this, it's for programmers (which I happen to be, luckily hah) -- https://developer.apple.com/library/mac/documentation/QuickTime/qtff/qtff.pdf

Awesome wiki on file formats... this one goes directly to what we're looking for, it has the 4CC's for video and audio, defines all of the top-level atoms: http://wiki.multimedia.cx/index.php?title=QuickTime_container

(root is implied, and is of course, "ftyp"; the indentation didn't carry over, but view the link for it)

moov

mvhd

trak

tkhd

edts

elst

mdia

...

stss

stsc

stsz

trak

trak

...

mdat

[data]

A 'meta' atom contains atoms containing human-readable textual data with meta information regarding the file. These atoms are marked with 4 bytes of course but the first byte is a value of 0xA9 (you saw this earlier in my example, it shows as the copyright symbol, so now you understand this normally cryptic line -- "Atom ©nam". The remaining 3 characters can be:

• nam: Name of song or video <--- So clearly, at this position, in this atom name, the song title would be stored, with 0xA9 before it. Same with these:
• cmt: General comment (atom name is ©cmt)
• alb: Album name (atom name would be ©alb)

Here's a real life output of a scanned file:

Atom "©nam" contains: Song Title

Atom "©ART" contains: Artist Name (why this is CAPS, I have no clue)

Atom "©alb" contains: Album Title

Atom "trkn" contains: 6 of 12 (now many of them stop having the 0xA9 char, and are all 4 letters-- I'm unsure but I THINK the "©" ones may be required?)

Atom "disk" contains: 1 of 1

Atom "©day" contains: 2000 (actually time/date... most of the time it's just the year. Apple puts a full timestamp in theirs ending with a Z)

Atom "cpil" contains: false (is compilation?)

Atom "pgap" contains: false (pregap/gapless?)

Atom "tmpo" contains: 0 (tempo/BPM)

Atom "©too" contains: iTunes 10.2.2.12, QuickTime 7.6.9 (encoding tool used)

The list of possible ones go on and on, I cut it off right there. Check the links above for all the dirty info if you want it.

Then there's special, super cryptic atoms, called user atoms... defined with names that are long hex UUID strings and are hidden, with the intention to hide it from the end user... may contain DRM stuff, or other relevant info such as info tying your account email/id to the file... which of course it is...

These look like this:

Atom "----" [com.apple.iTunes;iTunMOVI]

I'm going to stop right there, as most people are probably already long lost or unconcerned. This goes into something called RDNS (reverse dns -> notice the domain name is backwards? It would normally be typed as "iTunes.apple.com" -- that's the way normal DNS works on the internet... but enough. Notice only that the atom name is blanks "----" there can be many of these, all named this. If anyone is interested further or actually is intrigued by this and understands it, I'm up for discussion elsewhere...

Sorry for the length of this post. Now to find a file with the elusive, REAL "ITUNESADVISORY" atom. Thanks for the valuable info kstr79, without it I might still be barking up the wrong tree. It helped out! Any questions, I'll be glad to answer, but I think I scared most people off, LOL.

Michael Allbritton wrote:

No worries. It's an easy mistake to make because when Apple introduced the iTunes Store and said they would be using AAC (which really can be called .mp4) wrapped in their FairPlay DRM folks that had an anti-Apple agenda started saying that AAC was proprietary to the company. So, it is a bit of FUD that isn't quite dead yet. 😁

You're sort of misleading here. Yes, AAC is not an Apple proprietary format... neither is the MPEG-4 container... AAC is of course, Advanced Audio Coding... but when Apple starts making up proprietary things... that are clearly and markedly NON-STANDARD, then I have to say... it does start to become proprietary in some ways.

Real examples? The proper extention is .MP4 -- Apple made up its own file extentions to be more specific about the content:

M4A: Audio

M4V: Video

M4R: Ringtone

M4B: Audiobook?

M4P: DRM-protected AAC file

And so on... these are all not part of the standard. Apple also made up their own non-standard atoms/boxes... examples? First, a quote --

"The Apple Quicktime file format is an extremely well-defined file format. A little too well-defined, in fact. Some would even call it "over-engineered". The official Quicktime documentation is a magnificently detailed beast that gives equal time to explaining all parts of the spec, no matter how important or ignored a particular component may be in the actual implementation. The official spec can be a lot to digest at once and this document is intended to help interested programmers come up to speed on the Quicktime internals much more quickly.

This document emphasizes the components of the Quicktime file format that a programmer would need to know in order to write a general purpose Quicktime file decoder. This document also contains a discussion of decoding strategies.

Note that this document will probably never be complete since there is so much flexibility in the Quicktime format. But it is designed to cover the majority of QT files ever produced."

That sounds like proprietary if I've ever heard it. Further:

"Apple's Quicktime designers were thinking differently when they came up with the notion of an "atom" as "something that can contain other atoms". Atoms are chunks of data in that comprise a Quicktime file. Sometimes they contain data and sometimes they contain other atoms. An atom consists of a size, a type, and a data payload."

So, I agree with those "crazed anti-DRM zealots" more than I do with you, unfortunately. The audio encoding algorithm may be AAC, but everything surrounding that in the MPEG-4 container they have made into their own... If it were so simple they wouldn't need a nearly 8MB PDF file, over 400 pages long describing their file format. It's Apple proprietary. We also wouldn't have user defined tags tied to UUID's, defined as so:

Atom "----" [com.apple.iTunes;iTunSMPB] contains: 00000000 00000840 00000288 0000000000A34D38 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

Atom "----" [com.apple.iTunes;iTunNORM] contains: 00000E21 00001520 0000363F 0000410B 00003A0D 00003A0D 00007E88 00007E89 00003A0D 000045A9

iTunes SMPB and iTunes NORM, their version of ReplayGain or normalization... I believe they call it "Sound Check"... they made the name up, I don't know for sure.Which look what I just found -- Atom "rtng" contains: Inoffensive -- I thought rating would be the "star rating"... but it wouldn't be tagged "Inoffensive".

Who knows what all of their scary "--ID" atoms hold. I know "apID" contains your iTunes email/account name that it was purchased under... then we've got atID, cmID, plID, geID, sfID, etc... each populated with a string of numbers... who knows what, probably the serial of your iTunes install or hardware... other things tying your personal info to the file. Which is fine by me, I don't distribute what I get... even though I know how to strip the atom structure away and the UUID's and all that, hex edit the email out again from non-referenced places... (they want to be sure to tie you to that file! it's not "copy protection", it's just pretty much the opposite, "identity exposition"). So much for privacy, I'd rather just have the copy protection, I think, than have all kinds of personal info in every file I buy from them.

It's almost like -- Apple doesn't trust their consumer base... and I don't trust Apple, so... we're even 😀 But at least I educate myself on the sneaky stuff they do without really telling anyone... which all companies do, Apple just does it a little TOO much. They have TOO MUCH control over the market, their product-- both hardware and software... even their vendors and the consumers, they dictate every last nitpick detail. So I get in there and I get inside, see what they're doing that they don't want everyone to see...

That's why I'm the-worm-in-the-apple. Yeah, it's kind of clever, cause I made it up... but I'm also a slime covered, grotesque, squirming low-level life form... which really doesn't say much good for the general public!

So speak the truth, you cover up for the big Apple too much and defend everything they do. I'd say fanboy, but you could call me a troll and besides, I'm a fanboy, in truth, despite my distaste for the manufacturer. It's like that really hot stripper at the club... man, you'd love for her to be your wife but... dude -- she's a stripper at the club. Tainted and filthy, without morals... someone you don't want to be in a relationship with! The product is hot, it looks great from the superficial view... but when you get to the inner, more important aspects... she's likely akin to Apple -- rotten!

I kid. I have a love/hate relationship with them, and I'm constantly back and forth. I'm closer to my iPhone than I am to my own family. I spend more time messing with Apple products than I do with most things in life. Maybe that's why I hate them... because I love their stuff TOO MUCH. Deep psychology here... no need to go there. Just take it at face value, haha... I argued a counter point against your non-chalant comment, and I'm of the opinion that I spanked what you just claimed...? Friendly competition... I guess it's the guy thing... maybe I need to go boxing or do some XTREMEsports, LOL. Only half of what I say is serious, don't take it too rough! 😀

Whatever. Nothing in your insulting post changes the fact that AAC is not an audio format that is owned by Apple. It is an international standard. I won't be responding to you any more.

kstr79 wrote:

My mistake, I was under the impression it was the Apple Audio Codec, as I have heard it called. The Apple Audio Codec wiki redirects to the page you listed. 5 points to Gryffindor.

What about my points for arguing you back into "not so wrong" -- except AAC= Advanced Audio Coding... but if what you said about proprietary was untrue... what other company uses a metadata field named ITUNES anything?! If they did, they'd be sued!

You would be correct in saying that "Apple's implementation of the MPEG-4 container, using embedded AAC to encode the audio is a proprietary format."

Directly from the document entitled "Quicktime File Format Specification":

Important! The QuickTime File Format has been used as the basis of the MPEG-4 standard and the JPEG-2000

standard, developed by the International Organization for Standardization (ISO). Although these file types

have similar structures and contain many functionally identical elements, they are distinct file types.

Warning! Do not use this specification to interpret a file that conforms to a different specification,

however similar.

Apple's very own words! Say it loud and proud-- PRO-PRI-E-TARY! They literally formally explained that it's based off of the MPEG-4 and JPEG-2000 standards! Actually they worded it wrong, the QTFF was not been used as the basis of the MPEG-4 standard-- they have it reversed... the ISO defines the MPEG-4 specs. That's what makes them the INTERNATIONAL STANDARDS ORGANIZATION! They DEFINE the standards, they don't let APPLE do it then bum it off of them... yeah right. ISO is international, and it's a joint cooperation between many major entities. Apple wishes the ISO used their work as the basis for such a wide-scoped standard!

Their version/implementation USES the standard: ISO/IEC 14496-12(+some addendums/erratums) as their base reference. From the ISO document of that name, they say:

This technically identical text is published as ISO/IEC 14496-12 for MPEG-4, and as ISO/IEC 15444-12 for

JPEG 2000, and reference to this specification should be made accordingly. The recommendation is to

reference one, for example ISO/IEC 14496-12, and append to the reference a parenthetical comment

identifying the other, for example “(technically identical to ISO/IEC 15444-12)”.

How many points do you get for 2 consecutive spankings in a row?! I gotta rub it in, the case is too clear cut to deny! SPANKED! 😀 😀