Lossless MJPEG exports?

Question

Level 1

8 points

Lossless MJPEG exports?

We've just finished shooting a feature in MJPEG at the resolution of 4096 x 2160 with a Canon 1DC and are about to start cutting with FCPX on a Retina Macbook Pro with 16GB of RAM - so far viewing the rushes and making preliminary cuts has posed no problems and we're particularly pleased there is no lag or slowing down of the footage which simply plays like it was any other smaller format…

I have a question about MJPEG and FCPX and my own cutting methodology. When cutting, I like to copy quite a bit, so to speak. For instance, I will focus on a particular scene, cut it together and render it. Then will take the rendered file and design its sound, lay it all down, then render that. I will do this, individually for each scene in the film. Then I will dump the fifty or so scenes into a fresh timeline and render that also. Subsequently, I will lay in the score and re-render. So, as you can see, there are many generations of copying / rendering / exporting.

In the old days, I would have been going back to the neg or in analog video using a timecode and go back to the masters… but I prefer not to go through a similar process here. Therefore, what I need to know is whether the duplication I have outlined above will cause any degradation to my image whatsoever. To be totally clear, it's all MJPEG files at 4096 x 2160 and every single time I export something I am religiously doing it with all the best, highest, slowest settings and outputting at 4444 and so on - with so much reverence that it's almost as if I am handling a negative.

But is it possible I will lose anything from the image, even the slightest bit of quality, doing what I have described? If so, I will have to find another way. I would really appreciate any advice (or reassurance?) you experienced FCPX folks can throw my way, thank you for it deeply in advance and ask you to please remember this is not a request for help with cutting methodologies and so on (I have that part figured out over the years) but rather a straightforward technical query about whether repeatedly exporting MJPEG will cause the image to slowly, even slightly degrade, thus making my methodology a faulty one!

Is there, for instance, a particular setting or settings that will ensure the MJPEG remains as MJPEG until such time as I want to create various versions of the master later on?

I need to be darn sure that my master is 100% for many obvious reasons, not least of all that we'll be premiering in 4K cinemas.

MacBook Pro with Retina display, OS X Mountain Lion (10.8.3), FCPX

Posted on May 30, 2013 2:43 AM

Reply

Answer 1

Tom Wolsky

Level 10

152,809 points

May 30, 2013 4:29 AM in response to Jemappelle

Only exporting an uncompressed file will be completely transparent.

Reply

Answer 2

Jemappelle Author

Level 1

8 points

May 30, 2013 12:23 PM in response to Tom Wolsky

Tom Wolsky wrote:

Only exporting an uncompressed file will be completely transparent.

I know that's the case when one is using interframe compression, but did not think it applied to intraframe compression, MJPEG falling into the latter category.

Reply

Answer 3

Tom Wolsky

Level 10

152,809 points

May 30, 2013 12:32 PM in response to Jemappelle

Not sure what inter- or intra-frame compression has to do with it. DV is an intra-frame format and is very heavily compressed. M-JPEG is far from being a lossless codec, and can be quite heavily compressed. It's a very old codec, one of the oldest codecs used for high resolution video, and requires a high data rate to achieve good results. H.264, an inter-frame frame, can produce comparable results at lower data rates. Neither is lossless.

Reply

Answer 4

Jemappelle Author

Level 1

8 points

May 30, 2013 2:15 PM in response to Tom Wolsky

Sorry, but I am not trying to determine whether the format we shot in is compressed or not and am indeed well aware that it is compressed - in fact the compression is even referred to in the title of the format MJPEG!

Excuse me if I wasn't clear, but that's just not what my question related to in any way Tom 😁

What I am trying to determine is whether exporting MJPEG in FCPX must cause degradation, as it's not interframe compression that's used in MJPEG, but intraframe compression. Interframe compressed formats immediately degrade when edited because, for obvious reasons relating to the nature of interframe structure, individual frames cannot always be simply duplicated.

Intraframe formats - on the other hand - need not neccesarily degrade in theory when edited, as the individual frames can be simply duplicated, because there is no compression BETWEEN frames, only WITHIN each individual frame.

Am I making sense?

Reply

Answer 5

Tom Wolsky

Level 10

152,809 points

May 30, 2013 2:26 PM in response to Jemappelle

Yes, the image will be degraded if it's exported to M-JPEG, which is a lossy, compressed format. It will degrade further every time the process is repeated. That it's intraframe compression isn't relevant. The pixels are still be recompressed and re-encoded and there is loss. The further you go from the original the more apparent it will become. Again, a high quality interframe encode may show less degradation. Bottom line: you don't want to do this if you want to maintain a pristine image. You certainly don't want to do it with M-JPEG which is quite a bit more lossy than even ProRes 422.

Reply

Answer 6

Jemappelle Author

Level 1

8 points

May 30, 2013 2:51 PM in response to Tom Wolsky

I can't see any degradation with the naked eye and don't see why a NLE of this Calibre wouldn't be able to stitch shots together, duplicating each frame with its integrity fully intact.

If I have a JPEG on my desktop and I copy it, so there are two JPEGs now... you're not seriously suggesting the copy is inferior, that loss has occurred?

Because any NLE worth its salt, if transitions and effects are not used, if you are simply stitching shots together and there is no intraframe compression.... why would it degrade them...

Can you explain it to me more clearly?

Reply

Answer 7

Tom Wolsky

Level 10

152,809 points

May 30, 2013 3:13 PM in response to Jemappelle

It has absolutely nothing whatsoever to do with the application. It's the codec used. It's not a question of duplicating a frame intact, it's a question of the way a codec works. All JPEG codecs recompress every time the file is saved. Period. That's the way JPEG works.

The copy of a JPEG is inferior. If you make a copy of a copy of a copy the file degrades. That's the way the JPEG codec or ProRes or pretty much every codec works. It re-encodes the video. Animation and uncompressed are considered lossless codecs because you can copy them through multiple generations without loss. ProRes and certainly M-JPEG are considered lossy codecs. The pixels are encoded differently and have slightly less precision and clarity every time they are recompressed through another generation.

If you've tested it and it's OK for you, then fine.

I don't think I can be any clearer.

Reply

Answer 8

Jemappelle Author

Level 1

8 points

May 30, 2013 3:29 PM in response to Tom Wolsky

Okay, so you do think that if a Jpeg is sitting on my desktop and I select it and press Cmd C and then Cmd V and a second Jpeg appears, it's your belief this second Jpeg - rather than being a perfect copy of the data that comprised the first Jpeg - has in fact gone through a re-encode process and is inferior quality?

And further, that if I was to copy that copy... and then copy that copy.... after a hundred or thousand copies it would be really poor quality.

Sorry, Tom. That's not what I feel is happening or what I have expeirenced. Rather, I think I am just making an exact duplication of the original jpeg file. Yes, compression was used in its creation and might be used again if some kind of alteration was applied. But a straight copy? A straight copy of the data? No, I don't think any loss occurs when a straight copy of the data is made.

What's more, this very simple example of merely duplicating a Jpeg file (which remember happens when we send someone such a Jpeg by email or on a CD or by any route) that simple act of duplicating the 1s and 0s that make up the file... can be brought into a video context if each frame exists by itself. Indeed, I've done it in the past and it was with MPEG footage, albeit not MJPEG, it was MPEG TS. I exported the files hundreds of times from the NLE and there was no change in size or quality.

That's what I am seeing here too.

I don't need people to agree with me.

But if they disagree I need them to articulate why.

Reply

Answer 9

Tom Wolsky

Level 10

152,809 points

May 30, 2013 3:35 PM in response to Jemappelle

Okay, so you do think that if a Jpeg is sitting on my desktop and I select it and press Cmd C and then Cmd V and a second Jpeg appears, it's your belief this second Jpeg - rather than being a perfect copy of the data that comprised the first Jpeg - has in fact gone through a re-encode process and is inferior quality?

No. What you're describing is a file copy. There is obviously no recompression. You're copying the exact seems digital information into a new file. But if you open a JPEG file and do a save as in Photoshop for instance you create a new file that's recompressed. When you export a video file from any video application you're recompressing it. (There may be specific instances in specific formats where this does not occur, but that is not the case for M-JPEG.)

You can believe whatever you want, and if you're happy with the result and it's OK with you, then I'm not sure why you bothered asking.

That's the facts. JPEG and M-JPEG are not lossless. They recompress. The media degrades through every generation. Believe what you want, and do what you want.

Reply

Answer 10

Meg The Dog

Level 6

13,826 points

May 30, 2013 3:50 PM in response to Jemappelle

Jemappelle wrote:

So, as you can see, there are many generations of copying / rendering / exporting.

File Copy = Lossless digital transfer

Rendering = Recompression of frames

Exporting = Recompression of frames

MtD

Reply

Answer 11

Jemappelle Author

Level 1

8 points

May 30, 2013 4:47 PM in response to Meg The Dog

Meg The Dog wrote:

Jemappelle wrote:

So, as you can see, there are many generations of copying / rendering / exporting.

File Copy = Lossless digital transfer

Rendering = Recompression of frames

Exporting = Recompression of frames

MtD

Thanks for this, very constructive. See, I haven't always found these divisions to be so clear cut. Sometimes, when rendering or exporting, I have not experienced recompression of frames but instead the lossless digital transfer you ascribe only to file copying.

Once this happened when I was working with Mpeg2 TS from an old JVC HD1 - through extensive testing we were able to prove no recompression was occuring, the editing software was doing precisely what happens when you copy a file - unless you applied transition or effects.

Another time it happened when I was working with JPEG Movs from Anime Studio Pro - Vegas would not recompress them, it literally just duplicated the frames, like you were duplicating files individually, only placing them in different orders and positions according to one's edit...

Hence my original query about whether MJPEG can be dealt with this way. If it cannot, that's fine - but so far degradation has not presented itself to the naked eye and we've spat it in and out of FCPX 13 or 14 times.

Moreover, when I have asked people to explain why it should degraded, the answer has only been: because it degrades.

You see that I have not yet disproven my theory, which of course is what I am trying to do, and is the reason I "bothered asking" 😁

Reply

Answer 12

Meg The Dog

Level 6

13,826 points

May 30, 2013 7:19 PM in response to Jemappelle

Well, you are certainly free to work the way you want, but it does not make sense to me.

When image quality is paramount, we go to the Smoke and work in uncompressed.

MtD

Reply

Answer 13

Jemappelle Author

Level 1

8 points

May 31, 2013 2:28 AM in response to Meg The Dog

Elsewhere I've heard "ProRes 4444 is the closest you get to 'uncompressed.' But if you truly need zero compression of any kind, you'll need to get Compressor- $49 at the App Store"

LOL! I wonder would using Compressor allow the MJPEG frames to be merely duplicated and not re-compressed...

Any thoughts on that Meg?

😉

Reply

Answer 14

Russ H

Level 7

22,432 points

May 31, 2013 7:45 AM in response to Jemappelle

Jemappelle wrote:

Elsewhere I've heard "ProRes 4444 is the closest you get to 'uncompressed.'

PR 4444 is just PR HQ with an alpha channel.

Yes, PR is technically a lossy codec. But you should do the kind of testing you've done on other codecs and in other NLE's to see, as a practical matter, how meaningful that degradation is. My guess is you'll decide it's not meaningful.

I've done many generations of compressing PR 422 and my eye cannot tell the difference. (Perhaps it's my eye?) Here is an article that cites claims of 10 generations of compression without apparent loss.

Russ

Reply

Answer 15

Jemappelle Author

Level 1

8 points

May 31, 2013 8:07 AM in response to Russ H

Russ, thanks so much for the link... that's very constructive.

I've been reading other stuff too on the web this afternoon and it does appear the assertion upstream that "only exporting an uncompressed file will be completely transparent" is false.

A lossy, compressed format like MJPEG can be edited without any degradation, provided the compression is intraframe and not interframe. You seem to need a plug in ironically called Compressor to do it in FCPX although maybe there are other ways. And there is certainly other software that does it...

All you're really doing is rearranging the order of the frame, making data copies of 1s and 0s.

Often, it's not what happens when using an NLE, far various reasons, but it can be done if neccessary and for someone like me who rarely dissolves or uses effects but just wants to cut the picture, it a very valid route - which means timecode can be circumvented.

I'm wondering if FCPX is the right tool for me, actually.

Reply