Export process _adds_ stuttering to movies with frame-rate change

Question

Level 2

171 points

Export process _adds_ stuttering to movies with frame-rate change

SYNOPSIS: When I playback a certain movie project, called VFT, in iMovie 9.0.4, the motion is smooth. If I export it, and then view the export, the motion is jerky. I haven't found a way to view it where the motion is not jerky except to view it as a "project" in iMovie itself. There is a uniform stuttering exhibited in the exports. It looks like the film stops briefly several times a second. (For a very clear alternate example of this, get the Ren and Stimpy episodes "Black Hole" and "Space Madness". The title sequence background (with the moons, planets, etc. moving from right to left) is exactly the same, but appears to run smoothly in "Black Hole" and jerkily (5 jerks per second) in "Space Madness". Can anyone here explain this?) Note: The film VFT is characterized by a lot of moving scenery.

EXAMPLES OF THE STUTTERING: I picked the two parts of the movie (called "Visuals for Timesteps" (VFT)) that best exhibit the stuttering and put them in their own project. I exported the project from iMovie using "None" for compression and "Best" for quality. I then processed them through MPEG Streamclip according to smugmug.com's recommendations and posted it. You can see it at

http://betaneptune.smugmug.com/Movies/Stuttering-examples/26311872_Sh5Kz7#!i=219 2123682&k=mr9Vckn

THE FULL STORY: I made some Super 8 movies at 18 fps in the 70s and 80s. I had them transferred to digital. I received files from the transfer company in the form of QuickTime movie files. The format: Apple Motion JPEG A, 1440 x 1080, Millions. This was a high-definition scan. No interlacing. No blended frames. Each and every frame in the file is a digital copy of each and every frame in the film.

I imported these movie files into iMovie. Ideally one would edit them at 18 fps, but that's not an option with iMovie. So I created projects using 24 fps. To preserve the speed, iMovie doubles every third frame to "stretch" each 18 frames to 24. Why did I choose 24 fps? Easier editing, smaller files, and the hope of making 24 fps progressive DVDs or even BluRays one day. Furthermore, it was recommended by two independent sources, and it taxes the computer less. I was concerned that the 3:4 pulldown would give jerky motion, but playback in iMovie looked perfectly smooth. This surprised me, but it seemed fine, so I went with it.

One of the movies is called "Visuals for Timesteps" (VFT). This is a highly unusual film in that there is a lot of motion of the entire scene a lot of the time. While this looks fine when playback is done from the project browser, there is a uniform stuttering when playing exports. It looks like the movie stops several times a second, just what you might expect from the 3:4 "pulldown", so I assume it is. (My other films mostly look okay.) And it doesn't matter what app or hardware I use to view the export: QuickTime 7, QuickTime 10, MPEG Streamclip, iPod, iPad -- it matters not. (I'm talking about _local_ playback. The Net is not involved, though I get the same effect there, too, in Firefox and Safari.)

I tried different forms of compression: Cmd-E and "Export using Quicktime" with default settings, no compression, and AIC -- all to no avail, which didn't surprise me. I even imported the AIC back into iMovie just to see if there would still be stuttering. There was.

The original 18 fps files from the transfer company play back quite smoothly, with barely a hint of jitter in certain scenes, in QuickTime 7 and 10. The import in the project browser is just slightly more jittery, and again noticeable only upon careful scrutiny of certain shots. Also, the jittering doesn't make the film look like it stops several times a second. The motion appears quite uniform in time. This is far more acceptable (and I'm fine with it) than the stuttering in the exports.

What is going on? What "magic" is iMovie doing when viewing from the project browser that can't be done in exports?

And how can I fix this? I can redo the entire project at 30 fps, which would lessen the stuttering, I would think. But this would be a tremendous amount of work! There are 45 clips, each painstakingly timed, trimmed, and speed-adjusted to match the music. Many required single-frame precision! Many had to be individually cropped to crop out conspicuous defects in the form of dust, splotches, splice gaps, and splice sprocket holes. Also, a few clips needed color correction.

Is there a way to eliminate or at least lessen the stuttering other than a redo at 30 fps? Or an easier way to convert to 30 fps? Or could I create a small virtual disk, add the raw footage and project files, modify file names and folders if needed, and post it for, or send it to, those individuals who have iMovie? Any helpful comments and suggestions are welcome.

Running iMovie 9.0.4 on OS X 10.6.8.

Thanks!

AEF

iMovie '11, Mac OS X (10.6.8)

Posted on Nov 3, 2012 1:38 PM

Reply

Answer 1

Nov 3, 2012 4:41 PM in response to betaneptune

The stuttering is not that noticable to me, but that is probably because I haven't seen the original, so I don't know what I'm missing.

When you say you are exporting uncompressed, does that mean you are getting Motion JPEG as output? Or do you mean uncompressed 4:2:2 as output ?(very large files). At first I thought this could be an issue, but as I re-read your post, it appears that you have tried all the h.264 presets as well.

My first thought would be to redo them in 30 FPS.

Unfortunately, you can't change the frame rate without starting a whole new project. I don't think it would help you to switch the 24fps output to 30fps in another app. By the way, I don't think you can change the frame rate in the middle of the project in Final Cut Pro either.

I did a project like this with 8MM and I used 30, and it was not quite as smooth as the 18fps, but still not bad.

I just checked in Final Cut Pro X. FCPX gives you more control over how the frame rate is managed you have 4 options. As best I can tell, iMovie uses the first option, because iMovie does not do intermediate rendering.

Floor: The default setting. Final Cut Pro truncates down to the nearest integer during its calculation to match the clip’s frame rate to the project’s frame rate.
Nearest Neighbor: Final Cut Pro rounds to the nearest integer during its calculation to match the clip’s frame rate to the project’s frame rate. The Nearest Neighbor option reduces artifacts at the expense of visual stuttering. Rendering is required.
Frame Blending: Creates in-between frames by blending individual pixels of neighboring frames. Slow-motion clips created with Frame Blending appear to play back more smoothly than those created with the Floor or Nearest Neighbor setting. This setting provides better reduction of visual stuttering, but you may see some visual artifacts. Rendering is required.
Optical Flow: A type of frame blending that uses an optical flow algorithm to create new in-between frames. Final Cut Pro analyzes the clip to determine the directional movement of pixels, and then draws portions of the new frames based on the optical flow analysis. Choosing the Optical Flow option results in better reduction of visual stuttering, and Final Cut Pro spends a significant amount of time to fix visual artifacts.

for more see here.... http://help.apple.com/finalcutpro/mac/10.0.5/#ver3363b44e

I am doing a test on some of my own 8MM in FCP for the various options. Will post the results here.

Reply

Answer 2

Nov 3, 2012 6:44 PM in response to betaneptune

OK here are the results of my tests.

My initial reaction is that for Floor and Nearest Neighbor, 30fps is better, but for Frame Blending and Optical Flow, 24 fps is fine. I also have an opinion on which method produces the best results, but I would rather see what you think.

Here is 18fps motion jpeg conformed to a 24fps timeline. Rendered in ProRes 422 and converted to h.264 in Compressor. In 1080P

Here is 18fps motion jpeg conformed to a 30fps timeline. Rendered in ProRes 422 and converted to h.264 in Compressor. In 1080P

Reply

Answer 3

Nov 3, 2012 7:07 PM in response to AppleMan1958

I found a patent application that describes the 4 options in some detail.

http://www.google.com/patents/US20120210232

By the way, sorry for wearing you out on FCP. I know you probably don't want to buy it just for this.

Reply

Answer 4

betaneptune Author

Level 2

171 points

Nov 4, 2012 5:16 AM in response to AppleMan1958

Hi Appleman!

I wish I could show you the original internal working copy in iMovie, but I'd have to mail you a portable hard drive or invite you over! (Actually, the former would partly solve my problem. Since my original post, I stumbled upon how to do that: Copy the project by dragging it to an external drive icon in the project browser, and mail that to viewers who have iMovie. But would iMovie automatically look for it on an external drive?)

Yes, the stuttering is a little bit subtle, but it's there. Just to be absolutely sure, I "blind"-tested myself. I selected a 2s17f part of the internal working copy. (This chunk of footage ends with a dark evergreen tree immediately followed by an overpass. It's at 00:28 in the video on smugmug.) I copied that to a new project. I imported an AIC export of the movie and selected the same chunk of footage. I copied that to the same project. Then I selected both clips, and, using Cmd-C and Cmd-V, duplicated them 41 times, resulting in 84 clips that alternate between the original internal clip and the exported-and-re-imported AIC clip. Continuing, I covered the project browser with a record album (12" diameter), moved the pointer around randomly behind it, and pressed the spacebar. I repeated this a number of times, and correctly identified the stuttering clips every time without fail, quickly and without difficulty.

To better see the stuttering, focus on the trees and columns in the first clip, or on the dark evergreen tree just before the overpass at the end of the second clip.

By "uncompressed" I mean I did the following: Share > Export using QuickTime... > Options > Settings > Set Compression Type to None. I also took the liberty of setting Quality to Best. It was a short clip and also (I would think) is what makes corresponding single frames look the same. They do look the same. Finally, I uploaded the video to smugmug.com according to their directions at http://help.smugmug.com/customer/portal/articles/84569

QuickTime characterizes my local copy of the smugmug version as

User uploaded file

I haven't tried all the H.264 "presets". It would be time-consuming and I'd hope to get some guidance before undertaking such an endeavor. I thought that maybe the use of key frames has something to do with this, but since uncompressed exports exhibit the same stuttering, I'm not very optimistic about it! I did try AIC compression, thinking that that might make it the same as internal copy, but it didn't work. I thought re-importing the AIC copy might show no stuttering. It seemed like the equivalent of an undo operation, but the stuttering remained.

It appears that iMovie is doing something special when showing the movie that isn't done by anything else running an export. But what is it? Perhaps there's a setting in one of the apps that would play the export the way iMovie does. I looked, but couldn't find any such setting.

I studied the FCPX frame-rate options you mention.

Floor: Yes, this must be what iMovie is doing.

Nearest Neighbor: I don't think this would make any difference for 18 -> 24fps. I drew three frames next to four frames. One of the frames is half a frame off, and either way you round it you still get a doubling of every third frame.

Frame Blending: How would you edit this? I need the ability edit with single-frame precision. Could you edit a project and _then_ convert it to frame blending? Oh, perhaps by editing a "floor" clip and copying it to another project and then apply Frame Blending?

Optical Flow: Sounds like frame blending, only better, but more time-consuming. I'm guessing you'd use Frame Blending instead only if you were pressed for time. Is that right?

The descriptions mention compromising between artifacts and stuttering. What are these artifacts? I assume that things like squares at boundaries like you see in JPEG renditions and banding are also artifacts, but what do the artifacts mentioned in the FCPX help page look like?

Wow, I didn't know that Apple software posts its help pages online! How do you find such things?

Thanks for your help!

AEF

Reply

Answer 5

Nov 4, 2012 6:26 AM in response to betaneptune

I think you are correct that Floor and Nearest Neighbor produce near identical results for 18 to 24, and for 18 to 30.

Looking at my footage this morning, I can see why the 18 to 24 appears more stuttering. If you number the frames 1 to 18, it maps them to the 24 frames as 1 2 3 3 4 5 6 6 7 8 9 9 10 11 12 12 13 14 15 15 16 17 18 18. If you clap that out, it sounds like an engine starting up in first gear.

If you map the 18 frames to a 30 frame timeline, you get something like this.

1 1 2 2 3 4 4 5 5 6 7 7 8 8 9 10 10 11 11 12 13 13 14 14 15 16 16 17 17 18. If you clap that out, it sounds like the car is now in second gear.

In both frame blending and optical flow, you don't have the stuttering. It creates extra frames interpolated between the existing frames to create continuous motion.

As for the artifacts, if I look at the frame blending option slowly, the created frames look like they contain shadows. For example, when the baby (moi!) is waving his arms, one of the frames will have an arm that looks like a light shadow or motion artifact. But in optical flow, all pixels are clear. The artifacts are not noticable when you play it at speed for the same reason that you do not notice interlaced footage when played at speed on your TV. Your brain puts it together.

So optical flow looks better, but at a cost of extra analytical time and rendering time. If you have time, it gives the best result, but for most applications, I would think that frame blending would be sufficient.

One application where optical flow would give superior results is if you retime a clip to produce slow motion, say 25% of normal. In Floor, it would just repeat frames. In Frame Blending, it would create new frames, but you would see the shadowy artifacts for things in motion. In Optical flow, each frame would look like a photograph.

One additional option I did not mention yesterday is to use Compressor (about $50). You would have to pre-process the clips in compressor and use frame blending or optical flow. I find the Compressor interface a lot harder to use than the FCP interface, but looking at the manual, it does appear to have this capability. I can't help you much beyond that with Compressor. You would basically process each 18fps clip to the highest level of rate conform that you want, and create AIC files that you could use in iMovie.

Reply

Answer 6

Nov 4, 2012 10:25 AM in response to AppleMan1958

Here is an example of an interpolated frame in Frame Blending. You can see the artifacts, but they are not very noticable at full speed.

Here is the same frame in Optical Flow. There are no motion artifacts.

Reply

Answer 7

betaneptune Author

Level 2

171 points

Nov 4, 2012 11:08 AM in response to AppleMan1958

In reply to your first post that has movies:

Something's wrong here. I don't see the post of mine you responded to at Nov 4, 2012, 9:26 AM! I was writing my post and I was trying to use Cmd-Left_arrow to go to beg of line and it didn't work (Cmd-Right_arrow does go to end of line). I pressed it again and lost my editing session! To make a long story short, I screwed up my editing session and didn't think anything posted from it.

OK, back to the topic at hand!

I like Frame Blending better. Optical stutters too much in my view (pun not intended!).

I was focusing on the boundaries between the roofs and the sky, mostly the last one. This is because that is what it is easiest to detect stutter with and the houses moving by is a lot like my VFT movie. Now that you mention the waving, I reviewed the videos in their entirety. I see the same thing. There is more stutter in the Optical, but the picture is perhaps just a little bit clearer, perhaps in part due to reduced jitter.

[I think jitter can distinguished from stuttering this in this manner: Imagine something that appears to be vibrating and also appears to be moving in discrete pulses. Jitter is the former, while stutter is the latter.]

Note that it is difficult for me to play the videos slowly, being they're on what looks to me to be embedded YouTube.

OK, now my review!

24 fps:

Floor: stutters

Nearest: stutters to the same degree that Floor does

Blending: less stutter than Optical, but more "jitter"

Optical: more stutter than Blending, but less "jitter"

30 fps:

The same as 24, relative among the different types.

24 vs. 30 fps:

30 looks better than 24 most of the time. Sometimes they look about the same.

[I would have to see the videos played locally to make more precise judgements.]

So for panning motion I find Blending better. For non-panning, where there is little motion, Optical may be better. My movie is mostly of the former, so Blending would be better for me. Furthermore, I think stuttering bothers me much more than other defects.

If you freeze-frame the houses moving buy and look at the roofs, you see the roof/sky

It looks like the best option for me is to get FCPX, redo the entire movie at 30fps, then convert it to Frame Blending if that's possible. Unfortunately, my iMac's video card isn't supported by FCPX, which means I would have to buy a new Mac!

Thanks for going through the trouble to post examples of frame-rate conversion! I greatly appreciate it. Frame-rate changes are clearly one of the more challenging problems of filmmaking.

If you ever find out what "magic" iMovie performs to eliminate the stutter, please do tell. Could it perhaps be doing "frame blending" in real time?

AEF

Reply

Answer 8

betaneptune Author

Level 2

171 points

Nov 4, 2012 11:10 AM in response to AppleMan1958

Yes, I noticed that about Frame Blending. I can see that with roof/background boundaries do that when I freeze frame it on the right frames. But I still find Optical too jerky (stuttering).

I will try to compare this to what I see in iMovie. I have to find a scene that's similar enough.

I'll report back.

Thanks again for your help!

AEF

Reply

Answer 9

betaneptune Author

Level 2

171 points

Nov 4, 2012 11:17 AM in response to AppleMan1958

I took a quick look at the patent but don't understand it.

No problem "wearing me out on FCPX". The problem is that it doesn't support my iMac's graphics card, which means I'd have to buy a new one! (It's a ATI Radeon HD 2600 Pro card.) Bummer. But I've been considering getting a new Mac anyway.

AEF

Reply

Answer 10

betaneptune Author

Level 2

171 points

Nov 6, 2012 5:55 PM in response to betaneptune

Appleman, or anyone else, for that matter -- Can you explain the Ren and Stimpy puzzler in my original post? Why does the same background move perfectly smoothly in "Black Hole" but jerkily in "Space Madness"? You can also see the jerky effect (stuttering) in the opening title sequence of Star Trek TOS seasons 2 and 3. You see it when there is a close-up of the Enterprise moving from left to right in front of a planet. Would anyone care to take a stab at this?

Thanks!

Reply

Answer 11

betaneptune Author

Level 2

171 points

Nov 11, 2012 1:04 PM in response to betaneptune

OK, I finally have a reasonable theory on this. iMovie knows, at least it ceratinly can know, which frames it doubled to convert 18 fps to 24 fps. So during playback it can simply run at 18 fps, but skip the duplicated frames! So instead of 1 2 3 3 4 5 6 6 . . . at 24 fps you would get 1 2 3 4 5 6 . . . at 18 fps, and it would look great! I think this must be what it's doing. But why would the programmer bother? Perhaps it was for some reason easier this way. Anyway, if so, this leaves me no choice but to redo the entire project -- all 45 clips -- at 30 fps, which will lessen the effect. Without doing that it seems to me that I'd have to copy the project and its footage to external disks and ship them to my audience. Oh well.

That still leaves the Ren & Stimpy question.

AEF

Reply

Answer 12

betaneptune Author

Level 2

171 points

Nov 12, 2012 5:11 PM in response to betaneptune

Thought I put this QuickTime Info bubble in to answer your question. Maybe it didn't take. I'm trying again.

User uploaded file

AEF

Reply

Answer 13

Nov 12, 2012 7:40 PM in response to betaneptune

So you tried to export as uncompressed and it converted to h.264 anyway? I would think it would either export as Motion Jpeg or as uncompressed 4:2:2 (very large files). But it went to h.264 in standard definition size.

Reply

Answer 14

betaneptune Author

Level 2

171 points

Nov 12, 2012 7:49 PM in response to AppleMan1958

AppleMan1958 wrote:

So you tried to export as uncompressed and it converted to h.264 anyway? I would think it would either export as Motion Jpeg or as uncompressed 4:2:2 (very large files). But it went to h.264 in standard definition size.

No. I screwed something up. I must have given it the wrong filename. Let me get back to you. Stay tuned.

Reply

Answer 15

betaneptune Author

Level 2

171 points

Nov 12, 2012 8:38 PM in response to betaneptune

Appleman,

OK, here's what happened. Smugmug says to take your movie and run it through MPEG Streamclip and apply their settings. Well, if I'm going to compress something for their settings, shouldn't I start with an iMovie export that has no compression?

So I first exported it from iMovie with no compression, and labelled the file as such (first file listed in the ls command output below). Then I opened the file in MPEG Streamclip, set the controls according to smugmug's instructions, saved it (and, in the process, added "smugmug" to the file name so as to distinguish the two), and uploaded it to their site. So the "no compression" really only means the iMovie export. Adding "smugmug" means the same thing but processed for smugmug. Yeah, misleading.

You can find the smugmug.com instructions here: http://help.smugmug.com/customer/portal/articles/84569

I list the files below with the unix 'ls' command specifying creation dates. The first file is the export from iMovie without compression. Then I processed this file with MPEG Streamclip to produce the second file for smugmug. So I just added smugmug to the file name. You can see below that the smugmug version was made after the iMovie export and is considerably smaller than the uncompressed version. (The Unix command below sorted the files by creation date and displayed only the last 5 files.)

new-host:iMovie EXPORTS alanfeldman$ ls -ltrUog | tail -5

-rw-r--r--@ 1 1567954837 Nov 3 07:46 VFT D-3 16 stuttering examples no compression.mov

-rw-r--r--@ 1 8159828 Nov 3 08:00 VFT D-3 16 stuttering examples no compression smugmug.mov

-rw-r--r-- 1 38125 Nov 4 07:22 VFT D-3 16 stuttering examples no compression.jpeg

-rw-r--r--@ 1 157335389 Nov 4 15:20 VTF raw 24fps.m4v

-rw-r--r--@ 1 157494727 Nov 11 23:59 VFT raw 30fps.m4v

new-host:iMovie EXPORTS alanfeldman$

Yeah, it's misleading, but it's logical if you know what was done! Sorry for the confusion! The QuickTime info was for the smugmug version. The QuickTime 7 Info for the "no-compression" version is

That's a lot for a 53-second video! I'll have to ask smugmug if this is the right way to go about things, as it entails making a huge intermediate file.

AEF

Reply