Frame blending: Retiming/Variable Speed/Tracking ?

Hello.

I do not know a lot about these things. So, here are 3 questions (that could have better been split in 3 posts...)

1) From my personal and sometimes confused observations, I seem to come up with the following rule: frame blending should be used whenever the original fps of a clip multiplied with its playing speed is inferior to the current project's own fps. For instance: a project set at 50fps contains a clip shot at 30fps. Played at 50fps, I understand this clip will be slowed down since 30 frames will be used over the span of 50, and so should be played with frame blending as long as its playing speed is inferior to 50/30=166.67 %. Is this true or completely untrue ?

2) Let us imagine that for a reason or another, the variable speed option is used for the clip so as to skip some isolated flawed frames or group of frames (time relapse clips may require this in my experience). Is the frame blending then calculated between the remaining frames or are the skipped, eliminated ones still included in the interpolation (this I hope not) ?

3) Activating frame blending on a source clip that has some Analyze motion on (used for other objects' Match motions) completely derails the tracking and quite understandably so. Then, should frame blending be activated on the source clip before the tracking analysis is made and then everything is OK, or has the source clip to be rendered anyway at 50fps first, then reimported, while the initial tracking may still be used since its temporal data remain relevant and all the more so because they were obtained on the original clip ?


Thank you very much.

Posted on Mar 29, 2021 6:08 PM

Reply

Similar questions

8 replies

Apr 1, 2021 7:27 PM in response to joelle184

1) There might be a lack of data, but you will not notice it. Frame rate only becomes noticeable somewhere between 18 and 24 fps and very noticeable below 16fps — and that would depend on the frame rate of the original media, not its playback frame rate, provided it is about 20fps or higher. (I don't buy into 24fps is "cinematic" one bit... but on the other hand, I was one of those people who could see subliminal advertising in the 1960s).


2) In "modern" Motion, you're basically hosed. Optical flow does not work. Not like this:

https://www.youtube.com/watch?v=LSG9lTOUKDo

and it hasn't since 5.1.

The only workaround I can think of is to duplicate frames before and after the cut to fill the gap. (Using variable speed is basically the same thing I guess, just a lot more work.) A decent cross-dissolve would probably be better, or a "flash effect" — something short and stylish to cover the problem (just use it enough throughout to make it look like a "thematic" element). [I always look for an easy way out!]

You could try the technique in the video to use the present optical flow to create "tween" frames for you to cover the flaws — it might be acceptable [it will still just be frame blended]... but don't hold your breath.


3) I don't like "automatic" tracking in anything. I always seem to spend MORE time trying to get it to work than if I just go through and track manually. I'm no help here.


4) see the video... and yes — Motion 4 predates the changes in 5.1 (in other words, if you still have it and it runs, give it a shot!)


— you mentioned using Motion Blur. You could try adding Filters > Time > Widetime and keyframe the Amount to motion blur through the flawed frames (it's more "targeted"). Give enough frames in and out for the Amount to build up more naturally. Don't use Mix. It's not the same effect.


Mar 30, 2021 1:19 AM in response to joelle184

I'll start this response with: I hardly ever use Motion to edit video. I probably know less than you do.


1) I expect Motion to conform the playback of whatever frame rate footage you use to the project frame rate. Personally, I wouldn't worry about the details. Frame Blending (project) is on by default. It won't matter if your project frame rate is 10fps or 60fps, the video will play at exactly the same speed (although at 10fps, it will look a little different - not quite as smooth).


2) skipping frames and trying to repair the gap with frame blending is a dicey endeavor. Good luck with that!


3) do whatever repair you need to make, clone it and apply Analyze Motion to the clone. (Turn off the original clip once cloned.) Or you could export the video and bring it back in.


About the Frame Blending modes:


Motion's Optical Flow used to work GREAT! It could be used for morphing (close to it anyway). It was broken (IMO) some time ago and has not been fixed since (literally YEARS). Optical Flow now can distort images (in some cases) and looks like just a regular "blending" between frames — more like a cross fade (I don't care for how it looks in FCPX either...). Motion Blur is better handled on the project level (select Project > Properties and adjust the Samples and Shutter Angle to suit — turn on Motion Blur in the Render menu.) That leaves Blending and None. Blending and Optical Flow are more or less the same except Blending doesn't distort.


Again — take this all with a grain of salt because I don't use Motion for video. Maybe somebody who knows better can enlighten us both!



Mar 31, 2021 3:21 PM in response to fox_m

Hi fox. Thanx for helping.

1) I understand that the video will play at the same apparent speed, but if I play a 30fps in a 50fps project, is there not a lack of data for it to be played in the same conditions as in 50fps? Or do you mean that Motion keeps playing each and every clip at their individual speeds regardless of the project's fps (Then a slowed down clip (which is advised to be played with frame

blending) is slowed down if and only if it is played at a ≤100% speed)?

2) Maybe I should explain better what I usually do, frame blending or not. With Variable speed activated, let's say in elementary math terms that the function is the basic identity: f(x)=x. Variable speed has two natural keyframes programed: f(start)=start and f(end)=end. If I notice that frame y is flawed (as may be the case with time-lapse clips for instance), I just type (with Keyframes): f(y-1)=y-1 f(y)=y+1 and the minute speed discrepancy that ensues is compensated for over the rest of the clip because f(end) remains equal to end. I do this with every flawed frame and may even skip more than one in a leap. It appears seamless when it plays, at least to my eyes and with the footage I've used...

3) Motion seems to refuse any kind of tracking on a clone of an optical-flow-ed clip, or even a group made of it... Don't know why, but I had to export in 50fps first.

4) In Motion 4, I have good results with Optical flow and mediocre ones with Frame blending. Maybe Motion4 predates the changes you're talking about.

Apr 6, 2021 3:54 PM in response to fox_m

My bad: my Motion version is 5.4, not 4. So, yes, Optical flow doesn't give this morphing effect anylonger (I tested it in the reproduced context of this video of yours) and I agree it's a pity. Anyway, Optical flow still provides good results in very much slowed down clips at least to my eyes (see [0'36 - 0'56"] here https://youtu.be/0GrZBO8X8fQ where the clip with the buildings is 29.97fps played at 35% speed in a 50fps project, whereas the simple frame blending option did an AWFUL job).


As to tracking, I've come across some cases in which your methods (including the bezier line) are indeed better than automatic tracking: when the movement curve is clear with no fits and starts, no incessant back and forth, and no detailed stabilizing needed. But, when the footage is unsmooth, with some stabilizing needed (which is generally not completely satisfying with the stabilize behavior), how you can manage tracking other than with automatic tracking is beyond me... (=beyond my current capabilities). I recently had to do some tracking on this video https://www.videvo.net/video/young-boy-playing-piano-in-the-street-2/5804/ and had to use Stabilize first, then Automatic tracking to obtain this wishy-washy result at [1'47 - 2'11] on here again https://youtu.be/0GrZBO8X8fQ where the superimposed standing figure wriggles a bit due to an imperfect tracking (hence, the blended organ keys help a lot...). I tried manual tracking here with no result to speak of since the movement is too random. BTW, the preceding scene [1'21 - 1'46”] is far from perfect in this regard, but in the end manual tracking a la fox-m proved indeed slightly better than automatic tracking and way more simple, so it is manual tracking for this one. As to the opening scene, the time-lapse of the sky (played in variable speed with Kfs) skips some frames (sometimes more than one at a time) and I don't think it shows. I've yet to ascertain that values you enter as Kfs should always be integer numbers (to specify exact frames) since the retiming field reads decimals due to the slowing down of the clip...

Apr 7, 2021 1:30 AM in response to joelle184

I'd say you did a very good job of it. If the viewer wasn't looking for it, they'd probably wouldn't notice the flaws you see. It took me a couple of viewings to see what you were talking about.


I downloaded the Videvo clip and tried tracking it - I ran into the same problem. There really isn't a very good point in that video to track. I tried a corner on the piano - but there was a parallax issue. Then I used the brick floor with the best corner I could find and also ran into the wriggling problem you talked about. Attempting frame by frame keyframing seemed to make it worse. (I need a better onscreen control for targeting a tracking point which I am going to have to invent [when I have some time!]) I need to try some stuff out and then I'll get back with you about it.

Apr 11, 2021 6:54 PM in response to joelle184

OK, so I've tried it. Absent any tracking capabilities in FCPX, I think your plug-in may be of undeniable help for FCPX users at least and I don't even understand why it didn't exist yet. I'm no fan of FCPX but when you think of it, what help do you have there for manual tracking... I'd think of creating a null object (which is what your plug consists in) but I can't remember if FCPX allows for the linking of coordinates (?) as Motion does. Zooming in on the tracked area/target enhances the advantage and helps too.

The help is theoretically the same for Motion users... but they will tend to try automatic tracking first, which may be a mistake (as was my case in this aforementioned scene with the girl in a red dress where manual tracking ended up better). That said, I realize there is a un inevitable limit: the limbo in which some footage leaves you because the flaws are too big to be solved by mere stabilization (maybe several passes though...) and too small to be efficaciously managed and organised by manual tracking. So, no magic bullet.

nb: If the user displaces their keyed footage [from (0,0)] to begin with, it seems they'll have to use the Offset coordinates (as I had to). In reality, Offset is probably inevitable in a keying scenario.


This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Frame blending: Retiming/Variable Speed/Tracking ?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.