How to automatically delete duplicate frames in 30fps or 25fps files?

Question

Level 1

14 points

How to automatically delete duplicate frames in 30fps or 25fps files?

Hello,

I totally hate 24fps to 30fps (or 25fps to 30fps, or 24fps to 25fps) conversion which duplicates progressive frames. In 24 => 25 , a duplicate frame is added at the end of each second. In 24 => 30, one every 4th frame. In 25 => one every 5th frame. I'd like to restore the original fps in those files. Does anyone know of any app / utility that deletes these duplicate frames, as defined, without reeconding the file? I will explain what I do and what I know, which is not what I want 😉 :

In FCP, I open a, say, 30fps MOV, MP4 or even MPEG or TS file (in these two latter cases, just change their extension to .mod ; FCP may be iffy sometimes, but voilà! ).
I create a 24fps project.
I drag and drop a segment of the 30fps file which was converted from 24fps starting at the frame-one sequence, that is: 1-2-3-4-4 5-6-7-8-8 etc.
The 5th frame of each sequence is automatically deleted.
I then left-drag and right-drag an open the entire file in the timeline.
Reencoding works fine if the 12344 56788 sequence is repeated consistently. If the file comes from older films, chances are that the 24 => conversion has been done by chunks and the sequence does not repeat neatly.

Now, I know this could be done, but it's not what I want:

Extract all frames with FCP or MPEGStreamclip, delete the repeated frame files, rejoin with Time Lapse Assembler (free). Extremely time-consuming for long files, and the video may lose quality.
Apps like Loosless Frame Rate Converter (free) may work when no duplicate frames are found, that is, when a 24fps movie, for example, has been converted to 25fps just by speeding it up by 4% (I also hate that).

No, the idea is to input a file in an app or utility, tell the program to delete every nth frame, rejoin without reeconding. For instance, does anyone know if a script / method exists for MPEGStreamclip (which doesn't reencode) to do this?

Thanks!

Final Cut Pro X, Mac OS X (10.6.8), Trial version

Posted on Apr 27, 2015 5:47 AM

Reply

Answer 1

Apr 27, 2015 8:00 AM in response to Celso A.

YOu may want to take a look at Qtcoffee. It can do a lot of manipulations on movie files with great flexibility. Not sure if it will do what you want but I expect it will.

Reply

Answer 2

Celso A. Author

Level 1

14 points

Apr 27, 2015 9:33 AM in response to Luis Sequeira1

Thank you very much for the suggestion. I've downloaded Qtcoffee and looked at the documentation: http://www.3am.pair.com/QTCoffee%201.2.5%20Read%20Me.html . For what I can see, there's no specific command to delete every nth frame. All I can think of is splitting, say, a 30fps file (from <=24fps) into 5-frame chunks, then resplit these chunks by the last, repeated frame, discard these frame files, and lastly rejoin (cat) the remaining 4-frame chunks . A 1-hour video, for example, would generate 21,600 5-frame chunks, then divided by 2 = 43,200 files ! 😉 Not neat.

Now, if someone could come up with the QTcoffee syntax to (a) split the file, (b) delete the 5th frames, and (c) rejoin the remaining 4-frame chunks , all in one line, without producing the intermediate files (but in memory, perhaps), that could work. Would anyone dare to try that? 🙂 My knowledge doesn't get me that far at all.

Reply

Answer 3

Jul 13, 2016 11:17 AM in response to Celso A.

It is critical that the clip start at the beginning (frame 1) of this sequence —

For example: 1 2 3 4 4 1 2 3 ... Trim the clip as required. 24>30 & 25>30 duplicate rates vary.

Note: sometimes the duplicate frames are not regular over the length of a movie due to multiple edit patches from previous files. I've run into a variety of odd frame rate issues working with archival files.

To fix this when converting to a higher frame rate —

1. Set Compressor frame controls to interlace > reverse telecine*, duration > 100%.

2. Open the resulting clip in Cinema Tools and conform to 30, 29.97 etc. When conforming up to higher rate, the resulting clip will be shorter and, hence, faster. Vice versa converting down.

3. Return the shortened to the original speed using Apple Motion with Optical Blending set at 83.33%, 80%, etc.**

This is a slow process that eats available ram, but definitely produces the best results.

Or: For a less precise, but far faster method, use Frame Blending instead —

Compressor > Rate Conversion > Best. Duration > 83.3333%, 80% etc.

Similar with Motion or FPC. Results vary somewhat.

FPCX does this more conveniently when adding nonconforming clips to the timeline, but may not be faster, especially for Optical Blends — https://support.apple.com/kb/PH12648?local

* Compressor "interlace" control works with progressive clips.

** 25/30 = 83.33%, 30/25 = 120%. 24/30 = 80%, 30/24 = 125%. Etc.

Michael

loosethreadscinema.com

Reply

Answer 4

Sep 10, 2016 3:26 AM in response to Celso A.

I am not aware of any app that will do this automatically, and certainly not without re-encoding, since just about every video format known to man uses some form of temporal compression hence to manipulate individual frames requires re-encoding.

I developed my own method of doing this - it's partly-automated, clunky and not for the faint hearted - but much better than doing it manually on any clip more than a few seconds long.

First, I wrote an FXscript in FCP to detect duplicate frames and output the frame number of each duplicate to the Mac Console using the FXscript debug command. I then cut-and-paste the output of the Console into Text Wrangler to grep out the frame numbers (since the Console also spits out extraneous text) and past that into Excel.

At this point some manual checking is required since the cadence of the duplicated frames may not be constant if the clip is more than a minute long. For example, a 25 fps source converted to 29.97 fps will not simply have every 5th frame duplicated - eventually the sequence will spit out 9 unique frames before duplication restarts. Also, the FXscript I wrote to detect duplicate frames will trigger some false positives, especially with slow moving or still scenes. So by calculating the cadence in Excel you can fairly quickly strip out the false positives as well as pick up when the cadence skips, and come up with the correct list of duplicate frames.

I copy the duplicate frame list to a text file (called index.txt) and then use Quicktime Pro or Compressor to export the original clip as an image sequence in a folder, using whatever image format you think gives the highest quality. I then wrote an Applescript to read index.txt in this folder, delete all the duplicated frames so indicated, and run Quicktime to re-import the remaining image sequence as a new clip with the required frame rate and no duplicated frames. From there you can export to any movie format you like and restart editing.

Like I said - clunky and not for the faint hearted - but it works. I'm sure it would be possible for someone much smarter than me to write a FCP / FCX plugin to do this automatically. The hardest part would be to intelligently and correctly detect the cadence of the duplicated frames with no false positives.

Reply