Firestore FS-5 video with Final Cut X

Although I don't have that camera nor use the HDV codec, I can say that I am now downloading a software update that contains 1) royalty free SFX and new pre-sets for audio for FCP X and 2) a bunch of new codecs. I believe I saw something about HDV in the overview. There is also a PRO APP update.

Try the update (through software update) and see if that fixes the problem.

Yup I saw that update as well - that may have been the update that caused the problem, I hadn't tried the files before.

So I think this issue is Firestore originated rather than FCP X. Talking with the Apple tech, we tried the method to Save As in Quicktime that Brad suggested earlier. That produced a smaller file size, but the file itself looks identical to the naked eye (and the bit rate/resolution are all the same).

Now, take a Firestore mov file, and simple open it up in Quicktime, then open the Inspector in Quicktime and check the file size - the file size reported in Quicktime is smaller than the size of the file reported in the Finder. And that file size reported in Quicktime is the exact same size you get when you do a save as. I'm in the process of emailing all of this information to the Firestore folks, but my guess is that their implementation of HDV relied on some tricks that worked in earlier Apple HDV codecs but aren't official or something.

Either way, the Save As trick appears to work just great for now, at least until a more permanent solution is found.

topher1078 wrote:

So I think this issue is Firestore originated rather than FCP X. Talking with the Apple tech, we tried the method to Save As in Quicktime that Brad suggested earlier. That produced a smaller file size, but the file itself looks identical to the naked eye (and the bit rate/resolution are all the same).

Now, take a Firestore mov file, and simple open it up in Quicktime, then open the Inspector in Quicktime and check the file size - the file size reported in Quicktime is smaller than the size of the file reported in the Finder. And that file size reported in Quicktime is the exact same size you get when you do a save as. I'm in the process of emailing all of this information to the Firestore folks, but my guess is that their implementation of HDV relied on some tricks that worked in earlier Apple HDV codecs but aren't official or something.

Either way, the Save As trick appears to work just great for now, at least until a more permanent solution is found.

Very interesting; thanks for the report. Have you installed the items offered by Software Update following the FCPX installation? I'd love to be able to use "Save As" to work around this problem, but on my system QuickTime (both v.10 and v.7) won't open the media files at all. The only tool I've been able to use (subsequent to the FCPX+Software Update installation) to open these files is VLC.

Phil

Yes, normal Quicktime 7 and Quicktime X opened them just fine for me after the FCP/Codec installation. Try Compressor - I'm using that now to batch "save as" these files to HDV.

topher1078 wrote:

Yes, normal Quicktime 7 and Quicktime X opened them just fine for me after the FCP/Codec installation. Try Compressor - I'm using that now to batch "save as" these files to HDV.

I tried this technique on a different computer and was successful. I've updated my bug report with Apple to include details on the differences between the two systems; hopefully they (and/or Focus) will be able to make the required changes so that this doesn't turn into an even bigger problem.

Did you use Compressor by chance? I'm having problems replicating the Quicktime's "Save As..." functionality in Compressor - each time I output it's doing a full reencode, which I definitely want to avoid.

topher1078 wrote:

Did you use Compressor by chance?

No; I just used "save as" within QuickTime. I'm not sure it's possible to use Compressor without re-encoding.

Thanks to everyone for staying on this. I think we have a do-able workaround now, but clearly getting the bug fixed is very important. If there is anything new on the issue, please continue to update this thread.

So as a workaround for now, I created an Automator sequence that takes a batch of Quicktime files as input, opens them, does a save as, and quits. It's not the most elegant of solutions, but it is an automatic and fairly quick way of taking Firestore HDV files and making them work in FCP X.

(Contained is both an Automator workflow and application.)

(Sadly, even though I tested this like ten times, it's not working now for some reason - will reupload if I get it fixed.)

Guess what - Lion can't open Firestore HDV files - like, at all. You can't open them in Quicktime and do a Save As..., Final Cut X, and even Final Cut Pro 7, can't open them, etc. I know Firestore/Apple are working on the issue (as I keep talking with them about troubleshooting and such) but for now, if you rely on a Firestore, DO NOT upgrade to Lion unless you have a 100% way of converting these files using different program.

Thx for the warning. Pls post if anything develops.

After getting a reply in my thread below:

https://discussions.apple.com/thread/3147038

and pointing to this thread, I got even more curious about FCPX and Lion's inability to play these DTE files. My original hunch was that FCPX and Lion were not playing these files because the files were not flat. I don't believe this to be true anymore, because Lion can play a non-flat H.264 file just fine. See the data below:

I thought I'd try out a Python application called qtfaststart.py, located here:

https://github.com/danielgtaylor/qtfaststart

Essentially, all this python app does is rearrange atoms in a Quicktime file and then re-save an output file via the terminal. But it also has the ability to list all the atoms in a Quicktime file. Here are my tests--please note that "non-flat" Quicktime files are the ones with the moov atom AFTER the mdat atom.

1.

Non-Flat file (direct from DTE):

mdat (6642176 bytes)

moov (3664612 bytes)

(0 bytes)

2.

Flat file (re-saved in Quicktime under Snow Leopard as you can't open the DTE files in Lion):

ftyp (32 bytes)

moov (2510 bytes)

free (24 bytes)

wide (8 bytes)

mdat (4884354 bytes)

3.

M4V file from Handbrake encoded from original (not flat):

ftyp (28 bytes)

free (132 bytes)

mdat (95504 bytes)

moov (1804 bytes)

free (132 bytes)

4.

M4V file metadata edit in iTunes (making the file not flat if it was flat):

ftyp (28 bytes)

free (132 bytes)

mdat (95504 bytes)

moov (3899 bytes)

free (132 bytes)

5.

M4V file from iTunes after metadata edit in iTunes, flattened in Quicktime, saved as .MOV:

ftyp (32 bytes)

moov (5946 bytes)

free (16 bytes)

wide (8 bytes)

mdat (95520 bytes)

6.

M4V file from iTunes, processed with qtfaststart.py, resulting in flat file:

ftyp (28 bytes)

moov (3899 bytes)

mdat (95504 bytes)

The one thing I notice from the listing of the atoms in the Quicktime file is that the DTE files are missing one important atom: ftyp. Looking at the Quicktime atom documentation, this is an atom declaring the filetype. I assume that FCPX and Lion are now requiring the ftyp atom to be there, and this is why our DTE files are not working.

The unfortunate thing here, is that qtfaststart.py does not work on our files from the DTE. It will work with MP4 files just fine, but not with the DTE files, possibly because of the codec (not sure on that one, it's only speculation) it just hangs.

If someone is already talking with Apple and/or Focus Enhancements about this, perhaps they should look in to this, and maybe some code just needs to be shuffled around to ignore the ftyp atom, just like the behavior in Snow Leopard and FCP7.

I can sort of understand why FCPX doesn't work with these DTE files, because that's just an app with its own hooks for Quicktime, and import engine. What I don't understand, is why Quicktime X and Quicktime 7 Pro can open a mov without an ftyp atom in Snow Leopard, but when Lion is installed, Quicktime X and Quicktime 7 Pro cannot open these same files--is something very drastically different in the Quicktime engine that this all changed?

Message was edited by: markclea

Great investitagive work - I'm taking a look through everything you wrote.

But to answer your last question, something did change drastically from Snow Leopard to Lion. Almost all of the video codecs present on Lion were rewritten using Apple's new AVFoundation framework. Same codec, different implementation. This rewrite is one of the major reasons for Final Cut Pro X's speed, as it takes much better advantage of video card hardware in the system. So Final Cut Pro X on Snow Leopard uses these new codecs, but everything else on the system does not, which is why you can still open up these DTE files in Snow Leopard.

In Lion however, AVFoundation codecs replace all of the old ones, so that's why these files don't open at all on the system.

Definitely email what you've found to Focus Enhancements - they're pretty good about responding, or even just give them a call.

Here are more observations that I made while working with different workflows last night. Seems like my assumptions about FTYP being needed for Lion Quicktime and FCPX may not be correct.

In Snow Leopard:

I used three different tools, Premiere Pro, iTunes, and MetaX. All of these have the ability to add metadata to a Quicktime file.

Here's the atom data on the original clip again. This does not work in Lion Quicktime or FCPX.:

mdat (6642176 bytes)

moov (3664612 bytes)

(0 bytes)

Here's the atom data on a clip imported into a Premiere Pro (CS4) Project (not placed on a timeline, not exported, simply imported--PPCS4 writes metadata to the file in place on import). This file works in Lion Quicktime and FCPX. Note that this file is not considered flat, and does not contain the FTYP atom:

mdat (6642176 bytes)

moov (6499 bytes)

free (3658113 bytes)

Here's the atom data on a clip imported into iTunes and metadata added (in this case I just inserted an Artist attribute). This file works in Lion Quicktime and FCPX. Again, not flat, and no FTYP atom.:

mdat (6642176 bytes)

moov (5038 bytes)

free (3659574 bytes)

Here's the atom data on a clip imported into MetaX and metadata added, and the file re-written. This file works in Lion Quicktime and FCPX. Again, not flat, and no FTYP atom.:

mdat (6642176 bytes)

free (2689 bytes)

moov (2772 bytes)

free (3659151 bytes)

In Lion:

I tried the same tools above to add metadata to the clip, and all failed. I assume this is because all of these tools are using something in the Quicktime framework to add a metadata atom, and since Lion Quicktime and FCPX can't read the file to begin with, it has no way of adding the information to the clip.

I also tried something sort of weird in Lion--simply change the extension from .mov to .mpg. Lion Quicktime nor FCPX could read the file, but oddly enough, Quicktime 7 Pro in Lion could read the file, but it reports all the codecs incorrectly, and doesn't play very smooth, and does not play sound.

I think the key here, is that if we have some sort of tool that has the ability to write metadata to the file, without using the Quicktime framework at all, we'd be able to use these files directly in Lion, or at least have the capability of converting the file in Lion. Right now, there is no way to convert a file in Lion. All of the atom data software I've seen, that don't use the Quicktime framework natively, all require the FTYP atom to be present in order for the file to work correctly.