why does my h.264 file look so bad?

Question

Level 2

435 points

why does my h.264 file look so bad?

I've got a project from FinalCut 5 and i'm trying to compress an h.264 version to experiment. Every setting i try, the final result is less than satisfactory. Lots of artifacting, gradients and text from Motion are jagged. What am i doing wrong here? The footage was shot with a Sony HDV cam...downconverted to DV, all Motion files were encoded using the Animation codec, and there are some Nattress film effect filters applied to some of the clips.

I've tried exporting directly to compressor from FCP, exporting to uncompressed quicktime and then sending that to compressor, and tried exporting straight using quicktime compression. No matter what my h.264 settings, the final looks bad.

can anyone point me in the right direction?

Posted on Sep 26, 2005 6:02 AM

Reply

Answer 1

Waymen

Level 5

5,722 points

Sep 26, 2005 7:58 PM in response to pherplexed

What preset are you using for the H.264 encoding? What is the output size (WxH) of the video, what is the bit rate? H.264 might give you something that approaches a 2X advantage over MPEG2 but it can't produce miracles. If you are requesting a large output size at a very low bit rate it probably won't look that good. You are still going to need bit rates in the low Mbps range if you want really good looking video at sizes at or above 640 x 480.

Look at Apple's samples, where a 844 x 480 movie is encoded at about 3.6Mbps (including AAC audio).

Reply

Answer 2

pherplexed Author

Level 2

435 points

Sep 26, 2005 8:41 PM in response to Waymen

The last version i rendered was 852x480 px using 12.70Mbps bit rate. It still has bad gradients and jageddy text. I just downloaded the Final Cut Suite demo Apple had on their site and the specs on that are 800x450px at 761kbps and it looks amazing! gradients are smooth and text is crystal clear (not to mention it's only 21mb in size compared to my whopping 123mb!). There's a setting somewhere i'm missing i know it!!

thanks for any help.

Reply

Answer 3

Waymen

Level 5

5,722 points

Sep 26, 2005 9:29 PM in response to pherplexed

The last version i rendered was 852x480 px using 12.70Mbps bit rate

Given the resolution of your movie and that you are using H.264 that's an unnecessarily high bit rate. Since you think Apple's FCS demo looks so good why don't you try re-encoding a sample of that to see if the quality is maintained. Also, I'm sure Apple's demo was in a progressive format, you haven't said whether your source was interlaced or progressive and I think some of Apple's presets include deinterlacing which you may or may not want to do (check under the frame control and filter settings).

I have seen a few reports that QuickTime 7 may have some issues with RGB color space conversions but I don't see how that could be part of your problem since neither DV nor H.264 are using RGB. However, if you converted from RGB at any step in your workflow then perhaps that is part of the problem. I don't know any of the details concerning these reports, it may or may not be appropriate to your situation.

Reply

Answer 4

pherplexed Author

Level 2

435 points

Sep 27, 2005 7:23 PM in response to Waymen

I think you're right about apple's clip being done in a progressive format. our video footage was shot in interlaced mode and we're using the standard ntsc/dv sequence settings. i guess i'll play around with diff. sequence settings and see if i can at least get the content that came from Motion looking as top-notch as Apple's video!
thanks for the insight.

Reply

Answer 5

Joe Rice

Level 1

125 points

Oct 11, 2005 4:27 PM in response to pherplexed

One red flag that jumps out at me is the conversion to DV. Do you have the space to use a higher quality intermediate codec? If doing text and/or effects, I'd recommend using Uncompressed 8-bit 4:2:2 for your FCP sequence codec.

Converting to DV is going to resample color at 4:1:1, lowering chroma resolution, and causing jaggies especially in areas with high red or blue content.

Furthermore, in the sequence settings for the sequence where the downscaling from HD to SD is occurring, go to the Video Processing tab and set the Motion Filtering quality to best. That may improve the HD to SD conversion.

Joe

Reply

Answer 6

pherplexed Author

Level 2

435 points

Oct 11, 2005 5:27 PM in response to Joe Rice

Hey Joe--thanks so much for the insight. I had a feeling that downconverting to dv was part of the culprit. Also, your comment about resampling color at 4:1:1 (not entirely sure what that means) but you are absolutely right about what it will look like: the one text/effect sequence that looks the worst is red lettering!

I'll try the sequence and encoding settings you suggested and see what we get.

Thanks again (i gave you a couple of + votes for your help!)

Reply

Answer 7

Joe Rice

Level 1

125 points

Oct 11, 2005 5:54 PM in response to pherplexed

No problem - hope it helps.

Adam Wilt has a good intro to color sampling here:

http://www.adamwilt.com/DV-FAQ-tech.html#colorSampling

And some pictures here: http://www.adamwilt.com/pix-sampling.html

Basically it refers to one method digital video codecs reduce the amount of video data. For every four pixels across, the codec will store the luma value of each pixel (essentiall how dark/light it is), but will average the amount of chroma information for Cr and Cb (very generally red and blue) and only use one value for all four pixels for each of these.

In DV, where you've got a lot of blue or red information, this can show up jagged or blocky. If the source was DV, setting the sequence to uncompressed and applying the Color Smoothing 4:1:1 filter (in the Keying filters section of FCP) to the DV source can help.

In this case, since your material didn't originate as DV, just changing the sequence settings should do the trick.

Joe

Reply