YouTube imposes a data rate limit on uploads. For 1080, that limit is 8Mbps. If you upload anything with a data rate larger than that, YouTube is going to re-encode down smaller... you do *not* want that to happen.
FCPX exports h.264 somewhere in the region of 45-50Mbps (the nature of H.264 is that it is most likely to come in a good bit smaller than the "target" data rate... I won't go into details why here.) There are no options (other than picking a different target device) to alter FCPX's encoding method.
My advice is to "share" out of FCPX with ProRes 422 (LT should be good enough) and use Quicktime 7 Pro to transcode to H.264. It's fast, and it's very difficult to screw up the transcoding. When you're done, you can throw away the ProRes version.
Over the course of the workflow, you save considerable amount of time. ProRes 422 out of FCPX is usually quite fast particularly if you've already rendered your storyline. Converting in QT7Pro is also fairly quick (all things considered, and it IS h.264...). Uploading the very much smaller file size to YouTube is where it makes a world of difference. When I upload videos to YT, they are almost immediately ready to go (after 100% is reached, it only takes a minute or two before it's "live"... If YT has to reencode, it can take as long as you transcoding the H.264 from the original master before the video is ready to go live... )
Settings:
Export: Movie to Quicktime Movie
Options:
(the details in this capture show settings for 2K export at YouTube's 16Mbps limit — important detail here: all three checkboxes checked and Fast Start selected. You won't be using the Filter options.)
Settings:
(adjusted for 1080 — it's very important that Frame Reordering is OFF... when you Restrict the data rate, you cannot select Best for the Compressor Quality... always do Multi-pass. Let the compressor figure out Key Frames, it's smarter than we are.)
Size:
(Preserve aspect ratio might be a "legacy" consideration -- it corrected problems youtube used to have displaying HD in the "early days". It won't hurt your video and will guarantee it is played at the correct aspect ratio.)
Audio:
You can be a little more liberal with the audio settings, but higher values can bump your data rate up over the limit in some cases and you really cannot tell the difference in quality at higher rates. The Sound Quality slider position translates to 160kbps. VBR works just fine and YouTube does not "complain" about it (even though a constant bit rate is recommended in the specs.)
If you don't want to use QT7Pro, use Compressor. I personally do not trust third party software to do as good a job as Apple's software. Try to limit how far into Compressor you go for settings... you're likely to screw it up. Stick with the settings you see listed here. Always open the finished video in QT (either version.) Get Info on the file and check the data rate before uploading.
