This is my pet checklist for questions regarding FCP X performance - you may have already addressed some of the items but it's worth checking.
Make sure you're using the latest version of the application - FCP X 10.0.5 runs very well on my 2009 MacPro 2 x 2.26 GHz Quad-Core Intel Xeon with 16 GB RAM and ATI Radeon HD 5870 1024 MB. I run it with Lion 10.7.5.
First, check that you have at least 20% free space on your system drive.
For smooth playback without dropping frames, make sure 'Better Performance' is selected in the FCP X Preferences - Playback Tab.
If you have not already done so, move your Projects and Events to a fast (Firewire 800 or faster) external HD. Make sure the drive's formatted OS Extended (journalling's not required for video). You should always keep at least 20% free space on the Hard Drives that your Media, Projects and Events are on.
Check the spec of your Mac against the system requirements:
http://www.apple.com/finalcutpro/specs/
Check the spec of your graphics card. If it's listed here, it's not suitable:
http://support.apple.com/kb/HT4664
If you are getting crashes, there is some conflict on the OS. Create a new (admin) user account on your system and use FCP X from there - if it runs a lot better, there's a conflict and a clean install of the OS would be recommended.
Keep projects to 20 mins or less. If you have a long project, work on 20 min sections then paste these into a final project for export.
If your playback in FCP X is not good, I strongly recommend you use ProRes 422 Proxy - it edits and plays back like silk because the files are small but lightly compressed (not much packing and unpacking to do) - but remember to select 'Original or Optimised Media' (FCP X Preferences > Playback) just before you export your movie, otherwise it will be exported at low resolution.
The downside of 'Proxy' is that it looks awful. DON'T use Proxy when you're assessing things like video quality - especially focus.
If you have plenty of processor power, for the ultimate editing experience, create Optimised Media - most camera native files are highly compressed and need a great deal of processor power to play back - particularly if you add titles, filters or effects. ProRes 422 takes up much more hard drive space but is very lightly compressed. It edits and plays back superbly.
Personally, I work with XDCAM EX and h.264 from a Canon DSLR. Both of these run fine with my system, but I do transcode multicam clips.
Hide Audio Waveforms at all times when you don't need them (both in Browser and Storyline / Timeline). They take up a lot of processor power. (Use the switch icon at the bottom-right of your timeline to select a format without waveforms if you don't need them at the moment, then switch back when you do).
Create folders in the Project and Events libraries and put any projects you are not working on currently, in those folders. This will help a lot. There's a great application for this, called Event Manager X - for the tiny cost it's an invaluable application.
http://assistedediting.intelligentassistance.com/EventManagerX/
Unless you cannot edit and playback without it, turn off Background Rendering in Preferences (under Playback) - this will help general performance and you can always render when you need to by selecting the clip (or clips) and pressing Ctrl+R.
The biggest single improvement I saw in performance was when I upgraded the RAM from 8 GB to 16.
If you're getting crashes and none of this helps, you might have incompatible or corrupt media or project files. Move ALL your Events and Projects to temporary folders so that FCP X doesn't find them on launch. If it launches OK, re-introduce the Events and Projects one at a time, re-launching each time, so that you can track down the corrupt or incompatible files.
If FCP X doesn't launch (with no existing Events or Projects) I suggest you drag the application to the trash and reinstall it from the AppStore (using the same login that you used to buy it).
Andy
