Yeah, Neat Video does take a while. Sort of about 1/8 X. It's done wonders for me, though, as I have some Super 8 movies that have been through the projector a wee bit too many times! (Neat Video also has a dust and scratch removal feature, which I'm guessing you don't need.) It has trouble with stop-motion animation, though. You might have to make your moving objects.
An interesting advantage with Neat Video (I get no remunerations for saying nice things about it) is that you can fine-tune it by running its optimize function, which will try all combinations of number of CPU cores with and without your GPU.
There are times when you can disable it, either by unchecking it on the Video Inspector or toggling its own on/off switch (see the fine manual). If you're drawing a mask, e.g., or animating one, or doing color corrections, or whatever, and they're not affected by the noise reduction, disable the noise reduction. Otherwise it will run every time you make any changes! One thing I do, and I'm not certain how much it helps, is to type in numbers into the color controls (where possible) instead of smooth sliding. It'll be easier to keep track of where you are, at least -- due to the response delays you get.
Are you using the FCP noise reduction? The results of its sharpness function are kind of harsh, but while its degree can be adjusted, it cannot turned off. You'll probably get better results with Neat Video, but you have to help it by setting up a noise profile for each clip. Creating a good noise profile is very important. A bad profile will give you -- surprise! -- bad results.
Super NB: Do not use NR more than once in any given clip, e.g., at multiple layers in a compound clip. If you're compositing, only run it at the top level (actually I think you'd be best off putting the layers in a compound clip, and then run Neat Video on the compound clip). If you do use it more than once in any given clip, it will be really, really, really slow. Forget about getting a cup of coffee. You can go to the store to stock up. I once tried mixing FCP NR and NV NR. You have to run FCP first; otherwise, you get halos. I found that doing that can improve the sharpness, but I ultimately decided against it because FCP overshapening is a little to harsh, and it takes so goshdarn (can I use the raw version here?) long.
I don't recall the stabilization function being as slow, but every shot required different methods and settings.