Simon, I had the same problems. Export as quicktime self-contained, include DVDSP markers and DON'T check the recompress all frames box! Including the DVDSP markers makes sure the compression markers are exported as well. It makes sense to insert and name DVD chapter markers as well in FCP, as they will be included in the mv2 file you eventually import into DVDSP.
Then import the quicktime into compressor, and create a new preset that is 1-pass, NO VBR, 7.2 bitrate, motion estimator BEST. If you drag your movie into the preview window you can see all the compression markers that FCP automatically included, as well as the chapter markers (different color).
Submit the job. You can compress the audio to AC-3 the same time as you compress the video by adding the Dolby 2 function to your new 1-pass preset. All your audio must be 48K 16bit in FCP before you export, or it will go into the center channel on 5.1 sound systems.
Import the finished m2v and audio files into DVDSP. Notice how your chapter markers you inserted in FCP magically appear! Do your menus and buttons etc., then build the job. I don't use build/format as I have had better luck just dragging the master build folder that contains the VIDEO_TS folders into TOAST 6 and burning it at 1X DVD setting just to be safe.
All my previously artifacted movies, which are heavily effected with lots of transitions. NOW HAD NO ARTIFACTS WHATSOEVER! My thought of suicide or at least a Ben & Jerry's binge had passed!
All I did was a little tweaking, basically encoding using compressor's "lower quality" 1-pass setting, not the usual DVD 90-min. best setting, which the manual says to use for the best possible quality.
You can bypass Compressor stand-alone completely, because DVDSP uses Compressor to encode imported quicktimes as well, and you can change the settings to 1-pass 7.2 bitrate there as well. Compressor just has more advanced settings available.
FCP also uses Compressor when you "export using quicktime conversion." So it's hard to get away from using it unless you have a third party mpeg2 encoder. I'm thinking about BitVice, which markets its product as "Probably the best software MPEG2 encoder in the world"