There are more than one thing to this.
A. a DVD can never be any better than interlaced SD-Video (as on old CRT-TVs) - it's in the DVD Standard - whatever program used. Your Mac screen ar far more better.
The video was very shaky and jaggy. What could cause this?
Shaky - one very common origin to this is that iDVD is set to make PAL (25fps) or NTSC (29.97fps) (only choice there is as DVD Standard set's it) - and iDVD is feed 24fps or 50p or 60p or other more modern formats. OR that one tries to do a NTSC-DVD from PAL material or other way.
iDVD are not good in this frame rates conversions (fps) - So I use JES_Deinterlacer to do this and although it's free on Internet it does this conversion even better than pay-applications.
Jaggy - usually affects photos most and are due to the use of
• iMovie'08 to 11 - as they don't export OK over to iDVD but discard every second line in the picture - Better is to use iMovie HD6 or FinalCut as they can deliver 100% over to iDVD
• "Share to iDVD" was used - in any version of iMovie HD6 to 11 same problem. I do in iM'08 to11 - "Share to Media Browser" - and as LARGE - NOT HD or other resolution as this too degrades the final DVD result
Yours Bengt W