Chapters and Autoplay options

The easiest way is to put them all into one iMovie and set chapter markers within iMovie at the beginning of each project.

I can add these to iDVD as chapters and get each to play independently

No you can't get true 'CHAPTERS' to play independently. DVD chapters are like chapters in a book - a place to start. The DVD will then continue playing until the end of all the content.

but I would also like to have an option to play all back to back.

In iDVD, playback will start at the beginning and play to the end.

With iDVD, if you want to have the ability to play all the clips at once as well as each clip separately, you will need to have the content on the DVD twice; once as a separate clip and once as a movie of all the clips.

If you put the separate movies into one iMovie in the order you wish, they will all be joined as one movie.
You cannot put one iMovie project into another iMovie project but in iMovie 6 you can just drag the clips from one project into another project.
Others will have to address this re: iMovie versions 8/9/11 for you.