Playing back of AC3 would be an issue. You can install the right kind of component to allow that, but it would be a tough requirement to impose on everyone who wanted to view your movie. If you were to use MPEG StreamClip to do the demuxing, you could have it save the audio out as AIFF, and then use that in your movie, and compress to a standard codec to reduce the file size.
One thing you CAN do is use AC3 audio in Toast, so maybe make a DVD (to get a playable MPEG-2 VOB file) then using MPEG Streamclip export that to a QT format.