Quicktime X: Trimmed video does not make file size smaller than original.

What's going on here?

Hard to say for sure what with QT X being a "transitional" version of the application and MPG (MPEG-1) content traditionally not being a fully "edit compatible" compression format. However, based on your description of what is happening, I would guess that the trimming of your file in QT X merely resets the "in" and "out" points for playback and that when the file is "saved," the data is not physically altered but copied, in tact, to the new file for a "logical" playback of the portion you wanted and, thus, avoid problems with audio/video synchronization owing to the specifics of your editing points.

My normal recommendation is to use MPEG Streamclip (free) for the editing, merging, and fixing of timecodes of stream based (i.e., formats containing interspersed blocks of audio and video data in a single stream) as opposed to the multiple track, time reference based technology at the heart of QT.

It edits the video properly, but the output is always pixellated.

Are you converting, demuxing, or saving the data? Demux and save operations do not re-compress the the original data. If converting/recompressing the data and you are getting pixelation artifacts, try increasing the video data rate.

I still dont understand the Quicktime X trim problem still.

MPEG-1 (MPG) files contain a series of audio and video blocks of data which have been interspersed in a single data stream so as to be, among other things, self-regulating in terms of synchronization. Put simply, your MPEG-1 data blocks are not edit compatible at the frame level and QT was created for use as frame-level software. MPEG Streamclip is simply a bridge that allows you to either edit your content at the block level or convert the data from "blocks" to the "frames" of independent track data that QT was programmed to handle.

I dont think what I am experiencing is normal. It makes the whole trimming feature useless
It is quite normal. In fact, if you try to perform this editing in QT 7, you will find that the in/out markers will not even activate for use in trimming an MPEG-1 file since they are now "contextually" programmed to recognize what it cannot edit. Since the ability to trim files in QT X seems to have become a marketing issue, my guess is that the QT people at Apple came up with this method of "logical" trimming specifically for handling "muxed"content. And, if you do a bit more research in this forum, you will find the issue of trimming multiplexed compression formats is not a new one or an abnormal experience. If you really want to trim the content in QT X and share directly to YouTube, you can always use MPEG Streamclip to convert your files to high data rate/low compression QT edit specific format like DV/DV, DV/AIFF, AIC/AIFF, Animation/AIFF (if not editing in iMovie '08/09 and maybe not with QT X), H.264/AAC with unlimited data rates, etc.