What kind of applications do you want to run in Linux. If you do not need to do any processor intensive stuff in Linux and you have an Intel-based Mac, you could install Parallels, then install Linux in a virtual machine. See
http://www.parallels.com for details.
If you need to do processor intensive computing with Linux, then your best option is to buy a cheap PC, then install Linux and your apps on it. You can easily access the PC from your Mac desktop.
Terminal.app gives you a local shell session, so once you figure out how to connect manually to your linux box and log in, it's probably relatively easy to create a file that opens in Terminal.app and automatically does that login.
You cannot redirect remote X11 sessions to terminal in OS X (terminal is not X-capable). You can do that if you install Apple's X11.app, a package installer for which should be on the discs that came with your machine (i.e. the Options_install.mpkg). Once installed, X11.app will be in your Utilities folder - just double click it to start an xterm session.
The easiest thing to do then would be to ssh into the Matlab server:
The "-X" will enable X window forwarding from the remote machine to your local X11 session. The -l just tells ssh to login with the supplied userid (if that is different from the short name on you local Mac account).
You cannot display an X-window forwarded session in OS X terminal app. Terminal is incapable of interpreting and displaying X sessions.
This all must be done from a xterm in X11 (or whatever terminal emulator you choose to run in X11 - I personally use rxvt, xterm is the default installed with X11.app).
Of course you do not display the X-windows in the terminal. The X-windows are separate windows. But you can ssh using the Terminal.app, and initiate X applications from the shell prompt. The windows are forwarded by ssh. I do this with multiple remote hosts all day everyday. It works great.
I guess I just don't understand why you are using terminal at all. As you mentioned, X11.app needs to be running for that to work, so why even open terminal? Just fire up X11.app and do it all from an xterm session (I don't see the point of using two applications where one is all that is needed).
To make it easier, make an alias in your shell profile for the "ssh -X ..." command string. Then just double click on X11.app, and type the alias in xterm.
Terminal is a much nicer environment than xterm -- it supports many OS X features, such as drag and drop. Plus, I can use the method outlined above to create double-clickable icons for various hosts that I access. The X11 applications I use are not terminals. The original poster wants to run Matlab, which puts up various graphic windows; I use IDL, which is similar to Matlab. Terminal is for running command-line shells. X11 is used for Unix/Linux GUI applications.