Q: How do I close mail?

Clicking on the red 'x' closed all windows, but still runs in the background.

If you wish to close it down completely, Quit, the COMMAND + Q or Mail > Quit

This applys for most applications in OS X.

Force Quit .

Press command + option + esc keys together at the same time for 3 seconds.  Wait.

When Force Quit window appears, select Mail, if not already.

Press Force Quit button at the bottom of the window.       Wait.

Application will quit.

http://support.apple.com/kb/HT3411