How do you remove the swivel stand?

It is difficult to do, but try this:

You'll need a couple of different sized hex key wrenches. Put one into the hole on the metal cap on the bottom of the base, and, if you're extremely lucky, you'll be able to turn it counterclockwise and loosen it without the rest of it turning as well. If that works, the rest is easy to remove.

If it doesn't loosen by itself, look between the memory access panel and the upper part of the base, you'll see another smaller hex key hole there. The right sized hex key (the short end) will just fit into that hole to hold things in place while you loosen the metal cap.

Good luck.

You need a 4.5 mm hex key, I believe.

It helps to have two people - someone to hold the sliding base. I recently needed to remove the base on my eMac so that I could install additional memory. When I got someone to hold the base in place, it was a lot easier. Good luck!

This is tough...I just put one on. Turn your emac upside down. Slide the base toward the back of the computer. Fit the allen wrench ( I believe its 4.5) into the one hole on the base closest to the front of the computer. Then you need to bend the long part of the wrench in half. Yeah I know its kinda rigged but oh well. This is to get under the swivel for the back two screw holes. My husband was upset I bent it but there was no other way. If you intend to keep this stand off, you might be missing a screw. I had to take one out to install this thing.

BINGO:
I searched and found this. got really worried when I read about the problem because I am waiting for a swivel stand to come (via ebay) in the mail for the eMac I got recently (also via ebay). doubt that it will come with screws, so still might have to figure that out.. guage and length.
http://www.info.apple.com/usen/cip/pdf/emac/emac-installstand.pdf