How do I remove unwanted device from airport extreme

If you are saying that you have a wireless device connected to your network.....and you don't want it there.....then you need to change the password for the wireless network.......and then only give the password to users that you want to be able to connect to your network.

If a user who knows the password gives it to someone that you don't want to connect to your network, there is not much that you can do in that regard.

I changed the password but the person's computer has it set to join automatically

A computer cannot join a network automatically unless it has the correct password. If you changed the password, and the other computer can connect, then the other person must be able to access the router settings and find the password.

If I change the network name and password would that work?

Not if the other user knows how to see the settings on your router. He'll simply log on to the new network and enter the password. Unfortunately, security on the Apple routers is mostly an illusion. Anyone who knows anything about Apple routers knows this.

Are you using the same password for the Base Station as the wireless network? If you are, that is probably how the other user is getting in. You need to have a different password for the Base Station than you do for the wireless network.

Yes, but you will have to know the Wireless MAC Address, also known as an AirPort ID of the device to be able to set up a "rule" for that device. You can set up times that the device will be allowed to connect to the network each day.....or you can deny access altogether.

Again, this is mostly an illusion, and will not be of much use if the person knows how to get into the settings of your AirPort Express.

Access Control is really tricky....and it only can control a device that connects using wireless. If you have smart person there who knows that he can connect to the AirPort Express using a wired Ethernet connection, he can get into the settings that way and change things around if he wants.