In AirPort Utility 5.6, select the Extreme, then Manual Setup, then Internet pane > Internet tab, "Connection Sharing". Set it to "Off (Bridge Mode)".
In AirPort Utility 6.x, select the Extreme, click Edit, then Network > Router Mode > "Off (Bridge Mode)".
Click Update and allow the Extreme a moment or two to reset.
It's also a good idea to reset your entire network when adding a switch or any wired component for that matter. Power down your switch, modem, router, Extreme, then add the switch to your network and power everything up again.
That should fix it.
The settings are identical for the Express, and the order should not matter.
You should probably include a sketch of your proposed configuration - if you are using an original Express, its Ethernet port can be connected to a switch, but the means by which it connects to the Internet (wired or wirelessly) matters.
If your house power fails, devices power up independently when power returns. Each device has its own power-up time requirements, so the power-up sequence is not worth trying to control.
I hooked the Airport up next to my computer. It was in Bridge Mode. So I didn't change that.
I did notice that it was trying to setup its own network, so I changed the settings to make it extend my current wi-fi network (which is my goal)
So the incoming cable goes to the router (which has both wi-fi and a wired network), the router connects to the switch, the switch connects to the Airport (which is to extend the same wi-fi to a part of the house that gets no reception)
Unfortunately, when I moved the Airport Express to the other side of the network switch, again the Airport utility could no longer see it.
I've tested the network cable - It's OK
I've plugged a laptop into the cable, and the network lights at the plug show a connection made. (both at the computer and at the switch)
When I plug the Airport Express in, the lights at the network plug on the switch do not light (do not show a connection)
Why would an Airport with power and a good cable not "connect"? At one point in my trouble shooting, I think when I had reset the Airport, the light at the switch would flash every couple of seconds.
If the Express is connected to the switch, then do not use "Extend a wireless network". That will cause the Express to bridge your existing wireless network back to the wired LAN, creating a mess that is also likely to leave you unable to communicate with the Express, or prevent your network from functioning. If you find that occured, "hard reset" the confused Express and reconfigure it.
The setting you want is "Create a wireless network". Leave Connection Sharing in "Off (Bridge Mode)". If you provide the exact same wireless network name and security settings as your existing Wi-Fi, it will create its own wireless network. It just will happen to be indistinguishable from your existing one - that is normally ideal.