What is the make & model of your modem?
You pretty much stated the correct wiring. There should be an Ethernet cable connected between the modem and the WAN (circle of dots) port on the AirPort Express base station.
Anytime you change networking hardware it is always a good idea to perform a complete power recycle of that hardware. Check out the following AirPort User Tip for details. Please post back your results.