Hardwire Express units to my Extreme base station?

Can I use the RJ45 jack in the back of the Express units to hardwire them into the Extreme base station?

Yes, if you use the WAN "O" port on the Express.

When I tried this, I got no signal, so I figure it's either not possible or I need to reconfigure settings.

The AirPort Express devices were set up to expect a wireless connection from the AirPort Extreme. If you did nothing except add the Ethernet connection, the Express then saw two connections, both wired and wireless, was confused and did not know what to do. This also created a feedback loop over the network, effectively crashing the signal from the Express.

If you want to connect the Express using Ethernet, the correct solution would be to first reset the Express back to factory default settings. Then, establish the Ethernet connection before you set up the Express again. When the setup utility "wizard" sees the wired connection, it will configure the Express correctly to extend using Ethernet.