Check to insure that the Ethernet connection from the AT&T router is connected to the WAN "O" port on the AirPort Extreme.
Check the settings using AirPort Utility on the AirPort Extreme to make sure that it is set up this way:
Under the Wireless tab.......
Network Mode = Create a wireless network
Wireless Network Name = Exact same name as the AT&T network
Wireless Security = WPA2 Personal
Wireless Password = Exact same password as the AT&T network
Verify Password = Same as above
Under the Network tab.......
Router Mode = Off (Bridge Mode)
Click the Update button at the lower right of he window to save any changes that you might have made to the AirPort Extreme.
You now have what is known as a "roaming network". Both the AT&T and Apple router are providing a wireless network, but since the name of the network is the same on both routers, the network will act as if it is one "big" network.
The theory with a roaming network is that wireless devices will connect to the router or wireless access point with the best signal and then switch access points as a device moves from one area to another.
That's the theory. In practice, a Mac laptop will usually switch access points automatically when it moves from one area to another, but iOS devices like the iPhone and iPad, and other mobile devices and PCs will not.
So, you will need to get in the habit of temporarily turning off the WiFi on these devices when you move them from area to another, then turn the WiFi back on again and the device will usually pick up the stronger signal from the closer wireless access point.