Rpstephenson2001-
The missing 4G icon is probably an indication you are in an area that does not have good (or any) 4G coverage.
When you say "power off", do you mean holding the Sleep button until the "Slide to power off" arrow appears? If so, your iPad is probably getting a fresh start by establishing a new connection to AT&T when you turn it back on.
One thing you can try is to reset (reboot) your iPad. Hold both the Home and Sleep buttons until the Apple logo appears. Ignore the "Slide to power off" arrow. The iPad will restart after a couple of minutes.
Resetting this way will not hurt anything, and sometimes clears up mysterious problems. However it probably will not turn on the 4G icon if you are out of the 4G coverage area.
Fred