Can’t get the checking for mail to stop spinning

Did you try closing the mail app completely? Double tap the home button to get to the app switcher and swipe to find the mail app, then swipe up on the preview window to close it.

If that doesn’t work, try a forced restart, To force restart your device, press and hold both the Sleep/Wake and Home buttons for at least ten seconds, until you see the Apple logo.