Hello jmcgillivray,
Thank you for participating in the Apple Support Communities.
It sounds like VoiceOver is not staying in the Off position. I can imagine that this behavior would be very frustrating.
If VoiceOver toggles back to On by itself, even after you double tap the button, I recommend that you restart/reset your iPad:
Restart or reset your iPhone, iPad, or iPod touch - Apple Support
Next, you can also try this method to turn off VoiceOver using iTunes on a computer:
Managing Accessibility features using iTunes
- Connect your iPhone, iPad, or iPod touch to any computer with iTunes installed.
- In iTunes, select your device.
- From the Summary pane, click > Configure Universal Access in the Options section at the bottom.
- Select the feature you would like to use and click OK.
Note that if you're using iTunes 12, look for the option that says "Configure Accessibility" instead of Universal Access.
From:
Use Accessibility features in iOS - Apple Support
If the issue persists, reset the settings on your iPad:
Reset All Settings: All your preferences and settings are reset.
From:
Reset iPad settings - iPad User Guide
All the best,
Jeremy