That sounds frustrating, especially with a second-hand watch. What is probably happening is that the previous owner had the watch paired with their iPad and when they factory reset it, the Activation Lock or the pairing association is still tied to their Apple ID.
Here is what I would try:
First, since you bought it second-hand, try to reach out to the seller and ask them to unpair the watch from their iPad. They need to open the Watch app on their iPad, go to All Watches, tap the info button next to the watch, and choose Unpair. That will properly remove it from their account.
If that is not possible, your best option is to have the watch reset again but this time make sure it is fully erased. You can do this by putting the watch on its charger, then pressing and holding the side button until you see the power off slider. Then press and hold the Digital Crown until you see Erase all content and settings. Tap it and confirm.
Once it is fully erased, try setting it up fresh with your iPhone. Do not try to restore from a backup that might still have the old iPad association.
If none of this works, the watch might still be linked to the previous owner Apple ID through Activation Lock. In that case you would need to contact Apple Support and provide proof of purchase from the website where you bought it. They can help remove the lock from their end.
Let me know if any of these steps help!
