my restoring contact keeps for a long time

The iCloud sync is very, very slow. If you want to reset the sync and start over, try going to System Preferences/iCloud (Apple ID for Catalina or later) and stop Contacts syncing by unchecking the box. Wait a couple of minutes, then check syncing. Allow time for the sync to complete.

Last resort  - have you tried signing out of iCloud, restarting, and then signing back in?

See the Apple Support article below. 

Contacts, Calendars, or Reminders - Get help using iCloud.