Apple favorited photos completely scrambled after syncing favorites with google photos

I recently enabled the “sync favorites” feature in google photos which says “favorites from your Camera Roll will sync with Favorites in Google Photos.”

Seemed harmless. However, after turning on this feature, my favorited photos in apple photos got completely scrambled - it looks like random ones became favorited and unfavorited.

This very frustrating because I have tens of thousands of photos going back over 10 years and they’re all affected, so there’s no fast way to manually fix this. My camera roll organization is completely dependent on this favorites feature so my entire library is affected.

I’m a bit hesitant to turn off the feature in google photos since I manually did go through and fix a few thousand, and I’m afraid it’ll just mess it up again.

Please advise! Is there any way to fix this?