Many thanks FiremanJig for posting a solution that should work for some people.
Sadly not me! I made the same assumption as SILVAFOXX and thought it was all in the cloud. In addition, I did wait to wipe my old phone but as it looked like all the activity data was slowly appearing I thought it was safe to do so. Lesson learned.
Also to mention to save anyone time - I happened to have my old phone and watch still so I put old backups on those and then tried to follow FiremanJig’s solution, but no luck. It seems like as the backup hasn’t got the challenge data in there, it doesn’t resurface if you try to go back to the setup you had before.
Still annoyed, like others I had every challenge this year so far. But I can’t see any chance of getting it back so just have to move on.