I too have been having this problem.
For the moment it seems to have been fixed.
I preformed a restore on 3 separate occasions and each time it worked for
a Day then again it disconnected. After the last restore when
setting speaker up it complained about not seeing wifi turned out
even though my phone was showing that it was connected to my wifi
it was not. I really don't know if this had anything to do with the problem
as it connected the first 2 times then failed but I figured I would mention it
as it was part of the process.
After re-establishing my wifi connection the Homepod Mini finished setup
and has been working for a few days now.
Maybe just keep at it till something sticks. Wish I could tell you what finally
fixed mine but I have no clue.
All of this started with the 16.4 update and the .1 update did nothing to fix it.
I have 2 HomePods and this is only happening to one of them.
Have you connected the HomePod to your computer and preformed a restore?
Reset HomePod or HomePod mini - Apple Support