iPhone HOME app always asks me to "Teach Siri Your Voice" even after I have done it

Much like the description say, I've got my house set up with 3 Homepods and 3 AppleTV. Every time I launch the Home app it makes me teach SIRI my voice (default 5 questions). SIRI works on the home pods and AppleTVs and on my phone, but it asks for me to teach it my voice every time I launch the home app.

What can I do?