Siri Won't Recognize Voice Except Home Owner

Just got two new homepod minis and neither one will recognize more than one voice. They will only recognize the owner of the Home and not anyone added to the Home. I've tried almost everything I can find on the issue and Siri just won't recognize more than one person. So here is what I have tried:

1. reseting homepod
2. deleting other person from the home and adding them back
3. name in my contacts match the name in their contact information
4. Listen for Siri and location services are turned on.
5. Have turned off "Hey Siri" and turned it back on and done the whole setting up thing
6. Have removed homepod and added it back several times
7. All devices are on 15.4.1

I have read all the support articles so posting those won't help.

Nothing has worked. Even created a new home with the GF as the owner. Siri then recognizes her but won't recognize me. No matter what we have tried, she only recognizes one person at a time.

Anything else to try to get Siri to work as advertised?

Thanks