This is actually an issue. You shouldn’t have to “turn off” hey siri on other devices. The way it’s supppsed to work, is when you say hey Siri, all devices within range communicate with each other via Bluetooth and if the HomePod is in range, it’s supposed to take priority. It seems to be an issue, as I opened a support case with Apple as the same thing is happening with my Apple Watch and my HomePod. Apple has spent about 5 callbacks with me, capturing logs on both my watch and my iPhone and is currently working on it. For the OP, make sure bluetooth is on, on all your devices is the best I can suggest, but it may continue until Apple releases a fix.