Cease multi-room function with Homepods

I have two pairs of stereo Homepods. One pair is the default audio output for an Apple TV. I decided to try multi-room music so I asked Siri in the other room to "Play the same music as the Apple TV". That worked and the music plays on all four Homepods.

Now I want to undo my experiment. However Siri in the other room doesn't seem to recognise any command to do this. I tried "stop playing the same music as the Apple TV" and "no longer play the same music as the Apple TV" but Siri just has a guess at (unwanted) music with a similar title and plays it on all 4 Homepods. Saying "stop" stops the music on all 4, even if the music is coming from the Apple TV.

There doesn't seem any way to undo the connection with the Home app on an iPhone.

Any suggestions that Siri might understand would be appreciated.