I can pause video with Siri, but not play it

Occasionally, I am away from my Apple TV remote / my hands are dirty and I can’t use the iPhone remote.

I want to pause the video that’s playing (Apple TV+, Hulu, Netflix, etc), then resume it in a moment, using Siri (typically on my homepod).

PAUSING works. However, when I tell the Apple TV to play - it always starts playing music, instead of unpausing the video I was watching. “Unpause” doesn’t work either.

Does anyone know a way around this? Only being able to pause, but not play, makes this functionality rather unreliable and useless.