Safari video playback speed bug

Occasionally, I will experience this weird bug(?) in Safari 13.0.5 where all video playback speed is completely faulty: Regardless of the length of the video, or whether it's streaming or "pre-loaded" like on YouTube, it will play from the beginning of the video to end of the video in under a second or less and I will only see a couple of frames of the video.

I don't ever mess with the playback speed buttons on any online video players and going in and changing them when I get this bug doesn't slow down the video either. Closing the tab with the video or quitting and reopening Safari doesn't fix the problem.

Only way to fix it is to restart the Mac entirely.

I have a hunch that it's caused by me switching between devices with my AirPods (1. gen) since I'm assuming there's some sort of "delay compensation" going on to make the audio in the video sync with the AirPods. But it there doesn't seem to be a pattern to what triggers it. Some days it will happen 1-2 times, other days not at all, and I'll have weeks where it doesn't occur even though I'm switching back and fort between devices several times per day.

Does anyone know what causes this and if there is a fix?

The Mac in question is a 2018 Mac Mini running 10.15.3