AirPods Pro stop noise cancellation when using Zoom on Mac

I've found that when I join a Zoom meeting on my Mac, my AirPods Pro 2 gen switch from noise cancellation mode to some mode that doesn't fully block surrounding noise. This mode isn't quite the same as transparency, and it isn't the same as noise cancellation off, it's somewhere in the middle where some noise cancellation is happening, but some level of transparency is also happening.

When this happens I can correct the problem by going to the AirPods settings on my Mac, switching them to noise cancellation off, and then switching them back to noise cancellation. It is annoying to need to do this every time I join a Zoom meeting, though.

I've also noticed that if I join one Zoom meeting, go through the settings switch described above, then leave one meeting and join a new meeting without removing my AirPods, then this problem doesn't happen.

I've also tried using other apps with my AirPods, like Music, and this problem does not happen in that case, just with Zoom.

Has anyone else experienced this? Has anyone found a solution?