MacBook screen off after Vision Pro Virtual Display ends

I'm having an issue that after I end a Virtual Display Session to my MacBook when I open the MacBook to use normally, the screen remains off as if it's still in Virtual Display mode

I normally use Virtual Display in clamshell mode and plugged into a power source.

Vision Pro battery is also connected to a power source.

The issue occurs very frequently, but not every single time.

I can return the MacBook to normal by restarting it,

but I'd rather not have to restart every time if there's a way around it.

Apple Vision Pro M5 version

MacBook Pro 14" with M3 Max