macbook not waking up without external device

Usually I use macbook with external monitors. Sometimes, when I disconnect it from external monitor, and open it up later without any external device connected, it just won't wake up. Screen is black, keyboard is dead, touchbar is dead, power button is not working.

If I plug in power adapter, it makes "charging" sound. Sometimes I even get chat sound notification so apparently, mac is not totally asleep. Only thing that can wake it up in that case, is connecting it to external device (usually external monitor, in rare cases plugging in USB drive works). If I am away, I can only wait for it to lose all power and shuts down completely, until I can use it again.

I use MacBook Pro (13-inch, 2020) OS Big sur, but it happened in High sierra too.