Using a single 4K display (Samsung Odyssey G7) connected to an Nvidia RTX 3090 graphics card I was getting an immediate black screen when trying to navigate the app. System was non-responsive and had to hard reboot. I was having this issue with both the Apple Music app as well as the Apple Music Web Player running in Chrome. For me this was the clue that something about how the music is playing is crashing the graphics driver as the Web Player in Chrome would crash only once I started playing a song. Black screen but the music continued to play - so WinOS was still functional.
I went in, updated graphics driver and switched my monitor to refresh rate of 120Hz from the original 60Hz setting and this solved the problem. Now everything works fine.