Can the Big Sur OS DVD player play bluray disks?

It will only playback a commercial BluRay DVD with Third Party software. I prefer Blu-Ray Player Pro from MacGo. It allows full playback with full menu support. It's essentially the BluRay player that Apple should have built with the exception that it requires internet connection to initially retrieve the decryption Keys. It works flawlessly. There are "Free" hacks out there to just watch the DVD with VLC and other stuff, but I like having the full DVD experience with menus etc. You don't need any software to browse BluRay DVD's that are used for storing files. Finder will do this natively assuming you have a hardware BluRay device. To be fair Windows removed the ability to even decode regular DVD's natively in Windows 10. It comes down to licensing. It's expensive and MacGo side steps this by not installing all the keys with the software. You must rely on third party help on both platforms now.

I just set up a Mac Min (intel 2018 config not M1) and purchased a third party BlueRay player - A PIAEK Super Slim Drive. I'm also running MacOS Big Sur 11.0.1 (which given the 6 hours I have had with this OS I would avoid it at all costs).

I have put in a couple hours trying to get ANY DVD to play in it and have not had consistent success.

a) DVD Player (which does not reside in the Application folder but IS hidden in the System\Library\Coreservices\Application folder plays a Standard definition DVD unreliably (hangs, etc) but won't open a blueray drive.

b) "Final Video Player" app, claims it will play DVDs but does nothing.

c) "5KPlayer" app has a icon to launch DVDs but does nothing, even with a standard DVD in the drive.

I am still hunting for an app that works and am pretty frustrated.

So, I tried this on an M1 machine where half the apps are running under Rosetta2, but VLC and MakeMKV combined should get you going. MakeMKV has a VLC integration option, so that theoretically should work. I tend to just create a digital safety copy of my DVDs and BluRays, then throw those on Plex either as is, or converted with Handbrake to a smaller format (48 gigs to around 7) instead of playing the optical media all the time. Might wanna give that a go.

There is no Blu-ray support in macOS. Never has been. No Apple devices support Blu-ray discs. It is an all third-party device and software solution, providing one first confirms through the third-party specs, that the device is supported on current releases of macOS. Some devices are dependent on custom vendor Windows drivers for proper functionality and are paperweights on a Mac.

Software such as current releases of VLC and MacGo's Blu-ray Player Pro do well with a compatible Blu-ray device.

The Pioneer BDR-X12UBK states that it is compatible with El Capitan (10.11) thru Catalina (10.15). One would have to contact Pioneer as to whether is is compatible with Big Sur (11.1).

Ordinarily, I have a Blu-ray writer here, but it is on a road trip to Florida with a friend and I won't see it again until Spring. So no means to test here either, and I have no plans to purchase another Blu-ray writer.

Yep.

You need a BluRay device. DVD Player is just that. SD video.

We are asking about the software not hardware. If you have a BluRay player, will it play on a Mac or do you need additional 3rd party software?

I don't have a BluRay drive so I can't test it. But VLC is my goto media player on Mac, Windows and Linux. Just drop the disc on the app and it should start playing.