See the last post here by Musicfreak...
The hardware supports 24/192, but the operating system does not. The best you can do is optical out on a Mac at 24/96.
The way around this is by using a USB>SPDIF converter that comes with its own drivers. I use the M2Tech HiFace and can get up to 24/192 out of my Mac Mini.
And a related 24/192 thread...
These guys are 100% correct and Apple's documentation is misleading at best. The DAC chip is completely capable of 24/192 but the optical output is limited under Mac OS X to 24/96. I have booted Windows Vista Ultimate 64 bit a couple years ago and output 24/192 via the Mac Pro optical output, but this means nothing for any other piece of Mac hardware.
The hiFace will output up to 24/192 guaranteed. Here is a link to my review of the piece.
2011 Mini Which I think has the same DAC...
If you are using the optical out you aren't using the internal DAC at all. That would only come into play with the analog output. Due to cost constraints the analog bits after the DAC are likely not as good as a modestly priced USB or Firewire DAC. That is only a guess, as i've not had the opportunity to evaluate a MacMini (i have Dual G5 tower into Firewire DAC running 192kHz)
96 kHz is a limit of optical out of anything, and not a fault of Apple's.