yes, you're quite correct, Bairn76, the headphone port/jack cannot be used with Toslink on the Apple Silicon Macs
it USED to have a mini Toslink socket buried in there somewhere , on older models of Mac, and a mini Toslink digital optical plug would fit in there, and work quite nicely for digital optical audio/sound output, but that feature has been gone for some time now... you can get a USB or USB-C external sound card that could work.... here are the rough audio specs for your year of Mac Mini (the 2020 one) : Mac mini (M1, 2020) - Technical specifications (CA)
and, alas, as you can see, you get a headphone jack/socket, HDMI socket/port that supports multichannel audio out and that's about it...it also has two USB-A ports, 2 Thunderbolt 3 USB4 ports which support USB 3.1 Gen 2 and Thunderbolt 3.... so that's USB-C..... those are your options.... I'd go with a USB-A external sound card which supports Macs and offers Optical digital audio out of some kind, analog output, either via 3.5 mm headphone socket/jack or 2 RCA sockets or jacks.... This Amazon US link has something that should fit the bill well enough:
https://www.amazon.com/ASHATA-Converter-Digital-Windows-Windows2000/dp/B07Z3TQKFF/ref=sr_1_20?crid=2I3BSHPB58QP3&keywords=USB+external+sound+card&qid=1696110969&sprefix=usb+external+sound+card%2Caps%2C177&sr=8-20
copy and paste the whole link (the URL) into a new browser window and it should open in Amazon's website, says it works with Windows and Mac OS X OK, at least for stereo use.....
John B