MacOS Monterey M1 Mac mini stores iPhone & iPad backups, audiobooks and podcasts on the limited internal storage.

Locate backups of your iPhone, iPad, and iPod touch

From that same link above - This implies the Backups can be relocated to External Storage

• Don't edit, relocate, rename, or extract content from your backup files. For example, to restore from a relocated backup file, you need to move the backup file back to where it was in the Backup folder or the file won't work. Backups serve as a safe copy of almost all of your device's data and settings. So while you might be able to view or access a backup file's contents, the contents usually aren't stored in a format that you can read.

What is “Other” storage on a Mac, and how can I clean it out?

Free up storage space on your Mac

OmniDiskSweeper Safe to use

GrandPerspective 

How to delete Time Machine snapshots on your Mac

See used and available storage space on your Mac

Locate backups of your iPhone, iPad, and iPod touch

The final word from Apple on Managing the " Other " Category

Other: Contains files that don’t fall into the categories listed here. This category primarily includes files and data used by the system, such as log files, caches, VM files, and other runtime system resources. Also included are temporary files, fonts, app support files, and plug-ins. You can't manage the contents of this category. The contents are managed by macOS, and the category varies in size depending on the current state of your Mac.

Have you tried an external HDD attached via USB-A or thunderbolt/USB-C and access with Finder? I've never had a problem with that on any Mac we've ever owned. We've used HDDs and SSDs and depending on the transfers speeds you want both have worked extremely well.

Any file or folder you have on the internal drive can be placed on the external drive. just don't make the mistake of using your Time Machine backup drive. It needs to be dedicated to TM and nothing else.

The Original Aspect of your question " Is there a way to store these things on external HDDs without symlinks that are unreliable? Music and Photos can easily designated other storage locations, why not the others? " to which a reliable and accurate reply was offered.

IMHO, there has been enough suggestions / advise offered for the User ( you ) to make an informed choice as to the next course of action that is needed to possibly fix this computer issue.

Understand, the advise offered was and is in the best interests of the computer.

All advise / suggestions are done on a volunteer basis and in good faith.