iOS 18.0 with model A2843 AppleTV

Verify that the Apple TV box and the smart speakers are assigned to the same home and same room for HomeKit, and have the same Apple ID account for iCloud services.

Home app - App Store preview - Apple

Set up home theater audio with HomePod or HomePod mini and Apple TV 4K - Apple Support

HomePod “or” HomePod mini. I want to use both but can’t.

One kind (set) can be used as default speaker (set). More AirPlay speakers may be used for multi-room use, but not for default speaker use.

It also defaults to the system sound (tv speakers) which I do not want.

A set of HomePod speakers (that is set to the same room) can be made default, instead of TV speakers, by changing the Setting for Audio Output on Apple TV 4K.

AppleTV should be able to default to HomePods if available.

Yes. That is the intention of the second link. But that is for one or two (stereo pair) of the same kind.

Please contact Apple Support if your unit can’t be set accordingly.