I was attracted to have my Apple TV hooked as a music source to my DAC since I bought my first Apple TV (3rd version). The idea to keep the setup simple was tempting. Unfortunately this 48 kHz upsampling thing is a little bit disappointing.
By the way my current Apple TV 4th, as you know, comes without TOSLINK port, so I've to use an audio de-embedder to extract digital audio signal. For video it's perfect, nothing to complain.
For audio only use ... well, the sound isn't bad actually – still, you know, audio enthusiasts are always picky... ;-)
Eventually I found a sweet spot adding a device: an audio network streamer, ROON compatible and with AirPlay. My iTunes library is shared with the ROON server running on my Mac.
When I want to listen regular stuff like Apple Music, I simply AirPlay to my streamer: 44.1 kHz, no upsampling – good.
When I want to listen my local hi-res files I send them, again, to my streamer via ROON: bit-perfect audio – good.
When I want to listen audio from Apple TV Movies/TV, I get the audio from the HDMI-TOSLINK de-embedder – good.
Speaking of hearing: I'm 42 and can't hear some high frequencies anymore. It's sad. Anyway, I know this only because I listened test signals with my headphones. I still enjoy listening regular music which is made by fundamentals and harmonics, not only cold test frequencies.
Don't worry about "natural deafness": it's quite normal, just enjoy music. :-)
Speaking of cables: I've learned that spending too much in cables makes little sense. It's fine to have good cables instead of cheap/flimsy cables. There's a threshold above which the price/performance shrinks. Just my opinion, of course.
I agree: nowadays people are happy to listen music through earbuds and don't care for more sound quality. That's probably the reason why Apple hasn't stepped in the hi-res field (yet, I add, because I hope something will change in this regard).