I don't have the same TV and haven't experienced such an issue, but have you tried finding the best TV Settings for the ATV4K's 4K HDR mode and sticking with that (with ATV4K's Match Content set to Off)?
Assuming your TV's firmware is up-to-date and that all of its spiffy HDR-related and HDMI-CEC features are activated for the HDMI port the ATV4K is connected to, I would then go to the connected ATV4K and try the following:
- Settings > Video and Audio > VIDEO: Reset Video Settings
- Turn on HDR and verify Video Format = 4K HDR
- Test any improved Gamma suggestions and go with them if you can.
- Leave Match Content = Off (for now, at least).
From there, go to your TV's settings and turn off the Samsung's various video processing "enhancement" options off (sharpening, smoothness, etc.). These may cause a sort of double-jeopardy by processing the ATV4K's processed video stream. Then select your favorite Samsung picture mode (vivid, standard, movie, etc.) that's available while the Apple TV is displaying 4K HDR. Go back to the ATV4K Settings and do the color balance calibration process with your iPhone as a last step.
If you're like me, you may find that just sticking with the 4K HDR format looks fine for most if not all content you play through the Apple TV device. You've essentially upgraded the "Smart TV" aspect of your Samsung for a more current ATV4K with a more powerful chip. Let it do the heavy lifting in video processing. As a bonus, you wouldn't have to watch your TV go momentarily blank while switching video dynamic range and frame rates all the time.
Hopefully this approach will look good and bypass the question/issue you started with :-).