I'm a little bit surprised when you say "almost all users".
Please check the AVSforum website and you'll see how many times this issue has been brought up which is really frustrating.
John Archer from Forbes magazine has also informed Apple of the issue officially and after soooo many updates to ATV 4K, this issue is still there.
Apple IS NOT working fine with 24 fps materials, it converts 24 fps to 23.976 fps.
Please read Apple TV 4K related forums more often or please read the emails directly sent to apple regarding this issue. As an example, I put a link for you to just one of those forums where you can see many people complaining about this issue dating back to 2018 and Apple kept ignoring the issue. As of right now Apple TV 4K is a defective product since it is not able to output correct frame per second of Netflix and Amazon series and movies which are made in 24 fps.
The problem is that Apple TV 4K does not support 24 fps, but it does 23.976 fps. But that doesn't solve the problem with 24 fps movies and shows.
https://www.avsforum.com/threads/apple-tv-4k-lacks-support-for-24-000-video.2961262/