Sounds in both instances as though the AppleTV Remote is issuing identical signals to the ones the other devices use for volume.
I would suggest using the Remote app on an iPhone/iPad to control AppleTV , unless you can place AppleTV distant from the TV so that you have to point the Remote in a different part of the room to control it.
AppleTV Remotes send some kind of 'identity code' which is how Apple devices can be paired to different Apple remotes eg pair a Mac with one remote, AppleTV with another to prevent the remotes affectin each other.
Each remote you have is likely to have a different ID code but the number of codes may be finite so not impossible to have two the same. For this reason Apple has provided a way to change the UID of a Remote:
You might want to change the UID if you're setting up more than one Apple TV in a room, or if you're using a non-Apple remote. To change the UID, hold Menu and Select for six seconds.
Using your Apple remote for Apple TV - Apple Support
You could try that - if the existing UID of the Apple Remote forms part of the other devices remote code, changing the Apple Remote UID might solve the issue.