Two Apple TVs - remote interference?

Yes. Each remote will be paired with a specific Apple TV so there will be no interference.

I thought 4th gen Apple TV remotes use Bluetooth to communicate with the Apple TV unit and IR signals to control other devices such as volume on a TV?

There are some possible mixes of models that may cause unintended reactions: An Apple TV (4th generation or later) may react to the IR signals from an unpaired old style Apple Remote (aluminum). IR requires line-of-sight, so that should be easily avoidable.

Yes, but the older remotes can also control it by IR.

To stop this...

1. Take the unpaired silver remote (if the remote is not unpaired, press menu and rewind for 6 seconds while pointing at the paired device to unpair).
2. Make sure the silver remote can't control the other device (turn it round, cover it up or turn it off).
3. Hold it close to the Apple TV 4 and press the menu and rewind buttons for at least 6 seconds (you won't see anything happen while you do this)
4. Then, hold menu and select (centre button) until the light on the Apple TV 4 flashes.