Apple TV Remote

I received an Apple TV as a gift. It is set up, and works great, but how do I stop the remote from making changes to my iMac when I use it to operate the Apple TV? Is there a way to unpair the remote from my iMac and my Macbook? Thanks