Apple TV Remote

Yes, depending on TV remote model it is either rechargeable or battery replaceable.

Read here => Identify your Apple TV remote - Apple Support

It doesn't take long to charge, it depends on how much charge is left.

Note: You can use the remote while charging.

For the Siri Remote: Charge your Siri Remote - Apple Support

For the Apple Remote (aluminum): How to replace the battery in your Apple Remote - Apple Support