I would like to replace the apple tv3 with my new apple tv 4 that just arrived. what should I do?.

You don't need to install any software to get it connected and set up. Once you have it running, you can download additional apps, if you'd like.

Links to user manual for Apple TV 4K:

https://support.apple.com/guide/tv/welcome/tvos

https://books.apple.com/us/book/apple-tv-user-guide/id1475571643

It's always a good idea to make sure your old TV is update first to the latest software. If you haven't updated in awhile, there might be some new features or services you were missing out on. After that, yes you can switch out the pucks and set up the new one using your same AppleID that you were logged in on the old device. This should transfer and make available any content that you purchased previously. Here is a support article with more information: https://support.apple.com/en-us/HT205358

It is also recommended that you sign out of the old device, and restore it to factory settings. This way, any new recipient will not have access to your login. https://support.apple.com/en-us/HT202118