Welcome to the Apple Community.
Yes I would suspect your network is the problem here, however I doubt it just requires a replacement base station. If the base station is simply too far away replacing it is not going to help, likewise if you are experiencing interference a new base station would also be subject to the same interference.
Interference can be caused by other networks in the neighbourhood or from household electrical items.
You can download and install iStumbler (NetStumbler for windows users) to help you see which channels are used by neighbouring networks so that you can avoid them, but iStumbler will not see household items.
Refer to your router manual for instructions on changing your wifi channel.
There are other types of problems that can affect networks, but this is by far the most common, hence worth mentioning first.
I am trying to watch Margin Call and the first 34 minutes took 15 minutes to download. I am now stalled at that point waiting for the rest or perhaps just more of the movie to download. Our wireless is 300 mps which is the fastest I can find. If this is the way it always will be then I won't be a big user of Apple Tv