It isn't all that slow and should allow you to watch an HD movie with only a 30-40 minute wait.
There could be a few things that might affect you, some ISP's filter certain content or slow your connection speed down at certain times of the day. You may also have a problem on your local network, do you have any problems streaming movies from iTunes on your computer to the Apple TV.
The problem you encounter on the Apple TV when you need to download it again is because the Apple TV only has volatile memory, if you turn it off or do something else with it, you will lose anything you have downloaded.
You have several options that might help:
- You could switch to SD movies (settings > general > iTunes Store)
- You could download to a computer and stream from there, downloads to a computer wont disappear.
- You could investigate why it's taking up to 10 hours instead of 30-40 minutes.