That's not necessarily correct. At least in my case slides and videos for Stanford CS193P are downloading from deimos3.apple.com.
Annoyingly if I try to download "12. Persistence (November 3, 2011) - SD" from iTunes U on my iPad I get the error message:
Unable to Download Podcast
"12. Persistence (November 3, 2011) - SD" could not be downloaded at this time.
But if I then try from iTunes on my MacBook Pro sitting right beside it on the same network it actually starts downloading. Generally I've found downloads to be more reliable from my MacBook than on my iPad, but they're still amazingly slow (10 kilobytes per second seems the norm, regardless of whose network I'm on at the time).
I have the same problem on my podcast, I can download other but mine won't work. I can successfully donwload it on iTunes for Mac and sync it / play it on iPhone (iOS 5.1.1) but if I try to download it directly from iTunes Music app it will dislay the error.
I'm using feedburner to provide the feed. File is m4a with chapters & artwork hosted on networksolutions domain.
It's weird that it happens to some podcasts and not others. Haven't found anything online yet only other people with the same problem.
Turns out this is a problem for content producers, either their server or their file is not properly set for iPhone streaming.
In the case of my post it turned out to be the embedded artwork which had to be 600x600 in order for the iPhone to stream the file. On the server side you just have to allow streaming of the file type you are sending.
So contact them and let them know, its a problem that can be easily fixed on their side.