Answering my own question — after chatting with Apple support.
First off: This appears to be a technical glitch that affects some iPad users (since the last upgrade I assume, since I didn't experience this issue before, after any upgrades). Engineering reportedly is aware of this.
To fix:
- On your iPad, sign out of iCloud (Settings > Apple ID (scroll down) > Sign Out
- Restart your iPad
- Sign in to iCloud again (you will get several messages along the lines of This iPad will be using xyz now)
- Sign in to iTunes Store (do this first)
- Check TV app: the movies start appearing now
Note: Before signing in to iTunes Store, I checked my Books library to see if everything has populated there; books started populating, but I was signed out of the Book Store, and I wasn't not able to sign in through the Books app; only after I signed into the iTunes Store was I also signed into the Book Store; a bit weird UX...)