Your Mac Library from the Apple TV app there, can be made available on the Apple TV box through Home Sharing, and the Computers app. Is that what you were doing? That would not be using iCloud. Your home videos do not get uploaded anywhere just from adding it to your Library. Home Sharing requires that the computer is running and sharing the library. On older Macs (with iTunes), Home Sharing is configured a bit differently. Any wait time could come from waking the computer from sleep mode, perhaps? Home Sharing uses your home network only, so your ISP plan is irrelevant for this part.
Use Home Sharing to share media from a computer to other devices - Apple Support
Set up Media Sharing/Home Sharing on Mac - Apple Support
Home videos should be in .mp4, .mov, or .m4v format, with H.264 AVC or H.265 HEVC video and AAC (or AC3) audio. For best performance, media files can be “optimized” by putting the metadata at the start of the file, instead of normally at the end of the file, as well as interleaving streams by time chunks. While optimal, it is not strictly needed for modern equipment and OSes, as the computer can read the media file plenty fast enough normally with fast chips and SSD disks.
If you want to try for the effects of optimizing, then you can use e.g. Subler. Just open a file and then save as «new name», with Optimize checked. Then (re-)import the resulting file into your library. You may want to remove the previous version first from the library.
Alternatively, your home videos in the Photos app can be made available on the Apple TV box through iCloud Photo Library, and the Photos app on the Apple TV box.
Set up and use iCloud Photos - Apple Support
Use iCloud Photos on Mac - Apple Support
View your photos and videos on Apple TV - Apple Support
Alternatively, there are other third party solutions to sharing content from a computer (or NAS) to your Apple TV box.
Plex, Infuse, VLC, etc.
