It's worth noting that there are a few background processes that run when you are uploading initially... if you open Activity Monitor, look for a process called com.apple.photos.VideoConversionService or com.apple.photos.PhotoConversionService and see if it's consuming lots of CPU. This is what is stopping the upload - your Mac is converting your videos to a different format.
I think the reason for this process is it's converting any videos in your Library that won't run in a browser or an iOS device. The process cloudd also appears to be the process that handles the uploading once the conversion is done. Seems that it's one at a time (it won't upload other photos/videos while it's converting one).
Most of this is guesswork, but I'm pretty sure it's correct... I find that sometimes Apple hides information like this to its' detriment. Displaying this info somewhere would stop a lot of frustration. Maybe an Activity Window for Photos? It doesn't have to be visible all the time, maybe only appearing when background tasks are running, such as the initial upload of thousands of photos and videos, especially if there's another process doing something CPU intensive such as video conversion.
I found during the beta that com.apple.photos.PhotoConversionService would sometimes hang... not sure if they've improved this process and ironed out the bugs and better respawning.
Maybe in the future you will have the option to just upload the original video and the com.apple.photos.PhotoConversionService sits on Apple's servers doing all the legwork. One can only hope as this is a bit rubbish - my fans were on constantly and my MacBook Pro heated my bedroom for a few days while it did its' thing.
Good luck anyways, once it's all working it's pretty slick
