I think we are confused by your saying " upload the Albums (directly as files) on iCloud.com." This conflates a couple of different ideas. What's confusing is that iCloud has lots of parts. There is a Drive that acts like a internet external drive with folders and files like are stored on our Macs with the Finder. If that is what you mean, then you would need to File>Export the pictures to new folders you create in the iCloud Drive folder listed in the Finder sidebar. It would be easier to use a program like PowerPhotos to export albums for you-- it can export entires folders and keep the album structure. (This will take time, maybe days, to finish.)
The app Photos doesn't usually work with "files" in the Finder sense-- It stores the pictures with crazy seemingly meaningless names in a user not-safely-accessible "package," and uses a clever database to keep track of it all. That's why when you use Photos, you use the Export dialog rather than finder-like dragging or copying.
Another option is to use Preferences>iCloud to enable iCloud in Photos (with only one Mac Library, which you choose by identifying it in Preferences as your System Library, as léonie has said.) Then that entire library is copied to the Photos app in iCloud.com with exactly the same structure. This isn't a backup, it's a synchronization. If you delete a picture on you Mac, it will be deleted in iCloud. iCloud Photos will also automatically copy pictures from your iPhone and iPad if you turn on iCloud for Photos on those devices. Backups are kept on external drives (not iCloud) using Time Machine or Carbon Copy Cloner or some other reliable backup system.
Another option, which is very rare, is to have Photos refer to picture Files in Finder folders rather than keeping them in its own package. It's very easy for Photos to lose track of the pictures this way.
So you have to be sure which of these things you want. We can help if you need it.