In Photos for Mac it should look like this, when you ctrl-click "my Albums" in the sidebar (sorry, my Mac is currently set to use German as the primary language):
Photos at www.icloud.com is more limited - it is more an emergency access to your photos, when you are not able to use the Photos.app on one of your Apple devices. To make it easier to find your photos at on your iCloud webpage, you may want to structure your list of albums with folders and subfolders. if each subfolder is holding only a few albums, you can find them at a glance, without having to scroll a long list of albums. This way you can easily manage thousands of albums, even at www.icloud.com
For example - I have grouped my travel albums in folder for ten years, subfolders for each year, and albums for each major event in each yearly folder: It takes just a few clicks to drill down trough the folders to the album I want.