Hello everybody,
I don't know if anybody is still visiting this question, but I solved it.
check this out:
1 - go to you photos in icloud.com
2 - on the left you see sidebar that has multiple rows. Row #1 is called photos, row #2 album which is a folder with subfolders = your photo albums
3 - click on the row #2 that and now on the right (where you had your album opened a saw your photos) you should see those albums in an icon view
4 - simply click on any album you wish to delete and then on to top click on the trash can as you would to delete a photo
I worked for me. Hope it will for you.
m8