Is your system version macOS 10.15 Catalina?
The menus have been changed in Photos 5 on Catalina. Previously the "Image" menu has been showing an item "Delete ... photo ⌘⌫" when we select a photo in an album. Now the menu is only showing "Remove .. from album ⌫". The same goes for the contextual menus when we use the secondary click on a photo.
But the menus can be changed by holding down the command key ⌘. Holding down ⌘will change the "Remove .. from album ⌫" entry in the menus to "Delete ... photo ⌘⌫". That is probably, why the key combination is working ⌘⌫ is working right away - as soon as we touch ⌘ the menus are changing.