Mac Photos App Smart Album Not Working - Album, Is Not, Any

For when I am organising my photos I have created a smart album that has the following filter `Album` > `is not` > 'Any`. I then know these ones need to be moved into an Album.

However I went through some albums removing photos (using `Remove 1 photo from album`) from them expecting them to reappear in the smart folder where I could reorganise them but they don't appear.

I have been able to locate some within the all photos view and I have run an apple script... (How to find out the albums a photo is in: Jacques Rioux script, Version 2019) on them to confirm they are not in an album, yet they just won't appear in the smart album. Any ideas on where to go from here...