Photos as my screen saver on my iMac

It used to be possible to do that, but this was changed some years ago. The moment you press an arrow key you are out of the screensaver. I don't think there is a way to change that, unless Apple chooses to.

However, why not just watching the same photos doing a slideshow in Photos?

Manually advancing photo screensaver? - Apple Community

how to move to next picture in screen sav… - Apple Community

Consider GraphicConverter it has a powerful Browse Folder function, & many tools for manipulation, & a SlideShow Function that does what you want...

https://www.lemkesoft.de/en/products/graphicconverter/

http://www.lemkesoft.com/