Hello gleembru,
Do you have it set up to shuffle photos or are you saying you only wanted it at a specific photo?
Either way, verify the settings in System Preferences > Desktop & Screensaver. Make sure Change Picture: option is unchecked or if you want them to change photos but not at random, uncheck 'Random order.'
Also, if you mean they are rotating the photos at 90 degree, 180 degrees, etc. Check to make sure the photos aren't also rotated in the Photos app or folder that is being referenced.
If the settings still aren't working, restart your Mac and try again.
Thanks and have a great day.