Is there a way to select which photos are shown in stacks?

The stack widget keeps showing the same few photos which it has apparently chosen at random. None of the photos is from my favourites album, and several are photos I don’t look at often, so it’s unclear why they have been chosen. Is there a way to change that, and select what I want to see?