Adjust Date & Time: buggy? It's not consistently sticking

I regularly import photos into my iCloud account from a third-party app (I'm scanning some legacy family photos). Once they get into my iCloud account, they always inherit today's date, which makes sense.

I then go into macOS's Photos app, select them all, and use Image > Adjust Date and Time to batch-change their dates to the approximately correct date that the photos are from. This is so that my master Photos library can be in sequential date order.

What I'm finding is that, as soon as I use Adjust Date and Time, the selected photos do indeed pop back into the past. But, an hour or two later, sometimes they reappear at the bottom, with today's date. And no matter how many times I use Adjust Date and Time, sometimes it "sticks" and frequently it doesn't. The interval varies -- it can be an hour, it can be several hours, it can be next day, but sometimes photos jump BACK from the past to the present.

Has anyone else seen this bug? Or have any advice on fixing it once and for all? Ultimately, what I want to do is to be able to select photos, adjust their date to a date in the past, click OK, and have it work and stick forever.

I am running Sequoia 15.1.1 (the latest).