Photos exits from Album view back to Library when browsing or deleting photos

My Photos library contains about 44,000 images ranging from very old tiny resolution JPEGs to phones to RAW images off high end modern cameras (but no videos, I keep those elsewhere).

Lately, I find that when I am browsing through an album using the arrow keys on the keyboard, and using either Delete or Cmd + Delete to trim out the shots I don't want, my iMac slows down and then Photos eventually stops responding and exits from the Album view back to the Library view. This happens often, sometimes after viewing or deleting just a few images. The delete process also seems to take much longer than it used to.

Is there some sort of maintenance I need to run on my library, could something be confused in its internal metadata somewhere? I've had this library since the very earliest days of iPhoto, many machines (and versions of Mac OS) ago. I can't imagine such simple operations as "next" and "delete" should typically take so long.

I'm currently running Photos version 3.0 (3291.13.210) on a late 2013 iMac with High Sierra (10.13.6). I am not using iCloud Photo Library, My Photo Stream, or iCloud Photo Sharing.