Hi
Your drive is formatted ExFAT - this is not compatible with a photos library, and can result in gradual build up of errors in the database causing just such issues as this.
You are going to have to move the library to a volume with a compatible format.
1 - Backup your existing library. You are going to be copying the whole thing at least twice, and then possibly having a repair run. As a large data structure this will take some time, with a risk of data loss while it is ongoing.
2 - You will need to temporarily copy your library somewhere else WITH A COMPATIBLE FORMAT. This could be your system drive if you have enough space. Or a second external drive (USB or Thunderbolt), which is formatted APFS or MacOS extended (journaled). This might mean you need to buy (or borrow) an additional external drive. If so, you could use the new drive to run the library in the future, saving yourself one pass of copying the library.
3 - Reformat the external drive using disk utility. Again APFS (suggested for SSD) or MacOS Extended (journaled) (preferred for spinning drives). Do not use Case Sensitive. Be especially carful to choose the correct drive to format. Formatting completely and un-recoverably erases the contents, so you don't want to format the wrong one. Ideally unplug all other external drives, and make sure the drive you are formatting is in the external section of disk utility.
4 - Copy the library from the temporary storage to the newly formatted external drive. (You can skip this if you are going to leave it on the new external drive)
5 - Open the library on it's new home by holding the option key while starting photos. Go to photos preferences, and click the "use as system photo library" button.
Read also:
Move your Photos library to save space on your Mac – Apple Support (UK)
