You have to identify the "bad Apples" in your library, that cannot be processed by Photos and are causing the processes to waste time on trying to process them over and over again.
Have you checked below the "All Photos" album in the status bar, if there are items mentioned "On this Mac only" or "Unable to upload"? If yes, you can find these items with smart albums as described here: Check the status of your library in Photos for macOS - Apple Support
If the status bar does not show any special items, you are on your own.
Then you have to search systematically for videos or photos, that cannot be processed.
Which system version is running on your Mac? Starting with macOS 10.15 Catalina some image formats and video codecs are no longer supported by the newer system versions and older videos or image files in an unsupported format can cause photos to hang.
I would search systematically with smart albums for photos and videos in an unusual format by searching for the file name extension as described here: Script: Find Photos not in Any Album, for Photos 5 on Catalina
Then test the suspicious items by trying to import them into a new, empty library. Any item, that is giving an error message when trying to import it to a new, empty library needs to be converted to a more compatible format.
In addition to incompatible codecs the background processes could get stuck by photos with missing originals or damaged files or scanned or shared photos with incompatible color sync profiles embedded. Check any photo you downloaded from the web, if it is just a link and not a full image file.
As my library is an iCloud Photos Library and all photos are in iCloud, I am usually repairing it by downloading a fresh copy from iCloud to a new, empty library. This will ensure, that only photos compatible with the current system version will be downloaded from iCloud and my new library will usually be shipshape.
