Does a large Photos Library make Photos slow? Should we split the library?

The Pros and Cons of a large Photos Library

We know, that Photos should be able to handle at least a library with one million photos and videos, because it can migrate Aperture Libraries, and one million photos has been the limit for Aperture. It is tempting to let the library grow to find all photos and videos in one place. The library size is a trade-off between between making it possible to manage the library files and maintaining the photo libraries and between working efficiently within one library and finding the items we need for our projects together in one library.

• Pro: A large library with all photos in one place is how Photos should be used. Working with Photos and accessing our photos is most powerful, when we are finding all photos we need for our projects in the same library. We cannot search across all libraries at once. We have to open them in turn.
• Contra: But a large library makes it harder to manage the Photo Library packages and the background processing will take longer. The larger the library, the more it will try our patience, when we have to repair, upload it to iCloud, upgrade it, or back it up.

The library size is a trade-off between working efficiently within one library and finding the items we need for our projects together in one library and between making it possible to manage the Photo library packages and maintaining the photo libraries.

So, is its good idea to let a Photos Library get larger and larger, or should we split it into smaller, partial libraries?

As far as I can tell, the size of Photos Library matters, for several reasons:

• The larger the library, the harder it will be to maintain it:  Updating the library, repairing the library, backing it up, moving it to a new drive are taking longer and longer, the larger the library is. Even for a moderately sized library like my library with just 250GB the first Time Machine backup took more than a day. The upload to iCloud Photos took ten days. For larger libraries we can expect that upgrading the library or repairing it may take more than a day.
• The background processes after an upgrade will take considerably longer for a large library. 
• And we will need more additional free disk storage as working storage, when repairing the library or syncing it with iCloud. 
• Processes searching the whole library may take longer, for example the updating of smart albums.
• Some parts of the user interface in Photos have not been designed with a large library in mind. The People album cannot be sorted automatically or structured in any way, and with nearly thousand named people in the People album it is barely usable. The list of keywords is just a flat list, and the more keywords we define, the harder it will get to use them.
• If we are using iCloud Photos and want to sync the library across all our devices, the device with the least storage will be the bottleneck for the library size. Even with "optimise iPhone storage" we cannot sync a library of a 1 TB to an iPhone with 32GB of storage.

I would not let a Photos library become larger than it needs to be, to make it easier to maintain it. If we find that the library is too large for the volume it is on or repairing it takes too long, we should split it. But we have to plan the split according to our workflow.

We need all photos and videos  together in one library, that we need to use together. If we can split the library thematically into separate sets of related items, we should do it, for example separate private photos from work related photos.

I have noticed, that I only need my favourites and the photos I am currently working with on all devices, so I created a System Photos Library with my favourites and only the last imports. This library is syncing with iCloud to all devices.

Additionally, I have archive libraries with all photos for each shooting trip or vacation. These archive libraries are easy to back up and manage, because they are small.

Other possible reasons for making Photos slow

If Photos is slow, the library is not necessarily the reason for the slowness. There are several possible reasons why Photos may be slow, when you open your Photos Library.

• If you just recently upgraded to Catalina Photos may still be upgrading the library. This can take a few weeks until Photos is back to normal for a very large library.

• You may still have legacy media on your Photos Library, that Photos can no longer handle after the upgrade. Some codecs have been deprecated by the Catalina upgrade and Photos can no longer process some video formats or image formats that used to be supported by the earlier system versions. For example, you should remove all PDF files from the library. (How to Weed out Legacy Media in Photos for Mac - Apple Community)

• If your photos are in a RAW format, photos may be slow, if the RAW processing has changed. My Mac is currently slow as molasses, when I try to edit the DNG files in my library, because Photos is reprocessing them with the updated RAW support.

• Photos needs a lot of free storage in addition to the size of the Photos Library itself. The storage needs to be free, not just available.  You may want to check with Disk Utility, as the Finder is only showing "available" storage, not the unused storage. How much storage is free on your system volume and the volume with your Photos Library? On my Mac with the least storage Photos is unbearably slow, if the free storage is dropping below 40GB, even if the Finder is showing 100GB available, but it looks like the system will not purge the storage to free it immediately, if needed.  (This will probably depend on the size of the library. My library has currently 55000 photos and 300 videos). Most of the "available" storage may need purging, and that does not always seem to work. 
• Photos may be a bit slower, if your image files are huge, for example large RAW files or TIFF files. When I am working with huge TIFFs, larger than 100 MB, Photos is beach balling a lot.
• Photos may be slow, if you have hundred of smart albums, that need to scan the whole library for updating. In a large library I would be frugal with smart albums and delete the smart albums I do not need continually.
• Are you using iCloud Photos? If Apple is updating the Cloud Serves it can sometimes take very long to download the originals from Apple's cloud servers.

Photos can handle really large libraries, but we must not let the library grow indefinitely. The library size matters, when we have to move a library, to back it up, to copy it, to restore it, to repair it, to sync it again with iCloud. If we are using a portable computer, that we need to carry with us, it would be painful to have to wait several days for a library repair to finish, or a Time Machine backup to finish, before we can unplug the Mac to carry it with us. It depends on our patience, how long we are willing to wait, when we need to manage the library.

In addition to trying our patience the size of the volume, where we are storing the library is a limiting factor. I would always use a volume, with enough free storage to hold an additional copy of the library, so Photos will have additional storage when rebuilding or upgrading the library.