Photos Library size increases with switch to iCloud Photos

As Matti Haveri says, the format of the drive is critical. f this drive is in a an incompatible format, stop running Photos with it immediately!  A Photos Library can sit on an incompatible drive, but running it may corrupt the database.

A Photos Library is a dynamic thing-- it doesn't just hold a bunch of picture files. My "Favorites" Library that I synchronize with iCloud Photos says it's 120GB at iCloud.com, but it's 150 GB on my Mac. It's 142 GB on my iPad.

Photos generates thumbnail versions and smaller preview versions, sometimes in more than one size, so that we can flip through our pictures quickly without having to load each full sized image as we go. These extra versions may be removed if we haven't looked at them for a while, and more can be generated as we scan through. So the size of the Library will get larger and get smaller. Photos needs to have enough space on the volume its Library lives on to accommodate all this background activity.

When iCloud sees a new Library it has to do lots of work to make sure it's synchronized. A new OS also requires a rebuilding of the Library database and re-scanning. It could take a week or more for things to settle down. How long has it been?

jsunem wrote:

The photo library file is on an external hard drive with plenty of storage

What format is the external disk? It must be APFS or Mac OS Extended, non-Mac formats like exFAT or cloud storage are not supported.

"but I've read somewhere that Photos libraries can be come unwieldily or even possibly unstable when they get too large,"

Not necessarily. It is just a trade-off between the convenience of having all photos in one place and and the inconvenience of having to wait longer, when Photos is indexing the library or repairing it or syncing it again with iCloud.

Usually Photos is fast, even for a large Photos Library, because it is investing a lot of work in indexing the library and caching data for quick access. But when Photos needs to scan the library again and index the items after a system upgrade, or when it needs to sync a library again with iCloud Photos after it had to repair the library, you will have to wait a long time, if the library is large. Particularly the amount of maintenance work is growing with the size of the library, but this will annoy us mostly after major system upgrades. In between we will have only to be patient when backing up the library or copying it.

My strategy: Avoid repairing the library or turning off iCloud syncing. Be frugal when creating smart albums. Updating a smart album will depend on all photos in the library. And when you edit, delete or add items all smart albums will need to be updated. If you have many smart albums, you may be seeing the beachball spinning a lot when editing photos.

I try to keep my iCloud Photos Library as small as possible, so all photos I will need on all devices at all times are in my iCloud Photos Library (all my favorites, photos of important documents, and all new photos I am currently working). In addition to my iCloud Photos Library I am keeping an archive of all photos on an external drive (backed up on another drive). This archive is a collection of many small libraries, one for each year, and one for each photo safari.