It really will depend on your patience, how large you will let your library grow..
I try to keep the size of my Photos Library below 50000 items. When it gets larger, the upgrades of the library and the scanning for objects and faces will be very slow. My fastest Mac has an Apple Silicon M2 chip, but even this one is struggling with the library, when it needs updating.
To keep the library small, I am keeping only the favourite photos and the most recently taken photos in my iCloud Photos Library. The complete set of photos I am keeping is archived on external drives, in many smaller Photos Libraries. I am rarely needing the archive libraries, only when I notice that I need to restore a damaged image files or want to check, if Ithere are additional photos that I should add to iCloud.
I have occasionally asked a Genie at the Apple Store for Apple's recommendation of the Library size, and usually the answer is "at most 20000 items", but that would not do for me.