The Finder in Catalina is no longer updating the Info for the Photos Library immediately.
The size you are seeing in the Finder and the modification date may not be changed and updated for a long time. The Photos Library is actually a folder, bundled as a package. Photos is changing the internal folders in the package, but the Finder is not always reflecting this.
To see how many photos are actually stored inside the library package, you can open it and look inside. But be very careful not to modify anything.
- Reveal your Photos Library in the Finder (If Photos is running, go to the Menu Photos > Preferences > General. Click "Show in Finder". Then quit Photos.
- Ctrl-click your Photos library. In the contextual menu use the command "Show Package Contents".
- In the Finder Window that will open, select the folder "Originals". This folder is holding the original image files stored in your Photos Library. How large is it?
If the "originals" folder is smaller than the size of the total of photos shown in the Info for "All Photos" in Photos. Photos has not downloaded all originals from iCloud. Then you should force a download from iCloud by viewing the photos enlarged.
Even, if you have "Download originals to this Mac" enabled, you cannot be sure, that the photos will be downloaded immediately. Photos may be waiting with the download, until you are using the photos.
To check the iCloud Preferences: