Why does iCloud shared album share more pictures than what I have for a particular person

I wish to share pictures of my deceased father who famously avoided having his picture taken. I have 190 pictures of him in my library that I wish to share with my mother. I set up iCloud Shared Photo Library, choose my mother as a participant and then select my Dad. After a few seconds, it says I have 6,023 items from moments with him. Why is this different? It seems to add photos to the shared album which do not include him. What's going on?