Specifically because of this issue, I use both person-specific keywords to identify myself, my wife and a few other people we have a lot of pictures of, and then I use an umbrella keyword for "everyone else". Thus it's easy to create a smart group that excludes "everyone else" by that keyword.
name is Paul
name is Kim
keyword is not everyone else
It would be a bit of a job to keyword a large photo collection like this, but not
too bad, especially if you already have the positives keyworded. Basically you just create this smart group:
name is Paul
name is Kim
And then select all the photos that contain other people — even in a large collection this shouldn't be too huge a chore, because you're only after the intersection of two sets.
