Alright I am an idiot. I was celebrating too soon. I didn't test well before posting.
This is still sharing one screen, just sometimes two. So I am waiting for my friend to get back so I can test further.
He is logged in, and I was using VNC. I went to test if I could switch to my screen on the computer, which I could. But there is no way for me to get back to using VNC only. Anything I do, even reconnecting with my username and password, I am still on the physical screen too. I had this situation before using tigervnc viewer and just the main screen sharing password. So nothing is better.
The point is I need to be able to connect, disconnect, and reconnect without any interaction from a user on the physical screen.
This is just unacceptable. We need to be able to independently log in and out, without seeing each others' screens. This seems to only work if someone is on the physical screen and is busy, then the second user can connect with VNC. But sooner or later, both end up sharing the same screen, unless perhaps being very careful.
I will try with realvnc server installed on the mac mini and see if that is a solution.
The most disappointing thing about this, is that there are a number of situations for clients, where a Mac mini or Mac pro could prove to be a solution if multiple users could use the computer, without seeing each other's screens. So now the only option is *nix, that I know of. If the needed software is available.
I will post more details, and the results of testing with realvnc server installed.
But this still does not work correctly. It may be a not-very-good work-around for some situations. And maybe ours, but it would require sharing log-in passwords, which I do not like at all. And would never recommend such a solution for a client.