Here are some links from other versions of Primatte.
This movie demonstrates some general ideas:
http://cache.redgiantsoftware.com/assets/uploads/file/community-tutorial/primattekeyergettingstarted.mov
The Primatte Chromakey manual has some useful explanations starting on page 28:
http://www.digitalanarchy.com/manuals/Primatte2-manual.pdf
and on page 57, there's an explanation of the polyhedra.
As far as the 3D space goes, just think of your 3 color channels (RGB) being
the 3 dimensions of a cube. So any RGB color can be found somewhere inside the cube.
Now you select a green or blue from your screen. It's probably not a perfect color,
so it's a point somewhere in the cube near green or blue. If you made a small sphere
or polyhedron around that green point, you would get colors that are near the greenscreen color.
In fact, they might be located in the highlights or shadows of your screen, if the
lighting was less than perfect. Hopefully, you can define this little polyhedron
so that every color of the greenscreen is inside it, and so that colors from your
foreground image are outside. When you pull the key, the colors inside the polyhedron
produce a black alpha channel value. That's your background.
Of course, in the real world it's more complicated. There are probably some colors
that you need to manually asign to either foreground or background.
The cool thing about all the Primatte keyers is that, after defining the basic screen color,
you can add groups of colors to either the foreground or background by selecting
the appropriate operator and sampling those pixels. In addition, there are operators
for putting colors in a couple of in-between "shells" around the polyhedron.
For example, there might be some greenish colors that should not be part of the background,
since they are the result of spill. You can put them in the spill "shell" surrounding the
background polyhedron.
One confusing thing about all the documentation is that they talk about putting
pixels
inside or outside of the polyhedron. This is confusing because pixels are a
location
in the image. Actually, you are putting
colors in the various categories. Any pixel
that has that color will be assigned to the appropriate category (foreground, transparent,
spill, background).
I hope that helps.