Allow Certain IP Addresses through a Port

I have a mac mini server. I opened a pinhole on the Airport router but I need to now make it so that I only accept connections from specific IP addresses. Seems that the only way I have found is to turn the firewall on on the mini then define networks. Problem with that is there are like 8 addresses and they are not in a set range. Kind of surprised there is not a simpler way. What am I missing?