How to I restrict incoming FaceTime calls?

With the new iOS and Yosemite, it is handy to be able to use my iPhone for calls via my iPad and MacBook. However, now there is a new problem. It is easy enough to block an individual number, but I want something a little more robust. How can I restrict incoming FaceTime Audio/Video calls to just those people in my Contacts list? That is, without using "Do Not Disturb".