How can I let only calls at night from my Favorites list to ring but silent at all other times?

I don't want any sound from my phone during my waking hours, definitely not during meetings, and meals, etc. so I have my ringer in silent mode (the little switch on the side), and set my Sound setting to no vibration. I have an Apple watch, so I get notifications on my watch via vibration when I get calls during the day. Minimal disturbance. The problem is at night when I do not wear my watch. I like my phone or watch to make some noise in case any of my contacts in my Favorites list calls, or emergency calls from my family members (who are in my Favorites list). How can I do that?