How do I limit HomeKit user access to certain rooms?

I‘Ve set up a variety of Lutron switches around my house and can control them using the Home app on my devices. I’ve also given access to my wife and kids and their devices work as well. I’d like to give access to guests when they stay at our house but don’t want everyone having equal access to all switches. For example, I’d like to restrict access to bedroom lights and ceiling fan to just the person who lives in that bedroom, while giving them access to other controls such as living room, kitchen and garage. How do I select or change what switches or rooms each user can control?