It definitely does. With the kill switch off, if your VPN drops even for a few seconds, all traffic will go out as if you had no VPN until the VPN reconnects. And in my tests of PIA, that happens often. Normally you would not notice, but I did a Wireshark (network monitoring tool) capture and saw a lot of traffic go out sans VPN over the course of an hour with the VPN was on.
If you follow the steps I outlined and get a VPN configured in the System Settings instead of in the VPN vendor's app, Messages does not go around the VPN, so that's the good news. The bad news is that there is no kill switch setting. I saw my VPN drop, and Messages (TCP port 5223) leak many times. So as you say, what else is leaking out? When I just looked at all non-VPN traffic, the answer was - a lot. When I used the VPN app, whatever routing magic they do seems to address that.
So ... this is definitely not a perfect nor a permanent workaround. I would stay off the McDonald's Wi-Fi for now :-)