How do I take control of mic inputs on iPhone6?

When I cover speeches and lectures, I use my iPhone to record the line-level output from PA systems via a KBConnection XLR to TRRS adapter. It works great about half the time. The other half of the time the iPhone switches itself to using its own external microphones which is not what I want it to do.

A dedicated digital recorder like a Zoom or a Marantz is not an option, I have too much equipment to schlep as it is.

So, has anyone else encountered this problem and/or does anyone have a solution. Because it only happens sometimes, I assume the iPhone is using some kind of software-based switching between the inputs rather than purely electrical.

Is there a way to make it do what I need it to do?

In case you've never seen one, the adapter looks like this:

Tags: Field audio, sound production, mult box, press box, professional audio, dual track video, public address system, PA,