iphone 12 pro max telephoto lens interacting with binocular adaptor strangely

I bought an iphone 12 pro max with the intent to improve the telephoto performance of the iphone 11 when digiscoping behind a pair of binoculars. The thought was that the iphone 12 pro max has a 2.5x telephoto lens to allowing me to optically zoom in more so as not to see the circular field-of-view of the binocular (without having to use digital zoom). However, I encountered a very strange problem. When I precisely aligned the 2.5x telephoto lens (the upper-left corner camera) to be centered on the viewing hole of the Swarovski VPA variable phone adapter, the phone camera momentarily shows a image before quickly focuses to la la land with the picture goes away in the "photo" mode, only showing something really dim and out of focus, completely unrecognizable. When I switch to the "portrait" mode, it takes picture just fine, proving my mechanical alignment behind the phone adapter is very centered, but the "portrait" mode does not allow digital zoom. The "video" mode does not work either with the phone adapter centered on the 2.5x lens. When I take the phone adapter away, the iphone 12 pro max works just fine in both "video" and "photo" mode, so the phone by itself is working fine. I then tested the phone adapter with the hole centered on the 1x lens (the lower-left lens), it works just fine between 1x and 2.5x (with digital zoom), but goes dark beyond 2.5x which is expected because above 2.5x the phone switched to the 2.5x telephoto lens blocked by the phone adapter. I repeated this experiment many times, totally repeatable. 1.0x works, 2.5x does not when the Swarovski VPA phone adapter is centered on the respective lens. I can only explain the behavior the following way: There is something special about the 2.5x lens. When centered to a narrow circular opening such as the hole on the bino-phone adapter, it gets affected by the surrounding close range feature (the adapter hole side wall is just a few millimeters away surrounding the lens). The iphone got confused and focused to some strange distance or otherwise causing the camera to shutdown. Is there are solution to this problem for people using iphone 12 pro max behind binoculars for digiscoping? (In this case, the problem has nothing to do with binoculars, as I have not even hooked up the adapter to the binoculars yet, but it has everything to do with the relatively narrow opening of the bino-phone adapter.) If I cannot effectively use the 2.5x telephoto lens on iphone 12 pro max, it totally defeats the purpose of getting this new phone, as the slightly older iphone 11 with the 1x lens (plus digital zoom) works just fine with the same Swarovski VPA photo adapter. Is there a software trick I haven't learnt about iphone 12 camera, or is there a third-party app that allow me to bypass this totally unexpected problem? I can't imaging I am the only one trying to digiscoping with iphone 12 pro max using a phone adapter. Thanks!