AirPods Won't Connect During Call

I am having this issue. If I am on a call via the phone or speaker and want to switch over to the Air Pods, it usually wont work. I open the case, and pull out one of the Pods and put it in my ear. Nothing happens. I remove it and put it back in my ear. Nothing. I remove it and put it back in the case. Close the case. Reopen the case. Nothing. The Air Pods battery level box doesn't show up on the screen. If I put one back in my ear and then tap on the Speaker icon in the active call screen, sometimes I see AirPods listed and sometimes I don't. Even when I do see them listed and tap on Air Pods, it won't connect.

I have unpaired, repaired and tapped Forget This Device many, many times. Apple even replaced my Air Pods with a new set. They seem to work more often when I am not on a call, say when I want to listen to music only. But I bought them primarily for calls. Maybe there is something about activating the Air Pods during a call that is problematic.

I suspect this is an iOS 11 issue. But really, no idea.

Any suggestions would be appreciated.