Weird alien-sounding echoes only my voice, and only when I’m talking to one specific person

For the past couple of months, my iPhone SE has started echoing my voice in a weird, garbled way, but only when I’m on a call with one specific person. His voice sounds normal to me, and he doesn’t hear the garbled echoes of my voice— only I do. I have tried calling him from my work phone, and the same thing happened, only not quite as loud as when I on my iPhone. He has a “dumb” AT&T flip phone. We live on opposite coasts, but when we were together in person recently, sitting right beside one another, I called him to see if the noise would still be there, and it WAS. We switched phones and he talked into mine, and he had the same echoing distortion of his voice. So at least we know I’m not crazy! What could be causing this? It only happens when I talk to him — not on any other calls. It’s so loud and distracting that I can’t continue the conversation, and it’s making me very sad!

Thank you if you can help.