iPhone Mic Won't Work When Receiving Calls

Very strange issue with my phone. A lot of the time, however not all the time, if I get a phone call and I answer, the mic does not work. I can hear the person on the other end, however they cannot hear me. When I place a call, I can have a normal conversation, this only happens when someone calls me.

I've tried looking up my issue, but I only see situations where it is for both making or receiving calls, nothing particular to my situation. Has anyone dealt with this?