Can't always answer calls on iPhone 5 since ios 8.0

My dad use to have this problem on his phone on ios 7.?? but mine started doing it since ios 8

My phone will ring (I don't lock my phone) and I cannot swipe to answer but only like 1 in 10 calls. When it happened to my dad I found a workaround for him that if he pressed the volume buttons it would then work. That is not working for me however.

Things I have tried:

Resetting phone

Blanking out phone as new then restoring from backup

Blanking out phone as new then manually adding back all info <--- this was a massive 3 week PITA

Having Apple "Test" the phone (plugged it in and did a hardware test at store, showed good).

I seriously believe is to be an ios bug that just pops up at random. It does not seem to matter where I am or what I am doing or what programs I have installed. And the fact that my dad's phone did the exact same thing (but on a different ios version. Oh and he has not upgraded to ios 8 since he has a version that now works flawlessly).

I just get frustrated and have to keep calling people back (once the call goes to voicemail or they hang up and the screen goes black I can then open the phone with no problems).

I have no other problems with the phone, the touch screen has never given me any issues (other than answering) and when this is happening no button on the phone does anything at all. I can't even turn it off while it's ringing.

I just don't know what to do at this point. I have been fighting with this since the day I upgraded to ios 8. I keep hoping that a new upgrade will fix it like it did for my dad but so far no luck.