I can only hear media , keypad sounds through my headphones but my ringer works

So yesterday my iphone stopped playing sounds without my headphones plugged in. I can't hear the keypad sounds , alerts and media sounds without the headset being plugged in my iphone. However the ringer works !!

I tried restoring and backing it up

Cleaning the headset area with a small tissue

It stops working for a while then comes back on

What could I do ? It's driving me crazy