My iPod still makes sound after muting?

I've recently updated my iPod touch 4th Gen to 6.1.5 and if I hold the volume down button, and turn down the volume completely (media and normal) the lockscreen still makes sound, I still get notification sounds and typing still makes sounds. How do I fix this?

Thanks