Red badge on iPhone Phone app won't clear for voicemail

My telephone carrier has visual voicemail activated, and thus when I call voicemail it goes through as “call failed” and does not actually dial. My previous carrier did not have visual voicemail, so I am unsure if that impacts it. I recently got a voicemail, and the red notification badge has shown up on my phone app with no number. I can’t call voicemail to get rid of it, and all my visual voicemails are read. How do I get rid of the red badge?