delay before siri hears me

I have tried all of the troubleshooting steps, the microphone works fine for voice memos, videos, calls, and there is no delay when I have my headphones plugged in. I’ve noticed it’s less of a consistent delay and more of a waiting for me to speak delay. So if I hold my home button to initiate Siri and say nothing, the audio line never triggers as if it were picking up outside noise. It’s not until I actually say a few words that the line starts moving as though it were picking up sound, which is very frustrating since it almost as if I have to say a couple random words before I can begin telling Siri what I actually want. Also I don’t believe it’s a connection issue as I have tried using it on multiple WiFi networks as well as cellular data, and as I stated before Siri works perfectly fine when I have my headphones plugged in.