I have had this as an intermittent problem on my iPhone 13 mini, despite always updating to the latest version of IOS. My local Apple store replaced the display to no avail, and did various other diagnostic tests which revealed nothing. At their recommendation I reset all settings on the phone, but this did nothing and was a total pain to boot. The only other thing they could suggest was a full factory reset, and I wasn't prepared to go through that.
I did find that doing a forced restart cleared the problem for a while, but within a couple of weeks the slide-to-answer would start to get unreliable again. (I also found that doing a normal restart didn't work as well). So now I just do a forced restart once a week; that has so far kept me free of the problem for nearly six months, and hopefully it will continue to do so.
I daresay Apple will eventually address this problem, but until then I can live with this work-around.