When I am on the iPhone Xs and the screen is black, I have to tap once, then swipe up, then wait 2 seconds and swipe up again in order to open the phone. Is this an issue with the OS or is there an issue with my settings? Its really annoying since.

When I am on the iphone Xs, and I am speaking on the phone. The screen is black, then I tap to open it and swipe us, but it doesn't open. It goes to a screen with the time on it. Then I have to wait 2 seconds for it to swipe up again to open the phone. This is extremely annoying and I do it about 10-15 times per hour, when speaking on the phone so I need to find a solution to fix it. Is there a way that I can just tap and swipe to open the phone like I do when I am not speaking on the phone? This is extremely cumbersome. Please help.