Maps struggles keeping up with my location

I started having this problem recently that's made using turn-by-turn navigation impossible. It takes like 10-15 seconds to get my location, and when I'm navigating it can't seem to figure out my location. The circle is quite big and the arrow points in a different direction than I'm going, and because it only updates every 10 or 20 seconds I constantly miss turns. Once I'm off the route, it doesn't know what to do and just stalls (it won't reroute me, I have to find my way back to the original route that I chose when I hit "Go")

My wife is on the same carrier and has no issues. I've tried restarting the apps, airplane mode on/off, rebooting the phone, and today even plugged my phone into my MacBook and did a backup, clean restore then restored again from my backup and I'm still having the issue. It does this regardless of the maps app I'm using (same behavior in Apple Maps as Google Maps)

I dont know if it's related but the back of my phone is cracked, so maybe somehow an internal component like a GPS antenna(?) got damaged thats causing this? I have no idea whether it's a hardware or software issue, or even a network issue. I desperately need directions for work (I'm running around all day over a large area) so its been a major distress having this problem and I can't just go out and buy a new phone, I just got this one like 3 months ago. If someone could help me out I'd be so grateful!