That's really weird. My best suggestion is to reset your network settings.
Open Settings, go to General, scroll to the very bottom where it says Reset, and then Reset all Network Settings.
Note: This will erase all your previously saved network connections and passwords. AKA, when you do this, your iPhone won't remember the password of your wifi network and you will have to rejoin it manually, as well as enter in the password if you have one.
The upside is that it should hopefully make connecting to wifi as fast as it should be out of the box.
Hope I helped!