El Capitan - Safari causing system freeze

I have just recently updated to the final build of El Capitan coming from the GM release. Since then, I've been experiencing weird system freeze like everything freezes up except the mouse but can't click anything. It freezes around 5-7 seconds and then comes back normal. It is very irritating. Is anyone else experiencing this? I tried looking it over the internet and saw similar issues pointing out to Flash. Then I noticed closing Safari helps. I use it as my main browser and I don't want to switch to another just because of this issue so I uninstalled Flash but still to no avail. Everytime Safari comes up or when I begin to browse through sites, the random freeze occurs. I looked up Activity Monitor and noticed that a CPU Usage spike occurs everytime the freeze happens. I also noticed most processes with high CPU usage belonged to Safari. Now this makes me wonder whats causing this on the final build. This did not happen to me on the GM release. I have yet to do a clean install of El Capitan on my MB but resorted to open this up here first incase someone has found a fix for this. Please help.

OSX El Capitan 10.11 (15A284)

MacBook Pro (Retina, 13-inch, Mid 2014)

2.6 GHz Intel Core i5

8 GB 1600 MHz DDR3

Safari Version 9.0 (11601.1.56)