Safari has a weird issue with page load times, never could figure it out, so I just stopped using it.
Did you upgrade or install fresh? (if the first, see what happens with a fresh installation)
I run my own DNS on my network, and I also use Google/Sprint DNS, so in a nutshell, my browsing speed is maxed out, the only way to make the network run faster is for the techs upgrade the servers/websites on the other end of the wire.
Imagine, pressing enter and the page is loaded, that is how fast Mavericks runs on my computer.
Try some nslookup commands in terminal, time how long it takes to get a response/IP address (anything less than 3-5 seconds is normal)
nslookup apple.com
nslookup cnn.com
nslookup lifehacker.com
Let me know if you get NXdomain or no server found errors.