FWIW, whatever your connection speed is, you can only download or upload at the speed a server will accept. If you try to upload something to a site and that site is busy, or the route from your computer and the site has a slowdown somewhere, or it's just throttled, one of the servers are being worked on so the route has to be re-routed etc., that will affect the speeds.
As far as speedof.me, it doesn't seem all that accurate. It shows my download speed as 111% of what it actually is and my upload speed as 31% of what it is. SpeedTest.net reflects my actual UL/DL speeds.