Similar problem here too (quite high spec 2011 iMac 27") ; takes longer to boot up and Safari is slower under Yosemite than Mavericks.I have just installed Firefox and had a play around with it and it is much faster than safari ~ just couldn't get the appearance the way I liked so uninstalled. This morning I was using a five year old Compaq (averagely specced) and using Chrome it was much much faster than Safari is currently. I like Apple products (and just bought a new iPod classic for spare ) have, iPad etc but these 'upgrades' are like puling teeth for me. IOS 8 was useless on iPad 2 but improved slightly yesterday with IOS 8.1. I have little interest in technology as an end in itself, ( hifi and audio is my thing) just wanting it to work. I still can't send emails since Yosemite was installed last week as 'smtp servers are offline' ~ well, no they're not as email still works on iPad, android phones and compaq using the same servers. This really irritates. I was so impressed with iPad 2 when I got it ~ it set itself up virtually and I was sold on Apple's ease of use but I don't get back the 3 hours I spent last Friday checking settings, deleting accounts, re-adding email accounts and ultimately failing to fix it.
Yesterday I went to a famous dept store here in the UK to get an iPad mini with retina display as our iPad 2 is 3 years old but I left the store without buying it; just got sidetracked by looking at Samsung and believe it or not, Surface 3 program microsoft. Enough doubt in my mind after IOS7/8 and now Yosemite.
I still think Apple's products are beautifully designed and aesthetically and ergonomically impressive, but the Yosemite experience is draining; everyone glad handing about how great it is and it patently isn't. The design changes in the mac mini (lower performance than 2012 model ) and iMac (5K screen notwithstanding, much more difficult to make internal changes)) looks like arrogance.
Sorry for the negative rant but that's almost certainly me done