I finally upgraded to Yosemite a few days ago and have the same problem with bigger PDF files. I discovered two solutions. None of the the other solutions posted on this and similar threads worked for me.
1) Use Adobe Acrobat Reader instead. It seems to work better speed-wise, but of course, it's interface and functionality leave a lot to be desired.
2) (***Not for the faint of heart***)... Replace PDFkit framework with the one from Mavericks. Here's how:
a) BACK UP your system!!!!
b) Go to /System/Library/Frameworks/Quartz.framework/Versions/A/Frameworks
c) Copy PDFKit.framework to somewhere else on your computer for safe keeping in case you need to replace it. This is version 3.1 in Yosemite.
d) In the same folder on a Mavericks system, make a copy of the PDFKit.framework file. This should be version 2.9.2 in Mavericks.
e) Replace the file in folder (b) with the Mavericks version (d)
f) Restart the computer
So far, this has been working for me. Preview is back to normal speed. It's only been a short while and there's always the chance that this will break something else that I haven't found yet. If you do this, make sure you have a way to go back (reinstalling Yosemite would be one way).