Downgrading requires to have full, working backups of your system and your Aperture library from before the upgrade, so you can restore everything from your backups. You can neither downgrade the library nor the system by reinstalling 10.8.5 on top of 10.9.
Finding out, what is causing the memory problem requires requires a bit of diagnosis - it is not necessarily an Aperture bug. It might be caused by images or videos that cannot be processed, incompatible plug-ins or other software, hardware problems, system trouble.
I'd try to keep the Console open, when Aperture is running, to see disgnostic messages.
Also work with a new Aperture library for a while, to see, if the library is causing the problem. Try to remember, what your most recently installed applications and utilities were.
How is your Aperture drive connected? Directly or using a HUB?