I'm having this issue, too. It's not crashing. There is no crash report. It just quits.
While I can't find a way to disable App Nap just for Safari, as you can on other apps, I have been able to disable it on all apps.
http://hints.macworld.com/article.php?story=20140508050203105
But this didn't solve the issue. I looked and found some people had Genieo or InstallMac, but after doing a search of my computer and also running Adware Medic, I found neither of them. I also did a virus scan and it found nothing.
I cleared my history. It still quits with inactivity.
This seems to have started about the same time as my Yosemite update. Chrome will stay open, as well as other programs, it just seems to affect Safari.
It's more annoying than anything and mostly, I was worried it was a symptom of a larger problem. I still haven't found a solution for it.
Version 10.10.3
iMac 27 Inch, late 2009
3.06 GHz
16 GB