Logic Express 9

I'm having a problem with latency in Logic Express 9. I know how to adjust it in the audio settings, but no matter what I set my buffer size to, after a few seconds of playing, there is a delay between when I play and when I hear the sound through my speakers. I've done lots of recordings before using this same set up, but this just started to happen. Logic said I was running out of memory for a song I was recording, so I upgraded from 4 gigs of RAM to 20. This is my first time trying to record after installing the new RAM. I have my guitar plugged into a Zoom R16 pre-amp, and then into my computer (iMac 2.8 GHz i7 with 20GB of RAM). After a few seconds of playing using the built in guitar tones (and no other program open), the delay kicks in. Any suggestions other than changing the buffer size (which I've tried)?

Thanks