Fan blowing loud and only out one side of vent

My 2011 27" iMac running Mountain Lion has had an issue of having a loud blowing noise and air blowing relatively hard out the right side of the vent on top, but not the left. I installed an app to check the fan speeds because every thread I've seen on fan noise, usually that is the problem, but the speed seems fine. Tried vacumning the vent and intake as well as using canned air, but that didn't help. Blowing stops when I put it to sleep, and sometimes it will just stop after a long time of use, but it starts back up randomly or a few seconds after waking it up. Anyone else had this problem and know a solution?