I have just cleaned my Mac Pro. Yes you are on the right track. I pulled out the processor assembly at the bottom, the SuperDrive tray at the top (both very easy), and the graphics card (at little bit harder), and with compressed air in one hand, and vacuum cleaner in the other, I blowed and vacuumed out all the dust, there was heaps!
As others have suggested, use the compress air conservatively and in brief bursts, and only when the vacuum can't get to or dislodge some dust. I've had one graphics card fail due to the build up of dust in his fan fins, so I recommend getting it out and cleaning those little fins - which will probably be clogged with dust.
My Mac Pro now runs much quieter, and happier I suspect.