We've owned all 3 types... here's how I would describe them:
The g4 xserve is ridiculous... we only had one of these but the second the g5's were released, we sold thing this as fast as possible. It was far louder than some SGI servers I had.
We have 4 G5s, and they've been wonderful... I've seen the fan speeds range from 4000 - 7000 on average, depending on the temp of the room, and are typically very quiet overall... not so much that I'd keep them in the same room as my main work room, but they're excellent in a next door room.
The intel xserve was interesting... We have a quad 2.0, and at first thing thing was fairly loud (on average 9-10000 rpm), that we could hear it from another room with a door. Then summer came (we have a small studio so there was only one main AC unit for the entire place), and we simply couldnt keep that room cool when the server was on and it would jump up to 12000 when it wasnt doing anything.... and then with h264 compression would jump to 15000, which was starting to sound like a g4. At that point, we had a seperate AC unit put in, and we keep the room at 70, and since then its around 6-7000 normally, and 7-8000 when at full CPU, which is fairly good with our room layout.
Basically, the intel xserves, you can get very quiet if the room's cold enough, but its very interesting how much heat they spit out once you get them going, where as with the g5's, even if the intel was at 15000, the g5 in the same rack wouldnt go above 6000.