Are you using windows xp?
Is the phone charging when plugged in? Are you plugging directly into the usb port, not using a dock or usb hub? Laptop or desktop? Try a different usb port, preferably one in the back of the computer.
Open device manager and check to see if and where the iPhone shows up. The first place you would want to check is under usb. It might appear as usb mass storage device or something apple related. It could possibly also appear somewhere else in the list (imaging devices). Try using this:
http://support.apple.com/kb/TS1538