As said there is no separate setting to turn it off and leave all other sounds on, you can however quickly flip the silence switch before taking your picture. Does the trick for me!
I`m using an iPhone 4 too and trying to find a way to turn off the shutter sound.
Turning off the ring tone using the side switch does NOT work for me.
Turning the switch on the side to Silent works for me. However, there may be national laws involved. In some places it is required that cameras make a sound when a photo is taken. Your profile doesn't say where you are, but this might be an issue.