Jerky playback?

Items such as trees, lamposts, people, seem to be jumping across the screen when playing back panned footage shot with either my iPod Touch or a new Canon Powershot camera. It is not as noticeable in iPhoto playback though, and in iMovie it appears to be a intermittent fault too, i.e. it doesn't happen at the same points of the footage all the time....

I was wondering if it was something to do with the frames per second setting but this new camera is set to record iFrame compatible movies....

Any suggestions?