Vertically jumping/jittery frame after imovie export

After export the frames jump occassionally, as if there is a crop active that moves the frame up and down vertically.

Here is an example, the entire frame jumps by around 3-5 pixels every 4-7 seconds:

https://www.youtube.com/watch?v=1jJO0u0CjQM

Problem is with iMovie 10.1 on el capitan. Movies are shot with a sony nex-f3.

The video is completely stable when viewing on the camera and when watching in iMovie.

In some cases the jitter is extreme and going on all the time, other times it doesn't happen at all. I have recorded at various frame rates and shutter speeds, but the results are the same.