I don’t know whether the stabilization issue you are observing is fixable in iMovie. You might need to look for a more powerful app. Possibly you may be able to split the clips into segments to isolate the places where there is considerable zooming and provide a lesser degree of stabilization to those clips, or maybe eliminate the stabilization in those clips. A little hand shaking might be an acceptible trade off. That would be your call of course.
Also, stabilization is best applied by the camera during shooting, if it has a stabilization feature. Or does your camera have editing capabilities that include post recording stabilization correction? If so, you can do some correcting of the clip on your camera and re-export.
— Rich