Hi, Golan123,
Take a look at this thread link for some discussion about audio tracks. It may answer your question.
https://discussions.apple.com/thread/251112344
You are not doing anything wrong. Adding video above audio will impact the audio. If you hold down the Option and Command keys you can click along any audio track to relocate the little handle that connects it to the video clip above it, to lock it to the video clip at the new location of the handle. Then, when you slide the video clip along the timeline the video will stay with it.
As discussed in the above link, the easiest work flow is to add and adjust your video clips first, then add your audio clips afterwards. In your present project, you can select the audio clips as a batch and then do an Edit/Copy, Edit/Paste, them into a new project created for this purpose. Then you can copy and paste them back after you have finished editing your video clips in the original project.
-- Rich
-- Rich