Cant get trackpad to stop moving cursor on video

When watching a video I often use the progress bar at the bottom to move the video forward or backward to show a. piece of the video. Sometimes after doing this once or twice, I lose the ability to move the mouse without affecting the video. Eg I move to the right to increase the screen size on the button and it immediately fast-forwards the video. The mouse is well clear of the video box on the screen yet the movement still affects the video. Any suggestions as to how I can prevent this? or fix it? I have tried esc, caps option-esc etc..I am using Sequoia 15.1 but it used to do this with Ventura etc.