How do I permanently turn those shortcut windows?

You have encountered the Shelf, this being a fundamental feature of the iPadOS multitasking environment.  In iPadOS 26, the shelf will be seen when using the iPad in its Full-Screen Apps mode:

Settings > Multitasking & Gestures > Full-Screen Apps

If instead you select Windowed Apps mode, the Shelf will largely be suppressed from view.

When using Full-Screen Apps mode, the Shelf is triggered if you tap an App icon within the Dock - and you have multiple instances (windows) of the same App already open.

The Shelf, when triggered, shows a thumbnail of each running App-window; tapping a thumbnail image allows you to quickly switch between your open windows.

Your screenshot illustrates that you have currently two Google Chrome browser windows open.