The iPadOS15 multitasking environment permits you to run multiple “instances” of the same App, such as Mail - with each instance, perhaps having a different open inbox.
For example, if you have three running instances of Mail, tapping the icon you describe will open the shelf with three thumbnails - each corresponding to each running instance of Mail. Simply tapping a thumbnail will switch directly to that running instance of the Mail App. Simply swiping an individual thumbnail off the top of the screen will close the associated window/instance.
This article from Appleinsider has a useful explanation of this new feature along with many of the updates to multitasking:
https://appleinsider.com/articles/21/08/12/how-to-master-multitasking-on-ipad-and-ipad-pro-in-ipados-15
The iPad User Guide is also a rich source of frequently overlooked information. The Guide is available both online using a web browser (such as Safari), or as an Apple Books download:
iPad User Guide - Apple Support
https://books.apple.com/book/id1567104892