It's not because of the Thunderbolt 2 to 3 adapter. It's just that the M1 Macs do not support more than one Thunderbolt display. For example the LG UltraFine 4K Display
LG UltraFine 4K Display - Apple
has two Thunderbolt 3 ports and you can daisy-chain two from one Thunderbolt 3 port on a 2018 Intel Mac min with no adapters, but you can connect only one to a M1 Mac mini.
It's a limitation of the M1 chip. It can support two displays but has only enough circuitry to format one video stream to work on a shared USB/Thunderbolt bus. Such a bus has to be able to handle multiple data formats simultaneously bidirectionally. The video has to be formatted into labelled data packets. The second video stream from a M1 chip is just raw video data which can be used for a builtin display on a MacBook, iPad Pro, or iMac or a Mac mini's HDMI port.
This is a regression from Intel Macs that can handle more than one Thunderbolt display, but the M1 is just the first of Apple's CPU/Graphics chips. It it not suiltible for replacing all Intel Macs, so Apple still sells non-M1 Macs. Buy one of those if you need two Thunderbolt displays, or wait for Apple to coime out with more chips.