These forums are full of users reporting issues with M1 Macs and USB hard drives. I have seen a reference that USB devices using the JMicron USB controller were at fault, but some other USB controller chips may also have issues. The reports mention being able to see some folders, but not other folders/files. Unfortunately except for lots of posts on these forums and a few random bits elsewhere (credible though -- one product vendor briefly had a note on their website about the JMicron USB controllers not being compatible with an M1 Mac) I have not seen any in-depth investigation or report about this issue.
You may want to try connecting the USB drive to a USB3 hub to see if it might make any difference. Or try using another USB3 drive using a different USB controller chip or verify with the manufacturer that your USB drive is 100% compatible with an M1 Mac. I only briefly tested my USB3 adapter which uses the ASMedia USB chipset which seemed to function Ok.