Same issue here.. I've lost some confidence in this thing.
I just received a mac mini M1 late last week. I connected a 1TB external SSD using a short USB type C cable. To save space on the internal SSD, I moved my user home to the external drive (something similar to what I did on my imac).
I had noticed a few times I had received the same error you mentioned saying "not ejected properly" even though nothing was touched on the external drive.
Things worked fine for a few days until I had woke it from a sleep, and getting that error again, the drive disappeared from the finder. I rebooted, and when it came back, it behaved like I was logging in for the first time. The OS could not see the user directory or its files. I was able to see the drive, but when I went in command line and to the drive, no directories or files were visible.
In drive utility it said there was 250GB of space used, yet the OS was hiding the files/directories (btw: The external drive was formatted with APFS).
This was really frustrating because it took me a lot of effort to set up my user (since I was trying to migrate from my imac - just copying over docs, photos, etc).. and setting up my apps again. Now, all is lost and I have to start over. I am not sure if using APFS contributed to the issue, and perhaps I should have used the mac extended file system.
There are new security features on the mac mini m1 that can cause this?