Permission to access certain folders

I'm afraid a was mistaken when I thought the problem occurs only with VSCode. I tried some more apps and found out the following:

• Apps downloaded from the App Store don't have to ask before accessing Desktop, Documents or Downloads (at least it seems so to me)
• Apps downloaded from other sources seem to have to ask before accessing these locations. I tested with a handful of apps, some of them where able to access these folders even when I did not grant the permission to do so, some others were unable to access the folders in that case, but didn't ask a second time for this permission.
• Under Settings -> Security -> Privacy -> Files and Folders, the checkboxes of apps that can access these folders (even though I denied it) are unchecked as if the apps weren't allowed to access them
• I even removed an app completely from the list in Settings -> Security -> Privacy -> Files and Folders, but it still has access to the Desktop and also didn't ask again for permission again

So this feature clearly seems broken, I only wonder if other users can reproduce this odd behaviour.

As far as I understand, this feature isn't related to the folder permissions that can be viewed and set in Finder, but is meant to be an additional protection for some special folders that would normally be accessible for every app you run.

Thank you for your reply. I tried some more apps and perhaps I simply don't understand the concept behind this additional protection.

Terminal.app for example asks for permission, and if I don't grant it, it is unable to access said folders.

VSCode, as mentioned, also asks, but doesn't seem to care about the answer.

Other apps don't ask at all and are simply able to access those folders (e. g. Word, Libreoffice) (which is what I would have expected, as the folders belong to my account and those apps are running under my account). Those apps also don't appear in the list in the security settings at all.

It is not that this feature is very important to me (I wouldn't mind if any app I run was able to access my Desktop without asking for permission), I'd simply like to understand how it is supposed to work.