Mail App "On my Mac" language localization doesn't work

The reason of this issue is that, when first time Mail is launched on a user account, a hidden "On My Mac" account (just like your other mail accounts) is created with it's Description (name you give to a mail account) set according to the language of the Mail app. The Description of all accounts is stored in the Account database file ~/Library/Accounts/Accounts4.sqlite, under the "ZACCOUNT" table (this db is protected, an app with Full Disk Access is required to read and write this file).

You can find it by filtering "ZOWNINGBUNDLEID" column with value "com.apple.mail", it should give you at least one account and the "ZACCOUNTDESCRIPTION" column will be the Description of the "hidden On My Mac account", editing it do results the "On My Mac" being changed in the Mail app.

However I strongly advise against editing this database, as I have read online that a corrupt Account db won't get recreated automatically and you'll need to recreate your user account or even worse, reinstall macOS.

I suggest that you try restarting in Safe Mode and see if that changes anything.

I just did a quick test. My mac is in English, but I set Mail to use French.

I started Mail, and everything was in French, except that "On My Mac" remained in English.

I did not, however, have the opportunity to restart, so I don't know if that would have changed this.