Login screen uses language which is not the system's default (English)

My login screen seems to use the last chosen language on the system, not the default language (English), what makes my password be written in a different language.

In my case it's enough if I hit caps-lock (when it's hit in the other language, it changes to lowercase english), so I'm good.

But it's really annoying.

How can I just have the langage selector in the login screen?