I can't change my language while typing, in any app.

I have an iPad 3 and recently reset it to factory settings. It's running the latest iOS 8.2. Now I always set my iOS devices to default language English, but I since I do live and work in Germany, I also use German. I was able to add the secondary language just fine in the settings menu, but for some reason I can't switch to it in applications (Notes, Messages, OneNote are the ones I tried it in). Where the globe icon on the keyboard normally is, I can only see a smiley icon and I can switch to Emojis, but not to the secondary language. I did restart the iPad after adding the language, just to make sure.