Changing language on a per-app basis reverts back to default language...

So I am trying to study Japanese and I want to set my Google Chrome app to Japanese without having to change the language of my entire phone. So if I head to Settings, scroll all the way down to the list of apps I have, select on Chrome, I can change the language of the app itself, without having to change the entire phone's language. But if I close out Settings and Chrome from the multitasking window, it seems to revert back to English. I don't understand this feature if it doesn't work... any help would be appreciated. Thank you.