Audiobook management: how to tag files as audiobook and keep them that way?

I have many audio books. Those were read from the CD and thus end up in the music section. ⌘ + I > Options > Audiobooks allows me to move those files into the audiobook section.

The issues is, that flag is not retained once those files are removed from iTunes and then re-added. The files I tagged as "Audiobook" earlier are now back to the music section.

I find this immensely frustrating and confusing. Is there any way to keep the files tagged correctly so that they are automatically added to the audiobooks section when re-added?