You're correct that the key to making the item appear in iTunes' Audiobook library is the extension .m4b, but the change only succeeds if you start with an AAC file. You would change "m4a" to "m4b" by rightclicking the file in Windows Explorer and choosing Rename. If you don't see any file extensions for your files, go to the Tools menu in any Windows Explorer window and choose Folder Options. On the View tab, uncheck the option "Hide extensions for known file types."
Step-by-step instructions for making CDs into a proper audiobook are here:
http://discussions.apple.com/message.jspa?messageID=4218783#4218783