Hi,
My first thought would be to check the Messages > File Menu and the keystrokes listed for closing a chat and the like and then checking that against any "helper" app you might have that substitutes keystrokes for longer text.
I use TextExpander.
It is a System Preferences based item.
I use it to post the pic and time and Spec info in my signature block at the end of posts.
In the Preferences Pane it looks like this
(For the curious. I change to HTML mode then type the 4 characters listed as the abbreviation and it "pastes" it to the post. The Pic URL is grabbed after posting > then Editing and looking in the HTML mode again)
TextExpander is not the only app or Utility that can do this sort of thing.
I would also look at whether you are knocking the CMD key whilst trying to type Capital W as ****+CMD+W will close the chat.
However I am not entirely sure this would explain you being in the chat still and being able to attempt to scroll up and then finding nothing.
Are you actually logged in to Google at the same time ?
This may cause issues with Priorities.
I found in iChat 5 you could login to both and each would hold it's own part of any conversation.
Effectively a Buddy and myself both did it and realistically the iChat part was separate from the web browser login to the Web mail page and using Chat there.
I have not tested again in Messages but I have noticed the Priority setting in Messages is now different. It used to be 25 which is the same as the Google web page login but is now 0 (which itself might be a null amount).
9:46 pm Tuesday; February 11, 2014
iMac 2.5Ghz 5i 2011 (Mavericks 10.9)
G4/1GhzDual MDD (Leopard 10.5.8)
MacBookPro 2Gb (Snow Leopard 10.6.8)
Mac OS X (10.6.8),
Couple of iPhones and an iPad