I seem to have found the reason for this behaviour in my case. Hopefully, this should help those interested in fixing it in Notes app.
I am using OSX Mavericks [ 10.9.1 ] though.
I have notes on MacBook Pro synced up with my Google account [ GMail ]. I notice that this creates a label called "Notes" in my GMail account. It probably uses this to sync up the notes through GMail.
I also happen to sync my GMail locally with the MAIL app and wanted to stop syncing most of the labels in GMail. Once I disabled syncing of "Notes" label, immediately upon editing any item in Notes app, I started seeing this duplication. I tried reproducing this by enabling and disabling the "Notes" label in GMail. And whenever this label is disabled I always got duplicates [ and quite a few of them, depending on how much data I enter in my Notes item ] [ I use Priority Box feature in GMail though ]
My understanding of this problem is that, the Notes app tries to sync these items to the server. And since I disabled syncing anything that uses "Notes" label in GMail, Notes app failed to sync items. Any item that is failed to sync up with the server is stored locally in "Recovered Items" folder. And if you are viewing your Notes items through the "All Notes" folder, then you will see the list of failed items present in "Recovered Items" folder too. And this gives the illusion of multiple duplicates.
Try to fix your Notes app connection with your server [ either iCloud or anything else that you use ] and probably it might solve this issue.