I had a simple application which I dragged to the Mountain Lion finder toolbar and it acted as a Finder droplet. I could not then get rid of it - it would not drag off the toolbar.
Final answer was easy if a little unintuitive (to me).
a) customise the Finder toolbar (choose from Finder toolbar's context menu)
- but there's no sign of the droplet in the list of selectable Finder toolbar icons.
b) Now drag the droplet itself off the toolbar and 'poof' - it's gone.
Of course command-drag is even easier - if I'd read to the end of the comments here before answering. <sigh>
...Ian.