Deleting Podcasts does appear to be a hit-and-miss affair, as I've experienced just this past weekend.
Try deleting the episodes first. Once all the episodes are deleted, select the Podcast's header and use the delete key on your keyboard. That should finally remove the Podcast (title) from your iTunes Library.
You will probably have to find and delete the folder for that Podcast as a separate job.
I frequently found that using right-click/delete did not actually delete the header. On other ocassions (such as just now) I've had a pop-up asking me if I wish to delete all 1000 episodes in the Podcast. Not a reassuring message when I have no episodes downloaded and only about ten episodes listed with the cloud icon (suggesting that they are in the feed and waiting for me to download them).