I have the same thing, cleared the ~/Library/Calendars folder, emptied the trash, opened iCal and whoppa, all the calendars are still there. Even when I delete the calendars using Edit -> Delete, they won't go away. Trying to do all this due to MobileMe synching problems. But if I can't even delete calendars, my hopes of trying to solve this issue are diminishing quickly.
Ah well, what else to do on a Sunday morning, but to fight with iCal?