How can I delete collections in iBooks ?

You can only delete the ones that you created and you do that via the Edit button. When you tap on Collections, you will see a list of all of your categories. The Edit button is at the top of the list.

Tap Collection to open the list. Tap Edit. Any collection with a ➖ on the left can be deleted. Any collection without this is built in and cannot (typically "Books;' and "PDFs"