Would it be possible to publish a comic book on Apple Books?

I mean is it possible to publish them on the actual store. It doesn’t mention anything about being able to or even being allowed to publish comics and put them on the store.

Why don't you type "comic book" into the search field in the store so you can see what others have published? That should give you an idea about what is possible.

https://www.imore.com/how-publish-your-ebooks

Use iBooks Author to create and submit books to Apple Books or distribute anywhere on the web.

https://support.apple.com/en-us/HT201183

You should also look at what's being offered on Amazon/Kindle, I would imagine that has a much larger audience since it is not confined to Apple devices.