On a Mac with Mavericks (or Yosemite) installed on it you don't add them to iTunes, that is only used as a tool for syncing your books to the iBooks app on your iOS devices, iTunes isn't used for managing them.
For the books that are compatible with iBooks (you probably won't be able to copy the Mobi files into it) then you can add them to the iBooks app on your Mac either via the File > Add To Library menu option within the app, or by dragging and dropping them onto it.
If you connect your iPad to your Mac's iTunes they should then show on its Books tab for selecting and syncing to the iBooks app on the iPad
For the mobi book, you will need an app on your iPad that supports more books e.g. the Kindle app.
For Kindle, and possibly other mobi apps, you can add books to the app by selecting the iPad's Apps tab, scroll down the right-hand side of the screen to the file sharing section, select the app that you want to copy the book to, and add it to the app.