Depends on the book, and how portable you want it to be.
If you are just reading novels without illistrations and pictures that need to be studied, then an iPod Touch will do just fine. It is very portable, can be carried in a pocket with you everywhere, and pulled out to read anytime you are waiting in line, or have unexpected free time.
Even some PDF documents can be read on an iPod Touch 'IF' you get an app like GoodReader that will reflow PDF text so it is readable at a larger font (for example I have my car's owner's manuals on my iPod Touch, and I find it much easier to find things sitting in my car using the iPod Touch with search, then paging through the actual books; of course the same applies to having the owner's manuals on an iPad/iPad mini 🙂)
If you have to read technical books, and any other content that has illistrations, pictures, and examples that should not be reformated to fit the smaller screen, then an iPad or iPad mini is a better decision, as it can display the illistrations, pictures and examples as formatted by the author.
And while an iPod Touch can deal with PDF, an iPad or iPad mini just displays the PDF pages as composed by the author and they are large enough to be readable.
NOTE, some eBook readers support sync'ing between devices, so in theory you could start reading on an iPad/iPad mini, sync where you are to the cloud, then continue reading on an iPhone/iPod Touch getting where you stopped reading from the cloud (requires access to the internet, so could be a problem if your iPod Touch is not doing this while in a WiFi hotspot). So you could get Both 🙂
I have been reading fiction novels on a handheld device since 2001, starting with Palm devices, and switching to an iPod Touch when the 2nd generation came out. So far I've read about 700 ebooks this way. Your mileage may vary.