Connect the iOS device to your computer and restore via iTunes. Place the iOS device in Recovery Mode if necessary to allow the restore.
If recovery mode does not work try DFU mode.
For how to restore:
To restore from backup see the following. This will get the iPod like it was when the backup was made. This is you only possible way of getting your information back since when the iPod is in recovery mode (the screen show conect to iTunes) the iPod is already erased.
If you restore from iCloud backup the apps will be automatically downloaded. If you restore from iTunes backup the apps and music have to be in the iTunes library since synced media like apps and music are not included in the backup of the iOS device that iTunes makes.
You can redownload iTunes purchases by: