To delete an app just touch and hold the app until all the apps start to wiggle. Then tap the circled minus sign by the app you want to delete.
You can turn on a Restrictions that prevent deleting apps. In that case you will not see the circled minus sign.
You can't delete the apps that came with the iPod but yo can hide some of them by Restrictions.