App purchases are "final and nonrefundable" according to the iTunes Terms and Conditions, but if you contact Apple and grovel sufficiently, they may make an exception due to your clever child having found a way around your password.
Contact them here:
https://expresslane.apple.com/
Navigate the columns as follows: All Products and Services > iTunes > iTunes Store > Account Management >
No guarantees. If it's just a few dollars you might be better off considering it as an educational expense.
You should consider disabling App Store and / or In-App Purchases by using a Restrictions passcode. This presents a second layer of protection against unauthorized App Store purchases.
The Restrictions passcode does not have to be different from your lock screen password, but obviously it should be. Don't forget either of them because there is no way to retrieve them; they can only be reset by completely erasing your iPod Touch.