You can delete them from your device and/or your computer, but you can't currently permanently delete items from your purchase history, all you can do is hide them so that they are less visible : hiding and unhiding (hiding doesn't appear to work on iOS 6, but you should be able to hide them via your computer's iTunes).