You can't permanently delate any purchase, all you can do is hide them so that they don't show in the Purchased area for redownloading : Hide apps for your iPhone, iPad, iPod touch, or Mac - Apple Support
are they taking up space on my storage
They aren't, if they aren't downloaded they aren't taking up any space on the phone, and purchases don't use any of your iCloud allowance (iCloud contains your content backups).