How can I remove apps permenantly?

I have been deleting apps over time. However they seem to only bee removed from the screen. They are still listed in the AppStore Under updates and purchased apps and they open from there. How can I permenantly delete them?