how to completely empty mail

I thought I'd go through my mail app and empty trash of unwanted emails. When I'm in All Mail and going down the list deleting, they won't go completely away. What is going on and how do I really delete my deleted mail?