What system version are you using? Your profile says you are using a relatively modern OS version on a modern computer but you posted in a forum for 15 year old computers. How drive security is handled differs greatly.
Mail does store a cache on your computer. I don't know if this includes a copy of deleted messages. I suspect it does because I can go to my Mail application and click on Trash and it immediately shows what I have trashed recently whereas Mail typically takes a minute or two to finish updating from the server.
Some of the answer to this will depend upon the level of security you are wanting and in what scenario this is being done.