Managing storage of Mail attachments

I believe that it is not generally a good idea to go messing with system files created in the Library folders.

But can I delete a “Mail Downloads“ folder from the depths of Containers/com.apple.mail in Mac’s Library folder without upsetting the system?

And will deleting in this way also delete any attachment from original email ?

Although there are many posts about storage of Mail attachments I haven't managed to locate answers to these specific questions. Hopefully someone can point me in the right direction.