A .dmg file is a "Disk Image" file. Basically, it's a file that contains data just as it would be laid out on a disk. The computer can then open it up and treat it just as if a removable hard-disk has been inserted into the machine.
By default, Safari will "open" a disk image file when downloaded. When you "open" the disk image it does what UNIX calls "mounting the filesystem" contained in the file, that is to say, it takes the file snd sets it up just like you attached a disk to the system.
You'll have two icons, the disk image, and another for the "disk" contained on the image. You can eject the "disk" and you can trash the image (but it won't let you throw away the image file if the "disk" is mounted).