When you open a DMG to install an application, the file is mounted as another external drive or different volume. If you try to delete this DMG when it's mounted, the iMac will tell you that the file is in use, so you won't be able to delete it.
In order to delete the DMG without restarting, right-click the mounted DMG on the desktop and select "Eject". If you don't see it, open Finder, select Finder menu (on the menu bar) > Preferences > General, and mark "External drives". Another option to unmount it is doing it through the Finder window, so open a Finder window and you will see the mounted file in the sidebar, under "Devices". Just press the Eject button which appears next to your DMG name