Quitting apps

In High Sierra, most apps continue to run in the background, even after closing all of its windows.

This is confusing to me, because in Windows 10, when almost any app is running, it either has a window open/minimized, or it is pinned to the system tray and tucked away from the taskbar.

In short, is there a way to quit running apps by closing all of its windows instead of having to right click the icon on the dock?