dock icons will not stop running

The icons on my dock will not stop running. They have the small blue light under them when in use but the light does not go off when I close the program. I have been quitting them by holding down the curser on the icon until menu comes up to quit. Help, please. Thanks, Arudi