Perfect! I fixed my issue using Aaplluvr's 2nd suggestion.
First I tried their suggestion to do ALT/Option + Volume icon. For my, my headphones were already selected (JABRA Headset) for both input and output, which was strange. I also found that videos etc would still go through headphones.
So then I saw the second solution which was to go to the "Audio MIDI Setup" and play with the settings there.
What I changed was Right Clicking on my "Jabra UC VOICE 550 MS USB" headset (Or for other people, select the headphones you are trying to use).
Then I clicked on "Play alerts and sound effects through this device" and then that meant all 3 options were now greyed out and activated against that device. (Sound input, output, and alerts/effects).
Problem fixed!