Which iPod shuffle model do you have? This document shows the four types
The 1st and 2nd gen shuffle works a bit differently from 3rd and 4th gen.
Also, if 3rd or 4th gen, have you been using automatic syncing to load the shuffle, or have you been using the manual method (drag and drop songs on shuffle in iTunes)?
Assuming it's the current 4th gen (or 3rd gen), and you are loading it manually, this is how to remove songs... If the iTunes sidebar (along left side of window) is hidden, from the iTunes menu bar, under View, select Show Sidebar. The shuffle appears in the sidebar, under DEVICES. Click it there to select it.
Click on the small triangle to the left of the shuffle (in the sidebar) to drop down its content list, indented under the shuffle. Click on Music there. To the right, the list of songs currently on the shuffle is shown. Select the song you want to remove and press Delete on the keyboard. You can select more than one song at the same time.
Please post back if your shuffle is 1st or 2nd gen, or you are using automatic syncing to load the shuffle.