How are you currently trying to delete content from the device?
How you delete content from your iPod depends on how it is set up to receive content from your iTunes library whether that be via automatic syncing or manual management.
For automatic syncing, you must reconfigure your settings so that the album/tracks/content you wish to remove is no longer configured to sync to your iPod. For example, if the Sync only checked songs and videos option is enabled from under the iPod's Summary tab, make sure all the tracks for this album do NOT have a tick mark next to them in your iPod. Then resync your iPod with iTunes.
If that isn't the case, you'll want to look at the iPod's Music configuration tab. You can access this tab by selecting your iPod under Devices. This should bring you to the Summary tab over to the right. Look for the Music tab along the top and select it. Here you can configure how and what music syncs to your device. If you make any changes to this tab, be sure to hit the Apply button in the lower right hand corner to sync the updated changes to your device.
Otherwise, if you manually manage your iPod's contents, you must manually remove content from it. You can do this by plugging your iPod in and clicking the small black triangle next to your iPod from under the Devices section in the left hand pane of iTunes. This will open up a submenu of content that is indented underneath the name of your device, so go ahead and click on Music (or whatever content you are wanting to remove). Locate the track or tracks/content you wish to delete. Highlight them and hit the Delete key or choose Edit -> Delete.