It sounds like you are trying to manually manage the iPod's contents when you are already have it set up to automatically sync content from your iTunes library. If you want to be able to manually control what controls you add and remove from the device, enable the Manually manage music option from under the iPod's Summary tab in iTunes.
See here for more on manual managemenet.
Otherwise, if you want to continue to automatically sync your device with your library, you must configure the appropriate sync settings from under the iPod's Summary and Music tabs to no longer sync the content you wish to remove from the Nano. For example, if you have the Sync only checked songs option from under the Summary tab, make sure the music you want to sync to your Nano, has a tick mark next to it in your main iTunes library. If you don't want it on the Nano anymore, remove the tick mark and resync your iPod.
As for the issue of having a larger iTunes library than what your Nano is capable of holding, see this article for a few suggestions.