In Itunes advanced preferences, do you have "Keep iTunes Media folder organized" and "Copy files to iTunes Media folder when adding to library." checked?
If so, then your folders and songs are in one place and you will need to manual remove your other folder and files scattered accross your hard drive. For XP this is typically,
C:\Documents and Settings\username\My Documents\My Music\iTunes\