How do they show in the Music app on your device ? If they have the cloud icon against them then they are in the cloud - you can stop them showing via Settings > iTunes & App Store > Music (under the 'show all' heading) 'off' (or Settings > Music > Show All Music 'off').
If they are stored on your device, and you can't delete the tracks via Settings > General > Usage nor directly in the Music app, then try closing the Settings and Music apps via the device's taskbar and do a soft-reset and see if you can delete them after the device has restarted. Or see if you can delete them by connecting the device to you computer's iTunes and selecting the My Music section under 'on my device'
Close an app : Force an app to close in iOS - Apple Support
Soft-reset (i.e. a reboot) : press and hold both the sleep and home buttons for about 10 to 15 seconds (ignore the red slider), after which the Apple logo should appear and it should reboot.