My Delete key won't work

I'm having difficulty with the delete key. It won't work unless I highlight what I need to delete. But if I just want to delete an individual character without highlighting it, it will not work. What should I do?