The process named "screencapture" is normally only running when you have typed one of the special keyboard shortcuts (command+shift+4 and others) and terminates with either the press of the "esc" key (aka escape) or by actually doing the screen shot of the screen or window or area or whatever mode you told it to do with the shortcut command.
So, you're saying that you have a "screencapture" process visible listed in the "Activity Monitor" window and you can't kill it using the normal "x" icon in the upper left corner of the Activity Monitor window with that process highlighted?
On my High Sierra system, as soon as I type a screencapture shortcut command, e.g. command+shift+4 to capture a window, I see the screencapture process appear and the screen/mouse cursor changes to a bulls-eye, and I can't do ANYTHING on the system until either the escape key is pressed on keyboard or until I actually press the space key in a desired window and then click in that window to complete the screencapture and create the screencapture file. The only possible way I can think to actually try and kill that process would be to log in remotely from another system Terminal command and manually do a "kill -9" of the screencapture process id which you could obtain from the command:
ps -axwww |grep screencapture
So some things just don't quite make sense with your description. Are you on High Sierra version of macOS? You're not talking about the "Grab.app" program in the /Applications/Utilities folder window are you? Which exact screencapture keyboard shortcut are you using? Can you post what your Activity Monitor window shows for the screencapture process info?
Good luck...