I think you are missing the point here. There is no known problem with the keyboard and it's functionality on the iPhone. You do not need to remove the keyboard to fix it. I have it installed and use it, it does not make symols appear when using other English keyboards. If you are seeing symbols entered unintentionally there is a problem with your iPhone's software. To see symbols you must add the Emoji keyboard to the keyboard list. Even then you must select this keyboard when typing a message and physically select the icon you want. It may be possible if you have a different keyboard selected, other than US English, that has some built in short cuts for symbol. The US English keyboard does not.
If you jailbroke and modified the iPhone's software in any way it may have messed up the keyboards functionality. I suppose any app you installed to enhance keyboard function may have effect it, but I don't believe Apple would allow it intentionally. Any chat app you use may come with it's own keyboard, changing the system keyboards will probably not fix them. Restore will fix any software issues the iPhone may have although if caused by an app it may come back when it is installed. Restore will erase media files which are not included in the iPhone backup files.
Discussing jailbreak procedures of iPhones here is prohibited and I do not recommend it to fix any issue. I only mention it since some people have problems with iPod/iPhones after they have jailbroke them but fail to mention it in their initial post. It gets futile trying to help someone with a very rare issue such as this and after many attempts to fix it you are told the iPhone is jailbroke which caused the problem.
Again the best fix here is to Restore it back to a clean iOS install using a back up. If it still does it Restore it as new and add apps till you see the problem appear.