How do I change the default keyboard to use a third party app?

I an using a 3rd party keyboard app, and I want my messages, mail, etc to default to that app. It looks like I can delete the English keyboard to let the new keyboard be the only one available, but if I have a problem with it I want to be able to get the original English keyboard back. Is that the right way to make the third party keyboard the default?