First off, since it is an allergy to your skin, get that checked out. This could be a sign of a more serious medical issue.
Second, keep it in the case until you get your skin tested.
Third you might want to go to Apple (if available) after the first two have been done. If you have to handle the phone out of the case, use gloves. Explain the situation to them. Whatever happens with Apple, I don't know. If it was me, and I was with Apple, even though cosmetic damage is not covered, I would exchange it, because it's a medical issue and there is no damage that was user caused. I would go in with a doctors letter explaining this. They may not exchange it, but then again.
Fourth, do not take it out of the case unless you have to.