That shouldn't be happening. Try force closing the Calendars app by double-tapping the home button, locating Calendar, swiping up on the image above the app icon to close it, the tapping the home button again.
If it's still acting up, try going to Settings>iCloud, turn Calendars off, choose Delete from My iPhone when prompted (they will still be in iCloud), then after a few seconds turn Calendars back on again.