It's called a "badge" and it shows that there is an important message for you in the app. If it's on the Settings app it probably means that some setting is incorrect, like the password on an email account or a bluetooth device that failed to connect.