Texts sending from email not phone number

I just got a new iphone and some of my contacts are receiving my text messages from my gmail and not from my phone number. The settings in my phone are:

• You can receive imessages to and reply from: both my phone number and email are checked, but my phone number is grey as in I cannot select or unselect that. If I unselect my gmail in this section then I cannot send texts as they say not delivered.
• Start new conversations from: my phone number is checked but not my email

It is only sending from my email to some of my contacts, not everyone. How do I fix this?