iMessage not working between just one person and me?

My boyfriend and I both have iPhones; I have a 5 and he has a 4. The past few weeks we have not been able to iMessage only each other. We both can iMessage anyone else in our contacts and are able to send and receive them just fine, but we have not received any of each other's messages. He changed his messages settings so that he could be reached by e-mail as well and I was able to receive and send iMessages to/from his e-mail address, but not from his cell phone number. We have both tried turning iMessage on and off several times and resetting our phones. Normally I wouldn't care about not being able to text someone very often but he is in the military so that is our main form of communication. Any help would be appreciated; thanks!