Currently Being ModeratedJan 21, 2014 3:15 PM (in response to Nick Montoya)
It should, though you may need to use your Apple ID rather than a phone number. I don't know of any U.S. carrier who does data only plans for an iPhone. International data roaming is also expensive. If you live in the U.S. and have an unlocked phone, see about getting a TMobile SIM. They have free roaming in a lot of countries.
Note that there are far fewer public WiFi spots in Japan than in the U.S. Hotels sometimes have WiFi in the lobby but generally not in the rooms. The rooms may have an ethernet connection. I carry an Apple Airport Express for connecting to the ethernet cables in hotel rooms.