Q: Unknown data usage early morning

Eddie Strauss wrote:
What a fantastic post!

Agreed. I love Mr. Finch's posts. He beautifully summarizes and clarifies complicated dicussions with disparate issues. There should be a "Finch Summary" on every long thread.

How are you folks getting this level of detail? Is it on the printed bill or only online?

I'm not an AT&T customer. I wanted to see if I was experiencing similar noctunal emmissions of data, but Rogers (my carrier in Canada) tells me there is no way to get this level of detail for data usage. The closest we get is on the monthly bill, which gives a total for each day. There is also an online app (and an iPhone app) which gives cumulative unbilled totals, but no detail like what you're showing. Too bad.

AT&T prints data usage on the bill, apparently summarized to some degree, and also shows it on their web site in near real time (well, a few hours after the fact). AT&T also has an iPhone app that shows the same information, and an SMS query that will report total to date for the month.

I just looked at my AT&T bill for last month and my blood pressure is probably 20 points higher as a result.

After being warned that I had used 90% of my allotted usage with two weeks still left in my billing cycle, I practically quit using any data at all to avoid going over. Just two days before the billing cycle ended, AT&T recorded an unexplained transfer of 142,808KB at 2:46AM on a Sunday. That put my total for the month at 2,102,913KB. To answer the question asked earlier, they absolutely charged me the extra $10.00 for that 102,913KB extra data. I need to calm down before I call them again so I can present a rational argument and try to get some answers. Not sure what good it is going to do because I never got a call back from "Vanessa" who promised to research the problem last week.

When my new billing cycle started on Aug 4th, I reset the counters on my iPhone so I could more accurately compare it to what AT&T was recording. I have also been keeping a detailed log of my usage and comparing it to what shows up on the AT&T web site. So far in the first week my iPhone reports 24.8MB of data transferred which corresponds with what I would expect from my logged usage, but AT&T's web site claims that I used 73.15MB. The difference appears to be the total of two huge unexplained transfers (one for 32850KB and the other for 15411KB).

The only new observation now is that the unexplained transfers happened at about 10:00AM instead of 2:00AM. I guess AT&T realized that doing the transfers in the middle of the night was too obvious and is now trying to hide them in amongst all of the other transfers that occur during the day.

<Edited by Host>

chcn wrote:

Lawrence Finch wrote:
The only way to force it to be on WiFi is to leave the screen on. Once the screen goes off WiFi goes off. So if you turn off cellular data you will receive or send no data when the phone is on standby. No cellular, no WiFi.

Apparently if you don't have a 3G connection (out of range, SIM card removed, etc., or maybe if the cellular data is turned off) then iOS 4 will persist the WiFi connection even when the screen is locked. (It may also be a requirement that the phone be plugged in; I haven't tested this myself.) See:

http://forums.macrumors.com/showthread.php?t=936074
http://discussions.apple.com/thread.jspa?threadID=2470304

Definitely true. You can test this yourself if you are able to get the IP address of your phone. There are multiple ways of doing this - your WiFi router usually has a page in the admin console that tells you what devices are connected and what their IP addresses are. There are also iPhone apps that tell your what your WiFi and 3G IP addresses are.

Anyway, once you get your WiFi IP address, get to a computer on your WiFi network and try this.

First, open some form of command line interface (in Windows, run the command 'cmd', in Mac OS X use Terminal). Type 'ping <ip address>' and hit enter. You should see something that looks like this:

PING 10.2.204.130 (10.2.204.130): 56 data bytes
64 bytes from 10.2.204.130: icmp_seq=0 ttl=63 time=55.623 ms
64 bytes from 10.2.204.130: icmp_seq=1 ttl=63 time=86.491 ms
64 bytes from 10.2.204.130: icmp_seq=2 ttl=63 time=100.350 ms
64 bytes from 10.2.204.130: icmp_seq=3 ttl=63 time=23.572 ms

(10.2.204.130 is my phone's current address - yours will differ and so will the timings). All this does is that it sends small amounts of data to your phone asking your phone to reply if its there, and these lines mean that your phone replied.

Now, with cellular data on, lock your screen and wait a couple of minutes - it seems the amount of time it takes to disable the wifi varies. Sometimes its as little as 15s, but other times its longer. If you run the same command as before, you should now see this:

Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5

What this says is that your computer is no longer hearing back from your phone - i.e. your phone isn't on the WiFi network anymore.

Now, if you unlock your phone again and turn cellular data off and repeat this test, you should find that your phone will still respond to your computer even after it has been turned off for a while.
PING 10.2.204.130 (10.2.204.130): 56 data bytes
Request timeout for icmp_seq 0
64 bytes from 10.2.204.130: icmp_seq=1 ttl=63 time=80.645 ms
64 bytes from 10.2.204.130: icmp_seq=2 ttl=63 time=58.475 ms
64 bytes from 10.2.204.130: icmp_seq=3 ttl=63 time=82.140 ms
64 bytes from 10.2.204.130: icmp_seq=4 ttl=63 time=104.099 ms
64 bytes from 10.2.204.130: icmp_seq=5 ttl=63 time=25.705 ms

For this to happen, your phone doesn't need to be plugged in. I haven't tested the case where the phone is plugged in and cellular data is on.

These tests were all run on an iPhone 4 so YMMV if you have a different model - I don't know the exact differences.

Message was edited by: Lawrence T.

Message was edited by: Lawrence T. <added quote of message I'm replying to>

At last, someone with an experiment that makes sense. Thanks, Lawrence T!

So here's the next part of the story which might be more interesting.

First, for the folks who want to try these out but use a Windows machine, you want to use the command "ping -t xxx.xxx.xxx.xxx" where xxx.xxx.xxx.xxx is your ip address. If you don't use -t, your system will issue 4 pings at 1 second intervals and then stop. What we want though is for your system to keep issuing pings at 1 second intervals until YOU choose to stop it. And of course, to stop the pings, hit Ctrl-C.

First, the executive summary. In my earlier post, I outlined an experiment that shows an iPhone 4 turning its WiFi off after being asleep for some amount of time. Any communication with the phone will then happen over cellular data (and use your data plan). My next experiment (which had to wait until I got home because I didn't have my cable with me at work) was to repeat this with my phone plugged into my laptop and charging. It seems that as long as your phone is charging, the WiFi radio stays on the entire time your phone is asleep (its been about 5 minutes and my pings are still going strong).

Here are some details. I started off with cellular data off and my phone asleep. I started pinging the phone (and left it to run for the entire duration of this test).

64 bytes from 192.168.1.5: icmp_seq=0 ttl=64 time=70.692 ms
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=195.132 ms
64 bytes from 192.168.1.5: icmp_seq=2 ttl=64 time=113.573 ms
64 bytes from 192.168.1.5: icmp_seq=3 ttl=64 time=136.391 ms

No surprises there. I then turned the phone on, and turned 3G and cellular data both on (they were initially both off). Now, after performing this step it seems to be necessary to wait some time for your phone to get its 3G connection set up (something on the order of a minute or so). There are apps that tell you when you've acquired an IP address for your 3G connection (meaning you've successfully connected), or you can always turn the WiFi off and wait for the 3G icon to appear before turning WiFi back on again.

Now, you should have your phone with both WiFi and 3G on (and connected to the charger). If you look at the terminal running the pings, you should see no interruption in the pings. Now, disconnect your charger. Your phone should automatically wake from sleep - put it back to sleep or let it sleep automatically. After 15s (or maybe longer depending), your phone should drop its WiFi connection!


64 bytes from 192.168.1.5: icmp_seq=285 ttl=64 time=193.378 ms
64 bytes from 192.168.1.5: icmp_seq=286 ttl=64 time=411.805 ms
--------- WiFi goes byebye at this point ----------
Request timeout for icmp_seq 287
Request timeout for icmp_seq 288
Request timeout for icmp_seq 289
Request timeout for icmp_seq 290
Request timeout for icmp_seq 291

Now, plug the cable back into your phone. It should wake from sleep (and start syncing if you've plugged it into your computer). Your pings should start getting responses again as well:

Request timeout for icmp_seq 341
Request timeout for icmp_seq 342
Request timeout for icmp_seq 343
-------- Plugged back in and WiFi comes back on ------------
64 bytes from 192.168.1.5: icmp_seq=344 ttl=64 time=48.133 ms
64 bytes from 192.168.1.5: icmp_seq=345 ttl=64 time=73.917 ms
64 bytes from 192.168.1.5: icmp_seq=346 ttl=64 time=197.389 ms
64 bytes from 192.168.1.5: icmp_seq=347 ttl=64 time=47.084 ms

Now, wait for the phone to go to sleep. You should see that the pings continue uninterrupted even after the phone goes to sleep, which means that your WiFi is still on even though you did not turn cellular data off.

Having come to this happy conclusion, there remains some work to be done. First, its not clear the the WiFi radio is just taking a longer time to turn off - it might turn itself off after 15 minutes (or 30 minutes or 3.142 hours - its anyone's guess). What we all really care about though is having the WiFi radio stay on while we're asleep. So experiment no. 1 is to leave your phone charging, verify that the radio is still on after 5 minutes or so, KILL THE PINGS (so that there's no activity on the network going to the phone), and try pinging your phone again in the morning.

Experiment no. 2 is a little simpler - I'm going to redirect my pings to a log file and run the pings overnight just to confirm that this method at least keeps the WiFi radio on the entire night. If nothing else, this will guarantee that one method for keeping your WiFi on the entire night is to keep your phone plugged in and have a ping active.

Just because experiment no. 2 is easier and arguably more useful, I'm going to do that one first. I've also reset my usage counters, so we'll see what they say tomorrow morning. If nothing else, this could potentially prove that your phone uses the cellular network to send some data even though the WiFi radio is provably active.

Message was edited by: Lawrence T.

Here is a more interesting result. iPhone off charger, cellular data off, mail fetch set to manual, i.e. most conservative setting. Lock the phone and leave it for about 30 minutes. Then ping it:

$ping 172.16.0.250
PING 172.16.0.250 (172.16.0.250) 56(84) bytes of data.
From 172.16.0.1 icmp_seq=2 Destination Host Unreachable
From 172.16.0.1 icmp_seq=3 Destination Host Unreachable
From 172.16.0.1 icmp_seq=4 Destination Host Unreachable
64 bytes from 172.16.0.250: icmp_seq=6 ttl=64 time=154 ms
64 bytes from 172.16.0.250: icmp_seq=7 ttl=64 time=76.8 ms
64 bytes from 172.16.0.250: icmp_seq=8 ttl=64 time=95.0 ms
64 bytes from 172.16.0.250: icmp_seq=9 ttl=64 time=123 ms
64 bytes from 172.16.0.250: icmp_seq=10 ttl=64 time=46.4 ms
64 bytes from 172.16.0.250: icmp_seq=11 ttl=64 time=67.5 ms
64 bytes from 172.16.0.250: icmp_seq=12 ttl=64 time=90.7 ms
64 bytes from 172.16.0.250: icmp_seq=13 ttl=64 time=112 ms
64 bytes from 172.16.0.250: icmp_seq=14 ttl=64 time=32.4 ms
64 bytes from 172.16.0.250: icmp_seq=15 ttl=64 time=55.3 ms
64 bytes from 172.16.0.250: icmp_seq=16 ttl=64 time=79.8 ms
64 bytes from 172.16.0.250: icmp_seq=17 ttl=64 time=105 ms
64 bytes from 172.16.0.250: icmp_seq=18 ttl=64 time=21.0 ms
^C
--- 172.16.0.250 ping statistics ---
18 packets transmitted, 13 received, +3 errors, 27% packet loss, time 17015ms
rtt min/avg/max/mdev = 21.010/81.688/154.264/36.288 ms, pipe 3

And here is /var/log/messages:

Aug 12 00:36:14 localhost dhcpd: Wrote 0 deleted host decls to leases file.
Aug 12 00:36:14 localhost dhcpd: Wrote 0 new dynamic host decls to leases file.
Aug 12 00:36:14 localhost dhcpd: Wrote 10 leases to leases file.
Aug 12 00:36:14 localhost dhcpd: DHCPREQUEST for 172.16.0.250 from f8:1e:df:xx:xx:xx (Lightsp33d-iPhone) via eth1
Aug 12 00:36:14 localhost dhcpd: DHCPACK on 172.16.0.250 to f8:1e:df:xx:xx:xx (Lightsp33d-iPhone) via eth1

Which shows that the interface uses a Wake-On-LAN feature. That makes most sense, as power conservation is very important.

I expect something similar with cellular data turned on, although I haven't tested it... yawn. I don't care to test it with the phone plugged into the charger, since it can then do whatever it pleases, as there is power readily available.

Ah yes, here is the result with cellular data on, wifi on, off charger, mail fetch set to manual:

$ping 172.16.0.250
PING 172.16.0.250 (172.16.0.250) 56(84) bytes of data.
From 172.16.0.1 icmp_seq=2 Destination Host Unreachable
From 172.16.0.1 icmp_seq=3 Destination Host Unreachable
From 172.16.0.1 icmp_seq=4 Destination Host Unreachable
From 172.16.0.1 icmp_seq=6 Destination Host Unreachable
From 172.16.0.1 icmp_seq=7 Destination Host Unreachable
From 172.16.0.1 icmp_seq=8 Destination Host Unreachable
64 bytes from 172.16.0.250: icmp_seq=10 ttl=64 time=46.9 ms
64 bytes from 172.16.0.250: icmp_seq=11 ttl=64 time=73.6 ms
64 bytes from 172.16.0.250: icmp_seq=12 ttl=64 time=91.7 ms
64 bytes from 172.16.0.250: icmp_seq=13 ttl=64 time=118 ms
64 bytes from 172.16.0.250: icmp_seq=14 ttl=64 time=39.6 ms
64 bytes from 172.16.0.250: icmp_seq=15 ttl=64 time=64.4 ms
64 bytes from 172.16.0.250: icmp_seq=16 ttl=64 time=85.3 ms
^C
--- 172.16.0.250 ping statistics ---
16 packets transmitted, 7 received, +6 errors, 56% packet loss, time 15008ms
rtt min/avg/max/mdev = 39.651/74.339/118.543/25.150 ms, pipe 3

As predicted. And I see the same dhcp renewal in the messages log, as in my previous post.

I'm not completely sure what is happening yet, but this is certainly not Wake on LAN (or Wake on Wireless LAN). Wake on LAN and its variations are technologies that allow a _specially crafted_ packet to wake a sleeping machine. Your pings certainly don't qualify as specially crafted packets.

What's more likely and possible is that the phone power cycles the WiFi radio so it spends part of its time in an active state and part of the time in a sleep state. I don't have enough information to make any conclusions about this yet.

Will this lead to an answer to *my question*: *"What is in the large data transfers, where is it going and to whom?"*

Is it possible that the iPhone responds to the "Wake on Demand" feature of the new Airport Extremes?

brm123 wrote:
Will this lead to an answer to *my question*: *"What is in the large data transfers, where is it going and to whom?"*

Probably not. This was more an attempt to figure out either way whether data goes out over WiFi or your cellular connection when your phone is locked (and possibly ways to make sure data goes out over WiFi as much as possible).

To answer your question about the large data transfers, its probably best answered using a jailbroken phone and tcpdump on the phone. Alternatively, if you force WiFi and find that the phone is sending unknown large transfers over your WiFi network, you can set up a separate computer to record those packets. Might not tell you what's being sent, but will almost certainly tell you where those packets are going to.

*It seems there is an effort underway to divert this topic from the original question:*


*"What is in the large data transfers, where is it going and to whom?"*


*If you are experiencing and being charged for the unknown large data transfers by AT&T and still seeking an answer please chime in.*

rentamark wrote:
Is it possible that the iPhone responds to the "Wake on Demand" feature of the new Airport Extremes?

It's possible that this is something similar, but probably not that exact feature. The feature you refer to uses bonjour to communicate with sleeping machines - http://www.macworld.com/article/142468/2009/08/wakeondemand.html

Note that the article also mentions WMM power save, which is "a standby scanning mode that wakes a radio for extremely brief periods of time to scan for incoming traffic". It seems entirely possible that the iPhone uses this technology. I don't have any good answers for this yet.