Wednesday, January 7, 2009

Push services: Implementing persistent mobile TCP connections

As a result of my work on IMAP IDLE support in Android's default mail application, I have been experimenting with various strategies for implementing long-lived services and persistent connections that operate efficiently in a variety of circumstances. Several quirks about Android and mobile devices in general arose that could be of value to anyone implementing similar services.

For most protocols, you will need to implement some type of client-initiated keep alive at the application layer. For my purposes with IMAP I simply complied with the RFC and elected to leave IDLE mode then re-enter after 28 minutes of inactivity. On Android, you must use the AlarmManager service to wake the CPU for this task. You might be tempted to use a Handler for timing or even a simple thread with a looped sleep() however it should be noted that unless your application otherwise holds a WakeLock you cannot rely on any timing mechanism other than the AlarmManager. Once the screen goes blank, the CPU may sleep and once it does other timing mechanisms will block until the CPU wakes up again, regardless of any timeout paramters you supply.

After running my test for several days I noticed Android was mysteriously killing processes, claiming that the services implemented in them have "died", then restarting them just a few minutes later. No call to the service's onDestroy method will occur, and even on service restart you will only see a call to onCreate and not onStart. In order to compensate for this you are expected to store your state persistently and check for a discrepency during onCreate and then invoke startService for yourself if necessary. The SharedPreferences system can be handy for this.

Source code for a functional demonstration on this topic can be found at my android-random project page, under the module TestKeepAlive.


Justin (Google Employee) said...

You comment that onDestroy is not called when your Service is killed, and this is completely consistent with the Service lifecycle. You can not count on onDestory being called for anything. You can count on onStop being called. For a service you should consider it dead after it returns from onStop. For an Activity you should consider it dead after onPause is called.

Android Team @ Google

Daniel said...

I can see how your method keeps the TCP/IP session alive, but does it let you get asynchronous incoming data from the socket if the phone goes to sleep? Meaning, once the phone sleeps, before your 28 minute wake-up timer, will your Service get woken up in order to handle incoming data?

I'm working on K-9 email, and am interested in the various approaches to checking IMAP servers for new mail.

Chris said...


It's possible to set a PARTIAL_WAKE_LOCK with PowerManager.

Although, I'm not sure of the effect it would have on battery.

Daniel said...

Using a WakeLock, like I added to DAmail and K-9 for use with checking email while asleep, would have a huge impact on battery life. If the G1 is kept on using a WakeLock for ever, to allow IMAP IDLE traffic to come through at any time, the phone would not run long enough to be useful. The hard part, indeed, is to support IMAP IDLE without using a 100% duty cycle WakeLock.


Josh Guilfoyle said...

You do not need to do any special work to wake up when data is received on the socket. Even while sleeping, the radio is periodically listening for network events. If it didn't do this, it wouldn't make a very good phone :)

You will not require a WakeLock.

Daniel said...

I have implemented IMAP IDLE in K-9. Thanks for the pointers on how to handle the persistent connections.

It is not production release ready, but definitely works for receiving new mail via IMAP IDLE. I still need to handle the untagged FETCH messages to handle deletes and read status changes, and a few other cases.

Please see Issue 4 in K-9's Google Code site for details. The work is ongoing in the /k9mail/branches/issue4-1.X branch.

Mohsin said...

i am facing an problem?

server (web server) wants to send data( or some information) to mobile client (android based phone).
means simply notify to start work, then mobile start working on specific application.

i would like to do a SMS from server to mobile, to start working. buts its not the best idea.
as we can assign too an IP to android mobile phone in case of WiFi or USB connectivity.
but phone will use GPRS or GSM, so there is no unique IP of Mobile so that we can access mobile phone using Socket Programing
so there is any idea or tutorial for this mention problem.

dont give the example that GMAIL, or black beryy etc using these technology, actually tell me how technically its work
waiting for your good replies

tobsen said...

Great blog entry, really informative! I wonder though why google doesn't describe in more detail the way they push their gmails to the phone.

Dave said...

I am trying to implement a type of push service similar to IMAP using your example as a basis. What I have found is the ConnectionThread that is in the loop never runs when the phone goes to sleep. You mentioned in your comments that there is no need to use a WakeLock. I'm either doing something wrong or missing something.

Dave Arnold
Tacoma, WA

NickD said...


Did you ever resolve this? I'm curious because it seems one needs to do something to have the application thread be awake to handle the received data

Josh Guilfoyle said...

@NickD, nothing special is required to wake up to retrieve I/O that is being waited upon. The system will ensure that if data is available on the socket the CPU will awake and your app will have the opportunity to read it.

@Dave, I'm uncertain what could be wrong in your code but if you compile this sample program you will see that nothing special is required.

Erdal said...

Would it be possible to use this persistent tcp socket as the underlying socket for an HttpClient?

Maurice said...

Hi Josh,

Sometimes on the KeepAliveService I get a random The operation timed out. This happens on my Inputstream when it calls a read. I set a setSoTimeout to 30 mins like you suggested in the code but this timeout seems to be thrown much earlier.

This is followed by the cleanup at the bottom that just says socket not connected. This is bad for my server which handles threads listening to the client as it does not even throw the EOFException to do the cleanup on the server side. Any ideas?

Erdal said...

I noticed the use of System.currentTimeMillis() for scheduling the reconnects.

Would it be safer to use SystemClock.elapsedRealTime() instead? because this does not get affected by timezones changing?

Of course the alarm type would have to be changed to ELAPSED_REALTIME_WAKEUP.

Josh Guilfoyle said...

@Erdal, yes the System.currentTimeMillis() approach should not be used. You're correct that SystemClock.elapsedRealTime() is the appropriate replacement.

Also, re using this approach with HttpClient: there's actually no need. A properly configured HttpClient instance will re-use idle connections. The easiest way to do this safely is to use a static instance of your HttpClient along with the HttpClient#execute(HttpUriRequest, ResponseHandler) method. ResponseHandler is a convenient variant of execute that will ensure the stream is appropriately managed and released back into the pool of idle connections. Avoid calling HttpUriRequest#abort() unless you truly wish to close the underlying connection.