Tuesday, April 13, 2010

Logcat, Improved.

Logcat is a staple for most Android developers out there and I'm certainly no exception. I often have at least one terminal dedicated to logcat with sometimes many more with various combinations of options to control how I'm filtering the output.

Recently it occurred to me that most of this work is designed to separate my program from the noise of the entire platform. The numeric argument printed after the tag in the logcat output is the pid responsible for that log line which I had used in the past to do this sort of filtering but it was a pain when the app crashed or was reinstalled because the pid would change.

Enter my proclogcat script. This script tracks the pid as the process is killed and restarted and takes care of automating the adb shell ps | grep <process> logic on first launch. The best part is the script can be combined with Jeffrey Sharkey's excellent coloredlogcat script (or my modified version of it) for beautiful results.



Download: proclogcat

To use, simply copy it somewhere in your PATH and invoke either manually as adb logcat | proclogcat <process> or in a function as is discussed in the script source code.

Tuesday, December 15, 2009

Gracefully supporting multiple Android platform versions in a single release

The Android platform has been aggressively updating since version 1.0 and now we're starting to a see a much more interesting mix of device types, manufacturers, and even platform versions out in the wild. Unfortunately sometimes this can be frustrating for developers wanting to look forward to support new features and conveniences, but to still support devices that are on longer update cycles (like with the G1).

The pattern shown here will deal with multiple platform versions although can easily be applied in other situations. First of all, let's start with a preface about minSdkVersion, targetSdkVersion, and the Eclipse target platform. The *SdkVersion attributes are defined in the manifest <uses-sdk> tag and define the minimum platform version your app can be installed onto (and tested on!), and the highest version that you tested to and were aware of during development. It is important that you test your application on all versions between and including min and target. The Eclipse target platform is the specific version that Eclipse will be compiling against, this is what permits us to compile code that actually does link specifically against the newer platform features. This is usually set the same as your targetSdkVersion.

Now let's consider a practical example of a music player application which needs to implement a service in the foreground state during playback. Prior to API level 5, this was done with the Service.setForeground call, but level 5 and beyond deprecated this method due to widespread abuse. Instead, a new method was introduced (Service.startForeground) which can be used to achieve this affect as well as setting an ongoing notification in the status bar. In many ways this is handy as the notification and foreground state were naturally already tied together, now there's an API combining them. But problems start when you try to test new code using this method on platform versions below 2.0 (API level 5). Specifically, Dalvik will throw a VerifyError when attempting to initialize the class containing the call to startForeground for the first time, even if the call is in a conditional statement. This method does not exist on pre-2.0 devices, and so cannot be included in your code in this way.

A naive approach would be to simply use reflection to test for and execute startForeground, but thankfully Java offers a much more elegant design pattern for just this sort of thing. The basic idea is to create an abstract API that the rest of your application can access which hides the specific implementation of what's being performed, and does so in such a way that prevents the VM from initializing an unsupported class on an older platform. So you might try defining something like this:


public abstract class PlayerNotification {
public static PlayerNotification getInstance() {
if (Integer.parseInt(Build.VERSION.SDK) <= 4)
return PreEclair.Holder.sInstance;
else
return EclairAndBeyond.Holder.sInstance;
}

public abstract void showNotification(Service context, int id, Notification notification);
public abstract void hideNotification(Service context, int id);

private static class PreEclair extends PlayerNotification {
...
}

private static class EclairAndBeyond extends PlayerNotification {
...
}
}


Now your service could be modified to make use of this new abstract API as such:


public MyService extends Service {
private static final int NOTIF_PLAYING = 1;
private final PlayerNotification mNotification =
PlayerNotification.getInstance();

...

public void setForegroundAndShowNotification(Notification n) {
mNotification.showNotification(this, NOTIF_PLAYING, n);
}

public void stopForegroundAndHideNotification() {
mNotification.hideNotification(this, NOTIF_PLAYING);
}
}


Great, this sounds very simple and easy to follow. Let's return to the full implementation of PlayerNotification:


private static class PreEclair extends PlayerNotification {
private static class Holder {
private static final PreEclair sInstance = new PreEclair();
}
private NotificationManager getNotificationManager(Context context) {
return (NotificationManager)context.getSystemService(Context.NOTIFICATION_SERVICE);
}
public void showNotification(Service context, int id, Notification n) {
context.setForeground(true);
getNotificationManager(context).notify(id, n);
}
public void hideNotification(Service context, int id) {
context.setForeground(false);
getNotificationManager(context).cancel(id);
}
}

private static class EclairAndBeyond extends PlayerNotification {
private static class Holder {
private static final EclairAndBeyond sInstance = new EclairAndBeyond();
}
public void showNotification(Service context, int id, Notification n) {
context.startForeground(id, n);
}
public void hideNotification(Service context, int id) {
context.stopForeground(id);
}
}


And that's it as far as code goes! Assuming that you have already updated your AndroidManifest.xml to include the appropriate <uses-sdk> attributes, you're ready to start testing. Use the Android SDK tools to create AVDs for each of the major platform releases from your minimum supported version to your current target and deploy your app on each to make sure you have not made any mistakes.

For further reading about how Java guarantees this approach, read about the initialization on demand holder idiom. This is what allows us to prevent the wrong implementing class from initializing in the VM (and thus causing verification errors).

You can find two working examples of this pattern in my Five app: one using reflection and one matching the explained example

Tuesday, March 17, 2009

Building, running, and debugging Android source

There is a lot of confusion surrounding the work flow in the Android source tree, so allow me to simplify:
  1. Follow the initial instructions for downloading the source at:

    http://source.android.com/download

  2. Set up your environment to build the engineering build for the generic device and generic product. This is similar to the SDK, but with a few pieces missing.

    $ source build/envsetup.sh
    $ lunch 1

  3. To build for the first time:

    $ make

    If you have a multi-core system, you can build with make -jN where N is twice the number of cores on your machine. This should speed up the first build considerably.

  4. To launch the emulator from your build:

    $ ./out/host/<your-machine-type>/bin/emulator

    On my system <your-machine-type> is linux-x86.

    NOTE: The emulator knows where to find system and data images as a result of running lunch 1 above. This sets the environment variable ANDROID_PRODUCT_OUT to point to the target directory. For this example, it should be out/target/product/generic/.

  5. If you wish to make changes to the source code, there are handy utilities that have been exposed to your environment by source build/envsetup.sh above. For example, if you modify the Email app and just want to rebuild it:

    $ mmm packages/apps/Email

  6. To see your changes in the emulator you can run:

    $ adb remount
    $ adb sync


    Which will copy the regenerated Email.apk file into the emulator's /system/app folder, triggering the PackageManager to automatically reinstall it.

  7. Or if you change framework resources in frameworks/base/core/res/res/ you could regenerate framework-res.apk with:

    $ mmm frameworks/base/core/res

    Or if you modified even the framework itself you could run:

    $ mmm frameworks/base

    To sync these changes you must restart the running framework and sync, as with this handy sequence:

    $ adb remount
    $ adb shell stop
    $ adb sync
    $ adb shell start

  8. Finally, to debug your changes you can use the DDMS tool to select a process for debug and then attach Eclipse to it. If you have the Eclipse Android Development plugin installed, there is a special DDMS perspective which you can use to choose the process for debug. To attach Eclipse to it, see these instructions:

    http://source.android.com/using-eclipse

    This document also describes how to use Eclipse for development. Any IDE should work with the proper finagling though. Just note that the IDE won't really be an integrated environment: the final output of APKs, system.img, and even the generation of R.java files will have to be done by make!

    A note about the processes in Android:

    • system_process houses all things under frameworks/base/services. This includes the PackageManagerService, StatusBarService, etc. It has many, many threads (one for each service, and then one main UI thread), so be wary when debugging.
    • com.android.acore hosts Launcher (home), Contacts, etc. You can determine the apps/providers that run here by looking for android:process="android.process.acore" in the various AndroidManifest.xml files in packages/.

    Also remember that the "framework" (under frameworks/base/core/java) is not hosted by any one process. It is a library used by most processes, so to debug code there you can usually use a simple demo app that takes advantage of whatever you changed and debug that app's process. A useful trick for setting up your debug connection is to call Debug.waitForDebugger() during some startup part of an application or system service.

UPDATE 2009-07-24: The original ONE_SHOT_MAKEFILE line I gave for rebuilding the framework has been deprecated. mmm frameworks/base is now the recommended way to rebuild the framework code.

Wednesday, January 7, 2009

Push services: Implementing persistent mobile TCP connections

As a result of my work on IMAP IDLE support in Android's default mail application, I have been experimenting with various strategies for implementing long-lived services and persistent connections that operate efficiently in a variety of circumstances. Several quirks about Android and mobile devices in general arose that could be of value to anyone implementing similar services.

For most protocols, you will need to implement some type of client-initiated keep alive at the application layer. For my purposes with IMAP I simply complied with the RFC and elected to leave IDLE mode then re-enter after 28 minutes of inactivity. On Android, you must use the AlarmManager service to wake the CPU for this task. You might be tempted to use a Handler for timing or even a simple thread with a looped sleep() however it should be noted that unless your application otherwise holds a WakeLock you cannot rely on any timing mechanism other than the AlarmManager. Once the screen goes blank, the CPU may sleep and once it does other timing mechanisms will block until the CPU wakes up again, regardless of any timeout paramters you supply.

After running my test for several days I noticed Android was mysteriously killing processes, claiming that the services implemented in them have "died", then restarting them just a few minutes later. No call to the service's onDestroy method will occur, and even on service restart you will only see a call to onCreate and not onStart. In order to compensate for this you are expected to store your state persistently and check for a discrepency during onCreate and then invoke startService for yourself if necessary. The SharedPreferences system can be handy for this.

Source code for a functional demonstration on this topic can be found at my android-random project page, under the module TestKeepAlive.

Thursday, October 23, 2008

Working on IMAP IDLE support for Android's Email app

For all those screaming about the lack of push e-mail updates for regular IMAP accounts with the basic Email app in Android, I have been digging into the code for the past few days and am preparing a patch for IMAP IDLE support. For those that do not know, IMAP IDLE allows a client to maintain a persistent, light connection to the server that is then notified of new messages as they arrive. Many IMAP servers support this extension, including Microsoft Exchange which would thus allow push e-mail for those that need corporate e-mail options on the G1.

I will post an APK for folks to try once I have the support working well. Stay tuned...

Wednesday, August 27, 2008

Generate Callback Listener Helpers from AIDL

As I redesign Five I find myself tweaking/expanding the callback listeners in my interface which are handled through the RemoteCallbackList class in the service implementation. This class is great, however for each call code must be written similar to:

int N = mCallbacks.beginBroadcast();

for (int i = 0; i < N; i++) {
try {
mCallbacks.getBroadcastItem(i).onSomeEvent();
} catch (RemoteException e) {}
}

mCallbacks.finishBroadcast();

This can quickly pollute the service implementation, and even if you factor these calls out to a separate class it can be a pain to create and maintain them. So, I came up with a solution in the form of a Perl script which takes as input an AIDL file defining the callback interface and outputs a class implementing the extended RemoteCallbackList. To use, simply type:

$ ./aidl-cbliststub.pl < IFooListener.aidl > IFooListenerCallbackList.java

Download: aidl-cblistsub.pl

Monday, August 25, 2008

Android Instrumentation Example

As Android approaches maturity with its recent 0.9r1 release, I began to think its time that my main project updates to match. I decided to take another look at Android's instrumentation and unit testing features to build robust tests for my critical services and activities. Unfortunately, there isn't anything new in the way of documentation however there is now is an example we can use in ApiDemos.

Looking at the new ApiDemos we see that there is a new tests directory which contains a second AndroidManifest.xml with no user activities defined. This is important as this structure allows us to have two separate APKs, one for production and one for testing, so our main distribution and build doesn't need to be polluted. Unfortunately, this approach is not compatible with the Eclipse plugin, and so we must define everything in an external build environment.

For convenience, I packaged the complete ApiDemos project with my Ant build environment here: ApiDemos-instrumentation-build.tar.gz. To build, start the emulator and run the following commands:

$ adb shell rm /data/app/ApiDemos.apk
$ ant install
$ cd tests && ant install

It is essential above that you remove the old ApiDemos.apk as the instrumentation APK must be signed with the same key as the package it tests.

Once we've got both APKs installed we can begin running our unit tests. There is no UI in Android for this, however it can be invoked more conveniently through adb as such:

$ adb shell am instrument -w com.android.samples.tests/android.test.InstrumentationTestRunner

This will run all defined tests and print a dot character for each successful test, F for failure, or E for invocation error. See the javadoc in tests/src/com/android/samples/AllTests.java for more sample command lines to run individual test cases or specific tests within them.

My main project, Five, has recently been updated to include unit tests in the five-client component, with more test coverage to follow in the next few weeks. For a non-trivial test case, see my CacheServiceTest.

NOTE: Previously, I was using the masa Android plugin for Maven, however it has not yet been updated to support 0.9r1. I feel that using Maven for this would have been cleaner, but the Ant approach is sufficient for this simple demonstration. Once updated, I will return here and finish this example with a more sophisticated build environment to automate testing.