[Editor's Note: This is a guest post from Jenkins community member Tom Rini]

Alternatively: How to make your parallel jobs kick one last job at the end

Many of us have had occasion to think: "I could make this project build quicker if I could just run parts in parallel and then one final job to wrap it up."

Well, good news! Jenkins is here to help! With the Join Plugin you can do just that. Over on the confluence page it’s got a number of examples and fancy flow charts. But the take-away is that if you can describe the flow, you can make it happen. But you’re saying "wait, I need to pass information around between the jobs."

We’ve got that one covered for you too with the Parameterized Trigger Plugin. And here’s the best part, these two can work together! With both plugins installed you can follow the steps listed in the Build Parameters section of the Join Plugin.

And as they say, now you're cooking with gas!

In my capacity as Build Guy at Digg, I've written up a blog post on our new continuous deployment/code review/pre-tested commit workflow. We're using a combination of Hudson, Git and Gerrit, Selenium and more to make sure that every change going to Digg's new site has been thoroughly tested.

Read the whole post, with all the juicy details over on Digg's Technology Blog!

Over the past year Hudson has grown tremendously, both within the Java community and outside of it. Partially thanks to Titus Brown's PyCon 2010 Atlanta coverage of continuous integration for Python (which we've covered before), Hudson has made great strides within the Python community as well.

In my experience, the majority of Python developers are not using Hudson to build anything, unless they have C extensions, but rather to test their packages, which presents its own set of specific requirements for jobs. Jobs for testing Python code need to be able to reliaby reproduce an environment with the same set of dependencies from one run to the next in order to provide consistent testing. Unlike their Java counterparts, Python developers cannot rely on a powerful system like Maven2 for enumerating build/test targets or defining their project's dependencies in their jobs; fortunately, w e can have something close: virtualenv and pip.

Virtualenv does exactly what you might expect it to, it creates a "virtual environment" with custom site-packages directory, and modified python executable. Using virtualenv you can create a staged environment to use for running unit and integration tests. Adding pip alongside that and you have a fantastic Python package manager/installer to use with the virtual environment.

Below, I've outlined the steps required to use virtualenv and pip to automatically manage a custom environment for your Python jobs.

A few weeks ago I passed a job listing that I had found through one of my many Google Alerts for Hudson-related queries to Andrew (abayer), following up on one of those job listings Andrew recently signed an offer to join the nice folks over at Digg to be their resident "build guy." On its own I thought "great for Andrew!" and nothing more, then I saw this thread on reddit which poses the question:

Anyone here a build engineer, or part of the build team? Could you please share your experience?

It seems, to me at least, the notion of "release engineering" is making a come-back, particularly in the aging "Web 2.0" world where companies like Digg, Facebook, Reddit, Twitter, etc are anywhere from five to ten years old. As these companies have aged a couple of important things have happened, their code-base has aged increasing the possibility of bitrot, but they have also expanded in terms of headcount. Start-ups that might have once slighted larger corporations like Oracle, Cisco VMWare and IBM for their burdensome process and longer release schedules now find themselves ensnared with massive code bases, larger development teams and complicated deployments.

In this post I'll show a very basic tips on how to compile an Android project using Hudson. Specifically how I use Hudson to create release versions of my apps.

Debug vs Release

I'm assuming that you are using Eclipse with the ADT plugin. During development you can test your app on the emulator or a device and resources like R.java and aidl files are compiled for you automatically by the plugin. When it's time to release your app, you'll need to go through some steps:

• You must sign your app using a certificate
• You must update your AndroidManifest.xml to remove the android:debuggable attribute
• Replace your Google Maps API debug key with the one belonging to your release certificate (if you are using a MapView)
• Generate an apk package for the release and test it on a device or emulator

It would be nice to automate a few of these steps, and this is where Hudson comes in.

Automated builds: Ant

For automated builds the Android SDK uses Apache Ant, which Hudson has great support for. To generate a template build.xml you can use the android tool from the SDK using the following command: android create project -n template -t android-7 -p template -k dummy.pkg -a Dummy The target is specified as "android-7" meaning that we are building for Android 2.1. For apps that use MapView we would use "Google Inc.:Google APIs:7". It is a good idea to always target the latest SDK. From this template project we'll grab the build.properties and the build.xml and copy those to the Android project that we want to build. Edit build.xml and set the project name to your Android project name.

The local.properties file contains the path to the SDK root and shouldn't be checked in to version control. For our use we'll set the properties that are in that file on the Hudson job configuration page.

Running the build in Hudson

This part is easy: create a new freestyle job and let it be build with ant. The targets that we want to execute are clean release. Release will compile, package and sign your apk. Now to get this working right, some custom properties should be set (use the Advanced button).

sdk.dir=/Users/hugo/Code/android-sdk-mac target=Google Inc.:Google APIs:7 key.store=certs/rd-release.keystore key.alias=rainydays key.store.password=thisisnotmypassword key.alias.password=thisisnotmypassword The sdk.dir should point to the Android SDK root on your Hudson node. In my case I'm running Hudson locally on my machine. The target property refers to the SDK we want to use as mentioned earlier. The *key.** properties are related to signing of the apk. My strategy is to have a separate key store and private key for each application that I develop. I also check that keystore in to SVN. I also archive the **-release.apk* artifact so that I can download the latest release apk directly from Hudson. After completing these steps, you should be able to build your Android app with Hudson.

Updating the AndroidManifest for release

...But we're not done yet :) Remember what I said about updating the AndroidManifest.xml? For that we need to edit the build.xml, which by default contains nothing more then a <setup/> tag to pull in the Android SDK ant target definitions. For my Rainy Days application, I adjusted build.xml like this: ...
Removing debug attribute from AndroidManifest.xml Setting release maps key


What the above snippet does is removing the android:debuggable attribute from the AndroidManifest.xml and replacing the maps API key in res/layout/maplayout.xml with the correct key for release. The -package-resources target is pulled in from the Android android_rules.xml file.

Now when Hudson builds my app I get a ready to release apk that I can install on my device or emulator, which is pretty nice.

There is are some issues with this approach however. As you might have noticed:

• Replacements are done in the workspace, we are not really building exactly what's in svn
• Each new build should start out fresh for that reason, for example by using the svn revert option.

Additionally I can not yet tag the release version with the updated files, because the subversion tagging plugin doesn't support this by design. This could be worked around by adding svn statements in the build.xml however. For now I don't really mind as I make minor changes to the resource files, but I'll be looking at improving this situation.

Things to add: unit testing, coverage...

One thing that I'd really like to add is unit testing. This is a little bit more complicated though, since unit tests require a running emulator and a running emulator requires a gui. The Hudson Xvnc plugin could be very helpful here.

The Android build scripts for test projects already include EMMA output, it shouldn't be to hard to use the Hudson plugin for that.

When Hudson is running on a local machine, the Batch task plugin can automate installing the apk on a device to automate things further.

Summary

Building Android applications with Hudson is not that hard, since the builds are based on Ant. By hooking in to the standard Android build targets it's easy to update files like AndroidManifest.xml which in turn makes sure the release process is controlled and predictable.
Android unit tests depend on the emulator which is a little bit more challenging to set up, but Hudson already has some plugins available to make this easier.

Editor's Note: Hugo Visser is the developer of Rainy Days and Engine Watch for Android. You can follow him on Twitter and on his blog.