How to build sbt projects with Hudson and integrate test results

I recently spent some time fully automating my toolchain. My aim was to have a Continuous Integration (CI) server compile & test all my scala projects, publish the artifacts locally (for libraries) or create artifacts for deployment or download (for my blog engine and l-system generator). Luckily I had used sbt for all my scala projects almost from the start, so I only needed to automate the usage of sbt.

I chose [Hudson](“http://hudson-ci.org/") as my CI Server, as I have had good experience with it in the past (a friend of mine introduced it in the last company I worked for). There is no sbt plugin for Hudson, but Hudson offers a “free-style software” project type that simply issues shell commands. This is all you need for sbt projects, which is maybe why no one bothered to write a hudson-sbt-plugin yet. For me the free-style project was all I needed.

Basic Build Setup for a Simple Project

For my blog engine I need two scala projects: baselib (my own collection of utilities I have written for Scala) and liftblog (the actual blog written in Scala using Lift). The baselib is a very simple project which has hardly any dependencies and it just produces a .jar-file. I will use it as an example of how to integrate an sbt project into a Hudson build. As I have written above, just create a “free-style software project” and adjust it as follows:

Source Code Management

Nothing special here, just choose the SCM system you use and enter the repository location. As I use git, I first had to install the git plugin for hudson. I run Hudson on the same system where I have the git repo, so my url was just this:

file:///~/sbt_projects/baselib/.git

The three slashes where a little catch for me. I tried with one first and it didn’t work.

Build Triggers

I want the project built every time a change is checked in, so I chose “Poll SCM”. I want Hudson to check every five minutes, so this is my “Schedule” setting:

#every 5 minutes
*/5 * * * *

Build

This is where it finally gets (a little) interesting. I prefer to start the build with a complete clean of the project. If you omit that step, the whole process will be considerably faster, because only necessary changes are compiled. In my experience however, from time to time slight weird errors creep in if you don’t do a clean. So to be sure for starters, I stuck with a clean. If you are experimenting with plugins for your projects a lot and change/update them still often, you might even start with cleaning plugins. This is probably not what you want for most projects, though. As I am a firm believer in well-documented code, I also chose to create my scaladoc. As it is a lib I want to use in other projects, I publish the resulting jar to my local repo. All in all, this is what I came up with:

#only necessary if you use a plugin in the project that still changes often
#sbt clean-plugins
#comment this out for faster builds:
sbt clean
#you can comment this out if you don't want to check for updates of referenced libs:
sbt update
#the actual compilation of the code
sbt compile
#run unit tests
sbt test
#package documentation
sbt doc
#create artifact (e.g. .jar-file)
sbt package
#push to local repo
sbt publish-local
#if you want to publish to another repo you configured in your project, do this:
#sbt publish

That’s it. If any of the above steps fails, the shell command will finish with an error code and Hudson will count the build as failed. I could have trimmed this down a lot, as sbt test would call sbt compile anyway if necessary. I just prefer the process to be more fine grained, IMHO it helps debugging if something goes wrong. If you like it super-brief, the following should work as well:

#should compile, test, package, create docs & artifact
sbt package-all
sbt publish-local

Post-build Actions

The above is all you need to get your project built. If anything fails, Hudson will mark the project failed and you can check the console output for errors. One of the cool features of Hudson is however that you get a nice clickable overview page for your project where you can download artifacts (in the given example a .jar-file), browse the API documentation and get an overview over your tests. So we now need to tell Hudson where to find the stuff we just had him build. Sbt puts everything it creates into a directory named “target” in the project folder by default. So if you did not manually change the default paths, you should find everything there (probably in another subfolder named after the scala version used for compiling). To publish the javadoc, check the “Publish Javadoc” checkbox. If you did not change the default paths, you should have a path similar as the following for your docs:

target/scala_2.8.0/doc/main/api/

Depending on which Scala version you use, you might have to replace scala-2.8.0 to something else, e.g. scala-2.7.7. Next is the artifact. As Hudson can publish potentially more than one, we can use wildcards in the path, so there is no need to worry about your Scala version. Just check the “Archive the artifacts” checkbox and enter something like this:

target/scala_*/baselib_*.jar

Of course you need to replace baselib with the prefix of your jar file. As two more of my projects depend on my baselib, I have also checked the “Build other projects” checkbox and entered the names of the projects there in a comma-separated list.

If you run your build again now, you should be able to browse your API documentation from the project overview page and also be able to download the created .jar-files. We have left out one important feature though - the overview over our test results.

Integrating sbt Test Results into Hudson

The reason why I have left that out so far is that —to my knowledge— sbt cannot produce anything out-of-the-box from which hudson can parse the test results and display them. Hudson can understand test results in a certain XML format that is produced by JUnit. Unfortunately, sbt cannot create that XML format, so I was stuck at that point. There is a way out however: One can plug in custom test listeners into an sbt project. Those test listeners can react upon the test results. The default console output for example is created by a test listener that simpy prints to the console.

Adding a JUnitXML tests listener to your project

I did not find a plugin for JUnitXML output though, so I just wrote my own. It is still very beta and I make no guarantees as to its usability, but it works fine for me so far, so I have put it in my public maven repo so you can use it too if you like. I put the source under the MIT license, you can check it out from my git repo like this:

git clone http://git.henkelmann.eu/junit_xml_listener.git

The maven repo can be found under http://maven.henkelmann.eu, the organization is eu.henkelmann, the project name is junit_xml_listener and the current version is 0.2. In order to have the plugin generate the necessary XML output Hudson understands, you first have to add the plugin to your project. To do so, add the following lines to your plugin definition:

//tell sbt where to find the repo
val repo = "Christoph's Maven Repo" at "http://maven.henkelmann.eu/"
//include the junit xml listener 
val junitXml = "eu.henkelmann" % "junit_xml_listener" % "0.2"

Once you have done that, you need to add the JUnit xml listener to the default test listeners. To do so, add the following to your project definition:

//create a listener that writes to the normal output directory
def junitXmlListener: TestReportListener = new JUnitXmlTestsListener(outputPath.toString)
//add the new listener to the already configured ones
override def testListeners: Seq[TestReportListener] = super.testListeners ++ Seq(junitXmlListener)

Don’t forget to import the listener:

import eu.henkelmann.sbt.JUnitXmlTestsListener

The JunitXmlTestsListener takes the path to a folder as constructor argument. It creates a subfolder named “test-reports” in that folder and adds one XML file to that folder for each test suite run. The “outputPath” definition is the directory to which sbt writes all the results and should be present if you inherit from one of the default project types.

Add the test output to Hudson

Now the only thing you should have to do is to check the “Publish JUnit test result reports” checkbox and add the following path:

target/scala_*/test-reports/*.xml

Now you should get the same test overview for your Scala projects as for your JAVA projects.

The plugin still has one drawback, however: I did not find any way to get the duration for a single test run. I cannot measure it myself, because sbt always passes all test results per suite in one bunch after the tests are already over. So for now all test durations are displayed as zero. On the other hand the plugin should work no matter which test framework you use, as long as you use sbt to run the test.