I recently spent some time fully automating my toolchain. My aim was to have a Continuous Integration (CI) server compile & test all my scala projects, publish the artifacts locally (for libraries) or create artifacts for deployment or download (for my blog engine and l-system generator). Luckily I had used sbt for all my scala projects almost from the start, so I only needed to automate the usage of sbt.
I chose [Hudson](“http://hudson-ci.org/") as my CI Server, as I have had good experience with it in the past (a friend of mine introduced it in the last company I worked for). There is no sbt plugin for Hudson, but Hudson offers a “free-style software” project type that simply issues shell commands. This is all you need for sbt projects, which is maybe why no one bothered to write a hudson-sbt-plugin yet. For me the free-style project was all I needed.
Basic Build Setup for a Simple Project
For my blog engine I need two scala projects:
baselib (my own collection of utilities I have written for Scala) and
liftblog (the actual blog written in Scala using Lift). The
baselib is a very simple project which has hardly any dependencies and it just produces a .jar-file. I will use it as an example of how to integrate an sbt project into a Hudson build. As I have written above, just create a “free-style software project” and adjust it as follows:
Source Code Management
Nothing special here, just choose the SCM system you use and enter the repository location. As I use git, I first had to install the git plugin for hudson. I run Hudson on the same system where I have the git repo, so my url was just this:
The three slashes where a little catch for me. I tried with one first and it didn’t work.
I want the project built every time a change is checked in, so I chose “Poll SCM”. I want Hudson to check every five minutes, so this is my “Schedule” setting:
#every 5 minutes */5 * * * *
This is where it finally gets (a little) interesting. I prefer to start the build with a complete clean of the project. If you omit that step, the whole process will be considerably faster, because only necessary changes are compiled. In my experience however, from time to time slight weird errors creep in if you don’t do a clean. So to be sure for starters, I stuck with a clean. If you are experimenting with plugins for your projects a lot and change/update them still often, you might even start with cleaning plugins. This is probably not what you want for most projects, though. As I am a firm believer in well-documented code, I also chose to create my scaladoc. As it is a lib I want to use in other projects, I publish the resulting jar to my local repo. All in all, this is what I came up with:
#only necessary if you use a plugin in the project that still changes often #sbt clean-plugins #comment this out for faster builds: sbt clean #you can comment this out if you don't want to check for updates of referenced libs: sbt update #the actual compilation of the code sbt compile #run unit tests sbt test #package documentation sbt doc #create artifact (e.g. .jar-file) sbt package #push to local repo sbt publish-local #if you want to publish to another repo you configured in your project, do this: #sbt publish
That’s it. If any of the above steps fails, the shell command will finish with an error code and Hudson will count the build as failed. I could have trimmed this down a lot, as
sbt test would call
sbt compile anyway if necessary. I just prefer the process to be more fine grained, IMHO it helps debugging if something goes wrong. If you like it super-brief, the following should work as well:
#should compile, test, package, create docs & artifact sbt package-all sbt publish-local
The above is all you need to get your project built. If anything fails, Hudson will mark the project failed and you can check the console output for errors. One of the cool features of Hudson is however that you get a nice clickable overview page for your project where you can download artifacts (in the given example a .jar-file), browse the API documentation and get an overview over your tests. So we now need to tell Hudson where to find the stuff we just had him build. Sbt puts everything it creates into a directory named “target” in the project folder by default. So if you did not manually change the default paths, you should find everything there (probably in another subfolder named after the scala version used for compiling). To publish the javadoc, check the “Publish Javadoc” checkbox. If you did not change the default paths, you should have a path similar as the following for your docs:
Depending on which Scala version you use, you might have to replace
scala-2.8.0 to something else, e.g.
scala-2.7.7. Next is the artifact. As Hudson can publish potentially more than one, we can use wildcards in the path, so there is no need to worry about your Scala version. Just check the “Archive the artifacts” checkbox and enter something like this:
Of course you need to replace
baselib with the prefix of your jar file. As two more of my projects depend on my baselib, I have also checked the “Build other projects” checkbox and entered the names of the projects there in a comma-separated list.
If you run your build again now, you should be able to browse your API documentation from the project overview page and also be able to download the created .jar-files. We have left out one important feature though - the overview over our test results.
Integrating sbt Test Results into Hudson
The reason why I have left that out so far is that —to my knowledge— sbt cannot produce anything out-of-the-box from which hudson can parse the test results and display them. Hudson can understand test results in a certain XML format that is produced by JUnit. Unfortunately, sbt cannot create that XML format, so I was stuck at that point. There is a way out however: One can plug in custom test listeners into an sbt project. Those test listeners can react upon the test results. The default console output for example is created by a test listener that simpy prints to the console.
Adding a JUnitXML tests listener to your project
I did not find a plugin for JUnitXML output though, so I just wrote my own. It is still very beta and I make no guarantees as to its usability, but it works fine for me so far, so I have put it in my public maven repo so you can use it too if you like. I put the source under the MIT license, you can check it out from my git repo like this:
git clone http://git.henkelmann.eu/junit_xml_listener.git
The maven repo can be found under
http://maven.henkelmann.eu, the organization is
eu.henkelmann, the project name is
junit_xml_listener and the current version is
0.2. In order to have the plugin generate the necessary XML output Hudson understands, you first have to add the plugin to your project. To do so, add the following lines to your plugin definition:
//tell sbt where to find the repo val repo = "Christoph's Maven Repo" at "http://maven.henkelmann.eu/" //include the junit xml listener val junitXml = "eu.henkelmann" % "junit_xml_listener" % "0.2"
Once you have done that, you need to add the JUnit xml listener to the default test listeners. To do so, add the following to your project definition:
//create a listener that writes to the normal output directory def junitXmlListener: TestReportListener = new JUnitXmlTestsListener(outputPath.toString) //add the new listener to the already configured ones override def testListeners: Seq[TestReportListener] = super.testListeners ++ Seq(junitXmlListener)
Don’t forget to import the listener:
The JunitXmlTestsListener takes the path to a folder as constructor argument. It creates a subfolder named “test-reports” in that folder and adds one XML file to that folder for each test suite run. The “outputPath” definition is the directory to which sbt writes all the results and should be present if you inherit from one of the default project types.
Add the test output to Hudson
Now the only thing you should have to do is to check the “Publish JUnit test result reports” checkbox and add the following path:
Now you should get the same test overview for your Scala projects as for your JAVA projects.
The plugin still has one drawback, however: I did not find any way to get the duration for a single test run. I cannot measure it myself, because sbt always passes all test results per suite in one bunch after the tests are already over. So for now all test durations are displayed as zero. On the other hand the plugin should work no matter which test framework you use, as long as you use sbt to run the test.