wiki:AdvantagesOfJenkinsOverBuildbot
Last modified 2 years ago Last modified on 12/22/11 08:52:24

Advantages of Jenkins over Buildbot

Jenkins and Buildbot are both continuous-integration servers. WebKit currently uses Buildbot at http://build.webkit.org/. This page lists some advantages switching to Jenkins would bring.

Annotating build failures

As described in bug 59527, we'd like to be able to annotate build failures to show whether someone is working on them. Jenkins's Claim plugin would let us do just that.

Build times

Jenkins has built-in support for tracking the length of the build over time and graphing it.

Code coverage statistics

Jenkins's Cobertura plugin combined with coverage.py would allow us to track code coverage statistics for our Python unit tests. See an example from Apache's Jenkins server. You can even see individual lines of code that are or are not tested.

Determining when a test started failing

Jenkins has built-in support for determining when a test started failing, and will link you right to that build. See an example from Apache's Jenkins server. The "Age" column contains links to the first build in which each test failed. To take advantage of this we'd want to make all our testing tools output JUnitReport-style XML files. (TestWebKitAPI already knows how to do this if you pass it the --gtest_output=xml option.)

Graphs of aggregate test failures over time

Jenkins has built-in support for tracking the number of failing tests over time and graphing it. See an example from Apache's Jenkins server.

Graphs/tables of individual test duration and failures over time

Jenkins has built-in support for graphing test duration over time and generating tables of test failures over time. See an example from Apache's Jenkins server.