Advantages of Jenkins over Buildbot

​Jenkins and ​Buildbot are both continuous-integration servers. WebKit currently uses Buildbot at ​http://build.webkit.org/. This page lists some advantages switching to Jenkins would bring.

Annotating build failures

As described in ​bug 59527, we'd like to be able to annotate build failures to show whether someone is working on them. Jenkins's ​Claim plugin would let us do just that.

Build times

Jenkins has built-in support for tracking the length of the build over time and graphing it.

Code coverage statistics

Jenkins's ​Cobertura plugin combined with ​coverage.py would allow us to track code coverage statistics for our Python unit tests. See ​an example from Apache's Jenkins server. You can even see ​individual lines of code that are or are not tested.

Determining when a test started failing

Jenkins has built-in support for determining when a test started failing, and will link you right to that build. See ​an example from Apache's Jenkins server. The "Age" column contains links to the first build in which each test failed. To take advantage of this we'd want to make all our testing tools output ​JUnitReport-style XML files. (TestWebKitAPI already knows how to do this if you pass it the --gtest_output=xml option.)

Graphs of aggregate test failures over time

Jenkins has built-in support for tracking the number of failing tests over time and graphing it. See ​an example from Apache's Jenkins server.

Graphs/tables of individual test duration and failures over time

Jenkins has built-in support for graphing test duration over time and generating tables of test failures over time. See ​an example from Apache's Jenkins server.