WebKit BuildBot

The ​WebKit BuildBot is a Continuous Integration system for post-commit testing to help automate the compile/test cycle. After every commit to WebKit's repository it will schedule compilation and tests on a variety of different computers, known as build workers. For pre-commit testing, please see ​https://trac.webkit.org/wiki/EarlyWarningSystem

Running a Build Worker

Each build worker needs a unique username and password to be able to take part in the compile/test cycle. If you would like to contribute a bot, please add the bot (and builder if applicable) to config.json (using the usual patch submission process) and email ​admin@webkit.org to receive the bot password and Buildbot server restart.

Please ensure that you are familiar with ​building WebKit from source so that problems getting the bot up and running can be easily troubleshooted. The machine will need to have:

General requirements

1. Subversion installed and accessible on the PATH.
2. Set the computer to not go to sleep (under Energy Saver in System Preferences)
3. Install !buildbot-worker package:
   1. In a shell, sudo pip install buildbot-worker==2.10.1

Mac requirements

1. Everything in ​https://webkit.org/building/tools.html

Cygwin Win requirements

1. Everything in ​https://webkit.org/building/tools.html (if creating a builder, use DirectX SDK December 2004, NOT the FEB 2010 version)
   1. For the cygwin install, the actual minimal requirements are: apache, bc, bison, curl, diffutils, flex, gcc, gperf, make, patch, perl, python, readline, ruby, subversion, unzip, zip.
   2. From the instructions, save the file WebKitSupportLibrary.zip in a directory (e.g. c:\deps) and set WEBKITSUPPORTLIBRARIESZIPDIR=c:\deps
2. ​Safari 4 and QuickTime
3. ​Java
4. Set the following registry key:

   [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]
   "Auto"="0"
   

5. Under 64-bit versions of Windows, also set the following registry key:

   [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug]
   "Auto"="0"
   

6. Disable the error reporting UI (i.e., the dialog that says "[program name].exe has stopped working" with options to debug or close the application):
   1. Set the following registry key:

      [HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\Windows Error Reporting]
      "Dont Show UI"="1"
      

   2. Under 64-bit versions of Windows, also set the following registry key:

      [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\Windows Error Reporting]
      "Dont Show UI"="1"
      

7. Modify the group policy to block the UI:
   1. Run the gpedit application (C:\Windows\System\gpedit.msc).
   2. Select the "Computer Configuration\Administrative Templates\All Settings" node.
   3. Edit the "Prevent display of the user interface for critical errors" item.
   4. Enable the setting, then press OK.

Non-cygwin Win Requirements

1. Install ​Strawberry Perl. Make sure to run perl from the command line at least once. This will set up the library variables.
2. Install ​Python
3. Add Python directory on the system PATH variable (Control Panel -> System -> Advanced - > Environment Variables)
4. Add .PY to the system PATHEXT variable.
5. Download ​pywin32, unzip it and run setup install in the unzipped folder.
6. Install ​setuptools. If the installer fails due to not being able to find a registory entry, see ​http://selfsolved.com/problems/setuptools-06c11-fails-to-instal.
7. Run easy_install twisted.
8. Download ​buildbot-slave zip file, unzip it and run setup install in the unzipped folder.
9. Install ​Ruby.

GTK requirements

1. See BuildingGtk
2. Probably need to sudo apt-get install geoclue libsoup. (To be verified and moved to BuildingGtk).

Configuring a Build Worker

1. Open a shell inside a directory suitable to keep the worker copy of the WebKit repository (e.g.: mkdir /Volumes/Data/worker).
2. cd to the worker directory
3. Run buildbot-worker create-worker --keepalive 120 --umask 022 worker build.webkit.org:17000 [username] [password]
4. Edit info/admin and info/host to contain basic contact information and information about the bot.
5. Run buildbot-worker start to start the bot.

Troubleshooting

Perl Error on Windows

On Windows, if you get perl (...) fatal error - unable to remap (...) to same address as parent, you need to run

c:\cygwin\bin\ash
/bin/rebaseall -b 0x50000000 -o 0x80000
exit

or rebase individual files.

Timeouts on GStreamer related tests on Linux

If you're running the buildbot on a headless system, there will be problems with pulseaudio initialization in media tests. To resolve this:

1. Run pulseaudio in system mode by changing PULSEAUDIO_SYSTEM_START from 0 to 1 in etc/defaults/pulseaudio. Then (re)start the service service pulseaudio restart.
2. If you're seeing xcb_connection_has_error() returned true in the error output of run-webkit-tests when running media tests, it is likely to be caused by the x11-bell module of pulseaudio. If you don't need it, it's easy to uninstall for example by: apt-get remove --purge pulseaudio-module-x11

Running buildbot master locally

The following instruction describes how to run a buildbot master.

1. Install Buildbot
   1. sudo pip3 install buildbot==2.10.1 --no-binary :all:
   2. sudo pip3 install buildbot-console-view==2.10.1 buildbot-waterfall-view==2.10.1 buildbot-worker==2.10.1 buildbot-www==2.10.1
   3. sudo pip3 install Twisted --no-binary :all:
   4. sudo pip3 install lz4==3.1.0 six==1.15.0

2. Creating a master
   1. Checkout WebKit repository and cd to Tools/CISupport/build-webkit-org
   2. Run buildbot create-master
   3. Run make_passwords_json.py to generate necessary config files (passwords.json, auth.json and more).

3. Start / Stop buildbot

buildbot start or buildbot stop inside the directory you recreated in step 2.

4. Start / Stop build worker
   1. Modify worker's buildbot.tac
      • Set buildmaster_host to 'localhost' or whatever machine you're running master on
      • Set port to 17000 by default (different from the port master hosts http server)
      • Set workername and passwd to one that matches your passwords.json added in 2.3
   2. buildbot-worker start as usual.