wiki:BuildBot

WebKit BuildBot

The WebKit BuildBot is a Continuous Integration system for post-commit testing to help automate the compile/test cycle. After every commit to WebKit's repository it will schedule compilation and tests on a variety of different computers, known as build workers. For pre-commit testing, please see https://trac.webkit.org/wiki/EarlyWarningSystem

Running a Build Worker

Each build worker needs a unique username and password to be able to take part in the compile/test cycle. If you would like to contribute a bot, please add the bot (and builder if applicable) to config.json (using the usual patch submission process) and email admin@webkit.org to receive the bot password and Buildbot server restart.

Please ensure that you are familiar with building WebKit from source so that problems getting the bot up and running can be easily troubleshooted. The machine will need to have:

General requirements

  1. Subversion installed and accessible on the PATH.
  2. Set the computer to not go to sleep (under Energy Saver in System Preferences)
  3. Install !buildbot-worker package:
    1. In a shell, sudo pip install buildbot-worker==2.10.1

Mac requirements

  1. Everything in https://webkit.org/building/tools.html

Cygwin Win requirements

  1. Everything in https://webkit.org/building/tools.html (if creating a builder, use DirectX SDK December 2004, NOT the FEB 2010 version)
    1. For the cygwin install, the actual minimal requirements are: apache, bc, bison, curl, diffutils, flex, gcc, gperf, make, patch, perl, python, readline, ruby, subversion, unzip, zip.
    2. From the instructions, save the file WebKitSupportLibrary.zip in a directory (e.g. c:\deps) and set WEBKITSUPPORTLIBRARIESZIPDIR=c:\deps
  2. Safari 4 and QuickTime
  3. Java
  4. Set the following registry key:
    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]
    "Auto"="0"
    
  5. Under 64-bit versions of Windows, also set the following registry key:
    [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug]
    "Auto"="0"
    
  6. Disable the error reporting UI (i.e., the dialog that says "[program name].exe has stopped working" with options to debug or close the application):
    1. Set the following registry key:
      [HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\Windows Error Reporting]
      "Dont Show UI"="1"
      
    2. Under 64-bit versions of Windows, also set the following registry key:
      [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\Windows Error Reporting]
      "Dont Show UI"="1"
      
  7. Modify the group policy to block the UI:
    1. Run the gpedit application (C:\Windows\System\gpedit.msc).
    2. Select the "Computer Configuration\Administrative Templates\All Settings" node.
    3. Edit the "Prevent display of the user interface for critical errors" item.
    4. Enable the setting, then press OK.

Non-cygwin Win Requirements

  1. Install Strawberry Perl. Make sure to run perl from the command line at least once. This will set up the library variables.
  2. Install Python
  3. Add Python directory on the system PATH variable (Control Panel -> System -> Advanced - > Environment Variables)
  4. Add .PY to the system PATHEXT variable.
  5. Download pywin32, unzip it and run setup install in the unzipped folder.
  6. Install setuptools. If the installer fails due to not being able to find a registory entry, see http://selfsolved.com/problems/setuptools-06c11-fails-to-instal.
  7. Run easy_install twisted.
  8. Download buildbot-slave zip file, unzip it and run setup install in the unzipped folder.
  9. Install Ruby.

GTK requirements

  1. See BuildingGtk
  2. Probably need to sudo apt-get install geoclue libsoup. (To be verified and moved to BuildingGtk).

Configuring a Build Worker

  1. Open a shell inside a directory suitable to keep the worker copy of the WebKit repository (e.g.: mkdir /Volumes/Data/worker).
  2. cd to the worker directory
  3. Run buildbot-worker create-worker --keepalive 120 --umask 022 worker build.webkit.org:17000 [username] [password]
  4. Edit info/admin and info/host to contain basic contact information and information about the bot.
  5. Run buildbot-worker start to start the bot.

Troubleshooting

Perl Error on Windows

On Windows, if you get perl (...) fatal error - unable to remap (...) to same address as parent, you need to run

c:\cygwin\bin\ash
/bin/rebaseall -b 0x50000000 -o 0x80000
exit

or rebase individual files.

Timeouts on GStreamer related tests on Linux

If you're running the buildbot on a headless system, there will be problems with pulseaudio initialization in media tests. To resolve this:

  1. Run pulseaudio in system mode by changing PULSEAUDIO_SYSTEM_START from 0 to 1 in etc/defaults/pulseaudio. Then (re)start the service service pulseaudio restart.
  2. If you're seeing xcb_connection_has_error() returned true in the error output of run-webkit-tests when running media tests, it is likely to be caused by the x11-bell module of pulseaudio. If you don't need it, it's easy to uninstall for example by: apt-get remove --purge pulseaudio-module-x11

Running buildbot master locally

The following instruction describes how to run a buildbot master.

  1. Install Buildbot
    1. sudo pip3 install buildbot==2.10.1 --no-binary :all:
    2. sudo pip3 install buildbot-console-view==2.10.1 buildbot-waterfall-view==2.10.1 buildbot-worker==2.10.1 buildbot-www==2.10.1
    3. sudo pip3 install Twisted --no-binary :all:
    4. sudo pip3 install lz4==3.1.0 six==1.15.0
  1. Creating a master
    1. Checkout WebKit repository and cd to Tools/CISupport/build-webkit-org
    2. Run buildbot create-master
    3. Run make_passwords_json.py to generate necessary config files (passwords.json, auth.json and more).
  1. Start / Stop buildbot

buildbot start or buildbot stop inside the directory you recreated in step 2.

  1. Start / Stop build worker
    1. Modify worker's buildbot.tac
      • Set buildmaster_host to 'localhost' or whatever machine you're running master on
      • Set port to 17000 by default (different from the port master hosts http server)
      • Set workername and passwd to one that matches your passwords.json added in 2.3
    2. buildbot-worker start as usual.
Last modified 3 years ago Last modified on Feb 9, 2021 8:39:49 AM

Attachments (1)

Download all attachments as: .zip