Speed up the build

When you build WebKitGTK or WPEWebKit using the Flatpak SDK, ccache is enabled by default. More options are available, for distributed builds. See below.

sccache

This documentation applies to the WPE and GTK ports. From the ​sccache homepage:

​sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible, storing a cache in a remote storage using the Amazon Simple Cloud Storage Service (S3) API, the Google Cloud Storage (GCS) API, or Redis. It works as a client-server. The client spawns a server if one is not running already, and sends the wrapped command line as a request to the server, which then does the work and returns stdout/stderr for the job. The client-server model allows the server to be more efficient in its handling of the remote storage.

Sccache can also be used with local storage instead of remote.

Depending on your sccache-dist cloud deployment, you might need to ask for an authentication token to your sysadmin. Once you have it, generate the toolchain archives that will be used for the distributed compilation:

$ Tools/Scripts/webkit-flatpak -r -t token

You can specify the scheduler URL using -s scheduler-url, if you don't do it by default the scheduler url is Igalia's if it wasn't provided on the command-line.

Once the toolchains have been generated, they will be available in WebKitBuild/UserFlatpak/Toolchains/ and a sccache config file should be present in WebKitBuild/UserFlatpak/sccache.toml. Feel free to edit it, but keep in mind this is a generated file. To use a custom config file during the build, use the $SCCACHE_CONF env var (refer to the upstream sccache docs), otherwise the generated one will be used by default.

Here is an example of the config file contents:

[dist]
scheduler_url = "https://sccache.igalia.com"
[[dist.toolchains]]
compiler_executable = "/usr/bin/c++"
archive_compiler_executable = "/usr/bin/g++"
type = "path_override"
archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-gcc-df05b4231b1586d4bc07430d8353065c.tar.gz"

[[dist.toolchains]]
compiler_executable = "/usr/bin/clang++"
archive_compiler_executable = "/usr/bin/clang++"
type = "path_override"
archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-clang-1898673fc91e702aa44006e0195695a3.tar.gz"

[dist.auth]
token = "s3cr3t-tok3n"
type = "token"

The next step is to enable sccache in your build:

export WEBKIT_USE_SCCACHE=1
export NUMBER_OF_PROCESSORS=45

And then run build-webkit as usual. Open a network traffic monitor and watch for the uplink data rate. For debug builds a good downlink is needed as well, because the object files produced on the cloud builders are bigger. You can monitor the sccache stats with this command:

$ WEBKIT_USE_SCCACHE=1 webkit-flatpak -c watch -n 1 sccache -s

If you have access to a Redis instance, you can tell sccache to use it:

$ export SCCACHE_REDIS="redis://pass@host:port/dbpath"

IceCC

​Icecream was created by SUSE based on distcc. Like distcc, Icecream takes compile jobs from a build and distributes it among remote machines allowing a parallel build. But unlike distcc, Icecream uses a single central server called scheduler that dynamically assigns the compile jobs to multiple distributed daemons, choosing the fastest free one. This advantage pays off mostly for shared computers, if you're the only user on x machines, you have full control over them.

• You should have only one scheduler in your network.
• The scheduler and one of the daemons can be in the same host.

Installing

• Execute:

  $ sudo apt-get install icecc
  

• You can install icecc monitor too:

  $ sudo apt-get install icecc-monitor
  

icecc scheduler

• Configure scheduler to start by default (see /usr/share/doc/icecc/README.Debian):

  $ sudo update-rc.d icecc-scheduler enable
  

• After configuring the scheduler to start, it will do so on next reboot, but not sooner. You can start it manually:
  • Ubuntu

    $ sudo service icecc restart
    

  • Debian

    $ sudo service icecc-scheduler start
    

icecc daemon(s)

• iceccd (daemon) has to be able to find the scheduler in the network. By default, it makes so by sending a broadcast message. This may not work depending on your network topology (routers, firewalls, etc.). You can specify the actual address of the icecc-scheduler by stating it in the icecc config file:

  $ cat /etc/icecc/icecc.config
  ...
  #
  # If the daemon can't find the scheduler by broadcast (e.g. because
  # of a firewall) you can specify it.
  #
  # ICECC_SCHEDULER_HOST=""
  ICECC_SCHEDULER_HOST="<icecc-scheduler IP or host name>"
  ...
  

• Also, you may not want your local icecc daemon to run works from other hosts. For example, I want my laptop to be helped by my tower but I don't want my laptop to take works from my tower:

  $ cat /etc/icecc/icecc.config
  ...
  #
  # Specifies whether jobs submitted by other nodes are allowed to run on
  # this one.
  #
  # ICECC_ALLOW_REMOTE="yes"
  ICECC_ALLOW_REMOTE="no"
  ...
  

• Make sure you integrate it seamlessly with ccache by allowing enough jobs your local icecc daemon:

(CCache run preprocessor with calling icecc, which connects to your local icecc daemon, which run preprocessing locally. Don't set ICECC_MAX_JOBS="0" to forbid accepting remote jobs, because in this case CCache can preprocess only on one thread. The ideal setting for ICECC_MAX_JOBS is the number of your processors.)

$ cat /etc/icecc/icecc.config
...
#
# You can overwrite here the number of jobs to run in parallel. Per
# default this depends on the number of (virtual) CPUs installed.
#
# Note: a value of "0" is actually interpreted as "1", however it
# also sets ICECC_ALLOW_REMOTE="no".
#
# ICECC_MAX_JOBS=""
ICECC_MAX_JOBS="<number-of-your-processors>"
...

• You may want to restart the service after setting all the configuration appropriately:
  • Ubuntu

    $ sudo service icecc restart
    

  • Debian

    $ sudo service iceccd restart
    

• Then compile WebKit normally:

  $ Tools/Scripts/build-webkit --gtk
  

Enable IcecCC in build-webkit

To use icecc, export the CCACHE_PREFIX variable:

$ export CCACHE_PREFIX=icecc

As ccache is enabled by default in the SDK, it will now prefix compile commands with icecc. This is all you need to do, toolchains archives automatically generated during the SDK installation will be sent to the scheduler over the network. This is all pre-configured by default when build-webkit detected the CCACHE_PREFIX env var is set to icecc.

icecc troubleshooting

• If jobs are not being distributed, then check WebKitBuild/Release/rules.ninja is using /usr/lib/ccache/g++.

  rule CXX_COMPILER
    depfile = $DEP_FILE
    command = /usr/lib/ccache/g++   $DEFINES $FLAGS -MMD -MT $out -MF "$DEP_FILE" -o $out -c $in
    description = Building CXX object $out
  

• If you get strange errors when building try to clear the ccache cache (ccache -C) and start with a clean build

  $ Tools/Scripts/webkit-flatpak -c ccache -C
  $ rm -fr WebKitBuild/GTK/Release