Speed up the build

When you build WebKitGTK or WPEWebKit using the Flatpak SDK, ccache is enabled by default. More options are available, for distributed builds. See below.

sccache

This documentation applies to the WPE and GTK ports. From the ​sccache homepage:

​sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible, storing a cache in a remote storage using the Amazon Simple Cloud Storage Service (S3) API, the Google Cloud Storage (GCS) API, or Redis. It works as a client-server. The client spawns a server if one is not running already, and sends the wrapped command line as a request to the server, which then does the work and returns stdout/stderr for the job. The client-server model allows the server to be more efficient in its handling of the remote storage.

Sccache can also be used with local storage instead of remote.

Depending on your sccache-dist cloud deployment, you might need to ask for an authentication token to your sysadmin. Once you have it, generate the toolchain archives that will be used for the distributed compilation:

$ Tools/Scripts/webkit-flatpak -r -t token -s scheduler-url

By default the scheduler url is Igalia's if it wasn't provided on the command-line.

Once the toolchains have been generated, they will be available in WebKitBuild/Toolchains/ and a sccache config file should be present in WebKitBuild/UserFlatpak/sccache.toml. Feel free to edit it, but keep in mind this is a generated file. To use a custom config file during the build, use the $SCCACHE_CONF env var (refer to the upstream sccache docs), otherwise the generated one will be used by default.

Here is an example of the config file contents:

[dist]
scheduler_url = "https://sccache.igalia.com"
[[dist.toolchains]]
compiler_executable = "/usr/bin/c++"
archive_compiler_executable = "/usr/bin/g++"
type = "path_override"
archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-gcc-df05b4231b1586d4bc07430d8353065c.tar.gz"

[[dist.toolchains]]
compiler_executable = "/usr/bin/clang++"
archive_compiler_executable = "/usr/bin/clang++"
type = "path_override"
archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-clang-1898673fc91e702aa44006e0195695a3.tar.gz"

[dist.auth]
token = "s3cr3t-tok3n"
type = "token"

The next step is to enable sccache in your build:

export WEBKIT_USE_SCCACHE=1
export NUMBER_OF_PROCESSORS=45

And then run build-webkit as usual. Open a network traffic monitor and watch for the uplink data rate. For debug builds a good downlink is needed as well, because the object files produced on the cloud builders are bigger.

If you have access to a Redis instance, you can tell sccache to use it:

$ export SCCACHE_REDIS="redis://pass@host:port/dbpath"

IceCC

TODO