Changes between Version 1 and Version 2 of WebKitFlatpakSDK/SpeedUpBuild


Ignore:
Timestamp:
Aug 5, 2020 5:00:12 AM (2 years ago)
Author:
Philippe Normand
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • WebKitFlatpakSDK/SpeedUpBuild

    v1 v2  
    55=== sccache
    66
    7 [https://github.com/mozilla/sccache sccache] is available in the SDK. Depending on your sccache-dist cloud deployment, you might need to ask for an authentication token to your sysadmin. Once you have it, generate the toolchain archives that will be used for the distributed compilation:
     7This documentation applies to the WPE and GTK ports. From the [https://github.com/mozilla/sccache sccache] homepage:
     8
     9[https://github.com/mozilla/sccache sccache] is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible, storing a cache in a remote storage using the Amazon Simple Cloud Storage Service (S3) API, the Google Cloud Storage (GCS) API, or Redis. It works as a client-server. The client spawns a server if one is not running already, and sends the wrapped command line as a request to the server, which then does the work and returns stdout/stderr for the job. The client-server model allows the server to be more efficient in its handling of the remote storage.
     10
     11Sccache can also be used with local storage instead of remote.
     12
     13Depending on your sccache-dist cloud deployment, you might need to ask for an authentication token to your sysadmin. Once you have it, generate the toolchain archives that will be used for the distributed compilation:
    814
    915{{{
     16#!sh
    1017$ Tools/Scripts/webkit-flatpak -r -t token -s scheduler-url
    1118}}}
     
    1522Once the toolchains have been generated, they will be available in `WebKitBuild/Toolchains/` and a sccache config file should be present in `WebKitBuild/UserFlatpak/sccache.toml`. Feel free to edit it, but keep in mind this is a generated file. To use a custom config file during the build, use the `$SCCACHE_CONF` env var (refer to the upstream sccache docs), otherwise the generated one will be used by default.
    1623
     24Here is an example of the config file contents:
     25{{{
     26[dist]
     27scheduler_url = "https://sccache.igalia.com"
     28[[dist.toolchains]]
     29compiler_executable = "/usr/bin/c++"
     30archive_compiler_executable = "/usr/bin/g++"
     31type = "path_override"
     32archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-gcc-df05b4231b1586d4bc07430d8353065c.tar.gz"
     33
     34[[dist.toolchains]]
     35compiler_executable = "/usr/bin/clang++"
     36archive_compiler_executable = "/usr/bin/clang++"
     37type = "path_override"
     38archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-clang-1898673fc91e702aa44006e0195695a3.tar.gz"
     39
     40[dist.auth]
     41token = "s3cr3t-tok3n"
     42type = "token"
     43}}}
     44
     45
    1746The next step is to enable sccache in your build:
    1847
    1948{{{
     49#!sh
    2050export WEBKIT_USE_SCCACHE=1
    2151export NUMBER_OF_PROCESSORS=45
     
    2454And then run build-webkit as usual. Open a network traffic monitor and watch for the uplink data rate. For debug builds a good downlink is needed as well, because the object files produced on the cloud builders are bigger.
    2555
     56If you have access to a Redis instance, you can tell sccache to use it:
     57
     58{{{
     59#!sh
     60$ export SCCACHE_REDIS="redis://pass@host:port/dbpath"
     61}}}
     62
     63
    2664=== IceCC
    2765