Changes between Version 1 and Version 2 of WebKitFlatpakSDK/SpeedUpBuild

Timestamp:

Aug 5, 2020 5:00:12 AM (4 years ago)

Author:

Philippe Normand

Comment:

--

WebKitFlatpakSDK/SpeedUpBuild

@@ -5 +5 @@
 === sccache
 
-[https://github.com/mozilla/sccache sccache] is available in the SDK. Depending on your sccache-dist cloud deployment, you might need to ask for an authentication token to your sysadmin. Once you have it, generate the toolchain archives that will be used for the distributed compilation:
+This documentation applies to the WPE and GTK ports. From the [https://github.com/mozilla/sccache sccache] homepage:
+
+[https://github.com/mozilla/sccache sccache] is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible, storing a cache in a remote storage using the Amazon Simple Cloud Storage Service (S3) API, the Google Cloud Storage (GCS) API, or Redis. It works as a client-server. The client spawns a server if one is not running already, and sends the wrapped command line as a request to the server, which then does the work and returns stdout/stderr for the job. The client-server model allows the server to be more efficient in its handling of the remote storage.
+
+Sccache can also be used with local storage instead of remote.
+
+Depending on your sccache-dist cloud deployment, you might need to ask for an authentication token to your sysadmin. Once you have it, generate the toolchain archives that will be used for the distributed compilation:
 
 {{{
+#!sh
 $ Tools/Scripts/webkit-flatpak -r -t token -s scheduler-url
 }}}
@@ -15 +22 @@
 Once the toolchains have been generated, they will be available in `WebKitBuild/Toolchains/` and a sccache config file should be present in `WebKitBuild/UserFlatpak/sccache.toml`. Feel free to edit it, but keep in mind this is a generated file. To use a custom config file during the build, use the `$SCCACHE_CONF` env var (refer to the upstream sccache docs), otherwise the generated one will be used by default.
 
+Here is an example of the config file contents:
+{{{
+[dist]
+scheduler_url = "https://sccache.igalia.com"
+[[dist.toolchains]]
+compiler_executable = "/usr/bin/c++"
+archive_compiler_executable = "/usr/bin/g++"
+type = "path_override"
+archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-gcc-df05b4231b1586d4bc07430d8353065c.tar.gz"
+
+[[dist.toolchains]]
+compiler_executable = "/usr/bin/clang++"
+archive_compiler_executable = "/usr/bin/clang++"
+type = "path_override"
+archive = "/home/phil/WebKit/WebKitBuild/Toolchains/webkit-sdk-clang-1898673fc91e702aa44006e0195695a3.tar.gz"
+
+[dist.auth]
+token = "s3cr3t-tok3n"
+type = "token"
+}}}
+
+
 The next step is to enable sccache in your build:
 
 {{{
+#!sh
 export WEBKIT_USE_SCCACHE=1
 export NUMBER_OF_PROCESSORS=45
@@ -24 +54 @@
 And then run build-webkit as usual. Open a network traffic monitor and watch for the uplink data rate. For debug builds a good downlink is needed as well, because the object files produced on the cloud builders are bigger.
 
+If you have access to a Redis instance, you can tell sccache to use it:
+
+{{{
+#!sh
+$ export SCCACHE_REDIS="redis://pass@host:port/dbpath"
+}}}
+
+
 === IceCC