Changes between Version 3 and Version 4 of WebKitFlatpakSDK/SpeedUpBuild

Timestamp:

Aug 5, 2020 9:21:45 AM (4 years ago)

Author:

Philippe Normand

Comment:

--

WebKitFlatpakSDK/SpeedUpBuild

@@ -66 +66 @@
 === IceCC
 
-TODO
+
+[https://github.com/icecc/icecream Icecream] was created by SUSE based on distcc. Like distcc, Icecream takes compile jobs from a build and distributes it among remote machines allowing a parallel build. But unlike distcc, Icecream uses a single central server called scheduler that dynamically assigns the compile jobs to multiple distributed daemons, choosing the fastest free one. This advantage pays off mostly for shared computers, if you're the only user on x machines, you have full control over them.
+ * You should have only one scheduler in your network.
+ * The scheduler and one of the daemons can be in the same host.
+
+==== Installing
+
+ * Execute:
+{{{
+#!sh
+$ sudo apt-get install icecc
+}}}
+
+ * You can install icecc monitor too:
+{{{
+#!sh
+$ sudo apt-get install icecc-monitor
+}}}
+
+
+==== icecc scheduler
+
+ * Configure scheduler to start by default (see `/usr/share/doc/icecc/README.Debian`):
+{{{
+#!sh
+$ sudo update-rc.d icecc-scheduler enable
+}}}
+
+ * After configuring the scheduler to start, it will do so on next reboot, but not sooner. You can start it manually:
+   * Ubuntu
+{{{
+#!sh
+$ sudo service icecc restart
+}}}
+   * Debian
+{{{
+#!sh
+$ sudo service icecc-scheduler start
+}}}
+
+==== icecc daemon(s)
+
+ * iceccd (daemon) has to be able to find the scheduler in the network. By default, it makes so by sending a broadcast message. This may not work depending on your network topology (routers, firewalls, etc.). You can specify the actual address of the icecc-scheduler by stating it in the icecc config file:
+{{{
+#!sh
+$ cat /etc/icecc/icecc.config
+...
+#
+# If the daemon can't find the scheduler by broadcast (e.g. because
+# of a firewall) you can specify it.
+#
+# ICECC_SCHEDULER_HOST=""
+ICECC_SCHEDULER_HOST="<icecc-scheduler IP or host name>"
+...
+}}}
+
+ * Also, you may not want your local icecc daemon to run works from other hosts. For example, I want my laptop to be helped by my tower but I don't want my laptop to take works from my tower:
+{{{
+#!sh
+$ cat /etc/icecc/icecc.config
+...
+#
+# Specifies whether jobs submitted by other nodes are allowed to run on
+# this one.
+#
+# ICECC_ALLOW_REMOTE="yes"
+ICECC_ALLOW_REMOTE="no"
+...
+}}}
+
+ * Make sure you integrate it seamlessly with ccache by allowing enough jobs your local icecc daemon:
+(CCache run preprocessor with calling icecc, which connects to your local icecc daemon, which run preprocessing locally.
+Don't set ICECC_MAX_JOBS="0" to forbid accepting remote jobs, because in this case CCache can preprocess only on one thread.
+The ideal setting for ICECC_MAX_JOBS is the number of your processors.)
+{{{
+#!sh
+$ cat /etc/icecc/icecc.config
+...
+#
+# You can overwrite here the number of jobs to run in parallel. Per
+# default this depends on the number of (virtual) CPUs installed.
+#
+# Note: a value of "0" is actually interpreted as "1", however it
+# also sets ICECC_ALLOW_REMOTE="no".
+#
+# ICECC_MAX_JOBS=""
+ICECC_MAX_JOBS="<number-of-your-processors>"
+...
+}}}
+
+
+ * You may want to restart the service after setting all the configuration appropriately:
+   * Ubuntu
+{{{
+#!sh
+$ sudo service icecc restart
+}}}
+   * Debian
+{{{
+#!sh
+$ sudo service iceccd restart
+}}}
+
+ * Then compile WebKit normally:
+{{{
+#!sh
+$ Tools/Scripts/build-webkit --gtk
+}}}
+
+
+==== Enable IcecCC in build-webkit
+
+To use icecc, export the CCACHE_PREFIX variable:
+{{{
+#!sh
+$ export CCACHE_PREFIX=icecc
+}}}
+
+As ccache is enabled by default in the SDK, it will now prefix compile commands with icecc. This is all you need to do, toolchains archives automatically generated during the SDK installation will be sent to the scheduler over the network. This is all pre-configured by default when build-webkit detected the CCACHE_PREFIX env var is set to `icecc`.
+
+==== icecc troubleshooting
+
+* If jobs are not being distributed, then check WebKitBuild/Release/rules.ninja is using /usr/lib/ccache/g++.
+{{{
+rule CXX_COMPILER
+  depfile = $DEP_FILE
+  command = /usr/lib/ccache/g++   $DEFINES $FLAGS -MMD -MT $out -MF "$DEP_FILE" -o $out -c $in
+  description = Building CXX object $out
+}}}
+* If you get strange errors when building try to clear the ccache cache (ccache -C) and start with a clean build
+{{{
+#!sh
+$ Tools/Scripts/webkit-flatpak -c ccache -C
+$ rm -fr WebKitBuild/GTK/Release
+}}}