Changes between Version 3 and Version 4 of WebKitFlatpakSDK/SpeedUpBuild


Ignore:
Timestamp:
Aug 5, 2020 9:21:45 AM (2 years ago)
Author:
Philippe Normand
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • WebKitFlatpakSDK/SpeedUpBuild

    v3 v4  
    6666=== IceCC
    6767
    68 TODO
     68
     69[https://github.com/icecc/icecream Icecream] was created by SUSE based on distcc. Like distcc, Icecream takes compile jobs from a build and distributes it among remote machines allowing a parallel build. But unlike distcc, Icecream uses a single central server called scheduler that dynamically assigns the compile jobs to multiple distributed daemons, choosing the fastest free one. This advantage pays off mostly for shared computers, if you're the only user on x machines, you have full control over them.
     70 * You should have only one scheduler in your network.
     71 * The scheduler and one of the daemons can be in the same host.
     72
     73==== Installing
     74
     75 * Execute:
     76{{{
     77#!sh
     78$ sudo apt-get install icecc
     79}}}
     80
     81 * You can install icecc monitor too:
     82{{{
     83#!sh
     84$ sudo apt-get install icecc-monitor
     85}}}
     86
     87
     88==== icecc scheduler
     89
     90 * Configure scheduler to start by default (see `/usr/share/doc/icecc/README.Debian`):
     91{{{
     92#!sh
     93$ sudo update-rc.d icecc-scheduler enable
     94}}}
     95
     96 * After configuring the scheduler to start, it will do so on next reboot, but not sooner. You can start it manually:
     97   * Ubuntu
     98{{{
     99#!sh
     100$ sudo service icecc restart
     101}}}
     102   * Debian
     103{{{
     104#!sh
     105$ sudo service icecc-scheduler start
     106}}}
     107
     108==== icecc daemon(s)
     109
     110 * iceccd (daemon) has to be able to find the scheduler in the network. By default, it makes so by sending a broadcast message. This may not work depending on your network topology (routers, firewalls, etc.). You can specify the actual address of the icecc-scheduler by stating it in the icecc config file:
     111{{{
     112#!sh
     113$ cat /etc/icecc/icecc.config
     114...
     115#
     116# If the daemon can't find the scheduler by broadcast (e.g. because
     117# of a firewall) you can specify it.
     118#
     119# ICECC_SCHEDULER_HOST=""
     120ICECC_SCHEDULER_HOST="<icecc-scheduler IP or host name>"
     121...
     122}}}
     123
     124 * Also, you may not want your local icecc daemon to run works from other hosts. For example, I want my laptop to be helped by my tower but I don't want my laptop to take works from my tower:
     125{{{
     126#!sh
     127$ cat /etc/icecc/icecc.config
     128...
     129#
     130# Specifies whether jobs submitted by other nodes are allowed to run on
     131# this one.
     132#
     133# ICECC_ALLOW_REMOTE="yes"
     134ICECC_ALLOW_REMOTE="no"
     135...
     136}}}
     137
     138 * Make sure you integrate it seamlessly with ccache by allowing enough jobs your local icecc daemon:
     139(CCache run preprocessor with calling icecc, which connects to your local icecc daemon, which run preprocessing locally.
     140Don't set ICECC_MAX_JOBS="0" to forbid accepting remote jobs, because in this case CCache can preprocess only on one thread.
     141The ideal setting for ICECC_MAX_JOBS is the number of your processors.)
     142{{{
     143#!sh
     144$ cat /etc/icecc/icecc.config
     145...
     146#
     147# You can overwrite here the number of jobs to run in parallel. Per
     148# default this depends on the number of (virtual) CPUs installed.
     149#
     150# Note: a value of "0" is actually interpreted as "1", however it
     151# also sets ICECC_ALLOW_REMOTE="no".
     152#
     153# ICECC_MAX_JOBS=""
     154ICECC_MAX_JOBS="<number-of-your-processors>"
     155...
     156}}}
     157
     158
     159 * You may want to restart the service after setting all the configuration appropriately:
     160   * Ubuntu
     161{{{
     162#!sh
     163$ sudo service icecc restart
     164}}}
     165   * Debian
     166{{{
     167#!sh
     168$ sudo service iceccd restart
     169}}}
     170
     171 * Then compile WebKit normally:
     172{{{
     173#!sh
     174$ Tools/Scripts/build-webkit --gtk
     175}}}
     176
     177
     178==== Enable IcecCC in build-webkit
     179
     180To use icecc, export the CCACHE_PREFIX variable:
     181{{{
     182#!sh
     183$ export CCACHE_PREFIX=icecc
     184}}}
     185
     186As ccache is enabled by default in the SDK, it will now prefix compile commands with icecc. This is all you need to do, toolchains archives automatically generated during the SDK installation will be sent to the scheduler over the network. This is all pre-configured by default when build-webkit detected the CCACHE_PREFIX env var is set to `icecc`.
     187
     188==== icecc troubleshooting
     189
     190* If jobs are not being distributed, then check WebKitBuild/Release/rules.ninja is using /usr/lib/ccache/g++.
     191{{{
     192rule CXX_COMPILER
     193  depfile = $DEP_FILE
     194  command = /usr/lib/ccache/g++   $DEFINES $FLAGS -MMD -MT $out -MF "$DEP_FILE" -o $out -c $in
     195  description = Building CXX object $out
     196}}}
     197* If you get strange errors when building try to clear the ccache cache (ccache -C) and start with a clean build
     198{{{
     199#!sh
     200$ Tools/Scripts/webkit-flatpak -c ccache -C
     201$ rm -fr WebKitBuild/GTK/Release
     202}}}