Changes between Version 1 and Version 2 of JSCOnly/CrossBuildAndRemoteTestJSCLinux

Timestamp:

Feb 5, 2018, 4:05:58 PM (8 years ago)

Author:

clopez@igalia.com

Comment:

--

JSCOnly/CrossBuildAndRemoteTestJSCLinux

@@ -5 +5 @@
 This document explains how to cross-build the [[JSCOnly]] port for a target board (like a RaspberryPi) and run the JSC tests suite on the board remotely.[[BR]]
 To achieve that we build both an image for the board and a cross-toolchain with buildroot.
-
+Creating a helper script that exports the required variables documented below is a good idea.
+
+== Instructions for building the image and the toolchain with buildroot == 
+
+
+1. Download and unpack the last stable version of buildroot. Example:
+  {{{
+wget https://buildroot.org/downloads/buildroot-2017.11.2.tar.bz2
+tar xfa buildroot-2017.11.2.tar.bz2
+cd buildroot-2017.11.2
+  }}}
+
+2. pre-configure buildroot with your target board plus this config fragment:
+
+  2.1. download webkit_jsc_buildroot_extraconfig and save it as /tmp/webkit_jsc_buildroot_extraconfig
+
+  2.2. guess the name of your board defconfig (check the files in config/) and merge the default config for it with the attached one. For example, for configuring for using the RPi3 in ARM64 (Aarch64) mode we will do:
+    {{{
+support/kconfig/merge_config.sh configs/raspberrypi3_64_defconfig /tmp/webkit_jsc_buildbot_extraconfig 
+    }}}
+
+
+3. run "make menuconfig" and review your systems details, pay special attention the following sections:
+
+   3.1 Target options: review that the selected options are ok:
+
+     3.1.1 For example, if the  board is ```ARM32``` (```ARMv7```) is not the same to enable ```Thumb2``` instructions than not. Also the type of ```FPU``` selected is relevant.
+
+   3.2 Check that ```ICU``` will be built for the target:  (check the value ```BR2_PACKAGE_ICU=y``` )
+
+   3.3 Check that the compiler has C++ support enabled (```Toolchain -> Enable C++ support```)
+
+   3.4 Check also on ```Filesystem images``` the size of the image to be generated: Set it to at least to a 2GB size so it has enough free space for storing all the subproducts of the JSC tests.
+
+   3.5 Set the builroot hostdir (```Build options -> Host dir```) to the path where the toolchain should be installed. Example:
+    {{{
+$ cat .config|grep BR2_HOST_DIR
+BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"
+    }}}
+
+    Save the value of this variable for later.
+
+
+4. Build the buildroot toolchain and the board image.
+  {{{
+$ make -j $(nproc)
+  }}}
+  When it has finished you should have available in the path you defined for ```${BR2_HOST_DIR}``` the cross-toolchain. And you should have available in the subdir output/target of buildroot the image or rootfs that should be flashed on the board.
+
+
+==  Instructions for cross-compiling JSC  with the buildroot toolchain ==
+
+=== Defining the environment variables for the cross-compiler ===
+
+1. export this environment variables
+  {{{
+export BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"
+export CROSS_COMPILE="$(basename $(cat ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake|grep CMAKE_CXX_COMPILER|awk -F'"' '{print $2}')|sed "s/g++$//g")"
+export PATH="${BR2_HOST_DIR}/usr/bin:${PATH}"
+export CC="${CROSS_COMPILE}gcc"
+export CXX="${CROSS_COMPILE}g++"
+  }}}
+
+2. Check that the CROSS_COMPILE one evaluates to the cross-compiler name without the compiler suffix. Example:
+  {{{
+
+$ echo ${CROSS_COMPILE}
+aarch64-buildroot-linux-gnu-
+
+$ echo ${CROSS_COMPILE}gcc
+aarch64-buildroot-linux-gnu-gcc
+  }}}
+
+3. Pass the flag ```-DCMAKE_TOOLCHAIN_FILE``` to cmake to the path of the toolchainfile.cmake that buildroot generates. If you use the WebKit internal tooling for building (script Tools/Scripts/build-jsc) you can do that  by simply exporting this environment variable:
+  {{{
+export BUILD_JSC_ARGS="--cmakeargs=-DCMAKE_TOOLCHAIN_FILE=${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake" 
+  }}}
+
+=== Enabling ccache (optional, but recommended) === 
+
+If you wish to enable ccache, you can do this:
+
+1. First install ccache (the native from the host, via apt-get or similar)
+
+2. Create a ccache dir inside ```${BR2_HOST_DIR}``` and cd into it
+  {{{
+mkdir "${BR2_HOST_DIR}/usr/ccache"
+cd "${BR2_HOST_DIR}/usr/ccache"
+  }}}
+
+3. Create symlinks to ccache with the name of the CC and CXX compilers
+  {{{
+# Define CC and CXX like above
+ln -s "$(which ccache)" "${CXX}"
+ln -s "$(which ccache)" "${CC}"
+  }}}
+
+4. Replace on the ```${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake``` file the paths to the C and CXX compilers, so they point to ```${RELOCATED_HOST_DIR}/ccache``` instead of to ```${RELOCATED_HOST_DIR}/bin```
+  {{{
+sed -i '/set(CMAKE_C/s,/bin/,/ccache/,' ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake
+  }}}
+
+5. Now, '''each''' time you build, set your path with the ccache directory first
+  {{{
+export PATH="${BR2_HOST_DIR}/usr/ccache:${BR2_HOST_DIR}/usr/bin:${PATH}"
+  }}}
+  
+== Instructions for running the tests remotely on the board ==
+
+The script [https://trac.webkit.org/browser/trunk/Tools/Scripts/run-javascriptcore-tests run-javascriptcore-tests] allows to run the jsc tests remotely on the board.
+This script will generate a tarball bundled with the built jsc and all the libraries linking to it, plus all what is needed for run the tests.
+To make use of this follow the instructions below:
+
+=== Setup cross-ldd script ===
+
+For identifying which libraries link to the jsc binary, [https://trac.webkit.org/browser/trunk/Tools/Scripts/run-javascriptcore-tests run-javascriptcore-tests] executes the script ldd, but this tool (ldd) won't work out of the box on a cross-build environment.
+
+To fix that we use the attached cross-ldd.
+So we need to put this script renamed to ldd on the binary path of the cross-toolchain (```${BR2_HOST_DIR}/bin```)
+
+{{{
+# Save attached cross-ldd as /tmp/cross-ldd
+$ cp /tmp/cross-ldd ${BR2_HOST_DIR}/usr/bin/ldd
+# Important! make it executable:
+$ chmod +x ${BR2_HOST_DIR}/usr/bin/ldd
+}}}
+
+Check that it worked:
+
+{{{
+$ which ldd
+/home/buildbot/toolchains/aarch64/buildroot_2017.11.2/usr/bin/ldd
+
+# We need to export this environment variables every-time we want to use this.
+export BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"
+export CROSS_COMPILE="$(basename $(cat ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake|grep CMAKE_CXX_COMPILER|awk -F'"' '{print $2}')|sed "s/g++$//g")"
+export PATH="${BR2_HOST_DIR}/usr/bin:${PATH}"
+export CC="${CROSS_COMPILE}gcc"
+export CXX="${CROSS_COMPILE}g++"
+
+$ cd WebKit
+$ ldd WebKitBuild/Release/bin/jsc
+        libdl.so.2 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libdl.so.2 (0x00000000deadbeef)
+        libc.so.6 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libc.so.6 (0x00000000deadbeef)
+        ld-linux-aarch64.so.1 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/ld-linux-aarch64.so.1 (0x00000000deadbeef)
+        libJavaScriptCore.so.1 => /home/buildbot/jsconly/jsconly-linux-aarch64-release/build/WebKitBuild/Release/lib/libJavaScriptCore.so.1 (0x00000000deadc0de)
+        libicui18n.so.59 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libicui18n.so.59 (0x00000000deadbeef)
+        libicuuc.so.59 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libicuuc.so.59 (0x00000000deadbeef)
+        libicudata.so.59 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libicudata.so.59 (0x00000000deadbeef)
+        libpthread.so.0 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libpthread.so.0 (0x00000000deadbeef)
+        libatomic.so.1 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libatomic.so.1 (0x00000000deadbeef)
+        libstdc++.so.6 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libstdc++.so.6 (0x00000000deadbeef)
+        libm.so.6 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libm.so.6 (0x00000000deadbeef)
+        libgcc_s.so.1 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libgcc_s.so.1 (0x00000000deadbeef)
+
+}}}
+
+
+Note: Replace the above IP with the one from your board.
+
+=== Configure the SSH access to the board ===
+
+1. First copy your ssh key to the board so you can login passwordless
+   {{{
+$ ssh-keygen
+# default root password is "jsc"
+$ ssh-copy-id root@jsc-rpi3-64.board.local
+   }}}
+   Now double-check that you can ssh on the board without password:
+   {{{
+ssh-copy-id root@jsc-rpi3-64.board.local
+   }}}
+
+   Also accept any ssh key so next time it don't answers any question and the webkit tooling later can ssh automatically.
+
+2. Then configure the bencher config file on the board.
+  ssh on the board and create in the home directory a file named '.bencher' with the path where to copy the jsc products and tests for running.
+  {{{
+dev-machine $ ssh root@jsc-rpi3-64.board.local
+
+board-rpi64 # mkdir /root/jsc-temp
+board-rpi64 # echo '{"tempPath": "/root/jsc-temp"}' > /root/.bencher
+ }}}
+
+3. Now configure the remote config file
+  The Script Tools/Scripts/run-javascriptcore-tests also expects some config parameters with the IP of the board. You can pass that parameters by creating a file with this contents
+  {{{
+$ cat remote-jsc-tests-config.json 
+{"remote": "root@192.168.10.173:22"}
+  }}}
+
+4. Finally, execute run-javascriptcore-tests as follows:
+  {{{
+# NOTE: The cross-ldd script needs the  environment variables below set to work properly.
+export BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"
+export CROSS_COMPILE="$(basename $(cat ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake|grep CMAKE_CXX_COMPILER|awk -F'"' '{print $2}')|sed "s/g++$//g")"
+export PATH="${BR2_HOST_DIR}/usr/bin:${PATH}"
+export CC="${CROSS_COMPILE}gcc"
+export CXX="${CROSS_COMPILE}g++"
+
+Tools/Scripts/run-javascriptcore-tests --jsc-only --release --no-build --no-fail-fast --memory-limited  --remote-config-file ${pathtothefileabove}/remote-jsc-tests-config.json
+  }}}