wiki:JSCOnly/CrossBuildAndRemoteTestJSCLinux

Cross-Building and remote testing of JSC for Linux

This document explains how to cross-build the JSCOnly port for a target board (like a RaspberryPi) and run the JSC tests suite on the board remotely.
To achieve that we build both an image for the board and a cross-toolchain with buildroot.
Tip: if you are going to do this often, then creating a helper script to exports the required environment variables documented below is a good idea.

Using buildroot to build the toolchain for cross-compilation and the image for the board

  1. Download and unpack the last stable version of buildroot. Example:
    wget https://buildroot.org/downloads/buildroot-2017.11.2.tar.bz2
    tar xfa buildroot-2017.11.2.tar.bz2
    cd buildroot-2017.11.2
    
  • NOTE for RPi boards: It seems there is currently a bug in the image generation of buildroot-2017.11.2 for RPI2 and RPI3 in 32 bit mode (not for 64-bit mode) that results in images that don't boot. If you are targeting those and hit this problem then try to use instead the LTS version of buildroot (2017.02.9.tar.gz). If you are targeting RPI3 in 64-bit mode then use buildroot-2017.11.2 or later as the previous versions doesn't have support for it.
  1. pre-configure buildroot with your target board plus this config fragment:

2.1. download jsc_buildroot_extraconfig and save it to /tmp/

2.2. guess the name of your board defconfig (check the files in ${buildroot_dir}/config) and merge the default config for it with the attached one.
For example, for configuring for using the RPi3 in ARM64 (Aarch64) mode we will do:

support/kconfig/merge_config.sh -n configs/raspberrypi3_64_defconfig /tmp/jsc_buildroot_extraconfig
  1. run make menuconfig and review your systems details, pay special attention the following sections:
make menuconfig

3.1 Target options: review that the selected options are ok:

3.1.1 For example, if the board is ARM32 (ARMv7) is not the same to enable Thumb2 instructions than not. Also the type of FPU selected is relevant.

3.2 Check that ICU will be built for the target: (check the value BR2_PACKAGE_ICU=y )

3.3 Check that the compiler has C++ support enabled (Toolchain -> Enable C++ support)

3.4 Check also on Filesystem images the size of the image to be generated: Set it to at least to a 2GB size so it has enough free space for storing all the subproducts of the JSC tests.

3.5 Set the builroot hostdir (Build options -> Host dir) to the path where the toolchain should be installed. Example:

$ cat .config|grep BR2_HOST_DIR
BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"

Save the value of this variable for later.

  1. Build the buildroot toolchain and the board image.
    $ make -j $(nproc)
    
    When it has finished the cross-toolchain will be available in the path defined before for ${BR2_HOST_DIR}.
    And the image (or rootfs) that should be flashed to the board will be available in the subdir output/target of buildroot

Cross-compiling JSC with the buildroot toolchain

Now that we have the cross-toolchain built for the target board, building JSC is a matter of exporting the right environment variables before invoking the build-process
Tip: Its recommended start from a WebKit/JSC clean-build state to avoid problems with the CMake cache.

Defining the environment variables for the cross-compiler

  1. export this environment variables
    export BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"
    export CROSS_COMPILE="$(basename $(cat ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake|grep CMAKE_CXX_COMPILER|awk -F'"' '{print $2}')|sed "s/g++$//g")"
    export PATH="${BR2_HOST_DIR}/usr/bin:${PATH}"
    export CC="${CROSS_COMPILE}gcc"
    export CXX="${CROSS_COMPILE}g++"
    
  1. Check that the CROSS_COMPILE one evaluates to the cross-compiler name without the compiler suffix. Example:
    $ echo ${CROSS_COMPILE}
    aarch64-buildroot-linux-gnu-
    
    $ echo ${CROSS_COMPILE}gcc
    aarch64-buildroot-linux-gnu-gcc
    
  1. Pass the flag -DCMAKE_TOOLCHAIN_FILE to cmake to the path of the toolchainfile.cmake that buildroot generates.
    If you use the script build-jsc for building, then you can do that by exporting the environment variable BUILD_JSC_ARGS as follows:
    export BUILD_JSC_ARGS="--cmakeargs=-DCMAKE_TOOLCHAIN_FILE=${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake"
    Tools/Scripts/build-jsc --jsc-only --release
    

Enabling ccache (optional, but recommended)

If you wish to enable ccache, you can do this:

  1. First install ccache (the native from the host, via apt-get or similar)
  1. Create a ccache dir inside ${BR2_HOST_DIR} and cd into it
    mkdir "${BR2_HOST_DIR}/usr/ccache"
    cd "${BR2_HOST_DIR}/usr/ccache"
    
  1. Create symlinks to ccache with the name of the CC and CXX compilers
    # Define CC and CXX like above
    ln -s "$(which ccache)" "${CXX}"
    ln -s "$(which ccache)" "${CC}"
    
  1. Replace on the ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake file the paths to the C and CXX compilers, so they point to ${RELOCATED_HOST_DIR}/usr/ccache instead of to ${RELOCATED_HOST_DIR}/usr/bin
    sed -i '/set(CMAKE_C/s,/bin/,/ccache/,' ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake
    
  1. Now, each time you build, set your path with the ccache directory first
    export PATH="${BR2_HOST_DIR}/usr/ccache:${BR2_HOST_DIR}/usr/bin:${PATH}"
    

Tip: If you tried before a build without ccache and want to enable it now, then its recommended to re-start from a WebKit/JSC clean-build state to avoid problems with the CMake cache.

Running the JSC tests remotely on the board

The script run-javascriptcore-tests allows to run the jsc tests remotely on the board.
This script will generate a tarball bundled with the built jsc and all the libraries linking to it, plus all what is needed for run the tests.
To make use of this follow the instructions below:

Setup cross-ldd script

For identifying which libraries link to the jsc binary, run-javascriptcore-tests executes the script ldd, but this tool (ldd) won't work out of the box on a cross-build environment.

To fix that we use the attached cross-ldd. So we need to put this script renamed to ldd on the binary path of the cross-toolchain (${BR2_HOST_DIR}/bin)

# Fist save attached cross-ldd into /tmp
$ cp /tmp/cross-ldd ${BR2_HOST_DIR}/usr/bin/ldd

# IMPORTANT! make it executable:
$ chmod +x ${BR2_HOST_DIR}/usr/bin/ldd

Check that it worked:

$ which ldd
/home/buildbot/toolchains/aarch64/buildroot_2017.11.2/usr/bin/ldd

# We need to export this environment variables every-time we want to use this.
export BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"
export CROSS_COMPILE="$(basename $(cat ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake|grep CMAKE_CXX_COMPILER|awk -F'"' '{print $2}')|sed "s/g++$//g")"
export PATH="${BR2_HOST_DIR}/usr/bin:${PATH}"
export CC="${CROSS_COMPILE}gcc"
export CXX="${CROSS_COMPILE}g++"

$ cd WebKit
$ ldd WebKitBuild/Release/bin/jsc
        libdl.so.2 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libdl.so.2 (0x00000000deadbeef)
        libc.so.6 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libc.so.6 (0x00000000deadbeef)
        ld-linux-aarch64.so.1 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/ld-linux-aarch64.so.1 (0x00000000deadbeef)
        libJavaScriptCore.so.1 => /home/buildbot/jsconly/jsconly-linux-aarch64-release/build/WebKitBuild/Release/lib/libJavaScriptCore.so.1 (0x00000000deadc0de)
        libicui18n.so.59 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libicui18n.so.59 (0x00000000deadbeef)
        libicuuc.so.59 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libicuuc.so.59 (0x00000000deadbeef)
        libicudata.so.59 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libicudata.so.59 (0x00000000deadbeef)
        libpthread.so.0 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libpthread.so.0 (0x00000000deadbeef)
        libatomic.so.1 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libatomic.so.1 (0x00000000deadbeef)
        libstdc++.so.6 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/usr/lib/libstdc++.so.6 (0x00000000deadbeef)
        libm.so.6 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libm.so.6 (0x00000000deadbeef)
        libgcc_s.so.1 => /home/buildbot/toolchains/aarch64/buildroot_2017.11.2/aarch64-buildroot-linux-gnu/sysroot/lib/libgcc_s.so.1 (0x00000000deadbeef)

Note: Replace the above IP with the one from your board.

Configure the SSH access to the board

  1. First copy your ssh key to the board so you can login passwordless
    $ ssh-keygen
    # default root password is "jsc"
    $ ssh-copy-id root@jsc-rpi3-64.board.local
    
    Now double-check that you can ssh on the board without password:
    ssh-copy-id root@jsc-rpi3-64.board.local
    

Also accept any ssh key so next time it don't answers any question and the webkit tooling later can ssh automatically.

  1. Then configure the bencher config file on the board. ssh on the board and create in the home directory a file named '.bencher' with the path where to copy the jsc products and tests for running.
    dev-machine $ ssh root@jsc-rpi3-64.board.local
    
    board-rpi64 # mkdir /root/jsc-temp
    board-rpi64 # echo '{"tempPath": "/root/jsc-temp"}' > /root/.bencher
    
  1. Now configure the remote config file The script run-javascriptcore-tests expects some config parameters with the IP of the board. You can pass that parameters by creating a file with this contents
    # cat ${HOME}/remote-jsc-tests-config.json 
    {"remote": "root@jsc-rpi3-64.board.local:22"}
    
  1. Finally, execute run-javascriptcore-tests as follows:
    # NOTE: The cross-ldd script (which run-javascriptcore-tests executes) needs the environment variables below set to work properly.
    export BR2_HOST_DIR="/home/buildbot/toolchains/aarch64/buildroot_2017.11.2"
    export CROSS_COMPILE="$(basename $(cat ${BR2_HOST_DIR}/usr/share/buildroot/toolchainfile.cmake|grep CMAKE_CXX_COMPILER|awk -F'"' '{print $2}')|sed "s/g++$//g")"
    export PATH="${BR2_HOST_DIR}/usr/bin:${PATH}"
    export CC="${CROSS_COMPILE}gcc"
    export CXX="${CROSS_COMPILE}g++"
    
    Tools/Scripts/run-javascriptcore-tests --jsc-only --release --no-build --no-fail-fast --memory-limited  --remote-config-file ${pathtothefileabove}/remote-jsc-tests-config.json
    
Last modified 4 years ago Last modified on Feb 7, 2018 11:35:37 AM

Attachments (2)

Download all attachments as: .zip