Setting up development environment for Nokia N9

under construction

Build Qt5 and WebKit2 for N9 on Ubuntu 11.10 x64. Running MiniBrowser on the device.

1. Scratchbox Instructions

This section based on

• ​https://github.com/resworb/scripts
• ​https://trac.webkit.org/wiki/BuildingQt5OnHarmattan

Follow the instructions exactly!

1.1. Install and configure Scratchbox for Harmattan

(run@host)

  $ wget http://harmattan-dev.nokia.com/unstable/beta3/harmattan-sdk-setup.py 
  $ chmod +x harmattan-sdk-setup.py 
  $ sudo ./harmattan-sdk-setup.py admininstall 

Post install (run@host):

  $ newgrp sbox

or logout / login to get user access to /scratchbox/login.

Verify by writing (run@host)

  $ /scratchbox/login 

you will get prompt

  [sbox-HARMATTAN_X86: ~] >

and now you can switch to armel target (run@sbox)

  [sbox-HARMATTAN_X86: ~] >sb-conf select HARMATTAN_ARMEL
  [sbox-HARMATTAN_ARMEL: ~] >

CTRL+D to exit.

Sync resolv.conf and host.conf for network access in Scratchbox (run@host):

  $ sudo /scratchbox/sbin/sbox_sync

1.2. Prepare your environment

(run@host)

  $ mkdir /scratchbox/users/$USER/home/$USER/swork 
  $ ln -s /scratchbox/users/$USER/home/$USER/swork ~/swork

NB: You don't need the explicit swork directory as long as the script directory resides on the same level as the source directories.

You should now have a working dir (~/swork), residing in/accessible from Scratchbox, linked in the root of your user root. (Not root user!)

Go to swork directory (run@host):

  $ cd ~/swork

Download scripts for building (run@host):

  $ git clone https://github.com/resworb/scripts.git browser-scripts

Apply the pach attached to this page: ​browser-script.patch

  $ cd browser-scripts
  $ patch -p1 < browser-script.patch

Download testfonts (run@host):

  $ git clone git://gitorious.org/qtwebkit/testfonts.git ~/swork/testfonts

Install required dependencies inside scratchbox (run@host):

  $ /scratchbox/login
  [sbox-HARMATTAN_ARMEL: ~] > cd swork
  [sbox-HARMATTAN_ARMEL: ~/swork] > browser-scripts/builddeps-install.sh
  [sbox-HARMATTAN_ARMEL: ~/swork] > exit
  $

In order to get git installed inside Scratchbox please follow these steps (run@host):

  $ sudo su 
  $ echo "deb http://scratchbox.org/debian harmattan main" >> /etc/apt/sources.list.d/sb.list
  $ apt-get update
  $ apt-get install scratchbox-devkit-git
  $ exit

Add this to your PATH inside Scratchbox (preferrably add to .bashrc) (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~] > echo "export PATH=/scratchbox/devkits/git/bin:\$PATH" >> ~/.bashrc
  [sbox-HARMATTAN_ARMEL: ~] > . ./.bashrc

The next step is getting the source of Qt5 and WebKit (run@host):

  $ browser-scripts/clone-sources.sh

This will try to clone Qt5 and WebKit git repository to ~/swork/qt5 and ~/swork/webkit.

Warning: Sometimes Qt5 cloning fail because of connection timeout problems. This would cause that your ~/swork/qt5/qtscript direcory stays empty and WebKit build will miss QtScript/qscriptengine.h header. In this case delete ~/swork/qt5 and re-run clone-sources.sh .

If you have already a local git repo for WebKit, you can speed up the process by copying its contents to ~/swork/webkit directory. After copying you should run git reset --hard HEAD on it before running clone-sources.sh . You can update the sources by running (run@host):

  $ browser-scripts/update-sources.sh

1.3. Create ICECC toolchain (optional)

If you have multiple workstations configured to use ​ICECC, you can speed up building process significantly.

Create a toolchain for distributed building (run@host):

  $ cd ~/swork
  $ browser-scripts/create-icecc-env.sh > icecc-env.sh

The resulted icecc-env.sh file will be similar to this:

icecc_tarball=$(readlink -f $HOME/icecc/icecc-build)
parallel_builds="40"

if [ -f $icecc_tarball ] ; then
    export ICECC_VERSION="i386:$icecc_tarball,x86_64:$icecc_tarball"
    export PATH="/home/azbest/icecc/bin:$PATH"
    export MAKEFLAGS="$MAKEFLAGS -j$parallel_builds"
    export DEB_BUILD_OPTIONS="$DEB_BUILD_OPTIONS,parallel=$parallel_builds"
fi

Make the script runnable and execute it before building inside scratchbox when you need speedup.

1.4. Build QtWebKit

Login to scratchbox and optionally execute script for icecc (run@host):

  $ /scratchbox/login
  [sbox-HARMATTAN_ARMEL: ~] >
  [sbox-HARMATTAN_ARMEL: ~] > cd ~/swork
  [sbox-HARMATTAN_ARMEL: ~/swork] > . icecc-env.sh

Before building, you have to remove environment variable that was used to work with other Qt version (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > export -n QTDIR

You should also remove /usr/X11R6/bin and Qt from PATH. Check and modify your path variable (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > echo $PATH

You may get something similar to this:

/scratchbox/devkits/git/bin:/host_usr/bin:/scratchbox/devkits/hashutils-squeeze-sdk/bin:/scratchbox/devkits/debian-squeeze/bin:/scratchbox/devkits/perl/bin:/scratchbox/devkits/qemu/bin:/scratchbox/tools/bin:/targets/links/arch_tools/bin:/scratchbox/compilers/bin:/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin

Building Qt5 for Harmattan. You should use the --clean parameter (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > browser-scripts/qt5-build.sh --clean

When building is successful you get a similar message:

Build completed, run the following to use your new Qt build
export PATH=/scratchbox/users/$USER/home/$USER/swork/browser-scripts/../qt5/qtbase/bin:$PATH

So, follow this instuction that printed (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > export PATH=/scratchbox/users/$USER/home/$USER/swork/browser-scripts/../qt5/qtbase/bin:$PATH

Now, build !QtWebKit2 (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > browser-scripts/webkit-build.sh

If everything is right you should get similar result:

...
make[1]: Leaving directory `/home/azbest/swork/qtwebkit-webkit2-dev-builddir-armel-m6/Release'

===========================================================
 WebKit is now built (30m:22s). 
 To run QtTestBrowser with this newly-built code, use the
 "Tools/Scripts/run-launcher" script.
===========================================================

1.5. Updating build environment

If you need to update browser-scripts, run on host:

  $ cd ~/swork/browser-scripts
  $ git reset --hard HEAD
  $ git pull
  $ patch -p1 < ../browser-script.patch

If you neet to update Qt5 and WebKit, run on host:

  $ cd ~/swork/qt5 
  $ git submodule foreach --recursive git clean -fdx
  $ git clean -fdx
  $ git reset --hard HEAD
  $ cd ~/swork/webkit
  $ git clean -fdx
  $ git reset --hard HEAD
  $ cd ~/swork
  $ browser-scripts/update-sources.sh

Than rebuild binaries inside scratchbox:

  [sbox-HARMATTAN_ARMEL: ~] > cd ~/swork
  [sbox-HARMATTAN_ARMEL: ~] > browser-scripts/build-sources.sh

2. Configuring N9

2.1. Turn on developer mode

2.2. Enable root access over ssh

2.3. Install missing packages

3. How you get things running on the N9

3.1.a Transfer files to N9 with rsync

With rsync you can deploy the files to the device.

Required storage:

• (ext4)/home/user - 1GB
• (vfat)/home/user/MyDocs - 4.3GB

Download rsync for N9

• [​http://repo.pub.meego.com/home:/rzr:/harmattan/MeeGo_1.2_Harmattan_Maemo.org_MeeGo_1.2_Harmattan_standard/armel/rsync_3.0.9-1.0%7erzr1_armel.deb

Scp rsync installer to N9 as root:

  $ scp rsync_3.0.9-1.0~rzr1_armel.deb root@device:./
  $ ssh root@device
  RM696-40-4_PR_001:~# dpkg -i rsync_3.0.9-1.0~rzr1_armel.deb
  RM696-40-4_PR_001:~# exit
  $

Create public key for ssh connection, if you don't have it already:

  $ ssh-keygen -t rsa

Copy public key to N9:

  $ ssh user@device "mkdir .ssh"
  $ scp ~/.ssh/id_rsa.pub user@device:./.ssh/authorized_keys

Now, you don't have to enter password for 'user' when you connect to N9.

Download ​rsync-scripts.tar.gz and extract it to ~/swork.

Initialize directory tree for Qt and WebKit files:

  ~/swork/rsync-scripts$ init-rsync.sh

Sync files to device:

  ~/swork/rsync-scripts$ ./all-rsync.sh

This script will execute ./testfonts-rsync.sh, ./qt5-rsync.sh, ./webkit-tests-rsync.sh and ./webkit-build-rsync.sh. This can take some time on first run, so get some coffee or have a lunch.

If you don't want to transfer LayoutTests to N9, then you should use ./all-rsync.sh --no-layout-tests. Without this argument rsync always does re-transfer on LayoutTests because of the vfat filesystem's limitation.

3.1.b Mounting NFS on retail N9

The retail N9 has a very stricky security system, called AEGIS. You can read about this at: ​Harmattan Security Guide

The only known way to disable AEGIS restrictions is flashing an alternate kernel onto N9.

3.1.b.1 Flashing pached kernel to get open mode

Prerequisite (all of this can be found in /common/projects/WebKit/N9-FirmWare)

• Pached kernel: ​zImage-2.6.32.39-dfl61-20113701
• Original firmware 20.2011.40-4
• Nokia flasher: ​flasher_3.12.1_amd64.deb

It is recommended to flash back the device to factory state. If you have any importand data on the device, back it up first or you will lose it!

Install Nokia flasher utility and download the firmware and pached kernel file to a directory.

Turn off the device and disconnect it from usb. Flash the original firmware to the device:

  sudo flasher -F DFL61_HARMATTAN_20.2011.40-4_PR_LEGACY_001-OEM1-958_ARM.bin -F DFL61_HARMATTAN_20.2011.40-4.CENTRALEUROPE_EMMC_CENTRALEUROPE.bin -f

Do NOT disconnect usb cable. Do NOT restart the phone. Now, flash the pached kernel onto it:

  sudo flasher -f -a DFL61_HARMATTAN_20.2011.40-4_PR_LEGACY_001-OEM1-958_ARM.bin -k zImage-2.6.32.39-dfl61-20113701

Now, your N9 is ready. Disconnect it from usb. It will start booting.

3.1.b.2. Developer mode and required libraries

Enable developer mode: Settings / Security / Developer mode. After installation the device will do a reboot.

Change usb connection mode: Settings / Accessories / USB to "SDK" or "Always ask".

Set up wifi internet connection. Connect usb cable.

On N9 menu, select SDK Connectivity / USB. It will show your password for user developer. Leave it open.

Install required libs for NFS mount and add static password for 'user' user (run@host):

ssh developer@192.168.2.15

echo 'rootme' | devel-su
cd /root
apt-get install wget
wget http://maemo.cloud-7.de/HARM/N9/openmode_kernel_PR1.1/opensh/opensh.deb
wget http://repo.pub.meego.com/home:/rzr:/harmattan/MeeGo_1.2_Harmattan_Maemo.org_MeeGo_1.2_Harmattan_standard/armel/libwrap0_7.6.dbs-maemo6.1_armel.deb
wget http://repo.pub.meego.com/home:/rzr:/debian/harmattan/armel/portmap_6.0.0-2_armel.deb

dpkg -i opensh.deb
dpkg -i libwrap0_7.6.dbs-maemo6.1_armel.deb
dpkg -i portmap_6.0.0-2_armel.deb

passwd user

exit
exit

Create public key for ssh connection, if you don't have it already (run@host):

ssh-keygen -t rsa

Copy public key to N9 (run@host):

ssh user@device "mkdir .ssh"
scp ~/.ssh/id_rsa.pub user@device:./.ssh/authorized_keys

Now, you don't have to enter password for 'user' when you connect to N9.

3.1.b.3. Setup NFS on host computer (Ubuntu):

Install the required packages and share swork (run@host):

sudo apt-get install nfs-kernel-server portmap
sudo echo "/home/$USER/swork 192.168.2.15(rw,sync,no_root_squash)" >> /etc/exports 
sudo /etc/init.d/nfs-kernel-server restart

3.1.b.4. Setup NFS on N9

Create script for mounting NFS (run@host):

ssh user@192.168.2.15 "mkdir /home/user/swork; \
echo 'opensh -c \"/sbin/portmap; /sbin/modprobe nfs; mount -t nfs 192.168.2.14:/home/$USER/swork /home/user/swork\"' > nfsmount.sh; \
chmod +x nfsmount.sh "

Connect to N9 (run@host)

ssh user@192.168.2.15

Mount NFS if you didn't do it (run@n9).

. nfsmount.sh

3.2. Running MiniBrowser

3.2.1. Environment variables

Now create script for setting environment variables (run@host):

ssh user@192.168.2.15 "echo 'export QTDIR=/home/user/swork/qt5/qtbase' > nfsenv.sh; \
echo 'export QT_IMPORT_PATH=/home/user/swork/qt5/qtbase/imports' >> nfsenv.sh; \
echo 'export QML_IMPORT_PATH=/home/user/swork/qt5/qtbase/imports' >> nfsenv.sh; \
echo 'export LD_LIBRARY_PATH=/home/user/swork/qt5/qtbase/lib:/home/user/swork/qtwebkit-webkit2-dev-builddir-armel-m6/Release/lib' >> nfsenv.sh; \
echo 'export QT_PLUGIN_PATH=/home/user/swork/qt5/qtbase/plugins' >> nfsenv.sh; \
echo 'export QT_QPA_PLATFORM_PLUGIN_PATH=/home/user/swork/qt5/qtbase/plugins/platforms' >> nfsenv.sh; \
echo 'export QT_QPA_PLATFORM=xcb' >> nfsenv.sh; \
echo 'export WEBKIT_TESTFONTS=/home/user/swork/testfonts' >> nfsenv.sh; \
echo '# export TZ=/usr/share/zoneinfo/America/Los_Angeles' >> nfsenv.sh; \
echo 'export PATH=/home/user/swork/qt5/qtbase/bin:$PATH' >> nfsenv.sh"

Load environment variables (run@n9):

. nfsenv.sh

3.2.2. Launch browser

(run@n9)

~/swork/qtwebkit-webkit2-dev-builddir-armel-m6/Release/bin/MiniBrowser

4. Flash back N9 to factory state