Setting up development environment for Nokia N9

under construction

Build Qt5 and WebKit2 for N9 on Ubuntu 11.10 x64. Running MiniBrowser on the device.

1. Scratchbox Instructions

This section based on

• ​https://github.com/resworb/scripts
• ​https://trac.webkit.org/wiki/BuildingQt5OnHarmattan

Follow the instructions exactly!

1.1. Install and configure Scratchbox for Harmattan

(run@host)

  $ wget http://harmattan-dev.nokia.com/unstable/beta3/harmattan-sdk-setup.py 
  $ chmod +x harmattan-sdk-setup.py 
  $ sudo ./harmattan-sdk-setup.py admininstall 

Post install (run@host):

  $ newgrp sbox

or logout / login to get user access to /scratchbox/login.

Verify by writing (run@host)

  $ /scratchbox/login 

you will get prompt

  [sbox-HARMATTAN_X86: ~] >

and now you can switch to armel target (run@sbox)

  [sbox-HARMATTAN_X86: ~] >sb-conf select HARMATTAN_ARMEL
  [sbox-HARMATTAN_ARMEL: ~] >

CTRL+D to exit.

Sync resolv.conf and host.conf for network access in Scratchbox (run@host):

  $ sudo /scratchbox/sbin/sbox_sync

1.2. Prepare your environment

(run@host)

  $ mkdir /scratchbox/users/$USER/home/$USER/swork 
  $ ln -s /scratchbox/users/$USER/home/$USER/swork ~/swork

NB: You don't need the explicit swork directory as long as the script directory resides on the same level as the source directories.

You should now have a working dir (~/swork), residing in/accessible from Scratchbox, linked in the root of your user root. (Not root user!)

Go to swork directory (run@host):

  $ cd ~/swork

Download scripts for building (run@host):

  $ git clone https://github.com/resworb/scripts.git browser-scripts

Apply the patch attached to this page: ​browser-script.patch

  $ cd browser-scripts
  $ patch -p1 < browser-script.patch

Download testfonts (run@host):

  $ git clone git://gitorious.org/qtwebkit/testfonts.git ~/swork/testfonts

Install required dependencies inside scratchbox (run@host):

  $ /scratchbox/login
  [sbox-HARMATTAN_ARMEL: ~] > cd swork
  [sbox-HARMATTAN_ARMEL: ~/swork] > browser-scripts/builddeps-install.sh
  [sbox-HARMATTAN_ARMEL: ~/swork] > exit
  $

In order to get git installed inside Scratchbox please follow these steps (run@host):

  $ sudo su 
  $ echo "deb http://scratchbox.org/debian harmattan main" >> /etc/apt/sources.list.d/sb.list
  $ apt-get update
  $ apt-get install scratchbox-devkit-git
  $ exit

Add this to your PATH inside Scratchbox (preferrably add to .bashrc) (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~] > echo "export PATH=/scratchbox/devkits/git/bin:\$PATH" >> ~/.bashrc
  [sbox-HARMATTAN_ARMEL: ~] > . ./.bashrc

The next step is getting the source of Qt5 and WebKit (run@host):

  $ browser-scripts/clone-sources.sh

This will try to clone Qt5 and WebKit git repository to ~/swork/qt5 and ~/swork/webkit.

Warning: Sometimes Qt5 cloning fail because of connection timeout problems. This would cause that your ~/swork/qt5/qtscript direcory stays empty and WebKit build will miss QtScript/qscriptengine.h header. In this case delete ~/swork/qt5 and re-run clone-sources.sh .

If you have already a local git repo for WebKit, you can speed up the process by copying its contents to ~/swork/webkit directory. After copying you should run git reset --hard HEAD on it before running clone-sources.sh . You can update the sources by running (run@host):

  $ browser-scripts/update-sources.sh

1.3. Create ICECC toolchain (optional)

If you have multiple workstations configured to use ​ICECC, you can speed up building process significantly.

Create a toolchain for distributed building (run@host):

  $ cd ~/swork
  $ browser-scripts/create-icecc-env.sh

Save the printed output to a file called icecc-env.sh:

icecc_tarball=$(readlink -f $HOME/icecc/icecc-build)
parallel_builds="40"

if [ -f $icecc_tarball ] ; then
    export ICECC_VERSION="i386:$icecc_tarball,x86_64:$icecc_tarball"
    export PATH="/home/azbest/icecc/bin:$PATH"
    export MAKEFLAGS="$MAKEFLAGS -j$parallel_builds"
    export DEB_BUILD_OPTIONS="$DEB_BUILD_OPTIONS,parallel=$parallel_builds"
fi

Make the script runnable and execute it before building inside scratchbox when you need speedup.

1.4. Build QtWebKit

Login to scratchbox and optionally execute script for icecc (run@host):

  $ /scratchbox/login
  [sbox-HARMATTAN_ARMEL: ~] >
  [sbox-HARMATTAN_ARMEL: ~] > cd ~/swork
  [sbox-HARMATTAN_ARMEL: ~/swork] > . icecc-env.sh

Before building, you have to remove environment variable that was used to work with other Qt version (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > export -n QTDIR

You should also remove /usr/X11R6/bin and Qt from PATH. Check and modify your path variable (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > echo $PATH

You may get something similar to this:

/scratchbox/devkits/git/bin:/host_usr/bin:/scratchbox/devkits/hashutils-squeeze-sdk/bin:/scratchbox/devkits/debian-squeeze/bin:/scratchbox/devkits/perl/bin:/scratchbox/devkits/qemu/bin:/scratchbox/tools/bin:/targets/links/arch_tools/bin:/scratchbox/compilers/bin:/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin

Building Qt5 for Harmattan. You should use the --clean parameter (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > browser-scripts/qt5-build.sh --clean

When building is successful you get a similar message:

Build completed, run the following to use your new Qt build
export PATH=/scratchbox/users/$USER/home/$USER/swork/browser-scripts/../qt5/qtbase/bin:$PATH

So, follow this instuction that printed (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > export PATH=/scratchbox/users/$USER/home/$USER/swork/browser-scripts/../qt5/qtbase/bin:$PATH

Now, build !QtWebKit2 (run@sbox):

  [sbox-HARMATTAN_ARMEL: ~/swork] > browser-scripts/webkit-build.sh

If everything is right you should get similar result:

...
make[1]: Leaving directory `/home/azbest/swork/qtwebkit-webkit2-dev-builddir-armel-m6/Release'

===========================================================
 WebKit is now built (30m:22s). 
 To run QtTestBrowser with this newly-built code, use the
 "Tools/Scripts/run-launcher" script.
===========================================================

1.5. Updating build environment

If you need to update browser-scripts, run on host:

  $ cd ~/swork/browser-scripts
  $ git reset --hard HEAD
  $ git pull
  $ patch -p1 < ../browser-script.patch

If you neet to update Qt5 and WebKit, run on host:

  $ cd ~/swork/qt5 
  $ git submodule foreach --recursive git clean -fdx
  $ git clean -fdx
  $ git reset --hard HEAD
  $ cd ~/swork/webkit
  $ git clean -fdx
  $ git reset --hard HEAD
  $ cd ~/swork
  $ browser-scripts/update-sources.sh

Than rebuild binaries inside scratchbox:

  [sbox-HARMATTAN_ARMEL: ~] > cd ~/swork
  [sbox-HARMATTAN_ARMEL: ~] > browser-scripts/build-sources.sh

2. Configuring N9

2.1. Turn on developer mode

Notice: If you want to use NFS with retail N9, please read section 3.1.b.1 first.

There is a tutorial for this at ​http://paazio.nanbudo.fi/tutorials/qt-quick/developer-mode-in-nokia-n9

Enable developer mode: Settings / Security / Developer mode. After installation the device will do a reboot.

Change usb connection mode: Settings / Accessories / USB to "SDK" or "Always ask".

Set up wifi internet connection. Connect usb cable.

2.2. Enable root access over ssh

Login as 'developer' (run@host):

  $ ssh developer@192.168.2.15
developer@192.168.2.15's password: ****** (getting this from N9 usb connection)

Switch to root (run@n9):

  /home/developer $ devel-su
Password: rootme

You can create permanent password for default user 'user' (run@n9):

  ~ # passwd user
New password: password
Re-enter new password: password

Enable root login on ssh (run@n9):

  ~ #vi /etc/ssh/sshd_config

PermitRootLogin yes

After saving sshd_config, modify default ssh options (run@n9):

  ~ #echo -e 'mkdir -m 0755 -p /var/run/sshd\nexec /usr/sbin/sshd $SSHD_OPTS' >> /etc/default/ssh
  ~ #pkill -9 /usr/sbin/sshd

Now you can access your N9 as root over ssh. Be careful: it is a huge security risc. (run@host)

  $ ssh -X root@192.168.2.15
Password: rootme

Maybe you should change default root password, too.

2.3. Install missing packages

Resworb scripts has a solution for this.

This script uses name 'device' for N9. You can add this to your /etc/hosts file (run@host):

  $ sudo su
  $ echo -e '192.168.2.15\tdevice' >> /etc/hosts
  $ exit

Scrachbox should be configured to armel target befor running this script (run@host):

Run

  $ cd ~/swork
  $ browser-scripts/device-install-packages-n9.sh

It will ask for root password of the device.

3. How you get things running on the N9

3.1.a Transfer files to N9 with rsync

With rsync you can deploy the files to the device.

Required storage:

• (ext4)/home/user - 1GB
• (vfat)/home/user/MyDocs - 4.3GB

Download rsync for N9

• [​http://repo.pub.meego.com/home:/rzr:/harmattan/MeeGo_1.2_Harmattan_Maemo.org_MeeGo_1.2_Harmattan_standard/armel/rsync_3.0.9-1.0%7erzr1_armel.deb

Scp rsync installer to N9 as root (run@host):

  $ scp rsync_3.0.9-1.0~rzr1_armel.deb root@device:./
  $ ssh root@device
  RM696-40-4_PR_001:~# dpkg -i rsync_3.0.9-1.0~rzr1_armel.deb
  RM696-40-4_PR_001:~# exit
  $

Create public key for ssh connection, if you don't have it already (run@host):

  $ ssh-keygen -t rsa

Copy public key to N9 (run@host):

  $ ssh user@device "mkdir .ssh"
  $ scp ~/.ssh/id_rsa.pub user@device:./.ssh/authorized_keys

Now, you don't have to enter password for 'user' when you connect to N9.

Download ​rsync-scripts.tar.gz and extract it to ~/swork.

Initialize directory tree for Qt and WebKit files (run@host):

  ~/swork/rsync-scripts$ init-rsync.sh

Sync files to device:

  ~/swork/rsync-scripts$ ./all-rsync.sh

This script will execute ./testfonts-rsync.sh, ./qt5-rsync.sh, ./webkit-tests-rsync.sh and ./webkit-build-rsync.sh. This can take some time on first run, so get some coffee or have a lunch.

If you don't want to transfer LayoutTests to N9, then you should use ./all-rsync.sh --no-layout-tests. Without this argument rsync always does re-transfer on LayoutTests because of the vfat filesystem's limitation.

3.1.b Mounting NFS on retail N9

The retail N9 has a very stricky security system, called AEGIS. You can read about this at: ​Harmattan Security Guide

The only known way to disable AEGIS restrictions is flashing an alternate kernel onto N9.

Notice: This could void the warranty! After flashing the device will show a warranty void message while booting.

3.1.b.1 Flashing pached kernel to get open mode

Prerequisite

• Pached kernel: ​zImage-2.6.32.39-dfl61-20113701
• Original firmware 20.2011.40-4 (you can download it with ​NaviFirm+
• Nokia flasher: ​flasher_3.12.1_amd64.deb

It is recommended to flash back the device to factory state. If you have any importand data on the device, back it up first or you will lose it!

Install Nokia flasher utility and download the firmware and pached kernel file to a directory.

Turn off the device and disconnect it from usb. Flash the original firmware to the device:

  sudo flasher -F DFL61_HARMATTAN_20.2011.40-4_PR_LEGACY_001-OEM1-958_ARM.bin -F DFL61_HARMATTAN_20.2011.40-4.CENTRALEUROPE_EMMC_CENTRALEUROPE.bin -f

Do NOT disconnect usb cable. Do NOT restart the phone. Now, flash the pached kernel onto it:

  sudo flasher -f -a DFL61_HARMATTAN_20.2011.40-4_PR_LEGACY_001-OEM1-958_ARM.bin -k zImage-2.6.32.39-dfl61-20113701

Now, your N9 is ready. Disconnect it from usb. It will start booting.

3.1.b.2. Developer mode and required libraries

Enable developer mode: Settings / Security / Developer mode. After installation the device will do a reboot.

Change usb connection mode: Settings / Accessories / USB to "SDK" or "Always ask".

Set up wifi internet connection. Connect usb cable.

On N9 menu, select SDK Connectivity / USB. It will show your password for user developer. Leave it open.

Install required libs for NFS mount and add static password for 'user' user (run@host):

ssh developer@192.168.2.15

echo 'rootme' | devel-su
cd /root
apt-get install wget
wget http://maemo.cloud-7.de/HARM/N9/openmode_kernel_PR1.1/opensh/opensh.deb
wget http://repo.pub.meego.com/home:/rzr:/harmattan/MeeGo_1.2_Harmattan_Maemo.org_MeeGo_1.2_Harmattan_standard/armel/libwrap0_7.6.dbs-maemo6.1_armel.deb
wget http://repo.pub.meego.com/home:/rzr:/debian/harmattan/armel/portmap_6.0.0-2_armel.deb

dpkg -i opensh.deb
dpkg -i libwrap0_7.6.dbs-maemo6.1_armel.deb
dpkg -i portmap_6.0.0-2_armel.deb

passwd user

exit
exit

Create public key for ssh connection, if you don't have it already (run@host):

ssh-keygen -t rsa

Copy public key to N9 (run@host):

ssh user@device "mkdir .ssh"
scp ~/.ssh/id_rsa.pub user@device:./.ssh/authorized_keys

Now, you don't have to enter password for 'user' when you connect to N9.

3.1.b.3. Setup NFS on host computer (Ubuntu):

Install the required packages and share swork (run@host):

sudo apt-get install nfs-kernel-server portmap
sudo echo "/home/$USER/swork 192.168.2.15(rw,sync,no_root_squash)" >> /etc/exports 
sudo /etc/init.d/nfs-kernel-server restart

3.1.b.4. Setup NFS on N9

Create script for mounting NFS (run@host):

ssh user@192.168.2.15 "mkdir /home/user/swork; \
echo 'opensh -c \"/sbin/portmap; /sbin/modprobe nfs; mount -t nfs 192.168.2.14:/home/$USER/swork /home/user/swork\"' > nfsmount.sh; \
chmod +x nfsmount.sh "

Connect to N9 (run@host)

ssh user@192.168.2.15

Mount NFS if you didn't do it (run@n9).

opensh
. nfsmount.sh

3.2. Running MiniBrowser

3.2.1. Environment variables

Now create script for setting environment variables (run@host):

ssh user@192.168.2.15 "echo 'export QTDIR=/home/user/swork/qt5/qtbase' > n9env.sh; \
echo 'export QT_IMPORT_PATH=/home/user/swork/qt5/qtbase/imports' >> n9env.sh; \
echo 'export QML_IMPORT_PATH=/home/user/swork/qt5/qtbase/imports' >> n9env.sh; \
echo 'export LD_LIBRARY_PATH=/home/user/swork/qt5/qtbase/lib:/home/user/swork/qtwebkit-webkit2-dev-builddir-armel-m6/Release/lib' >> n9env.sh; \
echo 'export QT_PLUGIN_PATH=/home/user/swork/qt5/qtbase/plugins' >> n9env.sh; \
echo 'export QT_QPA_PLATFORM_PLUGIN_PATH=/home/user/swork/qt5/qtbase/plugins/platforms' >> n9env.sh; \
echo 'export QT_QPA_PLATFORM=xcb' >> n9env.sh; \
echo 'export WEBKIT_TESTFONTS=/home/user/swork/testfonts' >> n9env.sh; \
echo '# export TZ=/usr/share/zoneinfo/America/Los_Angeles' >> n9env.sh; \
echo 'export PATH=/home/user/swork/qt5/qtbase/bin:$PATH' >> n9env.sh"

Load environment variables (run@n9):

. n9env.sh

3.2.2. Launch browser

(run@n9)

~/swork/qtwebkit-webkit2-dev-builddir-armel-m6/Release/bin/MiniBrowser

4. Flash back N9 to factory state

If you want to restore the original factory state of the device or flashing went wrong you may need to flash the original firmware to the N9.

Flashing

• If your device cannot boot up at all, you need to use rd-mode switch steps.
• If you re-flash your device just for clean-up, only flashing (Step 2) is required.
• If you rewrite device with EMMC binary, than all your data will be lost.

Youtube example:

• ​http://www.youtube.com/watch?v=Vs2wyllNTCk&hd=1

Step 1 Disconnect your N9 device from usb. To turn on rd-mode and disable lifeguard-reset run:

sudo flasher --set-rd-flags=no-lifeguard-reset --enable-rd-mode

Now connect usb cable and wait for finding the device.

Step 2 Disconnect N9 from usb.

sudo flasher -F DFL61_HARMATTAN_20.2011.40-4_PR_LEGACY_001-OEM1-958_ARM.bin -F DFL61_HARMATTAN_20.2011.40-4.CENTRALEUROPE_EMMC_CENTRALEUROPE.bin -f -R

Now connect usb cable and wait for minimum 11% battery level, than flashing will be started.

Step 3 After flashing is complet and N9 device is rebooted disconnect it from usb. Turn off rd-mode and disable lifeguard-reset:

sudo flasher --clear-rd-flags=no-lifeguard-reset --disable-rd-mode

Connect usb cable...

More about R&D mode:

• ​http://wiki.maemo.org/R%26D_mode

Notes

If your device has hardware error (corrupted flash storage), you will get similar results as these:

Battery level 72 %, continuing.

image [state progress transfer flash speed]
---------------------------------------------------------------------
[x] cert-sw [finished 100 % 1 / 1 kB NA ]
[x] cmt-2nd [finished 100 % 95 / 95 kB NA ]
[x] cmt-algo [finished 100 % 789 / 789 kB NA ]
[_] cmt-mcusw [init 0 % 1024 / 6050 kB NA ]
[x] xloader [finished 100 % 23 / 23 kB NA ]
[x] secondary [finished 100 % 93 / 93 kB NA ]
[x] kernel [finished 100 % 2712 / 2712 kB 1271 kB/s]
[_] rootfs [init 0 % 12288 / 1102340 kB NA ]
ERROR: SU_GET_UPDATE_STATUS_REQ terminated with error code 1: Unknown error

image [state progress transfer flash speed]
---------------------------------------------------------------------
[x] cert-sw [finished 100 % 1 / 1 kB NA ]
[x] cmt-2nd [finished 100 % 95 / 95 kB NA ]
[x] cmt-algo [finished 100 % 789 / 789 kB NA ]
[x] cmt-mcusw [finished 100 % 6050 / 6050 kB 3317 kB/s]
[x] xloader [finished 100 % 23 / 23 kB NA ]
[x] secondary [finished 100 % 93 / 93 kB NA ]
[x] kernel [finished 100 % 2712 / 2712 kB 1271 kB/s]
[f] rootfs [init 0 % 13312 / 1102340 kB NA ]
Fetching error list:
========================================
bb5_rdc_cert_read failed
bb5_rdc_cert_read failed
bb5_rdc_cert_read failed
bb5_rdc_cert_read failed
bb5_rdc_cert_read failed
bb5_rdc_cert_read failed
mmc: Could not find device path under /sys/class/mmc_host/mmc0
mmc: Failed to init mmc block device
scconf_parse_entries failed: (null)
Unable to parse layout
mmc: Could not remove mount directory /tmp/sudmmc9J3FtB/mnt: No such file or dir
ectory
mmc: Could not unlink (null): Bad address
[Pipe 3] Finishing in error state with status 1