Running Tests for WebKitGTK+

Dependencies

Ensure that you've followed all dependency installation instructions. If you try to run tests without installing and updating both system and JHBuild (update-webkitgtk-libs) dependencies, it's likely that not all tests will pass.

Running the Tests

Simply run the run-webkit-tests script from the Tools directory. run-webkit-tests also supports the --debug argument to run the tests with the debug build of WebKitGTK+.

./Tools/Scripts/run-webkit-tests --gtk [--debug]

If you wish to run a single test you can do:

./Tools/Scripts/run-webkit-tests --gtk fast/forms/plaintext-mode-1.html

Unit Tests

The GTK+ port includes unit tests. To run the suite:

./Tools/Scripts/run-gtk-tests [--debug] [--verbose]

If you wish to run a single test you can do:

./Tools/Scripts/run-gtk-tests WebKitBuild/Release/Programs/WebKit2APITests/TestContextMenu

Pixel Tests

WebKitGTK+ supports running pixel tests, but they are not enabled by default. To run them simply pass the -p argument to run-webkit-tests:

./Tools/Scripts/run-webkit-tests --gtk -p

Running Tests in GDB

For debugging a test it may be useful to run it directly in DumpRenderTree (with gdb):

WebKitBuild/Debug/Programs/DumpRenderTree -v LayoutTests/fast/forms/plaintext-mode-1.html

Common Problems

Running python based test with multiprocess support

I'm running my development environment in a schroot with Ubuntu in an Ubuntu host.

It hapenned to me the following error when trying to run the tests

$ ./Tools/Scripts/run-webkit-tests --gtk --debug
Using port 'gtk'
Test configuration: <, x86, debug>
Placing test results in /opt/webkit/WebKit.git/WebKitBuild/Debug/layout-test-results
Baseline search path: gtk -> generic
Using Debug build
Pixel tests disabled
Regular timeout: 12000, slow test timeout: 60000
Command line: /opt/webkit/WebKit.git/WebKitBuild/Debug/Programs/DumpRenderTree -

Found 31495 tests; running 28693, skipping 2802.
Running 4 DumpRenderTrees in parallel over 727 shards (1 locked).

Releasing server lock ...
OSError raised: [Errno 38] Function not implemented
    File "/opt/webkit/WebKit.git/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 110, in run
      unexpected_result_count = manager.run(args)
    File "/opt/webkit/WebKit.git/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 411, in run
      self._run_tests(self._test_names, result_summary, int(self._options.child_processes))
    File "/opt/webkit/WebKit.git/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 464, in _run_tests
      return self._runner.run_tests(test_inputs, self._expectations, result_summary, num_workers, needs_http, needs_websockets, self._retrying)
    File "/opt/webkit/WebKit.git/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 141, in run_tests
      with message_pool.get(self, self._worker_factory, num_workers, self._port.worker_startup_delay_secs(), self._port.host) as pool:
    File "/opt/webkit/WebKit.git/Tools/Scripts/webkitpy/common/message_pool.py", line 61, in get
      return _MessagePool(caller, worker_factory, num_workers, worker_startup_delay_secs, host)
    File "/opt/webkit/WebKit.git/Tools/Scripts/webkitpy/common/message_pool.py", line 79, in __init__
      self._messages_to_worker = multiprocessing.Queue()
    File "/usr/lib/python2.7/multiprocessing/__init__.py", line 218, in Queue
      return Queue(maxsize)
    File "/usr/lib/python2.7/multiprocessing/queues.py", line 63, in __init__
      self._rlock = Lock()
    File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 147, in __init__
      SemLock.__init__(self, SEMAPHORE, 1, 1)
    File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__
      sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
Failed to execute Tools/Scripts/new-run-webkit-tests at ./Tools/Scripts/run-webkit-tests line 126.

The reason is explained in this post ​http://www.boris.co/2012/02/server-ubuntu-11.html

Python's _multithreading requires /dev/shm to work. In Ubuntu /dev/shm points to /run/shm by default. Rename (or delete, I doubt it is necessary anymore) existing /dev/shm.

Create /dev/shm directory

$ sudo -i (become root)
$ mkdir /dev/shm

Add to /etc/fstab mounting of /dev/shm command

tmpfs /dev/shm    tmpfs   defaults,noexec,nosuid     0     0

Mount all unmounted filesystems from /etc/fstab.

$ mount -a

You may need to do this into your schroot too.