Running Tests for WebKitGTK

Dependencies

Ensure that you've followed all dependency installation instructions. If you try to run tests without installing and updating both system and JHBuild (update-webkitgtk-libs) dependencies, it's likely that not all tests will pass.

Layout Tests (Functional Tests)

Running the Tests

Simply run the run-webkit-tests script from the Tools directory. run-webkit-tests also supports the --debug argument to run the tests with the debug build of WebKitGTK.

$ ./Tools/Scripts/run-webkit-tests --gtk [--debug]

If you wish to run a single test you can do:

$ ./Tools/Scripts/run-webkit-tests --gtk fast/forms/plaintext-mode-1.html

Pixel Tests

WebKitGTK supports running pixel tests, but they are not enabled by default. To run them simply pass the -p argument to run-webkit-tests:

$ ./Tools/Scripts/run-webkit-tests --gtk -p

Running Tests in GDB

For debugging a test it may be useful to run it directly in WebKitTestRunner (with gdb):

$ TEST_RUNNER_TEST_PLUGIN_PATH= TEST_RUNNER_INJECTED_BUNDLE_FILENAME=WebKitBuild/Debug/lib/libTestRunnerInjectedBundle.so WebKitBuild/Debug/bin/WebKitTestRunner -v LayoutTests/fast/forms/plaintext-mode-1.html

Debugging WebProcess in a test within flatpak:

TEST_RUNNER_TEST_PLUGIN_PATH= WEB_PROCESS_CMD_PREFIX='/usr/bin/gdbserver localhost:8080' webkit-flatpak --debug --gtk -c /app/webkit/WebKitBuild/Debug/bin/WebKitTestRunner "-v" "LayoutTests/fast/forms/plaintext-mode-1.html"

And, in another terminal, execute

$ Tools/Scripts/webkit-flatpak --debug --command=gdb /app/webkit/WebKitBuild/Debug/bin/WebKitWebProcess
Reading symbols from bin/WebKitWebProcess...done.
(gdb) target remote localhost:8080
(gdb) continue

HTTP tests

HTTP functional tests are stored at:

$ LayoutTests/http/tests/

Obviously, it is needed a local web server for running them. Depending on your distribution, when executing the tests, the run-webkit-tests script will launch the needed local web server. The configurations used for the web server can be found at:

$ LayoutTests/http/conf/

Sometimes, it is useful to change that configuration or to be able to just launch the web server and run some tests and checks manually. Instead of running the specific test through the run-webkit-tests script, we can do this calling:

$ ./Tools/Scripts/run-webkit-httpd

Unit Tests

The GTK+ port includes unit tests. To run the suite:

$ ./Tools/Scripts/run-gtk-tests [--debug]

If you wish to run a single test you can do:

$ ./Tools/Scripts/run-gtk-tests WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestWebExtensions

WebDriver Tests

The GTK+ port includes WebDriver tests. To run the suite:

$ python ./Tools/Scripts/run-webdriver-tests --json-output=webdriver_tests.json --release --gtk

If you wish to run a single test you can do:

python ./Tools/Scripts/run-webdriver-tests --json-output=webdriver_tests.json --release --gtk \
   imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py

WebDriver test expectations are stored at:

$ WedDriverTests/TestExpectations.json

Test expectations file format:

"imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py": {
   "subtests": {
      "testShouldAllowUsersToAcceptAnAlertInAFrame": {
         "expected": {"all": {"status": ["FAIL"], "bug": "webkit.org/b/181727"}}
      }
   }
}

Common Problems

chroot derived problems

There are some instructions to avoid common problems derived from having a chroot based working environment.

Propietary graphics drivers (NVIDIA/AMD) problems

For the tests we use Xvfb + LLVMPipe (software GL in a short), so we expect that libGL is provided by Mesa. However when proprietary graphics drivers are in use on the system, libGL is often provided by the driver.

To workaround this problem you can use the environment variable LD_LIBRARY_PATH to point it to where the libGL Mesa libraries are. On a Debian GNU/Linux system this can be achieved as follows:

LD_LIBRARY_PATH="/usr/lib/mesa-diverted/$(gcc -print-multiarch)" ./Tools/Scripts/run-webkit-tests --gtk [--debug]

More information related to the same issue ​can be found here

Further information

Further information about the functional and unit tests can be found in the genera documentation pages at the main page of the wiki.