wiki:WebInspectorTests

Version 2 (modified by Brian Burg, 10 years ago) (diff)

add notes about debugging tests; add notes about model tests

This page describes how various parts of the Web Inspector are tested.


Types of Tests

There are three flavors of inspector tests: tests that exercise the raw inspector protocol (independently of any particular frontend), tests that exercise the models and controllers underlying the user interface, and manual tests for the user interface. In some cases, model/controller tests are written instead of protocol tests if the protocol test would be unusually large or hard to debug.

There are no automated tests that exercise the user interface of the Web Inspector (UI tests). In practice, the Inspector UI changes frequently so such tests are very brittle, and are rarely worth the trouble of maintaining. Some manual tests focusing on UI exist in ManualTests/inspector/, but these are not maintained at the time of this writing.

Protocol Tests

Protocol tests are best used for features that require relatively few messages between the backend and frontend. They are more general---independent of any particular frontend---but it can be painful to write correct tests that work well with the asynchronous nature of the protocol.

Each test is an HTML file in the LayoutTests/inspector-protocol/ directory. The special test() method is evaluated in the inspector page (see below), and other scripts are executed in the test page (i.e., the page being inspected by the inspector).

Dummy inspector frontend

Many protocol and model tests do not need to persist across main frame navigations, nor do they need access all of the Web Inspector functionality. For such tests, the test harness creates a dummy inspector frontend by using window.open() from the test page, and establishes bidirectional communication with the child inspector page using window.postMessage and a message event handler. It then evaluates a subset of inspector resources into existence on the dummy page. Finally, it runs the provided test function by calling toString() on the function, sending to the child inspector page, and then using eval(). This functionality is implemented by LayoutTests/http/tests/inspector-protocol/resources/protocol-test.js on the test page, and LayoutTests/http/tests/inspector-protocol/resources/InspectorTest.js on the inspector page.

Model/Controller Tests

Model tests exercise the functionality of models and controllers specific to WebInspectorUI (the user interface included in WebKit trunk.)

Each test is an HTML file in the LayoutTests/inspector/ directory. The special test() method is evaluated in the inspector page (see below), and other scripts are executed in the test page (i.e., the page being inspected by the inspector).

Headless inspector frontend

Model tests are more heavyweight than protocol tests because they load a real instance of the web inspector into a separate WebView. This allows tests to persist across multiple page loads, but requires significantly more plumbing and setup time for each test. A summary of what happens is as follows:

  1. The test page is loaded by the test runner. Each test page must include inspector-test.js, which sets up the connection to the inspector page and provides other helpers. Its functions are in the namespace InspectorTestProxy, to remind you that it proxies calls from the test page to the inspector page.
  2. When the test page finishes loading, runTest() on the test page starts up the Inspector. The file Test.js initializes the inspecotr environment. Then, the frontend marshalls helper functions to the inspector page as well as the special test() method to be run.
  3. When the inspector finishes loading, it runs the marshalled test() in the inspector page.
  4. As the test runs, results are marshalled back to the test page by injecting and eval'ing JavaScript code.

Debugging Tests

Sometimes it can be hard to debug inspector tests (particularly, model tests) because the mechanisms for marshalling test results to the test page are fragile and can silence parse or other errors. We provide a flag that causes the test harness to synchronously log messages and errors to STDOUT rather than the test page. If your test mysteriously times out, set the flag InspectorTest.dumpMessagesToConsole (in Test.js) to true to enable this logging.