wiki:WebInspectorTests

Version 4 (modified by BJ Burg, 7 years ago) (diff)

Start cleaning up article about testing Web Inspector.

This page describes how various parts of the Web Inspector are tested.


Types of Tests

There are several types of inspector tests:

  • Protocol Tests exercise the inspector backend independently of any particular frontend.
  • Frontend Tests exercise the models and controllers underlying the Web Inspector user interface.
  • Manual Tests exercise the user interface in ways that are difficult to automate or require infrastructure that doesn't exist yet.
  • Library Tests exercise subsystems such as pretty printing or the protocol generator in isolation from a running Web Inspector instance.

To date, the Web Inspector has no automated tests that exercise the user interface. In practice, the Inspector UI changes frequently, so such tests tend to be brittle, and have traditionally not been worth the trouble of maintaining.

How Tests Execute

Each test is an HTML file in a per-domain directory within LayoutTests/inspector/. Some tests may additionally include external files, which are included in special resources/ directories that are automatically excluded from the test search path. All tests must decide which test harness to use by including either protocol-test.js or inspector-test.js.

When the test page finishes loading, it calls the runTest() method provided, which signals the test harness to set up a test inspector instance that inspects the test page. Each test page defines a special test() method, which is automatically marshalled and injected into the Inspector instance's context. Most scripts execute in the inspector's JavaScript context, and occasionally evaluate some code in the test page's context to trigger specific inspectable behaviors.

How to Write Tests

TODO

Protocol Tests

Protocol tests are appropriate for testing inspector features that require the use of a few commands and events between the backend and frontend, and do not require the inspected page to be reloaded. Protocol tests are fairly low-level, and exercise the Inspector backend independent of a particular frontend and its controllers and models. In other words, you cannot test Managers or other classes in the WebInspector namespace using a protocol test.

The protocol-test.js stub creates a dummy inspector frontend by using window.open() from the test page, and establishes bidirectional communication with the child inspector page using window.postMessage and a message event handler. The "inspector" page that is loaded into the iframe is located at Source/WebInspectorUI/Base/TestStub.html. The code that runs inside the Inspector frame (i.e., code within the test() method) has access to the protocol test harness, whose methods are prefixed with ProtocolTest. Protocol-specific methods for sending commands and awaiting on events are available in the InspectorProtocol namespace.

Model/Controller Tests

FIXME: re-edit from below.

Model tests exercise the functionality of models and controllers specific to WebInspectorUI (the user interface included in WebKit trunk.)

Each test is an HTML file in the LayoutTests/inspector/ directory. The special test() method is evaluated in the inspector page (see below), and other scripts are executed in the test page (i.e., the page being inspected by the inspector).

Headless inspector frontend

Model tests are more heavyweight than protocol tests because they load a real instance of the web inspector into a separate WebView. This allows tests to persist across multiple page loads, but requires significantly more plumbing and setup time for each test. A summary of what happens is as follows:

  1. The test page is loaded by the test runner. Each test page must include inspector-test.js, which sets up the connection to the inspector page and provides other helpers. Its functions are in the namespace InspectorTestProxy, to remind you that it proxies calls from the test page to the inspector page.
  2. When the test page finishes loading, runTest() executes on the test page and asks the test runner to start up the Inspector. The file Test.js is loaded by the Inspector and initializes its environment. Functions that execute in the Inspector use the InspectorTest namespace.
  3. When the inspector finishes loading and receives all marshalled code from the test page, it runs the test() method in the inspector page.
  4. As the test runs, results are marshalled back to the test page by injecting and eval'ing JavaScript code.

Debugging Tests

Sometimes it can be hard to debug inspector tests (particularly, model tests) because the mechanisms for marshalling test results to the test page are fragile and can silence parse or other errors. We provide a flag that causes the test harness to synchronously log messages and errors to STDOUT rather than the test page. If your test mysteriously times out, set the flag InspectorTest.dumpMessagesToConsole (in Test.js) to true to enable this logging.