Changes between Version 4 and Version 5 of WebInspectorTests


Ignore:
Timestamp:
Aug 28, 2015 4:09:26 PM (9 years ago)
Author:
BJ Burg
Comment:

More updates to Web Inspector test documentation.

Legend:

Unmodified
Added
Removed
Modified
  • WebInspectorTests

    v4 v5  
    1818 Each test is an HTML file in a per-domain directory within `LayoutTests/inspector/`. Some tests may additionally include external files, which are included in special `resources/` directories that are automatically excluded from the test search path. All tests must decide which test harness to use by including either `protocol-test.js` or `inspector-test.js`.
    1919
    20  When the test page finishes loading, it calls the `runTest()` method provided, which signals the test harness to set up a test inspector instance that inspects the test page. Each test page defines a special `test()` method, which is automatically marshalled and injected into the Inspector instance's context. Most scripts execute in the inspector's JavaScript context, and occasionally evaluate some code in the test page's context to trigger specific inspectable behaviors.
     20 When the test page finishes loading, it calls the `runTest()` method provided, which signals the test harness to set up a test inspector instance that inspects the test page. Each test page defines a special `test()` method, which is automatically marshalled and injected into the Inspector instance's context. Most scripts execute in the inspector's JavaScript context, and occasionally evaluate some code in the test page's context to log test results and to trigger specific inspectable behaviors.
     21
     22== Protocol Tests
     23
     24Protocol tests are appropriate for testing inspector features that require the use of a few commands and events between the backend and frontend, and do not require the inspected page to be reloaded. Protocol tests are fairly low-level, and exercise the Inspector backend independent of a particular frontend and its controllers and models. In other words, you cannot test Managers or other classes in the WebInspector namespace using a protocol test.
     25
     26The `protocol-test.js` stub creates a dummy inspector frontend by using `window.open()` from the test page, and establishes bidirectional communication with the __child__ inspector page using `window.postMessage` and a `message` event handler. The "inspector" page that is loaded into the iframe is located at `Source/WebInspectorUI/Base/TestStub.html`. The code that runs inside the Inspector frame (i.e., code within the test() method) has access to the protocol test harness, whose methods are prefixed with `ProtocolTest`. Protocol-specific methods for sending commands and awaiting on events are available in the InspectorProtocol namespace.
     27
     28== Frontend Tests
     29
     30Frontend tests exercise the functionality of models and controllers specific to WebInspectorUI (the user interface included in WebKit trunk). They use a real, headless Web Inspector frontend that persists across navigations of the inspected (test) page.
     31
     32The inspector-test.js stub creates a real (for WebKit2, separate process) inspector frontend. Instead of the normal Web Inspector base page (`Main.html`), it loads a smaller version (`Test.html`) which does not load Views and other code not used by tests. The code that runs inside the Inspector (i.e., code within the test() method) has access to the frontend test harness, whose methods are prefixed with `InspectorTest`. Like ordinary Web Inspector code, injected inspector code has full access to models and controllers in the `WebInspector` namespace. (However, as noted above, not all files are loaded in the test version of the Inspector. You may need to add additional files to `Test.html` when testing new code or adding inter-class dependencies.)
     33
     34== Manual Tests
     35
     36A manual test requires manual interaction with a test page to exercise specific behaviors. The test page should describe the necessary interaction steps, and the expected output/behavior.
     37
     38== Library Tests
     39
     40TODO
     41
     42-----
    2143
    2244== How to Write Tests
     
    2446TODO
    2547
    26 == Protocol Tests
     48== How to Debug Tests
    2749
    28 Protocol tests are appropriate for testing inspector features that require the use of a few commands and events between the backend and frontend, and do not require the inspected page to be reloaded. Protocol tests are fairly low-level, and exercise the Inspector backend independent of a particular frontend and its controllers and models. In other words, you cannot test Managers or other classes in the WebInspector namespace using a protocol test.
     50In general, the strategies for [wiki:"WebInspectorDebugging" debugging the Web Inspector] and debugging WebCore/WebKit2 apply the same to debugging inspector tests. Sometimes, tests can be more difficult to debug because the test harness' marshalling code can be broken by incorrectly written tests or bugs in the test harness. The test stubs provide several flags that enable extra or more reliable logging for debug purposes. Flags can be set in the corresponding `Test/TestStub.html` file for all test runs, or at the top of a `test()` method to only affect one test.
    2951
    30 The protocol-test.js stub creates a dummy inspector frontend by using `window.open()` from the test page, and establishes bidirectional communication with the __child__ inspector page using `window.postMessage` and a `message` event handler. The "inspector" page that is loaded into the iframe is located at `Source/WebInspectorUI/Base/TestStub.html`. The code that runs inside the Inspector frame (i.e., code within the test() method) has access to the protocol test harness, whose methods are prefixed with `ProtocolTest`. Protocol-specific methods for sending commands and awaiting on events are available in the InspectorProtocol namespace.
     52For protocol tests:
    3153
    32 == Model/Controller Tests
     54{{{
     55// Debug logging is synchronous on the test page.
     56ProtocolTest.forceDebugLogging = false;
    3357
    34 FIXME: re-edit from below.
     58// Tee all TestHarness commands to stderr from within the Inspector.
     59ProtocolTest.dumpActivityToSystemConsole = false;
    3560
    36 Model tests exercise the functionality of models and controllers specific to WebInspectorUI (the user interface included in WebKit trunk.)
     61// Best used in combination with dumpActivityToSystemConsole.
     62ProtocolTest.dumpInspectorProtocolMessages = false;
     63}}}
    3764
    38 Each test is an HTML file in the `LayoutTests/inspector/` directory. The special `test()` method is evaluated in the inspector page (see below), and other scripts are executed in the test page (i.e., the page being inspected by the inspector).
     65For frontend tests:
    3966
    40 === Headless inspector frontend
     67{{{
     68// Debug logging is synchronous on the test page.
     69InspectorTest.forceDebugLogging = false;
    4170
    42 Model tests are more heavyweight than protocol tests because they load a real instance of the web inspector into a separate WebView. This allows tests to persist across multiple page loads, but requires significantly more plumbing and setup time for each test. A summary of what happens is as follows:
     71// Tee all TestHarness commands to stderr from within the Inspector.
     72InspectorTest.dumpActivityToSystemConsole = false;
    4373
    44 1. The test page is loaded by the test runner. Each test page must include `inspector-test.js`, which sets up the connection to the inspector page and provides other helpers. Its functions are in the namespace `InspectorTestProxy`, to remind you that it proxies calls from the test page to the inspector page.
    45 2. When the test page finishes loading, `runTest()` executes on the test page and asks the test runner to start up the Inspector. The file `Test.js` is loaded by the Inspector and initializes its environment. Functions that execute in the Inspector use the `InspectorTest` namespace.
    46 3. When the inspector finishes loading and receives all marshalled code from the test page, it runs the `test()` method __in the inspector page__.
    47 4. As the test runs, results are marshalled back to the test page by injecting and eval'ing JavaScript code.
     74// Best used in combination with dumpActivityToSystemConsole.
     75    InspectorBackend.dumpInspectorProtocolMessages = false;
     76}}}
    4877
    49 == Debugging Tests
     78=== Attaching a Debugger to Tests with DumpRenderTree (WebKit1)
    5079
    51 Sometimes it can be hard to debug inspector tests (particularly, model tests) because the mechanisms for marshalling test results to the test page are fragile and can silence parse or other errors. We provide a flag that causes the test harness to synchronously log messages and errors to STDOUT rather than the test page. If your test mysteriously times out, set the flag `InspectorTest.dumpMessagesToConsole` (in `Test.js`) to true to enable this logging.
     80{{{
     81$ DYLD_FRAMEWORK_PATH=WebKitBuild/Debug lldb -- WebKitBuild/Debug/DumpRenderTree
     82(lldb) run LayoutTests/inspector/dom/focus.html
     83}}}
     84
     85To run DumpRenderTree in "server mode", which the run-webkit-tests uses to run multiple tests without restarting the process, make a file called "tests-to-run.txt" with one test per line, and launch this way instead:
     86
     87{{{
     88(lldb) process launch -i tests-to-run.txt "-"
     89}}}
     90
     91=== Attaching a Debugger to Tests with WebKitTestRunner (WebKit2)
     92
     93TODO