| 32 | | The inspector-test.js stub creates a real (for WebKit2, separate process) inspector frontend. Instead of the normal Web Inspector base page (`Main.html`), it loads a smaller version (`Test.html`) which does not load Views and other code not used by tests. The code that runs inside the Inspector (i.e., code within the test() method) has access to the frontend test harness, whose methods are prefixed with `InspectorTest`. Like ordinary Web Inspector code, injected inspector code has full access to models and controllers in the `WebInspector` namespace. (However, as noted above, not all files are loaded in the test version of the Inspector. You may need to add additional files to `Test.html` when testing new code or adding inter-class dependencies.) |
| | 32 | The `inspector-test.js` stub creates a real (for WebKit2, separate process) inspector frontend. Instead of the normal Web Inspector base page (`Main.html`), it loads a smaller version (`Test.html`) which does not load Views and other code not used by tests. The code that runs inside the Inspector (i.e., code within the test() method) has access to the frontend test harness, whose methods are prefixed with `InspectorTest`. Like ordinary Web Inspector code, injected inspector code has full access to models and controllers in the `WebInspector` namespace. (However, as noted above, not all files are loaded in the test version of the Inspector. You may need to add additional files to `Test.html` when testing new code or adding inter-class dependencies.) |
| 46 | | TODO |
| | 48 | The properties that we strive for when writing inspector tests are: |
| | 49 | |
| | 50 | * '''consistent''': a test should be consistent between runs, and not sporadically fail or time out. |
| | 51 | * '''robust''': a test should be robust to underlying changes in data structures or other minor changes to the code being exercised. It should not require frequent adjustments. |
| | 52 | * '''high coverage''': to uncover bugs and unaddressed situations, a test should exercise as many normal and exceptional code paths as possible. |
| | 53 | * '''self-documenting''': a test should act as executable documentation for the expected and unexpected use cases or behaviors of the code being exercised. |
| | 54 | |
| | 55 | With these properties in mind, here are a few hints for writing good tests: |
| | 56 | |
| | 57 | * Use good names in the test filename (`inspector/domain/description-of-test.html`), test suite name (`Domain.descriptionOfTest`), and in each test case's name (`TestSomethingInteresting`) and description (`"This test ensures something interesting"`). |
| | 58 | * Use `AsyncTestSuite` (documented below) to avoid common pitfalls involved in testing asynchronous code, such as when testing the result of a command sent to the inspector backend. |
| | 59 | * Use assertions to test invariants, pre-conditions, and post-conditions. Assertions should not need to be changed unless the code under test changes in significant ways. |
| | 60 | * Assertions should always document the expected condition in the message, usually using obligatory language such as "should be", "should contain", "should not", etc. For example, the following shows a good and bad assertion message: |
| | 61 | |
| | 62 | {{{ |
| | 63 | InspectorTest.expectThat(fontFamilyNames.length >= 5, "Has at least 5 fonts"); // BAD! |
| | 64 | InspectorTest.expectThat(fontFamilyNames.length >= 5, "Family names list should contain at least 5 fonts"); // GOOD! |
| | 65 | }}} |
| | 66 | |
| | 67 | * Use `expectThat` instead of `assert` whenever possible. The former will always add output to the test, prefixing the condition with `PASS:` or `FAIL:`; the latter only produces output when the condition evaluates to false. Why should the default be to always log? For someone trying to understand how a test works or debug a failing test, the extra output is very helpful in understanding control flow. |
| | 68 | * Log runtime states sparingly, and only those that may help to diagnose a failing test. Logging runtime states can make tests more self-documenting at the expense of reducing robustness. For example, if a test logs source code locations, these could change (and cause the test to fail) if text is added to or removed from relevant file. It is better to assert actual output against known-good outputs. Don't dump runtime state that is machine-dependent. |
| | 69 | |
| | 70 | == Important Test Fixtures |
| | 71 | |
| | 72 | Common to both protocol tests and frontend tests are the `TestHarness` and `TestSuite` classes. TestHarness and its subclasses (bound to the globals ProtocolTest or InspectorTest) provide basic mechanisms for logging, asserting, and starting or stopping the test. Protocol and frontend tests each have their own subclass of `TestHarness` which contains methods specific to one environment. |
| | 73 | |
| | 74 | `TestSuite` and its subclasses `AsyncTestSuite` and `SyncTestSuite` help us to write robust, well-documented, and fast tests. All new tests should use these classes. Each test file consists of one (or more) test suite(s). A suite consists of multiple test cases which execute sequentially in the order that they are added. If a test case fails, later test cases are skipped to avoid spurious failures caused by dependencies between test cases. Test cases are added to the suite imperatively, and then executed using the `runTestCases()` or `runTestCasesAndFinish()` methods. This allows for programmatic construction of test suites that exercise code using many different inputs. |
| | 75 | |
| | 76 | A `SyncTestSuite` executes its test cases synchronously, one after another in a loop. It is usually used for unit tests that do not require communication with the backend. Each test case provides a test method which takes no arguments and returns `true` or `false` to indicate test success or failure, respectively. |
| | 77 | |
| | 78 | An `AsyncTestSuite` executes its test cases asynchronously, one after another, by chaining together promises created for each test. Each test case provides a test method which takes two callback arguments: `resolve` and `reject`. At runtime, each test method is turned into a Promise; like a Promise, the test signals success by calling `resolve()`, and signals failure by calling `reject()` or throwing an `Error` instance. |
| | 126 | |
| | 127 | = Example Test (uses inspector-test.js, AsyncTestSuite) |
| | 128 | |
| | 129 | {{{ |
| | 130 | <!DOCTYPE html> |
| | 131 | <html> |
| | 132 | <head> |
| | 133 | <script src="../../http/tests/inspector/resources/inspector-test.js"></script> |
| | 134 | <script> |
| | 135 | function test() |
| | 136 | { |
| | 137 | let addedStyleSheet; |
| | 138 | let mainFrame = WebInspector.frameResourceManager.mainFrame; |
| | 139 | |
| | 140 | let suite = InspectorTest.createAsyncSuite("CSS.createStyleSheet"); |
| | 141 | |
| | 142 | suite.addTestCase({ |
| | 143 | name: "CheckNoStyleSheets", |
| | 144 | description: "Ensure there are no stylesheets.", |
| | 145 | test: (resolve, reject) => { |
| | 146 | InspectorTest.expectThat(WebInspector.cssStyleManager.styleSheets.length === 0, "Should be no stylesheets."); |
| | 147 | resolve(); |
| | 148 | } |
| | 149 | }); |
| | 150 | |
| | 151 | for (let i = 1; i <= 3; ++i) { |
| | 152 | suite.addTestCase({ |
| | 153 | name: "CreateInspectorStyleSheetCall" + i, |
| | 154 | description: "Should create a new inspector stylesheet.", |
| | 155 | test: (resolve, reject) => { |
| | 156 | CSSAgent.createStyleSheet(mainFrame.id); |
| | 157 | WebInspector.cssStyleManager.singleFireEventListener(WebInspector.CSSStyleManager.Event.StyleSheetAdded, function(event) { |
| | 158 | InspectorTest.expectThat(WebInspector.cssStyleManager.styleSheets.length === i, "Should increase the list of stylesheets."); |
| | 159 | InspectorTest.expectThat(event.data.styleSheet.origin === WebInspector.CSSStyleSheet.Type.Inspector, "Added StyleSheet origin should be 'inspector'."); |
| | 160 | InspectorTest.expectThat(event.data.styleSheet.isInspectorStyleSheet(), "StyleSheet.isInspectorStyleSheet() should be true."); |
| | 161 | InspectorTest.expectThat(event.data.styleSheet.parentFrame === mainFrame, "Added StyleSheet frame should be the main frame."); |
| | 162 | if (addedStyleSheet) |
| | 163 | InspectorTest.expectThat(event.data.styleSheet !== addedStyleSheet, "Added StyleSheet should be different from the last added stylesheet."); |
| | 164 | addedStyleSheet = event.data.styleSheet; |
| | 165 | resolve(); |
| | 166 | }); |
| | 167 | } |
| | 168 | }); |
| | 169 | } |
| | 170 | |
| | 171 | WebInspector.cssStyleManager.singleFireEventListener(WebInspector.CSSStyleManager.Event.StyleSheetRemoved, function(event) { |
| | 172 | InspectorTest.assert(false, "Should not be removing any StyleSheets in this test."); |
| | 173 | }); |
| | 174 | |
| | 175 | suite.runTestCasesAndFinish(); |
| | 176 | } |
| | 177 | </script> |
| | 178 | </head> |
| | 179 | <body onload="runTest()"> |
| | 180 | <p>Test CSS.createStyleSheet.</p> |
| | 181 | </body> |
| | 182 | </html> |
| | 183 | }}} |
