Associating Information with Tests
There are currently a few ways to associate information about a test with the test itself.
- You can describe which tests are expected to pass or fail for which ports using the TestExpectations file. These files are in LayoutTests/platform/*/TestExpectations, and each line describes one expected test result. If there is no line in TestExpectations for a particular test, is is expected to pass. The syntax of each line is:
webkit.org/b/?????? [ OS ] path/to/test.html [ ExpectedResult ]
One (somewhat) real example is:
webkit.org/b/148119 [ Mavericks Yosemite ] fast/text/trak-optimizeLegibility.html [ Pass Failure ]
which means that on Mavericks and Yosemite, fast/text/trak-optimizeLegibility.html is expected to either pass or fail.
Keep in mind that “Failure” means “text failure.” If the test is a reference test, and you expect it to fail, use the keyword ImageOnlyFailure.
- Another way to associate information is using a non-declarative mechanism: the JavaScript “window.internals” object. The is the preferred method of associating arbitrary semantics with a test. For example, if you want to say that your test should be paginated, you can say the following in a <script> tag in <head>:
window.internals.setPagination(“LeftToRightPaginated", 0);
This object is defined by the Internals.idl file in WebCore/testing, and is implemented by Internals.cpp. Note that these files are not compiled into WebCore itself, but instead have their own target called WebCoreTestSupport, which is depended on by our testing infrastructure (production builds don’t include it). In the Internals class, you have access to the current Document by calling contextDocument().
- A third way is useful for declaring information which must be available at web process launch (for example: private browsing mode). You can put text in the first line of the test which looks like
webkit-test-runner [ key=value otherkey=othervalue ]
This content is often put into a comment so it doesn’t affect the semantics of the test itself:
<!DOCTYPE html><!-- webkit-test-runner [ key=value otherkey=othervalue ] -->
These first lines are read by WebKitTestRunner just before the test starts running, in updateViewOptionsFromTestHeader(). WebKitTestRunner can react to the things it reads in any arbitrary way, including storing information in ContextConfigurationOptions so that it will tear down the web process pool and build up a new one if this information changes from test to test. Currently, only WebKitTestRunner reads this information, though there is no reason why the run-webkit-tests Python script couldn’t.
