Changes between Version 12 and Version 13 of TestExpectations
- Timestamp:
- Jun 3, 2012, 9:19:23 PM (13 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
TestExpectations
v12 v13 25 25 In all ports except for the Chromium ones, the convention is to check in the incorrect output as a platform-specific file, and then file a bug to track the incorrectness. For some tests, on some ports, the test is *never* expected to pass, in which case the test is added to the {{{Skipped}} files instead. We will also add tests to the Skipped files if it would affect the rest of the test run or cause NRWT itself to crash. 26 26 27 In the Chromium ports, the convention is to add a line of text to the test_expectations.txtfile (see below).27 In the Chromium ports, the convention is to add a line of text to the TestExpectations file (see below). 28 28 29 29 Lastly, we also support the concept of "reference tests", which check that two pages are rendered identically (pixel-by-pixel). As long as the two tests' output match, the tests pass. For more on reference tests, see [wiki:RefTests]. 30 30 31 == Suppressing failures using NRWT: the test_expectations.txtfile ==31 == Suppressing failures using NRWT: the TestExpectations file == 32 32 33 33 The test expectations file is found in a platform-specific directory under LayoutTests. I will use the Chromium version as an example. … … 69 69 3. BUGDPRANKE is a "placeholder" that indicates that no bug has been filed yet, but you should bug that individual about the status. 70 70 71 Configuration parameters describe which variations of your port the test expectation should apply to. Typically each test_expectations.txtfile is used for multiple variations of a test run, because each variation usually has a lot of failures in common, and it's easier to manage all of the failures in one place. The exact set of configuration parameters will vary from port to port. Here are the options supported for Chromium:71 Configuration parameters describe which variations of your port the test expectation should apply to. Typically each TestExpectations file is used for multiple variations of a test run, because each variation usually has a lot of failures in common, and it's easier to manage all of the failures in one place. The exact set of configuration parameters will vary from port to port. Here are the options supported for Chromium: 72 72 73 73 * LEOPARD SNOWLEOPARD XP VISTA WIN7 LUCID : these indicate particular versions of particular operating systems. Multiple options may appear on a single line, but duplicates are not allowed … … 136 136 137 137 ORWT has a much simpler mechanism. Tests are either expected to pass, or can be skipped. To skip a test, list it in the Skipped file for your platform. If your test produces output different from the "expected" version, check in the new (possibly incorrect) version in your platform-specific directory. See [wiki:LayoutTestSearchPath] for figuring out where that directory is. 138