Changes between Version 6 and Version 7 of TestExpectations

Timestamp:

Jul 7, 2011, 3:58:06 PM (14 years ago)

Author:

dpranke@chromium.org

Comment:

--

TestExpectations

@@ -14 +14 @@
 == How we manage tests that fail ==
 
-The primary function of the LayoutTests is as a *regression test suite*. This means that, while we care about whether a page is being rendered correctly, we care more about whether the page is being rendered the way we expect it to. In other words, we look more for changes in behavior than we do for correctness.
+The primary function of the LayoutTests is as a ''regression test suite''. This means that, while we care about whether a page is being rendered correctly, we care more about whether the page is being rendered the way we expect it to. In other words, we look more for changes in behavior than we do for correctness.
 
 All layout tests have "expected results", which may be one of several forms. The test may produce a text file containing javascript log messages, or a text rendering of the Render Tree. It may also produce a screen capture of the rendered page (if you are running with {{{--pixel-tests}}} enabled) as PNG files as well. For WebAudio tests, we can produce WAV files instead of either text or PNG files. For any of these types of tests, there are files checked into the LayoutTests directory named "-expected.{txt,png,wav}". In many (most?) cases, the output is expected to be generic and match on any webkit port.
@@ -72 +72 @@
    3. BUGDPRANKE is a "placeholder" that indicates that no bug has been filed yet, but you should bug that individual about the status. 
 
-
 Configuration parameters describe which variations of your port the test expectation should apply to. Typically each test_expectations.txt file is used for multiple variations of a test run, because each variation usually has a lot of failures in common, and it's easier to manage all of the failures in one place. The exact set of configuration parameters will vary from port to port. Here are the options supported for Chromium:
 
@@ -118 +117 @@
 Again, *duplicate expectations are not allowed*.
 
+You can verify that any changes you've made to an expectations file are correct by running:
+
+{{{
+% new-run-webkit-tests --lint-test-files
+}}}
+
+which will cycle through all of the possible combinations of configurations looking for errors and conflicts. It's not instantaneous, but shouldn't take more than a minute or two.
+
 == Suppressing failures using ORWT: Skipped files and checked-in failures ==