Changes between Version 8 and Version 9 of TestExpectations


Ignore:
Timestamp:
Jul 8, 2011, 6:51:39 AM (14 years ago)
Author:
Adam Roben
Comment:

A few small formatting tweaks, and replaced the Contents section with a page outline

Legend:

Unmodified
Added
Removed
Modified

  • TestExpectations

    v8 v9  
    11= Managing Test Expectations with {{{new-run-webkit-tests}}} and {{{old-run-webkit-tests}}} =
    22
    3 == Contents ==
    4 
    5  * How we manage tests that fail
    6  * Suppressing failures using NRWT: the test_expectations.txt file
    7  * Suppressing failures using ORWT: Skipped files and checked-in files.
     3[[PageOutline]]
    84
    95== See also ==
    106
    11  * [wiki:NewRunWebKitTests]
    12  * Triaging test failures
     7 * NewRunWebKitTests
     8 * TriagingTestFailures
    139
    1410== How we manage tests that fail ==
    … …  
    1612The primary function of the LayoutTests is as a ''regression test suite''. This means that, while we care about whether a page is being rendered correctly, we care more about whether the page is being rendered the way we expect it to. In other words, we look more for changes in behavior than we do for correctness.
    1713
    18 All layout tests have "expected results", which may be one of several forms. The test may produce a text file containing javascript log messages, or a text rendering of the Render Tree. It may also produce a screen capture of the rendered page (if you are running with {{{--pixel-tests}}} enabled) as PNG files as well. For WebAudio tests, we can produce WAV files instead of either text or PNG files. For any of these types of tests, there are files checked into the LayoutTests directory named "-expected.{txt,png,wav}". In many (most?) cases, the output is expected to be generic and match on any webkit port.
     14All layout tests have "expected results", which may be one of several forms. The test may produce a text file containing javascript log messages, or a text rendering of the Render Tree. It may also produce a screen capture of the rendered page as PNG files as well (if you are running with {{{--pixel-tests}}} enabled). For WebAudio tests, we can produce WAV files instead of either text or PNG files. For any of these types of tests, there are files checked into the LayoutTests directory named "-expected.{txt,png,wav}". In many (most?) cases, the output is expected to be generic and match on any webkit port.
    1915
    2016When the output doesn't match, there are two potential reasons for it: