Changes between Version 12 and Version 13 of TestExpectations


Ignore:
Timestamp:
Jun 3, 2012 9:19:23 PM (12 years ago)
Author:
rniwa@webkit.org
Comment:

Update per TestExpectations rename

Legend:

Unmodified
Added
Removed
Modified
  • TestExpectations

    v12 v13  
    2525In all ports except for the Chromium ones, the convention is to check in the incorrect output as a platform-specific file, and then file a bug to track the incorrectness. For some tests, on some ports, the test is *never* expected to pass, in which case the test is added to the {{{Skipped}} files instead. We will also add tests to the Skipped files if it would affect the rest of the test run or cause NRWT itself to crash.
    2626
    27 In the Chromium ports, the convention is to add a line of text to the test_expectations.txt file (see below).
     27In the Chromium ports, the convention is to add a line of text to the TestExpectations file (see below).
    2828
    2929Lastly, we also support the concept of "reference tests", which check that two pages are rendered identically (pixel-by-pixel). As long as the two tests' output match, the tests pass. For more on reference tests, see [wiki:RefTests].
    3030
    31 == Suppressing failures using NRWT: the test_expectations.txt file ==
     31== Suppressing failures using NRWT: the TestExpectations file ==
    3232
    3333The test expectations file is found in a platform-specific directory under LayoutTests. I will use the Chromium version as an example.
     
    6969   3. BUGDPRANKE is a "placeholder" that indicates that no bug has been filed yet, but you should bug that individual about the status.
    7070
    71 Configuration parameters describe which variations of your port the test expectation should apply to. Typically each test_expectations.txt file is used for multiple variations of a test run, because each variation usually has a lot of failures in common, and it's easier to manage all of the failures in one place. The exact set of configuration parameters will vary from port to port. Here are the options supported for Chromium:
     71Configuration parameters describe which variations of your port the test expectation should apply to. Typically each TestExpectations file is used for multiple variations of a test run, because each variation usually has a lot of failures in common, and it's easier to manage all of the failures in one place. The exact set of configuration parameters will vary from port to port. Here are the options supported for Chromium:
    7272
    7373 * LEOPARD SNOWLEOPARD XP VISTA WIN7 LUCID : these indicate particular versions of particular operating systems. Multiple options may appear on a single line, but duplicates are not allowed
     
    136136
    137137ORWT has a much simpler mechanism. Tests are either expected to pass, or can be skipped. To skip a test, list it in the Skipped file for your platform. If your test produces output different from the "expected" version, check in the new (possibly incorrect) version in your platform-specific directory. See [wiki:LayoutTestSearchPath] for figuring out where that directory is.
    138