Changes between Version 9 and Version 10 of TestExpectations

Jul 12, 2011 6:31:47 PM (9 years ago)



  • TestExpectations

    v9 v10  
    6666   1. BUGWK12345 indicates a bug in the WebKit bug database, and is equivalent to
    6767   2. BUGCR12345 indicates a bug in the Chromium bug database, and is equivalent to .
     68   3. BUGV8_12345 indicates a bug in the V8 bug database, .
    6869   3. BUGDPRANKE is a "placeholder" that indicates that no bug has been filed yet, but you should bug that individual about the status.
    121122which will cycle through all of the possible combinations of configurations looking for errors and conflicts. It's not instantaneous, but shouldn't take more than a minute or two.
     124== Rules of Thumb for Suppressing Failures ==
     126Here are some rules-of-thumb that you could apply when adding new expectations to the file:
     128  * Only use WONTFIX when you know for sure we will never, ever implement the capability, tested by the test
     129  * Use SKIP when the test:
     130     * throws JavaScript exception and makes text-only test manifest as pixel-test. This usually manifests in "Missing test expectations" failure.
     131     * disrupts running of the other tests. Although this is not typical, it may still be possible. Please make sure to give Pri-1 to the associated bug.
     132  * Try to specify platforms and configs as accurately as possible. If a test passes on all but on platform, it should only have that platform listed
     133  * If a test fails intermittently, use multiple expectations.
    123135== Suppressing failures using ORWT: Skipped files and checked-in failures ==