Changes between Version 9 and Version 10 of TestExpectations

Timestamp:

Jul 12, 2011, 6:31:47 PM (13 years ago)

Author:

dpranke@chromium.org

Comment:

--

TestExpectations

@@ -66 +66 @@
    1. BUGWK12345 indicates a bug in the WebKit bug database, and is equivalent to https://bugs.webkit.org/show_bug.cgi?id=12345
    2. BUGCR12345 indicates a bug in the Chromium bug database, and is equivalent to https://bugs.chromium.org/12345 .
+   3. BUGV8_12345 indicates a bug in the V8 bug database, http://code.google.com/p/v8/issues .
    3. BUGDPRANKE is a "placeholder" that indicates that no bug has been filed yet, but you should bug that individual about the status. 
 
@@ -121 +122 @@
 which will cycle through all of the possible combinations of configurations looking for errors and conflicts. It's not instantaneous, but shouldn't take more than a minute or two.
 
+== Rules of Thumb for Suppressing Failures ==
+
+Here are some rules-of-thumb that you could apply when adding new expectations to the file:
+
+  * Only use WONTFIX when you know for sure we will never, ever implement the capability, tested by the test
+  * Use SKIP when the test:
+     * throws JavaScript exception and makes text-only test manifest as pixel-test. This usually manifests in "Missing test expectations" failure.
+     * disrupts running of the other tests. Although this is not typical, it may still be possible. Please make sure to give Pri-1 to the associated bug.
+  * Try to specify platforms and configs as accurately as possible. If a test passes on all but on platform, it should only have that platform listed
+  * If a test fails intermittently, use multiple expectations.
+
 == Suppressing failures using ORWT: Skipped files and checked-in failures ==