Changes between Version 5 and Version 6 of NewRunWebKitTests


Ignore:
Timestamp:
Jul 7, 2011 3:11:44 PM (9 years ago)
Author:
dpranke@chromium.org
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • NewRunWebKitTests

    v5 v6  
    4242}}}
    4343
    44 and when the test completes, you'll get output like:
     44and when the test completes, you'll get output something like:
    4545
    4646{{{
    4747% new-run-webkit-tests
     48  animations/animation-direction-normal.html -> unexpected text diff mismatch
     49  compositing/reflections/nested-reflection-animated.html -> unexpected text diff mismatch
     50                                                         
     51Retrying 2 unexpected failure(s) ...
     52
     53  animations/animation-direction-normal.html -> unexpected text diff mismatch
     54  compositing/reflections/nested-reflection-animated.html -> unexpected text diff mismatch
     55                                                         
     5624894 tests ran as expected, 2 didn't:
     57
     58Regressions: Unexpected text diff mismatch : (10)
     59  animations/animation-direction-normal.html = TEXT
     60  compositing/reflections/nested-reflection-animated.html = TEXT
     61
     62% echo $?
     632
     64%
    4865}}}
    4966
     67In addition, by default, a web browser will be opened to an HTML page displaying the tests that failed, and links to the output from the tests. You can notice a few things about this test output:
     68
     69 1. Tests that fail are automatically retried, to see if the failure was "flaky" and only happens some of the time. A flaky failure (one that fails then passing during the retry is reported on screen, but does not count as an actual failure.
     70 2. The retries are all done serially in a single thread, to avoid any load-related or ordering-related issues that might be contributing to the flakiness.
     71
    5072=== More Advanced Scenarios ===