Changes between Version 1 and Version 2 of Rebaseline


Ignore:
Timestamp:
Mar 22, 2010, 3:09:09 PM (15 years ago)
Author:
Dimitri Glazkov
Comment:

Clean up rebaseline doc.

Legend:

Unmodified
Added
Removed
Modified
  • Rebaseline

    v1 v2  
    22
    33The tool is written in Python and lives in `WebKitTools/Scripts` ([http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/rebaseline-chromium-webkit-tests]).
    4 
    5 ''pasted from [http://code.google.com/p/chromium/wiki/RebaseliningTool], needs editing''
    64
    75== How does this tool work? ==
     
    1311  * For each test that has been rebaselined, remove this platform option from the test in _test_expectation.txt_. If no other platform options remaining after removal, delete the test from the file.
    1412
    15 After new baselines are generated, the tool launches a html page to compare the old and new baselines. The html can be disabled by option "--no_html_results".
     13After new baselines are generated, the tool launches a html page to compare the old and new baselines. The html can be disabled with `--no_html_results` option.
    1614
    1715== How to use it? ==
    18   * Make sure the test files (.html) exist at _src\webkit\data\layout_tests_ in repository. If not, add the html files there and commit them. By default, the rebaselining tool retrieves new baselines from the release version of buildbot layout test archive. If you add a new test, you need to wait for buildbot to run layout tests and the actual result for the test has been archived before you can rebaseline it using this tool. Check the *RELEASE* version of [http://build.chromium.org/buildbot/layout_test_results/ buildbot layout archive] to see whether your new test is available.
    19   * Update _test_expectations.txt_ at _src\webkit\tools\layout_tests_: add option *REBASELINE* to the tests that need to be rebaselined.
    20   * Run rebaselining script: *rebaseline.bat* for Windows and *rebaseline.sh* for Linux and Mac. The script can be found at _src\webkit\tools\layout_tests_. If you want to pull new baselines from webkit.org canary, run rebaseline.bat or rebaseline.sh with option *--webkit_canary*.
    21   * If the script runs successfully, new baselines (expected files) are added to SVN repository and the rebaselined tests are removed from _test_expectations.txt_.
     16  * Make sure the test files exist in repository and the build bots have cycled at least once after they were committed. If you are not sure, check the Release version of [http://build.chromium.org/buildbot/layout_test_results/ buildbot layout archive] to see whether your test results are available. By default, the rebaselining tool retrieves new baselines from the release version of buildbot layout test archive.
     17  * Update `test_expectations.txt`: add flag '''REBASELINE''' to the tests that need to be rebaselined.
     18  * Run rebaselining script: `WebKitTools/Scripts/rebaseline-chromium-webkit-tests`. If you want to pull new baselines from the webkit.org canary, use `--webkit_canary` (`-w`) option.
     19  * If the script runs successfully, new baselines (expected files) are added to SVN repository and the rebaselined tests are removed from `test_expectations.txt`.
    2220  * The tool launches a html page to compare the old and new baselines. Check the new baseline changes are expected.
    23   * Prepare a patch with the new baselines and _test_expectations.txt_, review the changes and then commit the patch.
     21  * Prepare a patch with the new baselines and `test_expectations.txt`, review the changes and then commit the patch.
    2422
    2523== Examples ==
    26 We need rebaseline test _LayoutTests/foo1.html_ for all three platforms and test _LayoutTests/foo2.html_ for *win* and *linux*.
     24We need rebaseline test `LayoutTests/foo1.html` for all three platforms and test `LayoutTests/foo2.html` for Chromium Win and Linux.
    2725
    28   * Make sure test files _foo1.html_ and _foo2.html_ have been committed at _src\webkit\data\layout_tests\LayoutTests_. If these are tests just added, make sure the layout tests ran at buildbot and the test results have been archived.
    29   * Update _test_expectations.txt_ to include the following lines:
     26  * If these are tests just added, make sure the layout tests ran at buildbot and the test results have been archived.
     27  * Update `test_expectations.txt` to include the following lines:
    3028{{{
    3129BUG1234 REBASELINE : LayoutTests/foo1.html = FAIL
    3230BUG5678 REBASELINE WIN LINUX : LayoutTests/foo2.html = FAIL
    33 }}}http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/rebaseline-chromium-webkit-tests
    34 
    35    If foo1 and foo2 already exist, add option "*REBASELINE*" to the existing tests.
    36   * Run rebaseline script on *Windows* machine:
    37 {{{
    38   open a command window
    39   change the directory to [YOUR CHROMUIM DIRECTORY]\src\webkit\tools\layout_tests
    40   rebaseline.bat
    4131}}}
    4232
    43   * Run rebaseline script on *Linux* or *Mac* machine
    44 {{{
    45   open a shell
    46   change the directory to [YOUR CHROMUIM DIRECTORY]/src/webkit/tools/layout_tests
    47   ./rebaseline.sh
    48 }}}
     33  * Run `WebKitTools/Scripts/rebaseline-chromium-webkit-tests`.
     34  * Check that tests `foo1` and `foo2` were removed from test_expectations.txt.
    4935
    50   NOTE: *you only need to run the script on ONE platform. The tool will auto generate new baselines for ALL three platforms*.
     36  * Create a patch, review the changes and commit.
    5137
    52   * Check tests foo1 and foo2 are removed from test_expectations.txt.
    53 
    54   * Create a patch
    55 {{{
    56 gcl change [YOUR PATCH NAME]
    57 }}}
    58   Review the changes and then commit it.  Don't check in with any tests still marked REBASELINE, as this will prevent others from cleanly using the tool.
    59 
    60   * The tool launches a html to compare new baselines with old baselines. Here is an example of the baseline result comparison html:
    61   [http://www.corp.google.com/~victorw/chromium/rebaseline_html/rebaseline.html Baseline Result Comparison HTML]
     38'''NOTE''': Don't check in with any tests still marked REBASELINE, as this will prevent others from cleanly using the tool.
    6239
    6340== Notes to GIT user ==