Rebaseline Server

The "Rebaseline Server" is a tool that makes it easy to do "mass" rebaselines (updating of pixel or text baselines for dozens or hundreds of layout tests). It makes it easy to review changes to baselines and update existing ones (and optionally move them, to handle OS transitions). It's known as a server because it runs a local HTTP server which makes it easy to have a cross-platform GUI for reviewing that is backed by Python code for doing the SCM (SVN/Git) operations on the filesystem.

Sample Usages

Leopard to Snow Leopard Transition

Snow Leopard has a different text anti-aliasing algorithm (and other small rendering tweaks) which requires many pixel baselines to be updated. The flow for doing this would be.

1. (On a Snow Leopard machine) run new-run-webkit-tests --tolerance 0 --pixel-tests [path/to/tests] (or omit the path to run all tests)
2. Observe that many tests failed with imaged differences
3. Run webkit-patch rebaseline-server WebKitBuild/{Debug|Release}/layout-test-results (or wherever your layout test results are localed)
4. ​http://localhost:8127/ should be launched in your browser, navigate to it if not
5. From the footer of the UI, set the baseline target to mac (since we want the default mac port baselines to reflect the newest shipping OS)
6. Similarly, set the Move current baselines to: dropdown to mac-leopard, so that the existing baselines are still used for bots that are on Leopard
7. Refer to the UI walkthrough for more details on how to use the GUI

If working on the Chromium port, the above steps can be used there too, except the baseline target is chromium-mac and existing baselines should be moved to chromium-mac-leopard.

Skia changes

​Skia is the graphics library used by the Chromium port on Windows and Linux; changes to it often require updates to pixel baselines.

1. (On a Windows or Linux machine) run new-run-webkit-tests --tolerance 0 --pixel-tests [path/to/tests] (or omit the path to run all tests)
2. Observe that many tests failed with imaged differences
3. Run webkit-patch rebaseline-server WebKitBuild/{Debug|Release}/layout-test-results (or wherever your layout test results are localed)
4. ​http://localhost:8127/ should be launched in your browser, navigate to it if not
5. From the footer of the UI, set the baseline target to chromium-win or chromium-win as appropriate.
6. Leave the Move current baselines to: dropdown set to Nowhere (replace)
7. Refer to the UI walkthrough for more details on how to use the GUI

Note that if you'd like to rebaseline both Windows and Linux in the same patch, you'll need to run the tests on both platforms by hand and copy the results file to the machine that's running the rebaseline server.

UI Walkthrough

Code location and design

The Python server lives at ​WebKitTools/Scripts/webkitpy/tool/commands/rebaselineserver.py, the GUI is at ​WebKitTools/Scripts/webkitpy/tool/commands/data/rebaselineserver/.

Caveats/Limitations

• new-run-webkit-tests must be used (the tool currently relies on the JSON output that only NRWT produces)
• The tool can currently only use local test results, it cannot pull from bots