= Rebaseline Server = The "Rebaseline Server" is a tool that makes it easy to do "mass" rebaselines (updating of pixel or text baselines for dozens or hundreds of layout tests). It makes it easy to review changes to baselines and update existing ones (and optionally move them, to handle OS transitions). It's known as a ''server'' because it runs a local HTTP server which makes it easy to have a cross-platform GUI for reviewing that is backed by Python code for doing the SCM (SVN/Git) operations on the filesystem. To use the tool, run: `webkit-patch rebaseline-server WebKitBuild/{Debug|Release}/layout-test-results` (after having run `new-run-webkit-tests`). See [#SampleUsages sample usages] below for more details. == UI Walkthrough == [[Image(overview.png, 1000px)]] === 1. Test selectors === [[Image(section-1.png)]] These menus let you choose which test(s) you want to rebaseline. The first menu breaks down tests by failure type. Within that failure type, the second menu filters by directory. The third menu lists individual tests. You can use the « and » buttons to move through the tests in the chosen directory (there are also [#Keyboardshortcuts keyboard shortcuts]). The __View test__ link shows the test's source on `trac.webkit.org.` === 2. Image results === Shows expected/actual/diff of the image results (if the test failed because of image diffs). The diff displays pixels that were the same at 25% opacity, and different ones as red. You can click on any image to use the [#Loupe loupe] to magnify that area. In the case of matching images but mismatching checksums, the diff area displays the checksums. === 3. Text results === Similarly to image results, this area displays expected/actual/diffs of text results if the test failed because of text differences. === 4. Loupe === === Keyboard shortcuts === The following keyboard shortcuts are supported: * left and right arrow: move through tests * q: add the current test to the rebaseline queue * x: remove the current test from the rebaseline queue * r: rebaseline all the tests in the queue == Sample Usages == === Leopard to Snow Leopard Transition === Snow Leopard has a different text anti-aliasing algorithm (and other small rendering tweaks) which requires many pixel baselines to be updated. The flow for doing this would be. 1. (On a Snow Leopard machine) run `new-run-webkit-tests --tolerance 0 --pixel-tests [path/to/tests]` (or omit the path to run all tests) 1. Observe that many tests failed with imaged differences 1. Run `webkit-patch rebaseline-server WebKitBuild/{Debug|Release}/layout-test-results` (or wherever your layout test results are localed) 1. http://localhost:8127/ should be launched in your browser, navigate to it if not 1. From the footer of the UI, set the '''baseline target''' to `mac` (since we want the default `mac` port baselines to reflect the newest shipping OS) 1. Similarly, set the '''Move current baselines to:''' dropdown to `mac-leopard`, so that the existing baselines are still used for bots that are on Leopard 1. Refer to the [#UIWalkthrough UI walkthrough] for more details on how to use the GUI 1. Commit the result patch with `webkit-patch` as usual If working on the Chromium port, the above steps can be used there too, except the baseline target is `chromium-mac` and existing baselines should be moved to `chromium-mac-leopard`. === Skia changes === [http://code.google.com/p/skia/ Skia] is the graphics library used by the Chromium port on Windows and Linux; changes to it often require updates to pixel baselines. 1. (On a Windows or Linux machine) run `new-run-webkit-tests --tolerance 0 --pixel-tests [path/to/tests]` (or omit the path to run all tests) 1. Observe that many tests failed with imaged differences 1. Run `webkit-patch rebaseline-server WebKitBuild/{Debug|Release}/layout-test-results` (or wherever your layout test results are localed) 1. http://localhost:8127/ should be launched in your browser, navigate to it if not 1. From the footer of the UI, set the '''baseline target''' to `chromium-win` or `chromium-win` as appropriate. 1. Leave the '''Move current baselines to:''' dropdown set to `Nowhere (replace)` 1. Refer to the [#UIWalkthrough UI walkthrough] for more details on how to use the GUI 1. Commit the result patch with `webkit-patch` as usual Note that if you'd like to rebaseline both Windows and Linux in the same patch, you'll need to run the tests on both platforms by hand and copy the results file to the machine that's running the rebaseline server (see the [#CaveatsLimitations caveats] section). == Code location and design == The Python server lives at [http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/tool/commands/rebaselineserver.py WebKitTools/Scripts/webkitpy/tool/commands/rebaselineserver.py], the GUI is at [http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/tool/commands/data/rebaselineserver/ WebKitTools/Scripts/webkitpy/tool/commands/data/rebaselineserver/]. The Python server parses the `unexpected_results.json`file [http://www.google.com/codesearch/p?hl=en#OAMlx_jo-ck/src/third_party/WebKit/WebKitTools/Scripts/webkitpy/layout_tests/run_webkit_tests.py&q=unexpected_results%20file:webkitpy&exact_package=chromium&l=823 generated] by `new-run-webkit-tests`. It then starts [http://docs.python.org/library/basehttpserver.html an HTTP server]. That server can respond with the list of failing tests, expected/actual output for a test (read from the test results directory), or to commands to rebaseline tests. The latter is done by using the [http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/common/checkout/scm.py scm.py] wrapper that makes it agnostic to Git vs. SVN checkouts. The UI populates various `` as well (see `queue.js`). == Caveats/Limitations == * `new-run-webkit-tests` must be used (the tool currently relies on the JSON output that only NRWT produces) * The tool can currently only use local test results, it cannot pull from bots * The UI works best on 30 inch monitor