wiki:Rebaseline

Version 20 (modified by ojan@chromium.org, 12 years ago) (diff)

--

Rebaselining Tool

The rebaselining tool automatically produces new expected results using the layout test results from buildbots. The goal is that after running this tool, the working copy will be ready to create a change containing all new baselines.

You can also use "webkit-patch garden-o-matic" to run garden-o-matic locally and do rebaselines from the GUI.

How does this tool work?

  • Grabs the results off the buildbot and puts them in the right place in your local repository.
  • Updates TestExpectations to remove the appropriate lines for rebaselined tests.
  • Runs webkit-patch optimize-baselines to remove duplicate baselines across ports.

How to use it?

Make sure the test files exist in repository (i.e. you've synced) and the build bots have cycled at least once. If you are not sure, check garden-o-matic or the buildbot waterfall to see whether your test results are available.

Gives you a list of bots to select from, then a list of failing tests on each selected bot:

webkit-patch rebaseline

Gives you a list of bots to select from then rebaselines the two tests on those bots:

webkit-patch rebaseline path/to/test1.html path/to/test2.html

Finds all the tests with a REBASELINE modifier in TestExpectations and rebaselines appropriately:

webkit-patch rebaseline-expectations

For example,

BUG1234 REBASELINE : path/to/test1.html = IMAGE
BUG1234 REBASELINE : path/to/test2.html = IMAGE+TEXT
BUG5678 REBASELINE WIN LINUX : path/to/test3.html = TEXT

Rebaselines just the png for test1.html, the png and txt for test2.html and just the txt for test3.html.

NOTE: Don't check in with any tests still marked REBASELINE, as this will prevent others from cleanly using the tool.