wiki:QtWebKitGardening

Version 3 (modified by andreas.kling@nokia.com, 13 years ago) (diff)

--

NOTE: This is a proposal in progress.

The purpose of QtWebKit gardening is to keep the Qt build and test bots green. There are a number of tasks for the gardener:

  • Reacting to bot redness.
  • Maintaining the LayoutTests skip lists.
  • Maintaining the LayoutTests expected results.
  • Maintaining the Qt API tests.

Let's look at each one in detail...

Maintaining the LayoutTests skip lists

These are currently the QtWebKit skip lists (black lists for LayoutTests):

  • LayoutTests/platform/qt/Skipped
  • LayoutTests/platform/qt-4.8/Skipped
  • LayoutTests/platform/qt-arm/Skipped
  • LayoutTests/platform/qt-linux/Skipped
  • LayoutTests/platform/qt-mac/Skipped
  • LayoutTests/platform/qt-win/Skipped
  • LayoutTests/platform/qt-wk2/Skipped

The "qt" list (LayoutTests/platform/qt/Skipped) is common and applies to all platforms. The other lists are for failures that are specific to that platform.

When a test starts failing, or a new test is added and fails on the Qt bot(s), the first thing to do is poking the person who caused the failure. If they are unavailable or insufficiently responsive, add the test to the appropriate Qt skip list and file a bug about it.

Regression bugs should have a title with format "REGRESSION(r######): [Qt] blah blah blah". Examples:

  • REGRESSION(r12345): [Qt] New test fast/images/super-cute-cat.html failing.
  • REGRESSION(r67890): [Qt] Caused three tests in canvas/philip/ to fail.

The bugs should be marked with the keywords "Qt, QtTriaged, Regression". The CC list should include the developers who authored and reviewed the offending changeset.

Maintaining the LayoutTests expected results

When new tests are added, it's sometimes necessary to have a platform-specific expected result, for instance if the output includes text metrics. These files live in LayoutTests/platform/qt*/foo/bar/baz-expected.txt and LayoutTests/platform/qt*/foo/bar/baz-expected.png (which corresponds to the test LayoutTests/foo/bar/baz.html)

For the text results, this is often as simple as running "webkit-patch rebaseline" and grabbing the results from the Qt bot. The screenshot (PNG) results are a little trickier, as they are currently not dumped by the bot, and need to be generated manually. These tests should be generated in an environment equivalent to that of the bot, i.e the Q-BAT VM from Szeged.

Attachments (5)

Download all attachments as: .zip