Version 2 (modified by Adam Roben, 12 years ago) (diff)

Slightly more prose-y

This guide will help you triage test failures on and get the bots back to being green.

Find out what is failing

There are two main ways to do this:

  • Browse (you should probably start with this one)
    1. Find recent builds that have failed
      • Windows:
        1. Go to
        2. Click on the name of a slave you're interested in to see a summary of its recent builds.
      • Other platforms:
        1. Go to
        2. Click on the name of a builder you're interested in to see a summary of its recent builds.
      • The Info column will tell you if any tests failed for that build.
      • To see the test output for a particular build, click on the link in the Build # column, then on view results, then on results.html
  • Use webkit-patch
    1. Run this command:
      webkit-patch failure-reason
    2. When prompted, specify which builder you're interested in.
    3. Press Enter to continue. webkit-patch will look back through the recent builds for that builder until it has found when all current failures were introduced.

Group failures by "root cause"

If a multiple tests are all failing for the same reason, they should be grouped together into a single root cause.

File one or more bugs for each root cause

If a test fails on multiple platforms and those platforms will need separate fixes, you should file one bug for each failing platform.

  1. Go to
  2. Include in your report:
    • The name(s) of the failing test(s)
    • What platform(s) the failures occur on
    • When the failures began, if known
    • A link to the failing output
      • If a single test had incorrect output, link to the pretty diff.
      • If multiple tests had incorrect output, or if the failure is a crash or hang, link to results.html.
  3. Apply keywords
    • LayoutTestFailure
    • Regression, if the failure is due to a regression in WebKit
    • PlatformOnly, if the test only fails on one platform

Get the bots green again

  • If the tests always fail with the same incorrect output, check in new results for the tests and include the bug URL in your ChangeLog
  • If the tests fail intermittently, or crash, or hang, add the tests to the appropriate Skipped files. Include a comment in the Skipped file with the bug URL and a brief description of how it fails (e.g., "Crashes", "Sometimes times out", etc.).