This guide will help you triage test failures on build.webkit.org and get the bots back to being green.

1. Find out what is failing. There are two main ways to do this:
   • Browse build.webkit.org (you should probably start with this one)
     1. Find recent builds that have failed
        • Windows:
          1. Go to ​http://build.webkit.org/buildslaves.
          2. Click on the name of a slave you're interested in to see a summary of its recent builds.
        • Other platforms:
          1. Go to ​http://build.webkit.org/builders.
          2. Click on the name of a builder you're interested in to see a summary of its recent builds.
        • The Info column will tell you if any tests failed for that build.
        • To see the test output for a particular build, click on the link in the Build # column, then on view results, then on results.html
   • Use webkit-patch
     1. Run this command:

        webkit-patch failure-reason
        

     2. When prompted, specify which builder you're interested in.
     3. Press Enter to continue. webkit-patch will look back through the recent builds for that builder until it has found when all current failures were introduced.
2. Group failures by "root causes"
   1. If a multiple tests are all failing for the same reason, they should be grouped together into a single root cause
3. File one or more bugs for each root cause. (If a test fails on multiple platforms and those platforms will need separate fixes, you should file one bug for each failing platform.)
   1. Go to ​http://webkit.org/new-bug
   2. Include in your report:
      • The name(s) of the failing test(s)
      • What platform(s) the failures occur on
      • When the failures began, if known
      • A link to the failing output
        • If a single test had incorrect output, link to the pretty diff.
        • If multiple tests had incorrect output, or if the failure is a crash or hang, link to results.html.
   3. Apply keywords
      • LayoutTestFailure
      • Regression, if the failure is due to a regression in WebKit
      • PlatformOnly, if the test only fails on one platform
4. Get the bots green again
   • If the tests always fail with the same incorrect output, check in new results for the tests and include the bug URL in your ChangeLog
   • If the tests fail intermittently, or crash, or hang, add the tests to the appropriate Skipped files. Include a comment in the Skipped file with the bug URL and a brief description of how it fails (e.g., "Crashes", "Sometimes times out", etc.).