Changes between Version 12 and Version 13 of WebKitGTK/Gardening/Howto


Ignore:
Timestamp:
Apr 17, 2020 6:36:05 AM (11 months ago)
Author:
Lauro Moura
Comment:

Update flakiness dashboard link

Legend:

Unmodified
Added
Removed
Modified
  • WebKitGTK/Gardening/Howto

    v12 v13  
    2626}}}
    27272. We pick one tests and we must research since '''when''' the test started to fail. Some ways of doing it:
    28   * '''The failure is recent (<50 releases)''': Check on the [https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html Flakiness Dashboard] the recent history of each test on the buildbots.
    29   * '''The failure is not so recent (<200 releases)''': Manually checking the stdio log for the layout tests on the [https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29?numbuilds=200 last 200 builds]. If the test passes on the older build available there , but not on the current build, then you can manually bisect the logs until you find when it started to fail. Maybe you can't find the exact release because not all releases were built or because the bot failed to compile between them. But at least you should able to allocate a shorter interval than you can bisect further by building webkit for each release on the interval and testing it.
     28  * '''The failure is recent (<50 revisions)''': Check on the [https://results.webkit.org/ Flakiness Dashboard] the recent history of each test on the buildbots.
     29  * '''The failure is not so recent (<200 revisions)''': Manually checking the stdio log for the layout tests on the [https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29?numbuilds=200 last 200 builds]. If the test passes on the older build available there , but not on the current build, then you can manually bisect the logs until you find when it started to fail. Maybe you can't find the exact release because not all releases were built or because the bot failed to compile between them. But at least you should able to allocate a shorter interval than you can bisect further by building webkit for each release on the interval and testing it.
    3030{{{
    3131  * media/video-controls-captions.html [ Timeout ]
    3232    --> 166964 < bug < 167006 (bot failed to compile webkit in between)
    3333}}}
    34   * '''The failure is not recent (>200 releases)''': Then you can check on the [https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20%28Tests%29/ raw results dir] the results for each release. You can proceed to do a manual bisection like in the previous case is explained.
     34  * '''The failure is not recent (>200 revisions)''': Then you can check on the [https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20%28Tests%29/ raw results dir] the results for each release. You can proceed to do a manual bisection like in the previous case is explained.
    3535    * This is useful also to check if the new baselines for the tests on the bot are the same that on your machine.
    3636  * '''!WebKit-!TestHunter''': There is a tool that checks the complete known history of webkitgtk layout test results and outputs intervals for the different results for a given test. Check it out at https://github.com/clopez/webkit-testhunter