Changes between Version 13 and Version 14 of WebKitGTK/Gardening/Howto


Ignore:
Timestamp:
Sep 6, 2022 7:54:35 PM (18 months ago)
Author:
Lauro Moura
Comment:

Update build.webkit.org links after migration to new Buildbot version last year

Legend:

Unmodified
Added
Removed
Modified
  • WebKitGTK/Gardening/Howto

    v13 v14  
    1111== Typical Workflow ==
    1212
    13 1. Check on the [https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29 release bot] which tests are failing. To do that we open the last build (or the penultimate if the last one still didn't finished running the layout tests) and we go to the log ("stdio") of the layout tests. (Ex: "8. layout-test 15 failures 5 new passes 25 flakes 1 missing results"). At the end of the log we can see something like:
     131. Check on the [https://build.webkit.org/#/builders/57 release bot] which tests are failing. To do that we open the last build (or the penultimate if the last one still didn't finished running the layout tests) and we go to the log ("stdio") of the layout tests. (Ex: "8. layout-test 15 failures 5 new passes 25 flakes 1 missing results"). At the end of the log we can see something like:
    1414{{{
    1515  Expected to fail, but passed: (3)
     
    27272. We pick one tests and we must research since '''when''' the test started to fail. Some ways of doing it:
    2828  * '''The failure is recent (<50 revisions)''': Check on the [https://results.webkit.org/ Flakiness Dashboard] the recent history of each test on the buildbots.
    29   * '''The failure is not so recent (<200 revisions)''': Manually checking the stdio log for the layout tests on the [https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29?numbuilds=200 last 200 builds]. If the test passes on the older build available there , but not on the current build, then you can manually bisect the logs until you find when it started to fail. Maybe you can't find the exact release because not all releases were built or because the bot failed to compile between them. But at least you should able to allocate a shorter interval than you can bisect further by building webkit for each release on the interval and testing it.
     29  * '''The failure is not so recent (<200 revisions)''': Manually checking the stdio log for the layout tests on the [https://build.webkit.org/#/builders/57 last 200 builds]. If the test passes on the older build available there , but not on the current build, then you can manually bisect the logs until you find when it started to fail. Maybe you can't find the exact release because not all releases were built or because the bot failed to compile between them. But at least you should able to allocate a shorter interval than you can bisect further by building webkit for each release on the interval and testing it.
    3030{{{
    3131  * media/video-controls-captions.html [ Timeout ]
    3232    --> 166964 < bug < 167006 (bot failed to compile webkit in between)
    3333}}}
    34   * '''The failure is not recent (>200 revisions)''': Then you can check on the [https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20%28Tests%29/ raw results dir] the results for each release. You can proceed to do a manual bisection like in the previous case is explained.
     34  * '''The failure is not recent (>200 revisions)''': Then you can check on the [https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/ raw results dir] the results for each release. You can proceed to do a manual bisection like in the previous case is explained.
    3535    * This is useful also to check if the new baselines for the tests on the bot are the same that on your machine.
    3636  * '''!WebKit-!TestHunter''': There is a tool that checks the complete known history of webkitgtk layout test results and outputs intervals for the different results for a given test. Check it out at https://github.com/clopez/webkit-testhunter
     
    5252==== The release bot. ====
    5353
    54 Our target to keep the tree green is the [https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29 GTK Linux 64-bit Release (Tests)] bot.
     54Our target to keep the tree green is the [https://build.webkit.org/#/builders/57 GTK Linux 64-bit Release (Tests)] bot.
    5555
    5656==== Other bots. ====
    5757
    58 You can check on the waterfall all the GTK bots: http://build.webkit.org/waterfall?category=GTK
     58You can check on the waterfall all the GTK bots: https://build.webkit.org/#/waterfall?tags=GTK
    5959
    6060