wiki:WPTUpdateBotDesign

This page contains some design notes about the proposal of a bot to help automating the WPT import of tests inside WebKit.

For reference check: https://lists.webkit.org/pipermail/webkit-dev/2020-March/031121.html


Config file:

# X => days since last sucessful try to start a new one
# Y => days since patch ready to say "ping.. please do anything"

State diagram:

  • Print internal state (prints WebKitBuild/$FileInternalState)
  • Clean all
  • Read WebKitBuild/$FileInternalState
    • lastBug has tag == SUCCESS and $date for $lastBug < X days:
      • print: "not doing anything because last bug $lastBug is less than X days"
      • return: green
    • NOT EXISTS $FileInternalState OR ($lastBug has tag != WORKING)
      • Open bug $currentBug
      • Create git branch bug-$currentBug
      • Get last WPT version from https://github.com/web-platform-tests/wpt.git
      • Update WPT version at LayoutTests/imported/w3c/resources/TestRepositories
      • Run script: Tools/Scripts/import-w3c-tests -v
      • Commit changes locally
      • Generate changelog
      • Commit changes locally (git)
      • Upload all changes (patch/HEAD) to $currentBug
      • Check that the upload worked (query bug; see patch attached)
      • Save $date and $currentBug in WebKitBuild/$FileInternalState with tag WORKING
      • return: green
    • $lastBug has tag == WORKING
      • Check bug state:
        • Closed and landed
          • Modify $FileInternalState and change the tag for $bug to SUCCESS
          • return: green.
        • Still open
          • Apply patch from bug $currentBug: Tools/Scripts/webkit-patch apply-from-bug --no-clean --no-update $currentBug
            • not applies
              • Close $currentBug saying that bug closed due to patch not longer applying. A new attempt will be made in other bug.
              • Modify $FileInternalState and change the tag for $bug to FAILURE
              • return: red (patch not longer applies)
            • applies:
              • Check EWS state for patch:
                • EWS: Still working
                  • bot-comment: "EWS $foo $bar still working, not doing anything"
                  • return: green
                • EWS: All green
                  • Patch has r-:
                    • comment: "i don't know how to handle an r-, sorry.. I will attempt a new update on a new bug"
                    • Modify $FileInternalState and change the tag for $bug to FAILURE
                    • return: red (patch had r-)
                  • Patch has r? or r+
                    • if older than Y days: ping, saying the patch still applies, please (if r+ say one thing, if r? say other)
                    • return: green.
                  • Patch does not have r?
                    • CC list of reviewers
                    • Set r? on patch and aks for review.
                    • If there is a $lastBug in $FileInternalState:
                      • comment on $lastBug that the next attempt is ready for review in $currentBug
                      • add $lastBug to "see also" of $currentBug
                    • return: green.
                • EWS: Some red
                  • Apply patch from bug $currentBug
                  • clean-unwantend-files: remove .rej files and changes on changelogs
                  • Update results/expectations from EWS: run script update-test-expectations-from-bugzilla
                    • QA: this step should fail if no changes are done. return: red (EWS red, but no changes to expectations)
                    • QA: match expectation files against patterns that may suggest a broken import, like "Can't find variable:"
                      • if it matches: return red (Requires manual intervention, seems the import broke something)
                  • Generate changelog
                  • Upload all changes (patch) to bug
                  • return: green

Notes: When checking EWS status is useful to use a while-list to only check for test-bots. For example: it would be ok to ignore a red on the check-style bot.

Last modified 16 months ago Last modified on Aug 3, 2020 10:16:29 PM