This page contains some design notes about the proposal of a bot to help automating the WPT import of tests inside WebKit.

For reference check: ​https://lists.webkit.org/pipermail/webkit-dev/2020-March/031121.html

---

Config file:

# X => days since last sucessful try to start a new one
# Y => days since patch ready to say "ping.. please do anything"

State diagram:

• Print internal state (prints WebKitBuild/$FileInternalState)
• Clean all
• Read WebKitBuild/$FileInternalState
  • lastBug has tag == SUCCESS and $date for $lastBug < X days:
    • print: "not doing anything because last bug $lastBug is less than X days"
    • return: green
  • NOT EXISTS $FileInternalState OR ($lastBug has tag != WORKING)
    • Open bug $currentBug
    • Create git branch bug-$currentBug
    • Get last WPT version from ​https://github.com/web-platform-tests/wpt.git
    • Update WPT version at LayoutTests/imported/w3c/resources/TestRepositories
    • Run script: Tools/Scripts/import-w3c-tests -v
    • Commit changes locally
    • Generate changelog
    • Commit changes locally (git)
    • Upload all changes (patch/HEAD) to $currentBug
    • Check that the upload worked (query bug; see patch attached)
    • Save $date and $currentBug in WebKitBuild/$FileInternalState with tag WORKING
    • return: green
  • $lastBug has tag == WORKING
    • Check bug state:
      • Closed and landed
        • Modify $FileInternalState and change the tag for $bug to SUCCESS
        • return: green.
      • Still open
        • Apply patch from bug $currentBug: Tools/Scripts/webkit-patch apply-from-bug --no-clean --no-update $currentBug
          • not applies
            • Close $currentBug saying that bug closed due to patch not longer applying. A new attempt will be made in other bug.
            • Modify $FileInternalState and change the tag for $bug to FAILURE
            • return: red (patch not longer applies)
          • applies:
            • Check EWS state for patch:
              • EWS: Still working
                • bot-comment: "EWS $foo $bar still working, not doing anything"
                • return: green
              • EWS: All green
                • Patch has r-:
                  • comment: "i don't know how to handle an r-, sorry.. I will attempt a new update on a new bug"
                  • Modify $FileInternalState and change the tag for $bug to FAILURE
                  • return: red (patch had r-)
                • Patch has r? or r+
                  • if older than Y days: ping, saying the patch still applies, please (if r+ say one thing, if r? say other)
                  • return: green.
                • Patch does not have r?
                  • CC list of reviewers
                  • Set r? on patch and aks for review.
                  • If there is a $lastBug in $FileInternalState:
                    • comment on $lastBug that the next attempt is ready for review in $currentBug
                    • add $lastBug to "see also" of $currentBug
                  • return: green.
              • EWS: Some red
                • Apply patch from bug $currentBug
                • clean-unwantend-files: remove .rej files and changes on changelogs
                • Update results/expectations from EWS: run script update-test-expectations-from-bugzilla
                  • QA: this step should fail if no changes are done. return: red (EWS red, but no changes to expectations)
                  • QA: match expectation files against patterns that may suggest a broken import, like "Can't find variable:"
                    • if it matches: return red (Requires manual intervention, seems the import broke something)
                • Generate changelog
                • Upload all changes (patch) to bug
                • return: green

Notes: When checking EWS status is useful to use a while-list to only check for test-bots. For example: it would be ok to ignore a red on the check-style bot.