This page contains some design notes about the proposal of a bot to help automating the WPT import of tests inside WebKit. For reference check: https://lists.webkit.org/pipermail/webkit-dev/2020-March/031121.html ---- Config file: {{{ # X => days since last sucessful try to start a new one # Y => days since patch ready to say "ping.. please do anything" }}} State diagram: - Print internal state (prints WebKitBuild/$FileInternalState) - Clean all - Read WebKitBuild/$FileInternalState - lastBug has tag == SUCCESS and $date for $lastBug < X days: - print: "not doing anything because last bug $lastBug is less than X days" - return: green - NOT EXISTS $FileInternalState OR ($lastBug has tag != WORKING) - Open bug $currentBug - Create git branch bug-$currentBug - Get last WPT version from https://github.com/web-platform-tests/wpt.git - Update WPT version at LayoutTests/imported/w3c/resources/TestRepositories - Run script: Tools/Scripts/import-w3c-tests -v - Commit changes locally - Generate changelog - Commit changes locally (git) - Upload all changes (patch/HEAD) to $currentBug - Check that the upload worked (query bug; see patch attached) - Save $date and $currentBug in WebKitBuild/$FileInternalState with tag WORKING - return: green - $lastBug has tag == WORKING - Check bug state: - Closed and landed - Modify $FileInternalState and change the tag for $bug to SUCCESS - return: green. - Still open - Apply patch from bug $currentBug: Tools/Scripts/webkit-patch apply-from-bug --no-clean --no-update $currentBug - not applies - Close $currentBug saying that bug closed due to patch not longer applying. A new attempt will be made in other bug. - Modify $FileInternalState and change the tag for $bug to FAILURE - return: red (patch not longer applies) - applies: - Check EWS state for patch: - EWS: Still working - bot-comment: "EWS $foo $bar still working, not doing anything" - return: green - EWS: All green - Patch has r-: - comment: "i don't know how to handle an r-, sorry.. I will attempt a new update on a new bug" - Modify $FileInternalState and change the tag for $bug to FAILURE - return: red (patch had r-) - Patch has r? or r+ - if older than Y days: ping, saying the patch still applies, please (if r+ say one thing, if r? say other) - return: green. - Patch does not have r? - CC list of reviewers - Set r? on patch and aks for review. - If there is a $lastBug in $FileInternalState: - comment on $lastBug that the next attempt is ready for review in $currentBug - add $lastBug to "see also" of $currentBug - return: green. - EWS: Some red - Apply patch from bug $currentBug - clean-unwantend-files: remove .rej files and changes on changelogs - Update results/expectations from EWS: run script update-test-expectations-from-bugzilla - QA: this step should fail if no changes are done. return: red (EWS red, but no changes to expectations) - QA: match expectation files against patterns that may suggest a broken import, like "Can't find variable:" - if it matches: return red (Requires manual intervention, seems the import broke something) - Generate changelog - Upload all changes (patch) to bug - return: green Notes: When checking EWS status is useful to use a while-list to only check for test-bots. For example: it would be ok to ignore a red on the check-style bot.