| | 1 | |
| | 2 | This page contains some design notes about the proposal of a bot to help automating the WPT import of tests inside WebKit. |
| | 3 | |
| | 4 | For reference check: https://lists.webkit.org/pipermail/webkit-dev/2020-March/031121.html |
| | 5 | |
| | 6 | ---- |
| | 7 | |
| | 8 | Config file: |
| | 9 | |
| | 10 | {{{ |
| | 11 | # X => days since last sucessful try to start a new one |
| | 12 | # Y => days since patch ready to say "ping.. please do anything" |
| | 13 | }}} |
| | 14 | |
| | 15 | State diagram: |
| | 16 | |
| | 17 | - Print internal state (prints WebKitBuild/$FileInternalState) |
| | 18 | - Clean all |
| | 19 | - Read WebKitBuild/$FileInternalState |
| | 20 | - lastBug has tag == SUCCESS and $date for $lastBug < X days: |
| | 21 | - print: "not doing anything because last bug $lastBug is less than X days" |
| | 22 | - return: green |
| | 23 | - NOT EXISTS $FileInternalState OR ($lastBug has tag != WORKING) |
| | 24 | - Open bug $currentBug |
| | 25 | - Create git branch bug-$currentBug |
| | 26 | - Get last WPT version from https://github.com/web-platform-tests/wpt.git |
| | 27 | - Update WPT version at LayoutTests/imported/w3c/resources/TestRepositories |
| | 28 | - Run script: Tools/Scripts/import-w3c-tests -v |
| | 29 | - Commit changes locally |
| | 30 | - Generate changelog |
| | 31 | - Commit changes locally (git) |
| | 32 | - Upload all changes (patch/HEAD) to $currentBug |
| | 33 | - Check that the upload worked (query bug; see patch attached) |
| | 34 | - Save $date and $currentBug in WebKitBuild/$FileInternalState with tag WORKING |
| | 35 | - return: green |
| | 36 | - $lastBug has tag == WORKING |
| | 37 | - Check bug state: |
| | 38 | - Closed and landed |
| | 39 | - Modify $FileInternalState and change the tag for $bug to SUCCESS |
| | 40 | - return: green. |
| | 41 | - Still open |
| | 42 | - Apply patch from bug $currentBug: Tools/Scripts/webkit-patch apply-from-bug --no-clean --no-update $currentBug |
| | 43 | - not applies |
| | 44 | - Close $currentBug saying that bug closed due to patch not longer applying. A new attempt will be made in other bug. |
| | 45 | - Modify $FileInternalState and change the tag for $bug to FAILURE |
| | 46 | - return: red (patch not longer applies) |
| | 47 | - applies: |
| | 48 | - Check EWS state for patch: |
| | 49 | - EWS: Still working |
| | 50 | - bot-comment: "EWS $foo $bar still working, not doing anything" |
| | 51 | - return: green |
| | 52 | - EWS: All green |
| | 53 | - Patch has r-: |
| | 54 | - comment: "i don't know how to handle an r-, sorry.. I will attempt a new update on a new bug" |
| | 55 | - Modify $FileInternalState and change the tag for $bug to FAILURE |
| | 56 | - return: red (patch had r-) |
| | 57 | - Patch has r? or r+ |
| | 58 | - if older than Y days: ping, saying the patch still applies, please (if r+ say one thing, if r? say other) |
| | 59 | - return: green. |
| | 60 | - Patch does not have r? |
| | 61 | - CC list of reviewers |
| | 62 | - Set r? on patch and aks for review. |
| | 63 | - If there is a $lastBug in $FileInternalState: |
| | 64 | - comment on $lastBug that the next attempt is ready for review in $currentBug |
| | 65 | - add $lastBug to "see also" of $currentBug |
| | 66 | - return: green. |
| | 67 | - EWS: Some red |
| | 68 | - Apply patch from bug $currentBug |
| | 69 | - clean-unwantend-files: remove .rej files and changes on changelogs |
| | 70 | - Update results/expectations from EWS: run script update-test-expectations-from-bugzilla |
| | 71 | - QA: this step should fail if no changes are done. return: red (EWS red, but no changes to expectations) |
| | 72 | - QA: match expectation files against patterns that may suggest a broken import, like "Can't find variable:" |
| | 73 | - if it matches: return red (Requires manual intervention, seems the import broke something) |
| | 74 | - Generate changelog |
| | 75 | - Upload all changes (patch) to bug |
| | 76 | - return: green |
| | 77 | |
| | 78 | Notes: When checking EWS status is useful to use a while-list to only check for test-bots. For example: it would be ok to ignore a red on the check-style bot. |
