Changes between Initial Version and Version 1 of WPTUpdateBotDesign


Ignore:
Timestamp:
Aug 3, 2020 10:16:29 PM (2 years ago)
Author:
clopez@igalia.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • WPTUpdateBotDesign

    v1 v1  
     1
     2This page contains some design notes about the proposal of a bot to help automating the WPT import of tests inside WebKit.
     3
     4For reference check: https://lists.webkit.org/pipermail/webkit-dev/2020-March/031121.html
     5
     6----
     7
     8Config file:
     9
     10{{{
     11# X => days since last sucessful try to start a new one
     12# Y => days since patch ready to say "ping.. please do anything"
     13}}}
     14
     15State diagram:
     16
     17- Print internal state (prints WebKitBuild/$FileInternalState)
     18- Clean all
     19- Read WebKitBuild/$FileInternalState
     20    - lastBug has tag == SUCCESS and $date for $lastBug < X days:
     21        - print: "not doing anything because last bug $lastBug is less than X days"
     22        - return: green
     23    - NOT EXISTS $FileInternalState OR ($lastBug has tag != WORKING)
     24        - Open bug $currentBug
     25        - Create git branch bug-$currentBug
     26        - Get last WPT version from https://github.com/web-platform-tests/wpt.git
     27        - Update WPT version at LayoutTests/imported/w3c/resources/TestRepositories
     28        - Run script: Tools/Scripts/import-w3c-tests -v
     29        - Commit changes locally
     30        - Generate changelog
     31        - Commit changes locally (git)
     32        - Upload all changes (patch/HEAD) to $currentBug
     33        - Check that the upload worked (query bug; see patch attached)
     34        - Save $date and $currentBug in WebKitBuild/$FileInternalState with tag WORKING
     35        - return: green
     36    - $lastBug has tag == WORKING
     37        - Check bug state:
     38            - Closed and landed
     39                - Modify $FileInternalState and change the tag for $bug to SUCCESS
     40                - return: green.
     41            - Still open
     42                - Apply patch from bug $currentBug: Tools/Scripts/webkit-patch apply-from-bug --no-clean --no-update $currentBug
     43                    - not applies
     44                        - Close $currentBug saying that bug closed due to patch not longer applying. A new attempt will be made in other bug.
     45                        - Modify $FileInternalState and change the tag for $bug to FAILURE
     46                        - return: red (patch not longer applies)
     47                    - applies:         
     48                        - Check EWS state for patch:
     49                            - EWS: Still working
     50                                - bot-comment: "EWS $foo $bar still working, not doing anything"
     51                                - return: green
     52                            - EWS: All green
     53                                - Patch has r-:
     54                                    - comment: "i don't know how to handle an r-, sorry.. I will attempt a new update on a new bug"
     55                                    - Modify $FileInternalState and change the tag for $bug to FAILURE
     56                                    - return: red (patch had r-)
     57                                - Patch has r? or r+
     58                                    - if older than Y days: ping, saying the patch still applies, please (if r+ say one thing, if r? say other)
     59                                    - return: green.
     60                                - Patch does not have r?
     61                                    - CC list of reviewers
     62                                    - Set r? on patch and aks for review.
     63                                    - If there is a $lastBug in $FileInternalState:
     64                                        - comment on $lastBug that the next attempt is ready for review in $currentBug
     65                                        - add $lastBug to "see also" of $currentBug
     66                                    - return: green.
     67                            - EWS: Some red
     68                                - Apply patch from bug $currentBug
     69                                - clean-unwantend-files: remove .rej files and changes on changelogs
     70                                - Update results/expectations from EWS: run script update-test-expectations-from-bugzilla
     71                                    - QA: this step should fail if no changes are done. return: red (EWS red, but no changes to expectations)
     72                                    - QA: match expectation files against patterns that may suggest a broken import, like "Can't find variable:"
     73                                        - if it matches: return red (Requires manual intervention, seems the import broke something)
     74                                - Generate changelog
     75                                - Upload all changes (patch) to bug
     76                                - return: green
     77
     78Notes: When checking EWS status is useful to use a while-list to only check for test-bots. For example: it would be ok to ignore a red on the check-style bot.