Changeset 244306 in webkit


Ignore:
Timestamp:
Apr 15, 2019 4:36:25 PM (5 years ago)
Author:
aakash_jain@apple.com
Message:

ews-build should clearly indicate flaky test failures
https://bugs.webkit.org/show_bug.cgi?id=196947

Reviewed by Lucas Forschler.

  • BuildSlaveSupport/ews-build/steps.py:

(AnalyzeAPITestsResults.analyzeResults):

Location:
trunk/Tools
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • trunk/Tools/BuildSlaveSupport/ews-build/steps.py

    r244283 r244306  
    793793        clean_tree_failures = getAPITestFailures(clean_tree_results)
    794794
    795         self._addToLog('stderr', '\nFailures in API Test first run: {}'.format(first_run_failures))
    796         self._addToLog('stderr', '\nFailures in API Test second run: {}'.format(first_run_failures))
    797         self._addToLog('stderr', '\nFailures in API Test on clean tree: {}'.format(clean_tree_failures))
    798795        failures_with_patch = first_run_failures.intersection(second_run_failures)
     796        flaky_failures = first_run_failures.union(second_run_failures) - first_run_failures.intersection(second_run_failures)
     797        flaky_failures_string = ', '.join([failure_name.replace('TestWebKitAPI.', '') for failure_name in flaky_failures])
    799798        new_failures = failures_with_patch - clean_tree_failures
    800799        new_failures_string = ', '.join([failure_name.replace('TestWebKitAPI.', '') for failure_name in new_failures])
     800
     801        self._addToLog('stderr', '\nFailures in API Test first run: {}'.format(first_run_failures))
     802        self._addToLog('stderr', '\nFailures in API Test second run: {}'.format(second_run_failures))
     803        self._addToLog('stderr', '\nFlaky Tests: {}'.format(flaky_failures))
     804        self._addToLog('stderr', '\nFailures in API Test on clean tree: {}'.format(clean_tree_failures))
    801805
    802806        if new_failures:
     
    813817            self.descriptionDone = 'Passed API tests'
    814818            message = 'Found {} pre-existing API tests failures'.format(len(clean_tree_failures))
     819            if flaky_failures:
     820                message += '. Flaky tests: {}'.format(flaky_failures_string)
    815821            self.build.buildFinished([message], SUCCESS)
    816822
  • trunk/Tools/ChangeLog

    r244299 r244306  
     12019-04-15  Aakash Jain  <aakash_jain@apple.com>
     2
     3        ews-build should clearly indicate flaky test failures
     4        https://bugs.webkit.org/show_bug.cgi?id=196947
     5
     6        Reviewed by Lucas Forschler.
     7
     8        * BuildSlaveSupport/ews-build/steps.py:
     9        (AnalyzeAPITestsResults.analyzeResults):
     10
    1112019-04-15  Brent Fulgham  <bfulgham@apple.com>
    212
Note: See TracChangeset for help on using the changeset viewer.