Changeset 252411 in webkit

Timestamp:

Nov 13, 2019, 8:21:56 AM (7 years ago)

Author:

aakash_jain@apple.com

Message:

Make webkitperl EWS robust against flakiness in webkitperl tests
​https://bugs.webkit.org/show_bug.cgi?id=204155

Reviewed by Jonathan Bedard.

• BuildSlaveSupport/ews-build/steps.py:

(RunWebKitPerlTests): Set haltOnFailure and flunkOnFailure to false.
(RunWebKitPerlTests.evaluateCommand): Add re-run step in case of failure.
(ReRunWebKitPerlTests):
(ReRunWebKitPerlTests.evaluateCommand): Overrided this to avoid running this step again.

• BuildSlaveSupport/ews-build/steps_unittest.py:

(TestReRunJavaScriptCoreTests): Added unit-tests.

Location:

trunk/Tools

Files:

• BuildSlaveSupport/ews-build/steps.py (modified) (2 diffs)
• BuildSlaveSupport/ews-build/steps_unittest.py (modified) (4 diffs)
• ChangeLog (modified) (1 diff)

trunk/Tools/BuildSlaveSupport/ews-build/steps.py

@@ -619 +619 @@
     description = ['webkitperl-tests running']
     descriptionDone = ['webkitperl-tests']
-    flunkOnFailure = True
+    flunkOnFailure = False
+    haltOnFailure = False
     command = ['perl', 'Tools/Scripts/test-webkitperl']
 
@@ -631 +632 @@
             return {u'step': unicode(message)}
         return {u'step': u'Failed webkitperl tests'}
+
+    def evaluateCommand(self, cmd):
+        rc = shell.ShellCommand.evaluateCommand(self, cmd)
+        if rc == FAILURE:
+            self.build.addStepsAfterCurrentStep([ReRunWebKitPerlTests()])
+        return rc
+
+
+class ReRunWebKitPerlTests(RunWebKitPerlTests):
+    name = 're-run-webkitperl-tests'
+    flunkOnFailure = True
+    haltOnFailure = True
+
+    def evaluateCommand(self, cmd):
+        return shell.ShellCommand.evaluateCommand(self, cmd)
 
 

trunk/Tools/BuildSlaveSupport/ews-build/steps_unittest.py

@@ -39 +39 @@
                    CompileJSC, CompileJSCToT, CompileWebKit, CompileWebKitToT, ConfigureBuild,
                    DownloadBuiltProduct, DownloadBuiltProductFromMaster, ExtractBuiltProduct, ExtractTestResults, InstallGtkDependencies, InstallWpeDependencies, KillOldProcesses,
-                   PrintConfiguration, ReRunAPITests, ReRunJavaScriptCoreTests, ReRunWebKitTests, RunAPITests, RunAPITestsWithoutPatch,
+                   PrintConfiguration, ReRunAPITests, ReRunJavaScriptCoreTests, ReRunWebKitPerlTests, ReRunWebKitTests, RunAPITests, RunAPITestsWithoutPatch,
                    RunBindingsTests, RunBuildWebKitOrgUnitTests, RunEWSBuildbotCheckConfig, RunEWSUnitTests, RunJavaScriptCoreTests, RunJSCTestsWithoutPatch, RunWebKit1Tests,
                    RunWebKitPerlTests, RunWebKitPyTests, RunWebKitTests, RunWebKitTestsWithoutPatch, TestWithFailureCount, Trigger, TransferToS3, UnApplyPatchIfRequired,
@@ -381 +381 @@
         return self.tearDownBuildStep()
 
-    def test_success(self):
+    def configureStep(self):
         self.setupStep(RunWebKitPerlTests())
+
+    def test_success(self):
+        self.configureStep()
         self.expectRemoteCommands(
             ExpectShell(workdir='wkdir',
@@ -395 +398 @@
 
     def test_failure(self):
-        self.setupStep(RunWebKitPerlTests())
+        self.configureStep()
         self.expectRemoteCommands(
             ExpectShell(workdir='wkdir',
@@ -410 +413 @@
         self.expectOutcome(result=FAILURE, state_string='Failed webkitperl tests')
         return self.runStep()
+
+
+class TestReRunJavaScriptCoreTests(TestRunWebKitPerlTests):
+    def configureStep(self):
+        self.setupStep(ReRunWebKitPerlTests())
 
 

trunk/Tools/ChangeLog

@@ +1 @@
+2019-11-13  Aakash Jain  <aakash_jain@apple.com>
+
+        Make webkitperl EWS robust against flakiness in webkitperl tests
+        https://bugs.webkit.org/show_bug.cgi?id=204155
+
+        Reviewed by Jonathan Bedard.
+
+        * BuildSlaveSupport/ews-build/steps.py:
+        (RunWebKitPerlTests): Set haltOnFailure and flunkOnFailure to false.
+        (RunWebKitPerlTests.evaluateCommand): Add re-run step in case of failure.
+        (ReRunWebKitPerlTests):
+        (ReRunWebKitPerlTests.evaluateCommand): Overrided this to avoid running this step again.
+        * BuildSlaveSupport/ews-build/steps_unittest.py:
+        (TestReRunJavaScriptCoreTests): Added unit-tests.
+
 2019-11-13  Aakash Jain  <aakash_jain@apple.com>