Changeset 281297 in webkit

Timestamp:

Aug 20, 2021, 12:50:20 AM (4 years ago)

Author:

youenn@apple.com

Message:

Remove all WTR output before Content-Type:text/plain for WebRTC tests in case of known excessive logging
​https://bugs.webkit.org/show_bug.cgi?id=229188

Reviewed by Jonathan Bedard.

Some logging beneath WebKit is making WebRTC tests flaky.
While we work on the cause of this bug, we can still beef-up our test output clean up steps to handle this logging.
Instead of stripping lines based on a known pattern, which does not always work as per current results,
we can simply remove all logging before the 'Content-Type:text/plain\n' string that WebKitTestRunner is adding before
printing the text generated by the test itself.
We update webkitpy scripts accordingly and add this new clean-up for:

• tests that have webrtc in their name
• tests whose output contain some known logging strings like 'LRP' or 'Negotiation String'.

This should hopefully be more robust while not hiding real regressions.

• Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:

(SingleTestRunner):

• Scripts/webkitpy/port/base.py:

(Port.logging_detectors_to_strip_text_start):

• Scripts/webkitpy/port/driver.py:

(DriverOutput.strip_text_start_if_needed):

• Scripts/webkitpy/port/mac.py:

(MacPort.logging_patterns_to_strip):
(MacPort.logging_detectors_to_strip_text_start):

Location:

trunk/Tools

Files:

• ChangeLog (modified) (1 diff)
• Scripts/webkitpy/layout_tests/controllers/single_test_runner.py (modified) (1 diff)
• Scripts/webkitpy/port/base.py (modified) (1 diff)
• Scripts/webkitpy/port/driver.py (modified) (1 diff)
• Scripts/webkitpy/port/mac.py (modified) (1 diff)

trunk/Tools/ChangeLog

@@ +1 @@
+2021-08-20  Youenn Fablet  <youenn@apple.com>
+
+        Remove all WTR output before Content-Type:text/plain for WebRTC tests in case of known excessive logging
+        https://bugs.webkit.org/show_bug.cgi?id=229188
+
+        Reviewed by Jonathan Bedard.
+
+        Some logging beneath WebKit is making WebRTC tests flaky.
+        While we work on the cause of this bug, we can still beef-up our test output clean up steps to handle this logging.
+        Instead of stripping lines based on a known pattern, which does not always work as per current results,
+        we can simply remove all logging before the 'Content-Type:text/plain\n' string that WebKitTestRunner is adding before
+        printing the text generated by the test itself.
+        We update webkitpy scripts accordingly and add this new clean-up for:
+        - tests that have webrtc in their name
+        - tests whose output contain some known logging strings like 'LRP' or 'Negotiation String'.
+        This should hopefully be more robust while not hiding real regressions.
+
+        * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
+        (SingleTestRunner):
+        * Scripts/webkitpy/port/base.py:
+        (Port.logging_detectors_to_strip_text_start):
+        * Scripts/webkitpy/port/driver.py:
+        (DriverOutput.strip_text_start_if_needed):
+        * Scripts/webkitpy/port/mac.py:
+        (MacPort.logging_patterns_to_strip):
+        (MacPort.logging_detectors_to_strip_text_start):
+
 2021-08-19  Carlos Alberto Lopez Perez  <clopez@igalia.com>
 

trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py

@@ -133 +133 @@
         driver_output.strip_patterns(patterns)
 
+        driver_output.strip_text_start_if_needed(self._port.logging_detectors_to_strip_text_start(self._driver_input().test_name))
+
         driver_output.strip_stderror_patterns(self._port.stderr_patterns_to_strip())
 

trunk/Tools/Scripts/webkitpy/port/base.py

@@ -1421 +1421 @@
         return []
 
+    def logging_detectors_to_strip_text_start(self, test_name):
+        return []
+
     def test_expectations_file_position(self):
         # By default baseline search path schema is i.e. port-wk2 -> wk2 -> port -> generic, so port expectations file is at second to last position.

trunk/Tools/Scripts/webkitpy/port/driver.py

@@ -115 +115 @@
             self.text = re.sub(pattern[0], pattern[1], self.text)
 
+    def strip_text_start_if_needed(self, detectors):
+        if not self.text or not len(detectors):
+            return
+
+        result = self.text.split('Content-Type: text/plain\n')
+        if len(result) != 2:
+            return
+
+        for detector in detectors:
+            if detector in result[0]:
+                self.text = result[1]
+                self.error += '\nRemoved logging from stdout:\n' + result[0]
+                return
+
     def strip_stderror_patterns(self, patterns):
         if not self.error:

trunk/Tools/Scripts/webkitpy/port/mac.py

@@ -291 +291 @@
         logging_patterns.append((re.compile('VP9 Info:.*\n'), ''))
 
-        # FIXME: Replace this with a defaults write after bots update to a build that has rdar://problem/74031400 fixed.
-        logging_patterns.append((r'Negotiation String:.*\n', ''))
-        logging_patterns.append((r'LRP.*\n', ''))
-        logging_patterns.append((r'.* \(seconds\) ', ''))
-        logging_patterns.append((r'.* 0  0 \[\]\n', ''))
-        logging_patterns.append((r'\[\]\n', ''))
-
         return logging_patterns
+
+    def logging_detectors_to_strip_text_start(self, test_name):
+        logging_detectors = []
+
+        if 'webrtc' in test_name:
+            logging_detectors.append('Negotiation String:')
+            logging_detectors.append('LRP')
+
+        return logging_detectors
 
     def stderr_patterns_to_strip(self):