Changeset 125954 in webkit

Timestamp:

Aug 17, 2012 5:25:44 PM (12 years ago)

Author:

dpranke@chromium.org

Message:

nrwt: don't fail tests if ImageDiff fails on the wk2 ports
​https://bugs.webkit.org/show_bug.cgi?id=94396

Reviewed by Ryosuke Niwa.

It appears that WTR is generating PNGs that ImageDiff doesn't
like, and so we're getting lots of "can not get ImageDiff"
errors (see bug 81962). The change in r124581 made this a test
failure, and this is causing the WK2 bots to frequently abort
with 500+ failures.

This change makes ImageDiff errors not be test failures just for
WK2 for now until we can better triage what's going on.

• Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:

(SingleTestRunner._compare_image):

Location:

trunk/Tools

Files:

• ChangeLog (modified) (1 diff)
• Scripts/webkitpy/layout_tests/controllers/single_test_runner.py (modified) (1 diff)

trunk/Tools/ChangeLog

@@ +1 @@
+2012-08-17  Dirk Pranke  <dpranke@chromium.org>
+
+        nrwt: don't fail tests if ImageDiff fails on the wk2 ports
+        https://bugs.webkit.org/show_bug.cgi?id=94396
+
+        Reviewed by Ryosuke Niwa.
+
+        It appears that WTR is generating PNGs that ImageDiff doesn't
+        like, and so we're getting lots of "can not get ImageDiff"
+        errors (see bug 81962). The change in r124581 made this a test
+        failure, and this is causing the WK2 bots to frequently abort
+        with 500+ failures. 
+
+        This change makes ImageDiff errors not be test failures just for
+        WK2 for now until we can better triage what's going on.
+
+        * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
+        (SingleTestRunner._compare_image):
+
 2012-08-17  James Robinson  <jamesr@chromium.org>
 

trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py

@@ -267 +267 @@
             diff_result = self._port.diff_image(driver_output.image, expected_driver_output.image)
             err_str = diff_result[2]
-            if err_str:
+            # FIXME: see https://bugs.webkit.org/show_bug.cgi?id=94277 and
+            # https://bugs.webkit.org/show_bug.cgi?id=81962; ImageDiff doesn't
+            # seem to be working with WTR properly and tons of tests are failing.
+            if err_str and not self._options.webkit_test_runner:
                 _log.warning('  %s : %s' % (self._test_name, err_str))
                 failures.append(test_failures.FailureImageHashMismatch())