Changeset 85960 in webkit


Ignore:
Timestamp:
May 6, 2011 11:35:21 AM (13 years ago)
Author:
abarth@webkit.org
Message:

2011-05-06 Adam Barth <abarth@webkit.org>

Reviewed by Eric Seidel.

The results.zip generated by the Chromium EWS are too big
https://bugs.webkit.org/show_bug.cgi?id=60352

Currently results.zip created by the Chromium EWS bot contain roughly
1000 tests that we expect to fail. These test results are not helpful
to contributors.

There are several approaches to solving this problem:

1) We could be more selective about which files to include in the zip.
2) We could ask new-run-webkit-tests not to generate results for

expected failures.

3) We could ask new-run-webkit-tests not to run tests that we expect to

fail.

This patch takes approach (3). Currently, we're ignoring the outcome
of failing tests anyway, so there's not much point in actually running
them. As a side benefit, this will speed up the test run by about 6%.
In the future, we might revisit this decision and choose another
approach.

  • Scripts/webkitpy/common/config/ports.py:
  • Scripts/webkitpy/common/config/ports_unittest.py:
Location:
trunk/Tools
Files:
3 edited

Legend:

Unmodified
Added
Removed
  • trunk/Tools/ChangeLog

    r85942 r85960  
     12011-05-06  Adam Barth  <abarth@webkit.org>
     2
     3        Reviewed by Eric Seidel.
     4
     5        The results.zip generated by the Chromium EWS are too big
     6        https://bugs.webkit.org/show_bug.cgi?id=60352
     7
     8        Currently results.zip created by the Chromium EWS bot contain roughly
     9        1000 tests that we expect to fail.  These test results are not helpful
     10        to contributors.
     11
     12        There are several approaches to solving this problem:
     13
     14        1) We could be more selective about which files to include in the zip.
     15        2) We could ask new-run-webkit-tests not to generate results for
     16           expected failures.
     17        3) We could ask new-run-webkit-tests not to run tests that we expect to
     18           fail.
     19
     20        This patch takes approach (3).  Currently, we're ignoring the outcome
     21        of failing tests anyway, so there's not much point in actually running
     22        them.  As a side benefit, this will speed up the test run by about 6%.
     23        In the future, we might revisit this decision and choose another
     24        approach.
     25
     26        * Scripts/webkitpy/common/config/ports.py:
     27        * Scripts/webkitpy/common/config/ports_unittest.py:
     28
    1292011-05-06  Gabor Rapcsanyi  <rgabor@webkit.org>
    230
  • trunk/Tools/Scripts/webkitpy/common/config/ports.py

    r85786 r85960  
    267267    @classmethod
    268268    def run_webkit_tests_command(cls):
    269         # FIXME: We should find a better way to do this.
     269        # FIXME: We should find a better way to do this. Some of these options
     270        # are specific to new-run-webkit-tests and some of them are due to
     271        # running in non-interactive mode.
    270272        return ["xvfb-run"] + ChromiumPort.run_webkit_tests_command() + [
    271273            "--results-directory=%s" % cls.results_directory,
     274            "--skip-failing-tests",
    272275            "--print=actual,config,expected,misc,slowest,unexpected,unexpected-results",
    273276        ]
  • trunk/Tools/Scripts/webkitpy/common/config/ports_unittest.py

    r85786 r85960  
    7171
    7272    def test_chromium_xvfb_port(self):
    73         self.assertEquals(ChromiumXVFBPort.run_webkit_tests_command(), ['xvfb-run', 'Tools/Scripts/new-run-webkit-tests', '--chromium', '--no-pixel-tests', '--results-directory=/tmp/layout-test-results', '--print=actual,config,expected,misc,slowest,unexpected,unexpected-results'])
     73        self.assertEquals(ChromiumXVFBPort.run_webkit_tests_command(), ['xvfb-run', 'Tools/Scripts/new-run-webkit-tests', '--chromium', '--no-pixel-tests', '--results-directory=/tmp/layout-test-results', '--skip-failing-tests', '--print=actual,config,expected,misc,slowest,unexpected,unexpected-results'])
    7474
    7575if __name__ == '__main__':
Note: See TracChangeset for help on using the changeset viewer.