Changeset 176830 in webkit


Ignore:
Timestamp:
Dec 4, 2014 5:37:02 PM (9 years ago)
Author:
ap@apple.com
Message:

Run http tests parallel
​https://bugs.webkit.org/show_bug.cgi?id=138958

Reviewed by Daniel Bates.

Remove the concept of "locked shard". Now http tests are just like any other tests.
We start HTTP and WebSocket servers at the start if we need them, and terminate them
when done with all the tests (not when the last http test runs, which is unnecessarily
unpredictable).

This makes debug tests run in 8 minutes and 12 seconds on my Mac Pro. Without the
patch, they used to take over 15 minutes.

As part of the fix, we no longer pass the number of servers to Apache. I don't
think that these parameters did what we wanted them to do; Apache handles the load
just fine without them.

The change applies to all platforms. I fixed everything I could find on Mac, and
Ossy told me that he's been running http tests in parallel for a long time. If
there is increased instability for some ports, it will need to be fixed - there is
generally nothing special about http tests at this point, and most code is
cross-platform in WebKit2.

  • Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py:

(LayoutTestRunner.init):
(LayoutTestRunner.run_tests):
(LayoutTestRunner.start_servers_with_lock):
(LayoutTestRunner._handle_started_test):
(Worker.handle):
(Sharder.init):
(Sharder.shard_tests):
(Sharder._shard_every_file):
(Sharder._shard_by_directory):
(LayoutTestRunner._handle_finished_test_list): Deleted.
(LayoutTestRunner._handle_finished_test_list.find): Deleted.
(Sharder._shard_in_two): Deleted.
(Sharder): Deleted.
(Sharder._resize_shards): Deleted.
(Sharder._resize_shards.divide_and_round_up): Deleted.
(Sharder._resize_shards.extract_and_flatten): Deleted.
(Sharder._resize_shards.split_at): Deleted.

  • Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:

(FakePrinter.print_workers_and_shards):
(LayoutTestRunnerTests.test_servers_started.start_http_server):
(LayoutTestRunnerTests.test_servers_started):
(SharderTests.get_shards):
(SharderTests.test_shard_by_dir):
(SharderTests.test_shard_every_file):
(SharderTests): Deleted.
(SharderTests.test_shard_in_two): Deleted.
(SharderTests.test_shard_in_two_has_no_locked_shards): Deleted.
(SharderTests.test_shard_in_two_has_no_unlocked_shards): Deleted.
(SharderTests.test_multiple_locked_shards): Deleted.

  • Scripts/webkitpy/layout_tests/controllers/manager_unittest.py:

(ManagerTest.test_needs_servers.get_manager):
(ManagerTest.integration_test_needs_servers.get_manager):
(ManagerTest.test_look_for_new_crash_logs.get_manager):
(ManagerTest):

  • Scripts/webkitpy/layout_tests/run_webkit_tests.py:

(_set_up_derived_options):

  • Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:

(RunTest.test_batch_size):
(RunTest.test_max_locked_shards): Deleted.

  • Scripts/webkitpy/layout_tests/servers/apache_http_server.py:

(LayoutTestApacheHttpd.init):

  • Scripts/webkitpy/layout_tests/servers/apache_http_server_unittest.py:

(TestLayoutTestApacheHttpd.test_start_cmd):

  • Scripts/webkitpy/layout_tests/servers/http_server.py:

(Lighttpd.init):

  • Scripts/webkitpy/layout_tests/servers/http_server_base.py:

(HttpServerBase.init):

  • Scripts/webkitpy/layout_tests/views/printing.py:

(Printer.print_workers_and_shards):

  • Scripts/webkitpy/port/base.py:

(Port.default_child_processes):
(Port.to.start_http_server):
(Port.default_max_locked_shards): Deleted.

  • Scripts/webkitpy/port/port_testcase.py:

(PortTestCase.make_port):
(PortTestCase.test_default_max_locked_shards): Deleted.

  • Scripts/webkitpy/port/test.py:

(TestPort.start_http_server):

Location:
trunk/Tools
Files:
14 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r176814 r176830  
     12014-12-04  Alexey Proskuryakov  <ap@apple.com>
     2
     3        Run http tests parallel
     4        https://bugs.webkit.org/show_bug.cgi?id=138958
     5
     6        Reviewed by Daniel Bates.
     7
     8        Remove the concept of "locked shard". Now http tests are just like any other tests.
     9        We start HTTP and WebSocket servers at the start if we need them, and terminate them
     10        when done with all the tests (not when the last http test runs, which is unnecessarily
     11        unpredictable).
     12
     13        This makes debug tests run in 8 minutes and 12 seconds on my Mac Pro. Without the
     14        patch, they used to take over 15 minutes.
     15
     16        As part of the fix, we no longer pass the number of servers to Apache. I don't
     17        think that these parameters did what we wanted them to do; Apache handles the load
     18        just fine without them.
     19
     20        The change applies to all platforms. I fixed everything I could find on Mac, and
     21        Ossy told me that he's been running http tests in parallel for a long time. If
     22        there is increased instability for some ports, it will need to be fixed - there is
     23        generally nothing special about http tests at this point, and most code is
     24        cross-platform in WebKit2.
     25
     26        * Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py:
     27        (LayoutTestRunner.__init__):
     28        (LayoutTestRunner.run_tests):
     29        (LayoutTestRunner.start_servers_with_lock):
     30        (LayoutTestRunner._handle_started_test):
     31        (Worker.handle):
     32        (Sharder.__init__):
     33        (Sharder.shard_tests):
     34        (Sharder._shard_every_file):
     35        (Sharder._shard_by_directory):
     36        (LayoutTestRunner._handle_finished_test_list): Deleted.
     37        (LayoutTestRunner._handle_finished_test_list.find): Deleted.
     38        (Sharder._shard_in_two): Deleted.
     39        (Sharder): Deleted.
     40        (Sharder._resize_shards): Deleted.
     41        (Sharder._resize_shards.divide_and_round_up): Deleted.
     42        (Sharder._resize_shards.extract_and_flatten): Deleted.
     43        (Sharder._resize_shards.split_at): Deleted.
     44        * Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:
     45        (FakePrinter.print_workers_and_shards):
     46        (LayoutTestRunnerTests.test_servers_started.start_http_server):
     47        (LayoutTestRunnerTests.test_servers_started):
     48        (SharderTests.get_shards):
     49        (SharderTests.test_shard_by_dir):
     50        (SharderTests.test_shard_every_file):
     51        (SharderTests): Deleted.
     52        (SharderTests.test_shard_in_two): Deleted.
     53        (SharderTests.test_shard_in_two_has_no_locked_shards): Deleted.
     54        (SharderTests.test_shard_in_two_has_no_unlocked_shards): Deleted.
     55        (SharderTests.test_multiple_locked_shards): Deleted.
     56        * Scripts/webkitpy/layout_tests/controllers/manager_unittest.py:
     57        (ManagerTest.test_needs_servers.get_manager):
     58        (ManagerTest.integration_test_needs_servers.get_manager):
     59        (ManagerTest.test_look_for_new_crash_logs.get_manager):
     60        (ManagerTest):
     61        * Scripts/webkitpy/layout_tests/run_webkit_tests.py:
     62        (_set_up_derived_options):
     63        * Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
     64        (RunTest.test_batch_size):
     65        (RunTest.test_max_locked_shards): Deleted.
     66        * Scripts/webkitpy/layout_tests/servers/apache_http_server.py:
     67        (LayoutTestApacheHttpd.__init__):
     68        * Scripts/webkitpy/layout_tests/servers/apache_http_server_unittest.py:
     69        (TestLayoutTestApacheHttpd.test_start_cmd):
     70        * Scripts/webkitpy/layout_tests/servers/http_server.py:
     71        (Lighttpd.__init__):
     72        * Scripts/webkitpy/layout_tests/servers/http_server_base.py:
     73        (HttpServerBase.__init__):
     74        * Scripts/webkitpy/layout_tests/views/printing.py:
     75        (Printer.print_workers_and_shards):
     76        * Scripts/webkitpy/port/base.py:
     77        (Port.default_child_processes):
     78        (Port.to.start_http_server):
     79        (Port.default_max_locked_shards): Deleted.
     80        * Scripts/webkitpy/port/port_testcase.py:
     81        (PortTestCase.make_port):
     82        (PortTestCase.test_default_max_locked_shards): Deleted.
     83        * Scripts/webkitpy/port/test.py:
     84        (TestPort.start_http_server):
     85
    1862014-12-04  Alexey Proskuryakov  <ap@apple.com>
    287

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py

    r176814 r176830  
    6868        self._results_directory = results_directory
    6969        self._test_is_slow = test_is_slow_fn
    70         self._sharder = Sharder(self._port.split_test, self._options.max_locked_shards)
     70        self._sharder = Sharder(self._port.split_test)
    7171        self._filesystem = self._port.host.filesystem
    7272
    … …  
    7878
    7979        self._current_run_results = None
    80         self._remaining_locked_shards = []
    8180        self._has_http_lock = False
    8281
    … …  
    9190        run_results = TestRunResults(self._expectations, len(test_inputs) + len(tests_to_skip))
    9291        self._current_run_results = run_results
    93         self._remaining_locked_shards = []
    9492        self._has_http_lock = False
    9593        self._printer.num_tests = len(test_inputs)
    … …  
    105103
    106104        self._printer.write_update('Sharding tests ...')
    107         locked_shards, unlocked_shards = self._sharder.shard_tests(test_inputs, int(self._options.child_processes), self._options.fully_parallel)
    108 
    109         # FIXME: We don't have a good way to coordinate the workers so that
    110         # they don't try to run the shards that need a lock if we don't actually
    111         # have the lock. The easiest solution at the moment is to grab the
    112         # lock at the beginning of the run, and then run all of the locked
    113         # shards first. This minimizes the time spent holding the lock, but
    114         # means that we won't be running tests while we're waiting for the lock.
    115         # If this becomes a problem in practice we'll need to change this.
    116 
    117         all_shards = locked_shards + unlocked_shards
    118         self._remaining_locked_shards = locked_shards
    119         if locked_shards and self._options.http:
    120             self.start_servers_with_lock(2 * min(num_workers, len(locked_shards)))
     105        all_shards = self._sharder.shard_tests(test_inputs, int(self._options.child_processes), self._options.fully_parallel)
     106
     107        if self._needs_http and self._options.http:
     108            self.start_servers_with_lock()
    121109
    122110        num_workers = min(num_workers, len(all_shards))
    123         self._printer.print_workers_and_shards(num_workers, len(all_shards), len(locked_shards))
     111        self._printer.print_workers_and_shards(num_workers, len(all_shards))
    124112
    125113        if self._options.dry_run:
    … …  
    198186        self._interrupt_if_at_failure_limits(run_results)
    199187
    200     def start_servers_with_lock(self, number_of_servers):
     188    def start_servers_with_lock(self):
    201189        self._printer.write_update('Acquiring http lock ...')
    202190        self._port.acquire_http_lock()
    203191        if self._needs_http:
    204192            self._printer.write_update('Starting HTTP server ...')
    205             self._port.start_http_server(number_of_servers=number_of_servers)
     193            self._port.start_http_server()
    206194        if self._needs_websockets:
    207195            self._printer.write_update('Starting WebSocket server ...')
    … …  
    230218        self._printer.print_started_test(test_input.test_name)
    231219
    232     def _handle_finished_test_list(self, worker_name, list_name):
    233         def find(name, test_lists):
    234             for i in range(len(test_lists)):
    235                 if test_lists[i].name == name:
    236                     return i
    237             return -1
    238 
    239         index = find(list_name, self._remaining_locked_shards)
    240         if index >= 0:
    241             self._remaining_locked_shards.pop(index)
    242             if not self._remaining_locked_shards:
    243                 self.stop_servers_with_lock()
    244 
    245220    def _handle_finished_test(self, worker_name, result, log_messages=[]):
    246221        self._update_summary_with_result(self._current_run_results, result)
    … …  
    282257        for test_input in test_inputs:
    283258            self._run_test(test_input, test_list_name)
    284         self._caller.post('finished_test_list', test_list_name)
    285259
    286260    def _update_test_input(self, test_input):
    … …  
    453427
    454428class Sharder(object):
    455     def __init__(self, test_split_fn, max_locked_shards):
     429    def __init__(self, test_split_fn):
    456430        self._split = test_split_fn
    457         self._max_locked_shards = max_locked_shards
    458431
    459432    def shard_tests(self, test_inputs, num_workers, fully_parallel):
    … …  
    463436        occur within the same directory.
    464437        Return:
    465             Two list of TestShards. The first contains tests that must only be
    466             run under the server lock, the second can be run whenever.
     438            A list of TestShards.
    467439        """
    468440
    … …  
    471443        # in prepare_lists as well.
    472444        if num_workers == 1:
    473             return self._shard_in_two(test_inputs)
     445            return [TestShard('all_tests', test_inputs)]
    474446        elif fully_parallel:
    475447            return self._shard_every_file(test_inputs)
    476448        return self._shard_by_directory(test_inputs, num_workers)
    477449
    478     def _shard_in_two(self, test_inputs):
    479         """Returns two lists of shards, one with all the tests requiring a lock and one with the rest.
    480 
    481         This is used when there's only one worker, to minimize the per-shard overhead."""
    482         locked_inputs = []
    483         unlocked_inputs = []
    484         for test_input in test_inputs:
    485             if test_input.needs_servers:
    486                 locked_inputs.append(test_input)
    487             else:
    488                 unlocked_inputs.append(test_input)
    489 
    490         locked_shards = []
    491         unlocked_shards = []
    492         if locked_inputs:
    493             locked_shards = [TestShard('locked_tests', locked_inputs)]
    494         if unlocked_inputs:
    495             unlocked_shards = [TestShard('unlocked_tests', unlocked_inputs)]
    496 
    497         return locked_shards, unlocked_shards
    498 
    499450    def _shard_every_file(self, test_inputs):
    500         """Returns two lists of shards, each shard containing a single test file.
     451        """Returns a list of shards, each shard containing a single test file.
    501452
    502453        This mode gets maximal parallelism at the cost of much higher flakiness."""
    503         locked_shards = []
    504         unlocked_shards = []
     454        shards = []
    505455        for test_input in test_inputs:
    506456            # Note that we use a '.' for the shard name; the name doesn't really
    507457            # matter, and the only other meaningful value would be the filename,
    508458            # which would be really redundant.
    509             if test_input.needs_servers:
    510                 locked_shards.append(TestShard('.', [test_input]))
    511             else:
    512                 unlocked_shards.append(TestShard('.', [test_input]))
    513 
    514         return locked_shards, unlocked_shards
     459            shards.append(TestShard('.', [test_input]))
     460
     461        return shards
    515462
    516463    def _shard_by_directory(self, test_inputs, num_workers):
    517         """Returns two lists of shards, each shard containing all the files in a directory.
     464        """Returns a lists of shards, each shard containing all the files in a directory.
    518465
    519466        This is the default mode, and gets as much parallelism as we can while
    520467        minimizing flakiness caused by inter-test dependencies."""
    521         locked_shards = []
    522         unlocked_shards = []
     468        shards = []
    523469        tests_by_dir = {}
    524470        # FIXME: Given that the tests are already sorted by directory,
    … …  
    531477        for directory, test_inputs in tests_by_dir.iteritems():
    532478            shard = TestShard(directory, test_inputs)
    533             if test_inputs[0].needs_servers:
    534                 locked_shards.append(shard)
    535             else:
    536                 unlocked_shards.append(shard)
     479            shards.append(shard)
    537480
    538481        # Sort the shards by directory name.
    539         locked_shards.sort(key=lambda shard: shard.name)
    540         unlocked_shards.sort(key=lambda shard: shard.name)
    541 
    542         # Put a ceiling on the number of locked shards, so that we
    543         # don't hammer the servers too badly.
    544 
    545         # FIXME: For now, limit to one shard or set it
    546         # with the --max-locked-shards. After testing to make sure we
    547         # can handle multiple shards, we should probably do something like
    548         # limit this to no more than a quarter of all workers, e.g.:
    549         # return max(math.ceil(num_workers / 4.0), 1)
    550         return (self._resize_shards(locked_shards, self._max_locked_shards, 'locked_shard'),
    551                 unlocked_shards)
    552 
    553     def _resize_shards(self, old_shards, max_new_shards, shard_name_prefix):
    554         """Takes a list of shards and redistributes the tests into no more
    555         than |max_new_shards| new shards."""
    556 
    557         # This implementation assumes that each input shard only contains tests from a
    558         # single directory, and that tests in each shard must remain together; as a
    559         # result, a given input shard is never split between output shards.
    560         #
    561         # Each output shard contains the tests from one or more input shards and
    562         # hence may contain tests from multiple directories.
    563 
    564         def divide_and_round_up(numerator, divisor):
    565             return int(math.ceil(float(numerator) / divisor))
    566 
    567         def extract_and_flatten(shards):
    568             test_inputs = []
    569             for shard in shards:
    570                 test_inputs.extend(shard.test_inputs)
    571             return test_inputs
    572 
    573         def split_at(seq, index):
    574             return (seq[:index], seq[index:])
    575 
    576         num_old_per_new = divide_and_round_up(len(old_shards), max_new_shards)
    577         new_shards = []
    578         remaining_shards = old_shards
    579         while remaining_shards:
    580             some_shards, remaining_shards = split_at(remaining_shards, num_old_per_new)
    581             new_shards.append(TestShard('%s_%d' % (shard_name_prefix, len(new_shards) + 1), extract_and_flatten(some_shards)))
    582         return new_shards
     482        shards.sort(key=lambda shard: shard.name)
     483
     484        return shards

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py

    r176814 r176830  
    5252        pass
    5353
    54     def print_workers_and_shards(self, num_workers, num_shards, num_locked_shards):
     54    def print_workers_and_shards(self, num_workers, num_shards):
    5555        pass
    5656
    … …  
    167167    def test_servers_started(self):
    168168
    169         def start_http_server(number_of_servers=None):
     169        def start_http_server():
    170170            self.http_started = True
    171171
    … …  
    190190        runner._needs_http = True
    191191        runner._needs_websockets = False
    192         runner.start_servers_with_lock(number_of_servers=4)
     192        runner.start_servers_with_lock()
    193193        self.assertEqual(self.http_started, True)
    194194        self.assertEqual(self.websocket_started, False)
    … …  
    200200        runner._needs_http = True
    201201        runner._needs_websockets = True
    202         runner.start_servers_with_lock(number_of_servers=4)
     202        runner.start_servers_with_lock()
    203203        self.assertEqual(self.http_started, True)
    204204        self.assertEqual(self.websocket_started, True)
    … …  
    210210        runner._needs_http = False
    211211        runner._needs_websockets = False
    212         runner.start_servers_with_lock(number_of_servers=4)
     212        runner.start_servers_with_lock()
    213213        self.assertEqual(self.http_started, False)
    214214        self.assertEqual(self.websocket_started, False)
    … …  
    235235        return TestInput(test_file, needs_servers=(test_file.startswith('http')))
    236236
    237     def get_shards(self, num_workers, fully_parallel, test_list=None, max_locked_shards=1):
     237    def get_shards(self, num_workers, fully_parallel, test_list=None):
    238238        port = TestPort(MockSystemHost())
    239         self.sharder = Sharder(port.split_test, max_locked_shards)
     239        self.sharder = Sharder(port.split_test)
    240240        test_list = test_list or self.test_list
    241241        return self.sharder.shard_tests([self.get_test_input(test) for test in test_list], num_workers, fully_parallel)
    … …  
    250250
    251251    def test_shard_by_dir(self):
    252         locked, unlocked = self.get_shards(num_workers=2, fully_parallel=False)
    253 
    254         # Note that although there are tests in multiple dirs that need locks,
    255         # they are crammed into a single shard in order to reduce the # of
    256         # workers hitting the server at once.
    257         self.assert_shards(locked,
    258              [('locked_shard_1',
    259                ['http/tests/security/view-source-no-refresh.html',
    260                 'http/tests/websocket/tests/unicode.htm',
    261                 'http/tests/websocket/tests/websocket-protocol-ignored.html',
    262                 'http/tests/xmlhttprequest/supported-xml-content-types.html'])])
    263         self.assert_shards(unlocked,
     252        result = self.get_shards(num_workers=2, fully_parallel=False)
     253
     254        self.assert_shards(result,
    264255            [('animations', ['animations/keyframes.html']),
    265256             ('dom/html/level2/html', ['dom/html/level2/html/HTMLAnchorElement03.html',
    266257                                      'dom/html/level2/html/HTMLAnchorElement06.html']),
    267258             ('fast/css', ['fast/css/display-none-inline-style-change-crash.html']),
     259             ('http/tests/security', ['http/tests/security/view-source-no-refresh.html']),
     260             ('http/tests/websocket/tests', ['http/tests/websocket/tests/unicode.htm', 'http/tests/websocket/tests/websocket-protocol-ignored.html']),
     261             ('http/tests/xmlhttprequest', ['http/tests/xmlhttprequest/supported-xml-content-types.html']),
    268262             ('ietestcenter/Javascript', ['ietestcenter/Javascript/11.1.5_4-4-c-1.html'])])
    269263
    270264    def test_shard_every_file(self):
    271         locked, unlocked = self.get_shards(num_workers=2, fully_parallel=True)
    272         self.assert_shards(locked,
     265        result = self.get_shards(num_workers=2, fully_parallel=True)
     266        self.assert_shards(result,
    273267            [('.', ['http/tests/websocket/tests/unicode.htm']),
     268             ('.', ['animations/keyframes.html']),
    274269             ('.', ['http/tests/security/view-source-no-refresh.html']),
    275270             ('.', ['http/tests/websocket/tests/websocket-protocol-ignored.html']),
    276              ('.', ['http/tests/xmlhttprequest/supported-xml-content-types.html'])]),
    277         self.assert_shards(unlocked,
    278             [('.', ['animations/keyframes.html']),
    279271             ('.', ['fast/css/display-none-inline-style-change-crash.html']),
     272             ('.', ['http/tests/xmlhttprequest/supported-xml-content-types.html']),
    280273             ('.', ['dom/html/level2/html/HTMLAnchorElement03.html']),
    281274             ('.', ['ietestcenter/Javascript/11.1.5_4-4-c-1.html']),
    282275             ('.', ['dom/html/level2/html/HTMLAnchorElement06.html'])])
    283 
    284     def test_shard_in_two(self):
    285         locked, unlocked = self.get_shards(num_workers=1, fully_parallel=False)
    286         self.assert_shards(locked,
    287             [('locked_tests',
    288               ['http/tests/websocket/tests/unicode.htm',
    289                'http/tests/security/view-source-no-refresh.html',
    290                'http/tests/websocket/tests/websocket-protocol-ignored.html',
    291                'http/tests/xmlhttprequest/supported-xml-content-types.html'])])
    292         self.assert_shards(unlocked,
    293             [('unlocked_tests',
    294               ['animations/keyframes.html',
    295                'fast/css/display-none-inline-style-change-crash.html',
    296                'dom/html/level2/html/HTMLAnchorElement03.html',
    297                'ietestcenter/Javascript/11.1.5_4-4-c-1.html',
    298                'dom/html/level2/html/HTMLAnchorElement06.html'])])
    299 
    300     def test_shard_in_two_has_no_locked_shards(self):
    301         locked, unlocked = self.get_shards(num_workers=1, fully_parallel=False,
    302              test_list=['animations/keyframe.html'])
    303         self.assertEqual(len(locked), 0)
    304         self.assertEqual(len(unlocked), 1)
    305 
    306     def test_shard_in_two_has_no_unlocked_shards(self):
    307         locked, unlocked = self.get_shards(num_workers=1, fully_parallel=False,
    308              test_list=['http/tests/websocket/tests/unicode.htm'])
    309         self.assertEqual(len(locked), 1)
    310         self.assertEqual(len(unlocked), 0)
    311 
    312     def test_multiple_locked_shards(self):
    313         locked, unlocked = self.get_shards(num_workers=4, fully_parallel=False, max_locked_shards=2)
    314         self.assert_shards(locked,
    315             [('locked_shard_1',
    316               ['http/tests/security/view-source-no-refresh.html',
    317                'http/tests/websocket/tests/unicode.htm',
    318                'http/tests/websocket/tests/websocket-protocol-ignored.html']),
    319              ('locked_shard_2',
    320               ['http/tests/xmlhttprequest/supported-xml-content-types.html'])])
    321 
    322         locked, unlocked = self.get_shards(num_workers=4, fully_parallel=False)
    323         self.assert_shards(locked,
    324             [('locked_shard_1',
    325               ['http/tests/security/view-source-no-refresh.html',
    326                'http/tests/websocket/tests/unicode.htm',
    327                'http/tests/websocket/tests/websocket-protocol-ignored.html',
    328                'http/tests/xmlhttprequest/supported-xml-content-types.html'])])

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/manager_unittest.py

    r174136 r176830  
    4747            port = Mock()  # FIXME: Use a tighter mock.
    4848            port.TEST_PATH_SEPARATOR = '/'
    49             manager = Manager(port, options=MockOptions(http=True, max_locked_shards=1), printer=Mock())
     49            manager = Manager(port, options=MockOptions(http=True), printer=Mock())
    5050            return manager
    5151
    … …  
    6060            host = MockHost()
    6161            port = host.port_factory.get()
    62             manager = Manager(port, options=MockOptions(test_list=None, http=True, max_locked_shards=1), printer=Mock())
     62            manager = Manager(port, options=MockOptions(test_list=None, http=True), printer=Mock())
    6363            return manager
    6464
    … …  
    8080            host = MockHost()
    8181            port = host.port_factory.get('test-mac-leopard')
    82             manager = Manager(port, options=MockOptions(test_list=None, http=True, max_locked_shards=1), printer=Mock())
     82            manager = Manager(port, options=MockOptions(test_list=None, http=True), printer=Mock())
    8383            return manager
    8484        host = MockHost()

  • trunk/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py

    r173937 r176830  
    332332        options.child_processes = os.environ.get("WEBKIT_TEST_CHILD_PROCESSES",
    333333                                                 str(port.default_child_processes()))
    334     if not options.max_locked_shards:
    335         options.max_locked_shards = int(os.environ.get("WEBKIT_TEST_MAX_LOCKED_SHARDS",
    336                                                        str(port.default_max_locked_shards())))
    337334
    338335    if not options.configuration:

  • trunk/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py

    r176814 r176830  
    225225            self.assertTrue(len(batch) <= 2, '%s had too many tests' % ', '.join(batch))
    226226
    227     def test_max_locked_shards(self):
    228         # Tests for the default of using one locked shard even in the case of more than one child process.
    229         if not self.should_test_processes:
    230             return
    231         save_env_webkit_test_max_locked_shards = None
    232         if "WEBKIT_TEST_MAX_LOCKED_SHARDS" in os.environ:
    233             save_env_webkit_test_max_locked_shards = os.environ["WEBKIT_TEST_MAX_LOCKED_SHARDS"]
    234             del os.environ["WEBKIT_TEST_MAX_LOCKED_SHARDS"]
    235         _, regular_output, _ = logging_run(['--debug-rwt-logging', '--child-processes', '2'], shared_port=False)
    236         try:
    237             self.assertTrue(any(['1 locked' in line for line in regular_output.buflist]))
    238         finally:
    239             if save_env_webkit_test_max_locked_shards:
    240                 os.environ["WEBKIT_TEST_MAX_LOCKED_SHARDS"] = save_env_webkit_test_max_locked_shards
    241 
    242227    def test_child_processes_2(self):
    243228        if self.should_test_processes:

  • trunk/Tools/Scripts/webkitpy/layout_tests/servers/apache_http_server.py

    r155411 r176830  
    4343
    4444class LayoutTestApacheHttpd(http_server_base.HttpServerBase):
    45     def __init__(self, port_obj, output_dir, additional_dirs=None, number_of_servers=None):
     45    def __init__(self, port_obj, output_dir, additional_dirs=None):
    4646        """Args:
    4747          port_obj: handle to the platform-specific routines
    4848          output_dir: the absolute path to the layout test result directory
    4949        """
    50         http_server_base.HttpServerBase.__init__(self, port_obj, number_of_servers)
     50        http_server_base.HttpServerBase.__init__(self, port_obj)
    5151        # We use the name "httpd" instead of "apache" to make our paths (e.g. the pid file: /tmp/WebKit/httpd.pid)
    5252        # match old-run-webkit-tests: https://bugs.webkit.org/show_bug.cgi?id=63956
    … …  
    115115                        '-c', "\'RemoveHandler .cgi .pl\'",
    116116                        '-c', "\'</Location>\'"]
    117 
    118         if self._number_of_servers:
    119             start_cmd += ['-c', "\'StartServers %d\'" % self._number_of_servers,
    120                           '-c', "\'MinSpareServers %d\'" % self._number_of_servers,
    121                           '-c', "\'MaxSpareServers %d\'" % self._number_of_servers]
    122117
    123118        stop_cmd = [executable,

  • trunk/Tools/Scripts/webkitpy/layout_tests/servers/apache_http_server_unittest.py

    r174136 r176830  
    5454        host.filesystem.write_text_file(test_port._path_to_apache_config_file(), '')
    5555
    56         server = LayoutTestApacheHttpd(test_port, "/mock/output_dir", number_of_servers=4)
     56        server = LayoutTestApacheHttpd(test_port, "/mock/output_dir")
    5757        server._check_that_all_ports_are_available = lambda: True
    5858        server._is_server_running_on_all_ports = lambda: True
    … …  
    6565        finally:
    6666            _, _, logs = oc.restore_output()
    67         self.assertIn("StartServers 4", logs)
    68         self.assertIn("MinSpareServers 4", logs)
    69         self.assertIn("MaxSpareServers 4", logs)
    7067        self.assertTrue(host.filesystem.exists("/mock/output_dir/httpd.conf"))

  • trunk/Tools/Scripts/webkitpy/layout_tests/servers/http_server.py

    r155411 r176830  
    4343    def __init__(self, port_obj, output_dir, background=False, port=None,
    4444                 root=None, run_background=None, additional_dirs=None,
    45                  layout_tests_dir=None, number_of_servers=None):
     45                 layout_tests_dir=None):
    4646        """Args:
    4747          output_dir: the absolute path to the layout test result directory
    4848        """
    4949        # Webkit tests
    50         http_server_base.HttpServerBase.__init__(self, port_obj, number_of_servers)
     50        http_server_base.HttpServerBase.__init__(self, port_obj)
    5151        self._name = 'lighttpd'
    5252        self._output_dir = output_dir

  • trunk/Tools/Scripts/webkitpy/layout_tests/servers/http_server_base.py

    r145122 r176830  
    4747    """A skeleton class for starting and stopping servers used by the layout tests."""
    4848
    49     def __init__(self, port_obj, number_of_servers=None):
     49    def __init__(self, port_obj):
    5050        self._executive = port_obj._executive
    5151        self._filesystem = port_obj._filesystem
    … …  
    5555        self._pid_file = None
    5656        self._port_obj = port_obj
    57         self._number_of_servers = number_of_servers
    5857
    5958        # We need a non-checkout-dependent place to put lock files, etc. We

  • trunk/Tools/Scripts/webkitpy/layout_tests/views/printing.py

    r170637 r176830  
    111111        self._print_debug('')
    112112
    113     def print_workers_and_shards(self, num_workers, num_shards, num_locked_shards):
     113    def print_workers_and_shards(self, num_workers, num_shards):
    114114        driver_name = self._port.driver_name()
    115115        if num_workers == 1:
    … …  
    118118        else:
    119119            self._print_default("Running %s in parallel." % (grammar.pluralize(num_workers, driver_name)))
    120             self._print_debug("(%d shards; %d locked)." % (num_shards, num_locked_shards))
     120            self._print_debug("(%d shards)." % num_shards)
    121121        self._print_default('')
    122122

  • trunk/Tools/Scripts/webkitpy/port/base.py

    r176629 r176830  
    160160        return self._executive.cpu_count()
    161161
    162     def default_max_locked_shards(self):
    163         """Return the number of "locked" shards to run in parallel (like the http tests)."""
    164         return 1
    165 
    166162    def worker_startup_delay_secs(self):
    167163        # FIXME: If we start workers up too quickly, DumpRenderTree appears
    … …  
    878874        pass
    879875
    880     def start_http_server(self, additional_dirs=None, number_of_servers=None):
     876    def start_http_server(self, additional_dirs=None):
    881877        """Start a web server. Raise an error if it can't start or is already running.
    882878
    … …  
    885881
    886882        if self._uses_apache():
    887             server = apache_http_server.LayoutTestApacheHttpd(self, self.results_directory(), additional_dirs=additional_dirs, number_of_servers=number_of_servers)
     883            server = apache_http_server.LayoutTestApacheHttpd(self, self.results_directory(), additional_dirs=additional_dirs)
    888884        else:
    889             server = http_server.Lighttpd(self, self.results_directory(), additional_dirs=additional_dirs, number_of_servers=number_of_servers)
     885            server = http_server.Lighttpd(self, self.results_directory(), additional_dirs=additional_dirs)
    890886
    891887        server.start()

  • trunk/Tools/Scripts/webkitpy/port/port_testcase.py

    r174136 r176830  
    9292        return port
    9393
    94     def test_default_max_locked_shards(self):
    95         port = self.make_port()
    96         port.default_child_processes = lambda: 16
    97         self.assertEqual(port.default_max_locked_shards(), 1)
    98         port.default_child_processes = lambda: 2
    99         self.assertEqual(port.default_max_locked_shards(), 1)
    100 
    10194    def test_default_timeout_ms(self):
    10295        self.assertEqual(self.make_port(options=MockOptions(configuration='Release')).default_timeout_ms(), 35000)

  • trunk/Tools/Scripts/webkitpy/port/test.py

    r175204 r176830  
    464464        return TestDriver
    465465
    466     def start_http_server(self, additional_dirs=None, number_of_servers=None):
     466    def start_http_server(self, additional_dirs=None):
    467467        pass
    468468
Note: See TracChangeset for help on using the changeset viewer.