Changeset 122234 in webkit


Ignore:
Timestamp:
Jul 10, 2012 10:33:41 AM (12 years ago)
Author:
ojan@chromium.org
Message:

Improve webkit-patch rebaseline to work for more cases
https://bugs.webkit.org/show_bug.cgi?id=90504

Reviewed by Adam Barth.

-Makes it work for the build.chromium.org bots.
-Lets you rebaseline all builders instead of just one.
-Lets you pass in the builders or tests to rebaseline.

  • Scripts/webkitpy/common/host.py:

(Host.buildbot_for_builder_name):

  • Scripts/webkitpy/common/net/buildbot/buildbot.py:

(Builder.init):
(Builder.latest_layout_test_results):
Provide a way to get to the LayoutTestResults of the latest build.
Most of the time we only need to get the latest one and the Chromium bots
only expose the full_results.json file for the latest build.

(Builder):
(Builder._fetch_file_from_results):
(Builder.fetch_layout_test_results):
Move these functions into Builder so that Builder and Build can both
fetch layout test results.

(Build.results_zip_url):
(Build.layout_test_results):

  • Scripts/webkitpy/common/net/buildbot/buildbot_unittest.py:

(BuilderTest.test_latest_layout_test_results):
(BuildTest.test_layout_test_results):

  • Scripts/webkitpy/common/system/user.py:

(User.prompt_with_multiple_lists):
Prompt with multiple sublists, but still have a global numbering.
This lets the build.chromium.org bots be clearly separated from the
build.webkit.org bots, which helps understand the builder names.

(User):
(User._wait_on_list_response):
(User.prompt_with_list):

  • Scripts/webkitpy/common/system/user_unittest.py:

(UserTest.test_prompt_with_multiple_lists):
(UserTest.test_prompt_with_multiple_lists.run_prompt_test):
(UserTest.test_prompt_with_multiple_lists.run_prompt_test.mock_raw_input):

  • Scripts/webkitpy/layout_tests/port/base.py:

(Port.is_chromium):
Provide a way to tell if a port is a Chromium port that doesn't involve string
manipulation on the port name or builder name.

  • Scripts/webkitpy/layout_tests/port/builders.py:

(builder_path_from_name):
(all_builder_names):
Memoizing here is incorrect because the test override _exact_matches,
so these can return different values. In either case, I'm pretty sure these
are not remotely hot codepaths. I manually inspected all call sites and they're
all in high-level calls (e.g. execute calls for webkit-patch commands) and not
called in a loop.

  • Scripts/webkitpy/layout_tests/port/chromium.py:

(ChromiumPort.is_chromium):

  • Scripts/webkitpy/tool/commands/rebaseline.py:

(RebaselineTest._results_url):
(AbstractParallelRebaselineCommand.init):
(Rebaseline.init):
(Rebaseline):
(Rebaseline._builders_to_pull_from):
(Rebaseline._tests_to_update):
(Rebaseline.execute):

  • Scripts/webkitpy/tool/commands/rebaseline_unittest.py:

(test_rebaseline.mock_builders_to_pull_from):
(test_rebaseline):
(test_rebaseline_command_line_flags):
(test_rebaseline_multiple_builders):
(test_rebaseline_multiple_builders.mock_builders_to_pull_from):
(test_rebaseline_multiple_builders.mock_tests_to_update):
(test_rebaseline_multiple_builders_and_tests_command_line):

  • Scripts/webkitpy/tool/mocktool.py:

(MockTool.irc):
(MockTool):
(MockTool.buildbot_for_builder_name):

Location:
trunk/Tools
Files:
13 edited

Legend:

Unmodified
Added
Removed
  • trunk/Tools/ChangeLog

    r122228 r122234  
     12012-07-09  Ojan Vafai  <ojan@chromium.org>
     2
     3        Improve webkit-patch rebaseline to work for more cases
     4        https://bugs.webkit.org/show_bug.cgi?id=90504
     5
     6        Reviewed by Adam Barth.
     7
     8        -Makes it work for the build.chromium.org bots.
     9        -Lets you rebaseline all builders instead of just one.
     10        -Lets you pass in the builders or tests to rebaseline.
     11
     12        * Scripts/webkitpy/common/host.py:
     13        (Host.buildbot_for_builder_name):
     14        * Scripts/webkitpy/common/net/buildbot/buildbot.py:
     15        (Builder.__init__):
     16        (Builder.latest_layout_test_results):
     17        Provide a way to get to the LayoutTestResults of the latest build.
     18        Most of the time we only need to get the latest one and the Chromium bots
     19        only expose the full_results.json file for the latest build.
     20
     21        (Builder):
     22        (Builder._fetch_file_from_results):
     23        (Builder.fetch_layout_test_results):
     24        Move these functions into Builder so that Builder and Build can both
     25        fetch layout test results.
     26
     27        (Build.results_zip_url):
     28        (Build.layout_test_results):
     29        * Scripts/webkitpy/common/net/buildbot/buildbot_unittest.py:
     30        (BuilderTest.test_latest_layout_test_results):
     31        (BuildTest.test_layout_test_results):
     32        * Scripts/webkitpy/common/system/user.py:
     33        (User.prompt_with_multiple_lists):
     34        Prompt with multiple sublists, but still have a global numbering.
     35        This lets the build.chromium.org bots be clearly separated from the
     36        build.webkit.org bots, which helps understand the builder names.
     37
     38        (User):
     39        (User._wait_on_list_response):
     40        (User.prompt_with_list):
     41        * Scripts/webkitpy/common/system/user_unittest.py:
     42        (UserTest.test_prompt_with_multiple_lists):
     43        (UserTest.test_prompt_with_multiple_lists.run_prompt_test):
     44        (UserTest.test_prompt_with_multiple_lists.run_prompt_test.mock_raw_input):
     45        * Scripts/webkitpy/layout_tests/port/base.py:
     46        (Port.is_chromium):
     47        Provide a way to tell if a port is a Chromium port that doesn't involve string
     48        manipulation on the port name or builder name.
     49
     50        * Scripts/webkitpy/layout_tests/port/builders.py:
     51        (builder_path_from_name):
     52        (all_builder_names):
     53        Memoizing here is incorrect because the test override _exact_matches,
     54        so these can return different values. In either case, I'm pretty sure these
     55        are not remotely hot codepaths. I manually inspected all call sites and they're
     56        all in high-level calls (e.g. execute calls for webkit-patch commands) and not
     57        called in a loop.
     58
     59        * Scripts/webkitpy/layout_tests/port/chromium.py:
     60        (ChromiumPort.is_chromium):
     61        * Scripts/webkitpy/tool/commands/rebaseline.py:
     62        (RebaselineTest._results_url):
     63        (AbstractParallelRebaselineCommand.__init__):
     64        (Rebaseline.__init__):
     65        (Rebaseline):
     66        (Rebaseline._builders_to_pull_from):
     67        (Rebaseline._tests_to_update):
     68        (Rebaseline.execute):
     69        * Scripts/webkitpy/tool/commands/rebaseline_unittest.py:
     70        (test_rebaseline.mock_builders_to_pull_from):
     71        (test_rebaseline):
     72        (test_rebaseline_command_line_flags):
     73        (test_rebaseline_multiple_builders):
     74        (test_rebaseline_multiple_builders.mock_builders_to_pull_from):
     75        (test_rebaseline_multiple_builders.mock_tests_to_update):
     76        (test_rebaseline_multiple_builders_and_tests_command_line):
     77        * Scripts/webkitpy/tool/mocktool.py:
     78        (MockTool.irc):
     79        (MockTool):
     80        (MockTool.buildbot_for_builder_name):
     81
    1822012-07-10  Alice Cheng  <alice_cheng@apple.com>
    283
  • trunk/Tools/Scripts/webkitpy/common/host.py

    r119785 r122234  
    139139        return self._checkout
    140140
     141    def buildbot_for_builder_name(self, name):
     142        if self.port_factory.get_from_builder_name(name).is_chromium():
     143            return self.chromium_buildbot()
     144        return self.buildbot
     145
    141146    @memoized
    142147    def chromium_buildbot(self):
  • trunk/Tools/Scripts/webkitpy/common/net/buildbot/buildbot.py

    r121852 r122234  
    3434
    3535import webkitpy.common.config.urls as config_urls
     36from webkitpy.common.memoized import memoized
    3637from webkitpy.common.net.failuremap import FailureMap
    3738from webkitpy.common.net.layouttestresults import LayoutTestResults
     
    6566    def accumulated_results_url(self):
    6667        return None
     68
     69    def latest_layout_test_results_url(self):
     70        return self.accumulated_results_url() or self.latest_cached_build().results_url();
     71
     72    @memoized
     73    def latest_layout_test_results(self):
     74        return self.fetch_layout_test_results(self.latest_layout_test_results_url())
     75
     76    def _fetch_file_from_results(self, results_url, file_name):
     77        # It seems this can return None if the url redirects and then returns 404.
     78        result = urllib2.urlopen("%s/%s" % (results_url, file_name))
     79        if not result:
     80            return None
     81        # urlopen returns a file-like object which sometimes works fine with str()
     82        # but sometimes is a addinfourl object.  In either case calling read() is correct.
     83        return result.read()
     84
     85    def fetch_layout_test_results(self, results_url):
     86        # FIXME: This should cache that the result was a 404 and stop hitting the network.
     87        results_file = NetworkTransaction(convert_404_to_None=True).run(lambda: self._fetch_file_from_results(results_url, "full_results.json"))
     88        if not results_file:
     89            results_file = NetworkTransaction(convert_404_to_None=True).run(lambda: self._fetch_file_from_results(results_url, "results.html"))
     90
     91        # results_from_string accepts either ORWT html or NRWT json.
     92        return LayoutTestResults.results_from_string(results_file)
    6793
    6894    def url_encoded_name(self):
     
    230256        self._revision = revision
    231257        self._is_green = is_green
    232         self._layout_test_results = None
    233258
    234259    @staticmethod
     
    246271        return "%s.zip" % self.results_url()
    247272
    248     def _fetch_file_from_results(self, file_name):
    249         # It seems this can return None if the url redirects and then returns 404.
    250         result = urllib2.urlopen("%s/%s" % (self.results_url(), file_name))
    251         if not result:
    252             return None
    253         # urlopen returns a file-like object which sometimes works fine with str()
    254         # but sometimes is a addinfourl object.  In either case calling read() is correct.
    255         return result.read()
    256 
     273    @memoized
    257274    def layout_test_results(self):
    258         if self._layout_test_results:
    259             return self._layout_test_results
    260 
    261         # FIXME: This should cache that the result was a 404 and stop hitting the network.
    262         results_file = NetworkTransaction(convert_404_to_None=True).run(lambda: self._fetch_file_from_results("full_results.json"))
    263         if not results_file:
    264             results_file = NetworkTransaction(convert_404_to_None=True).run(lambda: self._fetch_file_from_results("results.html"))
    265 
    266         # results_from_string accepts either ORWT html or NRWT json.
    267         self._layout_test_results = LayoutTestResults.results_from_string(results_file)
    268         return self._layout_test_results
     275        return self._builder.fetch_layout_test_results(self.results_url())
    269276
    270277    def builder(self):
  • trunk/Tools/Scripts/webkitpy/common/net/buildbot/buildbot_mock.py

    r121800 r122234  
    4949    def accumulated_results_url(self):
    5050        return "http://example.com/f/builders/%s/results/layout-test-results" % self.name()
     51
     52    def latest_layout_test_results_url(self):
     53        return self.accumulated_results_url()
    5154
    5255    def force_build(self, username, comments):
  • trunk/Tools/Scripts/webkitpy/common/net/buildbot/buildbot_unittest.py

    r121447 r122234  
    4949            )
    5050            results = [self._mock_test_result(testname) for testname in failure(build_number)]
    51             build._layout_test_results = LayoutTestResults(results)
     51            layout_test_results = LayoutTestResults(results)
     52            def mock_layout_test_results():
     53                return layout_test_results
     54            build.layout_test_results = mock_layout_test_results
    5255            return build
    5356        self.builder._fetch_build = _mock_fetch_build
     
    5760        self.builder = Builder(u"Test Builder \u2661", self.buildbot)
    5861        self._install_fetch_build(lambda build_number: ["test1", "test2"])
     62
     63    def test_latest_layout_test_results(self):
     64        self.builder.fetch_layout_test_results = lambda results_url: LayoutTestResults([self._mock_test_result(testname) for testname in ["test1", "test2"]])
     65        self.builder.accumulated_results_url = lambda: "http://dummy_url.org"
     66        self.assertTrue(self.builder.latest_layout_test_results())
    5967
    6068    def test_find_regression_window(self):
     
    148156        buildbot = BuildBot()
    149157        builder = Builder(u"Foo Builder (test)", buildbot)
     158        builder._fetch_file_from_results = lambda results_url, file_name: None
    150159        build = Build(builder, None, None, None)
    151         build._fetch_file_from_results = lambda file_name: None
    152160        # Test that layout_test_results() returns None if the fetch fails.
    153161        self.assertEqual(build.layout_test_results(), None)
  • trunk/Tools/Scripts/webkitpy/common/system/user.py

    r116042 r122234  
    7575
    7676    @classmethod
    77     def prompt_with_list(cls, list_title, list_items, can_choose_multiple=False, raw_input=raw_input):
     77    def prompt_with_multiple_lists(cls, list_title, subtitles, lists, can_choose_multiple=False, raw_input=raw_input):
     78        item_index = 0
     79        cumulated_list = []
    7880        print list_title
    79         i = 0
    80         for item in list_items:
    81             i += 1
    82             print "%2d. %s" % (i, item)
     81        for i in range(len(subtitles)):
     82            print "\n" + subtitles[i]
     83            for item in lists[i]:
     84                item_index += 1
     85                print "%2d. %s" % (item_index, item)
     86            cumulated_list += lists[i]
     87        return cls._wait_on_list_response(cumulated_list, can_choose_multiple, raw_input)
    8388
    84         # Loop until we get valid input
     89    @classmethod
     90    def _wait_on_list_response(cls, list_items, can_choose_multiple, raw_input):
    8591        while True:
    8692            if can_choose_multiple:
     
    99105                    continue
    100106                return list_items[result]
     107
     108    @classmethod
     109    def prompt_with_list(cls, list_title, list_items, can_choose_multiple=False, raw_input=raw_input):
     110        print list_title
     111        i = 0
     112        for item in list_items:
     113            i += 1
     114            print "%2d. %s" % (i, item)
     115        return cls._wait_on_list_response(list_items, can_choose_multiple, raw_input)
    101116
    102117    def edit(self, files):
  • trunk/Tools/Scripts/webkitpy/common/system/user_unittest.py

    r68768 r122234  
    5151            return None
    5252        self.assertEqual(User.prompt("input", repeat=self.repeatsRemaining, raw_input=mock_raw_input), None)
     53
     54    def test_prompt_with_multiple_lists(self):
     55        def run_prompt_test(inputs, expected_result, can_choose_multiple=False):
     56            def mock_raw_input(message):
     57                return inputs.pop(0)
     58            output_capture = OutputCapture()
     59            actual_result = output_capture.assert_outputs(
     60                self,
     61                User.prompt_with_multiple_lists,
     62                args=["title", ["subtitle1", "subtitle2"], [["foo", "bar"], ["foobar", "barbaz"]]],
     63                kwargs={"can_choose_multiple": can_choose_multiple, "raw_input": mock_raw_input},
     64                expected_stdout="title\n\nsubtitle1\n 1. foo\n 2. bar\n\nsubtitle2\n 3. foobar\n 4. barbaz\n")
     65            self.assertEqual(actual_result, expected_result)
     66            self.assertEqual(len(inputs), 0)
     67
     68        run_prompt_test(["1"], "foo")
     69        run_prompt_test(["badinput", "2"], "bar")
     70        run_prompt_test(["3"], "foobar")
     71        run_prompt_test(["4"], "barbaz")
     72
     73        run_prompt_test(["1,2"], ["foo", "bar"], can_choose_multiple=True)
     74        run_prompt_test(["  1,  2   "], ["foo", "bar"], can_choose_multiple=True)
     75        run_prompt_test(["all"], ["foo", "bar", 'foobar', 'barbaz'], can_choose_multiple=True)
     76        run_prompt_test([""], ["foo", "bar", 'foobar', 'barbaz'], can_choose_multiple=True)
     77        run_prompt_test(["  "], ["foo", "bar", 'foobar', 'barbaz'], can_choose_multiple=True)
     78        run_prompt_test(["badinput", "all"], ["foo", "bar", 'foobar', 'barbaz'], can_choose_multiple=True)
    5379
    5480    def test_prompt_with_list(self):
  • trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py

    r121512 r122234  
    655655        return False
    656656
     657    def is_chromium(self):
     658        return False
     659
    657660    def name(self):
    658661        """Returns a name that uniquely identifies this particular type of port
  • trunk/Tools/Scripts/webkitpy/layout_tests/port/builders.py

    r121852 r122234  
    9797
    9898
    99 @memoized
    10099def all_builder_names():
    101100    return sorted(set(_exact_matches.keys()))
    102101
    103102
    104 @memoized
    105103def all_port_names():
    106104    return sorted(set(map(lambda x: x["port_name"], _exact_matches.values()) + _ports_without_builders))
  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium.py

    r121812 r122234  
    111111        self._chromium_base_dir_path = None
    112112
     113    def is_chromium(self):
     114        return True
     115
    113116    def default_pixel_tests(self):
    114117        return True
  • trunk/Tools/Scripts/webkitpy/tool/commands/rebaseline.py

    r121852 r122234  
    8282
    8383    def _results_url(self, builder_name):
    84         port = self._tool.port_factory.get_from_builder_name(builder_name)
    85         # FIXME: Come up with a better way than string manipulation to see if the port is a chromium port.
    86         if port.name().startswith('chromium-'):
    87             return self._tool.chromium_buildbot().builder_with_name(builder_name).accumulated_results_url()
    88         return self._tool.buildbot.builder_with_name(builder_name).latest_cached_build().results_url()
     84        return self._tool.buildbot_for_builder_name(builder_name).builder_with_name(builder_name).latest_layout_test_results_url()
    8985
    9086    def _baseline_directory(self, builder_name):
     
    230226
    231227class AbstractParallelRebaselineCommand(AbstractDeclarativeCommand):
    232     def __init__(self):
    233         options = [
     228    def __init__(self, options=None):
     229        options = options or []
     230        options.extend([
    234231            optparse.make_option('--no-optimize', dest='optimize', action='store_false', default=True,
    235232                help=('Do not optimize/de-dup the expectations after rebaselining '
    236233                      '(default is to de-dup automatically). '
    237                       'You can use "webkit-patch optimize-baselines" to optimize separately.')),
    238         ]
     234                      'You can use "webkit-patch optimize-baselines" to optimize separately.'))])
    239235        AbstractDeclarativeCommand.__init__(self, options=options)
    240236
     
    370366class Rebaseline(AbstractParallelRebaselineCommand):
    371367    name = "rebaseline"
    372     help_text = "Replaces local expected.txt files with new results from build bots"
    373 
    374     # FIXME: This should share more code with FailureReason._builder_to_explain
    375     def _builder_to_pull_from(self):
    376         builder_statuses = self._tool.buildbot.builder_statuses()
    377         red_statuses = [status for status in builder_statuses if not status["is_green"]]
    378         _log.info("%s failing" % (pluralize("builder", len(red_statuses))))
    379         builder_choices = [status["name"] for status in red_statuses]
    380         chosen_name = self._tool.user.prompt_with_list("Which builder to pull results from:", builder_choices)
    381         # FIXME: prompt_with_list should really take a set of objects and a set of names and then return the object.
    382         for status in red_statuses:
    383             if status["name"] == chosen_name:
    384                 return (self._tool.buildbot.builder_with_name(chosen_name), status["build_number"])
    385 
    386     def _tests_to_update(self, build):
    387         failing_tests = build.layout_test_results().tests_matching_failure_types([test_failures.FailureTextMismatch])
    388         return self._tool.user.prompt_with_list("Which test(s) to rebaseline:", failing_tests, can_choose_multiple=True)
    389 
    390     def execute(self, options, args, tool):
    391         builder, build_number = self._builder_to_pull_from()
    392         build = builder.build(build_number)
    393 
    394         builder_name = builder.name()
     368    help_text = "Replaces local expected.txt files with new results from build bots. Shows the list of failing tests on the builders if no test names are provided."
     369    argument_names = "[TEST_NAMES]"
     370
     371    def __init__(self):
     372        options = [
     373            optparse.make_option("--builders", default=None, action="append", help="Comma-separated-list of builders to pull new baselines from (can also be provided multiple times)"),
     374        ]
     375        AbstractParallelRebaselineCommand.__init__(self, options=options)
     376
     377    def _builders_to_pull_from(self):
     378        chromium_buildbot_builder_names = []
     379        webkit_buildbot_builder_names = []
     380        for name in builders.all_builder_names():
     381            if self._tool.port_factory.get_from_builder_name(name).is_chromium():
     382                chromium_buildbot_builder_names.append(name)
     383            else:
     384                webkit_buildbot_builder_names.append(name)
     385
     386        titles = ["build.webkit.org bots", "build.chromium.org bots"]
     387        lists = [webkit_buildbot_builder_names, chromium_buildbot_builder_names]
     388
     389        chosen_names = self._tool.user.prompt_with_multiple_lists("Which builder to pull results from:", titles, lists, can_choose_multiple=True)
     390        return [self._builder_with_name(name) for name in chosen_names]
     391
     392    def _builder_with_name(self, name):
     393        return self._tool.buildbot_for_builder_name(name).builder_with_name(name)
     394
     395    def _tests_to_update(self, builder):
     396        failing_tests = builder.latest_layout_test_results().tests_matching_failure_types([test_failures.FailureTextMismatch])
     397        return self._tool.user.prompt_with_list("Which test(s) to rebaseline for %s:" % builder.name(), failing_tests, can_choose_multiple=True)
     398
     399    def execute(self, options, args, tool):
     400        if options.builders:
     401            builders = []
     402            for builder_names in options.builders:
     403                builders += [self._builder_with_name(name) for name in builder_names.split(",")]
     404        else:
     405            builders = self._builders_to_pull_from()
     406
    395407        test_list = {}
    396         for test in self._tests_to_update(build):
    397             test_list[test] = {builder_name: ['txt']}
     408
     409        for builder in builders:
     410            tests = args or self._tests_to_update(builder)
     411            for test in tests:
     412                if test not in test_list:
     413                    test_list[test] = {}
     414                # FIXME: Allow for choosing the suffixes.
     415                test_list[test][builder.name()] = ['txt']
     416
     417        if options.verbose:
     418            print "rebaseline-json: " + str(test_list)
     419
    398420        self._rebaseline(options, test_list)
  • trunk/Tools/Scripts/webkitpy/tool/commands/rebaseline_unittest.py

    r121852 r122234  
    352352            tool.executive = MockExecutive(should_log=True)
    353353
    354             def mock_builder_to_pull_from():
    355                 return MockBuilder('MOCK builder'), 1234
     354            def mock_builders_to_pull_from():
     355                return [MockBuilder('MOCK builder')]
    356356
    357357            def mock_tests_to_update(build):
    358358                return ['mock/path/to/test.html']
    359359
    360             command._builder_to_pull_from = mock_builder_to_pull_from
     360            command._builders_to_pull_from = mock_builders_to_pull_from
    361361            command._tests_to_update = mock_tests_to_update
     362
     363            expected_stdout = """rebaseline-json: {'mock/path/to/test.html': {'MOCK builder': ['txt']}}
     364"""
    362365
    363366            expected_stderr = """MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder', '--test', 'mock/path/to/test.html'], cwd=/mock-checkout
     
    365368"""
    366369
    367             OutputCapture().assert_outputs(self, command.execute, [MockOptions(optimize=True), [], tool], expected_stderr=expected_stderr)
     370            OutputCapture().assert_outputs(self, command.execute, [MockOptions(optimize=True, builders=None, verbose=True), [], tool], expected_stdout=expected_stdout, expected_stderr=expected_stderr)
    368371
    369372        finally:
    370373            builders._exact_matches = old_exact_matches
     374
     375    def test_rebaseline_command_line_flags(self):
     376        old_exact_matches = builders._exact_matches
     377        try:
     378            builders._exact_matches = {
     379                "MOCK builder": {"port_name": "test-mac-leopard", "specifiers": set(["mock-specifier"])},
     380            }
     381
     382            command = Rebaseline()
     383            tool = MockTool()
     384            command.bind_to_tool(tool)
     385
     386            for port_name in tool.port_factory.all_port_names():
     387                port = tool.port_factory.get(port_name)
     388                for path in port.expectations_files():
     389                    tool.filesystem.write_text_file(path, '')
     390
     391            tool.executive = MockExecutive(should_log=True)
     392
     393            expected_stdout = """rebaseline-json: {'mock/path/to/test.html': {'MOCK builder': ['txt']}}
     394"""
     395
     396            expected_stderr = """MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder', '--test', 'mock/path/to/test.html'], cwd=/mock-checkout
     397MOCK run_command: ['echo', 'optimize-baselines', '--suffixes', 'txt', 'mock/path/to/test.html'], cwd=/mock-checkout
     398"""
     399
     400            builder = "MOCK builder"
     401            test = "mock/path/to/test.html"
     402            OutputCapture().assert_outputs(self, command.execute, [MockOptions(optimize=True, builders=[builder], verbose=True), [test], tool], expected_stdout=expected_stdout, expected_stderr=expected_stderr)
     403
     404        finally:
     405            builders._exact_matches = old_exact_matches
     406
     407    def test_rebaseline_multiple_builders(self):
     408        old_exact_matches = builders._exact_matches
     409        try:
     410            builders._exact_matches = {
     411                "MOCK builder": {"port_name": "test-mac-leopard", "specifiers": set(["mock-specifier"])},
     412                "MOCK builder2": {"port_name": "test-mac-snowleopard", "specifiers": set(["mock-specifier2"])},
     413            }
     414
     415            command = Rebaseline()
     416            tool = MockTool()
     417            command.bind_to_tool(tool)
     418
     419            for port_name in tool.port_factory.all_port_names():
     420                port = tool.port_factory.get(port_name)
     421                for path in port.expectations_files():
     422                    tool.filesystem.write_text_file(path, '')
     423
     424            tool.executive = MockExecutive(should_log=True)
     425
     426            def mock_builders_to_pull_from():
     427                return [MockBuilder('MOCK builder'), MockBuilder('MOCK builder2')]
     428
     429            def mock_tests_to_update(build):
     430                return ['mock/path/to/test.html']
     431
     432            command._builders_to_pull_from = mock_builders_to_pull_from
     433            command._tests_to_update = mock_tests_to_update
     434
     435            expected_stdout = """rebaseline-json: {'mock/path/to/test.html': {'MOCK builder2': ['txt'], 'MOCK builder': ['txt']}}
     436"""
     437
     438            expected_stderr = """MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder2', '--test', 'mock/path/to/test.html'], cwd=/mock-checkout
     439MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder', '--test', 'mock/path/to/test.html'], cwd=/mock-checkout
     440MOCK run_command: ['echo', 'optimize-baselines', '--suffixes', 'txt', 'mock/path/to/test.html'], cwd=/mock-checkout
     441"""
     442
     443            OutputCapture().assert_outputs(self, command.execute, [MockOptions(optimize=True, builders=None, verbose=True), [], tool], expected_stdout=expected_stdout, expected_stderr=expected_stderr)
     444
     445        finally:
     446            builders._exact_matches = old_exact_matches
     447
     448    def test_rebaseline_multiple_builders_and_tests_command_line(self):
     449        old_exact_matches = builders._exact_matches
     450        try:
     451            builders._exact_matches = {
     452                "MOCK builder": {"port_name": "test-mac-leopard", "specifiers": set(["mock-specifier"])},
     453                "MOCK builder2": {"port_name": "test-mac-snowleopard", "specifiers": set(["mock-specifier2"])},
     454                "MOCK builder3": {"port_name": "test-mac-snowleopard", "specifiers": set(["mock-specifier2"])},
     455            }
     456
     457            command = Rebaseline()
     458            tool = MockTool()
     459            command.bind_to_tool(tool)
     460
     461            for port_name in tool.port_factory.all_port_names():
     462                port = tool.port_factory.get(port_name)
     463                for path in port.expectations_files():
     464                    tool.filesystem.write_text_file(path, '')
     465
     466            tool.executive = MockExecutive(should_log=True)
     467
     468            expected_stdout = """rebaseline-json: {'mock/path/to/test.html': {'MOCK builder2': ['txt'], 'MOCK builder': ['txt'], 'MOCK builder3': ['txt']}, 'mock/path/to/test2.html': {'MOCK builder2': ['txt'], 'MOCK builder': ['txt'], 'MOCK builder3': ['txt']}}
     469"""
     470
     471            expected_stderr = """MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder2', '--test', 'mock/path/to/test.html'], cwd=/mock-checkout
     472MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder', '--test', 'mock/path/to/test.html'], cwd=/mock-checkout
     473MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder2', '--test', 'mock/path/to/test2.html'], cwd=/mock-checkout
     474MOCK run_command: ['echo', 'rebaseline-test-internal', '--suffixes', 'txt', '--builder', 'MOCK builder', '--test', 'mock/path/to/test2.html'], cwd=/mock-checkout
     475MOCK run_command: ['echo', 'optimize-baselines', '--suffixes', 'txt', 'mock/path/to/test.html'], cwd=/mock-checkout
     476MOCK run_command: ['echo', 'optimize-baselines', '--suffixes', 'txt', 'mock/path/to/test2.html'], cwd=/mock-checkout
     477"""
     478
     479            OutputCapture().assert_outputs(self, command.execute, [MockOptions(optimize=True, builders=["MOCK builder,MOCK builder2", "MOCK builder3"], verbose=True), ["mock/path/to/test.html", "mock/path/to/test2.html"], tool], expected_stdout=expected_stdout, expected_stderr=expected_stderr)
     480
     481        finally:
     482            builders._exact_matches = old_exact_matches
  • trunk/Tools/Scripts/webkitpy/tool/mocktool.py

    r120488 r122234  
    3030
    3131from webkitpy.common.host_mock import MockHost
     32from webkitpy.common.net.buildbot.buildbot_mock import MockBuildBot
    3233from webkitpy.common.net.statusserver_mock import MockStatusServer
    3334from webkitpy.common.net.irc.irc_mock import MockIRC
     
    8384    def irc(self):
    8485        return self._irc
     86
     87    def buildbot_for_builder_name(self, name):
     88        return MockBuildBot()
Note: See TracChangeset for help on using the changeset viewer.