Changeset 125123 in webkit


Ignore:
Timestamp:
Aug 8, 2012 5:19:23 PM (12 years ago)
Author:
rniwa@webkit.org
Message:

Add Perf EWS IRC bot
​https://bugs.webkit.org/show_bug.cgi?id=92913

Reviewed by Dirk Pranke.

Adds the preliminary implementation of perfalizer, a IRC bot that runs performance tests
based on IRC requests.

  • Scripts/webkitpy/common/checkout/scm/scm_mock.py:

(MockSCM.head_svn_revision): The actual implementation returns string, so match that.
(MockSCM.svn_revision): Ditto.

  • Scripts/webkitpy/common/net/bugzilla/bugzilla.py:

(Bugzilla.add_attachment_to_bug): Added mimetype.

  • Scripts/webkitpy/common/net/bugzilla/bugzilla_mock.py:

(MockBugzilla.add_attachment_to_bug): Ditto.

  • Scripts/webkitpy/common/system/filesystem.py:

(FileSystem.copytree): Added.

  • Scripts/webkitpy/common/system/filesystem_mock.py:

(MockFileSystem.copytree): Added.

  • Scripts/webkitpy/performance_tests/perftestsrunner_unittest.py: Rebaselined tests

that had assumed revision numbers were integers.
(test_run_with_json_output):
(test_run_with_description):
(test_run_generates_json_by_default):
(test_run_generates_and_show_results_page):
(test_run_with_json_source):
(test_run_with_multiple_repositories):

  • Scripts/webkitpy/tool/bot/flakytestreporter_unittest.py:
  • Scripts/webkitpy/tool/commands/init.py:
  • Scripts/webkitpy/tool/commands/perfalizer.py: Added.

(PerfalizerTask):
(PerfalizerTask.init):
(PerfalizerTask._copy_build_product_without_patch):
(PerfalizerTask.run):
(PerfalizerTask.parent_command):
(PerfalizerTask.run_webkit_patch):
(PerfalizerTask._json_path):
(PerfalizerTask._results_page_path):
(PerfalizerTask._run_perf_test):
(PerfalizerTask.run_command):
(PerfalizerTask.command_passed):
(PerfalizerTask.command_failed):
(PerfalizerTask.refetch_patch):
(PerfalizerTask.expected_failures):
(PerfalizerTask.build_style):
(PerfTest): IRC command.
(PerfTest.execute):
(Perfalizer): IRC bot.
(Perfalizer.begin_work_queue):
(Perfalizer.work_item_log_path):
(Perfalizer._is_old_failure):
(Perfalizer.next_work_item):
(Perfalizer.process_work_item):
(Perfalizer.handle_unexpected_error):
(Perfalizer.handle_script_error):

  • Scripts/webkitpy/tool/commands/perfalizer_unittest.py: Added.

(PerfalizerTaskTest):
(PerfalizerTaskTest._create_and_run_perfalizer):
(PerfalizerTaskTest._create_and_run_perfalizer.logger):
(PerfalizerTaskTest._create_and_run_perfalizer.run_webkit_patch):
(PerfalizerTaskTest._create_and_run_perfalizer.run_perf_test):
(PerfalizerTaskTest.test_run):
(PerfalizerTaskTest.test_run_with_clean_fails):
(PerfalizerTaskTest.test_run_with_update_fails):
(PerfalizerTaskTest.test_run_with_build_fails):
(PerfalizerTaskTest.test_run_with_perf_test_fails):
(PerfalizerTaskTest.test_run_without_results_page):

  • Scripts/webkitpy/tool/commands/queues_unittest.py:

(AbstractPatchQueueTest.test_upload_results_archive_for_patch): Rebaselined.

  • Scripts/webkitpy/tool/commands/upload_unittest.py:

(test_attach_to_bug): Ditto.
(test_attach_to_bug_no_description_or_comment): Ditto.

Location:
trunk/Tools
Files:
2 added
11 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r125115 r125123  
     12012-08-08  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        Add Perf EWS IRC bot
     4        https://bugs.webkit.org/show_bug.cgi?id=92913
     5
     6        Reviewed by Dirk Pranke.
     7
     8        Adds the preliminary implementation of perfalizer, a IRC bot that runs performance tests
     9        based on IRC requests.
     10
     11        * Scripts/webkitpy/common/checkout/scm/scm_mock.py:
     12        (MockSCM.head_svn_revision): The actual implementation returns string, so match that.
     13        (MockSCM.svn_revision): Ditto.
     14        * Scripts/webkitpy/common/net/bugzilla/bugzilla.py:
     15        (Bugzilla.add_attachment_to_bug): Added mimetype.
     16        * Scripts/webkitpy/common/net/bugzilla/bugzilla_mock.py:
     17        (MockBugzilla.add_attachment_to_bug): Ditto.
     18        * Scripts/webkitpy/common/system/filesystem.py:
     19        (FileSystem.copytree): Added.
     20        * Scripts/webkitpy/common/system/filesystem_mock.py:
     21        (MockFileSystem.copytree): Added.
     22        * Scripts/webkitpy/performance_tests/perftestsrunner_unittest.py: Rebaselined tests
     23        that had assumed revision numbers were integers.
     24        (test_run_with_json_output):
     25        (test_run_with_description):
     26        (test_run_generates_json_by_default):
     27        (test_run_generates_and_show_results_page):
     28        (test_run_with_json_source):
     29        (test_run_with_multiple_repositories):
     30        * Scripts/webkitpy/tool/bot/flakytestreporter_unittest.py:
     31        * Scripts/webkitpy/tool/commands/__init__.py:
     32        * Scripts/webkitpy/tool/commands/perfalizer.py: Added.
     33        (PerfalizerTask):
     34        (PerfalizerTask.__init__):
     35        (PerfalizerTask._copy_build_product_without_patch):
     36        (PerfalizerTask.run):
     37        (PerfalizerTask.parent_command):
     38        (PerfalizerTask.run_webkit_patch):
     39        (PerfalizerTask._json_path):
     40        (PerfalizerTask._results_page_path):
     41        (PerfalizerTask._run_perf_test):
     42        (PerfalizerTask.run_command):
     43        (PerfalizerTask.command_passed):
     44        (PerfalizerTask.command_failed):
     45        (PerfalizerTask.refetch_patch):
     46        (PerfalizerTask.expected_failures):
     47        (PerfalizerTask.build_style):
     48        (PerfTest): IRC command.
     49        (PerfTest.execute):
     50        (Perfalizer): IRC bot.
     51        (Perfalizer.begin_work_queue):
     52        (Perfalizer.work_item_log_path):
     53        (Perfalizer._is_old_failure):
     54        (Perfalizer.next_work_item):
     55        (Perfalizer.process_work_item):
     56        (Perfalizer.handle_unexpected_error):
     57        (Perfalizer.handle_script_error):
     58        * Scripts/webkitpy/tool/commands/perfalizer_unittest.py: Added.
     59        (PerfalizerTaskTest):
     60        (PerfalizerTaskTest._create_and_run_perfalizer):
     61        (PerfalizerTaskTest._create_and_run_perfalizer.logger):
     62        (PerfalizerTaskTest._create_and_run_perfalizer.run_webkit_patch):
     63        (PerfalizerTaskTest._create_and_run_perfalizer.run_perf_test):
     64        (PerfalizerTaskTest.test_run):
     65        (PerfalizerTaskTest.test_run_with_clean_fails):
     66        (PerfalizerTaskTest.test_run_with_update_fails):
     67        (PerfalizerTaskTest.test_run_with_build_fails):
     68        (PerfalizerTaskTest.test_run_with_perf_test_fails):
     69        (PerfalizerTaskTest.test_run_without_results_page):
     70        * Scripts/webkitpy/tool/commands/queues_unittest.py:
     71        (AbstractPatchQueueTest.test_upload_results_archive_for_patch): Rebaselined.
     72        * Scripts/webkitpy/tool/commands/upload_unittest.py:
     73        (test_attach_to_bug): Ditto.
     74        (test_attach_to_bug_no_description_or_comment): Ditto.
     75
    1762012-08-08  Dirk Pranke  <dpranke@chromium.org>
    277

  • trunk/Tools/Scripts/webkitpy/common/checkout/scm/scm_mock.py

    r118557 r125123  
    7070
    7171    def head_svn_revision(self):
    72         return 1234
     72        return '1234'
    7373
    7474    def svn_revision(self, path):
    75         return 5678
     75        return '5678'
    7676
    7777    def create_patch(self, git_commit, changed_files=None):

  • trunk/Tools/Scripts/webkitpy/common/net/bugzilla/bugzilla.py

    r121269 r125123  
    579579        return "bug-%s-%s.%s" % (bug_id, timestamp(), extension)
    580580
    581     def add_attachment_to_bug(self,
    582                               bug_id,
    583                               file_or_string,
    584                               description,
    585                               filename=None,
    586                               comment_text=None):
     581    def add_attachment_to_bug(self, bug_id, file_or_string, description, filename=None, comment_text=None, mimetype=None):
    587582        self.authenticate()
    588583        log('Adding attachment "%s" to %s' % (description, self.bug_url_for_bug_id(bug_id)))
    … …  
    591586        file_object = self._file_object_for_upload(file_or_string)
    592587        filename = filename or self._filename_for_upload(file_object, bug_id)
    593         self._fill_attachment_form(description, file_object, filename=filename)
     588        self._fill_attachment_form(description, file_object, filename=filename, mimetype=mimetype)
    594589        if comment_text:
    595590            log(comment_text)

  • trunk/Tools/Scripts/webkitpy/common/net/bugzilla/bugzilla_mock.py

    r121269 r125123  
    394394            bug_id, cc, comment_text))
    395395
    396     def add_attachment_to_bug(self,
    397                               bug_id,
    398                               file_or_string,
    399                               description,
    400                               filename=None,
    401                               comment_text=None):
    402         log("MOCK add_attachment_to_bug: bug_id=%s, description=%s filename=%s" % (bug_id, description, filename))
     396    def add_attachment_to_bug(self, bug_id, file_or_string, description, filename=None, comment_text=None, mimetype=None):
     397        log("MOCK add_attachment_to_bug: bug_id=%s, description=%s filename=%s mimetype=%s" %
     398            (bug_id, description, filename, mimetype))
    403399        if comment_text:
    404400            log("-- Begin comment --")

  • trunk/Tools/Scripts/webkitpy/common/system/filesystem.py

    r112168 r125123  
    263263        shutil.rmtree(path, ignore_errors=True)
    264264
     265    def copytree(self, source, destination):
     266        shutil.copytree(source, destination)
     267
    265268    def split(self, path):
    266269        """Return (dirname, basename + '.' + ext)"""

  • trunk/Tools/Scripts/webkitpy/common/system/filesystem_mock.py

    r118979 r125123  
    379379        self.dirs = set(filter(lambda d: not d.startswith(path), self.dirs))
    380380
     381    def copytree(self, source, destination):
     382        source = self.normpath(source)
     383        destination = self.normpath(destination)
     384
     385        for source_file in self.files:
     386            if source_file.startswith(source):
     387                destination_path = self.join(destination, self.relpath(source_file, source))
     388                self.maybe_make_directory(self.dirname(destination_path))
     389                self.files[destination_path] = self.files[source_file]
     390
    381391    def split(self, path):
    382392        idx = path.rfind(self.sep)

  • trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner_unittest.py

    r124949 r125123  
    269269            {"Bindings/event-target-wrapper": {"max": 1510, "avg": 1489.05, "median": 1487, "min": 1471, "stdev": 14.46, "unit": "ms"},
    270270            "inspector/pass.html:group_name:test_name": 42},
    271             "webkit-revision": 5678, "branch": "webkit-trunk"})
     271            "webkit-revision": "5678", "branch": "webkit-trunk"})
    272272
    273273    def test_run_with_description(self):
    … …  
    279279            {"Bindings/event-target-wrapper": {"max": 1510, "avg": 1489.05, "median": 1487, "min": 1471, "stdev": 14.46, "unit": "ms"},
    280280            "inspector/pass.html:group_name:test_name": 42},
    281             "webkit-revision": 5678, "branch": "webkit-trunk"})
     281            "webkit-revision": "5678", "branch": "webkit-trunk"})
    282282
    283283    def create_runner_and_setup_results_template(self, args=[]):
    … …  
    311311            {"Bindings/event-target-wrapper": {"max": 1510, "avg": 1489.05, "median": 1487, "min": 1471, "stdev": 14.46, "unit": "ms"},
    312312            "inspector/pass.html:group_name:test_name": 42},
    313             "webkit-revision": 5678, "branch": "webkit-trunk"}])
     313            "webkit-revision": "5678", "branch": "webkit-trunk"}])
    314314
    315315        self.assertTrue(filesystem.isfile(output_json_path))
    … …  
    325325        expected_entry = {"timestamp": 123456789, "results": {"Bindings/event-target-wrapper":
    326326            {"max": 1510, "avg": 1489.05, "median": 1487, "min": 1471, "stdev": 14.46, "unit": "ms"},
    327             "inspector/pass.html:group_name:test_name": 42}, "webkit-revision": 5678, "branch": "webkit-trunk"}
     327            "inspector/pass.html:group_name:test_name": 42}, "webkit-revision": "5678", "branch": "webkit-trunk"}
    328328
    329329        self.maxDiff = None
    … …  
    374374            {"Bindings/event-target-wrapper": {"max": 1510, "avg": 1489.05, "median": 1487, "min": 1471, "stdev": 14.46, "unit": "ms"},
    375375            "inspector/pass.html:group_name:test_name": 42},
    376             "webkit-revision": 5678, "branch": "webkit-trunk",
     376            "webkit-revision": "5678", "branch": "webkit-trunk",
    377377            "key": "value"})
    378378
    … …  
    395395            {"Bindings/event-target-wrapper": {"max": 1510, "avg": 1489.05, "median": 1487, "min": 1471, "stdev": 14.46, "unit": "ms"},
    396396            "inspector/pass.html:group_name:test_name": 42.0},
    397             "webkit-revision": 5678, "some-revision": 5678, "branch": "webkit-trunk"})
     397            "webkit-revision": "5678", "some-revision": "5678", "branch": "webkit-trunk"})
    398398
    399399    def test_run_with_upload_json(self):

  • trunk/Tools/Scripts/webkitpy/tool/bot/flakytestreporter_unittest.py

    r116572 r125123  
    126126cc: abarth@webkit.org
    127127blocked: 50856
    128 MOCK add_attachment_to_bug: bug_id=60001, description=Failure diff from mock-bot-id filename=failure.diff
     128MOCK add_attachment_to_bug: bug_id=60001, description=Failure diff from mock-bot-id filename=failure.diff mimetype=None
    129129MOCK bug comment: bug_id=50000, cc=None
    130130--- Begin comment ---

  • trunk/Tools/Scripts/webkitpy/tool/commands/__init__.py

    r116414 r125123  
    1313from webkitpy.tool.commands.gardenomatic import GardenOMatic
    1414from webkitpy.tool.commands.openbugs import OpenBugs
     15from webkitpy.tool.commands.perfalizer import Perfalizer
    1516from webkitpy.tool.commands.prettydiff import PrettyDiff
    1617from webkitpy.tool.commands.queries import *

  • trunk/Tools/Scripts/webkitpy/tool/commands/queues_unittest.py

    r124354 r125123  
    174174        queue._options.port = None
    175175        patch = queue._tool.bugs.fetch_attachment(10001)
    176         expected_stderr = """MOCK add_attachment_to_bug: bug_id=50000, description=Archive of layout-test-results from bot filename=layout-test-results.zip
     176        expected_stderr = """MOCK add_attachment_to_bug: bug_id=50000, description=Archive of layout-test-results from bot filename=layout-test-results.zip mimetype=None
    177177-- Begin comment --
    178178The attached test failures were seen while running run-webkit-tests on the mock-queue.
    … …  
    406406--- End comment ---
    407407
    408 MOCK add_attachment_to_bug: bug_id=50002, description=Failure diff from bot filename=failure.diff
     408MOCK add_attachment_to_bug: bug_id=50002, description=Failure diff from bot filename=failure.diff mimetype=None
    409409MOCK bug comment: bug_id=50002, cc=None
    410410--- Begin comment ---
    … …  
    413413--- End comment ---
    414414
    415 MOCK add_attachment_to_bug: bug_id=50002, description=Archive of layout-test-results from bot filename=layout-test-results.zip
     415MOCK add_attachment_to_bug: bug_id=50002, description=Archive of layout-test-results from bot filename=layout-test-results.zip mimetype=None
    416416MOCK bug comment: bug_id=50000, cc=None
    417417--- Begin comment ---

  • trunk/Tools/Scripts/webkitpy/tool/commands/upload_unittest.py

    r101834 r125123  
    8080        options.comment = "extra comment"
    8181        options.description = "file description"
    82         expected_stderr = """MOCK add_attachment_to_bug: bug_id=50000, description=file description filename=None
     82        expected_stderr = """MOCK add_attachment_to_bug: bug_id=50000, description=file description filename=None mimetype=None
    8383-- Begin comment --
    8484extra comment
    … …  
    9191        options.comment = None
    9292        options.description = None
    93         expected_stderr = """MOCK add_attachment_to_bug: bug_id=50000, description=file.txt filename=None
     93        expected_stderr = """MOCK add_attachment_to_bug: bug_id=50000, description=file.txt filename=None mimetype=None
    9494"""
    9595        self.assert_execute_outputs(AttachToBug(), [50000, "path/to/file.txt"], options=options, expected_stderr=expected_stderr)
Note: See TracChangeset for help on using the changeset viewer.