Changeset 226009 in webkit


Ignore:
Timestamp:
Dec 16, 2017 6:54:13 PM (6 years ago)
Author:
commit-queue@webkit.org
Message:

Add a script to automate W3c web-platform-tests pull request creations from WebKit commits
https://bugs.webkit.org/show_bug.cgi?id=169462

Patch by Youenn Fablet <youenn@apple.com> on 2017-12-16
Reviewed by Darin Adler.

Adding some git helper routines used by WPT exporter.
Copying WPT github utility files from Chromium.
Updating web.py/web_mock.py to cope with these new files.

Implementing test exporter in test_exporter.py.
This script takes a WebKit commit as input and will create a WPT commit in a local WPT clone.
It will then push the commit to a public GitHub repository identified by a username parameter.
This parameter is passed through the command line or searched through git config/environment variables.

The script can optionally create a pull request to the official WPT GitHub repository.
User must provide a GitHub token to allow the script to make the PR on behalf of the user.
In that case, a comment is added to the corresponding bugzilla if a bug ID is given or can be found from the change log,
to easily link the pull request with the bugzilla bug.

  • Scripts/export-w3c-test-changes: Added.
  • Scripts/webkitpy/common/checkout/scm/git.py:

(Git.reset_hard):
(Git):
(Git.am):
(Git.commit):
(Git.format_patch):
(Git.request_pull):
(Git.remote):
(Git.push):
(Git.checkout_new_branch):

  • Scripts/webkitpy/common/net/web.py:

(Web.request): Copied from Chromium.

  • Scripts/webkitpy/common/net/web_mock.py: Copied needed code from Chromium.
  • Scripts/webkitpy/w3c/test_exporter.py: Added.

(TestExporter):
(TestExporter.init):
(TestExporter._init_repository):
(TestExporter.download_and_commit_patch):
(TestExporter.clean):
(TestExporter.create_branch_with_patch):
(TestExporter.push_to_public_repository):
(TestExporter.make_pull_request):
(TestExporter.delete_local_branch):
(TestExporter.create_git_patch):
(TestExporter.create_upload_remote_if_needed):
(TestExporter.do_export):
(parse_args):
(configure_logging):
(configure_logging.LogHandler):
(configure_logging.LogHandler.format):
(main):

  • Scripts/webkitpy/w3c/test_exporter_unittest.py: Added.

(TestExporterTest):
(TestExporterTest.MockBugzilla):
(TestExporterTest.MockBugzilla.init):
(TestExporterTest.MockBugzilla.fetch_bug_dictionary):
(TestExporterTest.MockBugzilla.post_comment_to_bug):
(TestExporterTest.MockGit):
(TestExporterTest.MockGit.clone):
(TestExporterTest.MockGit.init):
(TestExporterTest.MockGit.fetch):
(TestExporterTest.MockGit.checkout):
(TestExporterTest.MockGit.reset_hard):
(TestExporterTest.MockGit.push):
(TestExporterTest.MockGit.format_patch):
(TestExporterTest.MockGit.delete_branch):
(TestExporterTest.MockGit.checkout_new_branch):
(TestExporterTest.MockGit.am):
(TestExporterTest.MockGit.commit):
(TestExporterTest.MockGit.remote):
(TestExporterTest.test_export):

  • Scripts/webkitpy/w3c/common.py: Copied from chromium.
  • Scripts/webkitpy/w3c/wpt_github.py: Copied from chromium.
  • Scripts/webkitpy/w3c/wpt_github_mock.py: Copied from chromium.
  • Scripts/webkitpy/w3c/wpt_github_unittest.py: Copied from chromium.
Location:
trunk/Tools
Files:
6 added
4 edited
1 copied

Legend:

Unmodified
Added
Removed
  • trunk/Tools/ChangeLog

    r226008 r226009  
     12017-12-16  Youenn Fablet  <youenn@apple.com>
     2
     3        Add a script to automate W3c web-platform-tests pull request creations from WebKit commits
     4        https://bugs.webkit.org/show_bug.cgi?id=169462
     5
     6        Reviewed by Darin Adler.
     7
     8        Adding some git helper routines used by WPT exporter.
     9        Copying WPT github utility files from Chromium.
     10        Updating web.py/web_mock.py to cope with these new files.
     11
     12        Implementing test exporter in test_exporter.py.
     13        This script takes a WebKit commit as input and will create a WPT commit in a local WPT clone.
     14        It will then push the commit to a public GitHub repository identified by a username parameter.
     15        This parameter is passed through the command line or searched through git config/environment variables.
     16
     17        The script can optionally create a pull request to the official WPT GitHub repository.
     18        User must provide a GitHub token to allow the script to make the PR on behalf of the user.
     19        In that case, a comment is added to the corresponding bugzilla if a bug ID is given or can be found from the change log,
     20        to easily link the pull request with the bugzilla bug.
     21
     22        * Scripts/export-w3c-test-changes: Added.
     23        * Scripts/webkitpy/common/checkout/scm/git.py:
     24        (Git.reset_hard):
     25        (Git):
     26        (Git.am):
     27        (Git.commit):
     28        (Git.format_patch):
     29        (Git.request_pull):
     30        (Git.remote):
     31        (Git.push):
     32        (Git.checkout_new_branch):
     33        * Scripts/webkitpy/common/net/web.py:
     34        (Web.request): Copied from Chromium.
     35        * Scripts/webkitpy/common/net/web_mock.py: Copied needed code from Chromium.
     36        * Scripts/webkitpy/w3c/test_exporter.py: Added.
     37        (TestExporter):
     38        (TestExporter.__init__):
     39        (TestExporter._init_repository):
     40        (TestExporter.download_and_commit_patch):
     41        (TestExporter.clean):
     42        (TestExporter.create_branch_with_patch):
     43        (TestExporter.push_to_public_repository):
     44        (TestExporter.make_pull_request):
     45        (TestExporter.delete_local_branch):
     46        (TestExporter.create_git_patch):
     47        (TestExporter.create_upload_remote_if_needed):
     48        (TestExporter.do_export):
     49        (parse_args):
     50        (configure_logging):
     51        (configure_logging.LogHandler):
     52        (configure_logging.LogHandler.format):
     53        (main):
     54        * Scripts/webkitpy/w3c/test_exporter_unittest.py: Added.
     55        (TestExporterTest):
     56        (TestExporterTest.MockBugzilla):
     57        (TestExporterTest.MockBugzilla.__init__):
     58        (TestExporterTest.MockBugzilla.fetch_bug_dictionary):
     59        (TestExporterTest.MockBugzilla.post_comment_to_bug):
     60        (TestExporterTest.MockGit):
     61        (TestExporterTest.MockGit.clone):
     62        (TestExporterTest.MockGit.__init__):
     63        (TestExporterTest.MockGit.fetch):
     64        (TestExporterTest.MockGit.checkout):
     65        (TestExporterTest.MockGit.reset_hard):
     66        (TestExporterTest.MockGit.push):
     67        (TestExporterTest.MockGit.format_patch):
     68        (TestExporterTest.MockGit.delete_branch):
     69        (TestExporterTest.MockGit.checkout_new_branch):
     70        (TestExporterTest.MockGit.am):
     71        (TestExporterTest.MockGit.commit):
     72        (TestExporterTest.MockGit.remote):
     73        (TestExporterTest.test_export):
     74        * Scripts/webkitpy/w3c/common.py: Copied from chromium.
     75        * Scripts/webkitpy/w3c/wpt_github.py: Copied from chromium.
     76        * Scripts/webkitpy/w3c/wpt_github_mock.py: Copied from chromium.
     77        * Scripts/webkitpy/w3c/wpt_github_unittest.py: Copied from chromium.
     78
    1792017-12-16  Dan Bernstein  <mitz@apple.com>
    280
  • trunk/Tools/Scripts/webkitpy/common/checkout/scm/git.py

    r225742 r226009  
    3030import datetime
    3131import logging
    32 import os
    3332import re
    3433
     
    3837from .commitmessage import CommitMessage
    3938from .scm import AuthenticationError, SCM, commit_error_handler
    40 from .svn import SVN, SVNRepository
     39from .svn import SVNRepository
    4140
    4241_log = logging.getLogger(__name__)
     
    508507        return self._run_git(['submodule', 'deinit', '-f', '.'])
    509508
    510     def _branch_ref_exists(self, branch_ref):
     509    def branch_ref_exists(self, branch_ref):
    511510        return self._run_git(['show-ref', '--quiet', '--verify', branch_ref], return_exit_code=True) == 0
    512511
    513512    def delete_branch(self, branch_name):
    514         if self._branch_ref_exists('refs/heads/' + branch_name):
     513        if self.branch_ref_exists('refs/heads/' + branch_name):
    515514            self._run_git(['branch', '-D', branch_name])
    516515
     
    523522        if not remote_branch_refs:
    524523            remote_master_ref = 'refs/remotes/origin/master'
    525             if not self._branch_ref_exists(remote_master_ref):
     524            if not self.branch_ref_exists(remote_master_ref):
    526525                raise ScriptError(message="Can't find a branch to diff against. svn-remote.svn.fetch is not in the git config and %s does not exist" % remote_master_ref)
    527526            return remote_master_ref
     
    585584        return self._run_git(['fetch', remote])
    586585
     586    # Reset current HEAD to the specified commit.
     587    def reset_hard(self, commit):
     588        return self._run_git(['reset', '--hard', commit])
     589
     590    def apply_mail_patch(self, options):
     591        return self._run_git(['apply'] + options)
     592
     593    def commit(self, options):
     594        return self._run_git(['commit'] + options)
     595
     596    def format_patch(self, options):
     597        return self._run_git(['format-patch'] + options)
     598
     599    def request_pull(self, options):
     600        return self._run_git(['request-pull'] + options)
     601
     602    def remote(self, options):
     603        return self._run_git(['remote'] + options)
     604
     605    def push(self, options):
     606        return self._run_git(['push'] + options)
     607
     608    def local_config(self, key):
     609        return self._run_git(['config', '--get', '--local', key], error_handler=Executive.ignore_error)
     610
     611    def set_local_config(self, key, value):
     612        return self._run_git(['config', '--add', '--local', key, value], error_handler=Executive.ignore_error)
     613
     614    def checkout_new_branch(self, branch_name):
     615        return self._run_git(['checkout', '-b', branch_name])
     616
    587617    def checkout(self, revision, quiet=None):
    588618        command = ['checkout', revision]
  • trunk/Tools/Scripts/webkitpy/common/net/web.py

    r225742 r226009  
    3535    def get_binary(self, url, convert_404_to_None=False):
    3636        return NetworkTransaction(convert_404_to_None=convert_404_to_None).run(lambda: urllib2.urlopen(url).read())
     37
     38    def request(self, method, url, data, headers=None):
     39        opener = urllib2.build_opener(urllib2.HTTPHandler)
     40        request = urllib2.Request(url=url, data=data)
     41        request.get_method = lambda: method
     42
     43        if headers:
     44            for key, value in headers.items():
     45                request.add_header(key, value)
     46        return opener.open(request)
  • trunk/Tools/Scripts/webkitpy/common/net/web_mock.py

    r225742 r226009  
    2828
    2929import StringIO
    30 
     30import urllib2
    3131
    3232class MockWeb(object):
    33     def __init__(self, urls=None):
     33    def __init__(self, urls=None, responses=[]):
    3434        self.urls = urls or {}
    3535        self.urls_fetched = []
     36        self.responses = responses
    3637
    3738    def get_binary(self, url, convert_404_to_None=False):
     
    4142        return "MOCK Web result, convert 404 to None=%s" % convert_404_to_None
    4243
     44    def request(self, method, url, data, headers=None):  # pylint: disable=unused-argument
     45        return MockResponse(self.responses.pop(0))
     46
     47
     48class MockResponse(object):
     49    def __init__(self, values):
     50        self.status_code = values['status_code']
     51        self.url = ''
     52        self.body = values.get('body', '')
     53
     54        if int(self.status_code) >= 400:
     55            raise urllib2.HTTPError(
     56                url=self.url,
     57                code=self.status_code,
     58                msg='Received error status code: {}'.format(self.status_code),
     59                hdrs={},
     60                fp=None)
     61
     62    def getcode(self):
     63        return self.status_code
     64
     65    def read(self):
     66        return self.body
    4367
    4468# FIXME: Classes which are using Browser probably want to use Web instead.
  • trunk/Tools/Scripts/webkitpy/w3c/wpt_github_unittest.py

    r226008 r226009  
    1 # Copyright (C) 2011 Google Inc. All rights reserved.
     1# Copyright 2017 The Chromium Authors. All rights reserved.
    22#
    33# Redistribution and use in source and binary forms, with or without
     
    2727# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    2828
    29 import StringIO
     29import base64
     30import unittest
     31
     32from webkitpy.common.host_mock import MockHost
     33from webkitpy.w3c.wpt_github import WPTGitHub, MergeError
    3034
    3135
    32 class MockWeb(object):
    33     def __init__(self, urls=None):
    34         self.urls = urls or {}
    35         self.urls_fetched = []
     36class WPTGitHubTest(unittest.TestCase):
    3637
    37     def get_binary(self, url, convert_404_to_None=False):
    38         self.urls_fetched.append(url)
    39         if url in self.urls:
    40             return self.urls[url]
    41         return "MOCK Web result, convert 404 to None=%s" % convert_404_to_None
     38    def setUp(self):
     39        self.wpt_github = WPTGitHub(MockHost(), user='rutabaga', token='decafbad')
    4240
     41    def test_init(self):
     42        self.assertEqual(self.wpt_github.user, 'rutabaga')
     43        self.assertEqual(self.wpt_github.token, 'decafbad')
    4344
    44 # FIXME: Classes which are using Browser probably want to use Web instead.
    45 class MockBrowser(object):
    46     params = {}
    47 
    48     def open(self, url):
    49         pass
    50 
    51     def select_form(self, name):
    52         pass
    53 
    54     def __setitem__(self, key, value):
    55         self.params[key] = value
    56 
    57     def submit(self):
    58         return StringIO.StringIO()
    59 
    60     def set_handle_robots(self, value):
    61         pass
     45    def test_auth_token(self):
     46        self.assertEqual(
     47            self.wpt_github.auth_token(),
     48            base64.encodestring('rutabaga:decafbad').strip())
Note: See TracChangeset for help on using the changeset viewer.