Changeset 181479 in webkit


Ignore:
Timestamp:
Mar 13, 2015 9:00:03 AM (9 years ago)
Author:
youenn.fablet@crf.canon.fr
Message:

WebKit test infrastructure should automate the process of cloning W3C test suite and importing tests from it
​https://bugs.webkit.org/show_bug.cgi?id=134764

Reviewed by Ryosuke Niwa.

LayoutTests/imported/w3c:

TestRepositories defines the git repositories used to import tests.
It defines for each repository the infrastructure paths to always import, the paths to skip and the revision to use.
ImportExpectartions defines the tests to import.

  • resources/ImportExpectations: Added.
  • resources/TestRepositories: Added.

Tools:

TestDownloader is introduced to download W3C tests from the official
W3C tests repositories (​https://github.com/w3c/web-platform-tests.git
and ​https://github.com/w3c/csswg-test.git).
TestDownloader clones the repositories and checkouts revisions of the tests
defined in LayoutTests/imported/w3c/resources/TestRepositories.

Similarly to ​https://codereview.chromium.org/148173016,
TestDownloader copies part of the files based on the rules
defined in LayoutTests/imported/w3c/resources/ImportExpectations.

TestDownloader is used by TestImporter when no source directory is provided.

  • Scripts/webkitpy/w3c/test_downloader.py: Added.

(TestDownloader):
(TestDownloader.default_options):
(TestDownloader.load_test_repositories):
(TestDownloader.init):
(TestDownloader.checkout_test_repository):
(TestDownloader._init_paths_from_expectations):
(TestDownloader._add_test_suite_paths):
(TestDownloader._empty_directory):
(TestDownloader.copy_tests):
(TestDownloader.download_tests):

  • Scripts/webkitpy/w3c/test_importer.py:

(main):
(parse_args):
(TestImporter.init):
(TestImporter.do_import):
(TestImporter.test_downloader):
(TestImporter.find_importable_tests.should_keep_subdir):
(TestImporter.should_convert_test_harness_links):
(TestImporter.import_tests):
(TestImporter.write_import_log):

Location:
trunk
Files:
3 added
3 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/imported/w3c/ChangeLog

    r181146 r181479  
     12015-03-13  Youenn Fablet  <youenn.fablet@crf.canon.fr>
     2
     3        WebKit test infrastructure should automate the process of cloning W3C test suite and importing tests from it
     4        https://bugs.webkit.org/show_bug.cgi?id=134764
     5
     6        Reviewed by Ryosuke Niwa.
     7
     8        TestRepositories defines the git repositories used to import tests.
     9        It defines for each repository the infrastructure paths to always import, the paths to skip and the revision to use.
     10        ImportExpectartions defines the tests to import.
     11
     12        * resources/ImportExpectations: Added.
     13        * resources/TestRepositories: Added.
     14
    1152015-03-06  Carlos Alberto Lopez Perez <clopez@igalia.com> and Youenn Fablet  <youenn.fablet@crf.canon.fr>
    216

  • trunk/Tools/ChangeLog

    r181451 r181479  
     12015-03-13  Youenn Fablet  <youenn.fablet@crf.canon.fr>
     2
     3        WebKit test infrastructure should automate the process of cloning W3C test suite and importing tests from it
     4        https://bugs.webkit.org/show_bug.cgi?id=134764
     5
     6        Reviewed by Ryosuke Niwa.
     7
     8        TestDownloader is introduced to download W3C tests from the official
     9        W3C tests repositories (https://github.com/w3c/web-platform-tests.git
     10        and https://github.com/w3c/csswg-test.git).
     11        TestDownloader clones the repositories and checkouts revisions of the tests
     12        defined in LayoutTests/imported/w3c/resources/TestRepositories.
     13
     14        Similarly to https://codereview.chromium.org/148173016,
     15        TestDownloader copies part of the files based on the rules
     16        defined in LayoutTests/imported/w3c/resources/ImportExpectations.
     17
     18        TestDownloader is used by TestImporter when no source directory is provided.
     19
     20        * Scripts/webkitpy/w3c/test_downloader.py: Added.
     21        (TestDownloader):
     22        (TestDownloader.default_options):
     23        (TestDownloader.load_test_repositories):
     24        (TestDownloader.__init__):
     25        (TestDownloader.checkout_test_repository):
     26        (TestDownloader._init_paths_from_expectations):
     27        (TestDownloader._add_test_suite_paths):
     28        (TestDownloader._empty_directory):
     29        (TestDownloader.copy_tests):
     30        (TestDownloader.download_tests):
     31        * Scripts/webkitpy/w3c/test_importer.py:
     32        (main):
     33        (parse_args):
     34        (TestImporter.__init__):
     35        (TestImporter.do_import):
     36        (TestImporter.test_downloader):
     37        (TestImporter.find_importable_tests.should_keep_subdir):
     38        (TestImporter.should_convert_test_harness_links):
     39        (TestImporter.import_tests):
     40        (TestImporter.write_import_log):
     41
    1422015-03-12  Alexey Proskuryakov  <ap@apple.com>
    243

  • trunk/Tools/Scripts/webkitpy/w3c/test_importer.py

    r181014 r181479  
    22
    33# Copyright (C) 2013 Adobe Systems Incorporated. All rights reserved.
     4# Copyright (C) 2015 Canon Inc. All rights reserved.
    45#
    56# Redistribution and use in source and binary forms, with or without
    … …  
    2930
    3031"""
    31  This script imports a directory of W3C CSS tests into WebKit.
    32 
    33  You must have checked out the W3C repository to your local drive.
     32 This script imports W3C tests into WebKit, either from a local folder or by cloning W3C CSS and WPT repositories.
    3433
    3534 This script will import the tests into WebKit following these rules:
    3635
    37     - All tests are imported into LayoutTests/w3c
     36    - All tests are by default imported into LayoutTests/w3c
    3837
    3938    - Tests will be imported into a directory tree that
    40       mirrors the CSS Mercurial repo. For example, <csswg_repo_root>/css2.1 is brought in
    41       as LayoutTests/csswg/css2.1, maintaining the entire directory structure under that
     39      mirrors the CSS and WPT repositories. For example, <csswg_repo_root>/css2.1 should be brought in
     40      as LayoutTests/imported/w3c/csswg-tests/css2.1, maintaining the entire directory structure under that
    4241
    4342    - By default, only reftests and jstest are imported. This can be overridden with a -a or --all
    … …  
    4847      This can also be overridden by a -n or --no-overwrite flag
    4948
     49    - If no import_directory is provided, the script will download the tests from the W3C github repositories.
     50      The selection of tests and folders to import will be based on the following files:
     51         1. LayoutTests/imported/w3c/resources/TestRepositories lists the repositories to clone, the corresponding revision to checkout and the infrastructure folders that need to be imported/skipped.
     52         2. LayoutTests/imported/w3c/resources/ImportExpectations list the test suites or tests to NOT import.
     53
    5054    - All files are converted to work in WebKit:
    51          1. Paths to testharness.js files are modified point to Webkit's copy of them in
    52             LayoutTests/resources, using the correct relative path from the new location
     55         1. Paths to testharness.js files are modified to point to Webkit's copy of them in
     56            LayoutTests/resources, using the correct relative path from the new location.
     57            This is applied to CSS tests but not to WPT tests.
    5358         2. All CSS properties requiring the -webkit-vendor prefix are prefixed - this current
    5459            list of what needs prefixes is read from Source/WebCore/CSS/CSSProperties.in
    … …  
    8287from webkitpy.w3c.test_parser import TestParser
    8388from webkitpy.w3c.test_converter import convert_for_webkit
     89from webkitpy.w3c.test_downloader import TestDownloader
    8490
    8591CHANGESET_NOT_AVAILABLE = 'Not Available'
    … …  
    9096def main(_argv, _stdout, _stderr):
    9197    options, args = parse_args(_argv)
    92     import_dir = args[0]
     98    import_dir = args[0] if args else None
    9399    filesystem = FileSystem()
    94     if not filesystem.exists(import_dir):
     100    if import_dir and not filesystem.exists(import_dir):
    95101        sys.exit('Source directory %s not found!' % import_dir)
    96102
    … …  
    118124
    119125def parse_args(args):
    120     parser = argparse.ArgumentParser(prog='import-w3c-tests w3c_test_source_directory')
     126    parser = argparse.ArgumentParser(prog='import-w3c-tests [w3c_test_source_directory]')
    121127
    122128    parser.add_argument('-n', '--no-overwrite', dest='overwrite', action='store_false', default=True,
    123129        help='Flag to prevent duplicate test files from overwriting existing tests. By default, they will be overwritten')
    124130    parser.add_argument('-l', '--no-links-conversion', dest='convert_test_harness_links', action='store_false', default=True,
    125         help='Do not change links (testharness js or css e.g.). By default, links are converted to point to WebKit testharness files.')
     131       help='Do not change links (testharness js or css e.g.). This option only applies when providing a source directory, in which case by default, links are converted to point to WebKit testharness files. When tests are downloaded from W3C repository, links are converted for CSS tests and remain unchanged for WPT tests')
    126132
    127133    parser.add_argument('-a', '--all', action='store_true', default=False,
    … …  
    129135    parser.add_argument('-d', '--dest-dir', dest='destination', default='w3c',
    130136        help='Import into a specified directory relative to the LayoutTests root. By default, imports into w3c')
     137
     138    list_of_repositories = ' or '.join([test_repository['name'] for test_repository in TestDownloader.load_test_repositories()])
    131139    parser.add_argument('-t', '--test-path', action='append', dest='test_paths', default=[],
    132         help='Import only tests in the supplied subdirectory of the w3c_test_directory. Can be supplied multiple times to give multiple paths')
     140         help='Import only tests in the supplied subdirectory of the source directory. Can be supplied multiple times to give multiple paths. For tests directly cloned from W3C repositories, use ' + list_of_repositories + ' prefixes to filter specific tests')
     141
     142    parser.add_argument('-v', '--verbose', action='store_true', default=False,
     143         help='Print maximal log')
     144    parser.add_argument('--no-fetch', action='store_false', dest='fetch', default=True,
     145         help='Do not fetch the repositories. By default, repositories are fetched if a source directory is not provided')
     146    parser.add_argument('--import-all', action='store_true', default=False,
     147         help='Ignore the ImportExpectations file. All tests will be imported. This option only applies when tests are downloaded from W3C repository')
    133148
    134149    options, args = parser.parse_known_args(args)
    135     if len(args) != 1:
     150    if len(args) > 1:
    136151        parser.error('Incorrect number of arguments')
    137152    return options, args
    … …  
    151166
    152167        self.destination_directory = webkit_finder.path_from_webkit_base("LayoutTests", options.destination)
     168        self.layout_tests_w3c_path = webkit_finder.path_from_webkit_base('LayoutTests', 'imported', 'w3c')
     169        self.tests_download_path = webkit_finder.path_from_webkit_base('WebKitBuild', 'w3c-tests')
     170
     171        self._test_downloader = None
    153172
    154173        self.import_list = []
     174        self._importing_downloaded_tests = source_directory is None
    155175
    156176    def do_import(self):
    157         if len(self.options.test_paths) == 0:
     177        if not self.source_directory:
     178            _log.info('Downloading W3C test repositories')
     179            self.source_directory = self.filesystem.join(self.tests_download_path, 'to-be-imported')
     180            self.filesystem.maybe_make_directory(self.tests_download_path)
     181            self.filesystem.maybe_make_directory(self.source_directory)
     182            self.test_downloader().download_tests(self.source_directory, self.options.test_paths)
     183
     184        if not self.options.test_paths or self._importing_downloaded_tests:
    158185            self.find_importable_tests(self.source_directory)
    159186        else:
    160187            for test_path in self.options.test_paths:
    161188                self.find_importable_tests(self.filesystem.join(self.source_directory, test_path))
     189
    162190        self.import_tests()
     191
     192    def test_downloader(self):
     193        if not self._test_downloader:
     194            download_options = TestDownloader.default_options()
     195            download_options.fetch = self.options.fetch
     196            download_options.verbose = self.options.verbose
     197            download_options.import_all = self.options.import_all
     198            self._test_downloader = TestDownloader(self.tests_download_path, self.host, download_options)
     199        return self._test_downloader
    163200
    164201    def should_skip_file(self, filename):
    … …  
    172209    def find_importable_tests(self, directory):
    173210        def should_keep_subdir(filesystem, path):
     211            if self._importing_downloaded_tests:
     212                return True
    174213            subdir = path[len(directory):]
    175214            DIRS_TO_SKIP = ('work-in-progress', 'tools', 'support')
    … …  
    237276                self.import_list.append({'dirname': root, 'copy_list': copy_list,
    238277                    'reftests': reftests, 'jstests': jstests, 'total_tests': total_tests})
     278
     279    def should_convert_test_harness_links(self, test):
     280        if self._importing_downloaded_tests:
     281            for test_repository in self.test_downloader().test_repositories:
     282                if test.startswith(test_repository['name']):
     283                    return test_repository['convert_test_harness_links']
     284            return True
     285        return self.options.convert_test_harness_links
    239286
    240287    def import_tests(self):
    … …  
    304351                if 'html' in str(mimetype[0]) or 'xml' in str(mimetype[0])  or 'css' in str(mimetype[0]):
    305352                    try:
    306                         converted_file = convert_for_webkit(new_path, filename=orig_filepath, reference_support_info=reference_support_info, convert_test_harness_links=self.options.convert_test_harness_links)
     353                        converted_file = convert_for_webkit(new_path, filename=orig_filepath, reference_support_info=reference_support_info, convert_test_harness_links=self.should_convert_test_harness_links(subpath))
    307354                    except:
    308355                        _log.warn('Failed converting %s', orig_filepath)
    … …  
    383430        import_log.append('The tests in this directory were imported from the W3C repository.\n')
    384431        import_log.append('Do NOT modify these tests directly in Webkit.\n')
    385         import_log.append('Instead, push changes to the W3C CSS repo:\n')
    386         import_log.append('\thttp://hg.csswg.org/test\n')
    387         import_log.append('Or create a pull request on the W3C CSS github:\n')
    388         import_log.append('\thttps://github.com/w3c/csswg-test\n\n')
     432        import_log.append('Instead, create a pull request on the W3C CSS or WPT github:\n')
     433        import_log.append('\thttps://github.com/w3c/csswg-test\n')
     434        import_log.append('\thttps://github.com/w3c/web-platform-tests\n\n')
    389435        import_log.append('Then run the Tools/Scripts/import-w3c-tests in Webkit to reimport\n\n')
    390436        import_log.append('Do NOT modify or remove this file\n\n')
Note: See TracChangeset for help on using the changeset viewer.