Changeset 276762 in webkit


Ignore:
Timestamp:
Apr 29, 2021, 2:45:20 AM (4 years ago)
Author:
Sam Sneddon
Message:

Make sure webkitpy tests pass on Linux and on more Python versions
​https://bugs.webkit.org/show_bug.cgi?id=225157

Reviewed by Jonathan Bedard.

This gets us passing the webkitpy tests, run under pytest, on Linux,
under Python 2.7, 3.6, 3.7, 3.8, 3.9, and alpha 3.10.

  • Scripts/libraries/webkitcorepy/webkitcorepy/init.py:

Conditionally install setuptools 56 on Python 3, keep setuptools
44.1.1 on Python 2. (44.1.1 is the last release supporting Python 2,
but due to removals doesn't install on Python 3.10.)

  • Scripts/webkitpy/browserperfdash/browserperfdash_unittest.py:

(FakeBrowserDriver): Implement a FakeBrowserDriver which does nothing.
(FakeBrowserDriver.init):
(FakeBrowserDriver.prepare_env):
(FakeBrowserDriver.prepare_initial_env):
(FakeBrowserDriver.restore_env):
(FakeBrowserDriver.restore_env_after_all_testing):
(FakeBrowserDriver.close_browsers):
(FakeBrowserDriver.launch_url):
(FakeBrowserDriver.launch_webdriver):
(BrowserPerfDashRunnerTest.test_can_construct_runner_object_minimum_parameters):
Use the FakeBrowserRunner rather than the platform/browser
default. (This fixes this test on Linux when no minibrowser-gtk is
available.)

  • Scripts/webkitpy/common/checkout/scm/scm_unittest.py:

(GitTest.setUp): Set user.name/user.email to ensure they're set.
(GitSVNTest._setup_git_checkout): Set user.name/user.email to ensure
they're set.

  • Scripts/webkitpy/common/net/credentials.py: Conditionally import

keyring; the rest of the code already supports keyring being
None. (keyring doesn't have any release supporting both Python 2 and
3.10, but updating keyring pulls in new dependencies and is left as
future work.)

  • Scripts/webkitpy/common/prettypatch_unittest.py:

(PrettyPatchTest.check_ruby): Also check Ruby version.
(test_pretty_diff_encodings): Explicitly skip; don't just return.
(test_pretty_print_empty_string): Explicitly skip; don't just return.

  • Scripts/webkitpy/common/system/logtesting.py:

(TestLogStream.write): Python 3.6's logging appears to put new lines
following a message in a new message, which breaks many tests.

  • Scripts/webkitpy/pytest.ini: Broaden ignored warnings.
  • Scripts/webkitpy/results/upload_unittest.py:

(UploadTest.normalize): collections -> collections.abc for Py3.10

  • Scripts/webkitpy/xcode/simulated_device.py:

(SimulatedDeviceManager._create_device_with_runtime): Use modern
plistlib API.
(SimulatedDevice.state): Use modern plistlib API.

Location:
trunk/Tools
Files:
10 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r276744 r276762  
     12021-04-29  Sam Sneddon  <gsnedders@apple.com>
     2
     3        Make sure webkitpy tests pass on Linux and on more Python versions
     4        https://bugs.webkit.org/show_bug.cgi?id=225157
     5
     6        Reviewed by Jonathan Bedard.
     7
     8        This gets us passing the webkitpy tests, run under pytest, on Linux,
     9        under Python 2.7, 3.6, 3.7, 3.8, 3.9, and alpha 3.10.
     10
     11        * Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py:
     12        Conditionally install setuptools 56 on Python 3, keep setuptools
     13        44.1.1 on Python 2. (44.1.1 is the last release supporting Python 2,
     14        but due to removals doesn't install on Python 3.10.)
     15        * Scripts/webkitpy/browserperfdash/browserperfdash_unittest.py:
     16        (FakeBrowserDriver): Implement a FakeBrowserDriver which does nothing.
     17        (FakeBrowserDriver.__init__):
     18        (FakeBrowserDriver.prepare_env):
     19        (FakeBrowserDriver.prepare_initial_env):
     20        (FakeBrowserDriver.restore_env):
     21        (FakeBrowserDriver.restore_env_after_all_testing):
     22        (FakeBrowserDriver.close_browsers):
     23        (FakeBrowserDriver.launch_url):
     24        (FakeBrowserDriver.launch_webdriver):
     25        (BrowserPerfDashRunnerTest.test_can_construct_runner_object_minimum_parameters):
     26        Use the FakeBrowserRunner rather than the platform/browser
     27        default. (This fixes this test on Linux when no minibrowser-gtk is
     28        available.)
     29        * Scripts/webkitpy/common/checkout/scm/scm_unittest.py:
     30        (GitTest.setUp): Set user.name/user.email to ensure they're set.
     31        (GitSVNTest._setup_git_checkout): Set user.name/user.email to ensure
     32        they're set.
     33        * Scripts/webkitpy/common/net/credentials.py: Conditionally import
     34        keyring; the rest of the code already supports keyring being
     35        None. (keyring doesn't have any release supporting both Python 2 and
     36        3.10, but updating keyring pulls in new dependencies and is left as
     37        future work.)
     38        * Scripts/webkitpy/common/prettypatch_unittest.py:
     39        (PrettyPatchTest.check_ruby): Also check Ruby version.
     40        (test_pretty_diff_encodings): Explicitly skip; don't just return.
     41        (test_pretty_print_empty_string): Explicitly skip; don't just return.
     42        * Scripts/webkitpy/common/system/logtesting.py:
     43        (TestLogStream.write): Python 3.6's logging appears to put new lines
     44        following a message in a new message, which breaks many tests.
     45        * Scripts/webkitpy/pytest.ini: Broaden ignored warnings.
     46        * Scripts/webkitpy/results/upload_unittest.py:
     47        (UploadTest.normalize): collections -> collections.abc for Py3.10
     48        * Scripts/webkitpy/xcode/simulated_device.py:
     49        (SimulatedDeviceManager._create_device_with_runtime): Use modern
     50        plistlib API.
     51        (SimulatedDevice.state): Use modern plistlib API.
     52
    1532021-04-28  Devin Rousso  <drousso@apple.com>
    254

  • trunk/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/__init__.py

    r275353 r276762  
    4343if sys.version_info > (3, 0):
    4444    AutoInstall.register(Package('mock', Version(4)))
     45    AutoInstall.register(Package('setuptools', Version(56, 0, 0)))
    4546else:
    4647    AutoInstall.register(Package('mock', Version(3, 0, 5)))
     48    AutoInstall.register(Package('setuptools', Version(44, 1, 1)))
    4749    if platform.system() == 'Windows':
    4850        AutoInstall.register(Package('win_inet_pton', Version(1, 1, 0), pypi_name='win-inet-pton'))
    … …  
    5860AutoInstall.register(Package('pyparsing', Version(2, 4, 7)))
    5961AutoInstall.register(Package('requests', Version(2, 24)))
    60 AutoInstall.register(Package('setuptools', Version(44, 1, 1)))
    6162AutoInstall.register(Package('setuptools_scm', Version(5, 0, 2), pypi_name='setuptools-scm'))
    6263AutoInstall.register(Package('socks', Version(1, 7, 1), pypi_name='PySocks'))

  • trunk/Tools/Scripts/webkitpy/browserperfdash/browserperfdash_unittest.py

    r239522 r276762  
    2828import os
    2929
    30 from webkitpy.benchmark_runner.run_benchmark import default_browser, default_platform, benchmark_runner_subclasses
    3130from webkitpy.benchmark_runner.benchmark_runner import BenchmarkRunner
    32 from webkitpy.benchmark_runner.webserver_benchmark_runner import WebServerBenchmarkRunner
    33 from webkitpy.benchmark_runner.webdriver_benchmark_runner import WebDriverBenchmarkRunner
    34 from webkitpy.benchmark_runner.browser_driver.browser_driver_factory import BrowserDriverFactory
     31from webkitpy.benchmark_runner.browser_driver.browser_driver import BrowserDriver
     32from webkitpy.benchmark_runner.browser_driver.browser_driver_factory import (
     33    BrowserDriverFactory,
     34)
     35from webkitpy.benchmark_runner.run_benchmark import (
     36    benchmark_runner_subclasses,
     37)
    3538
    3639
    3740_log = logging.getLogger(__name__)
     41
     42
     43class FakeBrowserDriver(BrowserDriver):
     44    browser_name = None
     45    process_search_list = []
     46    platform = "fake"
     47
     48    def __init__(self):
     49        self.process_name = "fake/process"
     50
     51    def prepare_env(self, config):
     52        pass
     53
     54    def prepare_initial_env(self, config):
     55        pass
     56
     57    def restore_env(self):
     58        pass
     59
     60    def restore_env_after_all_testing(self):
     61        pass
     62
     63    def close_browsers(self):
     64        pass
     65
     66    def launch_url(self, url, options, browser_build_path, browser_path):
     67        pass
     68
     69    def launch_webdriver(self, url, driver):
     70        pass
     71
     72
     73BrowserDriverFactory.add_browser_driver("fake", None, FakeBrowserDriver)
    3874
    3975
    … …  
    6197        plan_list = BenchmarkRunner.available_plans()
    6298        build_dir = os.path.abspath(os.curdir)
    63         runner = FakeBenchmarkRunner(plan_list[0], False, 1, build_dir, "/tmp/testOutput.txt", default_platform(), default_browser(), None)
     99        runner = FakeBenchmarkRunner(
     100            plan_list[0], False, 1, build_dir, "/tmp/testOutput.txt", "fake", None, None
     101        )
    64102        self.assertTrue(runner.execute())

  • trunk/Tools/Scripts/webkitpy/common/checkout/scm/scm_unittest.py

    r276436 r276762  
    10231023
    10241024        os.chdir(self.untracking_checkout_path)
     1025        run_command(['git', 'config', 'user.name', 'scm_unittest'])
     1026        run_command(['git', 'config', 'user.email', 'scm_unittest@example.com'])
    10251027        write_into_file_at_path('foo_file', 'foo')
    10261028        run_command(['git', 'add', 'foo_file'])
    … …  
    10331035        run_command(['git', 'clone', '--quiet', self.untracking_checkout_path, self.tracking_git_checkout_path])
    10341036        os.chdir(self.tracking_git_checkout_path)
     1037        run_command(['git', 'config', 'user.name', 'scm_unittest'])
     1038        run_command(['git', 'config', 'user.email', 'scm_unittest@example.com'])
    10351039        self.tracking_scm = detect_scm_system(self.tracking_git_checkout_path)
    10361040
    … …  
    11751179        run_silent(['git', 'svn', 'clone', '-T', 'trunk', '--prefix', '', self.svn_repo_url, self.git_checkout_path])
    11761180        os.chdir(self.git_checkout_path)
     1181        run_command(['git', 'config', 'user.name', 'scm_unittest'])
     1182        run_command(['git', 'config', 'user.email', 'scm_unittest@example.com'])
    11771183        run_silent(['git', 'branch', '-m', 'trunk'])
    11781184

  • trunk/Tools/Scripts/webkitpy/common/net/credentials.py

    r271506 r276762  
    4242
    4343with OutputCapture():
    44     import keyring
     44    try:
     45        import keyring
     46    except ImportError:
     47        keyring = None
    4548
    4649_log = logging.getLogger(__name__)

  • trunk/Tools/Scripts/webkitpy/common/prettypatch_unittest.py

    r251955 r276762  
    3030import sys
    3131import unittest
     32from distutils.version import StrictVersion
    3233
    3334from webkitpy.common.system.executive import Executive
    … …  
    3940        executive = Executive()
    4041        try:
    41             result = executive.run_command(['ruby', '--version'])
     42            result = executive.run_command(['ruby', '-e', 'print(RUBY_VERSION)'])
    4243        except OSError as e:
    4344            return False
    44         return True
     45        # PrettyPatch relies on WEBrick, which was removed from the Ruby stdlib in 3
     46        return StrictVersion(result) < StrictVersion("3.0.0")
    4547
    4648    _diff_with_multiple_encodings = """
    … …  
    6971    def test_pretty_diff_encodings(self):
    7072        if not self.check_ruby():
     73            self.skipTest("no/unsupported Ruby")
    7174            return
    7275
    7376        if sys.platform.startswith('win'):
    74             # FIXME: disabled due to https://bugs.webkit.org/show_bug.cgi?id=93192
     77            self.skipTest("FIXME: disabled due to https://bugs.webkit.org/show_bug.cgi?id=93192")
    7578            return
    7679
    … …  
    8285    def test_pretty_print_empty_string(self):
    8386        if not self.check_ruby():
     87            self.skipTest("no/unsupported Ruby")
    8488            return
    8589

  • trunk/Tools/Scripts/webkitpy/common/system/logtesting.py

    r174136 r276762  
    6262    # http://docs.python.org/library/logging.html#module-logging.handlers
    6363    def write(self, message):
    64         self.messages.append(message)
     64        if message == "\n" and self.messages:
     65            self.messages[-1] += "\n"
     66        else:
     67            self.messages.append(message)
    6568
    6669    def flush(self):

  • trunk/Tools/Scripts/webkitpy/pytest.ini

    r276512 r276762  
    99    ignore:The 'warn' method is deprecated, use 'warning' instead:DeprecationWarning
    1010    ignore:cannot collect test class 'Test[^']*' because it has a __init__ constructor:pytest.PytestCollectionWarning
    11     ignore:the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses:DeprecationWarning
    12     ignore:Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.[0-9]+ it will stop working:DeprecationWarning
     11    ignore:the imp module is deprecated in favour of importlib.*:DeprecationWarning
     12    ignore:Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, ?and in 3.[0-9]+ it will stop working:DeprecationWarning
    1313    ignore:The readPlist function is deprecated, use load\(\) instead:DeprecationWarning
    1414    ignore:inspect.getargspec\(\) is deprecated since Python 3.0, use inspect.signature\(\) or inspect.getfullargspec\(\):DeprecationWarning
     15    ignore:The distutils package is deprecated.*:DeprecationWarning
     16    ignore:ssl module. PROTOCOL_TLS is deprecated.*:DeprecationWarning
     17    ignore:AutoInstall.find_spec() not found; falling back to find_module():ImportWarning

  • trunk/Tools/Scripts/webkitpy/results/upload_unittest.py

    r265574 r276762  
    3030import unittest
    3131
     32try:
     33    from collections.abc import Iterable, Mapping
     34except ImportError:
     35    from collections import Iterable, Mapping
     36
    3237from webkitpy.results.upload import Upload
    3338from webkitpy.thirdparty import mock
    … …  
    5661        if isinstance(data, basestring):
    5762            return str(data)
    58         elif isinstance(data, collections.Mapping):
     63        elif isinstance(data, Mapping):
    5964            return dict(map(UploadTest.normalize, data.items()))
    60         elif isinstance(data, collections.Iterable):
     65        elif isinstance(data, Iterable):
    6166            return type(data)(map(UploadTest.normalize, data))
    6267        return data

  • trunk/Tools/Scripts/webkitpy/xcode/simulated_device.py

    r276735 r276762  
    2424import json
    2525import logging
    26 import plistlib
    2726import re
    2827import time
    … …  
    3534from webkitpy.port.device import Device
    3635from webkitpy.xcode.device_type import DeviceType
     36
     37try:
     38    from plistlib import load as readPlist
     39except ImportError:
     40    from plistlib import readPlist
    3741
    3842_log = logging.getLogger(__name__)
    … …  
    107111        # Find device type. If we can't parse the device type, ignore this device.
    108112        try:
    109             device_type_string = SimulatedDeviceManager._device_identifier_to_name[plistlib.readPlist(host.filesystem.open_binary_file_for_reading(device_plist))['deviceType']]
     113            device_type_string = SimulatedDeviceManager._device_identifier_to_name[readPlist(host.filesystem.open_binary_file_for_reading(device_plist))['deviceType']]
    110114            device_type = DeviceType.from_string(device_type_string, runtime.version)
    111115            assert device_type.software_variant == runtime.os_variant
    … …  
    558562
    559563        device_plist = self.filesystem.expanduser(self.filesystem.join(SimulatedDeviceManager.simulator_device_path, self.udid, 'device.plist'))
    560         self._state = int(plistlib.readPlist(self.filesystem.open_binary_file_for_reading(device_plist))['state'])
     564        self._state = int(readPlist(self.filesystem.open_binary_file_for_reading(device_plist))['state'])
    561565        self._last_updated_state = time.time()
    562566        return self._state
Note: See TracChangeset for help on using the changeset viewer.