⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 253804 in webkit


Ignore:
Timestamp:
Dec 19, 2019, 5:06:22 PM (7 years ago)
Author:
Jonathan Bedard
Message:

Python 3: Add support to run-webkit-tests
​https://bugs.webkit.org/show_bug.cgi?id=205291

Reviewed by Stephanie Lewis.

  • Scripts/test-webkitpy-python3: Add webkitpy.layout_tests.
  • Scripts/webkitpy/common/message_pool.py:

(_MessagePool._loop): Move exception inside of loop.
(_Message.repr): Use .format strings.
(_Worker.init): Ditto.

  • Scripts/webkitpy/common/wavediff.py:

(WaveDiff.init): Use Python 3 compatible BytesIO and StringIO.

  • Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py:

(LayoutTestFinder._read_test_names_from_file): Use .format string.
(LayoutTestFinder.split_into_chunks): Explicitly use integer division.

  • Scripts/webkitpy/layout_tests/controllers/manager.py:

(Manager._get_test_inputs): Use range over xrange.

  • Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:

(SingleTestRunner._handle_error): Use .format strings.

  • Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:

(JSONLayoutResultsGenerator._insert_failure_summaries): Use Python 3 compatible itervalues.

  • Scripts/webkitpy/layout_tests/models/test_results.py:

(TestResult.init): Sort type list.

  • Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:

(RunTest.setUp): Multiple processes are buggy in test-webkitpy.
(RunTest.test_basic): replace buflist with getvalue().
(RunTest.test_child_processes_2): Ditto.
(RunTest.test_child_processes_min): Ditto.
(RunTest.test_keyboard_interrupt): Ditto.
(RunTest.test_missing_and_unexpected_results): Compare dictionaries instead of json strings.
(RunTest.test_crash_with_stderr): Ditto.
(RunTest.test_reftest_should_not_use_naming_convention_if_not_listed_in_reftestlist): Ditto.
(EndToEndTest.test_reftest_with_two_notrefs): Ditto.

  • Scripts/webkitpy/layout_tests/views/metered_stream.py:

(MeteredStream.write): Flush stream after writing.
(MeteredStream._erase_last_partial_line): Ditto.

  • Scripts/webkitpy/layout_tests/views/printing.py:

(Printer._print_directory_timings): Can't compare string to integer.
(Printer._print_statistics_for_test_timings): Use integer division.

  • Scripts/webkitpy/port/darwin.py:

(DarwinPort._merge_crash_logs): Use items over iteritems.

  • Scripts/webkitpy/port/device.py:

(Device.hash): Allow hashing of devices.

  • Scripts/webkitpy/port/device_port.py:

(DevicePort._install): Use range over xrange.
(DevicePort.setup_test_run): Ditto.
(DevicePort.clean_up_test_run): Ditto.

  • Scripts/webkitpy/port/driver.py:

(DriverOutput.init): Text should be decoded, audio encoded.
(Driver.init):
(Driver.run_test):
(Driver._parse_child_processes_output): Output is byte array.
(Driver._check_for_driver_timeout): Ditto.
(Driver._check_for_address_sanitizer_violation): Error lines are byte arrays.
(Driver._check_for_driver_crash_or_unresponsiveness): Ditto.
(Driver._read_optional_image_block): If a block is base64 encoded, we want the decoded
content, otherwise, we want the encoded content.
(Driver._read_header): Lines are byte arrays, decode them before processing.
(Driver._process_stdout_line): Blocks are byte arrays.
(Driver._strip_eof): Lines should be byte arrays, not strings.
(Driver._read_block): Standardize encoding in blocks.
(ContentBlock.init): Content should be a byte array.
(ContentBlock.decode_content): Attempt to decode content.

  • Scripts/webkitpy/port/driver_unittest.py:

(DriverTest.test_read_binary_block): Content should be encoded.
(DriverTest.test_read_base64_block): Ditto.
(DriverTest.test_check_for_driver_crash): ServerProcess output should be a byte array.

  • Scripts/webkitpy/port/image_diff.py:

(ImageDiffer.diff_image): ImageDiff output is in byte arrays.
(ImageDiffer._read): Ditto.

  • Scripts/webkitpy/port/server_process.py:

(ServerProcess.write): Encode data before writing it.

  • Scripts/webkitpy/port/server_process_mock.py:

(MockServerProcess.init): Convert string mock output to bytes.
(MockServerProcess.read_stdout_line): Stdout lines are byte arrays.
(MockServerProcess.read_stdout): Ditto.

  • Scripts/webkitpy/port/simulator_process.py:

(SimulatorProcess.NonBlockingFileFromSocket.close): Don't double close socket in Python 3.
(SimulatorProcess._start): Stdin should be a binary stream.

  • Scripts/webkitpy/port/test.py:

(unit_test_list): Convert audio streams to byte arrays.

  • Scripts/webkitpy/xcode/simulated_device.py:

(SimulatedDevice.is_usable): Decode xcrun output.
(SimulatedDevice.install_app): Use xrange over range.

Location:
trunk/Tools
Files:
23 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r253786 r253804  
     12019-12-19  Jonathan Bedard  <jbedard@apple.com>
     2
     3        Python 3: Add support to run-webkit-tests
     4        https://bugs.webkit.org/show_bug.cgi?id=205291
     5
     6        Reviewed by Stephanie Lewis.
     7
     8        * Scripts/test-webkitpy-python3: Add webkitpy.layout_tests.
     9        * Scripts/webkitpy/common/message_pool.py:
     10        (_MessagePool._loop): Move exception inside of loop.
     11        (_Message.__repr__): Use .format strings.
     12        (_Worker.__init__): Ditto.
     13        * Scripts/webkitpy/common/wavediff.py:
     14        (WaveDiff.__init__): Use Python 3 compatible BytesIO and StringIO.
     15        * Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py:
     16        (LayoutTestFinder._read_test_names_from_file): Use .format string.
     17        (LayoutTestFinder.split_into_chunks): Explicitly use integer division.
     18        * Scripts/webkitpy/layout_tests/controllers/manager.py:
     19        (Manager._get_test_inputs): Use range over xrange.
     20        * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
     21        (SingleTestRunner._handle_error): Use .format strings.
     22        * Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:
     23        (JSONLayoutResultsGenerator._insert_failure_summaries): Use Python 3 compatible itervalues.
     24        * Scripts/webkitpy/layout_tests/models/test_results.py:
     25        (TestResult.__init__): Sort type list.
     26        * Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
     27        (RunTest.setUp): Multiple processes are buggy in test-webkitpy.
     28        (RunTest.test_basic): replace buflist with getvalue().
     29        (RunTest.test_child_processes_2): Ditto.
     30        (RunTest.test_child_processes_min): Ditto.
     31        (RunTest.test_keyboard_interrupt): Ditto.
     32        (RunTest.test_missing_and_unexpected_results): Compare dictionaries instead of json strings.
     33        (RunTest.test_crash_with_stderr): Ditto.
     34        (RunTest.test_reftest_should_not_use_naming_convention_if_not_listed_in_reftestlist): Ditto.
     35        (EndToEndTest.test_reftest_with_two_notrefs): Ditto.
     36        * Scripts/webkitpy/layout_tests/views/metered_stream.py:
     37        (MeteredStream.write): Flush stream after writing.
     38        (MeteredStream._erase_last_partial_line): Ditto.
     39        * Scripts/webkitpy/layout_tests/views/printing.py:
     40        (Printer._print_directory_timings): Can't compare string to integer.
     41        (Printer._print_statistics_for_test_timings): Use integer division.
     42        * Scripts/webkitpy/port/darwin.py:
     43        (DarwinPort._merge_crash_logs): Use items over iteritems.
     44        * Scripts/webkitpy/port/device.py:
     45        (Device.__hash__): Allow hashing of devices.
     46        * Scripts/webkitpy/port/device_port.py:
     47        (DevicePort._install): Use range over xrange.
     48        (DevicePort.setup_test_run): Ditto.
     49        (DevicePort.clean_up_test_run): Ditto.
     50        * Scripts/webkitpy/port/driver.py:
     51        (DriverOutput.__init__): Text should be decoded, audio encoded.
     52        (Driver.__init__):
     53        (Driver.run_test):
     54        (Driver._parse_child_processes_output): Output is byte array.
     55        (Driver._check_for_driver_timeout): Ditto.
     56        (Driver._check_for_address_sanitizer_violation): Error lines are byte arrays.
     57        (Driver._check_for_driver_crash_or_unresponsiveness): Ditto.
     58        (Driver._read_optional_image_block): If a block is base64 encoded, we want the decoded
     59        content, otherwise, we want the encoded content.
     60        (Driver._read_header): Lines are byte arrays, decode them before processing.
     61        (Driver._process_stdout_line): Blocks are byte arrays.
     62        (Driver._strip_eof): Lines should be byte arrays, not strings.
     63        (Driver._read_block): Standardize encoding in blocks.
     64        (ContentBlock.__init__): Content should be a byte array.
     65        (ContentBlock.decode_content): Attempt to decode content.
     66        * Scripts/webkitpy/port/driver_unittest.py:
     67        (DriverTest.test_read_binary_block): Content should be encoded.
     68        (DriverTest.test_read_base64_block): Ditto.
     69        (DriverTest.test_check_for_driver_crash): ServerProcess output should be a byte array.
     70        * Scripts/webkitpy/port/image_diff.py:
     71        (ImageDiffer.diff_image): ImageDiff output is in byte arrays.
     72        (ImageDiffer._read): Ditto.
     73        * Scripts/webkitpy/port/server_process.py:
     74        (ServerProcess.write): Encode data before writing it.
     75        * Scripts/webkitpy/port/server_process_mock.py:
     76        (MockServerProcess.__init__): Convert string mock output to bytes.
     77        (MockServerProcess.read_stdout_line): Stdout lines are byte arrays.
     78        (MockServerProcess.read_stdout): Ditto.
     79        * Scripts/webkitpy/port/simulator_process.py:
     80        (SimulatorProcess.NonBlockingFileFromSocket.close): Don't double close socket in Python 3.
     81        (SimulatorProcess._start): Stdin should be a binary stream.
     82        * Scripts/webkitpy/port/test.py:
     83        (unit_test_list): Convert audio streams to byte arrays.
     84        * Scripts/webkitpy/xcode/simulated_device.py:
     85        (SimulatedDevice.is_usable): Decode xcrun output.
     86        (SimulatedDevice.install_app): Use xrange over range.
     87
    1882019-12-19  Matt Lewis  <jlewis3@apple.com>
    289

  • trunk/Tools/Scripts/test-webkitpy-python3

    r253392 r253804  
    3838  'webkitpy.browserperfdash',
    3939  'webkitpy.common',
    40   'webkitpy.layout_tests.controllers',
    41   'webkitpy.layout_tests.layout_package',
    42   'webkitpy.layout_tests.models',
    43   'webkitpy.layout_tests.servers',
    44   'webkitpy.layout_tests.views',
     40  'webkitpy.layout_tests',
    4541  'webkitpy.performance_tests',
    4642  'webkitpy.port',

  • trunk/Tools/Scripts/webkitpy/common/message_pool.py

    r252616 r253804  
    186186
    187187    def _loop(self, block):
    188         try:
    189             while True:
    190                 if len(self._workers_stopped) == len(self._workers):
    191                     block = False
     188        while True:
     189            if len(self._workers_stopped) == len(self._workers):
     190                block = False
     191
     192            try:
    192193                message = self._messages_to_manager.get(block)
    193                 self._log_messages(message.logs)
    194                 if message.from_user:
    195                     self._caller.handle(message.name, message.src, *message.args)
    196                     continue
    197                 method = getattr(self, '_handle_' + message.name)
    198                 assert method, 'bad message %s' % repr(message)
    199                 method(message.src, *message.args)
    200         except queue.Empty:
    201             pass
     194            except queue.Empty:
     195                break
     196
     197            self._log_messages(message.logs)
     198            if message.from_user:
     199                self._caller.handle(message.name, message.src, *message.args)
     200                continue
     201            method = getattr(self, '_handle_' + message.name)
     202            assert method, 'bad message %s' % repr(message)
     203            method(message.src, *message.args)
    202204
    203205
    … …  
    216218
    217219    def __repr__(self):
    218         return '_Message(src=%s, name=%s, args=%s, from_user=%s, logs=%s)' % (self.src, self.name, self.args, self.from_user, self.logs)
     220        return '_Message(src={src}, name={name}, args={args}, from_user={from_user}, logs={logs})'.format(
     221            src=self.src,
     222            name=self.name,
     223            args=self.args,
     224            from_user=self.from_user,
     225            logs=self.logs,
     226        )
    219227
    220228
    … …  
    224232        self.host = host
    225233        self.worker_number = worker_number
    226         self.name = 'worker/%d' % worker_number
     234        self.name = 'worker/{}'.format(worker_number)
    227235        self.log_messages = []
    228236        self.log_level = log_level

  • trunk/Tools/Scripts/webkitpy/common/wavediff.py

    r250375 r253804  
    2626import wave
    2727
    28 from io import BytesIO
     28from webkitpy.common.unicode_compatibility import BytesIO, StringIO
    2929
    3030
    … …  
    3636
    3737    def __init__(self, in1, in2):
    38         if isinstance(in1, file):
    39             waveFile1 = wave.open(in1, 'rb')
     38        if isinstance(in1, str):
     39            waveFile1 = wave.open(StringIO(in1), 'r')
    4040        else:
    4141            waveFile1 = wave.open(BytesIO(in1), 'rb')
    42         if isinstance(in2, file):
    43             waveFile1 = wave.open(in2, 'rb')
     42        if isinstance(in2, str):
     43            waveFile2 = wave.open(StringIO(in2), 'r')
    4444        else:
    4545            waveFile2 = wave.open(BytesIO(in2), 'rb')
    … …  
    6767        allData1 = self._readSamples(waveFile1, sampleWidth1, frameCount1 * channelCount1)
    6868        allData2 = self._readSamples(waveFile2, sampleWidth2, frameCount2 * channelCount2)
    69         results = map(self._diffSample, allData1, allData2, xrange(max(frameCount1 * channelCount1, frameCount2 * channelCount2)))
     69        results = list(map(self._diffSample, allData1, allData2, range(max(frameCount1 * channelCount1, frameCount2 * channelCount2))))
    7070
    7171        cumulativeSampleDiff = sum(results)
    72         differingSampleCount = len(filter(bool, results))
     72        differingSampleCount = len(list(filter(bool, results)))
    7373        self._filesAreIdentical = not differingSampleCount
    74         self._filesAreIdenticalWithinTolerance = not len(filter(lambda x: x > self._tolerance, results))
     74        self._filesAreIdenticalWithinTolerance = not len(list(filter(lambda x: x > self._tolerance, results)))
    7575
    7676        if differingSampleCount:

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py

    r244701 r253804  
    103103                if e.errno == errno.ENOENT:
    104104                    _log.critical('')
    105                     _log.critical('--test-list file "%s" not found' % file)
     105                    _log.critical('--test-list file "{}" not found'.format(filenames))
    106106                raise
    107107        return tests
    … …  
    177177                rounded_tests = (num_tests + test_size - (num_tests % test_size))
    178178
    179             chunk_len = rounded_tests / test_size
     179            chunk_len = rounded_tests // test_size
    180180            slice_start = chunk_len * (chunk_num - 1)
    181181            # It does not mind if we go over test_size.

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py

    r252443 r253804  
    146146    def _get_test_inputs(self, tests_to_run, repeat_each, iterations, device_type=None):
    147147        test_inputs = []
    148         for _ in xrange(iterations):
     148        for _ in range(iterations):
    149149            for test in tests_to_run:
    150                 for _ in xrange(repeat_each):
     150                for _ in range(repeat_each):
    151151                    test_inputs.append(self._test_input_for_file(test, device_type=device_type))
    152152        return test_inputs

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py

    r239989 r253804  
    3232import time
    3333
     34from webkitpy.common.unicode_compatibility import decode_for
    3435from webkitpy.layout_tests.controllers import test_result_writer
    3536from webkitpy.port.driver import DriverInput, DriverOutput
    … …  
    215216            _log.debug("%s %s output stderr lines:" % (self._worker_name, testname))
    216217        for line in driver_output.error.splitlines():
    217             _log.debug("  %s" % line)
     218            _log.debug("  {}".format(decode_for(line, str)))
    218219        return failures
    219220

  • trunk/Tools/Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py

    r240150 r253804  
    3232from webkitpy.layout_tests.models import test_expectations
    3333from webkitpy.layout_tests.models import test_failures
     34from webkitpy.common.iteration_compatibility import itervalues
    3435
    3536
    … …  
    133134
    134135        num_fixable = 0
    135         for expectation in self._expectations.itervalues():
     136        for expectation in itervalues(self._expectations):
    136137            num_fixable += len(expectation.model().get_tests_with_timeline(test_expectations.NOW))
    137138

  • trunk/Tools/Scripts/webkitpy/layout_tests/models/test_results.py

    r251112 r253804  
    4848        self.test_run_time = test_run_time or 0  # The time taken to execute the test itself.
    4949        self.has_stderr = has_stderr
    50         self.reftest_type = reftest_type or []
     50        self.reftest_type = sorted(reftest_type or [])
    5151        self.pid = pid
    5252        self.references = references or []

  • trunk/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py

    r253219 r253804  
    3030
    3131import json
     32import sys
    3233import unittest
    3334
     35from webkitpy.common.unicode_compatibility import StringIO
    3436from webkitpy.common.system import outputcapture, path
    3537from webkitpy.common.system.crashlogs_unittest import make_mock_crash_report_darwin
    … …  
    177179        # FIXME: Remove this when we fix test-webkitpy to work
    178180        # properly on cygwin (bug 63846).
    179         self.should_test_processes = not self._platform.is_win()
     181        # FIXME: Multiprocessing doesn't do well when nested in Python 3 (https://bugs.webkit.org/show_bug.cgi?id=205280)
     182        self.should_test_processes = not self._platform.is_win() and sys.version_info < (3, 0)
    180183
    181184    def test_basic(self):
    … …  
    196199            details.initial_results.total - details.initial_results.expected_skips - len(details.initial_results.unexpected_results_by_name),
    197200            len(details.initial_results.unexpected_results_by_name))
    198         self.assertTrue(one_line_summary in logging_stream.buflist)
     201        self.assertTrue(one_line_summary in logging_stream.getvalue())
    199202
    200203        # Ensure the results were summarized properly.
    … …  
    220223            _, regular_output, _ = logging_run(
    221224                ['--debug-rwt-logging', '--child-processes', '2'], shared_port=False)
    222             self.assertTrue(any(['Running 2 ' in line for line in regular_output.buflist]))
     225            self.assertTrue(any(['Running 2 ' in line for line in regular_output.getvalue().splitlines()]))
    223226
    224227    def test_child_processes_min(self):
    … …  
    227230                ['--debug-rwt-logging', '--child-processes', '2', '-i', 'passes/passes', 'passes'],
    228231                tests_included=True, shared_port=False)
    229             self.assertTrue(any(['Running 1 ' in line for line in regular_output.buflist]))
     232            self.assertTrue(any(['Running 1 ' in line for line in regular_output.getvalue().splitlines()]))
    230233
    231234    def test_dryrun(self):
    … …  
    273276        if self.should_test_processes:
    274277            _, regular_output, _ = logging_run(['failures/expected/keyboard.html', 'passes/text.html', '--child-processes', '2', '--force'], tests_included=True, shared_port=False)
    275             self.assertTrue(any(['Interrupted, exiting' in line for line in regular_output.buflist]))
     278            self.assertTrue(any(['Interrupted, exiting' in line for line in regular_output.getvalue().splitlines()]))
    276279
    277280    def test_no_tests_found(self):
    … …  
    477480        file_list = host.filesystem.written_files.keys()
    478481        self.assertEqual(details.exit_code, 1)
    479         expected_token = '"unexpected":{"text-image-checksum.html":{"report":"REGRESSION","expected":"PASS","actual":"IMAGE+TEXT","image_diff_percent":1},"missing_text.html":{"report":"MISSING","expected":"PASS","is_missing_text":true,"actual":"MISSING"}'
    480         json_string = host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')
    481         self.assertTrue(json_string.find(expected_token) != -1)
    482         self.assertTrue(json_string.find('"num_regressions":1') != -1)
    483         self.assertTrue(json_string.find('"num_flaky":0') != -1)
    484         self.assertTrue(json_string.find('"num_missing":1') != -1)
     482        expected_dictionary = {
     483            'version': 4,
     484            'fixable': 3,
     485            'skipped': 0,
     486            'num_passes': 0,
     487            'num_flaky': 0,
     488            'num_missing': 1,
     489            'num_regressions': 1,
     490            'uses_expectations_file': True,
     491            'interrupted': False,
     492            'layout_tests_dir': '/test.checkout/LayoutTests',
     493            'has_pretty_patch': False,
     494            'pixel_tests_enabled': True,
     495            'other_crashes': {},
     496            'date': '10:10AM on December 13, 2019',
     497            'tests': {
     498                'failures': {
     499                    'expected': {
     500                        'missing_image.html': {
     501                            'expected': 'PASS MISSING',
     502                            'actual': 'MISSING',
     503                            'is_missing_image': True,
     504                        },
     505                    }, 'unexpected': {
     506                        'missing_text.html': {
     507                            'report': 'MISSING',
     508                            'expected': 'PASS',
     509                            'actual': 'MISSING',
     510                            'is_missing_text': True,
     511                        }, 'text-image-checksum.html': {
     512                            'report': 'REGRESSION',
     513                            'expected': 'PASS',
     514                            'actual': 'IMAGE+TEXT',
     515                            'image_diff_percent': 1,
     516                        },
     517                    },
     518                },
     519            },
     520        }
     521        actual_dictionary = json.loads(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')[len('ADD_RESULTS('):-2])
     522        self.assertEqual(
     523            sorted(list(expected_dictionary['tests']['failures']['expected'])),
     524            sorted(list(actual_dictionary['tests']['failures']['expected'])),
     525        )
     526        self.assertEqual(
     527            sorted(list(expected_dictionary['tests']['failures']['unexpected'])),
     528            sorted(list(actual_dictionary['tests']['failures']['unexpected'])),
     529        )
     530        self.assertEqual(expected_dictionary['num_regressions'], actual_dictionary['num_regressions'])
     531        self.assertEqual(expected_dictionary['num_flaky'], actual_dictionary['num_flaky'])
     532        self.assertEqual(expected_dictionary['num_missing'], actual_dictionary['num_missing'])
    485533
    486534    def test_pixel_test_directories(self):
    … …  
    518566        host = MockHost()
    519567        _, regular_output, _ = logging_run(['failures/unexpected/crash-with-stderr.html'], tests_included=True, host=host)
    520         self.assertTrue(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json').find('{"crash-with-stderr.html":{"report":"REGRESSION","expected":"PASS","actual":"CRASH","has_stderr":true}}') != -1)
     568        actual_dictionary = json.loads(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')[len('ADD_RESULTS('):-2])
     569        expected_dictionary = {
     570            'version': 4,
     571            'fixable': 1,
     572            'skipped': 0,
     573            'num_passes': 0,
     574            'num_flaky': 0,
     575            'num_missing': 0,
     576            'num_regressions': 1,
     577            'uses_expectations_file': True,
     578            'interrupted': False,
     579            'layout_tests_dir': '/test.checkout/LayoutTests',
     580            'has_pretty_patch': False,
     581            'pixel_tests_enabled': True,
     582            'other_crashes': {},
     583            'date': '10:18AM on December 13, 2019',
     584            'tests': {
     585                'failures': {
     586                    'unexpected': {
     587                        'crash-with-stderr.html': {
     588                            'has_stderr': True,
     589                            'report': 'REGRESSION',
     590                            'expected': 'PASS',
     591                            'actual': 'CRASH',
     592                        },
     593                    },
     594                },
     595            },
     596        }
     597        self.assertEqual(
     598            sorted(list(expected_dictionary['tests']['failures']['unexpected'])),
     599            sorted(list(actual_dictionary['tests']['failures']['unexpected'])),
     600        )
    521601
    522602    def test_no_image_failure_with_image_diff(self):
    … …  
    741821        host = MockHost()
    742822        _, err, _ = logging_run(['--no-show-results', 'reftests/foo/'], tests_included=True, host=host)
    743         json_string = host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')
    744         self.assertTrue(json_string.find('"unlistedtest.html":{"report":"MISSING","expected":"PASS","is_missing_text":true,"actual":"MISSING","is_missing_image":true}') != -1)
    745         self.assertTrue(json_string.find('"num_regressions":4') != -1)
    746         self.assertTrue(json_string.find('"num_flaky":0') != -1)
    747         self.assertTrue(json_string.find('"num_missing":1') != -1)
     823        expected_dictionary = {
     824            'version': 4,
     825            'fixable': 5,
     826            'skipped': 0,
     827            'num_passes': 3,
     828            'num_flaky': 0,
     829            'num_missing': 1,
     830            'num_regressions': 4,
     831            'uses_expectations_file': True,
     832            'interrupted': False,
     833            'layout_tests_dir': '/test.checkout/LayoutTests',
     834            'has_pretty_patch': False,
     835            'pixel_tests_enabled': True,
     836            'other_crashes': {},
     837            'date': '10:27AM on December 13, 2019',
     838            'tests': {
     839                'reftests': {
     840                    'foo': {
     841                        'multiple-both-failure.html': {
     842                            'reftest_type': ['!=', '=='],
     843                            'report': 'REGRESSION',
     844                            'expected': 'PASS',
     845                            'actual': 'IMAGE',
     846                        }, 'multiple-match-failure.html': {
     847                            'reftest_type': ['=='],
     848                            'report': 'REGRESSION',
     849                            'expected': 'PASS',
     850                            'actual': 'IMAGE',
     851                            'image_diff_percent': 1,
     852                        }, 'multiple-mismatch-failure.html': {
     853                            'reftest_type': ['!='],
     854                            'report': 'REGRESSION',
     855                            'expected': 'PASS',
     856                            'actual': 'IMAGE',
     857                        }, 'test.html': {
     858                            'reftest_type': ['=='],
     859                            'report': 'REGRESSION',
     860                            'expected': 'PASS',
     861                            'actual': 'IMAGE',
     862                            'image_diff_percent': None,
     863                        }, 'unlistedtest.html': {
     864                            'report': 'MISSING',
     865                            'expected': 'PASS',
     866                            'actual': 'MISSING',
     867                            'is_missing_text': True,
     868                            'is_missing_image': True,
     869                        },
     870                    },
     871                },
     872            },
     873        }
     874        actual_dictionary = json.loads(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')[len('ADD_RESULTS('):-2])
     875
     876        self.assertEqual(
     877            expected_dictionary['tests']['reftests']['foo']['unlistedtest.html'],
     878            actual_dictionary['tests']['reftests']['foo']['unlistedtest.html'],
     879        )
     880        self.assertEqual(expected_dictionary['num_regressions'], actual_dictionary['num_regressions'])
     881        self.assertEqual(expected_dictionary['num_flaky'], actual_dictionary['num_flaky'])
     882        self.assertEqual(expected_dictionary['num_missing'], actual_dictionary['num_missing'])
    748883
    749884    def test_additional_platform_directory(self):
    … …  
    9491084        self.assertTrue("multiple-mismatch-success.html" not in json["tests"]["reftests"]["foo"])
    9501085        self.assertTrue("multiple-both-success.html" not in json["tests"]["reftests"]["foo"])
    951         self.assertEqual(json["tests"]["reftests"]["foo"]["multiple-match-failure.html"],
    952             {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["=="], "image_diff_percent": 1, "report": "REGRESSION"})
    953         self.assertEqual(json["tests"]["reftests"]["foo"]["multiple-mismatch-failure.html"],
    954             {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["!="], "report": "REGRESSION"})
    955         self.assertEqual(json["tests"]["reftests"]["foo"]["multiple-both-failure.html"],
    956             {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["==", "!="], "report": "REGRESSION"})
     1086        self.assertEqual(
     1087            json["tests"]["reftests"]["foo"]["multiple-match-failure.html"],
     1088            {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["=="], "image_diff_percent": 1, "report": "REGRESSION"},
     1089        )
     1090        self.assertEqual(
     1091            json["tests"]["reftests"]["foo"]["multiple-mismatch-failure.html"],
     1092            {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["!="], "report": "REGRESSION"},
     1093        )
     1094        self.assertEqual(
     1095            json["tests"]["reftests"]["foo"]["multiple-both-failure.html"],
     1096            {"expected": "PASS", "actual": "IMAGE", "reftest_type": sorted(["==", "!="]), "report": "REGRESSION"},
     1097        )
    9571098
    9581099

  • trunk/Tools/Scripts/webkitpy/layout_tests/views/metered_stream.py

    r252560 r253804  
    110110        try:
    111111            self._stream.write(timestamp_string + txt)
     112            self._stream.flush()
    112113        except UnicodeEncodeError:
    113114            output = ''
    … …  
    118119                    output += '?'
    119120            self._stream.write(output)
     121            self._stream.flush()
    120122
    121123    def writeln(self, txt, now=None, pid=None):
    … …  
    126128        self._stream.write(self._erasure(self._last_partial_line))
    127129        self._last_partial_line = ''
     130        self._stream.flush()
    128131
    129132    def flush(self):

  • trunk/Tools/Scripts/webkitpy/layout_tests/views/printing.py

    r248270 r253804  
    240240        min_seconds_to_print = 10
    241241        for timing in timings:
    242             if timing[0] > min_seconds_to_print:
     242            if timing[1] > min_seconds_to_print:
    243243                self._print_debug("  %s took %s seconds to run %s tests." % timing)
    244244        self._print_debug("")
    … …  
    255255
    256256        if num_tests % 2 == 1:
    257             median = timings[((num_tests - 1) / 2) - 1]
    258         else:
    259             lower = timings[num_tests / 2 - 1]
    260             upper = timings[num_tests / 2]
     257            median = timings[(num_tests - 1) // 2 - 1]
     258        else:
     259            lower = timings[num_tests // 2 - 1]
     260            upper = timings[num_tests // 2]
    261261            median = (float(lower + upper)) / 2
    262262

  • trunk/Tools/Scripts/webkitpy/port/darwin.py

    r252779 r253804  
    9999
    100100    def _merge_crash_logs(self, logs, new_logs, crashed_processes):
    101         for test, crash_log in new_logs.iteritems():
     101        for test, crash_log in new_logs.items():
    102102            try:
    103103                if test.split('-')[0] == 'Sandbox':

  • trunk/Tools/Scripts/webkitpy/port/device.py

    r248270 r253804  
    117117    def __repr__(self):
    118118        return u'{}'.format(self.platform_device)
     119
     120    def __hash__(self):
     121        return hash(self.udid)

  • trunk/Tools/Scripts/webkitpy/port/device_port.py

    r248270 r253804  
    9696            return
    9797
    98         for i in xrange(self.child_processes()):
     98        for i in range(self.child_processes()):
    9999            device = self.target_host(i)
    100100            _log.debug(u'Installing to {}'.format(device))
    … …  
    187187        self._install()
    188188
    189         for i in xrange(self.child_processes()):
     189        for i in range(self.child_processes()):
    190190            host = self.target_host(i)
    191191            host.prepare_for_testing(
    … …  
    202202        # Failure to teardown devices can leave things in a bad state.
    203203        exception_list = []
    204         for i in xrange(self.child_processes()):
     204        for i in range(self.child_processes()):
    205205            device = self.target_host(i)
    206206            if not device:

  • trunk/Tools/Scripts/webkitpy/port/driver.py

    r252948 r253804  
    4040from webkitpy.common.system import path
    4141from webkitpy.common.system.profiler import ProfilerFactory
     42from webkitpy.common.unicode_compatibility import encode_if_necessary, decode_for
    4243
    4344
    … …  
    8788            crashed_pid=None, crash_log=None, pid=None):
    8889        # FIXME: Args could be renamed to better clarify what they do.
    89         self.text = text
     90        self.text = decode_for(text, str) if text else None
    9091        self.image = image  # May be empty-string if the test crashes.
    9192        self.image_hash = image_hash
    9293        self.image_diff = None  # image_diff gets filled in after construction.
    93         self.audio = audio  # Binary format is port-dependent.
     94        self.audio = encode_if_necessary(audio) if audio else None  # Binary format is port-dependent.
    9495        self.crash = crash
    9596        self.crashed_process_name = crashed_process_name
    … …  
    162163        # FIXME: We should probably remove _read_first_block and _read_optional_image_block and
    163164        # instead scope these locally in run_test.
    164         self.error_from_test = str()
     165        self.error_from_test = ''
    165166        self.err_seen_eof = False
    166167
    … …  
    197198        self._driver_timed_out = False
    198199        self._crash_report_from_driver = None
    199         self.error_from_test = str()
     200        self.error_from_test = ''
    200201        self.err_seen_eof = False
    201202
    … …  
    215216        text, audio = self._read_first_block(deadline, driver_input.test_name)  # First block is either text or audio
    216217        image, actual_image_hash = self._read_optional_image_block(deadline, driver_input.test_name)  # The second (optional) block is image data.
     218        text = decode_for(text, str)
    217219
    218220        crashed = self.has_crashed()
    … …  
    228230            out, err = self._server_process.stop(self._port.driver_stop_timeout() if stop_when_done else 0.0)
    229231            if out:
    230                 text += out
     232                text += decode_for(out, str)
    231233            if err:
    232                 self.error_from_test += err
     234                self.error_from_test += decode_for(err, str)
    233235            self._server_process = None
    234236
    … …  
    281283
    282284        for line in output.splitlines():
    283             m = re.match('^([^:]+): ([0-9]+)$', line)
     285            m = re.match(b'^([^:]+): ([0-9]+)$', line)
    284286            if m:
    285                 process_name = m.group(1)
    286                 process_id = m.group(2)
     287                process_name = decode_for(m.group(1), str)
     288                process_id = decode_for(m.group(2), str)
    287289                child_processes[process_name].append(process_id)
    288290
    … …  
    541543
    542544    def _check_for_driver_timeout(self, out_line):
    543         if out_line.startswith("#PID UNRESPONSIVE - "):
    544             match = re.match('#PID UNRESPONSIVE - (\S+)', out_line)
    545             child_process_name = match.group(1) if match else 'WebProcess'
    546             match = re.search('pid (\d+)', out_line)
     545        if out_line.startswith(b"#PID UNRESPONSIVE - "):
     546            match = re.match(b'#PID UNRESPONSIVE - (\S+)', out_line)
     547            child_process_name = decode_for(match.group(1), str) if match else 'WebProcess'
     548            match = re.search(b'pid (\d+)', out_line)
    547549            child_process_pid = int(match.group(1)) if match else None
    548550            err_line = 'Wait on notifyDone timed out, process ' + child_process_name + ' pid = ' + str(child_process_pid)
    … …  
    555557
    556558    def _check_for_address_sanitizer_violation(self, error_line):
    557         if "ERROR: AddressSanitizer" in error_line:
     559        if b"ERROR: AddressSanitizer" in error_line:
    558560            return True
    559561
    560562    def _check_for_driver_crash_or_unresponsiveness(self, error_line):
    561         crashed_check = error_line.rstrip('\r\n')
    562         if crashed_check == "#CRASHED":
     563        crashed_check = error_line.rstrip(b'\r\n')
     564        if crashed_check == b"#CRASHED":
    563565            self._crashed_process_name = self._server_process.process_name()
    564566            self._crashed_pid = self._server_process.system_pid()
    565567            return True
    566         elif error_line.startswith("#CRASHED - "):
    567             match = re.match('#CRASHED - (\S+)', error_line)
    568             self._crashed_process_name = match.group(1) if match else 'WebProcess'
    569             match = re.search('pid (\d+)', error_line)
     568        elif error_line.startswith(b"#CRASHED - "):
     569            match = re.match(b'#CRASHED - (\S+)', error_line)
     570            self._crashed_process_name = decode_for(match.group(1), str) if match else 'WebProcess'
     571            match = re.search(b'pid (\d+)', error_line)
    570572            self._crashed_pid = int(match.group(1)) if match else None
    571573            _log.debug('%s crash, pid = %s' % (self._crashed_process_name, str(self._crashed_pid)))
    572574            return True
    573         elif error_line.startswith("#PROCESS UNRESPONSIVE - "):
    574             match = re.match('#PROCESS UNRESPONSIVE - (\S+)', error_line)
    575             child_process_name = match.group(1) if match else 'WebProcess'
    576             match = re.search('pid (\d+)', error_line)
     575        elif error_line.startswith(b"#PROCESS UNRESPONSIVE - "):
     576            match = re.match(b'#PROCESS UNRESPONSIVE - (\S+)', error_line)
     577            child_process_name = decode_for(match.group(1), str) if match else 'WebProcess'
     578            match = re.search(b'pid (\d+)', error_line)
    577579            child_process_pid = int(match.group(1)) if match else None
    578580            _log.debug('%s is unresponsive, pid = %s' % (child_process_name, str(child_process_pid)))
    … …  
    580582            if child_process_pid:
    581583                self._port.sample_process(child_process_name, child_process_pid, self._target_host)
    582             self.error_from_test += error_line
     584            self.error_from_test += decode_for(error_line, str)
    583585            self._server_process.write('#SAMPLE FINISHED\n', True)  # Must be able to ignore a broken pipe here, target process may already be closed.
    584586            return True
    … …  
    629631        block = self._read_block(deadline, test_name, wait_for_stderr_eof=True)
    630632        if block.content and block.content_type == 'image/png':
    631             return (block.decoded_content, block.content_hash)
     633            return (block.decoded_content if block.encoding == 'base64' else block.content, block.content_hash)
    632634        return (None, block.content_hash)
    633635
    634636    def _read_header(self, block, line, header_text, header_attr, header_filter=None):
    635637        if line.startswith(header_text) and getattr(block, header_attr) is None:
    636             value = line.split()[1]
     638            value = decode_for(line.split()[1], str)
    637639            if header_filter:
    638640                value = header_filter(value)
    … …  
    642644
    643645    def _process_stdout_line(self, block, line):
    644         if (self._read_header(block, line, 'Content-Type: ', 'content_type')
    645             or self._read_header(block, line, 'Content-Transfer-Encoding: ', 'encoding')
    646             or self._read_header(block, line, 'Content-Length: ', '_content_length', int)
    647             or self._read_header(block, line, 'ActualHash: ', 'content_hash')
    648             or self._read_header(block, line, 'DumpMalloc: ', 'malloc')
    649             or self._read_header(block, line, 'DumpJSHeap: ', 'js_heap')):
    650             return
     646        for header in [
     647            (b'Content-Type: ', 'content_type', None),
     648            (b'Content-Transfer-Encoding: ', 'encoding', None),
     649            (b'Content-Length: ', '_content_length', int),
     650            (b'ActualHash: ', 'content_hash', None),
     651            (b'DumpMalloc: ', 'malloc', None),
     652            (b'DumpJSHeap: ', 'js_heap', None),
     653        ]:
     654            if self._read_header(block, line, header[0], header[1], header[2]):
     655                return
     656
    651657        # Note, we're not reading ExpectedHash: here, but we could.
    652658        # If the line wasn't a header, we just append it to the content.
    653         block.content += line
     659        block.content = encode_if_necessary(block.content) + line
    654660
    655661    def _strip_eof(self, line):
    656         if line and line.endswith("#EOF\n"):
     662        if line and line.endswith(b"#EOF\n"):
    657663            return line[:-5], True
    658664        return line, False
    … …  
    689695            if out_line:
    690696                self._check_for_driver_timeout(out_line)
    691                 if out_line[-1] != "\n":
     697                if out_line[-1] != '\n' and out_line[-1] != 10:
    692698                    _log.error("  %s -> Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug." % test_name)
    693699                content_length_before_header_check = block._content_length
    … …  
    696702                # Don't wait until we're done with headers, just read the binary blob right now.
    697703                if content_length_before_header_check != block._content_length:
    698                     block.content = self._server_process.read_stdout(deadline, block._content_length)
     704                    block.content = encode_if_necessary(self._server_process.read_stdout(deadline, block._content_length))
    699705
    700706            if err_line:
    … …  
    703709                elif self._check_for_address_sanitizer_violation(err_line):
    704710                    asan_violation_detected = True
    705                     self._crash_report_from_driver = ""
     711                    self._crash_report_from_driver = b''
    706712                    # ASan report starts with a nondescript line, we only detect the second line.
    707713                    end_of_previous_error_line = self.error_from_test.rfind('\n', 0, -1)
    … …  
    709715                        self.error_from_test = self.error_from_test[:end_of_previous_error_line]
    710716                    else:
    711                         self.error_from_test = ""
     717                        self.error_from_test = ''
    712718                    # Symbolication can take a very long time, give it 10 extra minutes to finish.
    713719                    # FIXME: This can likely be removed once <rdar://problem/18701447> is fixed.
    714720                    deadline += 10 * 60 * 1000
    715721                if asan_violation_detected:
    716                     self._crash_report_from_driver += err_line
     722                    self._crash_report_from_driver += decode_for(err_line, str)
    717723                else:
    718                     self.error_from_test += err_line
     724                    self.error_from_test += decode_for(err_line, str)
    719725
    720726        if asan_violation_detected and not self._crashed_process_name:
    … …  
    739745        self._content_length = None
    740746        # Content is treated as binary data even though the text output is usually UTF-8.
    741         self.content = str()  # FIXME: Should be bytearray() once we require Python 2.6.
     747        self.content = b''
    742748        self.decoded_content = None
    743749        self.malloc = None
    … …  
    748754            self.decoded_content = base64.b64decode(self.content)
    749755        else:
    750             self.decoded_content = self.content
     756            try:
     757                self.decoded_content = decode_for(self.content, str)
     758            except UnicodeDecodeError:
     759                self.decoded_content = None
    751760
    752761

  • trunk/Tools/Scripts/webkitpy/port/driver_unittest.py

    r251808 r253804  
    183183        self.assertEqual(content_block.content_type, 'image/png')
    184184        self.assertEqual(content_block.content_hash, 'actual')
    185         self.assertEqual(content_block.content, '12345678\n')
     185        self.assertEqual(content_block.content, b'12345678\n')
    186186        self.assertEqual(content_block.decoded_content, '12345678\n')
    187187        driver._server_process = None
    … …  
    202202        self.assertEqual(content_block.content_hash, 'actual')
    203203        self.assertEqual(content_block.encoding, 'base64')
    204         self.assertEqual(content_block.content, 'MTIzNDU2NzgK')
     204        self.assertEqual(content_block.content, b'MTIzNDU2NzgK')
    205205        self.assertEqual(content_block.decoded_content, b'12345678\n')
    206206
    … …  
    248248
    249249        driver._server_process = FakeServerProcess(False)
    250         assert_crash(driver, '', False, None, None)
    251 
    252         driver._crashed_process_name = None
    253         driver._crashed_pid = None
    254         driver._server_process = FakeServerProcess(False)
    255         driver._driver_timed_out = False
    256         assert_crash(driver, '#CRASHED\n', True, 'FakeServerProcess', 1234)
    257 
    258         driver._crashed_process_name = None
    259         driver._crashed_pid = None
    260         driver._server_process = FakeServerProcess(False)
    261         driver._driver_timed_out = False
    262         assert_crash(driver, '#CRASHED - WebProcess\n', True, 'WebProcess', None)
    263 
    264         driver._crashed_process_name = None
    265         driver._crashed_pid = None
    266         driver._server_process = FakeServerProcess(False)
    267         driver._driver_timed_out = False
    268         assert_crash(driver, '#CRASHED - WebProcess (pid 8675)\n', True, 'WebProcess', 8675)
    269 
    270         driver._crashed_process_name = None
    271         driver._crashed_pid = None
    272         driver._server_process = FakeServerProcess(False)
    273         driver._driver_timed_out = False
    274         assert_crash(driver, '#PROCESS UNRESPONSIVE - WebProcess (pid 8675)\n', True, None, None, True)
    275 
    276         driver._crashed_process_name = None
    277         driver._crashed_pid = None
    278         driver._server_process = FakeServerProcess(False)
    279         driver._driver_timed_out = False
    280         assert_crash(driver, '#CRASHED - renderer (pid 8675)\n', True, 'renderer', 8675)
     250        assert_crash(driver, b'', False, None, None)
     251
     252        driver._crashed_process_name = None
     253        driver._crashed_pid = None
     254        driver._server_process = FakeServerProcess(False)
     255        driver._driver_timed_out = False
     256        assert_crash(driver, b'#CRASHED\n', True, 'FakeServerProcess', 1234)
     257
     258        driver._crashed_process_name = None
     259        driver._crashed_pid = None
     260        driver._server_process = FakeServerProcess(False)
     261        driver._driver_timed_out = False
     262        assert_crash(driver, b'#CRASHED - WebProcess\n', True, 'WebProcess', None)
     263
     264        driver._crashed_process_name = None
     265        driver._crashed_pid = None
     266        driver._server_process = FakeServerProcess(False)
     267        driver._driver_timed_out = False
     268        assert_crash(driver, b'#CRASHED - WebProcess (pid 8675)\n', True, 'WebProcess', 8675)
     269
     270        driver._crashed_process_name = None
     271        driver._crashed_pid = None
     272        driver._server_process = FakeServerProcess(False)
     273        driver._driver_timed_out = False
     274        assert_crash(driver, b'#PROCESS UNRESPONSIVE - WebProcess (pid 8675)\n', True, None, None, True)
     275
     276        driver._crashed_process_name = None
     277        driver._crashed_pid = None
     278        driver._server_process = FakeServerProcess(False)
     279        driver._driver_timed_out = False
     280        assert_crash(driver, b'#CRASHED - renderer (pid 8675)\n', True, 'renderer', 8675)
    281281
    282282        driver._crashed_process_name = None
    … …  
    284284        driver._server_process = FakeServerProcess(True)
    285285        driver._driver_timed_out = False
    286         assert_crash(driver, '', True, 'FakeServerProcess', 1234)
     286        assert_crash(driver, b'', True, 'FakeServerProcess', 1234)
    287287
    288288    def test_creating_a_port_does_not_write_to_the_filesystem(self):

  • trunk/Tools/Scripts/webkitpy/port/image_diff.py

    r244080 r253804  
    3636
    3737from webkitpy.port import server_process
     38from webkitpy.common.unicode_compatibility import decode_for
    3839
    3940
    … …  
    5859                self._start(tolerance)
    5960            # Note that although we are handed 'old', 'new', ImageDiff wants 'new', 'old'.
    60             self._process.write('Content-Length: %d\n%sContent-Length: %d\n%s' % (
    61                 len(actual_contents), actual_contents,
    62                 len(expected_contents), expected_contents))
     61            self._process.write('Content-Length: {}\n'.format(len(actual_contents)))
     62            self._process.write(actual_contents)
     63            self._process.write('Content-Length: {}\n'.format(len(expected_contents)))
     64            self._process.write(expected_contents)
    6365            return self._read()
    6466        except IOError as exception:
    … …  
    8486                break
    8587
    86             if output.startswith('diff'):  # This is the last line ImageDiff prints.
     88            if output.startswith(b'diff'):  # This is the last line ImageDiff prints.
    8789                break
    8890
    89             if output.startswith('Content-Length'):
    90                 m = re.match('Content-Length: (\d+)', output)
     91            if output.startswith(b'Content-Length'):
     92                m = re.match(b'Content-Length: (\d+)', output)
    9193                content_length = int(m.group(1))
    9294                output_image = self._process.read_stdout(deadline, content_length)
    … …  
    9496                break
    9597
    96         stderr = self._process.pop_all_buffered_stderr()
     98        stderr = decode_for(self._process.pop_all_buffered_stderr(), str)
    9799        err_str = ''
    98100        if stderr:
    … …  
    104106
    105107        diff_percent = 0
    106         if output and output.startswith('diff'):
    107             m = re.match('diff: (.+)% (passed|failed)', output)
    108             if m.group(2) == 'passed':
     108        if output and output.startswith(b'diff'):
     109            m = re.match(b'diff: (.+)% (passed|failed)', output)
     110            if m.group(2) == b'passed':
    109111                return (None, 0, None)
    110112            diff_percent = float(m.group(1))

  • trunk/Tools/Scripts/webkitpy/port/server_process.py

    r253410 r253804  
    3636import time
    3737
     38from webkitpy.common.system.executive import ScriptError
     39from webkitpy.common.unicode_compatibility import encode_if_necessary
     40
    3841# Note that although win32 python does provide an implementation of
    3942# the win32 select API, it only works on sockets, and not on the named pipes
    … …  
    4750    import os
    4851    import select
    49 
    50 from webkitpy.common.system.executive import ScriptError
    5152
    5253
    … …  
    179180            self._start()
    180181        try:
    181             self._proc.stdin.write(bytes)
     182            self._proc.stdin.write(encode_if_necessary(bytes))
    182183            self._proc.stdin.flush()
    183         except IOError as e:
     184        except (IOError, ValueError) as e:
    184185            self.stop(0.0)
    185186            # stop() calls _reset(), so we have to set crashed to True after calling stop()

  • trunk/Tools/Scripts/webkitpy/port/server_process_mock.py

    r244080 r253804  
    2727# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    2828
     29from webkitpy.common.unicode_compatibility import encode_if_necessary
     30
    2931
    3032class MockServerProcess(object):
    3133    def __init__(self, port_obj=None, name=None, cmd=None, env=None, universal_newlines=False, lines=None, crashed=False, target_host=None):
    3234        self.timed_out = False
    33         self.lines = lines or []
     35        self.lines = [encode_if_necessary(line) for line in (lines or [])]
    3436        self.crashed = crashed
    3537        self.writes = []
    … …  
    5355        if self.has_crashed():
    5456            return None
    55         return self.lines.pop(0) + "\n"
     57        return self.lines.pop(0) + b'\n'
    5658
    5759    def has_available_stdout(self):
    … …  
    6870            remaining_size = size - len(first_line) - 1
    6971            if not remaining_size:
    70                 return first_line + "\n"
    71             return first_line + "\n" + self.read_stdout(deadline, remaining_size)
     72                return first_line + b'\n'
     73            return first_line + b'\n' + self.read_stdout(deadline, remaining_size)
    7274        result = self.lines[0][:size]
    7375        self.lines[0] = self.lines[0][size:]

  • trunk/Tools/Scripts/webkitpy/port/simulator_process.py

    r252275 r253804  
    2323
    2424import os
     25import sys
    2526import time
    2627
    … …  
    6869        def close(self):
    6970            result = self._file.close()
    70             self.socket.close()
     71            # Closing the file implicitly closes the socket in Python 3
     72            if sys.version_info < (3, 0):
     73                self.socket.close()
    7174            return result
    7275
    … …  
    104107            try:
    105108                # This order matches the client side connections in Tools/TestRunnerShared/IOSLayoutTestCommunication.cpp setUpIOSLayoutTestCommunication()
    106                 stdin = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'w')
     109                stdin = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'wb')
    107110                stdout = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'rb')
    108111                stderr = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'rb')

  • trunk/Tools/Scripts/webkitpy/port/test.py

    r239989 r253804  
    3333from webkitpy.common.system.crashlogs import CrashLogs
    3434from webkitpy.common.version_name_map import PUBLIC_TABLE, VersionNameMap
     35from webkitpy.common.unicode_compatibility import decode_for, encode_if_necessary
    3536
    3637
    … …  
    112113
    113114def unit_test_list():
    114     silent_audio = "RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
    115     silent_audio_with_single_bit_difference = "RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
    116     audio2 = "RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x40\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
     115    silent_audio = b"RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
     116    silent_audio_with_single_bit_difference = b"RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
     117    audio2 = b"RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x40\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
    117118    tests = TestList()
    118119    tests.add('failures/expected/crash.html', crash=True)
    … …  
    435436
    436437    def diff_image(self, expected_contents, actual_contents, tolerance=None):
     438        expected_contents = encode_if_necessary(expected_contents)
     439        actual_contents = encode_if_necessary(actual_contents)
    437440        diffed = actual_contents != expected_contents
    438441        if not actual_contents and not expected_contents:
    … …  
    440443        if not actual_contents or not expected_contents:
    441444            return (True, 0, None)
    442         if 'ref' in expected_contents:
     445        if b'ref' in expected_contents:
    443446            assert tolerance == 0
    444447        if diffed:
    445             return ("< %s\n---\n> %s\n" % (expected_contents, actual_contents), 1, None)
     448            return ("< {}\n---\n> {}\n".format(
     449                decode_for(expected_contents, str),
     450                decode_for(actual_contents, str),
     451            ), 1, None)
    446452        return (None, 0, None)
    447453
    … …  
    564570        test = self._port._tests[test_name]
    565571        if test.keyboard:
    566             raise KeyboardInterrupt
     572            raise KeyboardInterrupt()
    567573        if test.exception:
    568574            raise ValueError('exception from ' + test_name)

  • trunk/Tools/Scripts/webkitpy/xcode/simulated_device.py

    r252738 r253804  
    564564            return True
    565565
    566         system_processes = self.executive.run_command([SimulatedDeviceManager.xcrun, 'simctl', 'spawn', self.udid, 'launchctl', 'print', 'system'], decode_output=False)
     566        system_processes = self.executive.run_command([SimulatedDeviceManager.xcrun, 'simctl', 'spawn', self.udid, 'launchctl', 'print', 'system'], decode_output=True)
    567567        if re.search(r'"{}"'.format(home_screen_service), system_processes) or re.search(r'A\s+{}'.format(home_screen_service), system_processes):
    568568            return True
    … …  
    605605    def install_app(self, app_path, env=None):
    606606        # Even after carousel is running, it takes a few seconds for watchOS to allow installs.
    607         for i in xrange(self.NUM_INSTALL_RETRIES):
     607        for i in range(self.NUM_INSTALL_RETRIES):
    608608            exit_code = self.executive.run_command(['xcrun', 'simctl', 'install', self.udid, app_path], return_exit_code=True)
    609609            if exit_code == 0:
Note: See TracChangeset for help on using the changeset viewer.