Changeset 253804 in webkit


Ignore:
Timestamp:
Dec 19, 2019 5:06:22 PM (4 years ago)
Author:
Jonathan Bedard
Message:

Python 3: Add support to run-webkit-tests
https://bugs.webkit.org/show_bug.cgi?id=205291

Reviewed by Stephanie Lewis.

  • Scripts/test-webkitpy-python3: Add webkitpy.layout_tests.
  • Scripts/webkitpy/common/message_pool.py:

(_MessagePool._loop): Move exception inside of loop.
(_Message.repr): Use .format strings.
(_Worker.init): Ditto.

  • Scripts/webkitpy/common/wavediff.py:

(WaveDiff.init): Use Python 3 compatible BytesIO and StringIO.

  • Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py:

(LayoutTestFinder._read_test_names_from_file): Use .format string.
(LayoutTestFinder.split_into_chunks): Explicitly use integer division.

  • Scripts/webkitpy/layout_tests/controllers/manager.py:

(Manager._get_test_inputs): Use range over xrange.

  • Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:

(SingleTestRunner._handle_error): Use .format strings.

  • Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:

(JSONLayoutResultsGenerator._insert_failure_summaries): Use Python 3 compatible itervalues.

  • Scripts/webkitpy/layout_tests/models/test_results.py:

(TestResult.init): Sort type list.

  • Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:

(RunTest.setUp): Multiple processes are buggy in test-webkitpy.
(RunTest.test_basic): replace buflist with getvalue().
(RunTest.test_child_processes_2): Ditto.
(RunTest.test_child_processes_min): Ditto.
(RunTest.test_keyboard_interrupt): Ditto.
(RunTest.test_missing_and_unexpected_results): Compare dictionaries instead of json strings.
(RunTest.test_crash_with_stderr): Ditto.
(RunTest.test_reftest_should_not_use_naming_convention_if_not_listed_in_reftestlist): Ditto.
(EndToEndTest.test_reftest_with_two_notrefs): Ditto.

  • Scripts/webkitpy/layout_tests/views/metered_stream.py:

(MeteredStream.write): Flush stream after writing.
(MeteredStream._erase_last_partial_line): Ditto.

  • Scripts/webkitpy/layout_tests/views/printing.py:

(Printer._print_directory_timings): Can't compare string to integer.
(Printer._print_statistics_for_test_timings): Use integer division.

  • Scripts/webkitpy/port/darwin.py:

(DarwinPort._merge_crash_logs): Use items over iteritems.

  • Scripts/webkitpy/port/device.py:

(Device.hash): Allow hashing of devices.

  • Scripts/webkitpy/port/device_port.py:

(DevicePort._install): Use range over xrange.
(DevicePort.setup_test_run): Ditto.
(DevicePort.clean_up_test_run): Ditto.

  • Scripts/webkitpy/port/driver.py:

(DriverOutput.init): Text should be decoded, audio encoded.
(Driver.init):
(Driver.run_test):
(Driver._parse_child_processes_output): Output is byte array.
(Driver._check_for_driver_timeout): Ditto.
(Driver._check_for_address_sanitizer_violation): Error lines are byte arrays.
(Driver._check_for_driver_crash_or_unresponsiveness): Ditto.
(Driver._read_optional_image_block): If a block is base64 encoded, we want the decoded
content, otherwise, we want the encoded content.
(Driver._read_header): Lines are byte arrays, decode them before processing.
(Driver._process_stdout_line): Blocks are byte arrays.
(Driver._strip_eof): Lines should be byte arrays, not strings.
(Driver._read_block): Standardize encoding in blocks.
(ContentBlock.init): Content should be a byte array.
(ContentBlock.decode_content): Attempt to decode content.

  • Scripts/webkitpy/port/driver_unittest.py:

(DriverTest.test_read_binary_block): Content should be encoded.
(DriverTest.test_read_base64_block): Ditto.
(DriverTest.test_check_for_driver_crash): ServerProcess output should be a byte array.

  • Scripts/webkitpy/port/image_diff.py:

(ImageDiffer.diff_image): ImageDiff output is in byte arrays.
(ImageDiffer._read): Ditto.

  • Scripts/webkitpy/port/server_process.py:

(ServerProcess.write): Encode data before writing it.

  • Scripts/webkitpy/port/server_process_mock.py:

(MockServerProcess.init): Convert string mock output to bytes.
(MockServerProcess.read_stdout_line): Stdout lines are byte arrays.
(MockServerProcess.read_stdout): Ditto.

  • Scripts/webkitpy/port/simulator_process.py:

(SimulatorProcess.NonBlockingFileFromSocket.close): Don't double close socket in Python 3.
(SimulatorProcess._start): Stdin should be a binary stream.

  • Scripts/webkitpy/port/test.py:

(unit_test_list): Convert audio streams to byte arrays.

  • Scripts/webkitpy/xcode/simulated_device.py:

(SimulatedDevice.is_usable): Decode xcrun output.
(SimulatedDevice.install_app): Use xrange over range.

Location:
trunk/Tools
Files:
23 edited

Legend:

Unmodified
Added
Removed
  • trunk/Tools/ChangeLog

    r253786 r253804  
     12019-12-19  Jonathan Bedard  <jbedard@apple.com>
     2
     3        Python 3: Add support to run-webkit-tests
     4        https://bugs.webkit.org/show_bug.cgi?id=205291
     5
     6        Reviewed by Stephanie Lewis.
     7
     8        * Scripts/test-webkitpy-python3: Add webkitpy.layout_tests.
     9        * Scripts/webkitpy/common/message_pool.py:
     10        (_MessagePool._loop): Move exception inside of loop.
     11        (_Message.__repr__): Use .format strings.
     12        (_Worker.__init__): Ditto.
     13        * Scripts/webkitpy/common/wavediff.py:
     14        (WaveDiff.__init__): Use Python 3 compatible BytesIO and StringIO.
     15        * Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py:
     16        (LayoutTestFinder._read_test_names_from_file): Use .format string.
     17        (LayoutTestFinder.split_into_chunks): Explicitly use integer division.
     18        * Scripts/webkitpy/layout_tests/controllers/manager.py:
     19        (Manager._get_test_inputs): Use range over xrange.
     20        * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
     21        (SingleTestRunner._handle_error): Use .format strings.
     22        * Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:
     23        (JSONLayoutResultsGenerator._insert_failure_summaries): Use Python 3 compatible itervalues.
     24        * Scripts/webkitpy/layout_tests/models/test_results.py:
     25        (TestResult.__init__): Sort type list.
     26        * Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:
     27        (RunTest.setUp): Multiple processes are buggy in test-webkitpy.
     28        (RunTest.test_basic): replace buflist with getvalue().
     29        (RunTest.test_child_processes_2): Ditto.
     30        (RunTest.test_child_processes_min): Ditto.
     31        (RunTest.test_keyboard_interrupt): Ditto.
     32        (RunTest.test_missing_and_unexpected_results): Compare dictionaries instead of json strings.
     33        (RunTest.test_crash_with_stderr): Ditto.
     34        (RunTest.test_reftest_should_not_use_naming_convention_if_not_listed_in_reftestlist): Ditto.
     35        (EndToEndTest.test_reftest_with_two_notrefs): Ditto.
     36        * Scripts/webkitpy/layout_tests/views/metered_stream.py:
     37        (MeteredStream.write): Flush stream after writing.
     38        (MeteredStream._erase_last_partial_line): Ditto.
     39        * Scripts/webkitpy/layout_tests/views/printing.py:
     40        (Printer._print_directory_timings): Can't compare string to integer.
     41        (Printer._print_statistics_for_test_timings): Use integer division.
     42        * Scripts/webkitpy/port/darwin.py:
     43        (DarwinPort._merge_crash_logs): Use items over iteritems.
     44        * Scripts/webkitpy/port/device.py:
     45        (Device.__hash__): Allow hashing of devices.
     46        * Scripts/webkitpy/port/device_port.py:
     47        (DevicePort._install): Use range over xrange.
     48        (DevicePort.setup_test_run): Ditto.
     49        (DevicePort.clean_up_test_run): Ditto.
     50        * Scripts/webkitpy/port/driver.py:
     51        (DriverOutput.__init__): Text should be decoded, audio encoded.
     52        (Driver.__init__):
     53        (Driver.run_test):
     54        (Driver._parse_child_processes_output): Output is byte array.
     55        (Driver._check_for_driver_timeout): Ditto.
     56        (Driver._check_for_address_sanitizer_violation): Error lines are byte arrays.
     57        (Driver._check_for_driver_crash_or_unresponsiveness): Ditto.
     58        (Driver._read_optional_image_block): If a block is base64 encoded, we want the decoded
     59        content, otherwise, we want the encoded content.
     60        (Driver._read_header): Lines are byte arrays, decode them before processing.
     61        (Driver._process_stdout_line): Blocks are byte arrays.
     62        (Driver._strip_eof): Lines should be byte arrays, not strings.
     63        (Driver._read_block): Standardize encoding in blocks.
     64        (ContentBlock.__init__): Content should be a byte array.
     65        (ContentBlock.decode_content): Attempt to decode content.
     66        * Scripts/webkitpy/port/driver_unittest.py:
     67        (DriverTest.test_read_binary_block): Content should be encoded.
     68        (DriverTest.test_read_base64_block): Ditto.
     69        (DriverTest.test_check_for_driver_crash): ServerProcess output should be a byte array.
     70        * Scripts/webkitpy/port/image_diff.py:
     71        (ImageDiffer.diff_image): ImageDiff output is in byte arrays.
     72        (ImageDiffer._read): Ditto.
     73        * Scripts/webkitpy/port/server_process.py:
     74        (ServerProcess.write): Encode data before writing it.
     75        * Scripts/webkitpy/port/server_process_mock.py:
     76        (MockServerProcess.__init__): Convert string mock output to bytes.
     77        (MockServerProcess.read_stdout_line): Stdout lines are byte arrays.
     78        (MockServerProcess.read_stdout): Ditto.
     79        * Scripts/webkitpy/port/simulator_process.py:
     80        (SimulatorProcess.NonBlockingFileFromSocket.close): Don't double close socket in Python 3.
     81        (SimulatorProcess._start): Stdin should be a binary stream.
     82        * Scripts/webkitpy/port/test.py:
     83        (unit_test_list): Convert audio streams to byte arrays.
     84        * Scripts/webkitpy/xcode/simulated_device.py:
     85        (SimulatedDevice.is_usable): Decode xcrun output.
     86        (SimulatedDevice.install_app): Use xrange over range.
     87
    1882019-12-19  Matt Lewis  <jlewis3@apple.com>
    289
  • trunk/Tools/Scripts/test-webkitpy-python3

    r253392 r253804  
    3838  'webkitpy.browserperfdash',
    3939  'webkitpy.common',
    40   'webkitpy.layout_tests.controllers',
    41   'webkitpy.layout_tests.layout_package',
    42   'webkitpy.layout_tests.models',
    43   'webkitpy.layout_tests.servers',
    44   'webkitpy.layout_tests.views',
     40  'webkitpy.layout_tests',
    4541  'webkitpy.performance_tests',
    4642  'webkitpy.port',
  • trunk/Tools/Scripts/webkitpy/common/message_pool.py

    r252616 r253804  
    186186
    187187    def _loop(self, block):
    188         try:
    189             while True:
    190                 if len(self._workers_stopped) == len(self._workers):
    191                     block = False
     188        while True:
     189            if len(self._workers_stopped) == len(self._workers):
     190                block = False
     191
     192            try:
    192193                message = self._messages_to_manager.get(block)
    193                 self._log_messages(message.logs)
    194                 if message.from_user:
    195                     self._caller.handle(message.name, message.src, *message.args)
    196                     continue
    197                 method = getattr(self, '_handle_' + message.name)
    198                 assert method, 'bad message %s' % repr(message)
    199                 method(message.src, *message.args)
    200         except queue.Empty:
    201             pass
     194            except queue.Empty:
     195                break
     196
     197            self._log_messages(message.logs)
     198            if message.from_user:
     199                self._caller.handle(message.name, message.src, *message.args)
     200                continue
     201            method = getattr(self, '_handle_' + message.name)
     202            assert method, 'bad message %s' % repr(message)
     203            method(message.src, *message.args)
    202204
    203205
     
    216218
    217219    def __repr__(self):
    218         return '_Message(src=%s, name=%s, args=%s, from_user=%s, logs=%s)' % (self.src, self.name, self.args, self.from_user, self.logs)
     220        return '_Message(src={src}, name={name}, args={args}, from_user={from_user}, logs={logs})'.format(
     221            src=self.src,
     222            name=self.name,
     223            args=self.args,
     224            from_user=self.from_user,
     225            logs=self.logs,
     226        )
    219227
    220228
     
    224232        self.host = host
    225233        self.worker_number = worker_number
    226         self.name = 'worker/%d' % worker_number
     234        self.name = 'worker/{}'.format(worker_number)
    227235        self.log_messages = []
    228236        self.log_level = log_level
  • trunk/Tools/Scripts/webkitpy/common/wavediff.py

    r250375 r253804  
    2626import wave
    2727
    28 from io import BytesIO
     28from webkitpy.common.unicode_compatibility import BytesIO, StringIO
    2929
    3030
     
    3636
    3737    def __init__(self, in1, in2):
    38         if isinstance(in1, file):
    39             waveFile1 = wave.open(in1, 'rb')
     38        if isinstance(in1, str):
     39            waveFile1 = wave.open(StringIO(in1), 'r')
    4040        else:
    4141            waveFile1 = wave.open(BytesIO(in1), 'rb')
    42         if isinstance(in2, file):
    43             waveFile1 = wave.open(in2, 'rb')
     42        if isinstance(in2, str):
     43            waveFile2 = wave.open(StringIO(in2), 'r')
    4444        else:
    4545            waveFile2 = wave.open(BytesIO(in2), 'rb')
     
    6767        allData1 = self._readSamples(waveFile1, sampleWidth1, frameCount1 * channelCount1)
    6868        allData2 = self._readSamples(waveFile2, sampleWidth2, frameCount2 * channelCount2)
    69         results = map(self._diffSample, allData1, allData2, xrange(max(frameCount1 * channelCount1, frameCount2 * channelCount2)))
     69        results = list(map(self._diffSample, allData1, allData2, range(max(frameCount1 * channelCount1, frameCount2 * channelCount2))))
    7070
    7171        cumulativeSampleDiff = sum(results)
    72         differingSampleCount = len(filter(bool, results))
     72        differingSampleCount = len(list(filter(bool, results)))
    7373        self._filesAreIdentical = not differingSampleCount
    74         self._filesAreIdenticalWithinTolerance = not len(filter(lambda x: x > self._tolerance, results))
     74        self._filesAreIdenticalWithinTolerance = not len(list(filter(lambda x: x > self._tolerance, results)))
    7575
    7676        if differingSampleCount:
  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py

    r244701 r253804  
    103103                if e.errno == errno.ENOENT:
    104104                    _log.critical('')
    105                     _log.critical('--test-list file "%s" not found' % file)
     105                    _log.critical('--test-list file "{}" not found'.format(filenames))
    106106                raise
    107107        return tests
     
    177177                rounded_tests = (num_tests + test_size - (num_tests % test_size))
    178178
    179             chunk_len = rounded_tests / test_size
     179            chunk_len = rounded_tests // test_size
    180180            slice_start = chunk_len * (chunk_num - 1)
    181181            # It does not mind if we go over test_size.
  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py

    r252443 r253804  
    146146    def _get_test_inputs(self, tests_to_run, repeat_each, iterations, device_type=None):
    147147        test_inputs = []
    148         for _ in xrange(iterations):
     148        for _ in range(iterations):
    149149            for test in tests_to_run:
    150                 for _ in xrange(repeat_each):
     150                for _ in range(repeat_each):
    151151                    test_inputs.append(self._test_input_for_file(test, device_type=device_type))
    152152        return test_inputs
  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py

    r239989 r253804  
    3232import time
    3333
     34from webkitpy.common.unicode_compatibility import decode_for
    3435from webkitpy.layout_tests.controllers import test_result_writer
    3536from webkitpy.port.driver import DriverInput, DriverOutput
     
    215216            _log.debug("%s %s output stderr lines:" % (self._worker_name, testname))
    216217        for line in driver_output.error.splitlines():
    217             _log.debug("  %s" % line)
     218            _log.debug("  {}".format(decode_for(line, str)))
    218219        return failures
    219220
  • trunk/Tools/Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py

    r240150 r253804  
    3232from webkitpy.layout_tests.models import test_expectations
    3333from webkitpy.layout_tests.models import test_failures
     34from webkitpy.common.iteration_compatibility import itervalues
    3435
    3536
     
    133134
    134135        num_fixable = 0
    135         for expectation in self._expectations.itervalues():
     136        for expectation in itervalues(self._expectations):
    136137            num_fixable += len(expectation.model().get_tests_with_timeline(test_expectations.NOW))
    137138
  • trunk/Tools/Scripts/webkitpy/layout_tests/models/test_results.py

    r251112 r253804  
    4848        self.test_run_time = test_run_time or 0  # The time taken to execute the test itself.
    4949        self.has_stderr = has_stderr
    50         self.reftest_type = reftest_type or []
     50        self.reftest_type = sorted(reftest_type or [])
    5151        self.pid = pid
    5252        self.references = references or []
  • trunk/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py

    r253219 r253804  
    3030
    3131import json
     32import sys
    3233import unittest
    3334
     35from webkitpy.common.unicode_compatibility import StringIO
    3436from webkitpy.common.system import outputcapture, path
    3537from webkitpy.common.system.crashlogs_unittest import make_mock_crash_report_darwin
     
    177179        # FIXME: Remove this when we fix test-webkitpy to work
    178180        # properly on cygwin (bug 63846).
    179         self.should_test_processes = not self._platform.is_win()
     181        # FIXME: Multiprocessing doesn't do well when nested in Python 3 (https://bugs.webkit.org/show_bug.cgi?id=205280)
     182        self.should_test_processes = not self._platform.is_win() and sys.version_info < (3, 0)
    180183
    181184    def test_basic(self):
     
    196199            details.initial_results.total - details.initial_results.expected_skips - len(details.initial_results.unexpected_results_by_name),
    197200            len(details.initial_results.unexpected_results_by_name))
    198         self.assertTrue(one_line_summary in logging_stream.buflist)
     201        self.assertTrue(one_line_summary in logging_stream.getvalue())
    199202
    200203        # Ensure the results were summarized properly.
     
    220223            _, regular_output, _ = logging_run(
    221224                ['--debug-rwt-logging', '--child-processes', '2'], shared_port=False)
    222             self.assertTrue(any(['Running 2 ' in line for line in regular_output.buflist]))
     225            self.assertTrue(any(['Running 2 ' in line for line in regular_output.getvalue().splitlines()]))
    223226
    224227    def test_child_processes_min(self):
     
    227230                ['--debug-rwt-logging', '--child-processes', '2', '-i', 'passes/passes', 'passes'],
    228231                tests_included=True, shared_port=False)
    229             self.assertTrue(any(['Running 1 ' in line for line in regular_output.buflist]))
     232            self.assertTrue(any(['Running 1 ' in line for line in regular_output.getvalue().splitlines()]))
    230233
    231234    def test_dryrun(self):
     
    273276        if self.should_test_processes:
    274277            _, regular_output, _ = logging_run(['failures/expected/keyboard.html', 'passes/text.html', '--child-processes', '2', '--force'], tests_included=True, shared_port=False)
    275             self.assertTrue(any(['Interrupted, exiting' in line for line in regular_output.buflist]))
     278            self.assertTrue(any(['Interrupted, exiting' in line for line in regular_output.getvalue().splitlines()]))
    276279
    277280    def test_no_tests_found(self):
     
    477480        file_list = host.filesystem.written_files.keys()
    478481        self.assertEqual(details.exit_code, 1)
    479         expected_token = '"unexpected":{"text-image-checksum.html":{"report":"REGRESSION","expected":"PASS","actual":"IMAGE+TEXT","image_diff_percent":1},"missing_text.html":{"report":"MISSING","expected":"PASS","is_missing_text":true,"actual":"MISSING"}'
    480         json_string = host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')
    481         self.assertTrue(json_string.find(expected_token) != -1)
    482         self.assertTrue(json_string.find('"num_regressions":1') != -1)
    483         self.assertTrue(json_string.find('"num_flaky":0') != -1)
    484         self.assertTrue(json_string.find('"num_missing":1') != -1)
     482        expected_dictionary = {
     483            'version': 4,
     484            'fixable': 3,
     485            'skipped': 0,
     486            'num_passes': 0,
     487            'num_flaky': 0,
     488            'num_missing': 1,
     489            'num_regressions': 1,
     490            'uses_expectations_file': True,
     491            'interrupted': False,
     492            'layout_tests_dir': '/test.checkout/LayoutTests',
     493            'has_pretty_patch': False,
     494            'pixel_tests_enabled': True,
     495            'other_crashes': {},
     496            'date': '10:10AM on December 13, 2019',
     497            'tests': {
     498                'failures': {
     499                    'expected': {
     500                        'missing_image.html': {
     501                            'expected': 'PASS MISSING',
     502                            'actual': 'MISSING',
     503                            'is_missing_image': True,
     504                        },
     505                    }, 'unexpected': {
     506                        'missing_text.html': {
     507                            'report': 'MISSING',
     508                            'expected': 'PASS',
     509                            'actual': 'MISSING',
     510                            'is_missing_text': True,
     511                        }, 'text-image-checksum.html': {
     512                            'report': 'REGRESSION',
     513                            'expected': 'PASS',
     514                            'actual': 'IMAGE+TEXT',
     515                            'image_diff_percent': 1,
     516                        },
     517                    },
     518                },
     519            },
     520        }
     521        actual_dictionary = json.loads(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')[len('ADD_RESULTS('):-2])
     522        self.assertEqual(
     523            sorted(list(expected_dictionary['tests']['failures']['expected'])),
     524            sorted(list(actual_dictionary['tests']['failures']['expected'])),
     525        )
     526        self.assertEqual(
     527            sorted(list(expected_dictionary['tests']['failures']['unexpected'])),
     528            sorted(list(actual_dictionary['tests']['failures']['unexpected'])),
     529        )
     530        self.assertEqual(expected_dictionary['num_regressions'], actual_dictionary['num_regressions'])
     531        self.assertEqual(expected_dictionary['num_flaky'], actual_dictionary['num_flaky'])
     532        self.assertEqual(expected_dictionary['num_missing'], actual_dictionary['num_missing'])
    485533
    486534    def test_pixel_test_directories(self):
     
    518566        host = MockHost()
    519567        _, regular_output, _ = logging_run(['failures/unexpected/crash-with-stderr.html'], tests_included=True, host=host)
    520         self.assertTrue(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json').find('{"crash-with-stderr.html":{"report":"REGRESSION","expected":"PASS","actual":"CRASH","has_stderr":true}}') != -1)
     568        actual_dictionary = json.loads(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')[len('ADD_RESULTS('):-2])
     569        expected_dictionary = {
     570            'version': 4,
     571            'fixable': 1,
     572            'skipped': 0,
     573            'num_passes': 0,
     574            'num_flaky': 0,
     575            'num_missing': 0,
     576            'num_regressions': 1,
     577            'uses_expectations_file': True,
     578            'interrupted': False,
     579            'layout_tests_dir': '/test.checkout/LayoutTests',
     580            'has_pretty_patch': False,
     581            'pixel_tests_enabled': True,
     582            'other_crashes': {},
     583            'date': '10:18AM on December 13, 2019',
     584            'tests': {
     585                'failures': {
     586                    'unexpected': {
     587                        'crash-with-stderr.html': {
     588                            'has_stderr': True,
     589                            'report': 'REGRESSION',
     590                            'expected': 'PASS',
     591                            'actual': 'CRASH',
     592                        },
     593                    },
     594                },
     595            },
     596        }
     597        self.assertEqual(
     598            sorted(list(expected_dictionary['tests']['failures']['unexpected'])),
     599            sorted(list(actual_dictionary['tests']['failures']['unexpected'])),
     600        )
    521601
    522602    def test_no_image_failure_with_image_diff(self):
     
    741821        host = MockHost()
    742822        _, err, _ = logging_run(['--no-show-results', 'reftests/foo/'], tests_included=True, host=host)
    743         json_string = host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')
    744         self.assertTrue(json_string.find('"unlistedtest.html":{"report":"MISSING","expected":"PASS","is_missing_text":true,"actual":"MISSING","is_missing_image":true}') != -1)
    745         self.assertTrue(json_string.find('"num_regressions":4') != -1)
    746         self.assertTrue(json_string.find('"num_flaky":0') != -1)
    747         self.assertTrue(json_string.find('"num_missing":1') != -1)
     823        expected_dictionary = {
     824            'version': 4,
     825            'fixable': 5,
     826            'skipped': 0,
     827            'num_passes': 3,
     828            'num_flaky': 0,
     829            'num_missing': 1,
     830            'num_regressions': 4,
     831            'uses_expectations_file': True,
     832            'interrupted': False,
     833            'layout_tests_dir': '/test.checkout/LayoutTests',
     834            'has_pretty_patch': False,
     835            'pixel_tests_enabled': True,
     836            'other_crashes': {},
     837            'date': '10:27AM on December 13, 2019',
     838            'tests': {
     839                'reftests': {
     840                    'foo': {
     841                        'multiple-both-failure.html': {
     842                            'reftest_type': ['!=', '=='],
     843                            'report': 'REGRESSION',
     844                            'expected': 'PASS',
     845                            'actual': 'IMAGE',
     846                        }, 'multiple-match-failure.html': {
     847                            'reftest_type': ['=='],
     848                            'report': 'REGRESSION',
     849                            'expected': 'PASS',
     850                            'actual': 'IMAGE',
     851                            'image_diff_percent': 1,
     852                        }, 'multiple-mismatch-failure.html': {
     853                            'reftest_type': ['!='],
     854                            'report': 'REGRESSION',
     855                            'expected': 'PASS',
     856                            'actual': 'IMAGE',
     857                        }, 'test.html': {
     858                            'reftest_type': ['=='],
     859                            'report': 'REGRESSION',
     860                            'expected': 'PASS',
     861                            'actual': 'IMAGE',
     862                            'image_diff_percent': None,
     863                        }, 'unlistedtest.html': {
     864                            'report': 'MISSING',
     865                            'expected': 'PASS',
     866                            'actual': 'MISSING',
     867                            'is_missing_text': True,
     868                            'is_missing_image': True,
     869                        },
     870                    },
     871                },
     872            },
     873        }
     874        actual_dictionary = json.loads(host.filesystem.read_text_file('/tmp/layout-test-results/full_results.json')[len('ADD_RESULTS('):-2])
     875
     876        self.assertEqual(
     877            expected_dictionary['tests']['reftests']['foo']['unlistedtest.html'],
     878            actual_dictionary['tests']['reftests']['foo']['unlistedtest.html'],
     879        )
     880        self.assertEqual(expected_dictionary['num_regressions'], actual_dictionary['num_regressions'])
     881        self.assertEqual(expected_dictionary['num_flaky'], actual_dictionary['num_flaky'])
     882        self.assertEqual(expected_dictionary['num_missing'], actual_dictionary['num_missing'])
    748883
    749884    def test_additional_platform_directory(self):
     
    9491084        self.assertTrue("multiple-mismatch-success.html" not in json["tests"]["reftests"]["foo"])
    9501085        self.assertTrue("multiple-both-success.html" not in json["tests"]["reftests"]["foo"])
    951         self.assertEqual(json["tests"]["reftests"]["foo"]["multiple-match-failure.html"],
    952             {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["=="], "image_diff_percent": 1, "report": "REGRESSION"})
    953         self.assertEqual(json["tests"]["reftests"]["foo"]["multiple-mismatch-failure.html"],
    954             {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["!="], "report": "REGRESSION"})
    955         self.assertEqual(json["tests"]["reftests"]["foo"]["multiple-both-failure.html"],
    956             {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["==", "!="], "report": "REGRESSION"})
     1086        self.assertEqual(
     1087            json["tests"]["reftests"]["foo"]["multiple-match-failure.html"],
     1088            {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["=="], "image_diff_percent": 1, "report": "REGRESSION"},
     1089        )
     1090        self.assertEqual(
     1091            json["tests"]["reftests"]["foo"]["multiple-mismatch-failure.html"],
     1092            {"expected": "PASS", "actual": "IMAGE", "reftest_type": ["!="], "report": "REGRESSION"},
     1093        )
     1094        self.assertEqual(
     1095            json["tests"]["reftests"]["foo"]["multiple-both-failure.html"],
     1096            {"expected": "PASS", "actual": "IMAGE", "reftest_type": sorted(["==", "!="]), "report": "REGRESSION"},
     1097        )
    9571098
    9581099
  • trunk/Tools/Scripts/webkitpy/layout_tests/views/metered_stream.py

    r252560 r253804  
    110110        try:
    111111            self._stream.write(timestamp_string + txt)
     112            self._stream.flush()
    112113        except UnicodeEncodeError:
    113114            output = ''
     
    118119                    output += '?'
    119120            self._stream.write(output)
     121            self._stream.flush()
    120122
    121123    def writeln(self, txt, now=None, pid=None):
     
    126128        self._stream.write(self._erasure(self._last_partial_line))
    127129        self._last_partial_line = ''
     130        self._stream.flush()
    128131
    129132    def flush(self):
  • trunk/Tools/Scripts/webkitpy/layout_tests/views/printing.py

    r248270 r253804  
    240240        min_seconds_to_print = 10
    241241        for timing in timings:
    242             if timing[0] > min_seconds_to_print:
     242            if timing[1] > min_seconds_to_print:
    243243                self._print_debug("  %s took %s seconds to run %s tests." % timing)
    244244        self._print_debug("")
     
    255255
    256256        if num_tests % 2 == 1:
    257             median = timings[((num_tests - 1) / 2) - 1]
    258         else:
    259             lower = timings[num_tests / 2 - 1]
    260             upper = timings[num_tests / 2]
     257            median = timings[(num_tests - 1) // 2 - 1]
     258        else:
     259            lower = timings[num_tests // 2 - 1]
     260            upper = timings[num_tests // 2]
    261261            median = (float(lower + upper)) / 2
    262262
  • trunk/Tools/Scripts/webkitpy/port/darwin.py

    r252779 r253804  
    9999
    100100    def _merge_crash_logs(self, logs, new_logs, crashed_processes):
    101         for test, crash_log in new_logs.iteritems():
     101        for test, crash_log in new_logs.items():
    102102            try:
    103103                if test.split('-')[0] == 'Sandbox':
  • trunk/Tools/Scripts/webkitpy/port/device.py

    r248270 r253804  
    117117    def __repr__(self):
    118118        return u'{}'.format(self.platform_device)
     119
     120    def __hash__(self):
     121        return hash(self.udid)
  • trunk/Tools/Scripts/webkitpy/port/device_port.py

    r248270 r253804  
    9696            return
    9797
    98         for i in xrange(self.child_processes()):
     98        for i in range(self.child_processes()):
    9999            device = self.target_host(i)
    100100            _log.debug(u'Installing to {}'.format(device))
     
    187187        self._install()
    188188
    189         for i in xrange(self.child_processes()):
     189        for i in range(self.child_processes()):
    190190            host = self.target_host(i)
    191191            host.prepare_for_testing(
     
    202202        # Failure to teardown devices can leave things in a bad state.
    203203        exception_list = []
    204         for i in xrange(self.child_processes()):
     204        for i in range(self.child_processes()):
    205205            device = self.target_host(i)
    206206            if not device:
  • trunk/Tools/Scripts/webkitpy/port/driver.py

    r252948 r253804  
    4040from webkitpy.common.system import path
    4141from webkitpy.common.system.profiler import ProfilerFactory
     42from webkitpy.common.unicode_compatibility import encode_if_necessary, decode_for
    4243
    4344
     
    8788            crashed_pid=None, crash_log=None, pid=None):
    8889        # FIXME: Args could be renamed to better clarify what they do.
    89         self.text = text
     90        self.text = decode_for(text, str) if text else None
    9091        self.image = image  # May be empty-string if the test crashes.
    9192        self.image_hash = image_hash
    9293        self.image_diff = None  # image_diff gets filled in after construction.
    93         self.audio = audio  # Binary format is port-dependent.
     94        self.audio = encode_if_necessary(audio) if audio else None  # Binary format is port-dependent.
    9495        self.crash = crash
    9596        self.crashed_process_name = crashed_process_name
     
    162163        # FIXME: We should probably remove _read_first_block and _read_optional_image_block and
    163164        # instead scope these locally in run_test.
    164         self.error_from_test = str()
     165        self.error_from_test = ''
    165166        self.err_seen_eof = False
    166167
     
    197198        self._driver_timed_out = False
    198199        self._crash_report_from_driver = None
    199         self.error_from_test = str()
     200        self.error_from_test = ''
    200201        self.err_seen_eof = False
    201202
     
    215216        text, audio = self._read_first_block(deadline, driver_input.test_name)  # First block is either text or audio
    216217        image, actual_image_hash = self._read_optional_image_block(deadline, driver_input.test_name)  # The second (optional) block is image data.
     218        text = decode_for(text, str)
    217219
    218220        crashed = self.has_crashed()
     
    228230            out, err = self._server_process.stop(self._port.driver_stop_timeout() if stop_when_done else 0.0)
    229231            if out:
    230                 text += out
     232                text += decode_for(out, str)
    231233            if err:
    232                 self.error_from_test += err
     234                self.error_from_test += decode_for(err, str)
    233235            self._server_process = None
    234236
     
    281283
    282284        for line in output.splitlines():
    283             m = re.match('^([^:]+): ([0-9]+)$', line)
     285            m = re.match(b'^([^:]+): ([0-9]+)$', line)
    284286            if m:
    285                 process_name = m.group(1)
    286                 process_id = m.group(2)
     287                process_name = decode_for(m.group(1), str)
     288                process_id = decode_for(m.group(2), str)
    287289                child_processes[process_name].append(process_id)
    288290
     
    541543
    542544    def _check_for_driver_timeout(self, out_line):
    543         if out_line.startswith("#PID UNRESPONSIVE - "):
    544             match = re.match('#PID UNRESPONSIVE - (\S+)', out_line)
    545             child_process_name = match.group(1) if match else 'WebProcess'
    546             match = re.search('pid (\d+)', out_line)
     545        if out_line.startswith(b"#PID UNRESPONSIVE - "):
     546            match = re.match(b'#PID UNRESPONSIVE - (\S+)', out_line)
     547            child_process_name = decode_for(match.group(1), str) if match else 'WebProcess'
     548            match = re.search(b'pid (\d+)', out_line)
    547549            child_process_pid = int(match.group(1)) if match else None
    548550            err_line = 'Wait on notifyDone timed out, process ' + child_process_name + ' pid = ' + str(child_process_pid)
     
    555557
    556558    def _check_for_address_sanitizer_violation(self, error_line):
    557         if "ERROR: AddressSanitizer" in error_line:
     559        if b"ERROR: AddressSanitizer" in error_line:
    558560            return True
    559561
    560562    def _check_for_driver_crash_or_unresponsiveness(self, error_line):
    561         crashed_check = error_line.rstrip('\r\n')
    562         if crashed_check == "#CRASHED":
     563        crashed_check = error_line.rstrip(b'\r\n')
     564        if crashed_check == b"#CRASHED":
    563565            self._crashed_process_name = self._server_process.process_name()
    564566            self._crashed_pid = self._server_process.system_pid()
    565567            return True
    566         elif error_line.startswith("#CRASHED - "):
    567             match = re.match('#CRASHED - (\S+)', error_line)
    568             self._crashed_process_name = match.group(1) if match else 'WebProcess'
    569             match = re.search('pid (\d+)', error_line)
     568        elif error_line.startswith(b"#CRASHED - "):
     569            match = re.match(b'#CRASHED - (\S+)', error_line)
     570            self._crashed_process_name = decode_for(match.group(1), str) if match else 'WebProcess'
     571            match = re.search(b'pid (\d+)', error_line)
    570572            self._crashed_pid = int(match.group(1)) if match else None
    571573            _log.debug('%s crash, pid = %s' % (self._crashed_process_name, str(self._crashed_pid)))
    572574            return True
    573         elif error_line.startswith("#PROCESS UNRESPONSIVE - "):
    574             match = re.match('#PROCESS UNRESPONSIVE - (\S+)', error_line)
    575             child_process_name = match.group(1) if match else 'WebProcess'
    576             match = re.search('pid (\d+)', error_line)
     575        elif error_line.startswith(b"#PROCESS UNRESPONSIVE - "):
     576            match = re.match(b'#PROCESS UNRESPONSIVE - (\S+)', error_line)
     577            child_process_name = decode_for(match.group(1), str) if match else 'WebProcess'
     578            match = re.search(b'pid (\d+)', error_line)
    577579            child_process_pid = int(match.group(1)) if match else None
    578580            _log.debug('%s is unresponsive, pid = %s' % (child_process_name, str(child_process_pid)))
     
    580582            if child_process_pid:
    581583                self._port.sample_process(child_process_name, child_process_pid, self._target_host)
    582             self.error_from_test += error_line
     584            self.error_from_test += decode_for(error_line, str)
    583585            self._server_process.write('#SAMPLE FINISHED\n', True)  # Must be able to ignore a broken pipe here, target process may already be closed.
    584586            return True
     
    629631        block = self._read_block(deadline, test_name, wait_for_stderr_eof=True)
    630632        if block.content and block.content_type == 'image/png':
    631             return (block.decoded_content, block.content_hash)
     633            return (block.decoded_content if block.encoding == 'base64' else block.content, block.content_hash)
    632634        return (None, block.content_hash)
    633635
    634636    def _read_header(self, block, line, header_text, header_attr, header_filter=None):
    635637        if line.startswith(header_text) and getattr(block, header_attr) is None:
    636             value = line.split()[1]
     638            value = decode_for(line.split()[1], str)
    637639            if header_filter:
    638640                value = header_filter(value)
     
    642644
    643645    def _process_stdout_line(self, block, line):
    644         if (self._read_header(block, line, 'Content-Type: ', 'content_type')
    645             or self._read_header(block, line, 'Content-Transfer-Encoding: ', 'encoding')
    646             or self._read_header(block, line, 'Content-Length: ', '_content_length', int)
    647             or self._read_header(block, line, 'ActualHash: ', 'content_hash')
    648             or self._read_header(block, line, 'DumpMalloc: ', 'malloc')
    649             or self._read_header(block, line, 'DumpJSHeap: ', 'js_heap')):
    650             return
     646        for header in [
     647            (b'Content-Type: ', 'content_type', None),
     648            (b'Content-Transfer-Encoding: ', 'encoding', None),
     649            (b'Content-Length: ', '_content_length', int),
     650            (b'ActualHash: ', 'content_hash', None),
     651            (b'DumpMalloc: ', 'malloc', None),
     652            (b'DumpJSHeap: ', 'js_heap', None),
     653        ]:
     654            if self._read_header(block, line, header[0], header[1], header[2]):
     655                return
     656
    651657        # Note, we're not reading ExpectedHash: here, but we could.
    652658        # If the line wasn't a header, we just append it to the content.
    653         block.content += line
     659        block.content = encode_if_necessary(block.content) + line
    654660
    655661    def _strip_eof(self, line):
    656         if line and line.endswith("#EOF\n"):
     662        if line and line.endswith(b"#EOF\n"):
    657663            return line[:-5], True
    658664        return line, False
     
    689695            if out_line:
    690696                self._check_for_driver_timeout(out_line)
    691                 if out_line[-1] != "\n":
     697                if out_line[-1] != '\n' and out_line[-1] != 10:
    692698                    _log.error("  %s -> Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug." % test_name)
    693699                content_length_before_header_check = block._content_length
     
    696702                # Don't wait until we're done with headers, just read the binary blob right now.
    697703                if content_length_before_header_check != block._content_length:
    698                     block.content = self._server_process.read_stdout(deadline, block._content_length)
     704                    block.content = encode_if_necessary(self._server_process.read_stdout(deadline, block._content_length))
    699705
    700706            if err_line:
     
    703709                elif self._check_for_address_sanitizer_violation(err_line):
    704710                    asan_violation_detected = True
    705                     self._crash_report_from_driver = ""
     711                    self._crash_report_from_driver = b''
    706712                    # ASan report starts with a nondescript line, we only detect the second line.
    707713                    end_of_previous_error_line = self.error_from_test.rfind('\n', 0, -1)
     
    709715                        self.error_from_test = self.error_from_test[:end_of_previous_error_line]
    710716                    else:
    711                         self.error_from_test = ""
     717                        self.error_from_test = ''
    712718                    # Symbolication can take a very long time, give it 10 extra minutes to finish.
    713719                    # FIXME: This can likely be removed once <rdar://problem/18701447> is fixed.
    714720                    deadline += 10 * 60 * 1000
    715721                if asan_violation_detected:
    716                     self._crash_report_from_driver += err_line
     722                    self._crash_report_from_driver += decode_for(err_line, str)
    717723                else:
    718                     self.error_from_test += err_line
     724                    self.error_from_test += decode_for(err_line, str)
    719725
    720726        if asan_violation_detected and not self._crashed_process_name:
     
    739745        self._content_length = None
    740746        # Content is treated as binary data even though the text output is usually UTF-8.
    741         self.content = str()  # FIXME: Should be bytearray() once we require Python 2.6.
     747        self.content = b''
    742748        self.decoded_content = None
    743749        self.malloc = None
     
    748754            self.decoded_content = base64.b64decode(self.content)
    749755        else:
    750             self.decoded_content = self.content
     756            try:
     757                self.decoded_content = decode_for(self.content, str)
     758            except UnicodeDecodeError:
     759                self.decoded_content = None
    751760
    752761
  • trunk/Tools/Scripts/webkitpy/port/driver_unittest.py

    r251808 r253804  
    183183        self.assertEqual(content_block.content_type, 'image/png')
    184184        self.assertEqual(content_block.content_hash, 'actual')
    185         self.assertEqual(content_block.content, '12345678\n')
     185        self.assertEqual(content_block.content, b'12345678\n')
    186186        self.assertEqual(content_block.decoded_content, '12345678\n')
    187187        driver._server_process = None
     
    202202        self.assertEqual(content_block.content_hash, 'actual')
    203203        self.assertEqual(content_block.encoding, 'base64')
    204         self.assertEqual(content_block.content, 'MTIzNDU2NzgK')
     204        self.assertEqual(content_block.content, b'MTIzNDU2NzgK')
    205205        self.assertEqual(content_block.decoded_content, b'12345678\n')
    206206
     
    248248
    249249        driver._server_process = FakeServerProcess(False)
    250         assert_crash(driver, '', False, None, None)
    251 
    252         driver._crashed_process_name = None
    253         driver._crashed_pid = None
    254         driver._server_process = FakeServerProcess(False)
    255         driver._driver_timed_out = False
    256         assert_crash(driver, '#CRASHED\n', True, 'FakeServerProcess', 1234)
    257 
    258         driver._crashed_process_name = None
    259         driver._crashed_pid = None
    260         driver._server_process = FakeServerProcess(False)
    261         driver._driver_timed_out = False
    262         assert_crash(driver, '#CRASHED - WebProcess\n', True, 'WebProcess', None)
    263 
    264         driver._crashed_process_name = None
    265         driver._crashed_pid = None
    266         driver._server_process = FakeServerProcess(False)
    267         driver._driver_timed_out = False
    268         assert_crash(driver, '#CRASHED - WebProcess (pid 8675)\n', True, 'WebProcess', 8675)
    269 
    270         driver._crashed_process_name = None
    271         driver._crashed_pid = None
    272         driver._server_process = FakeServerProcess(False)
    273         driver._driver_timed_out = False
    274         assert_crash(driver, '#PROCESS UNRESPONSIVE - WebProcess (pid 8675)\n', True, None, None, True)
    275 
    276         driver._crashed_process_name = None
    277         driver._crashed_pid = None
    278         driver._server_process = FakeServerProcess(False)
    279         driver._driver_timed_out = False
    280         assert_crash(driver, '#CRASHED - renderer (pid 8675)\n', True, 'renderer', 8675)
     250        assert_crash(driver, b'', False, None, None)
     251
     252        driver._crashed_process_name = None
     253        driver._crashed_pid = None
     254        driver._server_process = FakeServerProcess(False)
     255        driver._driver_timed_out = False
     256        assert_crash(driver, b'#CRASHED\n', True, 'FakeServerProcess', 1234)
     257
     258        driver._crashed_process_name = None
     259        driver._crashed_pid = None
     260        driver._server_process = FakeServerProcess(False)
     261        driver._driver_timed_out = False
     262        assert_crash(driver, b'#CRASHED - WebProcess\n', True, 'WebProcess', None)
     263
     264        driver._crashed_process_name = None
     265        driver._crashed_pid = None
     266        driver._server_process = FakeServerProcess(False)
     267        driver._driver_timed_out = False
     268        assert_crash(driver, b'#CRASHED - WebProcess (pid 8675)\n', True, 'WebProcess', 8675)
     269
     270        driver._crashed_process_name = None
     271        driver._crashed_pid = None
     272        driver._server_process = FakeServerProcess(False)
     273        driver._driver_timed_out = False
     274        assert_crash(driver, b'#PROCESS UNRESPONSIVE - WebProcess (pid 8675)\n', True, None, None, True)
     275
     276        driver._crashed_process_name = None
     277        driver._crashed_pid = None
     278        driver._server_process = FakeServerProcess(False)
     279        driver._driver_timed_out = False
     280        assert_crash(driver, b'#CRASHED - renderer (pid 8675)\n', True, 'renderer', 8675)
    281281
    282282        driver._crashed_process_name = None
     
    284284        driver._server_process = FakeServerProcess(True)
    285285        driver._driver_timed_out = False
    286         assert_crash(driver, '', True, 'FakeServerProcess', 1234)
     286        assert_crash(driver, b'', True, 'FakeServerProcess', 1234)
    287287
    288288    def test_creating_a_port_does_not_write_to_the_filesystem(self):
  • trunk/Tools/Scripts/webkitpy/port/image_diff.py

    r244080 r253804  
    3636
    3737from webkitpy.port import server_process
     38from webkitpy.common.unicode_compatibility import decode_for
    3839
    3940
     
    5859                self._start(tolerance)
    5960            # Note that although we are handed 'old', 'new', ImageDiff wants 'new', 'old'.
    60             self._process.write('Content-Length: %d\n%sContent-Length: %d\n%s' % (
    61                 len(actual_contents), actual_contents,
    62                 len(expected_contents), expected_contents))
     61            self._process.write('Content-Length: {}\n'.format(len(actual_contents)))
     62            self._process.write(actual_contents)
     63            self._process.write('Content-Length: {}\n'.format(len(expected_contents)))
     64            self._process.write(expected_contents)
    6365            return self._read()
    6466        except IOError as exception:
     
    8486                break
    8587
    86             if output.startswith('diff'):  # This is the last line ImageDiff prints.
     88            if output.startswith(b'diff'):  # This is the last line ImageDiff prints.
    8789                break
    8890
    89             if output.startswith('Content-Length'):
    90                 m = re.match('Content-Length: (\d+)', output)
     91            if output.startswith(b'Content-Length'):
     92                m = re.match(b'Content-Length: (\d+)', output)
    9193                content_length = int(m.group(1))
    9294                output_image = self._process.read_stdout(deadline, content_length)
     
    9496                break
    9597
    96         stderr = self._process.pop_all_buffered_stderr()
     98        stderr = decode_for(self._process.pop_all_buffered_stderr(), str)
    9799        err_str = ''
    98100        if stderr:
     
    104106
    105107        diff_percent = 0
    106         if output and output.startswith('diff'):
    107             m = re.match('diff: (.+)% (passed|failed)', output)
    108             if m.group(2) == 'passed':
     108        if output and output.startswith(b'diff'):
     109            m = re.match(b'diff: (.+)% (passed|failed)', output)
     110            if m.group(2) == b'passed':
    109111                return (None, 0, None)
    110112            diff_percent = float(m.group(1))
  • trunk/Tools/Scripts/webkitpy/port/server_process.py

    r253410 r253804  
    3636import time
    3737
     38from webkitpy.common.system.executive import ScriptError
     39from webkitpy.common.unicode_compatibility import encode_if_necessary
     40
    3841# Note that although win32 python does provide an implementation of
    3942# the win32 select API, it only works on sockets, and not on the named pipes
     
    4750    import os
    4851    import select
    49 
    50 from webkitpy.common.system.executive import ScriptError
    5152
    5253
     
    179180            self._start()
    180181        try:
    181             self._proc.stdin.write(bytes)
     182            self._proc.stdin.write(encode_if_necessary(bytes))
    182183            self._proc.stdin.flush()
    183         except IOError as e:
     184        except (IOError, ValueError) as e:
    184185            self.stop(0.0)
    185186            # stop() calls _reset(), so we have to set crashed to True after calling stop()
  • trunk/Tools/Scripts/webkitpy/port/server_process_mock.py

    r244080 r253804  
    2727# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    2828
     29from webkitpy.common.unicode_compatibility import encode_if_necessary
     30
    2931
    3032class MockServerProcess(object):
    3133    def __init__(self, port_obj=None, name=None, cmd=None, env=None, universal_newlines=False, lines=None, crashed=False, target_host=None):
    3234        self.timed_out = False
    33         self.lines = lines or []
     35        self.lines = [encode_if_necessary(line) for line in (lines or [])]
    3436        self.crashed = crashed
    3537        self.writes = []
     
    5355        if self.has_crashed():
    5456            return None
    55         return self.lines.pop(0) + "\n"
     57        return self.lines.pop(0) + b'\n'
    5658
    5759    def has_available_stdout(self):
     
    6870            remaining_size = size - len(first_line) - 1
    6971            if not remaining_size:
    70                 return first_line + "\n"
    71             return first_line + "\n" + self.read_stdout(deadline, remaining_size)
     72                return first_line + b'\n'
     73            return first_line + b'\n' + self.read_stdout(deadline, remaining_size)
    7274        result = self.lines[0][:size]
    7375        self.lines[0] = self.lines[0][size:]
  • trunk/Tools/Scripts/webkitpy/port/simulator_process.py

    r252275 r253804  
    2323
    2424import os
     25import sys
    2526import time
    2627
     
    6869        def close(self):
    6970            result = self._file.close()
    70             self.socket.close()
     71            # Closing the file implicitly closes the socket in Python 3
     72            if sys.version_info < (3, 0):
     73                self.socket.close()
    7174            return result
    7275
     
    104107            try:
    105108                # This order matches the client side connections in Tools/TestRunnerShared/IOSLayoutTestCommunication.cpp setUpIOSLayoutTestCommunication()
    106                 stdin = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'w')
     109                stdin = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'wb')
    107110                stdout = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'rb')
    108111                stderr = SimulatorProcess._accept_connection_create_file(self._target_host.listening_socket, 'rb')
  • trunk/Tools/Scripts/webkitpy/port/test.py

    r239989 r253804  
    3333from webkitpy.common.system.crashlogs import CrashLogs
    3434from webkitpy.common.version_name_map import PUBLIC_TABLE, VersionNameMap
     35from webkitpy.common.unicode_compatibility import decode_for, encode_if_necessary
    3536
    3637
     
    112113
    113114def unit_test_list():
    114     silent_audio = "RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
    115     silent_audio_with_single_bit_difference = "RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
    116     audio2 = "RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x40\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
     115    silent_audio = b"RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
     116    silent_audio_with_single_bit_difference = b"RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
     117    audio2 = b"RIFF2\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00\x22\x56\x00\x00\x44\xAC\x00\x00\x02\x00\x10\x00data\x0E\x00\x00\x00\x40\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
    117118    tests = TestList()
    118119    tests.add('failures/expected/crash.html', crash=True)
     
    435436
    436437    def diff_image(self, expected_contents, actual_contents, tolerance=None):
     438        expected_contents = encode_if_necessary(expected_contents)
     439        actual_contents = encode_if_necessary(actual_contents)
    437440        diffed = actual_contents != expected_contents
    438441        if not actual_contents and not expected_contents:
     
    440443        if not actual_contents or not expected_contents:
    441444            return (True, 0, None)
    442         if 'ref' in expected_contents:
     445        if b'ref' in expected_contents:
    443446            assert tolerance == 0
    444447        if diffed:
    445             return ("< %s\n---\n> %s\n" % (expected_contents, actual_contents), 1, None)
     448            return ("< {}\n---\n> {}\n".format(
     449                decode_for(expected_contents, str),
     450                decode_for(actual_contents, str),
     451            ), 1, None)
    446452        return (None, 0, None)
    447453
     
    564570        test = self._port._tests[test_name]
    565571        if test.keyboard:
    566             raise KeyboardInterrupt
     572            raise KeyboardInterrupt()
    567573        if test.exception:
    568574            raise ValueError('exception from ' + test_name)
  • trunk/Tools/Scripts/webkitpy/xcode/simulated_device.py

    r252738 r253804  
    564564            return True
    565565
    566         system_processes = self.executive.run_command([SimulatedDeviceManager.xcrun, 'simctl', 'spawn', self.udid, 'launchctl', 'print', 'system'], decode_output=False)
     566        system_processes = self.executive.run_command([SimulatedDeviceManager.xcrun, 'simctl', 'spawn', self.udid, 'launchctl', 'print', 'system'], decode_output=True)
    567567        if re.search(r'"{}"'.format(home_screen_service), system_processes) or re.search(r'A\s+{}'.format(home_screen_service), system_processes):
    568568            return True
     
    605605    def install_app(self, app_path, env=None):
    606606        # Even after carousel is running, it takes a few seconds for watchOS to allow installs.
    607         for i in xrange(self.NUM_INSTALL_RETRIES):
     607        for i in range(self.NUM_INSTALL_RETRIES):
    608608            exit_code = self.executive.run_command(['xcrun', 'simctl', 'install', self.udid, app_path], return_exit_code=True)
    609609            if exit_code == 0:
Note: See TracChangeset for help on using the changeset viewer.