Changeset 71547 in webkit


Ignore:
Timestamp:
Nov 8, 2010 11:12:44 AM (13 years ago)
Author:
Adam Roben
Message:

Make webkitpy.common.system.executive_unittest pass when running under Win32 Python

Fixes <http://webkit.org/b/49033>.

Reviewed by Dave Levin and Eric Seidel.

  • Scripts/webkitpy/common/system/executive.py:

(Executive._run_command_with_teed_output): Pass the arguments through
encode_argument_if_needed rather than using Cygwin-specific code here.
(Executive.run_and_throw_if_fail): Use child_process_encoding to decode
the output.
(Executive.run_command): Use encode_argument_if_needed to encode the
arguments and child_process_encoding to decode the output.
(Executive._child_process_encoding): Returns the encoding that should be
used when communicating with child processes. On Windows we use mbcs,
which maps to the current code page. On all other platforms we use
UTF-8.
(Executive._should_encode_child_process_arguments): Returns True if
arguments to child processes need to be encoded. This is currently
only needed on Cygwin and Win32 Python 2.x.
(Executive._encode_argument_if_needed): Encode the argument using
child_process_encoding if we need to encode arguments to child
processes on this platform.

  • Scripts/webkitpy/common/system/executive_unittest.py:

(never_ending_command): Added. Returns arguments to run a command that
will not quit until we kill it. On Windows we use wmic, on other
platforms we use yes.
(ExecutiveTest.test_run_command_with_unicode): Changed to expect the
mbcs encoding to be used and for output from the child processes to
have been roundtripped through encode/decode on Win32 Python. When
UTF-8 is the encoding the roundtripping is undetectable, but with mbcs
it's possible that some characters will not be able to be converted
and will be replaced by question marks; the round-tripping allows us
to expect this result.

(ExecutiveTest.test_kill_process):
(ExecutiveTest.test_kill_all):
Use never_ending_command instead of invoking "yes" directly. Expect an
exit code of 1 when using Win32 Python, as that's what seems to happen.

Location:
trunk/WebKitTools
Files:
3 edited

Legend:

Unmodified
Added
Removed
  • trunk/WebKitTools/ChangeLog

    r71545 r71547  
     12010-11-05  Adam Roben  <aroben@apple.com>
     2
     3        Make webkitpy.common.system.executive_unittest pass when running under
     4        Win32 Python
     5
     6        Fixes <http://webkit.org/b/49033>.
     7
     8        Reviewed by Dave Levin and Eric Seidel.
     9
     10        * Scripts/webkitpy/common/system/executive.py:
     11        (Executive._run_command_with_teed_output): Pass the arguments through
     12        encode_argument_if_needed rather than using Cygwin-specific code here.
     13        (Executive.run_and_throw_if_fail): Use child_process_encoding to decode
     14        the output.
     15        (Executive.run_command): Use encode_argument_if_needed to encode the
     16        arguments and child_process_encoding to decode the output.
     17        (Executive._child_process_encoding): Returns the encoding that should be
     18        used when communicating with child processes. On Windows we use mbcs,
     19        which maps to the current code page. On all other platforms we use
     20        UTF-8.
     21        (Executive._should_encode_child_process_arguments): Returns True if
     22        arguments to child processes need to be encoded. This is currently
     23        only needed on Cygwin and Win32 Python 2.x.
     24        (Executive._encode_argument_if_needed): Encode the argument using
     25        child_process_encoding if we need to encode arguments to child
     26        processes on this platform.
     27
     28        * Scripts/webkitpy/common/system/executive_unittest.py:
     29        (never_ending_command): Added. Returns arguments to run a command that
     30        will not quit until we kill it. On Windows we use wmic, on other
     31        platforms we use yes.
     32        (ExecutiveTest.test_run_command_with_unicode): Changed to expect the
     33        mbcs encoding to be used and for output from the child processes to
     34        have been roundtripped through encode/decode on Win32 Python. When
     35        UTF-8 is the encoding the roundtripping is undetectable, but with mbcs
     36        it's possible that some characters will not be able to be converted
     37        and will be replaced by question marks; the round-tripping allows us
     38        to expect this result.
     39
     40        (ExecutiveTest.test_kill_process):
     41        (ExecutiveTest.test_kill_all):
     42        Use never_ending_command instead of invoking "yes" directly. Expect an
     43        exit code of 1 when using Win32 Python, as that's what seems to happen.
     44
    1452010-11-08  Adam Roben  <aroben@apple.com>
    246
  • trunk/WebKitTools/Scripts/webkitpy/common/system/executive.py

    r71338 r71547  
    4646
    4747from webkitpy.common.system.deprecated_logging import tee
     48from webkitpy.python24 import versioning
    4849
    4950
     
    105106    def _run_command_with_teed_output(self, args, teed_output):
    106107        args = map(unicode, args)  # Popen will throw an exception if args are non-strings (like int())
    107         if sys.platform == 'cygwin':
    108             # Cygwin's Python's os.execv doesn't support unicode command
    109             # arguments, and neither does Cygwin's execv itself.
    110             # FIXME: Using UTF-8 here will confuse Windows-native commands
    111             # which will expect arguments to be encoded using the current code
    112             # page.
    113             args = [arg.encode('utf-8') for arg in args]
     108        args = map(self._encode_argument_if_needed, args)
     109
    114110        child_process = subprocess.Popen(args,
    115111                                         stdout=subprocess.PIPE,
     
    150146        child_out_file.close()
    151147
    152         # We assume the child process output utf-8
    153148        if decode_output:
    154             child_output = child_output.decode("utf-8")
     149            child_output = child_output.decode(self._child_process_encoding())
    155150
    156151        if exit_code:
     
    311306        # FIXME: We may need to encode differently on different platforms.
    312307        if isinstance(input, unicode):
    313             input = input.encode("utf-8")
     308            input = input.encode(self._child_process_encoding())
    314309        return (subprocess.PIPE, input)
    315310
     
    339334        start_time = time.time()
    340335        args = map(unicode, args)  # Popen will throw an exception if args are non-strings (like int())
    341         if sys.platform == 'cygwin':
    342             # Cygwin's Python's os.execv doesn't support unicode command
    343             # arguments, and neither does Cygwin's execv itself.
    344             # FIXME: Using UTF-8 here will confuse Windows-native commands
    345             # which will expect arguments to be encoded using the current code
    346             # page.
    347             args = [arg.encode('utf-8') for arg in args]
     336        args = map(self._encode_argument_if_needed, args)
     337
    348338        stdin, string_to_communicate = self._compute_stdin(input)
    349339        stderr = subprocess.STDOUT if return_stderr else None
     
    356346                                   close_fds=self._should_close_fds())
    357347        output = process.communicate(string_to_communicate)[0]
     348
    358349        # run_command automatically decodes to unicode() unless explicitly told not to.
    359350        if decode_output:
    360             output = output.decode("utf-8")
     351            output = output.decode(self._child_process_encoding())
     352
    361353        # wait() is not threadsafe and can throw OSError due to:
    362354        # http://bugs.python.org/issue1731717
     
    375367            (error_handler or self.default_error_handler)(script_error)
    376368        return output
     369
     370    def _child_process_encoding(self):
     371        # Win32 Python 2.x uses CreateProcessA rather than CreateProcessW
     372        # to launch subprocesses, so we have to encode arguments using the
     373        # current code page.
     374        if sys.platform == 'win32' and versioning.compare_version(sys, '3.0')[0] < 0:
     375            return 'mbcs'
     376        # All other platforms use UTF-8.
     377        # FIXME: Using UTF-8 on Cygwin will confuse Windows-native commands
     378        # which will expect arguments to be encoded using the current code
     379        # page.
     380        return 'utf-8'
     381
     382    def _should_encode_child_process_arguments(self):
     383        # Cygwin's Python's os.execv doesn't support unicode command
     384        # arguments, and neither does Cygwin's execv itself.
     385        if sys.platform == 'cygwin':
     386            return True
     387
     388        # Win32 Python 2.x uses CreateProcessA rather than CreateProcessW
     389        # to launch subprocesses, so we have to encode arguments using the
     390        # current code page.
     391        if sys.platform == 'win32' and versioning.compare_version(sys, '3.0')[0] < 0:
     392            return True
     393
     394        return False
     395
     396    def _encode_argument_if_needed(self, argument):
     397        if not self._should_encode_child_process_arguments():
     398            return argument
     399        return argument.encode(self._child_process_encoding())
  • trunk/WebKitTools/Scripts/webkitpy/common/system/executive_unittest.py

    r71338 r71547  
    3838
    3939
     40def never_ending_command():
     41    """Arguments for a command that will never end (useful for testing process
     42    killing). It should be a process that is unlikely to already be running
     43    because all instances will be killed."""
     44    if sys.platform == 'win32':
     45        return ['wmic']
     46    return ['yes']
     47
     48
    4049class ExecutiveTest(unittest.TestCase):
    4150
     
    5665        to Executive.run* methods, and they will return unicode()
    5766        objects by default unless decode_output=False"""
     67        unicode_tor_input = u"WebKit \u2661 Tor Arne Vestb\u00F8!"
     68        if sys.platform == 'win32':
     69            encoding = 'mbcs'
     70        else:
     71            encoding = 'utf-8'
     72        encoded_tor = unicode_tor_input.encode(encoding)
     73        # On Windows, we expect the unicode->mbcs->unicode roundtrip to be
     74        # lossy. On other platforms, we expect a lossless roundtrip.
     75        if sys.platform == 'win32':
     76            unicode_tor_output = encoded_tor.decode(encoding)
     77        else:
     78            unicode_tor_output = unicode_tor_input
     79
    5880        executive = Executive()
    59         unicode_tor = u"WebKit \u2661 Tor Arne Vestb\u00F8!"
    60         utf8_tor = unicode_tor.encode("utf-8")
    6181
    62         output = executive.run_command(cat.command_arguments(), input=unicode_tor)
    63         self.assertEquals(output, unicode_tor)
     82        output = executive.run_command(cat.command_arguments(), input=unicode_tor_input)
     83        self.assertEquals(output, unicode_tor_output)
    6484
    65         output = executive.run_command(echo.command_arguments("-n", unicode_tor))
    66         self.assertEquals(output, unicode_tor)
     85        output = executive.run_command(echo.command_arguments("-n", unicode_tor_input))
     86        self.assertEquals(output, unicode_tor_output)
    6787
    68         output = executive.run_command(echo.command_arguments("-n", unicode_tor), decode_output=False)
    69         self.assertEquals(output, utf8_tor)
     88        output = executive.run_command(echo.command_arguments("-n", unicode_tor_input), decode_output=False)
     89        self.assertEquals(output, encoded_tor)
    7090
    7191        # Make sure that str() input also works.
    72         output = executive.run_command(cat.command_arguments(), input=utf8_tor, decode_output=False)
    73         self.assertEquals(output, utf8_tor)
     92        output = executive.run_command(cat.command_arguments(), input=encoded_tor, decode_output=False)
     93        self.assertEquals(output, encoded_tor)
    7494
    7595        # FIXME: We should only have one run* method to test
    76         output = executive.run_and_throw_if_fail(echo.command_arguments("-n", unicode_tor), quiet=True)
    77         self.assertEquals(output, unicode_tor)
     96        output = executive.run_and_throw_if_fail(echo.command_arguments("-n", unicode_tor_input), quiet=True)
     97        self.assertEquals(output, unicode_tor_output)
    7898
    79         output = executive.run_and_throw_if_fail(echo.command_arguments("-n", unicode_tor), quiet=True, decode_output=False)
    80         self.assertEquals(output, utf8_tor)
     99        output = executive.run_and_throw_if_fail(echo.command_arguments("-n", unicode_tor_input), quiet=True, decode_output=False)
     100        self.assertEquals(output, encoded_tor)
    81101
    82102    def test_kill_process(self):
    83103        executive = Executive()
    84         # We use "yes" because it loops forever.
    85         process = subprocess.Popen(["yes"], stdout=subprocess.PIPE)
     104        process = subprocess.Popen(never_ending_command(), stdout=subprocess.PIPE)
    86105        self.assertEqual(process.poll(), None)  # Process is running
    87106        executive.kill_process(process.pid)
    88107        # Note: Can't use a ternary since signal.SIGKILL is undefined for sys.platform == "win32"
    89108        if sys.platform == "win32":
    90             expected_exit_code = 0  # taskkill.exe results in exit(0)
     109            expected_exit_code = 1
    91110        else:
    92111            expected_exit_code = -signal.SIGKILL
     
    112131        executive = Executive()
    113132        # We use "yes" because it loops forever.
    114         process = subprocess.Popen(["yes"], stdout=subprocess.PIPE)
     133        process = subprocess.Popen(never_ending_command(), stdout=subprocess.PIPE)
    115134        self.assertEqual(process.poll(), None)  # Process is running
    116         executive.kill_all("yes")
     135        executive.kill_all(never_ending_command()[0])
    117136        # Note: Can't use a ternary since signal.SIGTERM is undefined for sys.platform == "win32"
    118         if sys.platform in ("win32", "cygwin"):
    119             expected_exit_code = 0  # taskkill.exe results in exit(0)
     137        if sys.platform == "cygwin":
     138            expected_exit_code = 0  # os.kill results in exit(0) for this process.
     139        elif sys.platform == "win32":
     140            expected_exit_code = 1
    120141        else:
    121142            expected_exit_code = -signal.SIGTERM
    122143        self.assertEqual(process.wait(), expected_exit_code)
    123144        # Killing again should fail silently.
    124         executive.kill_all("yes")
     145        executive.kill_all(never_ending_command()[0])
    125146
    126147    def test_check_running_pid(self):
Note: See TracChangeset for help on using the changeset viewer.