Changeset 101107 in webkit


Ignore:
Timestamp:
Nov 23, 2011, 3:23:21 PM (13 years ago)
Author:
eric@webkit.org
Message:

Add Environment object to Host and fix the GCC smartquotes trouble seen on the commit-queue
​https://bugs.webkit.org/show_bug.cgi?id=71983

Reviewed by Adam Barth.

We'll add more code to Environment overtime,
allowing us to mock out more of our direct interactions with os.environ.

This patch also makes run_command print the passed in environment.

  • Scripts/webkitpy/common/host_mock.py:
  • Scripts/webkitpy/common/system/environment.py: Copied from Tools/Scripts/webkitpy/tool/steps/build.py.
  • Scripts/webkitpy/common/system/environment_mock.py: Copied from Tools/Scripts/webkitpy/tool/steps/build.py.
  • Scripts/webkitpy/common/system/environment_unittest.py: Copied from Tools/Scripts/webkitpy/tool/steps/build.py.
  • Scripts/webkitpy/common/system/executive.py:
  • Scripts/webkitpy/common/system/executive_mock.py:
  • Scripts/webkitpy/layout_tests/port/webkit.py:
  • Scripts/webkitpy/layout_tests/port/webkit_unittest.py:
  • Scripts/webkitpy/tool/commands/download_unittest.py:
  • Scripts/webkitpy/tool/steps/build.py:
Location:
trunk/Tools
Files:
9 edited
2 copied

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r101089 r101107  
     12011-11-23  Eric Seidel  <eric@webkit.org>
     2
     3        Add Environment object to Host and fix the GCC smartquotes trouble seen on the commit-queue
     4        https://bugs.webkit.org/show_bug.cgi?id=71983
     5
     6        Reviewed by Adam Barth.
     7
     8        We'll add more code to Environment overtime,
     9        allowing us to mock out more of our direct interactions with os.environ.
     10
     11        This patch also makes run_command print the passed in environment.
     12
     13        * Scripts/webkitpy/common/host_mock.py:
     14        * Scripts/webkitpy/common/system/environment.py: Copied from Tools/Scripts/webkitpy/tool/steps/build.py.
     15        * Scripts/webkitpy/common/system/environment_mock.py: Copied from Tools/Scripts/webkitpy/tool/steps/build.py.
     16        * Scripts/webkitpy/common/system/environment_unittest.py: Copied from Tools/Scripts/webkitpy/tool/steps/build.py.
     17        * Scripts/webkitpy/common/system/executive.py:
     18        * Scripts/webkitpy/common/system/executive_mock.py:
     19        * Scripts/webkitpy/layout_tests/port/webkit.py:
     20        * Scripts/webkitpy/layout_tests/port/webkit_unittest.py:
     21        * Scripts/webkitpy/tool/commands/download_unittest.py:
     22        * Scripts/webkitpy/tool/steps/build.py:
     23
    1242011-11-23  Tor Arne Vestbø  <tor.arne.vestbo@nokia.com>
    225

  • trunk/Tools/Scripts/webkitpy/common/host.py

    r100558 r101107  
    2929
    3030import logging
     31import os
    3132import sys
    3233
    … …  
    9596                _log.debug('Failed to engage svn.bat Windows hack.')
    9697
     98    def copy_current_environment(self):
     99        return Environment(os.environ.copy())
     100
    97101    def _initialize_scm(self, patch_directories=None):
    98102        if sys.platform == "win32":

  • trunk/Tools/Scripts/webkitpy/common/host_mock.py

    r100558 r101107  
    3232from webkitpy.common.net.buildbot.buildbot_mock import MockBuildBot
    3333from webkitpy.common.net.web_mock import MockWeb
     34from webkitpy.common.system.environment import Environment
    3435from webkitpy.common.system.executive_mock import MockExecutive
    3536from webkitpy.common.system.filesystem_mock import MockFileSystem
    … …  
    6768        self._watch_list = MockWatchList()
    6869
     70    def copy_current_environment(self):
     71        return Environment({"MOCK_ENVIRON_COPY": '1'})
     72
    6973    def _initialize_scm(self, patch_directories=None):
    7074        pass

  • trunk/Tools/Scripts/webkitpy/common/system/environment.py

    r101104 r101107  
    1 # Copyright (C) 2010 Google Inc. All rights reserved.
    2 # 
     1# Copyright (C) 2011 Google Inc. All rights reserved.
     2#
    33# Redistribution and use in source and binary forms, with or without
    44# modification, are permitted provided that the following conditions are
    55# met:
    6 # 
     6#
    77#     * Redistributions of source code must retain the above copyright
    88# notice, this list of conditions and the following disclaimer.
    … …  
    1414# contributors may be used to endorse or promote products derived from
    1515# this software without specific prior written permission.
    16 # 
     16#
    1717# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
    1818# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
    … …  
    2727# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    2828
    29 from webkitpy.tool.steps.abstractstep import AbstractStep
    30 from webkitpy.tool.steps.options import Options
    31 from webkitpy.common.system.deprecated_logging import log
    3229
     30class Environment(object):
     31    def __init__(self, env=None):
     32        self.env = env or {}
    3333
    34 class Build(AbstractStep):
    35     @classmethod
    36     def options(cls):
    37         return AbstractStep.options() + [
    38             Options.build,
    39             Options.quiet,
    40             Options.build_style,
    41         ]
     34    def to_dictionary(self):
     35        return self.env
    4236
    43     def build(self, build_style):
    44         self._tool.executive.run_and_throw_if_fail(self._tool.port().build_webkit_command(build_style=build_style), self._options.quiet, cwd=self._tool.scm().checkout_root)
    45 
    46     def run(self, state):
    47         if not self._options.build:
    48             return
    49         log("Building WebKit")
    50         if self._options.build_style == "both":
    51             self.build("debug")
    52             self.build("release")
    53         else:
    54             self.build(self._options.build_style)
     37    def disable_gcc_smartquotes(self):
     38        # Technically we only need to set LC_CTYPE to disable current
     39        # smartquote behavior: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38363
     40        # Apple's XCode sets LC_ALL instead, probably to be future-proof.
     41        self.env['LC_ALL'] = 'C'

  • trunk/Tools/Scripts/webkitpy/common/system/environment_unittest.py

    r101104 r101107  
    1 # Copyright (C) 2010 Google Inc. All rights reserved.
    2 # 
     1# Copyright (C) 2011 Google Inc. All rights reserved.
     2#
    33# Redistribution and use in source and binary forms, with or without
    44# modification, are permitted provided that the following conditions are
    55# met:
    6 # 
    7 #     * Redistributions of source code must retain the above copyright
     6#
     7#    * Redistributions of source code must retain the above copyright
    88# notice, this list of conditions and the following disclaimer.
    9 #     * Redistributions in binary form must reproduce the above
     9#    * Redistributions in binary form must reproduce the above
    1010# copyright notice, this list of conditions and the following disclaimer
    1111# in the documentation and/or other materials provided with the
    1212# distribution.
    13 #     * Neither the name of Google Inc. nor the names of its
     13#    * Neither the name of Google Inc. nor the names of its
    1414# contributors may be used to endorse or promote products derived from
    1515# this software without specific prior written permission.
    16 # 
     16#
    1717# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
    1818# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
    … …  
    2727# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    2828
    29 from webkitpy.tool.steps.abstractstep import AbstractStep
    30 from webkitpy.tool.steps.options import Options
    31 from webkitpy.common.system.deprecated_logging import log
     29import unittest
     30
     31from .environment import Environment
    3232
    3333
    34 class Build(AbstractStep):
    35     @classmethod
    36     def options(cls):
    37         return AbstractStep.options() + [
    38             Options.build,
    39             Options.quiet,
    40             Options.build_style,
    41         ]
    42 
    43     def build(self, build_style):
    44         self._tool.executive.run_and_throw_if_fail(self._tool.port().build_webkit_command(build_style=build_style), self._options.quiet, cwd=self._tool.scm().checkout_root)
    45 
    46     def run(self, state):
    47         if not self._options.build:
    48             return
    49         log("Building WebKit")
    50         if self._options.build_style == "both":
    51             self.build("debug")
    52             self.build("release")
    53         else:
    54             self.build(self._options.build_style)
     34class EnvironmentTest(unittest.TestCase):
     35    def test_disable_gcc_smartquotes(self):
     36        environment = Environment({})
     37        environment.disable_gcc_smartquotes()
     38        env = environment.to_dictionary()
     39        self.assertEqual(env['LC_ALL'], 'C')

  • trunk/Tools/Scripts/webkitpy/common/system/executive.py

    r92188 r101107  
    109109        return sys.platform not in ('win32', 'cygwin')
    110110
    111     def _run_command_with_teed_output(self, args, teed_output, cwd=None):
     111    def _run_command_with_teed_output(self, args, teed_output, **kwargs):
    112112        args = map(unicode, args)  # Popen will throw an exception if args are non-strings (like int())
    113113        args = map(self._encode_argument_if_needed, args)
    … …  
    117117                                   stderr=self.STDOUT,
    118118                                   close_fds=self._should_close_fds(),
    119                                    cwd=cwd)
     119                                   **kwargs)
    120120
    121121        # Use our own custom wait loop because Popen ignores a tee'd
    … …  
    137137    # like "build-webkit" where we want to display to the user that we're building
    138138    # but still have the output to stuff into a log file.
    139     def run_and_throw_if_fail(self, args, quiet=False, decode_output=True, cwd=None):
     139    def run_and_throw_if_fail(self, args, quiet=False, decode_output=True, **kwargs):
    140140        # Cache the child's output locally so it can be used for error reports.
    141141        child_out_file = StringIO.StringIO()
    … …  
    145145            tee_stdout = dev_null
    146146        child_stdout = tee(child_out_file, tee_stdout)
    147         exit_code = self._run_command_with_teed_output(args, child_stdout, cwd=cwd)
     147        exit_code = self._run_command_with_teed_output(args, child_stdout, **kwargs)
    148148        if quiet:
    149149            dev_null.close()

  • trunk/Tools/Scripts/webkitpy/common/system/executive_mock.py

    r99381 r101107  
    4545        return pid in self._running_pids
    4646
    47     def run_and_throw_if_fail(self, args, quiet=False, cwd=None):
     47    def run_and_throw_if_fail(self, args, quiet=False, cwd=None, env=None):
    4848        if self._should_log:
    49             log("MOCK run_and_throw_if_fail: %s, cwd=%s" % (args, cwd))
     49            env_string = ""
     50            if env:
     51                env_string = ", env=%s" % env
     52            log("MOCK run_and_throw_if_fail: %s, cwd=%s%s" % (args, cwd, env_string))
    5053        if self._should_throw_when_run.intersection(args):
    5154            raise ScriptError("Exception for %s" % args)
    … …  
    5962                    return_exit_code=False,
    6063                    return_stderr=True,
    61                     decode_output=False):
     64                    decode_output=False,
     65                    env=None):
    6266        assert(isinstance(args, list) or isinstance(args, tuple))
    6367        if self._should_log:
    64             log("MOCK run_command: %s, cwd=%s" % (args, cwd))
     68            env_string = ""
     69            if env:
     70                env_string = ", env=%s" % env
     71            log("MOCK run_command: %s, cwd=%s%s" % (args, cwd, env_string))
    6572        if self._should_throw:
    6673            raise ScriptError("MOCK ScriptError")
    … …  
    93100                    return_exit_code=False,
    94101                    return_stderr=True,
    95                     decode_output=False):
     102                    decode_output=False,
     103                    env=None):
    96104        assert(isinstance(args, list) or isinstance(args, tuple))
    97105        if self._exception:

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py

    r100674 r101107  
    4242from webkitpy.common.memoized import memoized
    4343from webkitpy.common.net.buildbot import BuildBot
     44from webkitpy.common.system.environment import Environment
    4445from webkitpy.common.system.executive import Executive, ScriptError
    4546from webkitpy.layout_tests.port import builders, server_process, Port, Driver, DriverOutput
    … …  
    8182        return self._filesystem.join(self._webkit_baseline_path(expectations_directory), 'test_expectations.txt')
    8283
    83     def _driver_build_script_name(self):
    84         if self.get_option('webkit_test_runner'):
    85             return "build-webkittestrunner"
    86         return "build-dumprendertree"
    87 
    8884    def _port_flag_for_scripts(self):
    8985        # This is overrriden by ports which need a flag passed to scripts to distinguish the use of that port.
    … …  
    10298        return config_args
    10399
    104     def _run_script(self, script_name, args=None, include_configuration_arguments=True, decode_output=True):
     100    def _run_script(self, script_name, args=None, include_configuration_arguments=True, decode_output=True, env=None):
    105101        run_script_command = [self._config.script_path(script_name)]
    106102        if include_configuration_arguments:
    … …  
    108104        if args:
    109105            run_script_command.extend(args)
    110         return self._executive.run_command(run_script_command, cwd=self._config.webkit_base_dir(), decode_output=decode_output)
     106        return self._executive.run_command(run_script_command, cwd=self._config.webkit_base_dir(), decode_output=decode_output, env=env)
    111107
    112108    def _build_driver(self):
     109        environment = self.host.copy_current_environment()
     110        environment.disable_gcc_smartquotes()
     111        env = environment.to_dictionary()
     112
    113113        # FIXME: We build both DumpRenderTree and WebKitTestRunner for
    114114        # WebKitTestRunner runs because DumpRenderTree still includes
    … …  
    117117        # projects.
    118118        try:
    119             self._run_script("build-dumprendertree")
     119            self._run_script("build-dumprendertree", env=env)
    120120            if self.get_option('webkit_test_runner'):
    121                 self._run_script("build-webkittestrunner")
     121                self._run_script("build-webkittestrunner", env=env)
    122122        except ScriptError:
    123123            _log.error("Failed to build %s" % self.driver_name())

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit_unittest.py

    r100494 r101107  
    178178        port._executive = MockExecutive(should_log=True)
    179179        port._options = MockOptions(configuration="Release")  # This should not be necessary, but I think TestWebKitPort is actually reading from disk (and thus detects the current configuration).
    180         expected_stderr = "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release'], cwd=/mock-checkout\n"
     180        expected_stderr = "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release'], cwd=/mock-checkout, env={'LC_ALL': 'C', 'MOCK_ENVIRON_COPY': '1'}\n"
    181181        self.assertTrue(output.assert_outputs(self, port._build_driver, expected_stderr=expected_stderr))
    182182
    183183        # Make sure when passed --webkit-test-runner web build the right tool.
    184184        port._options = MockOptions(webkit_test_runner=True, configuration="Release")
    185         expected_stderr = "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release'], cwd=/mock-checkout\nMOCK run_command: ['Tools/Scripts/build-webkittestrunner', '--release'], cwd=/mock-checkout\n"
     185        expected_stderr = "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release'], cwd=/mock-checkout, env={'LC_ALL': 'C', 'MOCK_ENVIRON_COPY': '1'}\nMOCK run_command: ['Tools/Scripts/build-webkittestrunner', '--release'], cwd=/mock-checkout, env={'LC_ALL': 'C', 'MOCK_ENVIRON_COPY': '1'}\n"
    186186        self.assertTrue(output.assert_outputs(self, port._build_driver, expected_stderr=expected_stderr))
    187187
    … …  
    189189        port._executive = MockExecutive(should_log=True, should_throw=True)
    190190        # Because WK2 currently has to build both webkittestrunner and DRT, if DRT fails, that's the only one it tries.
    191         expected_stderr = "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release'], cwd=/mock-checkout\n"
     191        expected_stderr = "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release'], cwd=/mock-checkout, env={'LC_ALL': 'C', 'MOCK_ENVIRON_COPY': '1'}\n"
    192192        self.assertFalse(output.assert_outputs(self, port._build_driver, expected_stderr=expected_stderr))
    193193

  • trunk/Tools/Scripts/webkitpy/tool/commands/download_unittest.py

    r99140 r101107  
    133133Was that diff correct?
    134134Building WebKit
    135 MOCK run_and_throw_if_fail: ['mock-build-webkit'], cwd=/mock-checkout
     135MOCK run_and_throw_if_fail: ['mock-build-webkit'], cwd=/mock-checkout, env={'LC_ALL': 'C', 'MOCK_ENVIRON_COPY': '1'}
    136136Running Python unit tests
    137137MOCK run_and_throw_if_fail: ['mock-test-webkitpy'], cwd=/mock-checkout

  • trunk/Tools/Scripts/webkitpy/tool/steps/build.py

    r91210 r101107  
    4242
    4343    def build(self, build_style):
    44         self._tool.executive.run_and_throw_if_fail(self._tool.port().build_webkit_command(build_style=build_style), self._options.quiet, cwd=self._tool.scm().checkout_root)
     44        environment = self._tool.copy_current_environment()
     45        environment.disable_gcc_smartquotes()
     46        env = environment.to_dictionary()
     47
     48        build_webkit_command = self._tool.port().build_webkit_command(build_style=build_style)
     49        self._tool.executive.run_and_throw_if_fail(build_webkit_command, self._options.quiet,
     50            cwd=self._tool.scm().checkout_root, env=env)
    4551
    4652    def run(self, state):
Note: See TracChangeset for help on using the changeset viewer.