Changeset 121497 in webkit


Ignore:
Timestamp:
Jun 28, 2012 6:14:24 PM (12 years ago)
Author:
dpranke@chromium.org
Message:

derive ChromiumPort from WebKitPort in NRWT in order to support skipping tests if symbols are missing
​https://bugs.webkit.org/show_bug.cgi?id=89706

Reviewed by Ojan Vafai.

Try again to land the change first landed in r121363. This patch
adds a bunch more tests and reworks the handling of
port-specific default values for --pixel-tests and --time-out-ms
to be more consistent (adding a default_pixel_tests() method,
pushing the webkit default_timeout_ms() value up into base.py,
and overriding it properly in the chromium and apple mac ports.

Also change the logic in
run_webkit_tests._setup_derived_options() to not second-guess
what the port wants the timeout value to be for debug builds;
computing this in two different places led to several bugs.

This change also changes the Chromium unittest ports to derive
from ChromiumPortTestCase instead of PortTestCase, so that we
ensure that we're running the same tests on all port variants
more easily. There's more cleanup that can be done here, but
this is good enough for now

  • Scripts/webkitpy/layout_tests/port/base.py:

(Port.default_pixel_tests):
(Port):
(Port.default_timeout_ms):

  • Scripts/webkitpy/layout_tests/port/chromium.py:

(ChromiumPort):
(ChromiumPort.init):
(ChromiumPort.default_pixel_tests):
(ChromiumPort.default_timeout_ms):
(ChromiumPort.driver_name):
(ChromiumPort._driver_class):
(ChromiumPort._missing_symbol_to_skipped_tests):
(ChromiumPort.skipped_layout_tests):
(ChromiumPort.setup_test_run):
(ChromiumPort._path_to_image_diff):
(ChromiumPort._convert_path):

  • Scripts/webkitpy/layout_tests/port/chromium_android_unittest.py:

(ChromiumAndroidPortTest):
(ChromiumAndroidPortTest.test_expectations_files):

  • Scripts/webkitpy/layout_tests/port/chromium_linux.py:

(ChromiumLinuxPort._modules_to_search_for_symbols):

  • Scripts/webkitpy/layout_tests/port/chromium_linux_unittest.py:

(ChromiumLinuxPortTest):

  • Scripts/webkitpy/layout_tests/port/chromium_mac.py:

(ChromiumMacPort._modules_to_search_for_symbols):

  • Scripts/webkitpy/layout_tests/port/chromium_mac_unittest.py:

(ChromiumMacPortTest):

  • Scripts/webkitpy/layout_tests/port/chromium_port_testcase.py:

split off from chromium_unittest.

  • Scripts/webkitpy/layout_tests/port/chromium_win.py:

(ChromiumWinPort._modules_to_search_for_symbols):

  • Scripts/webkitpy/layout_tests/port/chromium_win_unittest.py:

(ChromiumWinTest):

  • Scripts/webkitpy/layout_tests/port/mac.py:

(MacPort.default_timeout_ms):

  • Scripts/webkitpy/layout_tests/port/mac_unittest.py:

(MacTest.test_default_timeout_ms):

  • Scripts/webkitpy/layout_tests/port/mock_drt.py:

(MockDRTPort.start_http_server):

  • Scripts/webkitpy/layout_tests/port/port_testcase.py:

(PortTestCase.test_default_timeout_ms):
(PortTestCase):
(PortTestCase.test_default_pixel_tests):

  • Scripts/webkitpy/layout_tests/port/test.py:

(TestPort.default_pixel_tests):

  • Scripts/webkitpy/layout_tests/port/webkit.py:

(WebKitPort._modules_to_search_for_symbols):
(WebKitPort):
(WebKitPort._symbols_string):
(WebKitPort._skipped_tests_for_unsupported_features):

  • Scripts/webkitpy/layout_tests/port/webkit_unittest.py:

(TestWebKitPort._symbols_string):
(TestWebKitPort._tests_for_disabled_features):

  • Scripts/webkitpy/layout_tests/run_webkit_tests.py:

(_set_up_derived_options):

Location:
trunk/Tools
Files:
19 edited
1 copied

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r121493 r121497  
     12012-06-28  Dirk Pranke  <dpranke@chromium.org>
     2
     3        derive ChromiumPort from WebKitPort in NRWT in order to support skipping tests if symbols are missing
     4        https://bugs.webkit.org/show_bug.cgi?id=89706
     5
     6        Reviewed by Ojan Vafai.
     7
     8        Try again to land the change first landed in r121363. This patch
     9        adds a bunch more tests and reworks the handling of
     10        port-specific default values for --pixel-tests and --time-out-ms
     11        to be more consistent (adding a default_pixel_tests() method,
     12        pushing the webkit default_timeout_ms() value up into base.py,
     13        and overriding it properly in the chromium and apple mac ports.
     14
     15        Also change the logic in
     16        run_webkit_tests._setup_derived_options() to not second-guess
     17        what the port wants the timeout value to be for debug builds;
     18        computing this in two different places led to several bugs.
     19
     20        This change also changes the Chromium unittest ports to derive
     21        from ChromiumPortTestCase instead of PortTestCase, so that we
     22        ensure that we're running the same tests on all port variants
     23        more easily. There's more cleanup that can be done here, but
     24        this is good enough for now
     25
     26        * Scripts/webkitpy/layout_tests/port/base.py:
     27        (Port.default_pixel_tests):
     28        (Port):
     29        (Port.default_timeout_ms):
     30        * Scripts/webkitpy/layout_tests/port/chromium.py:
     31        (ChromiumPort):
     32        (ChromiumPort.__init__):
     33        (ChromiumPort.default_pixel_tests):
     34        (ChromiumPort.default_timeout_ms):
     35        (ChromiumPort.driver_name):
     36        (ChromiumPort._driver_class):
     37        (ChromiumPort._missing_symbol_to_skipped_tests):
     38        (ChromiumPort.skipped_layout_tests):
     39        (ChromiumPort.setup_test_run):
     40        (ChromiumPort._path_to_image_diff):
     41        (ChromiumPort._convert_path):
     42        * Scripts/webkitpy/layout_tests/port/chromium_android_unittest.py:
     43        (ChromiumAndroidPortTest):
     44        (ChromiumAndroidPortTest.test_expectations_files):
     45        * Scripts/webkitpy/layout_tests/port/chromium_linux.py:
     46        (ChromiumLinuxPort._modules_to_search_for_symbols):
     47        * Scripts/webkitpy/layout_tests/port/chromium_linux_unittest.py:
     48        (ChromiumLinuxPortTest):
     49        * Scripts/webkitpy/layout_tests/port/chromium_mac.py:
     50        (ChromiumMacPort._modules_to_search_for_symbols):
     51        * Scripts/webkitpy/layout_tests/port/chromium_mac_unittest.py:
     52        (ChromiumMacPortTest):
     53        * Scripts/webkitpy/layout_tests/port/chromium_port_testcase.py:
     54        split off from chromium_unittest.
     55        * Scripts/webkitpy/layout_tests/port/chromium_win.py:
     56        (ChromiumWinPort._modules_to_search_for_symbols):
     57        * Scripts/webkitpy/layout_tests/port/chromium_win_unittest.py:
     58        (ChromiumWinTest):
     59        * Scripts/webkitpy/layout_tests/port/mac.py:
     60        (MacPort.default_timeout_ms):
     61        * Scripts/webkitpy/layout_tests/port/mac_unittest.py:
     62        (MacTest.test_default_timeout_ms):
     63        * Scripts/webkitpy/layout_tests/port/mock_drt.py:
     64        (MockDRTPort.start_http_server):
     65        * Scripts/webkitpy/layout_tests/port/port_testcase.py:
     66        (PortTestCase.test_default_timeout_ms):
     67        (PortTestCase):
     68        (PortTestCase.test_default_pixel_tests):
     69        * Scripts/webkitpy/layout_tests/port/test.py:
     70        (TestPort.default_pixel_tests):
     71        * Scripts/webkitpy/layout_tests/port/webkit.py:
     72        (WebKitPort._modules_to_search_for_symbols):
     73        (WebKitPort):
     74        (WebKitPort._symbols_string):
     75        (WebKitPort._skipped_tests_for_unsupported_features):
     76        * Scripts/webkitpy/layout_tests/port/webkit_unittest.py:
     77        (TestWebKitPort._symbols_string):
     78        (TestWebKitPort._tests_for_disabled_features):
     79        * Scripts/webkitpy/layout_tests/run_webkit_tests.py:
     80        (_set_up_derived_options):
     81
    1822012-06-28  Joshua Bell  <jsbell@chromium.org>
    283

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py

    r120865 r121497  
    137137        self._results_directory = None
    138138
    139     def default_test_timeout_ms(self):
    140         return 6 * 1000
     139    def default_pixel_tests(self):
     140        # FIXME: Disable until they are run by default on build.webkit.org.
     141        return False
     142
     143    def default_timeout_ms(self):
     144        return 35 * 1000
    141145
    142146    def wdiff_available(self):

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium.py

    r121406 r121497  
    4444from webkitpy.layout_tests.port.base import Port, VirtualTestSuite
    4545from webkitpy.layout_tests.port.driver import DriverOutput
    46 from webkitpy.layout_tests.port.webkit import WebKitDriver
     46from webkitpy.layout_tests.port.webkit import WebKitPort, WebKitDriver
    4747
    4848
    … …  
    5050
    5151
    52 class ChromiumPort(Port):
     52class ChromiumPort(WebKitPort):
    5353    """Abstract base class for Chromium implementations of the Port class."""
    5454
    … …  
    108108
    109109    def __init__(self, host, port_name, **kwargs):
    110         Port.__init__(self, host, port_name, **kwargs)
     110        super(ChromiumPort, self).__init__(host, port_name, **kwargs)
    111111        # All sub-classes override this, but we need an initial value for testing.
    112112        self._chromium_base_dir_path = None
     113
     114    def default_pixel_tests(self):
     115        return True
     116
     117    def default_timeout_ms(self):
     118        if self.get_option('configuration') == 'Debug':
     119            return 12 * 1000
     120        return 6 * 1000
    113121
    114122    def _check_file_exists(self, path_to_file, file_description,
    … …  
    131139        return True
    132140
     141    def driver_name(self):
     142        # FIXME: merge this with Port.driver_name, WebKitPort.driver_name
     143        if self.get_option('driver_name'):
     144            return self.get_option('driver_name')
     145        return 'DumpRenderTree'
    133146
    134147    def check_build(self, needs_http):
    … …  
    258271            return self._build_path(self.get_option('configuration'), 'layout-test-results')
    259272
     273    def _driver_class(self):
     274        return ChromiumDriver
     275
     276    def _missing_symbol_to_skipped_tests(self):
     277        # FIXME: Should WebKitPort have these definitions also?
     278        return {
     279            "ff_mp3_decoder": ["webaudio/codec-tests/mp3"],
     280            "ff_aac_decoder": ["webaudio/codec-tests/aac"],
     281        }
     282
     283    def skipped_layout_tests(self, test_list):
     284        # FIXME: Merge w/ WebKitPort.skipped_layout_tests()
     285        return set(self._skipped_tests_for_unsupported_features(test_list))
     286
    260287    def setup_test_run(self):
    261288        # Delete the disk cache if any to ensure a clean test run.
    … …  
    265292        if self._filesystem.exists(cachedir):
    266293            self._filesystem.rmtree(cachedir)
    267 
    268     def _driver_class(self):
    269         return ChromiumDriver
    270294
    271295    def start_helper(self):
    … …  
    366390    def _build_path(self, *comps):
    367391        return self._static_build_path(self._filesystem, self.get_option('build_directory'), self.path_from_chromium_base(), self.path_from_webkit_base(), *comps)
     392
     393    def _path_to_image_diff(self):
     394        binary_name = 'ImageDiff'
     395        return self._build_path(self.get_option('configuration'), binary_name)
    368396
    369397    def _check_driver_build_up_to_date(self, configuration):
    … …  
    401429            return cygpath(path)
    402430        return path
    403 
    404     def _path_to_image_diff(self):
    405         binary_name = 'ImageDiff'
    406         return self._build_path(self.get_option('configuration'), binary_name)
    407431
    408432

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_android_unittest.py

    r118244 r121497  
    3636
    3737from webkitpy.layout_tests.port import chromium_android
    38 from webkitpy.layout_tests.port import port_testcase
     38from webkitpy.layout_tests.port import chromium_port_testcase
    3939from webkitpy.layout_tests.port import Port
    4040
    4141
    42 class ChromiumAndroidPortTest(port_testcase.PortTestCase):
     42class ChromiumAndroidPortTest(chromium_port_testcase.ChromiumPortTestCase):
    4343    port_name = 'chromium-android'
    4444    port_maker = chromium_android.ChromiumAndroidPort
    … …  
    4949        self.assertTrue(port.get_option('enable_hardware_gpu'))
    5050        self.assertEquals(port.baseline_path(), port._webkit_baseline_path('chromium-android'))
     51
     52    def test_expectations_files(self):
     53        # FIXME: override this test temporarily while we're still upstreaming the android port and
     54        # using a custom expectations file.
     55        pass
    5156
    5257    @staticmethod

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_linux.py

    r121406 r121497  
    111111        return map(self._webkit_baseline_path, port_names)
    112112
     113    def _modules_to_search_for_symbols(self):
     114        return [self._build_path(self.get_option('configuration'), 'libffmpegsumo.so')]
     115
    113116    def check_build(self, needs_http):
    114117        result = chromium.ChromiumPort.check_build(self, needs_http)

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_linux_unittest.py

    r119649 r121497  
    3434
    3535from webkitpy.layout_tests.port import chromium_linux
    36 from webkitpy.layout_tests.port import port_testcase
     36from webkitpy.layout_tests.port import chromium_port_testcase
    3737
    3838
    39 class ChromiumLinuxPortTest(port_testcase.PortTestCase):
     39class ChromiumLinuxPortTest(chromium_port_testcase.ChromiumPortTestCase):
    4040    port_name = 'chromium-linux'
    4141    port_maker = chromium_linux.ChromiumLinuxPort

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_mac.py

    r121406 r121497  
    8989        return map(self._webkit_baseline_path, fallback_paths[self._version])
    9090
     91    def _modules_to_search_for_symbols(self):
     92        return [self._build_path(self.get_option('configuration'), 'ffmpegsumo.so')]
     93
    9194    def check_build(self, needs_http):
    9295        result = chromium.ChromiumPort.check_build(self, needs_http)

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_mac_unittest.py

    r120865 r121497  
    3030
    3131from webkitpy.layout_tests.port import chromium_mac
    32 from webkitpy.layout_tests.port import port_testcase
     32from webkitpy.layout_tests.port import chromium_port_testcase
    3333from webkitpy.tool.mocktool import MockOptions
    3434
    3535
    36 class ChromiumMacPortTest(port_testcase.PortTestCase):
     36class ChromiumMacPortTest(chromium_port_testcase.ChromiumPortTestCase):
    3737    os_name = 'mac'
    3838    os_version = 'leopard'

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_port_testcase.py

    r121495 r121497  
    2727# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    2828
    29 import StringIO
    30 import sys
    31 import time
    3229import unittest
    3330
    3431from webkitpy.common.system import logtesting
    35 from webkitpy.common.system.executive_mock import MockExecutive, MockExecutive2
    36 from webkitpy.common.system.filesystem_mock import MockFileSystem
     32from webkitpy.common.system.executive_mock import MockExecutive2
    3733from webkitpy.common.system.systemhost_mock import MockSystemHost
    38 from webkitpy.layout_tests.port.config_mock import MockConfig
    39 from webkitpy.thirdparty.mock import Mock
    4034from webkitpy.tool.mocktool import MockOptions
    4135
    42 import chromium
    4336import chromium_android
    4437import chromium_linux
    … …  
    4841from webkitpy.layout_tests.models.test_configuration import TestConfiguration
    4942from webkitpy.layout_tests.port import port_testcase
    50 from webkitpy.layout_tests.port.driver import DriverInput
    51 
    52 
    53 class ChromiumDriverTest(unittest.TestCase):
    54     def setUp(self):
    55         host = MockSystemHost()
    56         options = MockOptions(configuration='Release', additional_drt_flag=['--test-shell'])
    57         config = MockConfig(filesystem=host.filesystem, default_configuration='Release')
    58         self.port = chromium_mac.ChromiumMacPort(host, 'chromium-mac-snowleopard', options=options, config=config)
    59         self.driver = chromium.ChromiumDriver(self.port, worker_number=0, pixel_tests=True)
    60 
    61     def test_test_shell_command(self):
    62         expected_command = "test.html 2 checksum\n"
    63         self.assertEqual(self.driver._test_shell_command("test.html", 2, "checksum"), expected_command)
    64 
    65     def _assert_write_command_and_read_line(self, input=None, expected_line=None, expected_stdin=None, expected_crash=False):
    66         if not expected_stdin:
    67             if input:
    68                 expected_stdin = input
    69             else:
    70                 # We reset stdin, so we should expect stdin.getValue = ""
    71                 expected_stdin = ""
    72         self.driver._proc.stdin = StringIO.StringIO()
    73         line, did_crash = self.driver._write_command_and_read_line(input)
    74         self.assertEqual(self.driver._proc.stdin.getvalue(), expected_stdin)
    75         self.assertEqual(line, expected_line)
    76         self.assertEqual(did_crash, expected_crash)
    77 
    78     def test_write_command_and_read_line(self):
    79         self.driver._proc = Mock()  # FIXME: This should use a tighter mock.
    80         # Set up to read 3 lines before we get an IOError
    81         self.driver._proc.stdout = StringIO.StringIO("first\nsecond\nthird\n")
    82 
    83         unicode_input = u"I \u2661 Unicode"
    84         utf8_input = unicode_input.encode("utf-8")
    85         # Test unicode input conversion to utf-8
    86         self._assert_write_command_and_read_line(input=unicode_input, expected_stdin=utf8_input, expected_line="first\n")
    87         # Test str() input.
    88         self._assert_write_command_and_read_line(input="foo", expected_line="second\n")
    89         # Test input=None
    90         self._assert_write_command_and_read_line(expected_line="third\n")
    91         # Test reading from a closed/empty stream.
    92         # reading from a StringIO does not raise IOError like a real file would, so raise IOError manually.
    93         def mock_readline():
    94             raise IOError
    95         self.driver._proc.stdout.readline = mock_readline
    96         self._assert_write_command_and_read_line(expected_crash=True)
    97 
    98     def test_crash_log(self):
    99         self.driver._proc = Mock()
    100 
    101         # Simulate a crash by having stdout close unexpectedly.
    102         def mock_readline():
    103             raise IOError
    104         self.driver._proc.stdout.readline = mock_readline
    105         self.driver._proc.pid = 1234
    106 
    107         self.driver.test_to_uri = lambda test: 'mocktesturi'
    108         self.driver._port.driver_name = lambda: 'mockdriver'
    109         self.driver._port._get_crash_log = lambda name, pid, out, err, newer_than: 'mockcrashlog'
    110         driver_output = self.driver.run_test(DriverInput(test_name='some/test.html', timeout=1, image_hash=None, should_run_pixel_test=False))
    111         self.assertTrue(driver_output.crash)
    112         self.assertEqual(driver_output.crashed_process_name, 'mockdriver')
    113         self.assertEqual(driver_output.crashed_pid, 1234)
    114         self.assertEqual(driver_output.crash_log, 'mockcrashlog')
    115 
    116     def test_stop(self):
    117         self.pid = None
    118         self.wait_called = False
    119         self.driver._proc = Mock()  # FIXME: This should use a tighter mock.
    120         self.driver._proc.pid = 1
    121         self.driver._proc.stdin = StringIO.StringIO()
    122         self.driver._proc.stdout = StringIO.StringIO()
    123         self.driver._proc.stderr = StringIO.StringIO()
    124         self.driver._proc.poll = lambda: None
    125 
    126         def fake_wait():
    127             self.assertTrue(self.pid is not None)
    128             self.wait_called = True
    129 
    130         self.driver._proc.wait = fake_wait
    131 
    132         class FakeExecutive(object):
    133             def kill_process(other, pid):
    134                 self.pid = pid
    135                 self.driver._proc.poll = lambda: 2
    136 
    137         self.driver._port._executive = FakeExecutive()
    138         self.driver.KILL_TIMEOUT_DEFAULT = 0.01
    139         self.driver.stop()
    140         self.assertTrue(self.wait_called)
    141         self.assertEquals(self.pid, 1)
    142 
    143     def test_two_drivers(self):
    144 
    145         class MockDriver(chromium.ChromiumDriver):
    146             def __init__(self, port):
    147                 chromium.ChromiumDriver.__init__(self, port, worker_number=0, pixel_tests=False)
    148 
    149             def cmd_line(self, pixel_test, per_test_args):
    150                 return 'python'
    151 
    152         # get_option is used to get the timeout (ms) for a process before we kill it.
    153         driver1 = MockDriver(self.port)
    154         driver1._start(False, [])
    155         driver2 = MockDriver(self.port)
    156         driver2._start(False, [])
    157         # It's possible for driver1 to timeout when stopping if it's sharing stdin with driver2.
    158         start_time = time.time()
    159         driver1.stop()
    160         driver2.stop()
    161         self.assertTrue(time.time() - start_time < 20)
    162 
    163     def test_stop_cleans_up_properly(self):
    164         self.driver._test_shell = False
    165         self.driver.start(True, [])
    166         last_tmpdir = self.port._filesystem.last_tmpdir
    167         self.assertNotEquals(last_tmpdir, None)
    168         self.driver.stop()
    169         self.assertFalse(self.port._filesystem.isdir(last_tmpdir))
    170 
    171     def test_two_starts_cleans_up_properly(self):
    172         # clone the WebKitDriverTest tests here since we override start() and stop()
    173         self.driver._test_shell = False
    174         self.driver.start(True, [])
    175         last_tmpdir = self.port._filesystem.last_tmpdir
    176         self.driver._start(True, [])
    177         self.assertFalse(self.port._filesystem.isdir(last_tmpdir))
    178 
    179 class ChromiumPortTest(port_testcase.PortTestCase):
    180     port_name = 'chromium-mac'
    181     port_maker = chromium.ChromiumPort
     43
     44
     45class ChromiumPortTestCase(port_testcase.PortTestCase):
     46
     47    def test_default_timeout_ms(self):
     48        self.assertEquals(self.make_port(options=MockOptions(configuration='Release')).default_timeout_ms(), 6000)
     49        self.assertEquals(self.make_port(options=MockOptions(configuration='Debug')).default_timeout_ms(), 12000)
     50
     51    def test_default_pixel_tests(self):
     52        self.assertEquals(self.make_port().default_pixel_tests(), True)
     53
     54    def test_missing_symbol_to_skipped_tests(self):
     55        # Test that we get the chromium skips and not the webkit default skips
     56        port = self.make_port()
     57        skip_dict = port._missing_symbol_to_skipped_tests()
     58        self.assertTrue('ff_mp3_decoder' in skip_dict)
     59        self.assertFalse('WebGLShader' in skip_dict)
    18260
    18361    def test_all_test_configurations(self):
    … …  
    20381        ]))
    20482
    205     def test_driver_cmd_line(self):
    206         # Override this test since ChromiumPort doesn't implement driver_cmd_line().
    207         pass
    208 
    209     def test_check_build(self):
    210         # Override this test since ChromiumPort doesn't implement _path_to_driver().
    211         pass
    212 
    213     def test_check_wdiff(self):
    214         # Override this test since ChromiumPort doesn't implement _path_to_driver().
    215         pass
    216 
    21783    class TestMacPort(chromium_mac.ChromiumMacPort):
    21884        def __init__(self, options=None):
    … …  
    253119    def test_default_configuration(self):
    254120        mock_options = MockOptions()
    255         port = ChromiumPortTest.TestLinuxPort(options=mock_options)
     121        port = ChromiumPortTestCase.TestLinuxPort(options=mock_options)
    256122        self.assertEquals(mock_options.configuration, 'default')
    257123        self.assertTrue(port.default_configuration_called)
    258124
    259125        mock_options = MockOptions(configuration=None)
    260         port = ChromiumPortTest.TestLinuxPort(mock_options)
     126        port = ChromiumPortTestCase.TestLinuxPort(mock_options)
    261127        self.assertEquals(mock_options.configuration, 'default')
    262128        self.assertTrue(port.default_configuration_called)
    263129
    264130    def test_diff_image(self):
    265         class TestPort(ChromiumPortTest.TestLinuxPort):
     131        class TestPort(ChromiumPortTestCase.TestLinuxPort):
    266132            def _path_to_image_diff(self):
    267133                return "/path/to/image_diff"
    268134
    269         port = ChromiumPortTest.TestLinuxPort()
     135        port = ChromiumPortTestCase.TestLinuxPort()
    270136        mock_image_diff = "MOCK Image Diff"
    271137
    … …  
    319185class ChromiumPortLoggingTest(logtesting.LoggingTestCase):
    320186    def test_check_sys_deps(self):
    321         port = ChromiumPortTest.TestLinuxPort()
     187        port = ChromiumPortTestCase.TestLinuxPort()
    322188
    323189        # Success

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_unittest.py

    r121406 r121497  
    2828
    2929import StringIO
    30 import sys
    3130import time
    3231import unittest
    3332
    3433from webkitpy.common.system import logtesting
    35 from webkitpy.common.system.executive_mock import MockExecutive, MockExecutive2
    36 from webkitpy.common.system.filesystem_mock import MockFileSystem
     34from webkitpy.common.system.executive_mock import MockExecutive2
    3735from webkitpy.common.system.systemhost_mock import MockSystemHost
    3836from webkitpy.layout_tests.port.config_mock import MockConfig
    … …  
    4139
    4240import chromium
    43 import chromium_android
    44 import chromium_linux
    4541import chromium_mac
    46 import chromium_win
    4742
    48 from webkitpy.layout_tests.models.test_configuration import TestConfiguration
    49 from webkitpy.layout_tests.port import port_testcase
     43from webkitpy.layout_tests.port import chromium_port_testcase
    5044from webkitpy.layout_tests.port.driver import DriverInput
    5145
    … …  
    177171        self.assertFalse(self.port._filesystem.isdir(last_tmpdir))
    178172
    179 class ChromiumPortTest(port_testcase.PortTestCase):
    180     port_name = 'chromium-mac'
    181     port_maker = chromium.ChromiumPort
    182 
    183     def test_all_test_configurations(self):
    184         """Validate the complete set of configurations this port knows about."""
    185         port = self.make_port()
    186         self.assertEquals(set(port.all_test_configurations()), set([
    187             TestConfiguration('icecreamsandwich', 'x86', 'debug'),
    188             TestConfiguration('icecreamsandwich', 'x86', 'release'),
    189             TestConfiguration('leopard', 'x86', 'debug'),
    190             TestConfiguration('leopard', 'x86', 'release'),
    191             TestConfiguration('snowleopard', 'x86', 'debug'),
    192             TestConfiguration('snowleopard', 'x86', 'release'),
    193             TestConfiguration('lion', 'x86', 'debug'),
    194             TestConfiguration('lion', 'x86', 'release'),
    195             TestConfiguration('xp', 'x86', 'debug'),
    196             TestConfiguration('xp', 'x86', 'release'),
    197             TestConfiguration('win7', 'x86', 'debug'),
    198             TestConfiguration('win7', 'x86', 'release'),
    199             TestConfiguration('lucid', 'x86', 'debug'),
    200             TestConfiguration('lucid', 'x86', 'release'),
    201             TestConfiguration('lucid', 'x86_64', 'debug'),
    202             TestConfiguration('lucid', 'x86_64', 'release'),
    203         ]))
    204 
    205     def test_driver_cmd_line(self):
    206         # Override this test since ChromiumPort doesn't implement driver_cmd_line().
    207         pass
    208 
    209     def test_check_build(self):
    210         # Override this test since ChromiumPort doesn't implement _path_to_driver().
    211         pass
    212 
    213     def test_check_wdiff(self):
    214         # Override this test since ChromiumPort doesn't implement _path_to_driver().
    215         pass
    216 
    217     class TestMacPort(chromium_mac.ChromiumMacPort):
    218         def __init__(self, options=None):
    219             options = options or MockOptions()
    220             chromium_mac.ChromiumMacPort.__init__(self, MockSystemHost(os_name='mac', os_version='leopard'), 'chromium-mac-leopard', options=options)
    221 
    222         def default_configuration(self):
    223             self.default_configuration_called = True
    224             return 'default'
    225 
    226     class TestAndroidPort(chromium_android.ChromiumAndroidPort):
    227         def __init__(self, options=None):
    228             options = options or MockOptions()
    229             chromium_win.ChromiumAndroidPort.__init__(self, MockSystemHost(os_name='android', os_version='icecreamsandwich'), 'chromium-android', options=options)
    230 
    231         def default_configuration(self):
    232             self.default_configuration_called = True
    233             return 'default'
    234 
    235     class TestLinuxPort(chromium_linux.ChromiumLinuxPort):
    236         def __init__(self, options=None):
    237             options = options or MockOptions()
    238             chromium_linux.ChromiumLinuxPort.__init__(self, MockSystemHost(os_name='linux', os_version='lucid'), 'chromium-linux-x86', options=options)
    239 
    240         def default_configuration(self):
    241             self.default_configuration_called = True
    242             return 'default'
    243 
    244     class TestWinPort(chromium_win.ChromiumWinPort):
    245         def __init__(self, options=None):
    246             options = options or MockOptions()
    247             chromium_win.ChromiumWinPort.__init__(self, MockSystemHost(os_name='win', os_version='xp'), 'chromium-win-xp', options=options)
    248 
    249         def default_configuration(self):
    250             self.default_configuration_called = True
    251             return 'default'
    252 
    253     def test_default_configuration(self):
    254         mock_options = MockOptions()
    255         port = ChromiumPortTest.TestLinuxPort(options=mock_options)
    256         self.assertEquals(mock_options.configuration, 'default')
    257         self.assertTrue(port.default_configuration_called)
    258 
    259         mock_options = MockOptions(configuration=None)
    260         port = ChromiumPortTest.TestLinuxPort(mock_options)
    261         self.assertEquals(mock_options.configuration, 'default')
    262         self.assertTrue(port.default_configuration_called)
    263 
    264     def test_diff_image(self):
    265         class TestPort(ChromiumPortTest.TestLinuxPort):
    266             def _path_to_image_diff(self):
    267                 return "/path/to/image_diff"
    268 
    269         port = ChromiumPortTest.TestLinuxPort()
    270         mock_image_diff = "MOCK Image Diff"
    271 
    272         def mock_run_command(args):
    273             port._filesystem.write_binary_file(args[4], mock_image_diff)
    274             return 1
    275 
    276         # Images are different.
    277         port._executive = MockExecutive2(run_command_fn=mock_run_command)
    278         self.assertEquals(mock_image_diff, port.diff_image("EXPECTED", "ACTUAL")[0])
    279 
    280         # Images are the same.
    281         port._executive = MockExecutive2(exit_code=0)
    282         self.assertEquals(None, port.diff_image("EXPECTED", "ACTUAL")[0])
    283 
    284         # There was some error running image_diff.
    285         port._executive = MockExecutive2(exit_code=2)
    286         exception_raised = False
    287         try:
    288             port.diff_image("EXPECTED", "ACTUAL")
    289         except ValueError, e:
    290             exception_raised = True
    291         self.assertFalse(exception_raised)
    292 
    293     def test_expectations_files(self):
    294         port = self.make_port()
    295         port.port_name = 'chromium'
    296 
    297         expectations_path = port.path_to_test_expectations_file()
    298         chromium_overrides_path = port.path_from_chromium_base(
    299             'webkit', 'tools', 'layout_tests', 'test_expectations.txt')
    300         skia_overrides_path = port.path_from_chromium_base(
    301             'skia', 'skia_test_expectations.txt')
    302 
    303         port._options.builder_name = 'DUMMY_BUILDER_NAME'
    304         self.assertEquals(port.expectations_files(), [expectations_path, skia_overrides_path, chromium_overrides_path])
    305 
    306         port._options.builder_name = 'builder (deps)'
    307         self.assertEquals(port.expectations_files(), [expectations_path, skia_overrides_path, chromium_overrides_path])
    308 
    309         # A builder which does NOT observe the Chromium test_expectations,
    310         # but still observes the Skia test_expectations...
    311         port._options.builder_name = 'builder'
    312         self.assertEquals(port.expectations_files(), [expectations_path, skia_overrides_path])
    313 
    314     def test_expectations_ordering(self):
    315         # since we don't implement self.port_name in ChromiumPort.
    316         pass
    317 
    318173
    319174class ChromiumPortLoggingTest(logtesting.LoggingTestCase):
     175
     176    # FIXME: put this someplace more useful
    320177    def test_check_sys_deps(self):
    321         port = ChromiumPortTest.TestLinuxPort()
     178        port = chromium_port_testcase.ChromiumPortTestCase.TestLinuxPort()
    322179
    323180        # Success

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_win.py

    r121406 r121497  
    107107        return map(self._webkit_baseline_path, port_names)
    108108
     109    def _modules_to_search_for_symbols(self):
     110        # FIXME: we should return the path to the ffmpeg equivalents to detect if we have the mp3 and aac codecs installed.
     111        # See https://bugs.webkit.org/show_bug.cgi?id=89706.
     112        return []
     113
    109114    def check_build(self, needs_http):
    110115        result = chromium.ChromiumPort.check_build(self, needs_http)

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_win_unittest.py

    r121266 r121497  
    3333from webkitpy.common.system.executive_mock import MockExecutive
    3434from webkitpy.common.system.filesystem_mock import MockFileSystem
     35from webkitpy.layout_tests.port import chromium_port_testcase
    3536from webkitpy.layout_tests.port import chromium_win
    36 from webkitpy.layout_tests.port import port_testcase
    3737from webkitpy.tool.mocktool import MockOptions
    3838
    3939
    40 class ChromiumWinTest(port_testcase.PortTestCase):
     40class ChromiumWinTest(chromium_port_testcase.ChromiumPortTestCase):
    4141    port_name = 'chromium-win'
    4242    port_maker = chromium_win.ChromiumWinPort

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/mac.py

    r120738 r121497  
    6666            self.set_option_default("batch_size", 1000)
    6767
     68    def default_timeout_ms(self):
     69        if self.get_option('guard_malloc'):
     70            return 350 * 1000
     71        return super(MacPort, self).default_timeout_ms()
     72
    6873    def _build_driver_flags(self):
    6974        return ['ARCHS=i386'] if self.architecture() == 'x86' else []

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/mac_unittest.py

    r120738 r121497  
    4646        self.assertEqual(port._skipped_file_search_paths(), expected_paths)
    4747
     48    def test_default_timeout_ms(self):
     49        super(MacTest, self).test_default_timeout_ms()
     50        self.assertEquals(self.make_port(options=MockOptions(guard_malloc=True)).default_timeout_ms(), 350000)
     51
    4852    def test_skipped_file_search_paths(self):
    4953        self.assert_skipped_file_search_paths('mac-snowleopard', set(['mac-snowleopard', 'mac']))

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/mock_drt.py

    r114788 r121497  
    105105        pass
    106106
    107     def start_http_server(self):
     107    def start_http_server(self, number_of_servers):
    108108        pass
    109109

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/port_testcase.py

    r120236 r121497  
    4545
    4646
    47 class PortTestCase(unittest.TestCase):
     47class PortTestCase(object):
    4848    """Tests that all Port implementations must pass."""
    4949    HTTP_PORTS = (8000, 8080, 8443)
    … …  
    6262        port_name = self.port_maker.determine_full_port_name(host, options, port_name)
    6363        return self.port_maker(host, port_name, options=options, config=config, **kwargs)
     64
     65    def test_default_timeout_ms(self):
     66        self.assertEquals(self.make_port(options=MockOptions(configuration='Release')).default_timeout_ms(), 35000)
     67        self.assertEquals(self.make_port(options=MockOptions(configuration='Debug')).default_timeout_ms(), 35000)
     68
     69    def test_default_pixel_tests(self):
     70        self.assertEquals(self.make_port().default_pixel_tests(), False)
    6471
    6572    def test_driver_cmd_line(self):

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/test.py

    r120851 r121497  
    358358        self._version = version_map[port_name]
    359359
     360    def default_pixel_tests(self):
     361        return True
     362
    360363    def _path_to_driver(self):
    361364        # This routine shouldn't normally be called, but it is called by

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py

    r121406 r121497  
    5252
    5353class WebKitPort(Port):
    54     def __init__(self, host, port_name=None, **kwargs):
    55         Port.__init__(self, host, port_name=port_name, **kwargs)
    56 
    57         # FIXME: Disable pixel tests until they are run by default on build.webkit.org.
    58         self.set_option_default("pixel_tests", False)
    59         # WebKit ports expect a 35s timeout, or 350s timeout when running with -g/--guard-malloc.
    60         # FIXME: --guard-malloc is only supported on Mac, so this logic should be in mac.py.
    61         default_time_out_seconds = 350 if self.get_option('guard_malloc') else 35
    62         self.set_option_default("time_out_ms", default_time_out_seconds * 1000)
    6354
    6455    def driver_name(self):
    … …  
    265256        return 'nm'
    266257
    267     def _webcore_symbols_string(self):
    268         webcore_library_path = self._path_to_webcore_library()
    269         if not webcore_library_path:
    270             return None
    271         try:
    272             return self._executive.run_command([self.nm_command(), webcore_library_path], error_handler=Executive.ignore_error)
    273         except OSError, e:
    274             _log.warn("Failed to run nm: %s.  Can't determine WebCore supported features." % e)
    275         return None
     258    def _modules_to_search_for_symbols(self):
     259        path = self._path_to_webcore_library()
     260        if path:
     261            return [path]
     262        return []
     263
     264    def _symbols_string(self):
     265        symbols = ''
     266        for path_to_module in self._modules_to_search_for_symbols():
     267            try:
     268                symbols += self._executive.run_command([self.nm_command(), path_to_module], error_handler=Executive.ignore_error)
     269            except OSError, e:
     270                _log.warn("Failed to run nm: %s.  Can't determine supported features correctly." % e)
     271        return symbols
    276272
    277273    # Ports which use run-time feature detection should define this method and return
    … …  
    332328        if self._has_test_in_directories(self._missing_symbol_to_skipped_tests().values(), test_list):
    333329            # Runtime feature detection not supported, fallback to static dectection:
    334             # Disable any tests for symbols missing from the webcore symbol string.
    335             webcore_symbols_string = self._webcore_symbols_string()
    336             if webcore_symbols_string is not None:
    337                 return reduce(operator.add, [directories for symbol_substring, directories in self._missing_symbol_to_skipped_tests().items() if symbol_substring not in webcore_symbols_string], [])
     330            # Disable any tests for symbols missing from the executable or libraries.
     331            symbols_string = self._symbols_string()
     332            if symbols_string is not None:
     333                return reduce(operator.add, [directories for symbol_substring, directories in self._missing_symbol_to_skipped_tests().items() if symbol_substring not in symbols_string], [])
    338334
    339335        # Failed to get any runtime or symbol information, don't skip any tests.

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit_unittest.py

    r121406 r121497  
    5454        return [self.test_configuration()]
    5555
    56     def _webcore_symbols_string(self):
     56    def _symbols_string(self):
    5757        return self.symbols_string
    5858
    … …  
    6262    def _tests_for_disabled_features(self):
    6363        return ["accessibility", ]
    64 
    65 
    66 class WebKitPortUnitTests(unittest.TestCase):
    67     def test_default_options(self):
    68         # The WebKit ports override new-run-webkit-test default options.
    69         options = MockOptions(pixel_tests=None, time_out_ms=None)
    70         port = WebKitPort(MockSystemHost(), options=options)
    71         self.assertEquals(port._options.pixel_tests, False)
    72         self.assertEquals(port._options.time_out_ms, 35000)
    73 
    74         # Note that we don't override options if specified by the user.
    75         options = MockOptions(pixel_tests=True, time_out_ms=6000)
    76         port = WebKitPort(MockSystemHost(), options=options)
    77         self.assertEquals(port._options.pixel_tests, True)
    78         self.assertEquals(port._options.time_out_ms, 6000)
    7964
    8065

  • trunk/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py

    r120740 r121497  
    139139
    140140    if options.pixel_tests is None:
    141         options.pixel_tests = True
     141        options.pixel_tests = port.default_pixel_tests()
    142142
    143143    if not options.time_out_ms:
    144         if options.configuration == "Debug":
    145             options.time_out_ms = str(2 * port.default_test_timeout_ms())
    146         else:
    147             options.time_out_ms = str(port.default_test_timeout_ms())
     144        options.time_out_ms = str(port.default_timeout_ms())
    148145
    149146    options.slow_time_out_ms = str(5 * int(options.time_out_ms))
Note: See TracChangeset for help on using the changeset viewer.