Changeset 55373 in webkit


Ignore:
Timestamp:
Mar 1, 2010 12:41:29 PM (14 years ago)
Author:
dpranke@chromium.org
Message:

2010-03-01 Dirk Pranke <​dpranke@chromium.org>

Reviewed by David Levin.

General cleanup of error handling in new-run-webkit-tests.

Add Port.check_build() call that is separate from Port.check_sys_deps()
(and add a --nocheck-build flag to skip). This breaks a circular
dependency where you would start the layout test helper before
checking sys deps, but checking sys deps was the thing that told
you if your binaries where there.

Also, made Port.check_sys_deps(), start_helper() and stop_helper()
optional by providing default implementations in the base class
rather than requiring ports to implement the routines regardless
of whether or not they were needed.

Lastly, tweak a bunch of log messages to be cleaner, including
changing messages in thirdparty/autoinstall.py to be silent at
the default log level.

​http://bugs.webkit.org/show_bug.cgi?id=35416

  • Scripts/webkitpy/layout_tests/layout_package/test_expectations.py:
  • Scripts/webkitpy/layout_tests/port/base.py:
  • Scripts/webkitpy/layout_tests/port/chromium.py:
  • Scripts/webkitpy/layout_tests/port/chromium_linux.py:
  • Scripts/webkitpy/layout_tests/port/chromium_mac.py:
  • Scripts/webkitpy/layout_tests/port/chromium_win.py:
  • Scripts/webkitpy/layout_tests/port/mac.py:
  • Scripts/webkitpy/layout_tests/port/passing.py:
  • Scripts/webkitpy/layout_tests/port/test.py:
  • Scripts/webkitpy/layout_tests/run_webkit_tests.py:
  • Scripts/webkitpy/thirdparty/autoinstall.py
Location:
trunk/WebKitTools
Files:
12 edited

Legend:

Unmodified
Added
Removed

  • trunk/WebKitTools/ChangeLog

    r55372 r55373  
     12010-03-01  Dirk Pranke  <dpranke@chromium.org>
     2
     3        Reviewed by David Levin.
     4
     5        General cleanup of error handling in new-run-webkit-tests.
     6
     7        Add Port.check_build() call that is separate from Port.check_sys_deps()
     8        (and add a --nocheck-build flag to skip). This breaks a circular
     9        dependency where you would start the layout test helper before
     10        checking sys deps, but checking sys deps was the thing that told
     11        you if your binaries where there.
     12
     13        Also, made Port.check_sys_deps(), start_helper() and stop_helper()
     14        optional by providing default implementations in the base class
     15        rather than requiring ports to implement the routines regardless
     16        of whether or not they were needed.
     17
     18        Lastly, tweak a bunch of log messages to be cleaner, including
     19        changing messages in thirdparty/autoinstall.py to be silent at
     20        the default log level.
     21
     22        http://bugs.webkit.org/show_bug.cgi?id=35416
     23
     24        * Scripts/webkitpy/layout_tests/layout_package/test_expectations.py:
     25        * Scripts/webkitpy/layout_tests/port/base.py:
     26        * Scripts/webkitpy/layout_tests/port/chromium.py:
     27        * Scripts/webkitpy/layout_tests/port/chromium_linux.py:
     28        * Scripts/webkitpy/layout_tests/port/chromium_mac.py:
     29        * Scripts/webkitpy/layout_tests/port/chromium_win.py:
     30        * Scripts/webkitpy/layout_tests/port/mac.py:
     31        * Scripts/webkitpy/layout_tests/port/passing.py:
     32        * Scripts/webkitpy/layout_tests/port/test.py:
     33        * Scripts/webkitpy/layout_tests/run_webkit_tests.py:
     34        * Scripts/webkitpy/thirdparty/autoinstall.py
     35
    1362010-03-01  Dirk Pranke <dpranke@chromium.org>
    237

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/test_expectations.py

    r54760 r55373  
    634634            else:
    635635                build_type = 'RELEASE'
    636             print "\nFAILURES FOR PLATFORM: %s, BUILD_TYPE: %s" \
    637                 % (self._test_platform_name.upper(), build_type)
     636            logging.error('')
     637            logging.error("FAILURES FOR PLATFORM: %s, BUILD_TYPE: %s" %
     638                          (self._test_platform_name.upper(), build_type))
    638639
    639640            for error in self._non_fatal_errors:
    640641                logging.error(error)
     642            logging.error('')
    641643            if len(self._errors):
    642644                raise SyntaxError('\n'.join(map(str, self._errors)))

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/base.py

    r55372 r55373  
    6969        raise NotImplementedError('Port.baseline_search_path')
    7070
     71    def check_build(self, needs_http):
     72        """This routine is used to ensure that the build is up to date
     73        and all the needed binaries are present."""
     74        raise NotImplementedError('Port.check_build')
     75
    7176    def check_sys_deps(self, needs_http):
    7277        """If the port needs to do some runtime checks to ensure that the
    73         tests can be run successfully, they should be done here.
     78        tests can be run successfully, it should override this routine.
     79        This step can be skipped with --nocheck-sys-deps.
    7480
    7581        Returns whether the system is properly configured."""
    76         raise NotImplementedError('Port.check_sys_deps')
     82        return True
    7783
    7884    def compare_text(self, expected_text, actual_text):
    … …  
    382388
    383389    def start_helper(self):
    384         """Start a layout test helper if needed on this port. The test helper
    385         is used to reconfigure graphics settings and do other things that
    386         may be necessary to ensure a known test configuration."""
    387         raise NotImplementedError('Port.start_helper')
     390        """If a port needs to reconfigure graphics settings or do other
     391        things to ensure a known test configuration, it should override this
     392        method."""
     393        pass
    388394
    389395    def start_http_server(self):
    … …  
    409415    def stop_helper(self):
    410416        """Shut down the test helper if it is running. Do nothing if
    411         it isn't, or it isn't available."""
    412         raise NotImplementedError('Port.stop_helper')
     417        it isn't, or it isn't available. If a port overrides start_helper()
     418        it must override this routine as well."""
     419        pass
    413420
    414421    def stop_http_server(self):

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/chromium.py

    r55372 r55373  
    4343
    4444
    45 def check_file_exists(path_to_file, str):
    46     """Verify the executable is present where expected or log an error."""
     45def check_file_exists(path_to_file, file_description, override_step=None):
     46    """Verify the file is present where expected or log an error.
     47
     48    Args:
     49        file_name: The (human friendly) name or description of the file
     50            you're looking for (e.g., "HTTP Server"). Used for error logging.
     51        override_step: An optional string to be logged if the check fails."""
    4752    if not os.path.exists(path_to_file):
    48         logging.error('Unable to find %s at %s' % (str, path_to_file))
     53        logging.error('Unable to find %s' % file_description)
     54        logging.error('    at %s' % path_to_file)
     55        if override_step:
     56            logging.error('    %s' % override_step)
     57            logging.error('')
    4958        return False
    5059    return True
    … …  
    6170        return self._chromium_baseline_path(self._name)
    6271
    63     def check_sys_deps(self, needs_http):
     72    def check_build(self, needs_http):
    6473        result = True
    6574        test_shell_binary_path = self._path_to_driver()
    … …  
    6776                                   'test driver')
    6877        if result:
    69             result = (self._check_build_up_to_date(self._options.target)
     78            result = (self._check_driver_build_up_to_date(self._options.target)
    7079                      and result)
    71 
    72             proc = subprocess.Popen([test_shell_binary_path,
    73                                      '--check-layout-test-sys-deps'])
    74             if proc.wait() != 0:
    75                 logging.error('System dependencies check failed.')
    76                 logging.error('To override, invoke with --nocheck-sys-deps')
    77                 logging.error('')
    78                 result = False
    79 
    8080        else:
    8181            logging.error('')
    8282
     83        helper_path = self._path_to_helper()
     84        result = check_file_exists(helper_path,
     85                                   'layout test helper') and result
    8386
    8487        if not self._options.no_pixel_tests:
    8588            image_diff_path = self._path_to_image_diff()
    86             if not check_file_exists(image_diff_path, 'image diff exe'):
    87                 logging.error('To override, invoke with --no-pixel-tests')
    88                 logging.error('')
    89                 result = False
     89            result = check_file_exists(image_diff_path, 'image diff exe',
     90                'To override, invoke with --no-pixel-tests') and result
    9091
    9192        return result
     93
     94    def check_sys_deps(self, needs_http):
     95        proc = subprocess.Popen([test_shell_binary_path,
     96                                '--check-layout-test-sys-deps'])
     97        if proc.wait():
     98            logging.error('System dependencies check failed.')
     99            logging.error('To override, invoke with --nocheck-sys-deps')
     100            logging.error('')
     101            result = False
     102        return True
    92103
    93104    def path_from_chromium_base(self, *comps):
    … …  
    162173    #
    163174
    164     def _check_build_up_to_date(self, target):
     175    def _check_driver_build_up_to_date(self, target):
    165176        if target in ('Debug', 'Release'):
    166177            try:

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/chromium_linux.py

    r55123 r55373  
    5555                self._webkit_baseline_path('mac')]
    5656
    57     def check_sys_deps(self, needs_http):
    58         result = chromium.ChromiumPort.check_sys_deps(self, needs_http)
     57    def check_build(self, needs_http):
     58        result = chromium.ChromiumPort.check_build(self, needs_http)
    5959        if needs_http:
    6060            if self._options.use_apache:
    … …  
    102102            "apache2 config file") and result
    103103        if not result:
    104             logging.error('Please install using: "sudo apt-get install '
     104            logging.error('    Please install using: "sudo apt-get install '
    105105                          'apache2 libapache2-mod-php5"')
    106106            logging.error('')
    … …  
    115115            "LigHTTPd modules") and result
    116116        if not result:
    117             logging.error('Please install using: "sudo apt-get install '
     117            logging.error('    Please install using: "sudo apt-get install '
    118118                          'lighttpd php5-cgi"')
    119119            logging.error('')
    … …  
    123123        result = chromium.check_file_exists(self._path_to_wdiff(), 'wdiff')
    124124        if not result:
    125             logging.error('Please install using: "sudo apt-get install '
     125            logging.error('    Please install using: "sudo apt-get install '
    126126                          'wdiff"')
    127127            logging.error('')

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/chromium_mac.py

    r55210 r55373  
    5454                self._webkit_baseline_path('mac')]
    5555
    56     def check_sys_deps(self, needs_http):
    57         result = chromium.ChromiumPort.check_sys_deps(self, needs_http)
     56    def check_build(self, needs_http):
     57        result = chromium.ChromiumPort.check_build(self, needs_http)
    5858        result = self._check_wdiff_install() and result
    5959        if not result:

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/chromium_win.py

    r55123 r55373  
    6161        return dirs
    6262
    63     def check_sys_deps(self, needs_http):
    64         result = chromium.ChromiumPort.check_sys_deps(self, needs_http)
     63    def check_build(self, needs_http):
     64        result = chromium.ChromiumPort.check_build(self, needs_http)
    6565        if not result:
    6666            logging.error('For complete Windows build requirements, please '

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/mac.py

    r55208 r55373  
    6767        return dirs
    6868
    69     def check_sys_deps(self, needs_http):
    70         if executive.run_command([self.script_path("build-dumprendertree")], return_exit_code=True) != 0:
     69    def check_build(self, needs_http):
     70        if executive.run_command([self.script_path("build-dumprendertree")],
     71                                 return_exit_code=True):
    7172            return False
    7273
    … …  
    7677            return False
    7778
    78         # This should also validate that the ImageDiff path is valid (once this script knows how to use ImageDiff).
     79        # This should also validate that the ImageDiff path is valid
     80        # (once this script knows how to use ImageDiff).
    7981        # https://bugs.webkit.org/show_bug.cgi?id=34826
    8082        return True
    … …  
    98100        """Starts a new Driver and returns a handle to it."""
    99101        return MacDriver(self, image_path, options)
    100 
    101     def start_helper(self):
    102         # This port doesn't use a helper process.
    103         pass
    104 
    105     def stop_helper(self):
    106         # This port doesn't use a helper process.
    107         pass
    108102
    109103    def test_base_platform_names(self):

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/passing.py

    r55268 r55373  
    3939   Note that because this is really acting as a wrapper around the underlying
    4040   port, you must be able to run the underlying port as well
    41    (check_sys_deps() must pass and the layout_test_helper must work). You
    42    may be able to get around this with --nocheck-sys-deps and --nostart-helper.
     41   (check_build() and check_sys_deps() must pass and auxiliary binaries
     42   like layout_test_helper and httpd must work).
    4343
    4444   This implementation also modifies the test expectations so that all
    … …  
    6161    def __getattr__(self, name):
    6262        return getattr(self.__delegate, name)
    63  
     63
    6464    def start_driver(self, image_path, options):
    6565        return PassingDriver(self, image_path, options)
    … …  
    9999        if image_hash:
    100100            image_filename = self._port.expected_filename(test_name, '.png')
    101             image = file(image_filename, 'rb').read()
    102             output_file = file(self._image_path, 'w')
    103             output_file.write(image)
    104             output_file.close()
     101            try:
     102                image = file(image_filename, 'rb').read()
     103            except IOError:
     104                image = ''
     105            if self._image_path:
     106                output_file = file(self._image_path, 'w')
     107                output_file.write(image)
     108                output_file.close()
    105109            hash_filename = self._port.expected_filename(test_name,
    106110                '.checksum')
    107             hash = file(hash_filename, 'r').read()
     111            try:
     112                hash = file(hash_filename, 'r').read()
     113            except IOError:
     114                hash = ''
    108115        else:
    109116            hash = None

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/test.py

    r55372 r55373  
    5353        return [self.baseline_path()]
    5454
    55     def check_sys_deps(self, needs_http):
     55    def check_build(self, needs_http):
    5656        return True
     57
     58    def compare_text(self, expected_text, actual_text):
     59        return False
    5760
    5861    def diff_image(self, expected_filename, actual_filename,
    5962                   diff_filename=None):
    60         return False
    61 
    62     def compare_text(self, expected_text, actual_text):
    6363        return False
    6464
    … …  
    9494        pass
    9595
    96     def start_helper(self):
    97         pass
    98 
    9996    def stop_http_server(self):
    10097        pass
    10198
    10299    def stop_websocket_server(self):
    103         pass
    104 
    105     def stop_helper(self):
    106100        pass
    107101

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/run_webkit_tests.py

    r55341 r55373  
    553553            result_summary: summary object to populate with the results
    554554        """
     555        self._meter.update('Starting test shells ...')
    555556        threads = self._instantiate_test_shell_threads(file_list,
    556557                                                       result_summary)
    … …  
    615616
    616617        if self.needs_http():
     618            self._meter.update('Starting HTTP server ...')
    617619            self._port.start_http_server()
    618620
    619621        if self._contains_tests(self.WEBSOCKET_SUBDIR):
     622            self._meter.update('Starting WebSocket server ...')
    620623            self._port.start_websocket_server()
    621624            # self._websocket_secure_server.Start()
    … …  
    14691472                                   options.target == 'Debug')
    14701473
    1471     if not options.nostart_helper:
    1472         port_obj.start_helper()
     1474    meter.update("Checking build ...")
     1475    if not port_obj.check_build(test_runner.needs_http()):
     1476        sys.exit(1)
     1477
     1478    meter.update("Starting helper ...")
     1479    port_obj.start_helper()
    14731480
    14741481    # Check that the system dependencies (themes, fonts, ...) are correct.
    1475     if (not options.nocheck_sys_deps and
    1476          not port_obj.check_sys_deps(test_runner.needs_http())):
    1477         sys.exit(1)
     1482    if not options.nocheck_sys_deps:
     1483        meter.update("Checking system dependencies ...")
     1484        if not port_obj.check_sys_deps(test_runner.needs_http()):
     1485            sys.exit(1)
    14781486
    14791487    meter.update("Preparing tests ...")
    … …  
    14891497            test_runner.add_test_type(fuzzy_image_diff.FuzzyImageDiff)
    14901498
    1491     meter.update("Starting ...")
    14921499    has_new_failures = test_runner.run(result_summary)
    14931500
    1494     if not options.nostart_helper:
    1495         port_obj.stop_helper()
     1501    port_obj.stop_helper()
    14961502
    14971503    logging.debug("Exit status: %d" % has_new_failures)
    … …  
    16221628                             action="store_true", default=False,
    16231629                             help="run all tests in parallel")
    1624     option_parser.add_option("", "--nostart-helper",
    1625                              action="store_true", default=False,
    1626                              help="don't run layout_test_helper")
    16271630    option_parser.add_option("", "--chromium",
    16281631                             action="store_true", default=False,

  • trunk/WebKitTools/Scripts/webkitpy/thirdparty/autoinstall.py

    r55363 r55373  
    294294
    295295    def bind(self, package_name, url, zip_subpath):
    296         _logger.info("binding: %s -> %s subpath: %s" % (package_name, url, zip_subpath))
     296        _logger.debug("binding: %s -> %s subpath: %s" %
     297                      (package_name, url, zip_subpath))
    297298        self.packages[package_name] = (url, zip_subpath)
    298299
Note: See TracChangeset for help on using the changeset viewer.