Changeset 57956 in webkit


Ignore:
Timestamp:
Apr 20, 2010 11:55:46 PM (14 years ago)
Author:
eric@webkit.org
Message:

2010-04-19 Eric Seidel <eric@webkit.org>

Reviewed by Adam Barth.

new-run-webkit-tests has much higher startup latency than run-webkit-tests
https://bugs.webkit.org/show_bug.cgi?id=37643

I got rid of the -expected.checksum reads during startup.
This makes startup noticably better on my laptop.

  • Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py:
    • Use image_hash() instead of .image_hash now that expected.checksum file reads are done lazily.
  • Scripts/webkitpy/layout_tests/port/http_server_base.py:
    • Add debug logging for this sleep call. In my testing I never saw this sleep() hit.
  • Scripts/webkitpy/layout_tests/port/websocket_server.py:
    • Sleep a shorter interval to make websocket server startup more responsive. On my machine startup was taking around 1 second.
    • Remove the unconditional .5s delay on startup.
  • Scripts/webkitpy/layout_tests/run_webkit_tests.py:
    • Make image_hash file reads done lazily in a new image_hash() function.
    • Add a "Starting testing ..." meter update after DRT threads have been started, but before we get updates from the first one.
    • Rename variable "t" to a full english name to match WebKit style.
Location:
trunk/WebKitTools
Files:
6 edited

Legend:

Unmodified
Added
Removed
  • trunk/WebKitTools/ChangeLog

    r57948 r57956  
     12010-04-19  Eric Seidel  <eric@webkit.org>
     2
     3        Reviewed by Adam Barth.
     4
     5        new-run-webkit-tests has much higher startup latency than run-webkit-tests
     6        https://bugs.webkit.org/show_bug.cgi?id=37643
     7
     8        I got rid of the -expected.checksum reads during startup.
     9        This makes startup noticably better on my laptop.
     10
     11        * Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py:
     12         - Use image_hash() instead of .image_hash now that expected.checksum
     13           file reads are done lazily.
     14        * Scripts/webkitpy/layout_tests/port/http_server_base.py:
     15         - Add debug logging for this sleep call.
     16           In my testing I never saw this sleep() hit.
     17        * Scripts/webkitpy/layout_tests/port/websocket_server.py:
     18         - Sleep a shorter interval to make websocket server
     19           startup more responsive.  On my machine startup was
     20           taking around 1 second.
     21         - Remove the unconditional .5s delay on startup.
     22        * Scripts/webkitpy/layout_tests/run_webkit_tests.py:
     23         - Make image_hash file reads done lazily in a new image_hash() function.
     24         - Add a "Starting testing ..." meter update after DRT threads have
     25           been started, but before we get updates from the first one.
     26         - Rename variable "t" to a full english name to match WebKit style.
     27
    1282010-04-20  Daniel Bates  <dbates@rim.com>
    229
  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py

    r57934 r57956  
    157157        crash, timeout, actual_checksum, output, error = \
    158158            driver.run_test(test_info.uri.strip(), test_info.timeout,
    159                             test_info.image_hash)
     159                            test_info.image_hash())
    160160        end = time.time()
    161161        self._test_result = process_output(self._port,
     
    421421        # are generating a new baseline.  (Otherwise, an image from a
    422422        # previous run will be copied into the baseline.)
    423         image_hash = test_info.image_hash
     423        image_hash = test_info.image_hash()
    424424        if image_hash and self._test_args.new_baseline:
    425425            image_hash = ""
  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/test_files.py

    r56994 r57956  
    3737import glob
    3838import os
     39import time
     40
     41from webkitpy.common.system import logutils
     42
     43
     44_log = logutils.get_logger(__file__)
     45
    3946
    4047# When collecting test cases, we include any file with these extensions.
     
    5259          directory. glob patterns are ok.
    5360    """
     61    gather_start_time = time.time()
    5462    paths_to_walk = set()
    5563    # if paths is empty, provide a pre-defined list.
     
    7482
    7583        for root, dirs, files in os.walk(path):
    76             # don't walk skipped directories and sub directories
     84            # Don't walk skipped directories or their sub-directories.
    7785            if os.path.basename(root) in _skipped_directories:
    7886                del dirs[:]
    7987                continue
     88            # This copy and for-in is slightly inefficient, but
     89            # the extra walk avoidance consistently shaves .5 seconds
     90            # off of total walk() time on my MacBook Pro.
     91            for directory in dirs[:]:
     92                if directory in _skipped_directories:
     93                    dirs.remove(directory)
    8094
    8195            for filename in files:
     
    8498                    filename = os.path.normpath(filename)
    8599                    test_files.add(filename)
     100
     101    gather_time = time.time() - gather_start_time
     102    _log.debug("Test gathering took %f seconds" % gather_time)
    86103
    87104    return test_files
  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/http_server_base.py

    r55603 r57956  
    5050            if action():
    5151                return True
     52            _log.debug("Waiting for action: %s" % action)
    5253            time.sleep(1)
    5354
  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/websocket_server.py

    r57845 r57956  
    6565      True if the url is alive.
    6666    """
     67    sleep_time = 0.5
    6768    wait_time = 5
    6869    while wait_time > 0:
     
    7374        except IOError:
    7475            pass
    75         wait_time -= 1
    76         # Wait a second and try again.
    77         time.sleep(1)
     76        # Wait for sleep_time before trying again.
     77        wait_time -= sleep_time
     78        time.sleep(sleep_time)
    7879
    7980    return False
     
    210211                                         env=env)
    211212
    212         # Wait a bit before checking the liveness of the server.
    213         time.sleep(0.5)
    214 
    215213        if self._use_tls:
    216214            url = 'https'
  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/run_webkit_tests.py

    r57934 r57956  
    4545"""
    4646
     47from __future__ import with_statement
     48
     49import codecs
    4750import errno
    4851import glob
     
    123126        self.uri = port.filename_to_uri(filename)
    124127        self.timeout = timeout
    125         expected_hash_file = port.expected_filename(filename, '.checksum')
     128        # FIXME: Confusing that the file is .checksum and we call it "hash"
     129        self._expected_hash_path = port.expected_filename(filename, '.checksum')
     130        self._have_read_expected_hash = False
     131        self._image_hash = None
     132
     133    def _read_image_hash(self):
    126134        try:
    127             self.image_hash = open(expected_hash_file, "r").read()
     135            with codecs.open(self._expected_hash_path, "r", "ascii") as hash_file:
     136                return hash_file.read()
    128137        except IOError, e:
    129138            if errno.ENOENT != e.errno:
    130139                raise
    131             self.image_hash = None
     140
     141    def image_hash(self):
     142        # Read the image_hash lazily to reduce startup time.
     143        # This class is accessed across threads, but only one thread should
     144        # ever be dealing with any given TestInfo so no locking is needed.
     145        if not self._have_read_expected_hash:
     146            self._have_read_expected_hash = True
     147            self._image_hash = self._read_image_hash()
     148        return self._image_hash
    132149
    133150
     
    543560            # Create separate TestTypes instances for each thread.
    544561            test_types = []
    545             for t in self._test_types:
    546                 test_types.append(t(self._port,
     562            for test_type in self._test_types:
     563                test_types.append(test_type(self._port,
    547564                                    self._options.results_directory))
    548565
     
    585602        threads = self._instantiate_dump_render_tree_threads(file_list,
    586603                                                             result_summary)
     604        self._meter.update("Starting testing ...")
    587605
    588606        # Wait for the threads to finish and collect test failures.
Note: See TracChangeset for help on using the changeset viewer.