Changeset 58128 in webkit


Ignore:
Timestamp:
Apr 22, 2010 4:46:56 PM (14 years ago)
Author:
eric@webkit.org
Message:

2010-04-22 Eric Seidel <eric@webkit.org>

Reviewed by Adam Barth.

Add code to help debug new-run-webkit-test hangs on the Chromium bots
https://bugs.webkit.org/show_bug.cgi?id=38011

I can see no reasonable way to test this change.
Stubbing out sys._current_frames() and traceback.extract_stack
seems folly. Dumping real data would have line number
(and possibly other call stack) variance between runs.

  • Scripts/webkitpy/layout_tests/run_webkit_tests.py:
    • Add _dump_thread_states and _dump_thread_states_if_necessary to have our main thread dump the states of all threads every 60 seconds when running in verbose mode.
    • Better document what is going on in our main loop.
Location:
trunk/WebKitTools
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • trunk/WebKitTools/ChangeLog

    r58117 r58128  
     12010-04-22  Eric Seidel  <eric@webkit.org>
     2
     3        Reviewed by Adam Barth.
     4
     5        Add code to help debug new-run-webkit-test hangs on the Chromium bots
     6        https://bugs.webkit.org/show_bug.cgi?id=38011
     7
     8        I can see no reasonable way to test this change.
     9        Stubbing out sys._current_frames() and traceback.extract_stack
     10        seems folly.  Dumping real data would have line number
     11        (and possibly other call stack) variance between runs.
     12
     13        * Scripts/webkitpy/layout_tests/run_webkit_tests.py:
     14         - Add _dump_thread_states and _dump_thread_states_if_necessary
     15           to have our main thread dump the states of all threads every
     16           60 seconds when running in verbose mode.
     17         - Better document what is going on in our main loop.
     18
    1192010-04-22  Sam Weinig  <sam@webkit.org>
    220
  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/run_webkit_tests.py

    r58074 r58128  
    239239        self._retries = 0
    240240
     241        # Hack for dumping threads on the bots
     242        self._last_thread_dump = None
     243
    241244    def __del__(self):
    242245        _log.debug("flushing stdout")
     
    584587        return int(self._options.child_processes) == 1
    585588
     589    def _dump_thread_states(self):
     590        for thread_id, stack in sys._current_frames().items():
     591            # FIXME: Python 2.6 has thread.ident which we could
     592            # use to map from thread_id back to thread.name
     593            print "\n# Thread: %d" % thread_id
     594            for filename, lineno, name, line in traceback.extract_stack(stack):
     595                print 'File: "%s", line %d, in %s' % (filename, lineno, name)
     596                if line:
     597                    print "  %s" % (line.strip())
     598
     599    def _dump_thread_states_if_necessary(self):
     600        # HACK: Dump thread states every minute to figure out what's
     601        # hanging on the bots.
     602        if not self._options.verbose:
     603            return
     604        dump_threads_every = 60  # Dump every minute
     605        if not self._last_thread_dump:
     606            self._last_thread_dump = time.time()
     607        time_since_last_dump = time.time() - self._last_thread_dump
     608        if  time_since_last_dump > dump_threads_every:
     609            self._dump_thread_states()
     610            self._last_thread_dump = time.time()
     611
    586612    def _run_tests(self, file_list, result_summary):
    587613        """Runs the tests in the file_list.
     
    598624            result_summary: summary object to populate with the results
    599625        """
     626        # FIXME: We should use webkitpy.tool.grammar.pluralize here.
    600627        plural = ""
    601628        if self._options.child_processes > 1:
     
    613640        thread_timings = []
    614641        try:
     642            # Loop through all the threads waiting for them to finish.
    615643            for thread in threads:
     644                # FIXME: We'll end up waiting on the first thread the whole
     645                # time.  That means we won't notice exceptions on other
     646                # threads until the first one exits.
     647                # We should instead while True: in the outer loop
     648                # and then loop through threads joining and checking
     649                # isAlive and get_exception_info.  Exiting on any exception.
    616650                while thread.isAlive():
    617                     # Let it timeout occasionally so it can notice a
    618                     # KeyboardInterrupt. Actually, the timeout doesn't
    619                     # really matter: apparently it suffices to not use
    620                     # an indefinite blocking join for it to
    621                     # be interruptible by KeyboardInterrupt.
     651                    # Wake the main thread every 0.1 seconds so we
     652                    # can call update_summary in a timely fashion.
    622653                    thread.join(0.1)
     654                    # HACK: Used for debugging threads on the bots.
     655                    self._dump_thread_states_if_necessary()
    623656                    self.update_summary(result_summary)
     657
     658                # This thread is done, save off the timing information.
    624659                thread_timings.append({'name': thread.getName(),
    625660                                       'num_tests': thread.get_num_tests(),
    626661                                       'total_time': thread.get_total_time()})
    627662                test_timings.update(thread.get_directory_timing_stats())
    628                 individual_test_timings.extend(
    629                     thread.get_test_results())
     663                individual_test_timings.extend(thread.get_test_results())
    630664        except KeyboardInterrupt:
    631665            for thread in threads:
     
    641675                raise exception_info[0], exception_info[1], exception_info[2]
    642676
    643         # Make sure we pick up any remaining tests.
     677        # FIXME: This update_summary call seems unecessary.
     678        # Calls are already made right after join() above,
     679        # as well as from the individual threads themselves.
    644680        self.update_summary(result_summary)
    645681        return (thread_timings, test_timings, individual_test_timings)
Note: See TracChangeset for help on using the changeset viewer.