Changeset 217853 in webkit
- Timestamp:
- Jun 6, 2017 1:25:15 PM (7 years ago)
- Location:
- trunk/Tools
- Files:
-
- 17 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/Tools/ChangeLog
r217847 r217853 1 2017-06-06 Jonathan Bedard <jbedard@apple.com> 2 3 webkitpy: Process crash-logs for iOS devices 4 https://bugs.webkit.org/show_bug.cgi?id=171976 5 <rdar://problem/32134551> 6 7 Reviewed by David Kilzer. 8 9 When running layout tests on an iOS device, crash logs should be processed. 10 Implement crash log searching and parsing for iOS devices. 11 12 * Scripts/webkitpy/common/system/crashlogs.py: 13 (CrashLogs): Moved process regular expression for Darwin to class variable. 14 (CrashLogs.__init__): Accept optional list of crash logs to ignore. 15 (CrashLogs.find_newest_log): Add iOS as a potential platform. 16 (CrashLogs.find_all_logs): Ditto. 17 (CrashLogs._parse_darwin_crash_log): Share code for parsing of Darwin crash logs. 18 Do not assume that a Darwin crash log starts with the process. 19 (CrashLogs._find_newest_log_darwin): Remove .app in process name for iOS, use 20 shared code for parsing Darwin crash logs. 21 (CrashLogs._find_newest_log_darwin.is_crash_log): Skip crash logs passed into this 22 object so that crash logs already on the system before testing are not parsed. 23 (CrashLogs._find_newest_log_win.is_crash_log): Ditto. 24 (CrashLogs._find_all_logs_darwin.is_crash_log): Ditto. 25 (CrashLogs._find_all_logs_darwin): Use shared code for parsing Darwin crash logs. 26 * Scripts/webkitpy/common/system/crashlogs_unittest.py: 27 (make_mock_crash_report_darwin): Crash logs may not have their process on the first line. 28 * Scripts/webkitpy/common/system/systemhost.py: 29 (SystemHost.symbolicate_crash_log_if_needed): The symbolicated crash log for most 30 systems is just the crashlog, use this behavior by default. 31 * Scripts/webkitpy/common/system/systemhost_mock.py: 32 (MockSystemHost.symbolicate_crash_log_if_needed): The symbolicated crash log for most 33 systems is just the crashlog, use this behavior by default. 34 * Scripts/webkitpy/port/apple.py: 35 (ApplePort): Add a dictionary mapping hosts to a list of crash logs to be skipped. 36 (ApplePort.setup_test_run): Set the list of crash logs to be skipped to the crash logs on 37 the system before testing begins 38 * Scripts/webkitpy/port/base.py: 39 (Port._get_crash_log): Pass optional target host when getting crash logs. 40 * Scripts/webkitpy/port/darwin.py: 41 (DarwinPort._look_for_all_crash_logs_in_log_dir): Pass list of crash logs to be skipped to 42 CrashLogs object. 43 (DarwinPort._get_crash_log): Pass optional target host when getting crash logs, pass list of crash 44 logs to be skipped to CrashLogs object. 45 * Scripts/webkitpy/port/darwin_testcase.py: 46 (DarwinTest.test_get_crash_log): Removed unused local function. 47 (DarwinTest.test_get_crash_log.fake_time_cb): Deleted. 48 * Scripts/webkitpy/port/device.py: 49 (Device.symbolicate_crash_log_if_needed): If the platform device has a function with this 50 name, call it. Otherwise, assume the default behavior and read the file at the provided path. 51 * Scripts/webkitpy/port/driver.py: 52 (Driver._get_crash_log): Pass optional target host when getting crash logs. 53 * Scripts/webkitpy/port/gtk.py: 54 (GtkPort._get_crash_log): Pass optional target host when getting crash logs. 55 * Scripts/webkitpy/port/ios.py: Ditto. 56 (IOSPort.setup_test_run): Each device is treated as an independent host. Set the list of crash logs 57 to be skipped for each host. 58 * Scripts/webkitpy/port/ios_device.py: 59 (IOSDevicePort.path_to_crash_logs): Consult apple_additions for the path to crash logs. 60 (IOSDevicePort._look_for_all_crash_logs_in_log_dir): Search every connected device for 61 crash logs and pass list of crash logs to ignore to each instance of CrashLogs. 62 (IOSDevicePort._get_crash_log): Search the specified target host for a crash log if a target 63 host is specified. Else, search all connected devices for the specified crash-log. 64 (IOSDevicePort.look_for_new_crash_logs): Deleted. 65 * Scripts/webkitpy/port/ios_device_unittest.py: 66 (IOSDeviceTest.test_crashlog_path): Without apple_additions, an exception should be raised. 67 (IOSDeviceTest.test_get_crash_log): Ditto. 68 * Scripts/webkitpy/port/simulator_process.py: 69 (SimulatorProcess.process_name): Check the provided bundle for the process name. 70 * Scripts/webkitpy/port/win.py: 71 (WinPort._get_crash_log): Pass optional target host when getting crash logs, pass list of crash 72 logs to be skipped to CrashLogs object. 73 * Scripts/webkitpy/port/wpe.py: 74 (WPEPort._get_crash_log): Pass optional target host when getting crash logs. 75 1 76 2017-06-06 David Kilzer <ddkilzer@apple.com> 2 77 -
trunk/Tools/Scripts/webkitpy/common/system/crashlogs.py
r216776 r217853 28 28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 29 30 import codecs31 30 import datetime 31 import logging 32 32 import re 33 34 35 _log = logging.getLogger(__name__) 33 36 34 37 … … 37 40 GLOBAL_PID_REGEX = re.compile(r'\s+Global\s+PID:\s+\[(?P<pid>\d+)\]') 38 41 EXIT_PROCESS_PID_REGEX = re.compile(r'Exit process \d+:(?P<pid>\w+), code') 42 DARWIN_PROCESS_REGEX = re.compile(r'^Process:\s+(?P<process_name>.*) \[(?P<pid>\d+)\]$') 39 43 40 def __init__(self, host, crash_log_directory ):44 def __init__(self, host, crash_log_directory, crash_logs_to_skip=[]): 41 45 self._host = host 42 46 self._crash_log_directory = crash_log_directory 47 self._crash_logs_to_skip = crash_logs_to_skip 43 48 44 49 def find_newest_log(self, process_name, pid=None, include_errors=False, newer_than=None): 45 if self._host.platform.is_mac() :50 if self._host.platform.is_mac() or self._host.platform.is_ios(): 46 51 return self._find_newest_log_darwin(process_name, pid, include_errors, newer_than) 47 52 elif self._host.platform.is_win(): … … 50 55 51 56 def find_all_logs(self, include_errors=False, newer_than=None): 52 if self._host.platform.is_mac() :57 if self._host.platform.is_mac() or self._host.platform.is_ios(): 53 58 return self._find_all_logs_darwin(include_errors, newer_than) 54 59 return None 55 60 61 def _parse_darwin_crash_log(self, path): 62 contents = self._host.symbolicate_crash_log_if_needed(path) 63 if not contents: 64 return (None, None, None) 65 for line in contents.splitlines(): 66 match = CrashLogs.DARWIN_PROCESS_REGEX.match(line) 67 if match: 68 return (match.group('process_name'), int(match.group('pid')), contents) 69 return (None, None, contents) 70 56 71 def _find_newest_log_darwin(self, process_name, pid, include_errors, newer_than): 57 72 def is_crash_log(fs, dirpath, basename): 58 return basename.startswith(process_name + "_") and basename.endswith(".crash") 73 if self._crash_logs_to_skip and fs.join(dirpath, basename) in self._crash_logs_to_skip: 74 return False 75 return (basename.startswith(process_name + '_') and (basename.endswith('.crash')) or 76 (process_name in basename and basename.endswith('.ips'))) 59 77 60 78 logs = self._host.filesystem.files_under(self._crash_log_directory, file_filter=is_crash_log) 61 first_line_regex = re.compile(r'^Process:\s+(?P<process_name>.*) \[(?P<pid>\d+)\]$')62 79 errors = '' 63 80 for path in reversed(sorted(logs)): 64 81 try: 65 82 if not newer_than or self._host.filesystem.mtime(path) > newer_than: 66 f = self._host.filesystem.read_text_file(path) 67 match = first_line_regex.match(f[0:f.find('\n')]) 68 if match and match.group('process_name') == process_name and (pid is None or int(match.group('pid')) == pid): 69 return errors + f 83 parsed_name, parsed_pid, log_contents = self._parse_darwin_crash_log(path) 84 if parsed_name == process_name and (pid is None or parsed_pid == pid): 85 return errors + log_contents 70 86 except IOError, e: 71 87 if include_errors: … … 81 97 def _find_newest_log_win(self, process_name, pid, include_errors, newer_than): 82 98 def is_crash_log(fs, dirpath, basename): 99 if self._crash_logs_to_skip and fs.join(dirpath, basename) in self._crash_logs_to_skip: 100 return False 83 101 return basename.startswith("CrashLog") 84 102 … … 118 136 def _find_all_logs_darwin(self, include_errors, newer_than): 119 137 def is_crash_log(fs, dirpath, basename): 120 return basename.endswith(".crash") 138 if self._crash_logs_to_skip and fs.join(dirpath, basename) in self._crash_logs_to_skip: 139 return False 140 return basename.endswith('.crash') or basename.endswith('.ips') 121 141 122 142 logs = self._host.filesystem.files_under(self._crash_log_directory, file_filter=is_crash_log) 123 first_line_regex = re.compile(r'^Process:\s+(?P<process_name>.*) \[(?P<pid>\d+)\]$')124 143 errors = '' 125 144 crash_logs = {} … … 128 147 if not newer_than or self._host.filesystem.mtime(path) > newer_than: 129 148 result_name = "Unknown" 130 pid = 0 131 log_contents = self._host.filesystem.read_text_file(path) 149 parsed_name, parsed_pid, log_contents = self._parse_darwin_crash_log(path) 150 if not log_contents: 151 _log.warn('No data in crash log at {}'.format(path)) 152 continue 153 132 154 # Verify timestamp from log contents 133 155 crash_time = self.get_timestamp_from_log(log_contents) … … 137 159 continue 138 160 139 match = first_line_regex.match(log_contents[0:log_contents.find('\n')]) 140 if match: 141 process_name = match.group('process_name') 142 pid = str(match.group('pid')) 143 result_name = process_name + "-" + pid 161 if parsed_name: 162 result_name = parsed_name + "-" + str(parsed_pid) 144 163 145 164 while result_name in crash_logs: -
trunk/Tools/Scripts/webkitpy/common/system/crashlogs_unittest.py
r216776 r217853 35 35 36 36 def make_mock_crash_report_darwin(process_name, pid): 37 return """Process: {process_name} [{pid}] 37 return """Crash log may not start with Process line 38 Process: {process_name} [{pid}] 38 39 Path: /Volumes/Data/slave/snowleopard-intel-release-tests/build/WebKitBuild/Release/{process_name} 39 40 Identifier: {process_name} -
trunk/Tools/Scripts/webkitpy/common/system/systemhost.py
r133508 r217853 47 47 def make_file_lock(self, path): 48 48 return file_lock.FileLock(path) 49 50 def symbolicate_crash_log_if_needed(self, path): 51 return self.filesystem.read_text_file(path) -
trunk/Tools/Scripts/webkitpy/common/system/systemhost_mock.py
r133508 r217853 55 55 def make_file_lock(self, path): 56 56 return MockFileLock(path) 57 58 def symbolicate_crash_log_if_needed(self, path): 59 return self.filesystem.read_text_file(path) -
trunk/Tools/Scripts/webkitpy/port/apple.py
r216776 r217853 55 55 VERSION_FALLBACK_ORDER = [] 56 56 ARCHITECTURES = [] 57 _crash_logs_to_skip_for_host = {} 57 58 58 59 @classmethod … … 93 94 self._version = self._strip_port_name_prefix(port_name) 94 95 96 def setup_test_run(self, device_class=None): 97 self._crash_logs_to_skip_for_host[self.host] = self.host.filesystem.files_under(self.path_to_crash_logs()) 98 95 99 def default_timeout_ms(self): 96 100 if self.get_option('guard_malloc'): -
trunk/Tools/Scripts/webkitpy/port/base.py
r217572 r217853 1365 1365 raise NotImplementedError 1366 1366 1367 def _get_crash_log(self, name, pid, stdout, stderr, newer_than ):1367 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, target_host=None): 1368 1368 name_str = name or '<unknown process name>' 1369 1369 pid_str = str(pid or '<unknown>') -
trunk/Tools/Scripts/webkitpy/port/darwin.py
r216776 r217853 108 108 109 109 def _look_for_all_crash_logs_in_log_dir(self, newer_than): 110 crash_log = CrashLogs(self.host, self.path_to_crash_logs() )110 crash_log = CrashLogs(self.host, self.path_to_crash_logs(), crash_logs_to_skip=self._crash_logs_to_skip_for_host.get(self.host, [])) 111 111 return crash_log.find_all_logs(include_errors=True, newer_than=newer_than) 112 112 113 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, time_fn=None, sleep_fn=None, wait_for_log=True ):113 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, time_fn=None, sleep_fn=None, wait_for_log=True, target_host=None): 114 114 # Note that we do slow-spin here and wait, since it appears the time 115 115 # ReportCrash takes to actually write and flush the file varies when there are … … 118 118 sleep_fn = sleep_fn or time.sleep 119 119 crash_log = '' 120 crash_logs = CrashLogs( self.host, self.path_to_crash_logs())120 crash_logs = CrashLogs(target_host or self.host, self.path_to_crash_logs(), crash_logs_to_skip=self._crash_logs_to_skip_for_host.get(target_host or self.host, [])) 121 121 now = time_fn() 122 122 deadline = now + 5 * int(self.get_option('child_processes', 1)) -
trunk/Tools/Scripts/webkitpy/port/darwin_testcase.py
r216776 r217853 138 138 139 139 def test_get_crash_log(self): 140 # Darwin crash logs are tested elsewhere, so here we just make sure we don't crash.141 def fake_time_cb():142 times = [0, 20, 40]143 return lambda: times.pop(0)144 140 port = self.make_port(port_name=self.port_name) 145 141 port._get_crash_log('DumpRenderTree', 1234, None, None, time.time(), wait_for_log=False) -
trunk/Tools/Scripts/webkitpy/port/device.py
r216030 r217853 64 64 self.listening_socket = None 65 65 66 def symbolicate_crash_log_if_needed(self, path): 67 if hasattr(self.platform_device, 'symbolicate_crash_log_if_needed'): 68 return self.platform_device.symbolicate_crash_log_if_needed(path) 69 return self.filesystem.read_text_file(path) 70 66 71 @property 67 72 def executive(self): -
trunk/Tools/Scripts/webkitpy/port/driver.py
r217276 r217853 242 242 243 243 def _get_crash_log(self, stdout, stderr, newer_than): 244 return self._port._get_crash_log(self._crashed_process_name, self._crashed_pid, stdout, stderr, newer_than )244 return self._port._get_crash_log(self._crashed_process_name, self._crashed_pid, stdout, stderr, newer_than, target_host=self._target_host) 245 245 246 246 def _command_wrapper(self): -
trunk/Tools/Scripts/webkitpy/port/gtk.py
r210547 r217853 221 221 return super(GtkPort, self).check_sys_deps(needs_http) and self._driver_class().check_driver(self) 222 222 223 def _get_crash_log(self, name, pid, stdout, stderr, newer_than ):223 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, target_host=None): 224 224 name = "WebKitWebProcess" if name == "WebProcess" else name 225 225 return GDBCrashLogGenerator(name, pid, newer_than, self._filesystem, self._path_to_driver).generate_crash_log(stdout, stderr) -
trunk/Tools/Scripts/webkitpy/port/ios.py
r215965 r217853 125 125 126 126 for i in xrange(self.child_processes()): 127 self.target_host(i).prepare_for_testing( 127 host = self.target_host(i) 128 host.prepare_for_testing( 128 129 self.ports_to_forward(), 129 130 self.app_identifier_from_bundle(self._path_to_driver()), 130 131 self.layout_tests_dir(), 131 132 ) 133 self._crash_logs_to_skip_for_host[host] = host.filesystem.files_under(self.path_to_crash_logs()) 132 134 133 135 def clean_up_test_run(self): -
trunk/Tools/Scripts/webkitpy/port/ios_device.py
r216776 r217853 24 24 25 25 from webkitpy.common.memoized import memoized 26 from webkitpy.common.system.crashlogs import CrashLogs 26 27 from webkitpy.port.config import apple_additions 27 28 from webkitpy.port.ios import IOSPort … … 68 69 return port_name 69 70 71 def path_to_crash_logs(self): 72 if not apple_additions(): 73 raise RuntimeError(self.NO_ON_DEVICE_TESTING) 74 return apple_additions().ios_crash_log_path() 75 76 def _look_for_all_crash_logs_in_log_dir(self, newer_than): 77 log_list = {} 78 for device in self._device_for_worker_number_map(): 79 crash_log = CrashLogs(device, self.path_to_crash_logs(), crash_logs_to_skip=self._crash_logs_to_skip_for_host.get(device, [])) 80 log_list.update(crash_log.find_all_logs(include_errors=True, newer_than=newer_than)) 81 return log_list 82 83 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, time_fn=None, sleep_fn=None, wait_for_log=True, target_host=None): 84 if target_host: 85 return super(IOSDevicePort, self)._get_crash_log(name, pid, stdout, stderr, newer_than, time_fn=time_fn, sleep_fn=sleep_fn, wait_for_log=wait_for_log, target_host=target_host) 86 87 # We need to search every device since one was not specified. 88 for device in self._device_for_worker_number_map(): 89 stderr_out, crashlog = super(IOSDevicePort, self)._get_crash_log(name, pid, stdout, stderr, newer_than, time_fn=time_fn, sleep_fn=sleep_fn, wait_for_log=False, target_host=device) 90 if crashlog: 91 return (stderr, crashlog) 92 return (stderr, None) 93 70 94 # FIXME: These need device implementations <rdar://problem/30497991>. 71 def path_to_crash_logs(self):72 raise NotImplementedError73 74 95 def check_for_leaks(self, process_name, process_pid): 75 96 pass 76 77 def look_for_new_crash_logs(self, crashed_processes, start_time):78 return {}79 97 80 98 def look_for_new_samples(self, unresponsive_processes, start_time): … … 91 109 return 'ios-device' 92 110 93 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, time_fn=None, sleep_fn=None, wait_for_log=True):94 return (stderr, None)95 96 111 def _create_devices(self, device_class): 97 112 if not apple_additions(): -
trunk/Tools/Scripts/webkitpy/port/ios_device_unittest.py
r216776 r217853 21 21 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 22 22 23 import time 24 23 25 from webkitpy.port.ios_device import IOSDevicePort 24 26 from webkitpy.port import ios_testcase … … 38 40 self.assertEqual('ios-device', self.make_port().operating_system()) 39 41 42 def test_crashlog_path(self): 43 port = self.make_port() 44 with self.assertRaises(RuntimeError): 45 port.path_to_crash_logs() 46 47 def test_get_crash_log(self): 48 port = self.make_port(port_name=self.port_name) 49 with self.assertRaises(RuntimeError): 50 port._get_crash_log('DumpRenderTree', 1234, None, None, time.time(), wait_for_log=False) 51 40 52 # FIXME: Update tests when <rdar://problem/30497991> is completed. 41 def test_crashlog_path(self):42 pass43 44 53 def test_spindump(self): 45 54 pass -
trunk/Tools/Scripts/webkitpy/port/win.py
r216788 r217853 373 373 return self.results_directory() 374 374 375 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, time_fn=None, sleep_fn=None, wait_for_log=True ):375 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, time_fn=None, sleep_fn=None, wait_for_log=True, target_host=None): 376 376 # Note that we do slow-spin here and wait, since it appears the time 377 377 # ReportCrash takes to actually write and flush the file varies when there are … … 381 381 sleep_fn = sleep_fn or time.sleep 382 382 crash_log = '' 383 crash_logs = CrashLogs( self.host, self.path_to_crash_logs())383 crash_logs = CrashLogs(target_host or self.host, self.path_to_crash_logs(), crash_logs_to_skip=self._crash_logs_to_skip_for_host.get(target_host or self.host, [])) 384 384 now = time_fn() 385 385 # FIXME: delete this after we're sure this code is working ... -
trunk/Tools/Scripts/webkitpy/port/wpe.py
r216497 r217853 75 75 return self._built_executables_path('ImageDiff') 76 76 77 def _get_crash_log(self, name, pid, stdout, stderr, newer_than ):77 def _get_crash_log(self, name, pid, stdout, stderr, newer_than, target_host=None): 78 78 name = "WPEWebProcess" if name == "WebProcess" else name 79 79 return GDBCrashLogGenerator(name, pid, newer_than, self._filesystem, self._path_to_driver).generate_crash_log(stdout, stderr)
Note: See TracChangeset
for help on using the changeset viewer.