⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 243559 in webkit


Ignore:
Timestamp:
Mar 27, 2019, 1:25:15 PM (7 years ago)
Author:
ddkilzer@apple.com
Message:

run-webkit-tests should check for leaks in WebKit processes
<​https://webkit.org/b/193772>
<rdar://problem/46526680>

Reviewed by Ryosuke Niwa.

This works by doing the following:

  • Add a "#LIST CHILD PROCESSES" command to WebKitTestRunnner. The list of child processes are returned one per line: process.name: pid
  • Run the "#LIST CHILD PROCESSES" command just before the "#CHECK FOR WORLD LEAKS" command, and store the list of child processes on the ServerProcess object.
  • When the --leaks switch is handled, run check_for_leaks() on each child process after the main test harness.
  • DumpRenderTree/mac/DumpRenderTree.mm:

(handleControlCommand):

  • Use strncmp() instead of strcmp().
  • Add support for handling "#LIST CHILD PROCESSES" command.
  • Scripts/webkitpy/port/base.py:

(Port.check_for_leaks):

  • Scripts/webkitpy/port/darwin.py:

(DarwinPort.check_for_leaks):

  • Rename redundant 'process_pid' argument to 'process_id'.
  • Scripts/webkitpy/port/driver.py:

(Driver.do_post_tests_work):

  • Restructure the logic since "#CHECK FOR WORLD LEAKS" is no longer the only command this sends to WebKitTestRunner.
  • If the --leaks switch is present, send the "#LIST CHILD PROCESSES" to WebKitTestRunner and store the result using Port.set_webkit_processes().

(Driver._parse_child_processes_output):

  • Add helper method to parse list of child process names and process IDs returned from WebKitTestRunner.
  • Scripts/webkitpy/port/ios_device.py:

(IOSDevicePort.check_for_leaks):

  • Rename redundant 'process_pid' argument to 'process_id'.
  • Scripts/webkitpy/port/leakdetector.py:

(LeakDetector._parse_leaks_output):

  • Return early if there is no leaks_output.

(LeakDetector.check_for_leaks):

  • Rename redundant 'process_pid' argument to 'process_id'.
  • Scripts/webkitpy/port/server_process.py:

(ServerProcess.init):
(ServerProcess.child_processes):
(ServerProcess.set_child_processes):

  • Add instance variable to Port to store list of child process names and process IDs returned from WebKitTestRunner.

(ServerProcess._start):

  • Clear self._child_processes.

(ServerProcess.stop):

  • If self._child_processes is set, call self._port.check_for_leaks() for each child process.
  • Scripts/webkitpy/port/server_process_unittest.py:

(TrivialMockPort.check_for_leaks):

  • Rename redundant 'process_pid' argument to 'process_id'.
  • Scripts/webkitpy/port/simulator_process.py:

(SimulatorProcess.stop):

  • If self._child_processes is set, call self._port.check_for_leaks() for each child process.
  • Scripts/webkitpy/port/watch_device.py:

(WatchDevicePort.check_for_leaks):

  • Rename redundant 'process_pid' argument to 'process_id'.
  • WebKitTestRunner/TestController.cpp:

(WTR::TestController::dumpResponse):

  • Extract method from findAndDumpWorldLeaks() so that it may be reused by findAndDumpWebKitProcessIdentifiers().

(WTR::TestController::findAndDumpWebKitProcessIdentifiers):

  • Add method to output process name and process ID of both the WebContent and Networking processes.

(WTR::TestController::findAndDumpWorldLeaks):

  • Fix missing newline in output when there were no abandoned documents.
  • Call dumpResponse() for extracted code.

(WTR::TestController::handleControlCommand):

  • Restructure the logic for "#CHECK FOR WORLD LEAKS".
  • Use strncmp() instead of strcmp().
  • Call findAndDumpWebKitProcessIdentifiers() when "#LIST CHILD PROCESSES" command is sent.
  • WebKitTestRunner/TestController.h:

(WTR::TestController::dumpResponse):
(WTR::TestController::findAndDumpWebKitProcessIdentifiers):

  • Declare methods.
Location:
trunk/Tools
Files:
13 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r243534 r243559  
     12019-03-27  David Kilzer  <ddkilzer@apple.com>
     2
     3        run-webkit-tests should check for leaks in WebKit processes
     4        <https://webkit.org/b/193772>
     5        <rdar://problem/46526680>
     6
     7        Reviewed by Ryosuke Niwa.
     8
     9        This works by doing the following:
     10        - Add a "#LIST CHILD PROCESSES" command to WebKitTestRunnner.
     11          The list of child processes are returned one per line:
     12          process.name: pid
     13        - Run the "#LIST CHILD PROCESSES" command just before the
     14          "#CHECK FOR WORLD LEAKS" command, and store the list of child
     15          processes on the ServerProcess object.
     16        - When the `--leaks` switch is handled, run check_for_leaks() on
     17          each child process after the main test harness.
     18
     19        * DumpRenderTree/mac/DumpRenderTree.mm:
     20        (handleControlCommand):
     21        - Use strncmp() instead of strcmp().
     22        - Add support for handling "#LIST CHILD PROCESSES" command.
     23
     24        * Scripts/webkitpy/port/base.py:
     25        (Port.check_for_leaks):
     26        * Scripts/webkitpy/port/darwin.py:
     27        (DarwinPort.check_for_leaks):
     28        - Rename redundant 'process_pid' argument to 'process_id'.
     29
     30        * Scripts/webkitpy/port/driver.py:
     31        (Driver.do_post_tests_work):
     32        - Restructure the logic since "#CHECK FOR WORLD LEAKS" is no
     33          longer the only command this sends to WebKitTestRunner.
     34        - If the `--leaks` switch is present, send the
     35          "#LIST CHILD PROCESSES" to WebKitTestRunner and store the
     36          result using Port.set_webkit_processes().
     37        (Driver._parse_child_processes_output):
     38        - Add helper method to parse list of child process names and
     39          process IDs returned from WebKitTestRunner.
     40
     41        * Scripts/webkitpy/port/ios_device.py:
     42        (IOSDevicePort.check_for_leaks):
     43        - Rename redundant 'process_pid' argument to 'process_id'.
     44
     45        * Scripts/webkitpy/port/leakdetector.py:
     46        (LeakDetector._parse_leaks_output):
     47        - Return early if there is no leaks_output.
     48        (LeakDetector.check_for_leaks):
     49        - Rename redundant 'process_pid' argument to 'process_id'.
     50
     51        * Scripts/webkitpy/port/server_process.py:
     52        (ServerProcess.__init__):
     53        (ServerProcess.child_processes):
     54        (ServerProcess.set_child_processes):
     55        - Add instance variable to Port to store list of child process
     56          names and process IDs returned from WebKitTestRunner.
     57        (ServerProcess._start):
     58        - Clear self._child_processes.
     59        (ServerProcess.stop):
     60        - If self._child_processes is set, call
     61          self._port.check_for_leaks() for each child process.
     62
     63        * Scripts/webkitpy/port/server_process_unittest.py:
     64        (TrivialMockPort.check_for_leaks):
     65        - Rename redundant 'process_pid' argument to 'process_id'.
     66
     67        * Scripts/webkitpy/port/simulator_process.py:
     68        (SimulatorProcess.stop):
     69        - If self._child_processes is set, call
     70          self._port.check_for_leaks() for each child process.
     71
     72        * Scripts/webkitpy/port/watch_device.py:
     73        (WatchDevicePort.check_for_leaks):
     74        - Rename redundant 'process_pid' argument to 'process_id'.
     75
     76        * WebKitTestRunner/TestController.cpp:
     77        (WTR::TestController::dumpResponse):
     78        - Extract method from findAndDumpWorldLeaks() so that it may be
     79          reused by findAndDumpWebKitProcessIdentifiers().
     80        (WTR::TestController::findAndDumpWebKitProcessIdentifiers):
     81        - Add method to output process name and process ID of both the
     82          WebContent and Networking processes.
     83        (WTR::TestController::findAndDumpWorldLeaks):
     84        - Fix missing newline in output when there were no abandoned
     85          documents.
     86        - Call dumpResponse() for extracted code.
     87        (WTR::TestController::handleControlCommand):
     88        - Restructure the logic for "#CHECK FOR WORLD LEAKS".
     89        - Use strncmp() instead of strcmp().
     90        - Call findAndDumpWebKitProcessIdentifiers() when
     91          "#LIST CHILD PROCESSES" command is sent.
     92        * WebKitTestRunner/TestController.h:
     93        (WTR::TestController::dumpResponse):
     94        (WTR::TestController::findAndDumpWebKitProcessIdentifiers):
     95        - Declare methods.
     96
    1972019-03-27  Carlos Garcia Campos  <cgarcia@igalia.com>
    298

  • trunk/Tools/DumpRenderTree/mac/DumpRenderTree.mm

    r243482 r243559  
    11/*
    2  * Copyright (C) 2005-2018 Apple Inc. All rights reserved.
     2 * Copyright (C) 2005-2019 Apple Inc. All rights reserved.
    33 *           (C) 2007 Graham Dennis (graham.dennis@gmail.com)
    44 *
    … …  
    11681168static bool handleControlCommand(const char* command)
    11691169{
    1170     if (!strcmp("#CHECK FOR WORLD LEAKS", command)) {
    1171         // DumpRenderTree does not support checking for world leaks.
     1170    if (!strncmp("#CHECK FOR WORLD LEAKS", command, 22) || !strncmp("#LIST CHILD PROCESSES", command, 21)) {
     1171        // DumpRenderTree does not support checking for world leaks or listing child processes.
    11721172        WTF::String result("\n");
     1173        unsigned resultLength = result.length();
    11731174        printf("Content-Type: text/plain\n");
    1174         printf("Content-Length: %u\n", result.length());
    1175         fwrite(result.utf8().data(), 1, result.length(), stdout);
     1175        printf("Content-Length: %u\n", resultLength);
     1176        fwrite(result.utf8().data(), 1, resultLength, stdout);
    11761177        printf("#EOF\n");
    11771178        fprintf(stderr, "#EOF\n");

  • trunk/Tools/Scripts/webkitpy/port/base.py

    r241318 r243559  
    11# Copyright (C) 2010 Google Inc. All rights reserved.
    2 # Copyright (C) 2013 Apple Inc. All rights reserved.
     2# Copyright (C) 2013-2019 Apple Inc. All rights reserved.
    33#
    44# Redistribution and use in source and binary forms, with or without
    … …  
    372372        return result
    373373
    374     def check_for_leaks(self, process_name, process_pid):
     374    def check_for_leaks(self, process_name, process_id):
    375375        # Subclasses should check for leaks in the running process
    376376        # and print any necessary warnings if leaks are found.

  • trunk/Tools/Scripts/webkitpy/port/darwin.py

    r243373 r243559  
    1 # Copyright (C) 2014-2016 Apple Inc. All rights reserved.
     1# Copyright (C) 2014-2019 Apple Inc. All rights reserved.
    22#
    33# Redistribution and use in source and binary forms, with or without
    … …  
    5757        return list(reversed([self._filesystem.join(self._webkit_baseline_path(p), 'TestExpectations') for p in self.baseline_search_path(device_type=device_type)]))
    5858
    59     def check_for_leaks(self, process_name, process_pid):
     59    def check_for_leaks(self, process_name, process_id):
    6060        if not self.get_option('leaks'):
    6161            return
     62
    6263        # We could use http://code.google.com/p/psutil/ to get the process_name from the pid.
    63         self._leak_detector.check_for_leaks(process_name, process_pid)
     64        self._leak_detector.check_for_leaks(process_name, process_id)
    6465
    6566    def print_leaks_summary(self):

  • trunk/Tools/Scripts/webkitpy/port/driver.py

    r241656 r243559  
    11# Copyright (C) 2011 Google Inc. All rights reserved.
    2 # Copyright (c) 2015, 2016 Apple Inc. All rights reserved.
     2# Copyright (c) 2015-2019 Apple Inc. All rights reserved.
    33#
    44# Redistribution and use in source and binary forms, with or without
    … …  
    252252
    253253    def do_post_tests_work(self):
    254         if not self._port.get_option('world_leaks'):
    255             return None
    256 
    257254        if not self._server_process:
    258255            return None
    259256
    260         _log.debug('Checking for world leaks...')
    261         self._server_process.write('#CHECK FOR WORLD LEAKS\n')
    262         deadline = time.time() + 20
    263         block = self._read_block(deadline, '', wait_for_stderr_eof=True)
    264 
    265         _log.debug('World leak result: %s' % (block.decoded_content))
    266 
    267         return self._parse_world_leaks_output(block.decoded_content)
     257        if self._port.get_option('leaks'):
     258            _log.debug('Gathering child processes...')
     259            self._server_process.write('#LIST CHILD PROCESSES\n')
     260            deadline = time.time() + 20
     261            block = self._read_block(deadline, '', wait_for_stderr_eof=True)
     262            self._server_process.set_child_processes(self._parse_child_processes_output(block.decoded_content))
     263
     264        if self._port.get_option('world_leaks'):
     265            _log.debug('Checking for world leaks...')
     266            self._server_process.write('#CHECK FOR WORLD LEAKS\n')
     267            deadline = time.time() + 20
     268            block = self._read_block(deadline, '', wait_for_stderr_eof=True)
     269
     270            _log.debug('World leak result: %s' % (block.decoded_content))
     271
     272            return self._parse_world_leaks_output(block.decoded_content)
     273
     274        return None
     275
     276    @staticmethod
     277    def _parse_child_processes_output(output):
     278        child_processes = defaultdict(list)
     279
     280        for line in output.splitlines():
     281            m = re.match('^([^:]+): ([0-9]+)$', line)
     282            if m:
     283                process_name = m.group(1)
     284                process_id = m.group(2)
     285                child_processes[process_name].append(process_id)
     286
     287        return child_processes
    268288
    269289    def _parse_world_leaks_output(self, output):

  • trunk/Tools/Scripts/webkitpy/port/ios_device.py

    r241202 r243559  
    1 # Copyright (C) 2014-2018 Apple Inc. All rights reserved.
     1# Copyright (C) 2014-2019 Apple Inc. All rights reserved.
    22#
    33# Redistribution and use in source and binary forms, with or without
    … …  
    104104
    105105    # FIXME: These need device implementations <rdar://problem/30497991>.
    106     def check_for_leaks(self, process_name, process_pid):
     106    def check_for_leaks(self, process_name, process_id):
    107107        pass
    108108

  • trunk/Tools/Scripts/webkitpy/port/leakdetector.py

    r243373 r243559  
    11# Copyright (C) 2010 Google Inc. All rights reserved.
     2# Copyright (C) 2011-2019 Apple Inc. All rights reserved.
    23#
    34# Redistribution and use in source and binary forms, with or without
    … …  
    7677
    7778    def _parse_leaks_output(self, leaks_output):
     79        if not leaks_output:
     80            return 0, 0, 0
    7881        _, count, bytes = re.search(r'Process (?P<pid>\d+): (?P<count>\d+) leaks? for (?P<bytes>\d+) total', leaks_output).groups()
    7982        excluded_match = re.search(r'(?P<excluded>\d+) leaks? excluded', leaks_output)
    … …  
    117120        return total_leaks
    118121
    119     def check_for_leaks(self, process_name, process_pid):
     122    def check_for_leaks(self, process_name, process_id):
    120123        _log.debug("Checking for leaks in %s" % process_name)
    121124        try:
    122             leaks_filename = self.leaks_file_name(process_name, process_pid)
     125            leaks_filename = self.leaks_file_name(process_name, process_id)
    123126            leaks_output_path = self._filesystem.join(self._port.results_directory(), leaks_filename)
    124127            # Oddly enough, run-leaks (or the underlying leaks tool) does not seem to always output utf-8,
    125128            # thus we pass decode_output=False.  Without this code we've seen errors like:
    126129            # "UnicodeDecodeError: 'utf8' codec can't decode byte 0x88 in position 779874: unexpected code byte"
    127             self._port._run_script("run-leaks", self._leaks_args(process_name, process_pid), include_configuration_arguments=False, decode_output=False)
     130            self._port._run_script("run-leaks", self._leaks_args(process_name, process_id), include_configuration_arguments=False, decode_output=False)
    128131            leaks_output = self._filesystem.read_binary_file(leaks_output_path)
    129132        except ScriptError as e:

  • trunk/Tools/Scripts/webkitpy/port/server_process.py

    r234430 r243559  
    1 # Copyright (C) 2017 Apple Inc. All rights reserved.
     1# Copyright (C) 2017-2019 Apple Inc. All rights reserved.
    22# Copyright (C) 2010 Google Inc. All rights reserved.
    33#
    … …  
    7878        self._target_host = target_host or port_obj.host
    7979        self._pid = None
     80        self._child_processes = {}
    8081        self._reset()
    8182
    … …  
    8384        # FIXME: there should be a way to get win32 vs. cygwin from platforminfo.
    8485        self._use_win32_apis = sys.platform.startswith('win')
     86
     87    def child_processes(self):
     88        return self._child_processes
     89
     90    def set_child_processes(self, child_processes):
     91        self._child_processes = child_processes
    8592
    8693    def pid(self):
    … …  
    124131            universal_newlines=self._universal_newlines)
    125132        self._pid = self._proc.pid
     133        self._child_processes = {}
    126134        if not self._use_win32_apis:
    127135            self._set_file_nonblocking(self._proc.stdout)
    … …  
    365373        if self.poll() is None:
    366374            self._port.check_for_leaks(self.process_name(), self.pid())
     375            for child_process_name in self._child_processes.keys():
     376                for child_process_id in self._child_processes[child_process_name]:
     377                    self._port.check_for_leaks(child_process_name, child_process_id)
    367378
    368379        if self._proc.stdin:

  • trunk/Tools/Scripts/webkitpy/port/server_process_unittest.py

    r238534 r243559  
    11# Copyright (C) 2011 Google Inc. All rights reserved.
     2# Copyright (C) 2011-2019 Apple Inc. All rights reserved.
    23#
    34# Redistribution and use in source and binary forms, with or without
    … …  
    4748        return "/mock-results"
    4849
    49     def check_for_leaks(self, process_name, process_pid):
     50    def check_for_leaks(self, process_name, process_id):
    5051        pass
    5152

  • trunk/Tools/Scripts/webkitpy/port/simulator_process.py

    r240356 r243559  
    121121        if self.poll() is None:
    122122            self._port.check_for_leaks(self.process_name(), self.pid())
     123            for child_process_name in self._child_processes.keys():
     124                for child_process_id in self._child_processes[child_process_name]:
     125                    self._port.check_for_leaks(child_process_name, child_process_id)
    123126
    124127        if self._proc and self._proc.pid:

  • trunk/Tools/Scripts/webkitpy/port/watch_device.py

    r241202 r243559  
    1 # Copyright (C) 2018 Apple Inc. All rights reserved.
     1# Copyright (C) 2018-2019 Apple Inc. All rights reserved.
    22#
    33# Redistribution and use in source and binary forms, with or without
    … …  
    102102
    103103    # FIXME: These need device implementations <rdar://problem/30497991>.
    104     def check_for_leaks(self, process_name, process_pid):
     104    def check_for_leaks(self, process_name, process_id):
    105105        pass
    106106

  • trunk/Tools/WebKitTestRunner/TestController.cpp

    r243523 r243559  
    11/*
    2  * Copyright (C) 2010-2018 Apple Inc. All rights reserved.
     2 * Copyright (C) 2010-2019 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    10351035}
    10361036
     1037void TestController::dumpResponse(const String& result)
     1038{
     1039    unsigned resultLength = result.length();
     1040    printf("Content-Type: text/plain\n");
     1041    printf("Content-Length: %u\n", resultLength);
     1042    fwrite(result.utf8().data(), 1, resultLength, stdout);
     1043    printf("#EOF\n");
     1044    fprintf(stderr, "#EOF\n");
     1045    fflush(stdout);
     1046    fflush(stderr);
     1047}
     1048
     1049void TestController::findAndDumpWebKitProcessIdentifiers()
     1050{
     1051    StringBuilder builder;
     1052
     1053#if PLATFORM(COCOA)
     1054    builder.append(TestController::webProcessName());
     1055    builder.appendLiteral(": ");
     1056    pid_t webContentPID = WKPageGetProcessIdentifier(TestController::singleton().mainWebView()->page());
     1057    builder.appendNumber(webContentPID);
     1058    builder.append('\n');
     1059
     1060    builder.append(TestController::networkProcessName());
     1061    builder.appendLiteral(": ");
     1062    pid_t networkingPID = WKContextGetNetworkProcessIdentifier(m_context.get());
     1063    builder.appendNumber(networkingPID);
     1064    builder.append('\n');
     1065#else
     1066    builder.append('\n');
     1067#endif
     1068
     1069    dumpResponse(builder.toString());
     1070}
     1071
    10371072void TestController::findAndDumpWorldLeaks()
    10381073{
    … …  
    10571092        }
    10581093    } else
    1059         builder.append("no abandoned documents");
    1060 
    1061     String result = builder.toString();
    1062     printf("Content-Type: text/plain\n");
    1063     printf("Content-Length: %u\n", result.length());
    1064     fwrite(result.utf8().data(), 1, result.length(), stdout);
    1065     printf("#EOF\n");
    1066     fprintf(stderr, "#EOF\n");
    1067     fflush(stdout);
    1068     fflush(stderr);
     1094        builder.append("no abandoned documents\n");
     1095
     1096    dumpResponse(builder.toString());
    10691097}
    10701098
    … …  
    15961624bool TestController::handleControlCommand(const char* command)
    15971625{
    1598     if (!strcmp("#CHECK FOR WORLD LEAKS", command)) {
    1599         if (!m_checkForWorldLeaks) {
     1626    if (!strncmp("#CHECK FOR WORLD LEAKS", command, 22)) {
     1627        if (m_checkForWorldLeaks)
     1628            findAndDumpWorldLeaks();
     1629        else
    16001630            WTFLogAlways("WebKitTestRunner asked to check for world leaks, but was not run with --world-leaks");
    1601             return true;
    1602         }
    1603         findAndDumpWorldLeaks();
    16041631        return true;
    16051632    }
     1633
     1634    if (!strncmp("#LIST CHILD PROCESSES", command, 21)) {
     1635        findAndDumpWebKitProcessIdentifiers();
     1636        return true;
     1637    }
     1638
    16061639    return false;
    16071640}

  • trunk/Tools/WebKitTestRunner/TestController.h

    r243496 r243559  
    11/*
    2  * Copyright (C) 2010, 2015-2017 Apple Inc. All rights reserved.
     2 * Copyright (C) 2010-2019 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    355355
    356356    void didReceiveLiveDocumentsList(WKArrayRef);
     357    void dumpResponse(const String&);
     358    void findAndDumpWebKitProcessIdentifiers();
    357359    void findAndDumpWorldLeaks();
    358360
Note: See TracChangeset for help on using the changeset viewer.