Changeset 259652 in webkit


Ignore:
Timestamp:
Apr 7, 2020 11:41:13 AM (4 years ago)
Author:
sbarati@apple.com
Message:

RAMification should have a way of gathering vmmaps for each test at the end of each run
​https://bugs.webkit.org/show_bug.cgi?id=210060

Reviewed by Yusuke Suzuki.

PerformanceTests:

When I was debugging a memory issue, I needed to gather vmmaps for each
RAMification subtest. This patch is checking in the code I wrote to be able
to do this. It works by:

  • Passing in an argument to RAMification saying we want vmmaps at the end of

each subtest run.

  • RAMification invokes jsc with an environment variable that tells the shell

to wait for one character of input from stdin before exiting.

  • The jsc shell also disables the bmalloc scavenger while waiting for input so the

vmmap we take from the python runner script represents the "footprint" score
of the benchmark. If the scavenger ran, it would end up releasing too much
memory for the vmmap to be useful.

  • The python script runs a vmmap, and then communicates to the jsc process

when the vmmap finishes running.

  • JetStream2/RAMification.py:

(parseArgs):
(BaseRunner.init):
(BaseRunner.getResults):
(LocalRunner.runOneTest):
(main):
(main.runTestList):

Source/bmalloc:

  • bmalloc/Scavenger.cpp:

(bmalloc::Scavenger::scavenge):
(bmalloc::Scavenger::partialScavenge):

  • bmalloc/Scavenger.h:

(bmalloc::Scavenger::disable):

  • bmalloc/bmalloc.cpp:

(bmalloc::api::disableScavenger):

  • bmalloc/bmalloc.h:

Source/JavaScriptCore:

  • jsc.cpp:

(main):

Source/WTF:

  • wtf/FastMalloc.cpp:

(WTF::fastDisableScavenger):

  • wtf/FastMalloc.h:
Location:
trunk
Files:
12 edited

Legend:

Unmodified
Added
Removed

  • trunk/PerformanceTests/ChangeLog

    r259582 r259652  
     12020-04-07  Saam Barati  <sbarati@apple.com>
     2
     3        RAMification should have a way of gathering vmmaps for each test at the end of each run
     4        https://bugs.webkit.org/show_bug.cgi?id=210060
     5
     6        Reviewed by Yusuke Suzuki.
     7
     8        When I was debugging a memory issue, I needed to gather vmmaps for each
     9        RAMification subtest. This patch is checking in the code I wrote to be able
     10        to do this. It works by:
     11        - Passing in an argument to RAMification saying we want vmmaps at the end of
     12        each subtest run.
     13        - RAMification invokes jsc with an environment variable that tells the shell
     14        to wait for one character of input from stdin before exiting.
     15        - The jsc shell also disables the bmalloc scavenger while waiting for input so the
     16        vmmap we take from the python runner script represents the "footprint" score
     17        of the benchmark. If the scavenger ran, it would end up releasing too much
     18        memory for the vmmap to be useful.
     19        - The python script runs a vmmap, and then communicates to the jsc process
     20        when the vmmap finishes running.
     21
     22        * JetStream2/RAMification.py:
     23        (parseArgs):
     24        (BaseRunner.__init__):
     25        (BaseRunner.getResults):
     26        (LocalRunner.runOneTest):
     27        (main):
     28        (main.runTestList):
     29
    1302020-04-06  Saam Barati  <sbarati@apple.com>
    231

  • trunk/PerformanceTests/JetStream2/RAMification.py

    r259098 r259652  
    4444peakFootprintRE = re.compile("Peak Footprint: (\d+(?:.\d+)?)")
    4545
    46 TestResult = collections.namedtuple("TestResult", ["name", "returnCode", "footprint", "peakFootprint"])
     46TestResult = collections.namedtuple("TestResult", ["name", "returnCode", "footprint", "peakFootprint", "vmmapOutput"])
    4747
    4848ramification_dir = os.path.abspath(os.path.dirname(os.path.realpath(__file__)))
    … …  
    121121    parser.add_argument("-n", "--run-no-jit", dest="runNoJITTests", nargs="?", const=True, default=None, type=optStrToBool, metavar="true / false", help="Run no JIT tests [default]")
    122122    parser.add_argument("-o", "--output", dest="jsonFilename", type=str, default=None, metavar="JSON-output-file", help="Path to JSON output")
     123    parser.add_argument("-m", "--vmmap", dest="takeVmmap", action="store_true", default=False, help="Take a vmmap after each test")
    123124
    124125    args = parser.parse_args()
    … …  
    147148        self.rootDir = args.testDir
    148149        self.environmentVars = {}
     150        self.vmmapOutput = ""
    149151
    150152    def setup(self):
    … …  
    179181
    180182    def getResults(self):
    181         return TestResult(name=self.testName, returnCode=self.returnCode, footprint=self.footprint, peakFootprint=self.peakFootprint)
     183        return TestResult(name=self.testName, returnCode=self.returnCode, footprint=self.footprint, peakFootprint=self.peakFootprint, vmmapOutput=self.vmmapOutput)
    182184
    183185
    … …  
    201203        self.resetForTest(test)
    202204
    203         proc = subprocess.Popen(args, cwd=self.rootDir, env=self.environmentVars, stdout=subprocess.PIPE, stderr=None, shell=False)
     205        proc = subprocess.Popen(args, cwd=self.rootDir, env=self.environmentVars, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=None, shell=False)
    204206        while True:
    205207            line = proc.stdout.readline()
    … …  
    208210
    209211            self.processLine(line)
     212
     213            if "js shell waiting for input to exit" in line:
     214                self.vmmapOutput = subprocess.Popen(['vmmap', '--summary', '{}'.format(proc.pid)], shell=False, stderr=subprocess.PIPE, stdout=subprocess.PIPE).stdout.read()
     215                if sys.version_info[0] >= 3:
     216                    self.vmmapOutput = str(self.vmmapOutput, "utf-8")
     217                proc.stdin.write(b"done\n")
     218                proc.stdin.flush()
    210219
    211220            if line == "":
    … …  
    226235
    227236    testRunner = args.runner(args)
     237
     238
     239    if args.takeVmmap:
     240        testRunner.setEnv("JS_SHELL_WAIT_FOR_INPUT_TO_EXIT", "1")
    228241
    229242    dyldFrameworkPath = frameworkPathFromExecutablePath(args.jscCommand)
    … …  
    256269                if args.verbose:
    257270                    print("footprint: {}, peak footprint: {}".format(testResult.footprint, testResult.peakFootprint))
     271                    if testResult.vmmapOutput:
     272                        print(testResult.vmmapOutput)
    258273                else:
    259274                    print

  • trunk/Source/JavaScriptCore/ChangeLog

    r259646 r259652  
     12020-04-07  Saam Barati  <sbarati@apple.com>
     2
     3        RAMification should have a way of gathering vmmaps for each test at the end of each run
     4        https://bugs.webkit.org/show_bug.cgi?id=210060
     5
     6        Reviewed by Yusuke Suzuki.
     7
     8        * jsc.cpp:
     9        (main):
     10
    1112020-04-07  Yusuke Suzuki  <ysuzuki@apple.com>
    212

  • trunk/Source/JavaScriptCore/jsc.cpp

    r259636 r259652  
    25292529    EXCEPT(res = 3)
    25302530    finalizeStatsAtEndOfTesting();
     2531    if (getenv("JS_SHELL_WAIT_FOR_INPUT_TO_EXIT")) {
     2532        WTF::fastDisableScavenger();
     2533        fprintf(stdout, "\njs shell waiting for input to exit\n");
     2534        fflush(stdout);
     2535        getc(stdin);
     2536    }
    25312537
    25322538    jscExit(res);

  • trunk/Source/WTF/ChangeLog

    r259606 r259652  
     12020-04-07  Saam Barati  <sbarati@apple.com>
     2
     3        RAMification should have a way of gathering vmmaps for each test at the end of each run
     4        https://bugs.webkit.org/show_bug.cgi?id=210060
     5
     6        Reviewed by Yusuke Suzuki.
     7
     8        * wtf/FastMalloc.cpp:
     9        (WTF::fastDisableScavenger):
     10        * wtf/FastMalloc.h:
     11
    1122020-04-06  Ross Kirsling  <ross.kirsling@sony.com>
    213

  • trunk/Source/WTF/wtf/FastMalloc.cpp

    r253987 r259652  
    275275void fastEnableMiniMode() { }
    276276
     277void fastDisableScavenger() { }
     278
    277279void fastMallocDumpMallocStats() { }
    278280
    … …  
    633635}
    634636
     637void fastDisableScavenger()
     638{
     639    bmalloc::api::disableScavenger();
     640}
     641
    635642} // namespace WTF
    636643

  • trunk/Source/WTF/wtf/FastMalloc.h

    r253987 r259652  
    7777WTF_EXPORT_PRIVATE void fastEnableMiniMode();
    7878
     79WTF_EXPORT_PRIVATE void fastDisableScavenger();
     80
    7981struct FastMallocStatistics {
    8082    size_t reservedVMBytes;

  • trunk/Source/bmalloc/ChangeLog

    r259466 r259652  
     12020-04-07  Saam Barati  <sbarati@apple.com>
     2
     3        RAMification should have a way of gathering vmmaps for each test at the end of each run
     4        https://bugs.webkit.org/show_bug.cgi?id=210060
     5
     6        Reviewed by Yusuke Suzuki.
     7
     8        * bmalloc/Scavenger.cpp:
     9        (bmalloc::Scavenger::scavenge):
     10        (bmalloc::Scavenger::partialScavenge):
     11        * bmalloc/Scavenger.h:
     12        (bmalloc::Scavenger::disable):
     13        * bmalloc/bmalloc.cpp:
     14        (bmalloc::api::disableScavenger):
     15        * bmalloc/bmalloc.h:
     16
    1172020-04-03  David Kilzer  <ddkilzer@apple.com>
    218

  • trunk/Source/bmalloc/bmalloc/Scavenger.cpp

    r257554 r259652  
    205205void Scavenger::scavenge()
    206206{
     207    if (!m_isEnabled)
     208        return;
     209
    207210    UniqueLockHolder lock(m_scavengingMutex);
    208211
    … …  
    280283void Scavenger::partialScavenge()
    281284{
     285    if (!m_isEnabled)
     286        return;
     287
    282288    UniqueLockHolder lock(m_scavengingMutex);
    283289

  • trunk/Source/bmalloc/bmalloc/Scavenger.h

    r254871 r259652  
    7575    void enableMiniMode();
    7676
     77    // Used for debugging only.
     78    void disable() { m_isEnabled = false; }
     79
    7780private:
    7881    enum class State { Sleep, Run, RunSoon };
    … …  
    116119   
    117120    Vector<DeferredDecommit> m_deferredDecommits;
     121    bool m_isEnabled { true };
    118122};
    119123DECLARE_STATIC_PER_PROCESS_STORAGE(Scavenger);

  • trunk/Source/bmalloc/bmalloc/bmalloc.cpp

    r254781 r259652  
    136136}
    137137
     138void disableScavenger()
     139{
     140    if (!DebugHeap::tryGet())
     141        Scavenger::get()->disable();
     142}
     143
    138144} } // namespace bmalloc::api
    139145

  • trunk/Source/bmalloc/bmalloc/bmalloc.h

    r244244 r259652  
    130130BEXPORT void enableMiniMode();
    131131
     132// Used for debugging only.
     133BEXPORT void disableScavenger();
     134
    132135} // namespace api
    133136} // namespace bmalloc
Note: See TracChangeset for help on using the changeset viewer.