Changeset 277027 in webkit


Ignore:
Timestamp:
May 5, 2021, 10:28:13 AM (4 years ago)
Author:
mark.lam@apple.com
Message:

Enable incremental sweeping of GCAwareJITStubRoutines.
​https://bugs.webkit.org/show_bug.cgi?id=225376

Reviewed by Filip Pizlo.

This patch makes the following changes:

  1. Enhance JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines() to be able to run in an incremental time slice.
  1. Added JITStubRoutineSet::notifyHaveRoutinesToDelete() so that GCAwareJITStubRoutine::observeZeroRefCount() can flag that the GC may have some dead GCAwareJITStubRoutines to delete.
  1. Added JITStubRoutineSet::mayHaveRoutinesToDelete() so that clients can do a cheap check ahead of time to determine if there's work to do, and avoid calling JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines() altogether if not needed.
  1. Added Heap::mayHaveJITStubRoutinesToDelete() and Heap::deleteDeadJITStubRoutines() as wrappers around JITStubRoutineSet::mayHaveRoutinesToDelete() and JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines() because the use of the JITStubRoutineSet is a heap internal implementation detail.
  1. Enhanced the IncrementalSweeper to also call Heap::deleteDeadJITStubRoutines() if needed.
  1. Enhanced Heap::sweepSynchronously() to also call Heap::deleteDeadJITStubRoutines() if needed.
  1. Time slices for Heap::deleteDeadJITStubRoutines() is currently set at the current values:
    1. max of 1 ms (1/10 of the IncreamentalSweeper's time slice) when invoked from the IncreamentalSweeper.
    2. max of 5 ms when invoked from Heap::deleteUnmarkedCompiledCode().
    3. unlimited time (with a sanity check) when called from Heap::sweepSynchronously().

The choices of 1ms and 5ms were picked to not be too long, but would still delete
the bulk of the dead GCAwareJITStubRoutines quickly enough based on data from my
instrumented runs the CLI version of JetStream2.

I think these hardcoded values will do for now. If need be, we can try something
more sophisticated later.

  • CMakeLists.txt:
  • heap/Heap.cpp:

(JSC::Heap::deleteUnmarkedCompiledCode):
(JSC::Heap::sweepSynchronously):

  • heap/Heap.h:
  • heap/HeapInlines.h:

(JSC::Heap::mayHaveJITStubRoutinesToDelete):
(JSC::Heap::deleteDeadJITStubRoutines):

  • heap/IncrementalSweeper.cpp:

(JSC::IncrementalSweeper::doSweep):

  • heap/JITStubRoutineSet.cpp:

(JSC::JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines):

  • heap/JITStubRoutineSet.h:

(JSC::JITStubRoutineSet::mayHaveRoutinesToDelete):
(JSC::JITStubRoutineSet::notifyHaveRoutinesToDelete):
(JSC::JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines):

  • jit/GCAwareJITStubRoutine.cpp:

(JSC::GCAwareJITStubRoutine::observeZeroRefCount):

  • jit/JITStubRoutine.h:

(JSC::JITStubRoutine::createSelfManagedRoutine): Deleted.

Location:
trunk/Source/JavaScriptCore
Files:
10 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/CMakeLists.txt

    r276676 r277027  
    673673    heap/IsoSubspaceInlines.h
    674674    heap/IsoSubspacePerVM.h
     675    heap/JITStubRoutineSet.h
    675676    heap/LocalAllocator.h
    676677    heap/LocalAllocatorInlines.h

  • trunk/Source/JavaScriptCore/ChangeLog

    r276942 r277027  
     12021-05-05  Mark Lam  <mark.lam@apple.com>
     2
     3        Enable incremental sweeping of GCAwareJITStubRoutines.
     4        https://bugs.webkit.org/show_bug.cgi?id=225376
     5
     6        Reviewed by Filip Pizlo.
     7
     8        This patch makes the following changes:
     9
     10        1. Enhance JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines() to be able to
     11           run in an incremental time slice.
     12
     13        2. Added JITStubRoutineSet::notifyHaveRoutinesToDelete() so that
     14           GCAwareJITStubRoutine::observeZeroRefCount() can flag that the GC may have
     15           some dead GCAwareJITStubRoutines to delete.
     16
     17        3. Added JITStubRoutineSet::mayHaveRoutinesToDelete() so that clients can do
     18           a cheap check ahead of time to determine if there's work to do, and avoid
     19           calling JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines() altogether
     20           if not needed.
     21
     22        4. Added Heap::mayHaveJITStubRoutinesToDelete() and Heap::deleteDeadJITStubRoutines()
     23           as wrappers around JITStubRoutineSet::mayHaveRoutinesToDelete() and
     24           JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines() because the use of
     25           the JITStubRoutineSet is a heap internal implementation detail.
     26
     27        5. Enhanced the IncrementalSweeper to also call Heap::deleteDeadJITStubRoutines()
     28           if needed.
     29
     30        6. Enhanced Heap::sweepSynchronously() to also call Heap::deleteDeadJITStubRoutines()
     31           if needed.
     32
     33        7. Time slices for Heap::deleteDeadJITStubRoutines() is currently set at the
     34           current values:
     35           a. max of 1 ms (1/10 of the IncreamentalSweeper's time slice) when invoked from
     36              the IncreamentalSweeper.
     37           b. max of 5 ms when invoked from Heap::deleteUnmarkedCompiledCode().
     38           c. unlimited time (with a sanity check) when called from Heap::sweepSynchronously().
     39
     40           The choices of 1ms and 5ms were picked to not be too long, but would still delete
     41           the bulk of the dead GCAwareJITStubRoutines quickly enough based on data from my
     42           instrumented runs the CLI version of JetStream2.
     43
     44           I think these hardcoded values will do for now.  If need be, we can try something
     45           more sophisticated later.
     46
     47        * CMakeLists.txt:
     48        * heap/Heap.cpp:
     49        (JSC::Heap::deleteUnmarkedCompiledCode):
     50        (JSC::Heap::sweepSynchronously):
     51        * heap/Heap.h:
     52        * heap/HeapInlines.h:
     53        (JSC::Heap::mayHaveJITStubRoutinesToDelete):
     54        (JSC::Heap::deleteDeadJITStubRoutines):
     55        * heap/IncrementalSweeper.cpp:
     56        (JSC::IncrementalSweeper::doSweep):
     57        * heap/JITStubRoutineSet.cpp:
     58        (JSC::JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines):
     59        * heap/JITStubRoutineSet.h:
     60        (JSC::JITStubRoutineSet::mayHaveRoutinesToDelete):
     61        (JSC::JITStubRoutineSet::notifyHaveRoutinesToDelete):
     62        (JSC::JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines):
     63        * jit/GCAwareJITStubRoutine.cpp:
     64        (JSC::GCAwareJITStubRoutine::observeZeroRefCount):
     65        * jit/JITStubRoutine.h:
     66        (JSC::JITStubRoutine::createSelfManagedRoutine): Deleted.
     67
    1682021-05-03  Mark Lam  <mark.lam@apple.com>
    269

  • trunk/Source/JavaScriptCore/heap/Heap.cpp

    r276324 r277027  
    974974    // And CodeBlock destructor is assuming that CodeBlock gets destroyed before UnlinkedCodeBlock gets destroyed.
    975975    vm().forEachCodeBlockSpace([] (auto& space) { space.space.sweep(); });
    976     m_jitStubRoutines->deleteUnmarkedJettisonedStubRoutines();
     976    if (mayHaveJITStubRoutinesToDelete())
     977        deleteDeadJITStubRoutines(5_ms);
    977978}
    978979
    … …  
    10411042    m_objectSpace.sweepBlocks();
    10421043    m_objectSpace.shrink();
     1044
     1045    unsigned passes = 0;
     1046    while (mayHaveJITStubRoutinesToDelete()) {
     1047        constexpr Seconds unlimitedTime = 600_s;
     1048        deleteDeadJITStubRoutines(unlimitedTime);
     1049        RELEASE_ASSERT(passes++ < 100);
     1050    }
     1051
    10431052    if (UNLIKELY(Options::logGC())) {
    10441053        MonotonicTime after = MonotonicTime::now();

  • trunk/Source/JavaScriptCore/heap/Heap.h

    r276324 r277027  
    407407    bool isMarkingForGCVerifier() const { return m_isMarkingForGCVerifier; }
    408408
     409    static bool mayHaveJITStubRoutinesToDelete();
     410    void deleteDeadJITStubRoutines(Seconds timeSlice);
     411
    409412private:
    410413    friend class AllocatingScope;

  • trunk/Source/JavaScriptCore/heap/HeapInlines.h

    r273138 r277027  
    3030#include "HeapCellInlines.h"
    3131#include "IndexingHeader.h"
     32#include "JITStubRoutineSet.h"
    3233#include "JSCast.h"
    3334#include "Structure.h"
    … …  
    280281}
    281282
     283inline bool Heap::mayHaveJITStubRoutinesToDelete()
     284{
     285    return JITStubRoutineSet::mayHaveRoutinesToDelete();
     286}
     287
     288inline void Heap::deleteDeadJITStubRoutines(Seconds timeSlice)
     289{
     290    m_jitStubRoutines->deleteUnmarkedJettisonedStubRoutines(timeSlice);
     291}
     292
    282293} // namespace JSC

  • trunk/Source/JavaScriptCore/heap/IncrementalSweeper.cpp

    r261895 r277027  
    3737static constexpr double sweepTimeTotal = .10;
    3838static constexpr double sweepTimeMultiplier = 1.0 / sweepTimeTotal;
     39static constexpr Seconds deleteJITStubRoutinesTimeSlice = std::min(sweepTimeSlice / 10, 1_ms);
    3940
    4041void IncrementalSweeper::scheduleTimer()
    … …  
    5657void IncrementalSweeper::doSweep(VM& vm, MonotonicTime sweepBeginTime)
    5758{
    58     while (sweepNextBlock(vm)) {
     59    bool hasMoreBlocksToSweep = true;
     60    bool hasMoreWork = true;
     61    while (hasMoreWork) {
     62        if (hasMoreBlocksToSweep)
     63            hasMoreBlocksToSweep = sweepNextBlock(vm);
     64
     65        if (Heap::mayHaveJITStubRoutinesToDelete())
     66            vm.heap.deleteDeadJITStubRoutines(deleteJITStubRoutinesTimeSlice);
     67
     68        hasMoreWork = hasMoreBlocksToSweep || Heap::mayHaveJITStubRoutinesToDelete();
     69
    5970        Seconds elapsedTime = MonotonicTime::now() - sweepBeginTime;
    6071        if (elapsedTime < sweepTimeSlice)

  • trunk/Source/JavaScriptCore/heap/JITStubRoutineSet.cpp

    r275490 r277027  
    3030
    3131#include "GCAwareJITStubRoutine.h"
     32#include <algorithm>
    3233
    3334namespace JSC {
     35
     36using WTF::Range;
     37
     38bool JITStubRoutineSet::s_mayHaveRoutinesToDelete = false;
    3439
    3540JITStubRoutineSet::JITStubRoutineSet() { }
    … …  
    114119}
    115120
    116 void JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines()
     121void JITStubRoutineSet::deleteUnmarkedJettisonedStubRoutines(Seconds timeSlice)
    117122{
     123    ASSERT(s_mayHaveRoutinesToDelete);
     124
     125    MonotonicTime startTime = MonotonicTime::now();
     126    Seconds elapsedTime;
     127    constexpr unsigned maxBatchSize = 100;
     128
     129    unsigned endIndex = m_routines.size();
     130
     131    // Clear the s_mayHaveRoutinesToDelete flag before we start.
     132    // Destruction of a MarkingGCAwareJITStubRoutine can trigger more routines
     133    // to be deleted, and some of those may be the ones we have already iterated
     134    // pass.
     135    s_mayHaveRoutinesToDelete = false;
     136
    118137    unsigned srcIndex = 0;
    119     unsigned dstIndex = srcIndex;
    120     while (srcIndex < m_routines.size()) {
    121         Routine routine = m_routines[srcIndex++];
    122         if (!routine.routine->m_isJettisoned || routine.routine->m_mayBeExecuting) {
    123             m_routines[dstIndex++] = routine;
    124             continue;
     138    while (srcIndex < endIndex) {
     139        unsigned batchSize = std::min<unsigned>(maxBatchSize, endIndex - srcIndex);
     140        while (batchSize--) {
     141            Routine routine = m_routines[srcIndex];
     142            if (!routine.routine->m_isJettisoned || routine.routine->m_mayBeExecuting) {
     143                srcIndex++;
     144                continue;
     145            }
     146            m_routines[srcIndex] = m_routines[--endIndex];
     147
     148            routine.routine->deleteFromGC();
    125149        }
    126         routine.routine->deleteFromGC();
     150
     151        elapsedTime = MonotonicTime::now() - startTime;
     152        if (elapsedTime > timeSlice) {
     153            // We timed out. Assume there's more to do, and that we should check
     154            // again next time slice.
     155            s_mayHaveRoutinesToDelete = true;
     156            break;
     157        }
    127158    }
    128     m_routines.shrinkCapacity(dstIndex);
     159
     160    m_routines.shrinkCapacity(endIndex);
    129161}
    130162

  • trunk/Source/JavaScriptCore/heap/JITStubRoutineSet.h

    r273138 r277027  
    3232#include <wtf/Vector.h>
    3333
    34 using WTF::Range;
    35 
    3634namespace JSC {
    3735
    … …  
    6260    void prepareForConservativeScan();
    6361   
    64     void deleteUnmarkedJettisonedStubRoutines();
     62    void deleteUnmarkedJettisonedStubRoutines(Seconds timeSlice);
    6563
    6664    template<typename Visitor> void traceMarkedStubRoutines(Visitor&);
    67    
     65
     66    static bool mayHaveRoutinesToDelete() { return s_mayHaveRoutinesToDelete; }
     67    static void notifyHaveRoutinesToDelete() { s_mayHaveRoutinesToDelete = true; }
     68
    6869private:
    6970    void markSlow(uintptr_t address);
    … …  
    7475    };
    7576    Vector<Routine> m_routines;
    76     Range<uintptr_t> m_range { 0, 0 };
     77    WTF::Range<uintptr_t> m_range { 0, 0 };
     78
     79    static bool s_mayHaveRoutinesToDelete;
    7780};
    7881
    … …  
    9194    void mark(void*) { }
    9295    void prepareForConservativeScan() { }
    93     void deleteUnmarkedJettisonedStubRoutines() { }
     96    void deleteUnmarkedJettisonedStubRoutines(Seconds) { }
    9497    template<typename Visitor> void traceMarkedStubRoutines(Visitor&) { }
     98
     99    static bool mayHaveRoutinesToDelete() { return false; }
     100    static void notifyHaveRoutinesToDelete() { }
    95101};
    96102

  • trunk/Source/JavaScriptCore/jit/GCAwareJITStubRoutine.cpp

    r273138 r277027  
    6363
    6464    m_isJettisoned = true;
     65    JITStubRoutineSet::notifyHaveRoutinesToDelete();
    6566}
    6667

  • trunk/Source/JavaScriptCore/jit/JITStubRoutine.h

    r230748 r277027  
    11/*
    2  * Copyright (C) 2012-2018 Apple Inc. All rights reserved.
     2 * Copyright (C) 2012-2021 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    5454        , m_refCount(1)
    5555    {
    56     }
    57    
    58     // Use this if you want to pass a CodePtr to someone who insists on taking
    59     // a RefPtr<JITStubRoutine>.
    60     static Ref<JITStubRoutine> createSelfManagedRoutine(
    61         MacroAssemblerCodePtr<JITStubRoutinePtrTag> rawCodePointer)
    62     {
    63         return adoptRef(*new JITStubRoutine(MacroAssemblerCodeRef<JITStubRoutinePtrTag>::createSelfManagedCodeRef(rawCodePointer)));
    6456    }
    6557   
Note: See TracChangeset for help on using the changeset viewer.