Changeset 278875 in webkit


Ignore:
Timestamp:
Jun 15, 2021, 9:09:27 AM (4 years ago)
Author:
mark.lam@apple.com
Message:

Move setting of scratch buffer active lengths to the runtime functions.
​https://bugs.webkit.org/show_bug.cgi?id=227013
rdar://79325068

Reviewed by Keith Miller.

We previously emit JIT'ed code to set and unset the ScratchBuffer active length
around calls into C++ runtime functions. This was needed because the runtime
functions may allow GC to run, and GC needs to be able to scan the values stored
in the ScratchBuffer.

In this patch, we change it so that the runtime functions that need it will
declare an ActiveScratchBufferScope RAII object that will set the ScratchBuffer
active length, and unset it on exit. This allows us to:

  1. Emit less JIT code. The runtime function can take care of it.
  2. Elide setting the ScratchBuffer active length if not needed. The runtime functions know whether they can GC or not. They only need to set the active length if they can GC.

Note that scanning of the active ScratchBuffer is done synchronously on the
mutator thread via Heap::gatherScratchBufferRoots(), which is called as part of
the GC conservative root scan. This means there is no urgency / sequencing that
requires that the active length be set before calling into the runtime function.
Setting it in the runtime function itself is fine as long as it is done before
the function executes any operations that can GC.

This patch also made the following changes:

  1. Introduce ActiveScratchBufferScope RAII object used to set/unset the ScratchBuffer length in the runtime functions. ActiveScratchBufferScope takes the active length in units of number of stack slots / Registers / JSValues instead of bytes.
  1. Deleted ScratchRegisterAllocator::preserveUsedRegistersToScratchBufferForCall() and ScratchRegisterAllocator::restoreUsedRegistersFromScratchBufferForCall(). These functions are unused.

The reasoning behind what values to pass to ActiveScratchBufferScope, is any:

  1. AssemblyHelpers::debugCall() in AssemblyHelpers.cpp: The ScratchBuffer is only used for operationDebugPrintSpeculationFailure(), which now declares an ActiveScratchBufferScope.

The active length is GPRInfo::numberOfRegisters + FPRInfo::numberOfRegisters.
See scratchSize in AssemblyHelpers::debugCall().

  1. genericGenerationThunkGenerator() in FTLThunks.cpp: The scratch buffer size for determining the active length is requiredScratchMemorySizeInBytes().

However, genericGenerationThunkGenerator() generates code to call either
operationCompileFTLOSRExit() or operationCompileFTLLazySlowPath(). Both of
these functions will DeferGCForAWhile. Hence, GC cannot run, and we don't need
to set the active length here.

  1. compileArrayPush() in FTLLowerDFGToB3.cpp:

Cases Array::Int32, Array::Contiguous, or Array::Double calls
operationArrayPushMultiple() or operationArrayPushDoubleMultiple().

For operationArrayPushMultiple(), the active length is elementCount. See
computation of scratchSize.

For operationArrayPushDoubleMultiple(), we don't need to set the active length
because the ScratchBuffer only contains double values. The GC does not need
to scan those.

Case Array::ArrayStorage calls operationArrayPushMultiple().
The active length is elementCount. See computation of scratchSize.

compileNewArray() in FTLLowerDFGToB3.cpp:

Calls operationNewArray(). Active length is m_node->numChildren(), which is
passed to operationNewArray() as the size parameter. See computation of
scratchSize.

compileNewArrayWithSpread() in FTLLowerDFGToB3.cpp:

Calls operationNewArrayWithSpreadSlow(). Active length is m_node->numChildren(),
which is passes to operationNewArrayWithSpreadSlow() as the numItems parameter.
See computation of scratchSize.

  1. osrExitGenerationThunkGenerator() in DFGThunks.cpp:

Calls operationCompileOSRExit(). Active length is GPRInfo::numberOfRegisters +
FPRInfo::numberOfRegisters. See computation of scratchSize.

  1. compileNewArray() in DFGSpeculativeJIT.cpp:

Calls operationNewArray(). Active length is node->numChildren(), which is
passed in as the size parameter.

compileNewArrayWithSpread() in DFGSpeculativeJIT.cpp:

Calls operationNewArrayWithSpreadSlow(). Active length is node->numChildren(),
which is passed in as the numItems parameter.

compileArrayPush() in DFGSpeculativeJIT.cpp:

Calls operationArrayPushMultiple(). Active length is elementCount, which is
passed in as the elementCount parameter.

Calls operationArrayPushDoubleMultiple(). Active length is elementCount, but
we don't need to set it because the ScratchBuffer only contains double values.

  • dfg/DFGOSRExit.cpp:

(JSC::DFG::JSC_DEFINE_JIT_OPERATION):

  • dfg/DFGOperations.cpp:

(JSC::DFG::JSC_DEFINE_JIT_OPERATION):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::compileNewArray):
(JSC::DFG::SpeculativeJIT::compileNewArrayWithSpread):
(JSC::DFG::SpeculativeJIT::compileArrayPush):

  • dfg/DFGThunks.cpp:

(JSC::DFG::osrExitGenerationThunkGenerator):

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileArrayPush):
(JSC::FTL::DFG::LowerDFGToB3::compileNewArray):
(JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSpread):

  • ftl/FTLOSRExitCompiler.cpp:

(JSC::FTL::JSC_DEFINE_JIT_OPERATION):

  • ftl/FTLOperations.cpp:

(JSC::FTL::JSC_DEFINE_JIT_OPERATION):

  • ftl/FTLThunks.cpp:

(JSC::FTL::genericGenerationThunkGenerator):

  • jit/AssemblyHelpers.cpp:

(JSC::AssemblyHelpers::debugCall):

  • jit/ScratchRegisterAllocator.cpp:

(JSC::ScratchRegisterAllocator::preserveUsedRegistersToScratchBufferForCall): Deleted.
(JSC::ScratchRegisterAllocator::restoreUsedRegistersFromScratchBufferForCall): Deleted.

  • jit/ScratchRegisterAllocator.h:
  • runtime/VM.h:
  • runtime/VMInlines.h:

(JSC::ActiveScratchBufferScope::ActiveScratchBufferScope):
(JSC::ActiveScratchBufferScope::~ActiveScratchBufferScope):

Location:
trunk/Source/JavaScriptCore
Files:
14 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/ChangeLog

    r278859 r278875  
     12021-06-15  Mark Lam  <mark.lam@apple.com>
     2
     3        Move setting of scratch buffer active lengths to the runtime functions.
     4        https://bugs.webkit.org/show_bug.cgi?id=227013
     5        rdar://79325068
     6
     7        Reviewed by Keith Miller.
     8
     9        We previously emit JIT'ed code to set and unset the ScratchBuffer active length
     10        around calls into C++ runtime functions.  This was needed because the runtime
     11        functions may allow GC to run, and GC needs to be able to scan the values stored
     12        in the ScratchBuffer.
     13
     14        In this patch, we change it so that the runtime functions that need it will
     15        declare an ActiveScratchBufferScope RAII object that will set the ScratchBuffer
     16        active length, and unset it on exit.  This allows us to:
     17
     18        1. Emit less JIT code.  The runtime function can take care of it.
     19        2. Elide setting the ScratchBuffer active length if not needed.  The runtime
     20           functions know whether they can GC or not.  They only need to set the active
     21           length if they can GC.
     22
     23        Note that scanning of the active ScratchBuffer is done synchronously on the
     24        mutator thread via Heap::gatherScratchBufferRoots(), which is called as part of
     25        the GC conservative root scan.  This means there is no urgency / sequencing that
     26        requires that the active length be set before calling into the runtime function.
     27        Setting it in the runtime function itself is fine as long as it is done before
     28        the function executes any operations that can GC.
     29
     30        This patch also made the following changes:
     31
     32        1. Introduce ActiveScratchBufferScope RAII object used to set/unset the
     33           ScratchBuffer length in the runtime functions.  ActiveScratchBufferScope takes
     34           the active length in units of number of stack slots / Registers / JSValues
     35           instead of bytes.
     36
     37        2. Deleted ScratchRegisterAllocator::preserveUsedRegistersToScratchBufferForCall()
     38           and ScratchRegisterAllocator::restoreUsedRegistersFromScratchBufferForCall().
     39           These functions are unused.
     40
     41        The reasoning behind what values to pass to ActiveScratchBufferScope, is any:
     42
     43        1. AssemblyHelpers::debugCall() in AssemblyHelpers.cpp:
     44           The ScratchBuffer is only used for operationDebugPrintSpeculationFailure(),
     45           which now declares an ActiveScratchBufferScope.
     46
     47           The active length is GPRInfo::numberOfRegisters + FPRInfo::numberOfRegisters.
     48           See scratchSize in AssemblyHelpers::debugCall().
     49
     50        2. genericGenerationThunkGenerator() in FTLThunks.cpp:
     51           The scratch buffer size for determining the active length is
     52           requiredScratchMemorySizeInBytes(). 
     53
     54           However, genericGenerationThunkGenerator() generates code to call either
     55           operationCompileFTLOSRExit() or operationCompileFTLLazySlowPath().  Both of
     56           these functions will DeferGCForAWhile.  Hence, GC cannot run, and we don't need
     57           to set the active length here.
     58
     59        3. compileArrayPush() in FTLLowerDFGToB3.cpp:
     60
     61           Cases Array::Int32, Array::Contiguous, or Array::Double calls
     62           operationArrayPushMultiple() or operationArrayPushDoubleMultiple().
     63
     64           For operationArrayPushMultiple(), the active length is elementCount.  See
     65           computation of scratchSize.
     66
     67           For operationArrayPushDoubleMultiple(), we don't need to set the active length
     68           because the ScratchBuffer only contains double values.  The GC does not need
     69           to scan those.
     70
     71           Case Array::ArrayStorage calls operationArrayPushMultiple().
     72           The active length is elementCount.  See computation of scratchSize.
     73
     74           compileNewArray() in FTLLowerDFGToB3.cpp:
     75
     76           Calls operationNewArray().  Active length is m_node->numChildren(), which is
     77           passed to operationNewArray() as the size parameter.  See computation of
     78           scratchSize.
     79
     80           compileNewArrayWithSpread() in FTLLowerDFGToB3.cpp:
     81
     82           Calls operationNewArrayWithSpreadSlow().  Active length is m_node->numChildren(),
     83           which is passes to operationNewArrayWithSpreadSlow() as the numItems parameter.
     84           See computation of scratchSize.
     85
     86        4. osrExitGenerationThunkGenerator() in DFGThunks.cpp:
     87
     88           Calls operationCompileOSRExit().  Active length is GPRInfo::numberOfRegisters +
     89           FPRInfo::numberOfRegisters.  See computation of scratchSize.
     90
     91        5. compileNewArray() in DFGSpeculativeJIT.cpp:
     92
     93           Calls operationNewArray().  Active length is node->numChildren(), which is
     94           passed in as the size parameter.
     95
     96           compileNewArrayWithSpread() in DFGSpeculativeJIT.cpp:
     97
     98           Calls operationNewArrayWithSpreadSlow().  Active length is node->numChildren(),
     99           which is passed in as the numItems parameter.
     100
     101           compileArrayPush() in DFGSpeculativeJIT.cpp:
     102
     103           Calls operationArrayPushMultiple().  Active length is elementCount, which is
     104           passed in as the elementCount parameter.
     105
     106           Calls operationArrayPushDoubleMultiple().  Active length is elementCount, but
     107           we don't need to set it because the ScratchBuffer only contains double values.
     108
     109
     110        * dfg/DFGOSRExit.cpp:
     111        (JSC::DFG::JSC_DEFINE_JIT_OPERATION):
     112        * dfg/DFGOperations.cpp:
     113        (JSC::DFG::JSC_DEFINE_JIT_OPERATION):
     114        * dfg/DFGSpeculativeJIT.cpp:
     115        (JSC::DFG::SpeculativeJIT::compileNewArray):
     116        (JSC::DFG::SpeculativeJIT::compileNewArrayWithSpread):
     117        (JSC::DFG::SpeculativeJIT::compileArrayPush):
     118        * dfg/DFGThunks.cpp:
     119        (JSC::DFG::osrExitGenerationThunkGenerator):
     120        * ftl/FTLLowerDFGToB3.cpp:
     121        (JSC::FTL::DFG::LowerDFGToB3::compileArrayPush):
     122        (JSC::FTL::DFG::LowerDFGToB3::compileNewArray):
     123        (JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSpread):
     124        * ftl/FTLOSRExitCompiler.cpp:
     125        (JSC::FTL::JSC_DEFINE_JIT_OPERATION):
     126        * ftl/FTLOperations.cpp:
     127        (JSC::FTL::JSC_DEFINE_JIT_OPERATION):
     128        * ftl/FTLThunks.cpp:
     129        (JSC::FTL::genericGenerationThunkGenerator):
     130        * jit/AssemblyHelpers.cpp:
     131        (JSC::AssemblyHelpers::debugCall):
     132        * jit/ScratchRegisterAllocator.cpp:
     133        (JSC::ScratchRegisterAllocator::preserveUsedRegistersToScratchBufferForCall): Deleted.
     134        (JSC::ScratchRegisterAllocator::restoreUsedRegistersFromScratchBufferForCall): Deleted.
     135        * jit/ScratchRegisterAllocator.h:
     136        * runtime/VM.h:
     137        * runtime/VMInlines.h:
     138        (JSC::ActiveScratchBufferScope::ActiveScratchBufferScope):
     139        (JSC::ActiveScratchBufferScope::~ActiveScratchBufferScope):
     140
    11412021-06-14  Yusuke Suzuki  <ysuzuki@apple.com>
    2142

  • trunk/Source/JavaScriptCore/dfg/DFGOSRExit.cpp

    r277928 r278875  
    4242#include "OperandsInlines.h"
    4343#include "ProbeContext.h"
     44#include "VMInlines.h"
    4445
    4546#include <wtf/Scope.h>
    … …  
    145146    VM& vm = callFrame->deprecatedVM();
    146147    auto scope = DECLARE_THROW_SCOPE(vm);
     148    ActiveScratchBufferScope activeScratchBufferScope(vm, GPRInfo::numberOfRegisters + FPRInfo::numberOfRegisters);
    147149
    148150    if constexpr (validateDFGDoesGC) {
    … …  
    891893    VM& vm = callFrame->deprecatedVM();
    892894    NativeCallFrameTracer tracer(vm, callFrame);
     895    ActiveScratchBufferScope activeScratchBufferScope(vm, GPRInfo::numberOfRegisters + FPRInfo::numberOfRegisters);
    893896
    894897    SpeculationFailureDebugInfo* debugInfo = static_cast<SpeculationFailureDebugInfo*>(debugInfoRaw);

  • trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp

    r278462 r278875  
    10401040    CallFrame* callFrame = DECLARE_CALL_FRAME(vm);
    10411041    JITOperationPrologueCallFrameTracer tracer(vm, callFrame);
     1042    ActiveScratchBufferScope activeScratchBufferScope(vm, elementCount);
    10421043    auto scope = DECLARE_THROW_SCOPE(vm);
    10431044
    … …  
    10651066    CallFrame* callFrame = DECLARE_CALL_FRAME(vm);
    10661067    JITOperationPrologueCallFrameTracer tracer(vm, callFrame);
     1068    // Don't need a ActiveScratchBufferScope here because the scratch buffer only contains double values for this call.
    10671069    auto scope = DECLARE_THROW_SCOPE(vm);
    10681070
    … …  
    17331735    CallFrame* callFrame = DECLARE_CALL_FRAME(vm);
    17341736    JITOperationPrologueCallFrameTracer tracer(vm, callFrame);
    1735    
     1737    ActiveScratchBufferScope activeScratchBufferScope(vm, size);
     1738
    17361739    return bitwise_cast<char*>(constructArray(globalObject, arrayStructure, static_cast<JSValue*>(buffer), size));
    17371740}
    … …  
    30403043    CallFrame* callFrame = DECLARE_CALL_FRAME(vm);
    30413044    JITOperationPrologueCallFrameTracer tracer(vm, callFrame);
     3045    ActiveScratchBufferScope activeScratchBufferScope(vm, numItems);
    30423046    auto scope = DECLARE_THROW_SCOPE(vm);
    30433047

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r278662 r278875  
    91689168    flushRegisters();
    91699169
    9170     if (scratchSize) {
    9171         GPRTemporary scratch(this);
    9172 
    9173         // Tell GC mark phase how much of the scratch buffer is active during call.
    9174         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), scratch.gpr());
    9175         m_jit.storePtr(TrustedImmPtr(scratchSize), scratch.gpr());
    9176     }
    9177 
    91789170    GPRFlushedCallResult result(this);
    91799171
    … …  
    91829174        static_cast<void*>(buffer), size_t(node->numChildren()));
    91839175    m_jit.exceptionCheck();
    9184 
    9185     if (scratchSize) {
    9186         GPRTemporary scratch(this);
    9187 
    9188         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), scratch.gpr());
    9189         m_jit.storePtr(TrustedImmPtr(nullptr), scratch.gpr());
    9190     }
    91919176
    91929177    cellResult(result.gpr(), node, UseChildrenCalledExplicitly);
    … …  
    93409325    }
    93419326
    9342     {
    9343         GPRTemporary scratch(this);
    9344         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), scratch.gpr());
    9345         m_jit.storePtr(TrustedImmPtr(scratchSize), MacroAssembler::Address(scratch.gpr()));
    9346     }
    9347 
    93489327    flushRegisters();
    93499328
    … …  
    93539332    callOperation(operationNewArrayWithSpreadSlow, resultGPR, TrustedImmPtr::weakPointer(m_graph, globalObject), buffer, node->numChildren());
    93549333    m_jit.exceptionCheck();
    9355     {
    9356         GPRTemporary scratch(this);
    9357         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), scratch.gpr());
    9358         m_jit.storePtr(TrustedImmPtr(nullptr), MacroAssembler::Address(scratch.gpr()));
    9359     }
    93609334
    93619335    cellResult(resultGPR, node);
    … …  
    98769850        ScratchBuffer* scratchBuffer = vm().scratchBufferForSize(scratchSize);
    98779851        m_jit.move(TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())), bufferGPR);
    9878         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), storageLengthGPR);
    9879         m_jit.storePtr(TrustedImmPtr(scratchSize), MacroAssembler::Address(storageLengthGPR));
    98809852
    98819853        storageDone.link(&m_jit);
    … …  
    98929864
    98939865        addSlowPathGenerator(slowPathCall(m_jit.jump(), this, operationArrayPushMultiple, resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), baseGPR, bufferGPR, TrustedImm32(elementCount)));
    9894 
    9895         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), bufferGPR);
    9896         m_jit.storePtr(TrustedImmPtr(nullptr), MacroAssembler::Address(bufferGPR));
    98979866
    98989867        base.use();
    … …  
    99509919        ScratchBuffer* scratchBuffer = vm().scratchBufferForSize(scratchSize);
    99519920        m_jit.move(TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())), bufferGPR);
    9952         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), storageLengthGPR);
    9953         m_jit.storePtr(TrustedImmPtr(scratchSize), MacroAssembler::Address(storageLengthGPR));
    99549921
    99559922        storageDone.link(&m_jit);
    … …  
    99669933
    99679934        addSlowPathGenerator(slowPathCall(m_jit.jump(), this, operationArrayPushDoubleMultiple, resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), baseGPR, bufferGPR, TrustedImm32(elementCount)));
    9968 
    9969         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), bufferGPR);
    9970         m_jit.storePtr(TrustedImmPtr(nullptr), MacroAssembler::Address(bufferGPR));
    99719935
    99729936        base.use();
    … …  
    100319995        ScratchBuffer* scratchBuffer = vm().scratchBufferForSize(scratchSize);
    100329996        m_jit.move(TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())), bufferGPR);
    10033         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), storageLengthGPR);
    10034         m_jit.storePtr(TrustedImmPtr(scratchSize), MacroAssembler::Address(storageLengthGPR));
    100359997
    100369998        storageDone.link(&m_jit);
    … …  
    1004810010        addSlowPathGenerator(
    1004910011            slowPathCall(m_jit.jump(), this, operationArrayPushMultiple, resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), baseGPR, bufferGPR, TrustedImm32(elementCount)));
    10050 
    10051         m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), bufferGPR);
    10052         m_jit.storePtr(TrustedImmPtr(nullptr), MacroAssembler::Address(bufferGPR));
    1005310012
    1005410013        base.use();

  • trunk/Source/JavaScriptCore/dfg/DFGThunks.cpp

    r277928 r278875  
    3232#include "DFGJITCode.h"
    3333#include "DFGOSRExit.h"
     34#include "DFGOSRExitCompilerCommon.h"
    3435#include "FPRInfo.h"
    3536#include "GPRInfo.h"
    3637#include "LinkBuffer.h"
    3738#include "MacroAssembler.h"
    38 #include "DFGOSRExitCompilerCommon.h"
    3939
    4040namespace JSC { namespace DFG {
    … …  
    6363    }
    6464   
    65     // Tell GC mark phase how much of the scratch buffer is active during call.
    66     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::regT0);
    67     jit.storePtr(MacroAssembler::TrustedImmPtr(scratchSize), MacroAssembler::Address(GPRInfo::regT0));
    68 
    6965    // Set up one argument.
    7066    jit.move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
    … …  
    7268
    7369    MacroAssembler::Call functionCall = jit.call(OperationPtrTag);
    74 
    75     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::regT0);
    76     jit.storePtr(MacroAssembler::TrustedImmPtr(nullptr), MacroAssembler::Address(GPRInfo::regT0));
    7770
    7871    for (unsigned i = 0; i < FPRInfo::numberOfRegisters; ++i) {

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

    r278696 r278875  
    61636163            ASSERT(scratchSize);
    61646164            ScratchBuffer* scratchBuffer = vm().scratchBufferForSize(scratchSize);
    6165             m_out.storePtr(m_out.constIntPtr(scratchSize), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    61666165            ValueFromBlock slowBufferResult = m_out.anchor(m_out.constIntPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())));
    61676166            m_out.jump(setup);
    … …  
    61936192                operation = &operationArrayPushDoubleMultiple;
    61946193            ValueFromBlock slowResult = m_out.anchor(vmCall(Int64, operation, weakPointer(globalObject), base, buffer, m_out.constInt32(elementCount)));
    6195             m_out.storePtr(m_out.constIntPtr(0), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    61966194            m_out.jump(continuation);
    61976195
    … …  
    62696267            ASSERT(scratchSize);
    62706268            ScratchBuffer* scratchBuffer = vm().scratchBufferForSize(scratchSize);
    6271             m_out.storePtr(m_out.constIntPtr(scratchSize), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    62726269            ValueFromBlock slowBufferResult = m_out.anchor(m_out.constIntPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())));
    62736270            m_out.jump(setup);
    … …  
    62876284            m_out.appendTo(slowCallPath, continuation);
    62886285            ValueFromBlock slowResult = m_out.anchor(vmCall(Int64, operationArrayPushMultiple, weakPointer(globalObject), base, buffer, m_out.constInt32(elementCount)));
    6289             m_out.storePtr(m_out.constIntPtr(0), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    62906286            m_out.jump(continuation);
    62916287
    … …  
    72797275            m_out.store64(valueToStore, m_out.absolute(buffer + operandIndex));
    72807276        }
    7281        
    7282         m_out.storePtr(
    7283             m_out.constIntPtr(scratchSize), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    7284        
     7277
    72857278        LValue result = vmCall(
    72867279            Int64, operationNewArray, weakPointer(globalObject),
    72877280            weakStructure(structure), m_out.constIntPtr(buffer),
    72887281            m_out.constIntPtr(m_node->numChildren()));
    7289        
    7290         m_out.storePtr(m_out.intPtrZero, m_out.absolute(scratchBuffer->addressOfActiveLength()));
    7291        
     7282
    72927283        setJSValue(result);
    72937284    }
    … …  
    75257516        }
    75267517
    7527         m_out.storePtr(m_out.constIntPtr(scratchSize), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    75287518        LValue result = vmCall(Int64, operationNewArrayWithSpreadSlow, weakPointer(globalObject), m_out.constIntPtr(buffer), m_out.constInt32(m_node->numChildren()));
    7529         m_out.storePtr(m_out.constIntPtr(0), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    75307519
    75317520        setJSValue(result);

  • trunk/Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp

    r277928 r278875  
    524524
    525525    VM& vm = callFrame->deprecatedVM();
     526    // Don't need an ActiveScratchBufferScope here because we DeferGCForAWhile below.
    526527
    527528    if constexpr (validateDFGDoesGC) {

  • trunk/Source/JavaScriptCore/ftl/FTLOperations.cpp

    r278351 r278875  
    11/*
    2  * Copyright (C) 2014-2020 Apple Inc. All rights reserved.
     2 * Copyright (C) 2014-2021 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    731731{
    732732    VM& vm = callFrame->deprecatedVM();
     733    // Don't need an ActiveScratchBufferScope here because we DeferGCForAWhile.
    733734
    734735    // We cannot GC. We've got pointers in evil places.

  • trunk/Source/JavaScriptCore/ftl/FTLThunks.cpp

    r278656 r278875  
    7777   
    7878    saveAllRegisters(jit, buffer);
    79    
    80     // Tell GC mark phase how much of the scratch buffer is active during call.
    81     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::nonArgGPR1);
    82     jit.storePtr(MacroAssembler::TrustedImmPtr(requiredScratchMemorySizeInBytes()), GPRInfo::nonArgGPR1);
    8379
    8480    jit.loadPtr(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
    … …  
    9591   
    9692    jit.move(GPRInfo::returnValueGPR, GPRInfo::regT0);
    97    
    98     // Make sure we tell the GC that we're not using the scratch buffer anymore.
    99     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::regT1);
    100     jit.storePtr(MacroAssembler::TrustedImmPtr(nullptr), GPRInfo::regT1);
    101    
     93
    10294    // Prepare for tail call.
    10395    while (numberOfRequiredPops--)

  • trunk/Source/JavaScriptCore/jit/AssemblyHelpers.cpp

    r278656 r278875  
    967967    }
    968968
    969     // Tell GC mark phase how much of the scratch buffer is active during call.
    970     move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::regT0);
    971     storePtr(TrustedImmPtr(scratchSize), GPRInfo::regT0);
    972 
    973969#if CPU(X86_64) || CPU(ARM_THUMB2) || CPU(ARM64) || CPU(MIPS)
    974970    move(TrustedImmPtr(buffer), GPRInfo::argumentGPR2);
    … …  
    982978    move(TrustedImmPtr(tagCFunctionPtr<OperationPtrTag>(function)), scratch);
    983979    call(scratch, OperationPtrTag);
    984 
    985     move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::regT0);
    986     storePtr(TrustedImmPtr(nullptr), GPRInfo::regT0);
    987980
    988981    for (unsigned i = 0; i < FPRInfo::numberOfRegisters; ++i) {

  • trunk/Source/JavaScriptCore/jit/ScratchRegisterAllocator.cpp

    r261755 r278875  
    162162}
    163163
    164 void ScratchRegisterAllocator::preserveUsedRegistersToScratchBufferForCall(MacroAssembler& jit, ScratchBuffer* scratchBuffer, GPRReg scratchGPR)
    165 {
    166     RegisterSet usedRegisters = usedRegistersForCall();
    167     if (!usedRegisters.numberOfSetRegisters())
    168         return;
    169    
    170     unsigned count = 0;
    171     for (GPRReg reg = MacroAssembler::firstRegister(); reg <= MacroAssembler::lastRegister(); reg = MacroAssembler::nextRegister(reg)) {
    172         if (usedRegisters.get(reg)) {
    173             jit.storePtr(reg, static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer()) + count);
    174             count++;
    175         }
    176         if (GPRInfo::toIndex(reg) != GPRInfo::InvalidIndex
    177             && scratchGPR == InvalidGPRReg
    178             && !m_lockedRegisters.get(reg) && !m_scratchRegisters.get(reg))
    179             scratchGPR = reg;
    180     }
    181     RELEASE_ASSERT(scratchGPR != InvalidGPRReg);
    182     for (FPRReg reg = MacroAssembler::firstFPRegister(); reg <= MacroAssembler::lastFPRegister(); reg = MacroAssembler::nextFPRegister(reg)) {
    183         if (usedRegisters.get(reg)) {
    184             jit.move(MacroAssembler::TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer()) + count), scratchGPR);
    185             count++;
    186             jit.storeDouble(reg, scratchGPR);
    187         }
    188     }
    189     RELEASE_ASSERT(count * sizeof(JSValue) == desiredScratchBufferSizeForCall());
    190    
    191     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), scratchGPR);
    192     jit.storePtr(MacroAssembler::TrustedImmPtr(static_cast<size_t>(count * sizeof(JSValue))), scratchGPR);
    193 }
    194 
    195 void ScratchRegisterAllocator::restoreUsedRegistersFromScratchBufferForCall(MacroAssembler& jit, ScratchBuffer* scratchBuffer, GPRReg scratchGPR)
    196 {
    197     RegisterSet usedRegisters = usedRegistersForCall();
    198     if (!usedRegisters.numberOfSetRegisters())
    199         return;
    200    
    201     if (scratchGPR == InvalidGPRReg) {
    202         // Find a scratch register.
    203         for (unsigned i = GPRInfo::numberOfRegisters; i--;) {
    204             if (m_lockedRegisters.getGPRByIndex(i) || m_scratchRegisters.getGPRByIndex(i))
    205                 continue;
    206             scratchGPR = GPRInfo::toRegister(i);
    207             break;
    208         }
    209     }
    210     RELEASE_ASSERT(scratchGPR != InvalidGPRReg);
    211    
    212     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), scratchGPR);
    213     jit.storePtr(MacroAssembler::TrustedImmPtr(nullptr), scratchGPR);
    214 
    215     // Restore double registers first.
    216     unsigned count = usedRegisters.numberOfSetGPRs();
    217     for (FPRReg reg = MacroAssembler::firstFPRegister(); reg <= MacroAssembler::lastFPRegister(); reg = MacroAssembler::nextFPRegister(reg)) {
    218         if (usedRegisters.get(reg)) {
    219             jit.move(MacroAssembler::TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer()) + (count++)), scratchGPR);
    220             jit.loadDouble(scratchGPR, reg);
    221         }
    222     }
    223        
    224     count = 0;
    225     for (GPRReg reg = MacroAssembler::firstRegister(); reg <= MacroAssembler::lastRegister(); reg = MacroAssembler::nextRegister(reg)) {
    226         if (usedRegisters.get(reg))
    227             jit.loadPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer()) + (count++), reg);
    228     }
    229 }
    230 
    231164unsigned ScratchRegisterAllocator::preserveRegistersToStackForCall(MacroAssembler& jit, const RegisterSet& usedRegisters, unsigned extraBytesAtTopOfStack)
    232165{

  • trunk/Source/JavaScriptCore/jit/ScratchRegisterAllocator.h

    r206525 r278875  
    9292    unsigned desiredScratchBufferSizeForCall() const;
    9393   
    94     void preserveUsedRegistersToScratchBufferForCall(MacroAssembler& jit, ScratchBuffer* scratchBuffer, GPRReg scratchGPR = InvalidGPRReg);
    95     void restoreUsedRegistersFromScratchBufferForCall(MacroAssembler& jit, ScratchBuffer* scratchBuffer, GPRReg scratchGPR = InvalidGPRReg);
    96 
    9794    static unsigned preserveRegistersToStackForCall(MacroAssembler& jit, const RegisterSet& usedRegisters, unsigned extraPaddingInBytes);
    9895    static void restoreRegistersFromStackForCall(MacroAssembler& jit, const RegisterSet& usedRegisters, const RegisterSet& ignore, unsigned numberOfStackBytesUsedForRegisterPreservation, unsigned extraPaddingInBytes);

  • trunk/Source/JavaScriptCore/runtime/VM.h

    r278656 r278875  
    280280#pragma warning(pop)
    281281#endif
     282
     283class ActiveScratchBufferScope {
     284public:
     285    ActiveScratchBufferScope(VM&, size_t activeScratchBufferSizeInJSValues);
     286    ~ActiveScratchBufferScope();
     287
     288private:
     289    ScratchBuffer* m_scratchBuffer;
     290};
    282291
    283292class VM : public ThreadSafeRefCounted<VM>, public DoublyLinkedListNode<VM> {

  • trunk/Source/JavaScriptCore/runtime/VMInlines.h

    r252177 r278875  
    11/*
    2  * Copyright (C) 2015-2019 Apple Inc. All rights reserved.
     2 * Copyright (C) 2015-2021 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    3232
    3333namespace JSC {
    34    
     34
     35inline ActiveScratchBufferScope::ActiveScratchBufferScope(VM& vm, size_t activeScratchBufferSizeInJSValues)
     36    : m_scratchBuffer(vm.scratchBufferForSize(activeScratchBufferSizeInJSValues * sizeof(EncodedJSValue)))
     37{
     38    // Tell GC mark phase how much of the scratch buffer is active during the call operation this scope is used in.
     39    if (m_scratchBuffer)
     40        m_scratchBuffer->u.m_activeLength = activeScratchBufferSizeInJSValues * sizeof(EncodedJSValue);
     41}
     42
     43inline ActiveScratchBufferScope::~ActiveScratchBufferScope()
     44{
     45    // Tell the GC that we're not using the scratch buffer anymore.
     46    if (m_scratchBuffer)
     47        m_scratchBuffer->u.m_activeLength = 0;
     48}
     49
    3550bool VM::ensureStackCapacityFor(Register* newTopOfStack)
    3651{
Note: See TracChangeset for help on using the changeset viewer.