Changeset 122768 in webkit


Ignore:
Timestamp:
Jul 16, 2012 3:08:21 PM (12 years ago)
Author:
fpizlo@apple.com
Message:

DFG PutById transition should handle storage allocation, and inline it
https://bugs.webkit.org/show_bug.cgi?id=91337

Reviewed by Oliver Hunt.

This enables the patching of DFG PutById to handle the out-of-line storage
allocation case. Furthermore, it inlines out-of-line storage allocation (and
reallocation) into the generated stubs.

To do this, this patch adds the ability to store the relevant register
allocation state (i.e. the set of in-use registers) in the structure stub
info so that the stub generation code can more flexibly select scratch
registers: sometimes it needs none, sometimes one - or sometimes up to
three. Moreover, to make the stub generation register allocation simple and
maintainable, this patch introduces a reusable scratch register allocator
class. This register allocator understands that some registers are in use by
the main path code and so must be spilled as necessary, other registers are
locked for use in the stub itself and so cannot even be spilled, while still
others may be allocated for scratch purposes. A scratch register that is
used must be spilled. If a register is locked, it cannot be used as a
scratch register. If a register is used, it can be used as a scratch
register so long as it is spilled.

This is a sub-1% speed-up on V8 and neutral elsewhere.

  • GNUmakefile.list.am:
  • JavaScriptCore.xcodeproj/project.pbxproj:
  • assembler/MacroAssemblerCodeRef.h:

(FunctionPtr):
(JSC::FunctionPtr::FunctionPtr):

  • bytecode/StructureStubInfo.h:
  • dfg/DFGCCallHelpers.h:

(JSC::DFG::CCallHelpers::setupArgumentsWithExecState):
(CCallHelpers):

  • dfg/DFGGPRInfo.h:
  • dfg/DFGJITCompiler.cpp:

(JSC::DFG::JITCompiler::link):

  • dfg/DFGJITCompiler.h:

(JSC::DFG::PropertyAccessRecord::PropertyAccessRecord):
(PropertyAccessRecord):

  • dfg/DFGOperations.cpp:
  • dfg/DFGOperations.h:
  • dfg/DFGRegisterBank.h:

(JSC::DFG::RegisterBank::isInUse):
(RegisterBank):

  • dfg/DFGRegisterSet.h: Added.

(DFG):
(RegisterSet):
(JSC::DFG::RegisterSet::RegisterSet):
(JSC::DFG::RegisterSet::asPOD):
(JSC::DFG::RegisterSet::copyInfo):
(JSC::DFG::RegisterSet::set):
(JSC::DFG::RegisterSet::setGPRByIndex):
(JSC::DFG::RegisterSet::clear):
(JSC::DFG::RegisterSet::get):
(JSC::DFG::RegisterSet::getGPRByIndex):
(JSC::DFG::RegisterSet::getFreeGPR):
(JSC::DFG::RegisterSet::setFPRByIndex):
(JSC::DFG::RegisterSet::getFPRByIndex):
(JSC::DFG::RegisterSet::setByIndex):
(JSC::DFG::RegisterSet::getByIndex):
(JSC::DFG::RegisterSet::numberOfSetGPRs):
(JSC::DFG::RegisterSet::numberOfSetFPRs):
(JSC::DFG::RegisterSet::numberOfSetRegisters):
(JSC::DFG::RegisterSet::setBit):
(JSC::DFG::RegisterSet::clearBit):
(JSC::DFG::RegisterSet::getBit):

  • dfg/DFGRepatch.cpp:

(JSC::DFG::generateProtoChainAccessStub):
(JSC::DFG::tryCacheGetByID):
(JSC::DFG::tryBuildGetByIDList):
(JSC::DFG::emitPutReplaceStub):
(JSC::DFG::emitPutTransitionStub):
(JSC::DFG::tryCachePutByID):
(JSC::DFG::tryBuildPutByIdList):

  • dfg/DFGScratchRegisterAllocator.h: Added.

(DFG):
(ScratchRegisterAllocator):
(JSC::DFG::ScratchRegisterAllocator::ScratchRegisterAllocator):
(JSC::DFG::ScratchRegisterAllocator::lock):
(JSC::DFG::ScratchRegisterAllocator::allocateScratch):
(JSC::DFG::ScratchRegisterAllocator::allocateScratchGPR):
(JSC::DFG::ScratchRegisterAllocator::allocateScratchFPR):
(JSC::DFG::ScratchRegisterAllocator::didReuseRegisters):
(JSC::DFG::ScratchRegisterAllocator::preserveReusedRegistersByPushing):
(JSC::DFG::ScratchRegisterAllocator::restoreReusedRegistersByPopping):
(JSC::DFG::ScratchRegisterAllocator::desiredScratchBufferSize):
(JSC::DFG::ScratchRegisterAllocator::preserveUsedRegistersToScratchBuffer):
(JSC::DFG::ScratchRegisterAllocator::restoreUsedRegistersFromScratchBuffer):

  • dfg/DFGSpeculativeJIT.h:

(SpeculativeJIT):
(JSC::DFG::SpeculativeJIT::usedRegisters):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::cachedGetById):
(JSC::DFG::SpeculativeJIT::cachedPutById):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::cachedGetById):
(JSC::DFG::SpeculativeJIT::cachedPutById):
(JSC::DFG::SpeculativeJIT::compile):

  • heap/CopiedAllocator.h:

(CopiedAllocator):
(JSC::CopiedAllocator::fastPathShouldSucceed):
(JSC):

Location:
trunk/Source/JavaScriptCore
Files:
17 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/JavaScriptCore/ChangeLog

    r122753 r122768  
     12012-07-15  Filip Pizlo  <fpizlo@apple.com>
     2
     3        DFG PutById transition should handle storage allocation, and inline it
     4        https://bugs.webkit.org/show_bug.cgi?id=91337
     5
     6        Reviewed by Oliver Hunt.
     7
     8        This enables the patching of DFG PutById to handle the out-of-line storage
     9        allocation case. Furthermore, it inlines out-of-line storage allocation (and
     10        reallocation) into the generated stubs. 
     11       
     12        To do this, this patch adds the ability to store the relevant register
     13        allocation state (i.e. the set of in-use registers) in the structure stub
     14        info so that the stub generation code can more flexibly select scratch
     15        registers: sometimes it needs none, sometimes one - or sometimes up to
     16        three. Moreover, to make the stub generation register allocation simple and
     17        maintainable, this patch introduces a reusable scratch register allocator
     18        class. This register allocator understands that some registers are in use by
     19        the main path code and so must be spilled as necessary, other registers are
     20        locked for use in the stub itself and so cannot even be spilled, while still
     21        others may be allocated for scratch purposes. A scratch register that is
     22        used must be spilled. If a register is locked, it cannot be used as a
     23        scratch register. If a register is used, it can be used as a scratch
     24        register so long as it is spilled.
     25       
     26        This is a sub-1% speed-up on V8 and neutral elsewhere.
     27
     28        * GNUmakefile.list.am:
     29        * JavaScriptCore.xcodeproj/project.pbxproj:
     30        * assembler/MacroAssemblerCodeRef.h:
     31        (FunctionPtr):
     32        (JSC::FunctionPtr::FunctionPtr):
     33        * bytecode/StructureStubInfo.h:
     34        * dfg/DFGCCallHelpers.h:
     35        (JSC::DFG::CCallHelpers::setupArgumentsWithExecState):
     36        (CCallHelpers):
     37        * dfg/DFGGPRInfo.h:
     38        * dfg/DFGJITCompiler.cpp:
     39        (JSC::DFG::JITCompiler::link):
     40        * dfg/DFGJITCompiler.h:
     41        (JSC::DFG::PropertyAccessRecord::PropertyAccessRecord):
     42        (PropertyAccessRecord):
     43        * dfg/DFGOperations.cpp:
     44        * dfg/DFGOperations.h:
     45        * dfg/DFGRegisterBank.h:
     46        (JSC::DFG::RegisterBank::isInUse):
     47        (RegisterBank):
     48        * dfg/DFGRegisterSet.h: Added.
     49        (DFG):
     50        (RegisterSet):
     51        (JSC::DFG::RegisterSet::RegisterSet):
     52        (JSC::DFG::RegisterSet::asPOD):
     53        (JSC::DFG::RegisterSet::copyInfo):
     54        (JSC::DFG::RegisterSet::set):
     55        (JSC::DFG::RegisterSet::setGPRByIndex):
     56        (JSC::DFG::RegisterSet::clear):
     57        (JSC::DFG::RegisterSet::get):
     58        (JSC::DFG::RegisterSet::getGPRByIndex):
     59        (JSC::DFG::RegisterSet::getFreeGPR):
     60        (JSC::DFG::RegisterSet::setFPRByIndex):
     61        (JSC::DFG::RegisterSet::getFPRByIndex):
     62        (JSC::DFG::RegisterSet::setByIndex):
     63        (JSC::DFG::RegisterSet::getByIndex):
     64        (JSC::DFG::RegisterSet::numberOfSetGPRs):
     65        (JSC::DFG::RegisterSet::numberOfSetFPRs):
     66        (JSC::DFG::RegisterSet::numberOfSetRegisters):
     67        (JSC::DFG::RegisterSet::setBit):
     68        (JSC::DFG::RegisterSet::clearBit):
     69        (JSC::DFG::RegisterSet::getBit):
     70        * dfg/DFGRepatch.cpp:
     71        (JSC::DFG::generateProtoChainAccessStub):
     72        (JSC::DFG::tryCacheGetByID):
     73        (JSC::DFG::tryBuildGetByIDList):
     74        (JSC::DFG::emitPutReplaceStub):
     75        (JSC::DFG::emitPutTransitionStub):
     76        (JSC::DFG::tryCachePutByID):
     77        (JSC::DFG::tryBuildPutByIdList):
     78        * dfg/DFGScratchRegisterAllocator.h: Added.
     79        (DFG):
     80        (ScratchRegisterAllocator):
     81        (JSC::DFG::ScratchRegisterAllocator::ScratchRegisterAllocator):
     82        (JSC::DFG::ScratchRegisterAllocator::lock):
     83        (JSC::DFG::ScratchRegisterAllocator::allocateScratch):
     84        (JSC::DFG::ScratchRegisterAllocator::allocateScratchGPR):
     85        (JSC::DFG::ScratchRegisterAllocator::allocateScratchFPR):
     86        (JSC::DFG::ScratchRegisterAllocator::didReuseRegisters):
     87        (JSC::DFG::ScratchRegisterAllocator::preserveReusedRegistersByPushing):
     88        (JSC::DFG::ScratchRegisterAllocator::restoreReusedRegistersByPopping):
     89        (JSC::DFG::ScratchRegisterAllocator::desiredScratchBufferSize):
     90        (JSC::DFG::ScratchRegisterAllocator::preserveUsedRegistersToScratchBuffer):
     91        (JSC::DFG::ScratchRegisterAllocator::restoreUsedRegistersFromScratchBuffer):
     92        * dfg/DFGSpeculativeJIT.h:
     93        (SpeculativeJIT):
     94        (JSC::DFG::SpeculativeJIT::usedRegisters):
     95        * dfg/DFGSpeculativeJIT32_64.cpp:
     96        (JSC::DFG::SpeculativeJIT::cachedGetById):
     97        (JSC::DFG::SpeculativeJIT::cachedPutById):
     98        (JSC::DFG::SpeculativeJIT::compile):
     99        * dfg/DFGSpeculativeJIT64.cpp:
     100        (JSC::DFG::SpeculativeJIT::cachedGetById):
     101        (JSC::DFG::SpeculativeJIT::cachedPutById):
     102        (JSC::DFG::SpeculativeJIT::compile):
     103        * heap/CopiedAllocator.h:
     104        (CopiedAllocator):
     105        (JSC::CopiedAllocator::fastPathShouldSucceed):
     106        (JSC):
     107
    11082012-07-16  Patrick Gansterer  <paroga@webkit.org>
    2109
  • trunk/Source/JavaScriptCore/GNUmakefile.list.am

    r122650 r122768  
    217217        Source/JavaScriptCore/dfg/DFGRedundantPhiEliminationPhase.h \
    218218        Source/JavaScriptCore/dfg/DFGRegisterBank.h \
     219        Source/JavaScriptCore/dfg/DFGRegisterSet.h \
    219220        Source/JavaScriptCore/dfg/DFGRepatch.cpp \
    220221        Source/JavaScriptCore/dfg/DFGRepatch.h \
    221222        Source/JavaScriptCore/dfg/DFGScoreBoard.h \
     223        Source/JavaScriptCore/dfg/DFGScratchRegisterAllocator.h \
    222224        Source/JavaScriptCore/dfg/DFGSilentRegisterSavePlan.h \
    223225        Source/JavaScriptCore/dfg/DFGSlowPathGenerator.h \
  • trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

    r122544 r122768  
    151151                0F766D3815AE4A1C008F363E /* StructureStubClearingWatchpoint.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F766D3615AE4A1A008F363E /* StructureStubClearingWatchpoint.cpp */; };
    152152                0F766D3915AE4A1F008F363E /* StructureStubClearingWatchpoint.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F766D3715AE4A1A008F363E /* StructureStubClearingWatchpoint.h */; settings = {ATTRIBUTES = (Private, ); }; };
     153                0F766D4415B2A3C0008F363E /* DFGRegisterSet.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F766D4215B2A3BD008F363E /* DFGRegisterSet.h */; settings = {ATTRIBUTES = (Private, ); }; };
     154                0F766D4615B3701F008F363E /* DFGScratchRegisterAllocator.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F766D4515B3701D008F363E /* DFGScratchRegisterAllocator.h */; settings = {ATTRIBUTES = (Private, ); }; };
    153155                0F7700921402FF3C0078EB39 /* SamplingCounter.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F7700911402FF280078EB39 /* SamplingCounter.cpp */; };
    154156                0F7B294A14C3CD29007C3DB1 /* DFGCCallHelpers.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F7B294814C3CD23007C3DB1 /* DFGCCallHelpers.h */; settings = {ATTRIBUTES = (Private, ); }; };
     
    896898                0F766D3615AE4A1A008F363E /* StructureStubClearingWatchpoint.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = StructureStubClearingWatchpoint.cpp; sourceTree = "<group>"; };
    897899                0F766D3715AE4A1A008F363E /* StructureStubClearingWatchpoint.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StructureStubClearingWatchpoint.h; sourceTree = "<group>"; };
     900                0F766D4215B2A3BD008F363E /* DFGRegisterSet.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGRegisterSet.h; path = dfg/DFGRegisterSet.h; sourceTree = "<group>"; };
     901                0F766D4515B3701D008F363E /* DFGScratchRegisterAllocator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGScratchRegisterAllocator.h; path = dfg/DFGScratchRegisterAllocator.h; sourceTree = "<group>"; };
    898902                0F77008E1402FDD60078EB39 /* SamplingCounter.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SamplingCounter.h; sourceTree = "<group>"; };
    899903                0F7700911402FF280078EB39 /* SamplingCounter.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SamplingCounter.cpp; sourceTree = "<group>"; };
     
    23052309                                0A4337BD1506219B00991C95 /* DFGRedundantPhiEliminationPhase.h */,
    23062310                                86EC9DC11328DF82002B2AD7 /* DFGRegisterBank.h */,
     2311                                0F766D4215B2A3BD008F363E /* DFGRegisterSet.h */,
    23072312                                86BB09BE138E381B0056702F /* DFGRepatch.cpp */,
    23082313                                86BB09BF138E381B0056702F /* DFGRepatch.h */,
    23092314                                86ECA3F9132DF25A002B2AD7 /* DFGScoreBoard.h */,
     2315                                0F766D4515B3701D008F363E /* DFGScratchRegisterAllocator.h */,
    23102316                                0F1E3A65153A21DF000F9456 /* DFGSilentRegisterSavePlan.h */,
    23112317                                0F1E3A501537C2CB000F9456 /* DFGSlowPathGenerator.h */,
     
    28512857                                0F766D3515AE253B008F363E /* JumpReplacementWatchpoint.h in Headers */,
    28522858                                0F766D3915AE4A1F008F363E /* StructureStubClearingWatchpoint.h in Headers */,
     2859                                0F766D4415B2A3C0008F363E /* DFGRegisterSet.h in Headers */,
     2860                                0F766D4615B3701F008F363E /* DFGScratchRegisterAllocator.h in Headers */,
    28532861                        );
    28542862                        runOnlyForDeploymentPostprocessing = 0;
  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerCodeRef.h

    r120786 r122768  
    127127    }
    128128
     129    template<typename returnType, typename argType1, typename argType2, typename argType3, typename argType4, typename argType5>
     130    FunctionPtr(returnType(*value)(argType1, argType2, argType3, argType4, argType5))
     131        : m_value((void*)value)
     132    {
     133        ASSERT_VALID_CODE_POINTER(m_value);
     134    }
     135
    129136// MSVC doesn't seem to treat functions with different calling conventions as
    130137// different types; these methods already defined for fastcall, below.
  • trunk/Source/JavaScriptCore/bytecode/StructureStubInfo.h

    r122544 r122768  
    3232
    3333#include "CodeOrigin.h"
     34#include "DFGRegisterSet.h"
    3435#include "Instruction.h"
    3536#include "JITStubRoutine.h"
     
    213214#endif
    214215                int8_t valueGPR;
    215                 int8_t scratchGPR;
     216                DFG::RegisterSetPOD usedRegisters;
    216217                int32_t deltaCallToDone;
    217218                int32_t deltaCallToStorageLoad;
  • trunk/Source/JavaScriptCore/dfg/DFGCCallHelpers.h

    r122392 r122768  
    230230    }
    231231
     232    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4)
     233    {
     234        resetCallArguments();
     235        addCallArgument(GPRInfo::callFrameRegister);
     236        addCallArgument(arg1);
     237        addCallArgument(arg2);
     238        addCallArgument(arg3);
     239        addCallArgument(arg4);
     240    }
     241
     242    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4, GPRReg arg5)
     243    {
     244        resetCallArguments();
     245        addCallArgument(GPRInfo::callFrameRegister);
     246        addCallArgument(arg1);
     247        addCallArgument(arg2);
     248        addCallArgument(arg3);
     249        addCallArgument(arg4);
     250        addCallArgument(arg5);
     251    }
     252
    232253    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4)
    233254    {
     
    643664    }
    644665
     666    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4)
     667    {
     668        poke(arg4);
     669        setupArgumentsWithExecState(arg1, arg2, arg3);
     670    }
     671
     672    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4, GPRReg arg5)
     673    {
     674        poke(arg5, 1);
     675        poke(arg4);
     676        setupArgumentsWithExecState(arg1, arg2, arg3);
     677    }
     678
    645679    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, TrustedImm32 arg3, TrustedImm32 arg4)
    646680    {
     
    722756
    723757#endif // NUMBER_OF_ARGUMENT_REGISTERS == 4
     758
     759#if NUMBER_OF_ARGUMENT_REGISTERS >= 5
     760    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4)
     761    {
     762        setupTwoStubArgs<GPRInfo::argumentGPR1, GPRInfo::argumentGPR4>(arg1, arg4);
     763        move(arg2, GPRInfo::argumentGPR2);
     764        move(arg3, GPRInfo::argumentGPR3);
     765        move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
     766    }
     767#endif
    724768
    725769    void setupResults(GPRReg destA, GPRReg destB)
  • trunk/Source/JavaScriptCore/dfg/DFGGPRInfo.h

    r122392 r122768  
    2727#define DFGGPRInfo_h
    2828
     29#include <wtf/Platform.h>
     30
    2931#if ENABLE(DFG_JIT)
    3032
    31 #include <assembler/MacroAssembler.h>
    32 #include <dfg/DFGRegisterBank.h>
     33#include "DFGRegisterBank.h"
     34#include "MacroAssembler.h"
    3335
    3436namespace JSC { namespace DFG {
  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp

    r122392 r122768  
    180180        info.patch.dfg.valueGPR = m_propertyAccesses[i].m_valueGPR;
    181181#endif
    182         info.patch.dfg.scratchGPR = m_propertyAccesses[i].m_scratchGPR;
     182        m_propertyAccesses[i].m_usedRegisters.copyInfo(info.patch.dfg.usedRegisters);
    183183        info.patch.dfg.registersFlushed = m_propertyAccesses[i].m_registerMode == PropertyAccessRecord::RegistersFlushed;
    184184    }
  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h

    r122392 r122768  
    3636#include "DFGGraph.h"
    3737#include "DFGRegisterBank.h"
     38#include "DFGRegisterSet.h"
    3839#include "JITCode.h"
    3940#include "LinkBuffer.h"
     
    170171        int8_t baseGPR,
    171172        int8_t valueGPR,
    172         int8_t scratchGPR,
     173        const RegisterSet& usedRegisters,
    173174        RegisterMode registerMode = RegistersInUse)
    174175#elif USE(JSVALUE32_64)
     
    185186        int8_t valueTagGPR,
    186187        int8_t valueGPR,
    187         int8_t scratchGPR,
     188        const RegisterSet& usedRegisters,
    188189        RegisterMode registerMode = RegistersInUse)
    189190#endif
     
    205206#endif
    206207        , m_valueGPR(valueGPR)
    207         , m_scratchGPR(scratchGPR)
     208        , m_usedRegisters(usedRegisters)
    208209        , m_registerMode(registerMode)
    209210    {
     
    227228#endif
    228229    int8_t m_valueGPR;
    229     int8_t m_scratchGPR;
     230    RegisterSet m_usedRegisters;
    230231    RegisterMode m_registerMode;
    231232};
  • trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp

    r122392 r122768  
    12311231{
    12321232    return jsIsFunctionType(JSValue::decode(value));
     1233}
     1234
     1235void DFG_OPERATION operationReallocateStorageAndFinishPut(ExecState* exec, JSObject* base, Structure* structure, PropertyOffset offset, EncodedJSValue value)
     1236{
     1237    JSGlobalData& globalData = exec->globalData();
     1238    ASSERT(structure->outOfLineCapacity() > base->structure()->outOfLineCapacity());
     1239    ASSERT(!globalData.heap.storageAllocator().fastPathShouldSucceed(structure->outOfLineCapacity() * sizeof(JSValue)));
     1240    base->setStructureAndReallocateStorageIfNecessary(globalData, structure);
     1241    base->putDirectOffset(globalData, offset, JSValue::decode(value));
    12331242}
    12341243
  • trunk/Source/JavaScriptCore/dfg/DFGOperations.h

    r121280 r122768  
    181181size_t DFG_OPERATION operationIsObject(EncodedJSValue) WTF_INTERNAL;
    182182size_t DFG_OPERATION operationIsFunction(EncodedJSValue) WTF_INTERNAL;
     183void DFG_OPERATION operationReallocateStorageAndFinishPut(ExecState*, JSObject*, Structure*, PropertyOffset, EncodedJSValue) WTF_INTERNAL;
    183184
    184185// This method is used to lookup an exception hander, keyed by faultLocation, which is
  • trunk/Source/JavaScriptCore/dfg/DFGRegisterBank.h

    r107499 r122768  
    227227    }
    228228   
     229    bool isInUse(RegID reg) const
     230    {
     231        return isLocked(reg) || name(reg) != InvalidVirtualRegister;
     232    }
     233   
    229234#ifndef NDEBUG
    230235    void dump()
  • trunk/Source/JavaScriptCore/dfg/DFGRepatch.cpp

    r122595 r122768  
    3030
    3131#include "DFGCCallHelpers.h"
     32#include "DFGScratchRegisterAllocator.h"
    3233#include "DFGSpeculativeJIT.h"
    3334#include "DFGThunks.h"
     
    162163#endif
    163164    GPRReg resultGPR = static_cast<GPRReg>(stubInfo.patch.dfg.valueGPR);
    164     GPRReg scratchGPR = static_cast<GPRReg>(stubInfo.patch.dfg.scratchGPR);
     165    GPRReg scratchGPR = RegisterSet(stubInfo.patch.dfg.usedRegisters).getFreeGPR();
    165166    bool needToRestoreScratch = false;
    166167   
     
    232233#endif
    233234        GPRReg resultGPR = static_cast<GPRReg>(stubInfo.patch.dfg.valueGPR);
    234         GPRReg scratchGPR = static_cast<GPRReg>(stubInfo.patch.dfg.scratchGPR);
     235        GPRReg scratchGPR = RegisterSet(stubInfo.patch.dfg.usedRegisters).getFreeGPR();
    235236        bool needToRestoreScratch = false;
    236237       
     
    385386#endif
    386387        GPRReg resultGPR = static_cast<GPRReg>(stubInfo.patch.dfg.valueGPR);
    387         GPRReg scratchGPR = static_cast<GPRReg>(stubInfo.patch.dfg.scratchGPR);
     388        GPRReg scratchGPR = RegisterSet(stubInfo.patch.dfg.usedRegisters).getFreeGPR();
    388389       
    389390        CCallHelpers stubJit(globalData, codeBlock);
     
    630631#endif
    631632    GPRReg valueGPR = static_cast<GPRReg>(stubInfo.patch.dfg.valueGPR);
    632     GPRReg scratchGPR = static_cast<GPRReg>(stubInfo.patch.dfg.scratchGPR);
     633    GPRReg scratchGPR = RegisterSet(stubInfo.patch.dfg.usedRegisters).getFreeGPR();
    633634    bool needToRestoreScratch = false;
    634635#if ENABLE(GGC) || ENABLE(WRITE_BARRIER_PROFILING)
     
    723724#endif
    724725    GPRReg valueGPR = static_cast<GPRReg>(stubInfo.patch.dfg.valueGPR);
    725     GPRReg scratchGPR = static_cast<GPRReg>(stubInfo.patch.dfg.scratchGPR);
    726     bool needToRestoreScratch = false;
    727            
    728     ASSERT(scratchGPR != baseGPR);
    729            
    730     MacroAssembler stubJit;
    731            
     726   
     727    ScratchRegisterAllocator allocator(stubInfo.patch.dfg.usedRegisters);
     728    allocator.lock(baseGPR);
     729#if USE(JSVALUE32_64)
     730    allocator.lock(valueTagGPR);
     731#endif
     732    allocator.lock(valueGPR);
     733   
     734    CCallHelpers stubJit(globalData);
     735           
     736    GPRReg scratchGPR1 = allocator.allocateScratchGPR();
     737    ASSERT(scratchGPR1 != baseGPR);
     738    ASSERT(scratchGPR1 != valueGPR);
     739   
     740    bool needSecondScratch = false;
     741    bool needThirdScratch = false;
     742#if ENABLE(GGC) || ENABLE(WRITE_BARRIER_PROFILING)
     743    needSecondScratch = true;
     744#endif
     745    if (structure->outOfLineCapacity() != oldStructure->outOfLineCapacity()
     746        && oldStructure->outOfLineCapacity()) {
     747        needSecondScratch = true;
     748        needThirdScratch = true;
     749    }
     750
     751    GPRReg scratchGPR2;
     752    if (needSecondScratch) {
     753        scratchGPR2 = allocator.allocateScratchGPR();
     754        ASSERT(scratchGPR2 != baseGPR);
     755        ASSERT(scratchGPR2 != valueGPR);
     756        ASSERT(scratchGPR2 != scratchGPR1);
     757    } else
     758        scratchGPR2 = InvalidGPRReg;
     759    GPRReg scratchGPR3;
     760    if (needThirdScratch) {
     761        scratchGPR3 = allocator.allocateScratchGPR();
     762        ASSERT(scratchGPR3 != baseGPR);
     763        ASSERT(scratchGPR3 != valueGPR);
     764        ASSERT(scratchGPR3 != scratchGPR1);
     765        ASSERT(scratchGPR3 != scratchGPR2);
     766    } else
     767        scratchGPR3 = InvalidGPRReg;
     768           
     769    allocator.preserveReusedRegistersByPushing(stubJit);
     770
    732771    MacroAssembler::JumpList failureCases;
    733772           
    734     if (scratchGPR == InvalidGPRReg) {
    735         scratchGPR = SpeculativeJIT::selectScratchGPR(baseGPR, valueGPR);
    736         stubJit.push(scratchGPR);
    737         needToRestoreScratch = true;
    738     }
    739    
    740773    ASSERT(oldStructure->transitionWatchpointSetHasBeenInvalidated());
    741774   
     
    744777    addStructureTransitionCheck(
    745778        oldStructure->storedPrototype(), exec->codeBlock(), stubInfo, stubJit, failureCases,
    746         scratchGPR);
     779        scratchGPR1);
    747780           
    748781    if (putKind == NotDirect) {
     
    750783            addStructureTransitionCheck(
    751784                (*it)->storedPrototype(), exec->codeBlock(), stubInfo, stubJit, failureCases,
    752                 scratchGPR);
     785                scratchGPR1);
    753786        }
    754787    }
    755788
    756789#if ENABLE(GGC) || ENABLE(WRITE_BARRIER_PROFILING)
     790    ASSERT(needSecondScratch);
     791    ASSERT(scratchGPR2 != InvalidGPRReg);
    757792    // Must always emit this write barrier as the structure transition itself requires it
    758     GPRReg scratch2 = SpeculativeJIT::selectScratchGPR(baseGPR, valueGPR, scratchGPR);
    759     stubJit.push(scratch2);
    760     SpeculativeJIT::writeBarrier(stubJit, baseGPR, scratchGPR, scratch2, WriteBarrierForPropertyAccess);
    761     stubJit.pop(scratch2);
    762 #endif
     793    SpeculativeJIT::writeBarrier(stubJit, baseGPR, scratchGPR1, scratchGPR2, WriteBarrierForPropertyAccess);
     794#endif
     795   
     796    MacroAssembler::JumpList slowPath;
     797   
     798    bool scratchGPR1HasStorage = false;
     799   
     800    if (structure->outOfLineCapacity() != oldStructure->outOfLineCapacity()) {
     801        size_t newSize = structure->outOfLineCapacity() * sizeof(JSValue);
     802        CopiedAllocator* copiedAllocator = &globalData->heap.storageAllocator();
     803       
     804        if (!oldStructure->outOfLineCapacity()) {
     805            stubJit.loadPtr(&copiedAllocator->m_currentRemaining, scratchGPR1);
     806            slowPath.append(stubJit.branchSubPtr(MacroAssembler::Signed, MacroAssembler::TrustedImm32(newSize), scratchGPR1));
     807            stubJit.storePtr(scratchGPR1, &copiedAllocator->m_currentRemaining);
     808            stubJit.negPtr(scratchGPR1);
     809            stubJit.addPtr(MacroAssembler::AbsoluteAddress(&copiedAllocator->m_currentPayloadEnd), scratchGPR1);
     810            stubJit.subPtr(MacroAssembler::TrustedImm32(newSize), scratchGPR1);
     811        } else {
     812            size_t oldSize = oldStructure->outOfLineCapacity() * sizeof(JSValue);
     813            ASSERT(newSize > oldSize);
     814           
     815            // Optimistically assume that the old storage was the very last thing
     816            // allocated.
     817            stubJit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::offsetOfOutOfLineStorage()), scratchGPR3);
     818            stubJit.loadPtr(&copiedAllocator->m_currentPayloadEnd, scratchGPR2);
     819            stubJit.loadPtr(&copiedAllocator->m_currentRemaining, scratchGPR1);
     820            stubJit.subPtr(scratchGPR1, scratchGPR2);
     821            stubJit.subPtr(MacroAssembler::TrustedImm32(oldSize), scratchGPR2);
     822            MacroAssembler::Jump needFullRealloc =
     823                stubJit.branchPtr(MacroAssembler::NotEqual, scratchGPR2, scratchGPR3);
     824            slowPath.append(stubJit.branchSubPtr(MacroAssembler::Signed, MacroAssembler::TrustedImm32(newSize - oldSize), scratchGPR1));
     825            stubJit.storePtr(scratchGPR1, &copiedAllocator->m_currentRemaining);
     826            stubJit.move(scratchGPR2, scratchGPR1);
     827            MacroAssembler::Jump doneRealloc = stubJit.jump();
     828           
     829            needFullRealloc.link(&stubJit);
     830            slowPath.append(stubJit.branchSubPtr(MacroAssembler::Signed, MacroAssembler::TrustedImm32(newSize), scratchGPR1));
     831            stubJit.storePtr(scratchGPR1, &copiedAllocator->m_currentRemaining);
     832            stubJit.negPtr(scratchGPR1);
     833            stubJit.addPtr(MacroAssembler::AbsoluteAddress(&copiedAllocator->m_currentPayloadEnd), scratchGPR1);
     834            stubJit.subPtr(MacroAssembler::TrustedImm32(newSize), scratchGPR1);
     835            // We have scratchGPR1 = new storage, scratchGPR3 = old storage, scratchGPR2 = available
     836            for (size_t offset = 0; offset < oldSize; offset += sizeof(JSValue)) {
     837                stubJit.loadPtr(MacroAssembler::Address(scratchGPR3, offset), scratchGPR2);
     838                stubJit.storePtr(scratchGPR2, MacroAssembler::Address(scratchGPR1, offset));
     839            }
     840           
     841            doneRealloc.link(&stubJit);
     842        }
     843       
     844        stubJit.storePtr(scratchGPR1, MacroAssembler::Address(baseGPR, JSObject::offsetOfOutOfLineStorage()));
     845        scratchGPR1HasStorage = true;
     846    }
    763847
    764848    stubJit.storePtr(MacroAssembler::TrustedImmPtr(structure), MacroAssembler::Address(baseGPR, JSCell::structureOffset()));
     
    767851        stubJit.storePtr(valueGPR, MacroAssembler::Address(baseGPR, JSObject::offsetOfInlineStorage() + offsetInInlineStorage(slot.cachedOffset()) * sizeof(JSValue)));
    768852    else {
    769         stubJit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::offsetOfOutOfLineStorage()), scratchGPR);
    770         stubJit.storePtr(valueGPR, MacroAssembler::Address(scratchGPR, offsetInOutOfLineStorage(slot.cachedOffset()) * sizeof(JSValue)));
     853        if (!scratchGPR1HasStorage)
     854            stubJit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::offsetOfOutOfLineStorage()), scratchGPR1);
     855        stubJit.storePtr(valueGPR, MacroAssembler::Address(scratchGPR1, offsetInOutOfLineStorage(slot.cachedOffset()) * sizeof(JSValue)));
    771856    }
    772857#elif USE(JSVALUE32_64)
     
    775860        stubJit.store32(valueTagGPR, MacroAssembler::Address(baseGPR, JSObject::offsetOfInlineStorage() + offsetInInlineStorage(slot.cachedOffset()) * sizeof(JSValue) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag)));
    776861    } else {
    777         stubJit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::offsetOfOutOfLineStorage()), scratchGPR);
    778         stubJit.store32(valueGPR, MacroAssembler::Address(scratchGPR, offsetInOutOfLineStorage(slot.cachedOffset()) * sizeof(JSValue) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload)));
    779         stubJit.store32(valueTagGPR, MacroAssembler::Address(scratchGPR, offsetInOutOfLineStorage(slot.cachedOffset()) * sizeof(JSValue) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag)));
     862        if (!scratchGPR1HasStorage)
     863            stubJit.loadPtr(MacroAssembler::Address(baseGPR, JSObject::offsetOfOutOfLineStorage()), scratchGPR1);
     864        stubJit.store32(valueGPR, MacroAssembler::Address(scratchGPR1, offsetInOutOfLineStorage(slot.cachedOffset()) * sizeof(JSValue) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload)));
     865        stubJit.store32(valueTagGPR, MacroAssembler::Address(scratchGPR1, offsetInOutOfLineStorage(slot.cachedOffset()) * sizeof(JSValue) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag)));
    780866    }
    781867#endif
     
    784870    MacroAssembler::Jump failure;
    785871           
    786     if (needToRestoreScratch) {
    787         stubJit.pop(scratchGPR);
     872    if (allocator.didReuseRegisters()) {
     873        allocator.restoreReusedRegistersByPopping(stubJit);
    788874        success = stubJit.jump();
    789875
    790876        failureCases.link(&stubJit);
    791         stubJit.pop(scratchGPR);
     877        allocator.restoreReusedRegistersByPopping(stubJit);
    792878        failure = stubJit.jump();
    793879    } else
    794880        success = stubJit.jump();
    795            
     881   
     882    MacroAssembler::Call operationCall;
     883    MacroAssembler::Jump successInSlowPath;
     884   
     885    if (structure->outOfLineCapacity() != oldStructure->outOfLineCapacity()) {
     886        slowPath.link(&stubJit);
     887       
     888        allocator.restoreReusedRegistersByPopping(stubJit);
     889        ScratchBuffer* scratchBuffer = globalData->scratchBufferForSize(allocator.desiredScratchBufferSize());
     890        allocator.preserveUsedRegistersToScratchBuffer(stubJit, scratchBuffer, scratchGPR1);
     891#if USE(JSVALUE64)
     892        stubJit.setupArgumentsWithExecState(baseGPR, MacroAssembler::TrustedImmPtr(structure), MacroAssembler::TrustedImm32(slot.cachedOffset()), valueGPR);
     893#else
     894        stubJit.setupArgumentsWithExecState(baseGPR, MacroAssembler::TrustedImmPtr(structure), MacroAssembler::TrustedImm32(slot.cachedOffset()), valueTagGPR, valueGPR);
     895#endif
     896        operationCall = stubJit.call();
     897        allocator.restoreUsedRegistersFromScratchBuffer(stubJit, scratchBuffer, scratchGPR1);
     898        successInSlowPath = stubJit.jump();
     899    }
     900   
    796901    LinkBuffer patchBuffer(*globalData, &stubJit, exec->codeBlock());
    797902    patchBuffer.link(success, stubInfo.callReturnLocation.labelAtOffset(stubInfo.patch.dfg.deltaCallToDone));
    798     if (needToRestoreScratch)
     903    if (allocator.didReuseRegisters())
    799904        patchBuffer.link(failure, failureLabel);
    800905    else
    801906        patchBuffer.link(failureCases, failureLabel);
    802    
    803     stubRoutine = FINALIZE_CODE_FOR_STUB(
    804         patchBuffer,
    805         ("DFG PutById transition stub for CodeBlock %p, return point %p",
    806          exec->codeBlock(), stubInfo.callReturnLocation.labelAtOffset(
    807              stubInfo.patch.dfg.deltaCallToDone).executableAddress()));
     907    if (structure->outOfLineCapacity() != oldStructure->outOfLineCapacity()) {
     908        patchBuffer.link(operationCall, operationReallocateStorageAndFinishPut);
     909        patchBuffer.link(successInSlowPath, stubInfo.callReturnLocation.labelAtOffset(stubInfo.patch.dfg.deltaCallToDone));
     910    }
     911   
     912    stubRoutine =
     913        createJITStubRoutine(
     914            FINALIZE_CODE(
     915                patchBuffer,
     916                ("DFG PutById transition stub for CodeBlock %p, return point %p",
     917                 exec->codeBlock(), stubInfo.callReturnLocation.labelAtOffset(
     918                     stubInfo.patch.dfg.deltaCallToDone).executableAddress())),
     919            *globalData,
     920            exec->codeBlock()->ownerExecutable(),
     921            structure->outOfLineCapacity() != oldStructure->outOfLineCapacity(),
     922            structure);
    808923}
    809924
     
    830945                return false;
    831946           
    832             // skip optimizing the case where we need a realloc
    833             if (oldStructure->outOfLineCapacity() != structure->outOfLineCapacity())
     947            // Skip optimizing the case where we need a realloc, if we don't have
     948            // enough registers to make it happen.
     949            if (GPRInfo::numberOfRegisters < 6
     950                && oldStructure->outOfLineCapacity() != structure->outOfLineCapacity()
     951                && oldStructure->outOfLineCapacity())
    834952                return false;
    835953           
     
    8931011                return false;
    8941012           
    895             // skip optimizing the case where we need a realloc
    896             if (oldStructure->outOfLineCapacity() != structure->outOfLineCapacity())
     1013            // Skip optimizing the case where we need a realloc, if we don't have
     1014            // enough registers to make it happen.
     1015            if (GPRInfo::numberOfRegisters < 6
     1016                && oldStructure->outOfLineCapacity() != structure->outOfLineCapacity()
     1017                && oldStructure->outOfLineCapacity())
    8971018                return false;
    8981019           
  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

    r122392 r122768  
    272272    {
    273273        use(nodeUse.index());
     274    }
     275   
     276    RegisterSet usedRegisters()
     277    {
     278        RegisterSet result;
     279        for (unsigned i = GPRInfo::numberOfRegisters; i--;) {
     280            GPRReg gpr = GPRInfo::toRegister(i);
     281            if (m_gprs.isInUse(gpr))
     282                result.set(gpr);
     283        }
     284        for (unsigned i = FPRInfo::numberOfRegisters; i--;) {
     285            FPRReg fpr = FPRInfo::toRegister(i);
     286            if (m_fprs.isInUse(fpr))
     287                result.set(fpr);
     288        }
     289        return result;
    274290    }
    275291
     
    943959
    944960#if USE(JSVALUE64)
    945     void cachedGetById(CodeOrigin, GPRReg baseGPR, GPRReg resultGPR, GPRReg scratchGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill);
     961    void cachedGetById(CodeOrigin, GPRReg baseGPR, GPRReg resultGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill);
    946962    void cachedPutById(CodeOrigin, GPRReg base, GPRReg value, Edge valueUse, GPRReg scratchGPR, unsigned identifierNumber, PutKind, JITCompiler::Jump slowPathTarget = JITCompiler::Jump());
    947963#elif USE(JSVALUE32_64)
    948     void cachedGetById(CodeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, GPRReg scratchGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill);
     964    void cachedGetById(CodeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill);
    949965    void cachedPutById(CodeOrigin, GPRReg basePayloadGPR, GPRReg valueTagGPR, GPRReg valuePayloadGPR, Edge valueUse, GPRReg scratchGPR, unsigned identifierNumber, PutKind, JITCompiler::Jump slowPathTarget = JITCompiler::Jump());
    950966#endif
  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp

    r122678 r122768  
    504504}
    505505
    506 void SpeculativeJIT::cachedGetById(CodeOrigin codeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, GPRReg scratchGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode)
     506void SpeculativeJIT::cachedGetById(CodeOrigin codeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode)
    507507{
    508508    JITCompiler::DataLabelPtr structureToCompare;
     
    554554            tagLoadWithPatch, payloadLoadWithPatch, slowPath.get(), doneLabel,
    555555            safeCast<int8_t>(basePayloadGPR), safeCast<int8_t>(resultTagGPR),
    556             safeCast<int8_t>(resultPayloadGPR), safeCast<int8_t>(scratchGPR),
     556            safeCast<int8_t>(resultPayloadGPR), usedRegisters(),
    557557            spillMode == NeedToSpill ? PropertyAccessRecord::RegistersInUse : PropertyAccessRecord::RegistersFlushed));
    558558    addSlowPathGenerator(slowPath.release());
     
    596596            basePayloadGPR, identifier(identifierNumber));
    597597    }
     598    RegisterSet currentlyUsedRegisters = usedRegisters();
     599    currentlyUsedRegisters.clear(scratchGPR);
     600    ASSERT(currentlyUsedRegisters.get(basePayloadGPR));
     601    ASSERT(currentlyUsedRegisters.get(valueTagGPR));
     602    ASSERT(currentlyUsedRegisters.get(valuePayloadGPR));
    598603    m_jit.addPropertyAccess(
    599604        PropertyAccessRecord(
     
    603608            slowPath.get(), doneLabel, safeCast<int8_t>(basePayloadGPR),
    604609            safeCast<int8_t>(valueTagGPR), safeCast<int8_t>(valuePayloadGPR),
    605             safeCast<int8_t>(scratchGPR)));
     610            usedRegisters()));
    606611    addSlowPathGenerator(slowPath.release());
    607612}
     
    32993304            GPRReg resultTagGPR = resultTag.gpr();
    33003305            GPRReg resultPayloadGPR = resultPayload.gpr();
    3301             GPRReg scratchGPR;
    3302            
    3303             if (resultTagGPR == baseGPR)
    3304                 scratchGPR = resultPayloadGPR;
    3305             else
    3306                 scratchGPR = resultTagGPR;
    3307            
     3306
    33083307            base.use();
    33093308           
    3310             cachedGetById(node.codeOrigin, InvalidGPRReg, baseGPR, resultTagGPR, resultPayloadGPR, scratchGPR, node.identifierNumber());
     3309            cachedGetById(node.codeOrigin, InvalidGPRReg, baseGPR, resultTagGPR, resultPayloadGPR, node.identifierNumber());
    33113310           
    33123311            jsValueResult(resultTagGPR, resultPayloadGPR, m_compileIndex, UseChildrenCalledExplicitly);
     
    33223321        GPRReg resultTagGPR = resultTag.gpr();
    33233322        GPRReg resultPayloadGPR = resultPayload.gpr();
    3324         GPRReg scratchGPR;
    3325 
    3326         if (resultTagGPR == basePayloadGPR)
    3327             scratchGPR = resultPayloadGPR;
    3328         else
    3329             scratchGPR = resultTagGPR;
    33303323       
    33313324        base.use();
     
    33333326        JITCompiler::Jump notCell = m_jit.branch32(JITCompiler::NotEqual, baseTagGPR, TrustedImm32(JSValue::CellTag));
    33343327       
    3335         cachedGetById(node.codeOrigin, baseTagGPR, basePayloadGPR, resultTagGPR, resultPayloadGPR, scratchGPR, node.identifierNumber(), notCell);
     3328        cachedGetById(node.codeOrigin, baseTagGPR, basePayloadGPR, resultTagGPR, resultPayloadGPR, node.identifierNumber(), notCell);
    33363329       
    33373330        jsValueResult(resultTagGPR, resultPayloadGPR, m_compileIndex, UseChildrenCalledExplicitly);
     
    33553348            GPRReg resultPayloadGPR = resultPayload.gpr();
    33563349
    3357             GPRReg scratchGPR = selectScratchGPR(baseGPR, resultTagGPR, resultPayloadGPR);
    3358            
    33593350            base.use();
    33603351           
    33613352            flushRegisters();
    33623353           
    3363             cachedGetById(node.codeOrigin, InvalidGPRReg, baseGPR, resultTagGPR, resultPayloadGPR, scratchGPR, node.identifierNumber(), JITCompiler::Jump(), DontSpill);
     3354            cachedGetById(node.codeOrigin, InvalidGPRReg, baseGPR, resultTagGPR, resultPayloadGPR, node.identifierNumber(), JITCompiler::Jump(), DontSpill);
    33643355           
    33653356            jsValueResult(resultTagGPR, resultPayloadGPR, m_compileIndex, UseChildrenCalledExplicitly);
     
    33763367        GPRReg resultPayloadGPR = resultPayload.gpr();
    33773368
    3378         GPRReg scratchGPR = selectScratchGPR(baseTagGPR, basePayloadGPR, resultTagGPR, resultPayloadGPR);
    3379        
    33803369        base.use();
    33813370       
     
    33843373        JITCompiler::Jump notCell = m_jit.branch32(JITCompiler::NotEqual, baseTagGPR, TrustedImm32(JSValue::CellTag));
    33853374       
    3386         cachedGetById(node.codeOrigin, baseTagGPR, basePayloadGPR, resultTagGPR, resultPayloadGPR, scratchGPR, node.identifierNumber(), notCell, DontSpill);
     3375        cachedGetById(node.codeOrigin, baseTagGPR, basePayloadGPR, resultTagGPR, resultPayloadGPR, node.identifierNumber(), notCell, DontSpill);
    33873376       
    33883377        jsValueResult(resultTagGPR, resultPayloadGPR, m_compileIndex, UseChildrenCalledExplicitly);
  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp

    r122678 r122768  
    493493}
    494494
    495 void SpeculativeJIT::cachedGetById(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg resultGPR, GPRReg scratchGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode)
     495void SpeculativeJIT::cachedGetById(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg resultGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode)
    496496{
    497497    JITCompiler::DataLabelPtr structureToCompare;
     
    521521            codeOrigin, structureToCompare, structureCheck, propertyStorageLoad, loadWithPatch,
    522522            slowPath.get(), doneLabel, safeCast<int8_t>(baseGPR), safeCast<int8_t>(resultGPR),
    523             safeCast<int8_t>(scratchGPR),
     523            usedRegisters(),
    524524            spillMode == NeedToSpill ? PropertyAccessRecord::RegistersInUse : PropertyAccessRecord::RegistersFlushed));
    525525    addSlowPathGenerator(slowPath.release());
    526 
    527 
    528     if (scratchGPR != resultGPR && scratchGPR != InvalidGPRReg && spillMode == NeedToSpill)
    529         unlock(scratchGPR);
    530526}
    531527
     
    569565            identifier(identifierNumber));
    570566    }
     567    RegisterSet currentlyUsedRegisters = usedRegisters();
     568    currentlyUsedRegisters.clear(scratchGPR);
     569    ASSERT(currentlyUsedRegisters.get(baseGPR));
     570    ASSERT(currentlyUsedRegisters.get(valueGPR));
    571571    m_jit.addPropertyAccess(
    572572        PropertyAccessRecord(
    573573            codeOrigin, structureToCompare, structureCheck, propertyStorageLoad,
    574574            JITCompiler::DataLabelCompact(storeWithPatch.label()), slowPath.get(), doneLabel,
    575             safeCast<int8_t>(baseGPR), safeCast<int8_t>(valueGPR), safeCast<int8_t>(scratchGPR)));
     575            safeCast<int8_t>(baseGPR), safeCast<int8_t>(valueGPR), currentlyUsedRegisters));
    576576    addSlowPathGenerator(slowPath.release());
    577577}
     
    33283328            GPRReg baseGPR = base.gpr();
    33293329            GPRReg resultGPR = result.gpr();
    3330             GPRReg scratchGPR;
    3331            
    3332             if (resultGPR == baseGPR)
    3333                 scratchGPR = tryAllocate();
    3334             else
    3335                 scratchGPR = resultGPR;
    33363330           
    33373331            base.use();
    33383332           
    3339             cachedGetById(node.codeOrigin, baseGPR, resultGPR, scratchGPR, node.identifierNumber());
     3333            cachedGetById(node.codeOrigin, baseGPR, resultGPR, node.identifierNumber());
    33403334           
    33413335            jsValueResult(resultGPR, m_compileIndex, UseChildrenCalledExplicitly);
     
    33483342        GPRReg baseGPR = base.gpr();
    33493343        GPRReg resultGPR = result.gpr();
    3350         GPRReg scratchGPR;
    3351        
    3352         if (resultGPR == baseGPR)
    3353             scratchGPR = tryAllocate();
    3354         else
    3355             scratchGPR = resultGPR;
    33563344       
    33573345        base.use();
     
    33593347        JITCompiler::Jump notCell = m_jit.branchTestPtr(JITCompiler::NonZero, baseGPR, GPRInfo::tagMaskRegister);
    33603348       
    3361         cachedGetById(node.codeOrigin, baseGPR, resultGPR, scratchGPR, node.identifierNumber(), notCell);
     3349        cachedGetById(node.codeOrigin, baseGPR, resultGPR, node.identifierNumber(), notCell);
    33623350       
    33633351        jsValueResult(resultGPR, m_compileIndex, UseChildrenCalledExplicitly);
     
    33803368            GPRReg resultGPR = result.gpr();
    33813369           
    3382             GPRReg scratchGPR = selectScratchGPR(baseGPR, resultGPR);
    3383            
    33843370            base.use();
    33853371           
    33863372            flushRegisters();
    33873373           
    3388             cachedGetById(node.codeOrigin, baseGPR, resultGPR, scratchGPR, node.identifierNumber(), JITCompiler::Jump(), DontSpill);
     3374            cachedGetById(node.codeOrigin, baseGPR, resultGPR, node.identifierNumber(), JITCompiler::Jump(), DontSpill);
    33893375           
    33903376            jsValueResult(resultGPR, m_compileIndex, UseChildrenCalledExplicitly);
     
    33983384        GPRReg resultGPR = result.gpr();
    33993385       
    3400         GPRReg scratchGPR = selectScratchGPR(baseGPR, resultGPR);
    3401        
    34023386        base.use();
    34033387        flushRegisters();
     
    34053389        JITCompiler::Jump notCell = m_jit.branchTestPtr(JITCompiler::NonZero, baseGPR, GPRInfo::tagMaskRegister);
    34063390       
    3407         cachedGetById(node.codeOrigin, baseGPR, resultGPR, scratchGPR, node.identifierNumber(), notCell, DontSpill);
     3391        cachedGetById(node.codeOrigin, baseGPR, resultGPR, node.identifierNumber(), notCell, DontSpill);
    34083392       
    34093393        jsValueResult(resultGPR, m_compileIndex, UseChildrenCalledExplicitly);
  • trunk/Source/JavaScriptCore/heap/CopiedAllocator.h

    r122677 r122768  
    3434
    3535class CopiedAllocator {
    36     friend class JIT;
    3736public:
    3837    CopiedAllocator();
    3938   
     39    bool fastPathShouldSucceed(size_t bytes) const;
    4040    CheckedBoolean tryAllocate(size_t bytes, void** outPtr);
    4141    CheckedBoolean tryReallocate(void *oldPtr, size_t oldBytes, size_t newBytes);
     
    4747    bool isValid() { return !!m_currentBlock; }
    4848
    49 private:
    5049    CopiedBlock* currentBlock() { return m_currentBlock; }
    5150
     51    // Yes, these are public. No, that doesn't mean you can play with them.
     52    // If I had made them private then I'd have to list off all of the JIT
     53    // classes and functions that are entitled to modify these directly, and
     54    // that would have been gross.
    5255    size_t m_currentRemaining;
    5356    char* m_currentPayloadEnd;
     
    6063    , m_currentBlock(0)
    6164{
     65}
     66
     67inline bool CopiedAllocator::fastPathShouldSucceed(size_t bytes) const
     68{
     69    ASSERT(is8ByteAligned(reinterpret_cast<void*>(bytes)));
     70   
     71    return bytes <= m_currentRemaining;
    6272}
    6373
Note: See TracChangeset for help on using the changeset viewer.