⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 278656 in webkit


Ignore:
Timestamp:
Jun 9, 2021, 4:17:36 AM (5 years ago)
Author:
ysuzuki@apple.com
Message:

[JSC] Use DataIC for AccessCase
​https://bugs.webkit.org/show_bug.cgi?id=226072

Reviewed by Saam Barati and Filip Pizlo.

Source/JavaScriptCore:

This patch adds non-repatching IC for Baseline JIT in ARM64.
This does not work in non-ARM64 architectures (including X64) due to the use of link-register.

  1. We add non-repatching IC, which is enabled only in Baseline due to performance reason. We are using the existing IC in DFG and FTL. Non-repatching includes fast-path, and slow-path's operation function.
  2. We still keep InlineAccess in all tiers. Removing that causes 0.3 ~ 1.0% regression in Speedometer2. This means that we still need some repatching when we first introduce stubs.
  3. We add a mechanism to share generated code stubs in non-repatching IC. Currently, getter / setter / custom accessors are excluded since their code relies on JSGlobalObject, CodeBlock etc. which are not included in AccessCase's data structure.
  4. This patch still relies on that CodeBlock will be destroyed synchronously since we need to ensure that sharing-hash-table does not include already-dead JIT code stubs. We can fix it (e.g. annotating epoch to these stubs, bump them in finalizeUnconditionally), but we avoid doing that to prevent from further enlarging this patch. This patch is already significant in its size.
  5. Added callOperation(Address) support in CCallHelpers, which can save the target in nonArgGPR0, and call it so that we can use Address including GPR which is also used for arguments.

Performance is neutral in JetStream2 and Speedometer2. But it offers the way to remove some code generation in Baseline.

  • assembler/MacroAssemblerARM64E.h:

(JSC::MacroAssemblerARM64E::call):

  • bytecode/AccessCase.cpp:

(JSC::AccessCase::create):
(JSC::AccessCase::createTransition):
(JSC::AccessCase::createDelete):
(JSC::AccessCase::createCheckPrivateBrand):
(JSC::AccessCase::createSetPrivateBrand):
(JSC::AccessCase::fromStructureStubInfo):
(JSC::AccessCase::clone const):
(JSC::AccessCase::generateWithGuard):
(JSC::AccessCase::generateImpl):
(JSC::AccessCase::canBeShared):

  • bytecode/AccessCase.h:

(JSC::AccessCase::hash const):
(JSC::AccessCase::AccessCase):
(JSC::SharedJITStubSet::Hash::Key::Key):
(JSC::SharedJITStubSet::Hash::Key::isHashTableDeletedValue const):
(JSC::SharedJITStubSet::Hash::Key::operator==):
(JSC::SharedJITStubSet::Hash::hash):
(JSC::SharedJITStubSet::Hash::equal):
(JSC::SharedJITStubSet::Searcher::Translator::hash):
(JSC::SharedJITStubSet::Searcher::Translator::equal):
(JSC::SharedJITStubSet::PointerTranslator::hash):
(JSC::SharedJITStubSet::PointerTranslator::equal):
(JSC::SharedJITStubSet::add):
(JSC::SharedJITStubSet::remove):
(JSC::SharedJITStubSet::find):

  • bytecode/ByValInfo.h:

(JSC::ByValInfo::setUp):
(JSC::ByValInfo::offsetOfSlowOperation):
(JSC::ByValInfo::offsetOfNotIndexJumpTarget):
(JSC::ByValInfo::offsetOfBadTypeJumpTarget):

  • bytecode/CallLinkInfo.cpp:

(JSC::CallLinkInfo::initializeDirectCall):
(JSC::CallLinkInfo::setDirectCallTarget):

  • bytecode/CodeBlock.h:

(JSC::CodeBlock::useDataIC const):

  • bytecode/GetterSetterAccessCase.cpp:

(JSC::GetterSetterAccessCase::create):
(JSC::GetterSetterAccessCase::clone const):
(JSC::GetterSetterAccessCase::emitDOMJITGetter):

  • bytecode/GetterSetterAccessCase.h:
  • bytecode/InlineAccess.cpp:

(JSC::getScratchRegister):
(JSC::InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess):
(JSC::InlineAccess::rewireStubAsJumpInAccess):
(JSC::InlineAccess::resetStubAsJumpInAccess):
(JSC::InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess):
(JSC::InlineAccess::rewireStubAsJump): Deleted.

  • bytecode/InlineAccess.h:
  • bytecode/InstanceOfAccessCase.cpp:

(JSC::InstanceOfAccessCase::create):
(JSC::InstanceOfAccessCase::clone const):

  • bytecode/InstanceOfAccessCase.h:
  • bytecode/IntrinsicGetterAccessCase.cpp:

(JSC::IntrinsicGetterAccessCase::create):
(JSC::IntrinsicGetterAccessCase::clone const):

  • bytecode/IntrinsicGetterAccessCase.h:
  • bytecode/ModuleNamespaceAccessCase.cpp:

(JSC::ModuleNamespaceAccessCase::create):
(JSC::ModuleNamespaceAccessCase::clone const):

  • bytecode/ModuleNamespaceAccessCase.h:
  • bytecode/ObjectPropertyConditionSet.h:

(JSC::ObjectPropertyConditionSet::hash const):
(JSC::ObjectPropertyConditionSet::operator==):
(JSC::ObjectPropertyConditionSet::operator!=):

  • bytecode/PolymorphicAccess.cpp:

(JSC::AccessGenerationState::installWatchpoint):
(JSC::AccessGenerationState::succeed):
(JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions):
(JSC::PolymorphicAccess::addCases):
(JSC::PolymorphicAccess::addCase):
(JSC::PolymorphicAccess::visitWeak const):
(JSC::PolymorphicAccess::regenerate):

  • bytecode/PolymorphicAccess.h:

(JSC::AccessGenerationState::AccessGenerationState):
(JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions):

  • bytecode/ProxyableAccessCase.cpp:

(JSC::ProxyableAccessCase::create):
(JSC::ProxyableAccessCase::clone const):

  • bytecode/ProxyableAccessCase.h:
  • bytecode/StructureStubInfo.cpp:

(JSC::StructureStubInfo::StructureStubInfo):
(JSC::StructureStubInfo::addAccessCase):
(JSC::StructureStubInfo::reset):

  • bytecode/StructureStubInfo.h:

(JSC::StructureStubInfo::offsetOfCodePtr):
(JSC::StructureStubInfo::offsetOfSlowPathStartLocation):
(JSC::StructureStubInfo::offsetOfSlowOperation):
(JSC::StructureStubInfo::patchableJump): Deleted.

  • dfg/DFGJITCompiler.h:

(JSC::DFG::JITCompiler::appendCall):

  • dfg/DFGSlowPathGenerator.h:

(JSC::DFG::slowPathICCall):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::compileGetById):
(JSC::DFG::SpeculativeJIT::compileGetByIdFlush):
(JSC::DFG::SpeculativeJIT::compileDeleteById):
(JSC::DFG::SpeculativeJIT::compileDeleteByVal):
(JSC::DFG::SpeculativeJIT::compileInById):
(JSC::DFG::SpeculativeJIT::compileInByVal):
(JSC::DFG::SpeculativeJIT::compileGetPrivateNameByVal):
(JSC::DFG::SpeculativeJIT::compileGetPrivateNameById):
(JSC::DFG::SpeculativeJIT::compilePutPrivateNameById):
(JSC::DFG::SpeculativeJIT::compileCheckPrivateBrand):
(JSC::DFG::SpeculativeJIT::compileSetPrivateBrand):
(JSC::DFG::SpeculativeJIT::compileInstanceOfForCells):
(JSC::DFG::SpeculativeJIT::compileInstanceOf):
(JSC::DFG::SpeculativeJIT::compilePutByIdFlush):
(JSC::DFG::SpeculativeJIT::compilePutById):
(JSC::DFG::SpeculativeJIT::compilePutByIdDirect):
(JSC::DFG::SpeculativeJIT::cachedPutById):

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::callOperation):
(JSC::DFG::SpeculativeJIT::appendCall):
(JSC::DFG::SpeculativeJIT::appendCallSetResult):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::cachedGetById):
(JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::cachedGetById):
(JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis):
(JSC::DFG::SpeculativeJIT::compile):

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::getPrivateName):
(JSC::FTL::DFG::LowerDFGToB3::compilePrivateBrandAccess):
(JSC::FTL::DFG::LowerDFGToB3::cachedPutById):
(JSC::FTL::DFG::LowerDFGToB3::compileGetByVal):
(JSC::FTL::DFG::LowerDFGToB3::compileDelBy):
(JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq):

  • ftl/FTLSlowPathCall.cpp:

(JSC::FTL::SlowPathCallContext::SlowPathCallContext):
(JSC::FTL::SlowPathCallContext::keyWithTarget const):
(JSC::FTL::SlowPathCallContext::makeCall):

  • ftl/FTLSlowPathCall.h:

(JSC::FTL::callOperation):

  • ftl/FTLSlowPathCallKey.cpp:

(JSC::FTL::SlowPathCallKey::dump const):

  • ftl/FTLSlowPathCallKey.h:

(JSC::FTL::SlowPathCallKey::SlowPathCallKey):
(JSC::FTL::SlowPathCallKey::indirectOffset const):
(JSC::FTL::SlowPathCallKey::withCallTarget):
(JSC::FTL::SlowPathCallKey::operator== const):
(JSC::FTL::SlowPathCallKey::hash const):

  • ftl/FTLThunks.cpp:

(JSC::FTL::genericGenerationThunkGenerator):
(JSC::FTL::slowPathCallThunkGenerator):

  • jit/AssemblyHelpers.cpp:

(JSC::AssemblyHelpers::emitVirtualCall):

  • jit/CCallHelpers.cpp:

(JSC::CCallHelpers::emitJITCodeOver):

  • jit/CCallHelpers.h:

(JSC::CCallHelpers::ArgCollection::ArgCollection):
(JSC::CCallHelpers::ArgCollection::pushRegArg):
(JSC::CCallHelpers::ArgCollection::pushExtraRegArg):
(JSC::CCallHelpers::ArgCollection::pushNonArg):
(JSC::CCallHelpers::ArgCollection::addGPRArg):
(JSC::CCallHelpers::ArgCollection::addGPRExtraArg):
(JSC::CCallHelpers::ArgCollection::addStackArg):
(JSC::CCallHelpers::ArgCollection::addPoke):
(JSC::CCallHelpers::calculatePokeOffset):
(JSC::CCallHelpers::pokeForArgument):
(JSC::CCallHelpers::stackAligned):
(JSC::CCallHelpers::marshallArgumentRegister):
(JSC::CCallHelpers::setupArgumentsImpl):
(JSC::CCallHelpers::pokeArgumentsAligned):
(JSC::CCallHelpers::std::is_integral<CURRENT_ARGUMENT_TYPE>::value):
(JSC::CCallHelpers::std::is_pointer<CURRENT_ARGUMENT_TYPE>::value):
(JSC::CCallHelpers::setupArgumentsEntryImpl):
(JSC::CCallHelpers::setupArguments):
(JSC::CCallHelpers::setupArgumentsForIndirectCall):

  • jit/GCAwareJITStubRoutine.cpp:

(JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine):
(JSC::GCAwareJITStubRoutine::makeGCAware):
(JSC::GCAwareJITStubRoutine::observeZeroRefCount):
(JSC::PolymorphicAccessJITStubRoutine::PolymorphicAccessJITStubRoutine):
(JSC::PolymorphicAccessJITStubRoutine::observeZeroRefCount):
(JSC::PolymorphicAccessJITStubRoutine::computeHash):
(JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine):
(JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler):
(JSC::createICJITStubRoutine):
(JSC::createJITStubRoutine): Deleted.

  • jit/GCAwareJITStubRoutine.h:

(JSC::GCAwareJITStubRoutine::create):
(JSC::PolymorphicAccessJITStubRoutine::cases const):
(JSC::PolymorphicAccessJITStubRoutine::weakStructures const):
(JSC::PolymorphicAccessJITStubRoutine::hash const):

  • jit/GPRInfo.h:
  • jit/JIT.cpp:

(JSC::JIT::link):

  • jit/JIT.h:
  • jit/JITCall.cpp:

(JSC::JIT::emit_op_iterator_open):
(JSC::JIT::emitSlow_op_iterator_open):
(JSC::JIT::emit_op_iterator_next):
(JSC::JIT::emitSlow_op_iterator_next):

  • jit/JITCall32_64.cpp:

(JSC::JIT::emit_op_iterator_open):
(JSC::JIT::emit_op_iterator_next):

  • jit/JITCode.h:

(JSC::JITCode::useDataIC):

  • jit/JITInlineCacheGenerator.cpp:

(JSC::JITInlineCacheGenerator::JITInlineCacheGenerator):
(JSC::JITInlineCacheGenerator::finalize):
(JSC::JITByIdGenerator::JITByIdGenerator):
(JSC::JITByIdGenerator::finalize):
(JSC::JITByIdGenerator::generateFastCommon):
(JSC::JITGetByIdGenerator::JITGetByIdGenerator):
(JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator):
(JSC::JITPutByIdGenerator::JITPutByIdGenerator):
(JSC::JITDelByValGenerator::JITDelByValGenerator):
(JSC::JITDelByValGenerator::generateFastPath):
(JSC::JITDelByValGenerator::finalize):
(JSC::JITDelByIdGenerator::JITDelByIdGenerator):
(JSC::JITDelByIdGenerator::generateFastPath):
(JSC::JITDelByIdGenerator::finalize):
(JSC::JITInByValGenerator::JITInByValGenerator):
(JSC::JITInByValGenerator::generateFastPath):
(JSC::JITInByValGenerator::finalize):
(JSC::JITInByIdGenerator::JITInByIdGenerator):
(JSC::JITInstanceOfGenerator::JITInstanceOfGenerator):
(JSC::JITInstanceOfGenerator::generateFastPath):
(JSC::JITInstanceOfGenerator::finalize):
(JSC::JITGetByValGenerator::JITGetByValGenerator):
(JSC::JITGetByValGenerator::generateFastPath):
(JSC::JITGetByValGenerator::finalize):
(JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator):
(JSC::JITPrivateBrandAccessGenerator::generateFastPath):
(JSC::JITPrivateBrandAccessGenerator::finalize):

  • jit/JITInlineCacheGenerator.h:

(JSC::JITGetByIdGenerator::JITGetByIdGenerator): Deleted.
(JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): Deleted.
(JSC::JITPutByIdGenerator::JITPutByIdGenerator): Deleted.
(JSC::JITDelByValGenerator::JITDelByValGenerator): Deleted.
(JSC::JITDelByValGenerator::slowPathJump const): Deleted.
(JSC::JITDelByIdGenerator::JITDelByIdGenerator): Deleted.
(JSC::JITDelByIdGenerator::slowPathJump const): Deleted.
(JSC::JITInByIdGenerator::JITInByIdGenerator): Deleted.
(JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): Deleted.
(JSC::JITGetByValGenerator::JITGetByValGenerator): Deleted.
(JSC::JITGetByValGenerator::slowPathJump const): Deleted.
(JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): Deleted.
(JSC::JITPrivateBrandAccessGenerator::slowPathJump const): Deleted.

  • jit/JITInlines.h:

(JSC::JIT::emitLoadForArrayMode):
(JSC::JIT::appendCallWithExceptionCheck):
(JSC::JIT::appendCallWithExceptionCheckSetJSValueResult):
(JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile):

  • jit/JITOpcodes.cpp:

(JSC::JIT::emit_op_instanceof):
(JSC::JIT::emitSlow_op_instanceof):
(JSC::JIT::privateCompileHasIndexedProperty):
(JSC::JIT::emit_op_has_enumerable_indexed_property):
(JSC::JIT::emitSlow_op_has_enumerable_indexed_property):

  • jit/JITOpcodes32_64.cpp:

(JSC::JIT::emit_op_instanceof):
(JSC::JIT::privateCompileHasIndexedProperty):
(JSC::JIT::emit_op_has_enumerable_indexed_property):

  • jit/JITOperations.cpp:

(JSC::JSC_DEFINE_JIT_OPERATION):

  • jit/JITPropertyAccess.cpp:

(JSC::JIT::emit_op_get_by_val):
(JSC::JIT::emitSlow_op_get_by_val):
(JSC::JIT::slow_op_get_by_val_prepareCallGenerator):
(JSC::JIT::emit_op_get_private_name):
(JSC::JIT::emitSlow_op_get_private_name):
(JSC::JIT::slow_op_get_private_name_prepareCallGenerator):
(JSC::JIT::emit_op_set_private_brand):
(JSC::JIT::emitSlow_op_set_private_brand):
(JSC::JIT::emit_op_check_private_brand):
(JSC::JIT::emitSlow_op_check_private_brand):
(JSC::JIT::emit_op_put_by_val):
(JSC::JIT::emitGenericContiguousPutByVal):
(JSC::JIT::emitArrayStoragePutByVal):
(JSC::JIT::emitPutByValWithCachedId):
(JSC::JIT::emitSlow_op_put_by_val):
(JSC::JIT::slow_op_put_by_val_prepareCallGenerator):
(JSC::JIT::emit_op_put_private_name):
(JSC::JIT::emitSlow_op_put_private_name):
(JSC::JIT::slow_op_put_private_name_prepareCallGenerator):
(JSC::JIT::emit_op_del_by_id):
(JSC::JIT::emitSlow_op_del_by_id):
(JSC::JIT::slow_op_del_by_id_prepareCallGenerator):
(JSC::JIT::emit_op_del_by_val):
(JSC::JIT::emitSlow_op_del_by_val):
(JSC::JIT::slow_op_del_by_val_prepareCallGenerator):
(JSC::JIT::emit_op_try_get_by_id):
(JSC::JIT::emitSlow_op_try_get_by_id):
(JSC::JIT::emit_op_get_by_id_direct):
(JSC::JIT::emitSlow_op_get_by_id_direct):
(JSC::JIT::emit_op_get_by_id):
(JSC::JIT::emit_op_get_by_id_with_this):
(JSC::JIT::emitSlow_op_get_by_id):
(JSC::JIT::slow_op_get_by_id_prepareCallGenerator):
(JSC::JIT::emitSlow_op_get_by_id_with_this):
(JSC::JIT::slow_op_get_by_id_with_this_prepareCallGenerator):
(JSC::JIT::emit_op_put_by_id):
(JSC::JIT::emitSlow_op_put_by_id):
(JSC::JIT::slow_op_put_by_id_prepareCallGenerator):
(JSC::JIT::emit_op_in_by_id):
(JSC::JIT::emitSlow_op_in_by_id):
(JSC::JIT::emit_op_in_by_val):
(JSC::JIT::emitSlow_op_in_by_val):
(JSC::JIT::privateCompilePutByVal):
(JSC::JIT::privateCompilePutPrivateNameWithCachedId):
(JSC::JIT::privateCompilePutByValWithCachedId):
(JSC::JIT::emitDoubleLoad):
(JSC::JIT::emitContiguousLoad):
(JSC::JIT::emitArrayStorageLoad):
(JSC::JIT::emitIntTypedArrayPutByVal):
(JSC::JIT::emitFloatTypedArrayPutByVal):

  • jit/JITPropertyAccess32_64.cpp:

(JSC::JIT::emit_op_del_by_id):
(JSC::JIT::emit_op_del_by_val):
(JSC::JIT::emit_op_get_by_val):
(JSC::JIT::emit_op_get_private_name):
(JSC::JIT::emit_op_set_private_brand):
(JSC::JIT::emit_op_check_private_brand):
(JSC::JIT::emit_op_put_by_val):
(JSC::JIT::emitGenericContiguousPutByVal):
(JSC::JIT::emitArrayStoragePutByVal):
(JSC::JIT::emit_op_try_get_by_id):
(JSC::JIT::emit_op_get_by_id_direct):
(JSC::JIT::emit_op_get_by_id):
(JSC::JIT::emit_op_get_by_id_with_this):
(JSC::JIT::emit_op_put_by_id):
(JSC::JIT::emit_op_in_by_id):
(JSC::JIT::emit_op_in_by_val):

  • jit/JITStubRoutine.h:
  • jit/PolymorphicCallStubRoutine.cpp:

(JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine):

  • jit/Repatch.cpp:

(JSC::readPutICCallTarget):
(JSC::repatchSlowPathCall):
(JSC::tryCacheGetBy):
(JSC::repatchGetBy):
(JSC::tryCacheArrayGetByVal):
(JSC::repatchArrayGetByVal):
(JSC::tryCachePutByID):
(JSC::repatchPutByID):
(JSC::tryCacheDeleteBy):
(JSC::repatchDeleteBy):
(JSC::tryCacheInBy):
(JSC::repatchInBy):
(JSC::tryCacheCheckPrivateBrand):
(JSC::repatchCheckPrivateBrand):
(JSC::tryCacheSetPrivateBrand):
(JSC::repatchSetPrivateBrand):
(JSC::tryCacheInstanceOf):
(JSC::repatchInstanceOf):
(JSC::linkSlowFor):
(JSC::linkVirtualFor):
(JSC::resetGetBy):
(JSC::resetPutByID):
(JSC::resetDelBy):
(JSC::resetInBy):
(JSC::resetInstanceOf):
(JSC::resetCheckPrivateBrand):
(JSC::resetSetPrivateBrand):
(JSC::resetPatchableJump): Deleted.

  • jit/Repatch.h:
  • runtime/Options.cpp:

(JSC::Options::recomputeDependentOptions):

  • runtime/OptionsList.h:
  • runtime/StructureIDTable.h:
  • runtime/VM.cpp:

(JSC::VM::VM):
(JSC::VM::~VM):

  • runtime/VM.h:

Tools:

  • Scripts/run-jsc-stress-tests:
Location:
trunk
Files:
66 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/ChangeLog

    r278642 r278656  
     12021-06-08  Yusuke Suzuki  <ysuzuki@apple.com>
     2
     3        [JSC] Use DataIC for AccessCase
     4        https://bugs.webkit.org/show_bug.cgi?id=226072
     5
     6        Reviewed by Saam Barati and Filip Pizlo.
     7
     8        This patch adds non-repatching IC for Baseline JIT in ARM64.
     9        This does not work in non-ARM64 architectures (including X64) due to the use of link-register.
     10
     11        1. We add non-repatching IC, which is enabled only in Baseline due to performance reason. We are using the
     12           existing IC in DFG and FTL. Non-repatching includes fast-path, and slow-path's operation function.
     13        2. We still keep InlineAccess in all tiers. Removing that causes 0.3 ~ 1.0% regression in Speedometer2. This
     14           means that we still need some repatching when we first introduce stubs.
     15        3. We add a mechanism to share generated code stubs in non-repatching IC. Currently, getter / setter / custom
     16           accessors are excluded since their code relies on JSGlobalObject, CodeBlock etc. which are not included in
     17           AccessCase's data structure.
     18        4. This patch still relies on that CodeBlock will be destroyed synchronously since we need to ensure that sharing-hash-table
     19           does not include already-dead JIT code stubs. We can fix it (e.g. annotating epoch to these stubs, bump them in
     20           finalizeUnconditionally), but we avoid doing that to prevent from further enlarging this patch. This patch is already
     21           significant in its size.
     22        5. Added callOperation(Address) support in CCallHelpers, which can save the target in nonArgGPR0, and call it so that we can use
     23           Address including GPR which is also used for arguments.
     24
     25        Performance is neutral in JetStream2 and Speedometer2. But it offers the way to remove some code generation in Baseline.
     26
     27        * assembler/MacroAssemblerARM64E.h:
     28        (JSC::MacroAssemblerARM64E::call):
     29        * bytecode/AccessCase.cpp:
     30        (JSC::AccessCase::create):
     31        (JSC::AccessCase::createTransition):
     32        (JSC::AccessCase::createDelete):
     33        (JSC::AccessCase::createCheckPrivateBrand):
     34        (JSC::AccessCase::createSetPrivateBrand):
     35        (JSC::AccessCase::fromStructureStubInfo):
     36        (JSC::AccessCase::clone const):
     37        (JSC::AccessCase::generateWithGuard):
     38        (JSC::AccessCase::generateImpl):
     39        (JSC::AccessCase::canBeShared):
     40        * bytecode/AccessCase.h:
     41        (JSC::AccessCase::hash const):
     42        (JSC::AccessCase::AccessCase):
     43        (JSC::SharedJITStubSet::Hash::Key::Key):
     44        (JSC::SharedJITStubSet::Hash::Key::isHashTableDeletedValue const):
     45        (JSC::SharedJITStubSet::Hash::Key::operator==):
     46        (JSC::SharedJITStubSet::Hash::hash):
     47        (JSC::SharedJITStubSet::Hash::equal):
     48        (JSC::SharedJITStubSet::Searcher::Translator::hash):
     49        (JSC::SharedJITStubSet::Searcher::Translator::equal):
     50        (JSC::SharedJITStubSet::PointerTranslator::hash):
     51        (JSC::SharedJITStubSet::PointerTranslator::equal):
     52        (JSC::SharedJITStubSet::add):
     53        (JSC::SharedJITStubSet::remove):
     54        (JSC::SharedJITStubSet::find):
     55        * bytecode/ByValInfo.h:
     56        (JSC::ByValInfo::setUp):
     57        (JSC::ByValInfo::offsetOfSlowOperation):
     58        (JSC::ByValInfo::offsetOfNotIndexJumpTarget):
     59        (JSC::ByValInfo::offsetOfBadTypeJumpTarget):
     60        * bytecode/CallLinkInfo.cpp:
     61        (JSC::CallLinkInfo::initializeDirectCall):
     62        (JSC::CallLinkInfo::setDirectCallTarget):
     63        * bytecode/CodeBlock.h:
     64        (JSC::CodeBlock::useDataIC const):
     65        * bytecode/GetterSetterAccessCase.cpp:
     66        (JSC::GetterSetterAccessCase::create):
     67        (JSC::GetterSetterAccessCase::clone const):
     68        (JSC::GetterSetterAccessCase::emitDOMJITGetter):
     69        * bytecode/GetterSetterAccessCase.h:
     70        * bytecode/InlineAccess.cpp:
     71        (JSC::getScratchRegister):
     72        (JSC::InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess):
     73        (JSC::InlineAccess::rewireStubAsJumpInAccess):
     74        (JSC::InlineAccess::resetStubAsJumpInAccess):
     75        (JSC::InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess):
     76        (JSC::InlineAccess::rewireStubAsJump): Deleted.
     77        * bytecode/InlineAccess.h:
     78        * bytecode/InstanceOfAccessCase.cpp:
     79        (JSC::InstanceOfAccessCase::create):
     80        (JSC::InstanceOfAccessCase::clone const):
     81        * bytecode/InstanceOfAccessCase.h:
     82        * bytecode/IntrinsicGetterAccessCase.cpp:
     83        (JSC::IntrinsicGetterAccessCase::create):
     84        (JSC::IntrinsicGetterAccessCase::clone const):
     85        * bytecode/IntrinsicGetterAccessCase.h:
     86        * bytecode/ModuleNamespaceAccessCase.cpp:
     87        (JSC::ModuleNamespaceAccessCase::create):
     88        (JSC::ModuleNamespaceAccessCase::clone const):
     89        * bytecode/ModuleNamespaceAccessCase.h:
     90        * bytecode/ObjectPropertyConditionSet.h:
     91        (JSC::ObjectPropertyConditionSet::hash const):
     92        (JSC::ObjectPropertyConditionSet::operator==):
     93        (JSC::ObjectPropertyConditionSet::operator!=):
     94        * bytecode/PolymorphicAccess.cpp:
     95        (JSC::AccessGenerationState::installWatchpoint):
     96        (JSC::AccessGenerationState::succeed):
     97        (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions):
     98        (JSC::PolymorphicAccess::addCases):
     99        (JSC::PolymorphicAccess::addCase):
     100        (JSC::PolymorphicAccess::visitWeak const):
     101        (JSC::PolymorphicAccess::regenerate):
     102        * bytecode/PolymorphicAccess.h:
     103        (JSC::AccessGenerationState::AccessGenerationState):
     104        (JSC::AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions):
     105        * bytecode/ProxyableAccessCase.cpp:
     106        (JSC::ProxyableAccessCase::create):
     107        (JSC::ProxyableAccessCase::clone const):
     108        * bytecode/ProxyableAccessCase.h:
     109        * bytecode/StructureStubInfo.cpp:
     110        (JSC::StructureStubInfo::StructureStubInfo):
     111        (JSC::StructureStubInfo::addAccessCase):
     112        (JSC::StructureStubInfo::reset):
     113        * bytecode/StructureStubInfo.h:
     114        (JSC::StructureStubInfo::offsetOfCodePtr):
     115        (JSC::StructureStubInfo::offsetOfSlowPathStartLocation):
     116        (JSC::StructureStubInfo::offsetOfSlowOperation):
     117        (JSC::StructureStubInfo::patchableJump): Deleted.
     118        * dfg/DFGJITCompiler.h:
     119        (JSC::DFG::JITCompiler::appendCall):
     120        * dfg/DFGSlowPathGenerator.h:
     121        (JSC::DFG::slowPathICCall):
     122        * dfg/DFGSpeculativeJIT.cpp:
     123        (JSC::DFG::SpeculativeJIT::compileGetById):
     124        (JSC::DFG::SpeculativeJIT::compileGetByIdFlush):
     125        (JSC::DFG::SpeculativeJIT::compileDeleteById):
     126        (JSC::DFG::SpeculativeJIT::compileDeleteByVal):
     127        (JSC::DFG::SpeculativeJIT::compileInById):
     128        (JSC::DFG::SpeculativeJIT::compileInByVal):
     129        (JSC::DFG::SpeculativeJIT::compileGetPrivateNameByVal):
     130        (JSC::DFG::SpeculativeJIT::compileGetPrivateNameById):
     131        (JSC::DFG::SpeculativeJIT::compilePutPrivateNameById):
     132        (JSC::DFG::SpeculativeJIT::compileCheckPrivateBrand):
     133        (JSC::DFG::SpeculativeJIT::compileSetPrivateBrand):
     134        (JSC::DFG::SpeculativeJIT::compileInstanceOfForCells):
     135        (JSC::DFG::SpeculativeJIT::compileInstanceOf):
     136        (JSC::DFG::SpeculativeJIT::compilePutByIdFlush):
     137        (JSC::DFG::SpeculativeJIT::compilePutById):
     138        (JSC::DFG::SpeculativeJIT::compilePutByIdDirect):
     139        (JSC::DFG::SpeculativeJIT::cachedPutById):
     140        * dfg/DFGSpeculativeJIT.h:
     141        (JSC::DFG::SpeculativeJIT::callOperation):
     142        (JSC::DFG::SpeculativeJIT::appendCall):
     143        (JSC::DFG::SpeculativeJIT::appendCallSetResult):
     144        * dfg/DFGSpeculativeJIT32_64.cpp:
     145        (JSC::DFG::SpeculativeJIT::cachedGetById):
     146        (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis):
     147        (JSC::DFG::SpeculativeJIT::compile):
     148        * dfg/DFGSpeculativeJIT64.cpp:
     149        (JSC::DFG::SpeculativeJIT::cachedGetById):
     150        (JSC::DFG::SpeculativeJIT::cachedGetByIdWithThis):
     151        (JSC::DFG::SpeculativeJIT::compile):
     152        * ftl/FTLLowerDFGToB3.cpp:
     153        (JSC::FTL::DFG::LowerDFGToB3::getPrivateName):
     154        (JSC::FTL::DFG::LowerDFGToB3::compilePrivateBrandAccess):
     155        (JSC::FTL::DFG::LowerDFGToB3::cachedPutById):
     156        (JSC::FTL::DFG::LowerDFGToB3::compileGetByVal):
     157        (JSC::FTL::DFG::LowerDFGToB3::compileDelBy):
     158        (JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq):
     159        * ftl/FTLSlowPathCall.cpp:
     160        (JSC::FTL::SlowPathCallContext::SlowPathCallContext):
     161        (JSC::FTL::SlowPathCallContext::keyWithTarget const):
     162        (JSC::FTL::SlowPathCallContext::makeCall):
     163        * ftl/FTLSlowPathCall.h:
     164        (JSC::FTL::callOperation):
     165        * ftl/FTLSlowPathCallKey.cpp:
     166        (JSC::FTL::SlowPathCallKey::dump const):
     167        * ftl/FTLSlowPathCallKey.h:
     168        (JSC::FTL::SlowPathCallKey::SlowPathCallKey):
     169        (JSC::FTL::SlowPathCallKey::indirectOffset const):
     170        (JSC::FTL::SlowPathCallKey::withCallTarget):
     171        (JSC::FTL::SlowPathCallKey::operator== const):
     172        (JSC::FTL::SlowPathCallKey::hash const):
     173        * ftl/FTLThunks.cpp:
     174        (JSC::FTL::genericGenerationThunkGenerator):
     175        (JSC::FTL::slowPathCallThunkGenerator):
     176        * jit/AssemblyHelpers.cpp:
     177        (JSC::AssemblyHelpers::emitVirtualCall):
     178        * jit/CCallHelpers.cpp:
     179        (JSC::CCallHelpers::emitJITCodeOver):
     180        * jit/CCallHelpers.h:
     181        (JSC::CCallHelpers::ArgCollection::ArgCollection):
     182        (JSC::CCallHelpers::ArgCollection::pushRegArg):
     183        (JSC::CCallHelpers::ArgCollection::pushExtraRegArg):
     184        (JSC::CCallHelpers::ArgCollection::pushNonArg):
     185        (JSC::CCallHelpers::ArgCollection::addGPRArg):
     186        (JSC::CCallHelpers::ArgCollection::addGPRExtraArg):
     187        (JSC::CCallHelpers::ArgCollection::addStackArg):
     188        (JSC::CCallHelpers::ArgCollection::addPoke):
     189        (JSC::CCallHelpers::calculatePokeOffset):
     190        (JSC::CCallHelpers::pokeForArgument):
     191        (JSC::CCallHelpers::stackAligned):
     192        (JSC::CCallHelpers::marshallArgumentRegister):
     193        (JSC::CCallHelpers::setupArgumentsImpl):
     194        (JSC::CCallHelpers::pokeArgumentsAligned):
     195        (JSC::CCallHelpers::std::is_integral<CURRENT_ARGUMENT_TYPE>::value):
     196        (JSC::CCallHelpers::std::is_pointer<CURRENT_ARGUMENT_TYPE>::value):
     197        (JSC::CCallHelpers::setupArgumentsEntryImpl):
     198        (JSC::CCallHelpers::setupArguments):
     199        (JSC::CCallHelpers::setupArgumentsForIndirectCall):
     200        * jit/GCAwareJITStubRoutine.cpp:
     201        (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine):
     202        (JSC::GCAwareJITStubRoutine::makeGCAware):
     203        (JSC::GCAwareJITStubRoutine::observeZeroRefCount):
     204        (JSC::PolymorphicAccessJITStubRoutine::PolymorphicAccessJITStubRoutine):
     205        (JSC::PolymorphicAccessJITStubRoutine::observeZeroRefCount):
     206        (JSC::PolymorphicAccessJITStubRoutine::computeHash):
     207        (JSC::MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine):
     208        (JSC::GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler):
     209        (JSC::createICJITStubRoutine):
     210        (JSC::createJITStubRoutine): Deleted.
     211        * jit/GCAwareJITStubRoutine.h:
     212        (JSC::GCAwareJITStubRoutine::create):
     213        (JSC::PolymorphicAccessJITStubRoutine::cases const):
     214        (JSC::PolymorphicAccessJITStubRoutine::weakStructures const):
     215        (JSC::PolymorphicAccessJITStubRoutine::hash const):
     216        * jit/GPRInfo.h:
     217        * jit/JIT.cpp:
     218        (JSC::JIT::link):
     219        * jit/JIT.h:
     220        * jit/JITCall.cpp:
     221        (JSC::JIT::emit_op_iterator_open):
     222        (JSC::JIT::emitSlow_op_iterator_open):
     223        (JSC::JIT::emit_op_iterator_next):
     224        (JSC::JIT::emitSlow_op_iterator_next):
     225        * jit/JITCall32_64.cpp:
     226        (JSC::JIT::emit_op_iterator_open):
     227        (JSC::JIT::emit_op_iterator_next):
     228        * jit/JITCode.h:
     229        (JSC::JITCode::useDataIC):
     230        * jit/JITInlineCacheGenerator.cpp:
     231        (JSC::JITInlineCacheGenerator::JITInlineCacheGenerator):
     232        (JSC::JITInlineCacheGenerator::finalize):
     233        (JSC::JITByIdGenerator::JITByIdGenerator):
     234        (JSC::JITByIdGenerator::finalize):
     235        (JSC::JITByIdGenerator::generateFastCommon):
     236        (JSC::JITGetByIdGenerator::JITGetByIdGenerator):
     237        (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator):
     238        (JSC::JITPutByIdGenerator::JITPutByIdGenerator):
     239        (JSC::JITDelByValGenerator::JITDelByValGenerator):
     240        (JSC::JITDelByValGenerator::generateFastPath):
     241        (JSC::JITDelByValGenerator::finalize):
     242        (JSC::JITDelByIdGenerator::JITDelByIdGenerator):
     243        (JSC::JITDelByIdGenerator::generateFastPath):
     244        (JSC::JITDelByIdGenerator::finalize):
     245        (JSC::JITInByValGenerator::JITInByValGenerator):
     246        (JSC::JITInByValGenerator::generateFastPath):
     247        (JSC::JITInByValGenerator::finalize):
     248        (JSC::JITInByIdGenerator::JITInByIdGenerator):
     249        (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator):
     250        (JSC::JITInstanceOfGenerator::generateFastPath):
     251        (JSC::JITInstanceOfGenerator::finalize):
     252        (JSC::JITGetByValGenerator::JITGetByValGenerator):
     253        (JSC::JITGetByValGenerator::generateFastPath):
     254        (JSC::JITGetByValGenerator::finalize):
     255        (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator):
     256        (JSC::JITPrivateBrandAccessGenerator::generateFastPath):
     257        (JSC::JITPrivateBrandAccessGenerator::finalize):
     258        * jit/JITInlineCacheGenerator.h:
     259        (JSC::JITGetByIdGenerator::JITGetByIdGenerator): Deleted.
     260        (JSC::JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator): Deleted.
     261        (JSC::JITPutByIdGenerator::JITPutByIdGenerator): Deleted.
     262        (JSC::JITDelByValGenerator::JITDelByValGenerator): Deleted.
     263        (JSC::JITDelByValGenerator::slowPathJump const): Deleted.
     264        (JSC::JITDelByIdGenerator::JITDelByIdGenerator): Deleted.
     265        (JSC::JITDelByIdGenerator::slowPathJump const): Deleted.
     266        (JSC::JITInByIdGenerator::JITInByIdGenerator): Deleted.
     267        (JSC::JITInstanceOfGenerator::JITInstanceOfGenerator): Deleted.
     268        (JSC::JITGetByValGenerator::JITGetByValGenerator): Deleted.
     269        (JSC::JITGetByValGenerator::slowPathJump const): Deleted.
     270        (JSC::JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator): Deleted.
     271        (JSC::JITPrivateBrandAccessGenerator::slowPathJump const): Deleted.
     272        * jit/JITInlines.h:
     273        (JSC::JIT::emitLoadForArrayMode):
     274        (JSC::JIT::appendCallWithExceptionCheck):
     275        (JSC::JIT::appendCallWithExceptionCheckSetJSValueResult):
     276        (JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile):
     277        * jit/JITOpcodes.cpp:
     278        (JSC::JIT::emit_op_instanceof):
     279        (JSC::JIT::emitSlow_op_instanceof):
     280        (JSC::JIT::privateCompileHasIndexedProperty):
     281        (JSC::JIT::emit_op_has_enumerable_indexed_property):
     282        (JSC::JIT::emitSlow_op_has_enumerable_indexed_property):
     283        * jit/JITOpcodes32_64.cpp:
     284        (JSC::JIT::emit_op_instanceof):
     285        (JSC::JIT::privateCompileHasIndexedProperty):
     286        (JSC::JIT::emit_op_has_enumerable_indexed_property):
     287        * jit/JITOperations.cpp:
     288        (JSC::JSC_DEFINE_JIT_OPERATION):
     289        * jit/JITPropertyAccess.cpp:
     290        (JSC::JIT::emit_op_get_by_val):
     291        (JSC::JIT::emitSlow_op_get_by_val):
     292        (JSC::JIT::slow_op_get_by_val_prepareCallGenerator):
     293        (JSC::JIT::emit_op_get_private_name):
     294        (JSC::JIT::emitSlow_op_get_private_name):
     295        (JSC::JIT::slow_op_get_private_name_prepareCallGenerator):
     296        (JSC::JIT::emit_op_set_private_brand):
     297        (JSC::JIT::emitSlow_op_set_private_brand):
     298        (JSC::JIT::emit_op_check_private_brand):
     299        (JSC::JIT::emitSlow_op_check_private_brand):
     300        (JSC::JIT::emit_op_put_by_val):
     301        (JSC::JIT::emitGenericContiguousPutByVal):
     302        (JSC::JIT::emitArrayStoragePutByVal):
     303        (JSC::JIT::emitPutByValWithCachedId):
     304        (JSC::JIT::emitSlow_op_put_by_val):
     305        (JSC::JIT::slow_op_put_by_val_prepareCallGenerator):
     306        (JSC::JIT::emit_op_put_private_name):
     307        (JSC::JIT::emitSlow_op_put_private_name):
     308        (JSC::JIT::slow_op_put_private_name_prepareCallGenerator):
     309        (JSC::JIT::emit_op_del_by_id):
     310        (JSC::JIT::emitSlow_op_del_by_id):
     311        (JSC::JIT::slow_op_del_by_id_prepareCallGenerator):
     312        (JSC::JIT::emit_op_del_by_val):
     313        (JSC::JIT::emitSlow_op_del_by_val):
     314        (JSC::JIT::slow_op_del_by_val_prepareCallGenerator):
     315        (JSC::JIT::emit_op_try_get_by_id):
     316        (JSC::JIT::emitSlow_op_try_get_by_id):
     317        (JSC::JIT::emit_op_get_by_id_direct):
     318        (JSC::JIT::emitSlow_op_get_by_id_direct):
     319        (JSC::JIT::emit_op_get_by_id):
     320        (JSC::JIT::emit_op_get_by_id_with_this):
     321        (JSC::JIT::emitSlow_op_get_by_id):
     322        (JSC::JIT::slow_op_get_by_id_prepareCallGenerator):
     323        (JSC::JIT::emitSlow_op_get_by_id_with_this):
     324        (JSC::JIT::slow_op_get_by_id_with_this_prepareCallGenerator):
     325        (JSC::JIT::emit_op_put_by_id):
     326        (JSC::JIT::emitSlow_op_put_by_id):
     327        (JSC::JIT::slow_op_put_by_id_prepareCallGenerator):
     328        (JSC::JIT::emit_op_in_by_id):
     329        (JSC::JIT::emitSlow_op_in_by_id):
     330        (JSC::JIT::emit_op_in_by_val):
     331        (JSC::JIT::emitSlow_op_in_by_val):
     332        (JSC::JIT::privateCompilePutByVal):
     333        (JSC::JIT::privateCompilePutPrivateNameWithCachedId):
     334        (JSC::JIT::privateCompilePutByValWithCachedId):
     335        (JSC::JIT::emitDoubleLoad):
     336        (JSC::JIT::emitContiguousLoad):
     337        (JSC::JIT::emitArrayStorageLoad):
     338        (JSC::JIT::emitIntTypedArrayPutByVal):
     339        (JSC::JIT::emitFloatTypedArrayPutByVal):
     340        * jit/JITPropertyAccess32_64.cpp:
     341        (JSC::JIT::emit_op_del_by_id):
     342        (JSC::JIT::emit_op_del_by_val):
     343        (JSC::JIT::emit_op_get_by_val):
     344        (JSC::JIT::emit_op_get_private_name):
     345        (JSC::JIT::emit_op_set_private_brand):
     346        (JSC::JIT::emit_op_check_private_brand):
     347        (JSC::JIT::emit_op_put_by_val):
     348        (JSC::JIT::emitGenericContiguousPutByVal):
     349        (JSC::JIT::emitArrayStoragePutByVal):
     350        (JSC::JIT::emit_op_try_get_by_id):
     351        (JSC::JIT::emit_op_get_by_id_direct):
     352        (JSC::JIT::emit_op_get_by_id):
     353        (JSC::JIT::emit_op_get_by_id_with_this):
     354        (JSC::JIT::emit_op_put_by_id):
     355        (JSC::JIT::emit_op_in_by_id):
     356        (JSC::JIT::emit_op_in_by_val):
     357        * jit/JITStubRoutine.h:
     358        * jit/PolymorphicCallStubRoutine.cpp:
     359        (JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine):
     360        * jit/Repatch.cpp:
     361        (JSC::readPutICCallTarget):
     362        (JSC::repatchSlowPathCall):
     363        (JSC::tryCacheGetBy):
     364        (JSC::repatchGetBy):
     365        (JSC::tryCacheArrayGetByVal):
     366        (JSC::repatchArrayGetByVal):
     367        (JSC::tryCachePutByID):
     368        (JSC::repatchPutByID):
     369        (JSC::tryCacheDeleteBy):
     370        (JSC::repatchDeleteBy):
     371        (JSC::tryCacheInBy):
     372        (JSC::repatchInBy):
     373        (JSC::tryCacheCheckPrivateBrand):
     374        (JSC::repatchCheckPrivateBrand):
     375        (JSC::tryCacheSetPrivateBrand):
     376        (JSC::repatchSetPrivateBrand):
     377        (JSC::tryCacheInstanceOf):
     378        (JSC::repatchInstanceOf):
     379        (JSC::linkSlowFor):
     380        (JSC::linkVirtualFor):
     381        (JSC::resetGetBy):
     382        (JSC::resetPutByID):
     383        (JSC::resetDelBy):
     384        (JSC::resetInBy):
     385        (JSC::resetInstanceOf):
     386        (JSC::resetCheckPrivateBrand):
     387        (JSC::resetSetPrivateBrand):
     388        (JSC::resetPatchableJump): Deleted.
     389        * jit/Repatch.h:
     390        * runtime/Options.cpp:
     391        (JSC::Options::recomputeDependentOptions):
     392        * runtime/OptionsList.h:
     393        * runtime/StructureIDTable.h:
     394        * runtime/VM.cpp:
     395        (JSC::VM::VM):
     396        (JSC::VM::~VM):
     397        * runtime/VM.h:
     398
    13992021-06-08  Robin Morisset  <rmorisset@apple.com>
    2400

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64E.h

    r278576 r278656  
    226226    }
    227227
    228     ALWAYS_INLINE Call call(Address address, PtrTag tag)
     228    ALWAYS_INLINE void call(Address address, PtrTag tag)
    229229    {
    230230        ASSERT(tag != CFunctionPtrTag && tag != NoPtrTag);
    231231        ASSERT(!Options::useJITCage() || callerType(tag) == PtrTagCallerType::JIT);
    232232        load64(address, getCachedDataTempRegisterIDAndInvalidate());
    233         return call(dataTempRegister, tag);
    234     }
    235 
    236     ALWAYS_INLINE Call call(Address address, RegisterID tag)
     233        call(dataTempRegister, tag);
     234    }
     235
     236    ALWAYS_INLINE void call(Address address, RegisterID tag)
    237237    {
    238238        ASSERT(tag != dataTempRegister);
    239239        load64(address, getCachedDataTempRegisterIDAndInvalidate());
    240         return call(dataTempRegister, tag);
     240        call(dataTempRegister, tag);
    241241    }
    242242

  • trunk/Source/JavaScriptCore/bytecode/AccessCase.cpp

    r278253 r278656  
    6969}
    7070
    71 std::unique_ptr<AccessCase> AccessCase::create(VM& vm, JSCell* owner, AccessType type, CacheableIdentifier identifier, PropertyOffset offset, Structure* structure, const ObjectPropertyConditionSet& conditionSet, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain)
     71Ref<AccessCase> AccessCase::create(VM& vm, JSCell* owner, AccessType type, CacheableIdentifier identifier, PropertyOffset offset, Structure* structure, const ObjectPropertyConditionSet& conditionSet, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain)
    7272{
    7373    switch (type) {
    … …  
    106106    };
    107107
    108     return std::unique_ptr<AccessCase>(new AccessCase(vm, owner, type, identifier, offset, structure, conditionSet, WTFMove(prototypeAccessChain)));
    109 }
    110 
    111 std::unique_ptr<AccessCase> AccessCase::createTransition(
     108    return adoptRef(*new AccessCase(vm, owner, type, identifier, offset, structure, conditionSet, WTFMove(prototypeAccessChain)));
     109}
     110
     111RefPtr<AccessCase> AccessCase::createTransition(
    112112    VM& vm, JSCell* owner, CacheableIdentifier identifier, PropertyOffset offset, Structure* oldStructure, Structure* newStructure,
    113113    const ObjectPropertyConditionSet& conditionSet, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain)
    … …  
    123123    }
    124124
    125     return std::unique_ptr<AccessCase>(new AccessCase(vm, owner, Transition, identifier, offset, newStructure, conditionSet, WTFMove(prototypeAccessChain)));
    126 }
    127 
    128 std::unique_ptr<AccessCase> AccessCase::createDelete(
     125    return adoptRef(*new AccessCase(vm, owner, Transition, identifier, offset, newStructure, conditionSet, WTFMove(prototypeAccessChain)));
     126}
     127
     128Ref<AccessCase> AccessCase::createDelete(
    129129    VM& vm, JSCell* owner, CacheableIdentifier identifier, PropertyOffset offset, Structure* oldStructure, Structure* newStructure)
    130130{
    131131    RELEASE_ASSERT(oldStructure == newStructure->previousID());
    132132    ASSERT(!newStructure->outOfLineCapacity() || oldStructure->outOfLineCapacity());
    133     return std::unique_ptr<AccessCase>(new AccessCase(vm, owner, Delete, identifier, offset, newStructure, { }, { }));
    134 }
    135 
    136 std::unique_ptr<AccessCase> AccessCase::createCheckPrivateBrand(VM& vm, JSCell* owner, CacheableIdentifier identifier, Structure* structure)
    137 {
    138     return std::unique_ptr<AccessCase>(new AccessCase(vm, owner, CheckPrivateBrand, identifier, invalidOffset, structure, { }, { }));
    139 }
    140 
    141 std::unique_ptr<AccessCase> AccessCase::createSetPrivateBrand(
     133    return adoptRef(*new AccessCase(vm, owner, Delete, identifier, offset, newStructure, { }, { }));
     134}
     135
     136Ref<AccessCase> AccessCase::createCheckPrivateBrand(VM& vm, JSCell* owner, CacheableIdentifier identifier, Structure* structure)
     137{
     138    return adoptRef(*new AccessCase(vm, owner, CheckPrivateBrand, identifier, invalidOffset, structure, { }, { }));
     139}
     140
     141Ref<AccessCase> AccessCase::createSetPrivateBrand(
    142142    VM& vm, JSCell* owner, CacheableIdentifier identifier, Structure* oldStructure, Structure* newStructure)
    143143{
    144144    RELEASE_ASSERT(oldStructure == newStructure->previousID());
    145     return std::unique_ptr<AccessCase>(new AccessCase(vm, owner, SetPrivateBrand, identifier, invalidOffset, newStructure, { }, { }));
     145    return adoptRef(*new AccessCase(vm, owner, SetPrivateBrand, identifier, invalidOffset, newStructure, { }, { }));
    146146}
    147147
    … …  
    150150}
    151151
    152 std::unique_ptr<AccessCase> AccessCase::fromStructureStubInfo(
     152RefPtr<AccessCase> AccessCase::fromStructureStubInfo(
    153153    VM& vm, JSCell* owner, CacheableIdentifier identifier, StructureStubInfo& stubInfo)
    154154{
    … …  
    189189}
    190190
    191 std::unique_ptr<AccessCase> AccessCase::clone() const
    192 {
    193     std::unique_ptr<AccessCase> result(new AccessCase(*this));
     191Ref<AccessCase> AccessCase::clone() const
     192{
     193    auto result = adoptRef(*new AccessCase(*this));
    194194    result->resetState();
    195195    return result;
    … …  
    808808    m_state = Generated;
    809809
     810    JSGlobalObject* globalObject = state.m_globalObject;
    810811    CCallHelpers& jit = *state.jit;
    811812    StructureStubInfo& stubInfo = *state.stubInfo;
    … …  
    854855                } else {
    855856                    if (structure->hasMonoProto()) {
    856                         JSValue prototype = structure->prototypeForLookup(state.m_globalObject);
     857                        JSValue prototype = structure->prototypeForLookup(globalObject);
    857858                        RELEASE_ASSERT(prototype.isObject());
    858859                        jit.move(CCallHelpers::TrustedImmPtr(asObject(prototype)), baseForAccessGPR);
    … …  
    965966        allocator.lock(valueRegs);
    966967        allocator.lock(stubInfo.propertyRegs());
     968        if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     969            allocator.lock(stubInfo.m_stubInfoGPR);
    967970        allocator.lock(scratchGPR);
    968971       
    … …  
    10621065        allocator.lock(valueRegs);
    10631066        allocator.lock(stubInfo.propertyRegs());
     1067        if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     1068            allocator.lock(stubInfo.m_stubInfoGPR);
    10641069        allocator.lock(scratchGPR);
    10651070        GPRReg scratch2GPR = allocator.allocateScratchGPR();
    … …  
    11451150        allocator.lock(valueRegs);
    11461151        allocator.lock(stubInfo.propertyRegs());
     1152        if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     1153            allocator.lock(stubInfo.m_stubInfoGPR);
    11471154        allocator.lock(scratchGPR);
    11481155        GPRReg scratch2GPR = allocator.allocateScratchGPR();
    … …  
    12061213        allocator.lock(valueRegs);
    12071214        allocator.lock(stubInfo.propertyRegs());
     1215        if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     1216            allocator.lock(stubInfo.m_stubInfoGPR);
    12081217        allocator.lock(scratchGPR);
    12091218        GPRReg scratch2GPR = allocator.allocateScratchGPR();
    … …  
    13181327        allocator.lock(valueRegs);
    13191328        allocator.lock(stubInfo.propertyRegs());
     1329        if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     1330            allocator.lock(stubInfo.m_stubInfoGPR);
    13201331        allocator.lock(scratchGPR);
    13211332       
    … …  
    14181429
    14191430        if (condition.isWatchableAssumingImpurePropertyWatchpoint(PropertyCondition::WatchabilityEffort::EnsureWatchability)) {
    1420             state.installWatchpoint(condition);
     1431            state.installWatchpoint(codeBlock, condition);
    14211432            continue;
    14221433        }
    … …  
    14361447        // We will emit code that has a weak reference that isn't otherwise listed anywhere.
    14371448        Structure* structure = condition.object()->structure(vm);
    1438         state.weakReferences.append(WriteBarrier<JSCell>(vm, codeBlock, structure));
     1449        state.weakStructures.append(structure->id());
    14391450
    14401451        jit.move(CCallHelpers::TrustedImmPtr(condition.object()), scratchGPR);
    … …  
    15831594        // This also does the necessary calculations of whether or not we're an
    15841595        // exception handling call site.
    1585         AccessGenerationState::SpillState spillState = state.preserveLiveRegistersToStackForCall();
     1596        RegisterSet extraRegistersToPreserve;
     1597#if CPU(ARM64)
     1598        if (codeBlock->useDataIC())
     1599            extraRegistersToPreserve.set(ARM64Registers::lr);
     1600#endif
     1601        AccessGenerationState::SpillState spillState = state.preserveLiveRegistersToStackForCall(extraRegistersToPreserve);
    15861602
    15871603        auto restoreLiveRegistersFromStackForCall = [&](AccessGenerationState::SpillState& spillState, bool callHasReturnValue) {
    … …  
    17131729            jit.move(CCallHelpers::TrustedImmPtr(globalObject), GPRInfo::regT3);
    17141730            access.callLinkInfo()->emitSlowPath(vm, jit);
     1731
    17151732            if (m_type == Getter)
    17161733                jit.setupResults(valueRegs);
    … …  
    17241741            done.link(&jit);
    17251742
    1726             jit.addPtr(CCallHelpers::TrustedImm32((codeBlock->stackPointerOffset() * sizeof(Register)) - state.preservedReusedRegisterState.numberOfBytesPreserved - spillState.numberOfStackBytesUsedForRegisterPreservation),
    1727                 GPRInfo::callFrameRegister, CCallHelpers::stackPointerRegister);
     1743            int stackPointerOffset = (codeBlock->stackPointerOffset() * sizeof(Register)) - state.preservedReusedRegisterState.numberOfBytesPreserved - spillState.numberOfStackBytesUsedForRegisterPreservation;
     1744            jit.addPtr(CCallHelpers::TrustedImm32(stackPointerOffset), GPRInfo::callFrameRegister, CCallHelpers::stackPointerRegister);
    17281745            bool callHasReturnValue = isGetter();
    17291746            restoreLiveRegistersFromStackForCall(spillState, callHasReturnValue);
    … …  
    18461863
    18471864            jit.loadPtr(CCallHelpers::Address(baseGPR, JSProxy::targetOffset()), scratchGPR);
    1848             auto spillState = state.preserveLiveRegistersToStackForCallWithoutExceptions();
     1865
     1866            RegisterSet extraRegistersToPreserve;
     1867#if CPU(ARM64)
     1868            if (codeBlock->useDataIC())
     1869                extraRegistersToPreserve.set(ARM64Registers::lr);
     1870#endif
     1871            auto spillState = state.preserveLiveRegistersToStackForCallWithoutExceptions(extraRegistersToPreserve);
     1872
    18491873            jit.setupArguments<decltype(operationWriteBarrierSlowPath)>(CCallHelpers::TrustedImmPtr(&vm), scratchGPR);
    18501874            jit.prepareCallOperation(vm);
    … …  
    18761900        allocator.lock(stubInfo.baseRegs());
    18771901        allocator.lock(valueRegs);
     1902        if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     1903            allocator.lock(stubInfo.m_stubInfoGPR);
    18781904        allocator.lock(scratchGPR);
    18791905
    … …  
    19341960                extraRegistersToPreserve.set(baseGPR);
    19351961                extraRegistersToPreserve.set(valueRegs);
     1962#if CPU(ARM64)
     1963                if (codeBlock->useDataIC())
     1964                    extraRegistersToPreserve.set(ARM64Registers::lr);
     1965#endif
    19361966                AccessGenerationState::SpillState spillState = state.preserveLiveRegistersToStackForCall(extraRegistersToPreserve);
    19371967               
    … …  
    20322062        allocator.lock(valueRegs);
    20332063        allocator.lock(baseGPR);
     2064        if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     2065            allocator.lock(stubInfo.m_stubInfoGPR);
    20342066        allocator.lock(scratchGPR);
    20352067        ASSERT(structure()->transitionWatchpointSetHasBeenInvalidated());
    … …  
    22042236#endif // ASSERT_ENABLED
    22052237
     2238bool AccessCase::canBeShared(const AccessCase& lhs, const AccessCase& rhs)
     2239{
     2240    // We do not care m_state.
     2241    // And we say "false" if either of them have m_polyProtoAccessChain.
     2242    if (lhs.m_polyProtoAccessChain || rhs.m_polyProtoAccessChain)
     2243        return false;
     2244    if (lhs.additionalSet() || rhs.additionalSet())
     2245        return false;
     2246
     2247    if (lhs.m_type != rhs.m_type)
     2248        return false;
     2249    if (lhs.m_offset != rhs.m_offset)
     2250        return false;
     2251    if (lhs.m_viaProxy != rhs.m_viaProxy)
     2252        return false;
     2253    if (lhs.m_structure.get() != rhs.m_structure.get())
     2254        return false;
     2255    if (lhs.m_identifier != rhs.m_identifier)
     2256        return false;
     2257    if (lhs.m_conditionSet != rhs.m_conditionSet)
     2258        return false;
     2259
     2260    switch (lhs.m_type) {
     2261    case Load:
     2262    case Transition:
     2263    case Delete:
     2264    case DeleteNonConfigurable:
     2265    case DeleteMiss:
     2266    case Replace:
     2267    case Miss:
     2268    case GetGetter:
     2269    case InHit:
     2270    case InMiss:
     2271    case ArrayLength:
     2272    case StringLength:
     2273    case DirectArgumentsLength:
     2274    case ScopedArgumentsLength:
     2275    case CheckPrivateBrand:
     2276    case SetPrivateBrand:
     2277    case IndexedInt32Load:
     2278    case IndexedDoubleLoad:
     2279    case IndexedContiguousLoad:
     2280    case IndexedArrayStorageLoad:
     2281    case IndexedScopedArgumentsLoad:
     2282    case IndexedDirectArgumentsLoad:
     2283    case IndexedTypedArrayInt8Load:
     2284    case IndexedTypedArrayUint8Load:
     2285    case IndexedTypedArrayUint8ClampedLoad:
     2286    case IndexedTypedArrayInt16Load:
     2287    case IndexedTypedArrayUint16Load:
     2288    case IndexedTypedArrayInt32Load:
     2289    case IndexedTypedArrayUint32Load:
     2290    case IndexedTypedArrayFloat32Load:
     2291    case IndexedTypedArrayFloat64Load:
     2292    case IndexedStringLoad:
     2293    case InstanceOfGeneric:
     2294        return true;
     2295
     2296    case Getter:
     2297    case Setter: {
     2298        // Getter and Setter relies on CodeBlock, which makes sharing impossible.
     2299        return false;
     2300    }
     2301
     2302    case CustomValueGetter:
     2303    case CustomAccessorGetter:
     2304    case CustomValueSetter:
     2305    case CustomAccessorSetter: {
     2306        // They are embedding JSGlobalObject that are not tied to sharing JITStubRoutine.
     2307        return false;
     2308    }
     2309
     2310    case IntrinsicGetter: {
     2311        auto& lhsd = lhs.as<IntrinsicGetterAccessCase>();
     2312        auto& rhsd = rhs.as<IntrinsicGetterAccessCase>();
     2313        return lhsd.m_intrinsicFunction == rhsd.m_intrinsicFunction;
     2314    }
     2315
     2316    case ModuleNamespaceLoad: {
     2317        auto& lhsd = lhs.as<ModuleNamespaceAccessCase>();
     2318        auto& rhsd = rhs.as<ModuleNamespaceAccessCase>();
     2319        return lhsd.m_moduleNamespaceObject == rhsd.m_moduleNamespaceObject
     2320            && lhsd.m_moduleEnvironment == rhsd.m_moduleEnvironment
     2321            && lhsd.m_scopeOffset == rhsd.m_scopeOffset;
     2322    }
     2323
     2324    case InstanceOfHit:
     2325    case InstanceOfMiss: {
     2326        auto& lhsd = lhs.as<InstanceOfAccessCase>();
     2327        auto& rhsd = rhs.as<InstanceOfAccessCase>();
     2328        return lhsd.m_prototype == rhsd.m_prototype;
     2329    }
     2330    }
     2331
     2332    return true;
     2333}
     2334
    22062335} // namespace JSC
    22072336

  • trunk/Source/JavaScriptCore/bytecode/AccessCase.h

    r275490 r278656  
    2929
    3030#include "CacheableIdentifier.h"
     31#include "GCAwareJITStubRoutine.h"
     32#include "JITStubRoutine.h"
    3133#include "JSFunctionInlines.h"
    3234#include "ObjectPropertyConditionSet.h"
    3335#include "PolyProtoAccessChain.h"
    3436#include <wtf/CommaPrinter.h>
     37#include <wtf/VectorHash.h>
    3538
    3639namespace JSC {
    … …  
    8083// code. This allows us to only regenerate once we've accumulated (hopefully) more than one new
    8184// AccessCase.
    82 class AccessCase {
     85class AccessCase : public ThreadSafeRefCounted<AccessCase> {
    8386    WTF_MAKE_FAST_ALLOCATED_WITH_HEAP_IDENTIFIER(AccessCase);
    8487public:
    … …  
    148151    }
    149152
    150     static std::unique_ptr<AccessCase> create(VM&, JSCell* owner, AccessType, CacheableIdentifier, PropertyOffset = invalidOffset,
     153    static Ref<AccessCase> create(VM&, JSCell* owner, AccessType, CacheableIdentifier, PropertyOffset = invalidOffset,
    151154        Structure* = nullptr, const ObjectPropertyConditionSet& = ObjectPropertyConditionSet(), RefPtr<PolyProtoAccessChain>&& = nullptr);
    152155
    153     static std::unique_ptr<AccessCase> createTransition(VM&, JSCell* owner, CacheableIdentifier, PropertyOffset, Structure* oldStructure,
     156    static RefPtr<AccessCase> createTransition(VM&, JSCell* owner, CacheableIdentifier, PropertyOffset, Structure* oldStructure,
    154157        Structure* newStructure, const ObjectPropertyConditionSet&, RefPtr<PolyProtoAccessChain>&&);
    155158
    156     static std::unique_ptr<AccessCase> createDelete(VM&, JSCell* owner, CacheableIdentifier, PropertyOffset, Structure* oldStructure,
     159    static Ref<AccessCase> createDelete(VM&, JSCell* owner, CacheableIdentifier, PropertyOffset, Structure* oldStructure,
    157160        Structure* newStructure);
    158161
    159     static std::unique_ptr<AccessCase> createCheckPrivateBrand(VM&, JSCell* owner, CacheableIdentifier, Structure*);
    160     static std::unique_ptr<AccessCase> createSetPrivateBrand(VM&, JSCell* owner, CacheableIdentifier, Structure* oldStructure, Structure* newStructure);
     162    static Ref<AccessCase> createCheckPrivateBrand(VM&, JSCell* owner, CacheableIdentifier, Structure*);
     163    static Ref<AccessCase> createSetPrivateBrand(VM&, JSCell* owner, CacheableIdentifier, Structure* oldStructure, Structure* newStructure);
    161164   
    162     static std::unique_ptr<AccessCase> fromStructureStubInfo(VM&, JSCell* owner, CacheableIdentifier, StructureStubInfo&);
     165    static RefPtr<AccessCase> fromStructureStubInfo(VM&, JSCell* owner, CacheableIdentifier, StructureStubInfo&);
    163166
    164167    AccessType type() const { return m_type; }
    … …  
    254257    ALWAYS_INLINE void checkConsistency(StructureStubInfo&) { }
    255258#endif
    256    
     259
     260    unsigned hash() const
     261    {
     262        return computeHash(m_conditionSet.hash(), static_cast<unsigned>(m_type), m_viaProxy, m_structure.unvalidatedGet(), m_offset);
     263    }
     264
     265    static bool canBeShared(const AccessCase&, const AccessCase&);
     266
    257267protected:
    258268    AccessCase(VM&, JSCell* owner, AccessType, CacheableIdentifier, PropertyOffset, Structure*, const ObjectPropertyConditionSet&, RefPtr<PolyProtoAccessChain>&&);
    259     AccessCase(AccessCase&&) = default;
    260     AccessCase(const AccessCase& other) = default;
     269    AccessCase(AccessCase&& other)
     270        : m_type(WTFMove(other.m_type))
     271        , m_state(WTFMove(other.m_state))
     272        , m_viaProxy(WTFMove(other.m_viaProxy))
     273        , m_offset(WTFMove(other.m_offset))
     274        , m_structure(WTFMove(other.m_structure))
     275        , m_conditionSet(WTFMove(other.m_conditionSet))
     276        , m_polyProtoAccessChain(WTFMove(other.m_polyProtoAccessChain))
     277        , m_identifier(WTFMove(other.m_identifier))
     278    { }
     279
     280    AccessCase(const AccessCase& other)
     281        : m_type(other.m_type)
     282        , m_state(other.m_state)
     283        , m_viaProxy(other.m_viaProxy)
     284        , m_offset(other.m_offset)
     285        , m_structure(other.m_structure)
     286        , m_conditionSet(other.m_conditionSet)
     287        , m_polyProtoAccessChain(other.m_polyProtoAccessChain)
     288        , m_identifier(other.m_identifier)
     289    { }
    261290
    262291    AccessCase& operator=(const AccessCase&) = delete;
    … …  
    276305    // FIXME: This only exists because of how AccessCase puts post-generation things into itself.
    277306    // https://bugs.webkit.org/show_bug.cgi?id=156456
    278     virtual std::unique_ptr<AccessCase> clone() const;
     307    virtual Ref<AccessCase> clone() const;
    279308
    280309    // Perform any action that must be performed before the end of the epoch in which the case
    … …  
    316345};
    317346
     347class SharedJITStubSet {
     348    WTF_MAKE_FAST_ALLOCATED(SharedJITStubSet);
     349public:
     350    SharedJITStubSet() = default;
     351
     352    struct Hash {
     353        struct Key {
     354            Key() = default;
     355
     356            Key(GPRReg baseGPR, GPRReg valueGPR, GPRReg extraGPR, GPRReg stubInfoGPR, RegisterSet usedRegisters, PolymorphicAccessJITStubRoutine* wrapped)
     357                : m_wrapped(wrapped)
     358                , m_baseGPR(baseGPR)
     359                , m_valueGPR(valueGPR)
     360                , m_extraGPR(extraGPR)
     361                , m_stubInfoGPR(stubInfoGPR)
     362                , m_usedRegisters(usedRegisters)
     363            { }
     364
     365            Key(WTF::HashTableDeletedValueType)
     366                : m_wrapped(bitwise_cast<PolymorphicAccessJITStubRoutine*>(static_cast<uintptr_t>(1)))
     367            { }
     368
     369            bool isHashTableDeletedValue() const { return m_wrapped == bitwise_cast<PolymorphicAccessJITStubRoutine*>(static_cast<uintptr_t>(1)); }
     370
     371            friend bool operator==(const Key& a, const Key& b)
     372            {
     373                return a.m_wrapped == b.m_wrapped
     374                    && a.m_baseGPR == b.m_baseGPR
     375                    && a.m_valueGPR == b.m_valueGPR
     376                    && a.m_extraGPR == b.m_extraGPR
     377                    && a.m_stubInfoGPR == b.m_stubInfoGPR
     378                    && a.m_usedRegisters == b.m_usedRegisters;
     379            }
     380
     381            PolymorphicAccessJITStubRoutine* m_wrapped { nullptr };
     382            GPRReg m_baseGPR;
     383            GPRReg m_valueGPR;
     384            GPRReg m_extraGPR;
     385            GPRReg m_stubInfoGPR;
     386            RegisterSet m_usedRegisters;
     387        };
     388
     389        using KeyTraits = SimpleClassHashTraits<Key>;
     390
     391        static unsigned hash(const Key& p)
     392        {
     393            if (!p.m_wrapped)
     394                return 1;
     395            return p.m_wrapped->hash();
     396        }
     397
     398        static bool equal(const Key& a, const Key& b)
     399        {
     400            return a == b;
     401        }
     402
     403        static constexpr bool safeToCompareToEmptyOrDeleted = false;
     404    };
     405
     406    struct Searcher {
     407        struct Translator {
     408            static unsigned hash(const Searcher& searcher)
     409            {
     410                return PolymorphicAccessJITStubRoutine::computeHash(searcher.m_cases, searcher.m_weakStructures);
     411            }
     412
     413            static bool equal(const Hash::Key a, const Searcher& b)
     414            {
     415                if (a.m_baseGPR == b.m_baseGPR
     416                    && a.m_valueGPR == b.m_valueGPR
     417                    && a.m_extraGPR == b.m_extraGPR
     418                    && a.m_stubInfoGPR == b.m_stubInfoGPR
     419                    && a.m_usedRegisters == b.m_usedRegisters) {
     420                    // FIXME: The ordering of cases does not matter for sharing capabilities.
     421                    // We can potentially increase success rate by making this comparison / hashing non ordering sensitive.
     422                    const auto& aCases = a.m_wrapped->cases();
     423                    const auto& bCases = b.m_cases;
     424                    if (aCases.size() != bCases.size())
     425                        return false;
     426                    for (unsigned index = 0; index < bCases.size(); ++index) {
     427                        if (!AccessCase::canBeShared(*aCases[index], *bCases[index]))
     428                            return false;
     429                    }
     430                    const auto& aWeak = a.m_wrapped->weakStructures();
     431                    const auto& bWeak = b.m_weakStructures;
     432                    if (aWeak.size() != bWeak.size())
     433                        return false;
     434                    for (unsigned i = 0, size = aWeak.size(); i < size; ++i) {
     435                        if (aWeak[i] != bWeak[i])
     436                            return false;
     437                    }
     438                    return true;
     439                }
     440                return false;
     441            }
     442        };
     443
     444        GPRReg m_baseGPR;
     445        GPRReg m_valueGPR;
     446        GPRReg m_extraGPR;
     447        GPRReg m_stubInfoGPR;
     448        RegisterSet m_usedRegisters;
     449        const FixedVector<RefPtr<AccessCase>>& m_cases;
     450        const FixedVector<StructureID>& m_weakStructures;
     451    };
     452
     453    struct PointerTranslator {
     454        static unsigned hash(PolymorphicAccessJITStubRoutine* stub)
     455        {
     456            return stub->hash();
     457        }
     458
     459        static bool equal(const Hash::Key& key, PolymorphicAccessJITStubRoutine* stub)
     460        {
     461            return key.m_wrapped == stub;
     462        }
     463    };
     464
     465    void add(Hash::Key&& key)
     466    {
     467        m_stubs.add(WTFMove(key));
     468    }
     469
     470    void remove(PolymorphicAccessJITStubRoutine* stub)
     471    {
     472        auto iter = m_stubs.find<PointerTranslator>(stub);
     473        if (iter != m_stubs.end())
     474            m_stubs.remove(iter);
     475    }
     476
     477    PolymorphicAccessJITStubRoutine* find(const Searcher& searcher)
     478    {
     479        auto entry = m_stubs.find<SharedJITStubSet::Searcher::Translator>(searcher);
     480        if (entry != m_stubs.end())
     481            return entry->m_wrapped;
     482        return nullptr;
     483    }
     484
     485private:
     486    HashSet<Hash::Key, Hash, Hash::KeyTraits> m_stubs;
     487};
     488
    318489} // namespace JSC
    319490

  • trunk/Source/JavaScriptCore/bytecode/ByValInfo.h

    r273138 r278656  
    245245    }
    246246
    247     void setUp(CodeLocationJump<JSInternalPtrTag> notIndexJump, CodeLocationJump<JSInternalPtrTag> badTypeJump, CodeLocationLabel<ExceptionHandlerPtrTag> exceptionHandler, JITArrayMode arrayMode, ArrayProfile* arrayProfile, CodeLocationLabel<JSInternalPtrTag> doneTarget, CodeLocationLabel<JSInternalPtrTag> badTypeNextHotPathTarget, CodeLocationLabel<JSInternalPtrTag> slowPathTarget)
     247    void setUp(CodeLocationLabel<ExceptionHandlerPtrTag> exceptionHandler, JITArrayMode arrayMode, ArrayProfile* arrayProfile, CodeLocationLabel<JSInternalPtrTag> doneTarget, CodeLocationLabel<JSInternalPtrTag> badTypeNextHotPathTarget, CodeLocationLabel<JSInternalPtrTag> slowPathTarget)
    248248    {
    249         this->notIndexJump = notIndexJump;
    250         this->badTypeJump = badTypeJump;
    251249        this->exceptionHandler = exceptionHandler;
    252250        this->doneTarget = doneTarget;
    … …  
    263261    DECLARE_VISIT_AGGREGATE;
    264262
    265     CodeLocationJump<JSInternalPtrTag> notIndexJump;
    266     CodeLocationJump<JSInternalPtrTag> badTypeJump;
     263    static ptrdiff_t offsetOfSlowOperation() { return OBJECT_OFFSETOF(ByValInfo, m_slowOperation); }
     264    static ptrdiff_t offsetOfNotIndexJumpTarget() { return OBJECT_OFFSETOF(ByValInfo, m_notIndexJumpTarget); }
     265    static ptrdiff_t offsetOfBadTypeJumpTarget() { return OBJECT_OFFSETOF(ByValInfo, m_badTypeJumpTarget); }
     266
     267    FunctionPtr<OperationPtrTag> m_slowOperation;
     268
     269    union {
     270        CodeLocationLabel<JITStubRoutinePtrTag> m_notIndexJumpTarget;
     271        CodeLocationJump<JSInternalPtrTag> m_notIndexJump;
     272    };
     273    union {
     274        CodeLocationLabel<JITStubRoutinePtrTag> m_badTypeJumpTarget;
     275        CodeLocationJump<JSInternalPtrTag> m_badTypeJump;
     276    };
     277
    267278    CodeLocationLabel<ExceptionHandlerPtrTag> exceptionHandler;
    268279    CodeLocationLabel<JSInternalPtrTag> doneTarget;

  • trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.cpp

    r277838 r278656  
    421421    if (isTailCall()) {
    422422        RELEASE_ASSERT(fastPathStart());
    423         CCallHelpers::emitJITCodeOver(fastPathStart(), [&] (CCallHelpers& jit) {
     423        CCallHelpers::emitJITCodeOver(fastPathStart(), scopedLambda<void(CCallHelpers&)>([&](CCallHelpers& jit) {
    424424            auto jump = jit.jump();
    425425            jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
    426426                linkBuffer.link(jump, slowPathStart());
    427427            });
    428         }, "initialize direct call");
     428        }), "initialize direct call");
    429429    } else
    430430        MacroAssembler::repatchNearCall(u.codeIC.m_callLocation, slowPathStart());
    … …  
    437437    if (isTailCall()) {
    438438        RELEASE_ASSERT(fastPathStart());
    439         CCallHelpers::emitJITCodeOver(fastPathStart(), [&] (CCallHelpers& jit) {
     439        CCallHelpers::emitJITCodeOver(fastPathStart(), scopedLambda<void(CCallHelpers&)>([&](CCallHelpers& jit) {
    440440            // We reserved this many bytes for the jump at fastPathStart(). Make that
    441441            // code nops now so we fall through to the jump to the fast path.
    442             jit.emitNops(CCallHelpers::patchableJumpSize()); 
    443         }, "Setting direct call target");
     442            jit.emitNops(CCallHelpers::patchableJumpSize());
     443        }), "Setting direct call target");
    444444    }
    445445

  • trunk/Source/JavaScriptCore/bytecode/CodeBlock.h

    r278576 r278656  
    424424        return jitType() == JITType::BaselineJIT;
    425425    }
     426
     427    bool useDataIC() const
     428    {
     429        return JITCode::useDataIC(jitType());
     430    }
    426431   
    427432#if ENABLE(JIT)

  • trunk/Source/JavaScriptCore/bytecode/GetterSetterAccessCase.cpp

    r278253 r278656  
    4848}
    4949
    50 std::unique_ptr<AccessCase> GetterSetterAccessCase::create(
     50Ref<AccessCase> GetterSetterAccessCase::create(
    5151    VM& vm, JSCell* owner, AccessType type, CacheableIdentifier identifier, PropertyOffset offset, Structure* structure, const ObjectPropertyConditionSet& conditionSet,
    5252    bool viaProxy, WatchpointSet* additionalSet, FunctionPtr<CustomAccessorPtrTag> customGetter, JSObject* customSlotBase,
    … …  
    6262    };
    6363
    64     std::unique_ptr<GetterSetterAccessCase> result(new GetterSetterAccessCase(vm, owner, type, identifier, offset, structure, conditionSet, viaProxy, additionalSet, customSlotBase, WTFMove(prototypeAccessChain)));
     64    auto result = adoptRef(*new GetterSetterAccessCase(vm, owner, type, identifier, offset, structure, conditionSet, viaProxy, additionalSet, customSlotBase, WTFMove(prototypeAccessChain)));
    6565    result->m_domAttribute = domAttribute;
    6666    result->m_customAccessor = customGetter ? FunctionPtr<CustomAccessorPtrTag>(customGetter) : nullptr;
    … …  
    6868}
    6969
    70 std::unique_ptr<AccessCase> GetterSetterAccessCase::create(VM& vm, JSCell* owner, AccessType type, Structure* structure, CacheableIdentifier identifier, PropertyOffset offset,
     70Ref<AccessCase> GetterSetterAccessCase::create(VM& vm, JSCell* owner, AccessType type, Structure* structure, CacheableIdentifier identifier, PropertyOffset offset,
    7171    const ObjectPropertyConditionSet& conditionSet, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain, bool viaProxy,
    7272    FunctionPtr<CustomAccessorPtrTag> customSetter, JSObject* customSlotBase)
    7373{
    7474    ASSERT(type == Setter || type == CustomValueSetter || type == CustomAccessorSetter);
    75     std::unique_ptr<GetterSetterAccessCase> result(new GetterSetterAccessCase(vm, owner, type, identifier, offset, structure, conditionSet, viaProxy, nullptr, customSlotBase, WTFMove(prototypeAccessChain)));
     75    auto result = adoptRef(*new GetterSetterAccessCase(vm, owner, type, identifier, offset, structure, conditionSet, viaProxy, nullptr, customSlotBase, WTFMove(prototypeAccessChain)));
    7676    result->m_customAccessor = customSetter ? FunctionPtr<CustomAccessorPtrTag>(customSetter) : nullptr;
    7777    return result;
    … …  
    9292}
    9393
    94 std::unique_ptr<AccessCase> GetterSetterAccessCase::clone() const
    95 {
    96     std::unique_ptr<GetterSetterAccessCase> result(new GetterSetterAccessCase(*this));
     94Ref<AccessCase> GetterSetterAccessCase::clone() const
     95{
     96    auto result = adoptRef(*new GetterSetterAccessCase(*this));
    9797    result->resetState();
    9898    return result;
    … …  
    140140    allocator.lock(stubInfo.baseRegs());
    141141    allocator.lock(valueRegs);
     142    if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     143        allocator.lock(stubInfo.m_stubInfoGPR);
    142144    allocator.lock(scratchGPR);
    143145
    … …  
    224226        registersToSpillForCCall.set(reg);
    225227    registersToSpillForCCall.exclude(RegisterSet::registersToNotSaveForCCall());
     228#if CPU(ARM64)
     229    CodeBlock* codeBlock = jit.codeBlock();
     230    if (codeBlock->useDataIC())
     231        registersToSpillForCCall.set(ARM64Registers::lr);
     232#endif
    226233
    227234    AccessCaseSnippetParams params(state.m_vm, WTFMove(regs), WTFMove(gpScratch), WTFMove(fpScratch));

  • trunk/Source/JavaScriptCore/bytecode/GetterSetterAccessCase.h

    r278253 r278656  
    5454    void emitDOMJITGetter(AccessGenerationState&, const DOMJIT::GetterSetter*, GPRReg baseForGetGPR);
    5555
    56     static std::unique_ptr<AccessCase> create(
     56    static Ref<AccessCase> create(
    5757        VM&, JSCell* owner, AccessType, CacheableIdentifier, PropertyOffset, Structure*,
    5858        const ObjectPropertyConditionSet&, bool viaProxy, WatchpointSet* additionalSet, FunctionPtr<CustomAccessorPtrTag> customGetter,
    5959        JSObject* customSlotBase, std::optional<DOMAttributeAnnotation>, RefPtr<PolyProtoAccessChain>&&);
    6060
    61     static std::unique_ptr<AccessCase> create(VM&, JSCell* owner, AccessType, Structure*, CacheableIdentifier, PropertyOffset,
     61    static Ref<AccessCase> create(VM&, JSCell* owner, AccessType, Structure*, CacheableIdentifier, PropertyOffset,
    6262        const ObjectPropertyConditionSet&, RefPtr<PolyProtoAccessChain>&&, bool viaProxy = false,
    6363        FunctionPtr<CustomAccessorPtrTag> customSetter = nullptr, JSObject* customSlotBase = nullptr);
    6464
    6565    void dumpImpl(PrintStream&, CommaPrinter&) const final;
    66     std::unique_ptr<AccessCase> clone() const final;
     66    Ref<AccessCase> clone() const final;
    6767
    6868    ~GetterSetterAccessCase() final;

  • trunk/Source/JavaScriptCore/bytecode/InlineAccess.cpp

    r277305 r278656  
    217217    allocator.lock(stubInfo.valueTagGPR);
    218218#endif
     219    if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     220        allocator.lock(stubInfo.m_stubInfoGPR);
    219221    GPRReg scratch = allocator.allocateScratchGPR();
    220222    if (allocator.didReuseRegisters())
    … …  
    380382}
    381383
    382 void InlineAccess::rewireStubAsJump(StructureStubInfo& stubInfo, CodeLocationLabel<JITStubRoutinePtrTag> target)
    383 {
    384     CCallHelpers jit;
    385 
    386     auto jump = jit.jump();
    387 
    388     // We don't need a nop sled here because nobody should be jumping into the middle of an IC.
    389     bool needsBranchCompaction = false;
    390     LinkBuffer linkBuffer(jit, stubInfo.start, jit.m_assembler.buffer().codeSize(), LinkBuffer::Profile::InlineCache, JITCompilationMustSucceed, needsBranchCompaction);
    391     RELEASE_ASSERT(linkBuffer.isValid());
    392     linkBuffer.link(jump, target);
    393 
    394     FINALIZE_CODE(linkBuffer, NoPtrTag, "InlineAccess: linking constant jump");
     384void InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(CodeBlock* codeBlock, StructureStubInfo& stubInfo, CodeLocationLabel<JITStubRoutinePtrTag> target)
     385{
     386    if (codeBlock->useDataIC()) {
     387        stubInfo.m_codePtr = target;
     388        return;
     389    }
     390
     391    CCallHelpers::emitJITCodeOver(stubInfo.start.retagged<JSInternalPtrTag>(), scopedLambda<void(CCallHelpers&)>([&](CCallHelpers& jit) {
     392        // We don't need a nop sled here because nobody should be jumping into the middle of an IC.
     393        auto jump = jit.jump();
     394        jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
     395            linkBuffer.link(jump, target);
     396        });
     397    }), "InlineAccess: linking constant jump");
     398}
     399
     400void InlineAccess::rewireStubAsJumpInAccess(CodeBlock* codeBlock, StructureStubInfo& stubInfo, CodeLocationLabel<JITStubRoutinePtrTag> target)
     401{
     402    if (codeBlock->useDataIC()) {
     403        // If it is not GetById-like-thing, we do not emit nop sled (e.g. GetByVal).
     404        // The code is already an indirect jump, and only thing we should do is replacing m_codePtr.
     405        if (stubInfo.hasConstantIdentifier) {
     406            // If m_codePtr is pointing to stubInfo.slowPathStartLocation, this means that InlineAccess code is not a stub one.
     407            // We rewrite this with the stub-based dispatching code once, and continue using it until we reset the code.
     408            if (stubInfo.m_codePtr.executableAddress() == stubInfo.slowPathStartLocation.executableAddress()) {
     409                CCallHelpers::emitJITCodeOver(stubInfo.start.retagged<JSInternalPtrTag>(), scopedLambda<void(CCallHelpers&)>([&](CCallHelpers& jit) {
     410                    jit.move(CCallHelpers::TrustedImmPtr(&stubInfo), stubInfo.m_stubInfoGPR);
     411                    jit.call(CCallHelpers::Address(stubInfo.m_stubInfoGPR, StructureStubInfo::offsetOfCodePtr()), JITStubRoutinePtrTag);
     412                    auto jump = jit.jump();
     413                    auto doneLocation = stubInfo.doneLocation;
     414                    jit.addLinkTask([=](LinkBuffer& linkBuffer) {
     415                        linkBuffer.link(jump, doneLocation);
     416                    });
     417                }), "InlineAccess: linking stub call");
     418            }
     419        }
     420        stubInfo.m_codePtr = target;
     421        return;
     422    }
     423
     424    CCallHelpers::emitJITCodeOver(stubInfo.start.retagged<JSInternalPtrTag>(), scopedLambda<void(CCallHelpers&)>([&](CCallHelpers& jit) {
     425        // We don't need a nop sled here because nobody should be jumping into the middle of an IC.
     426        auto jump = jit.jump();
     427        jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
     428            linkBuffer.link(jump, target);
     429        });
     430    }), "InlineAccess: linking constant jump");
     431}
     432
     433void InlineAccess::resetStubAsJumpInAccess(CodeBlock*, StructureStubInfo& stubInfo)
     434{
     435    CCallHelpers::emitJITCodeOver(stubInfo.start.retagged<JSInternalPtrTag>(), scopedLambda<void(CCallHelpers&)>([&](CCallHelpers& jit) {
     436        // We don't need a nop sled here because nobody should be jumping into the middle of an IC.
     437        auto jump = jit.jump();
     438        auto slowPathStartLocation = stubInfo.slowPathStartLocation;
     439        jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
     440            linkBuffer.link(jump, slowPathStartLocation);
     441        });
     442    }), "InlineAccess: linking constant jump");
     443}
     444
     445void InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess(CodeBlock* codeBlock, StructureStubInfo& stubInfo)
     446{
     447    rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, stubInfo.slowPathStartLocation);
    395448}
    396449

  • trunk/Source/JavaScriptCore/bytecode/InlineAccess.h

    r242252 r278656  
    105105    static bool isCacheableStringLength(StructureStubInfo&);
    106106    static bool generateArrayLength(StructureStubInfo&, JSArray*);
    107     static void rewireStubAsJump(StructureStubInfo&, CodeLocationLabel<JITStubRoutinePtrTag>);
    108107    static bool generateSelfInAccess(StructureStubInfo&, Structure*);
    109108    static bool generateStringLength(StructureStubInfo&);
     109
     110    static void rewireStubAsJumpInAccessNotUsingInlineAccess(CodeBlock*, StructureStubInfo&, CodeLocationLabel<JITStubRoutinePtrTag>);
     111    static void rewireStubAsJumpInAccess(CodeBlock*, StructureStubInfo&, CodeLocationLabel<JITStubRoutinePtrTag>);
     112    static void resetStubAsJumpInAccessNotUsingInlineAccess(CodeBlock*, StructureStubInfo&);
     113    static void resetStubAsJumpInAccess(CodeBlock*, StructureStubInfo&);
    110114
    111115    // This is helpful when determining the size of an IC on

  • trunk/Source/JavaScriptCore/bytecode/InstanceOfAccessCase.cpp

    r261755 r278656  
    3333namespace JSC {
    3434
    35 std::unique_ptr<AccessCase> InstanceOfAccessCase::create(
     35Ref<AccessCase> InstanceOfAccessCase::create(
    3636    VM& vm, JSCell* owner, AccessType accessType, Structure* structure,
    3737    const ObjectPropertyConditionSet& conditionSet, JSObject* prototype)
    3838{
    39     return std::unique_ptr<AccessCase>(new InstanceOfAccessCase(vm, owner, accessType, structure, conditionSet, prototype));
     39    return adoptRef(*new InstanceOfAccessCase(vm, owner, accessType, structure, conditionSet, prototype));
    4040}
    4141
    … …  
    4646}
    4747
    48 std::unique_ptr<AccessCase> InstanceOfAccessCase::clone() const
     48Ref<AccessCase> InstanceOfAccessCase::clone() const
    4949{
    50     std::unique_ptr<InstanceOfAccessCase> result(new InstanceOfAccessCase(*this));
     50    auto result = adoptRef(*new InstanceOfAccessCase(*this));
    5151    result->resetState();
    5252    return result;

  • trunk/Source/JavaScriptCore/bytecode/InstanceOfAccessCase.h

    r261567 r278656  
    3535public:
    3636    using Base = AccessCase;
     37    friend class AccessCase;
    3738   
    38     static std::unique_ptr<AccessCase> create(
     39    static Ref<AccessCase> create(
    3940        VM&, JSCell*, AccessType, Structure*, const ObjectPropertyConditionSet&,
    4041        JSObject* prototype);
    … …  
    4344   
    4445    void dumpImpl(PrintStream&, CommaPrinter&) const final;
    45     std::unique_ptr<AccessCase> clone() const final;
     46    Ref<AccessCase> clone() const final;
    4647   
    4748    ~InstanceOfAccessCase() final;

  • trunk/Source/JavaScriptCore/bytecode/IntrinsicGetterAccessCase.cpp

    r275490 r278656  
    4040}
    4141
    42 std::unique_ptr<AccessCase> IntrinsicGetterAccessCase::create(VM& vm, JSCell* owner, CacheableIdentifier identifier, PropertyOffset offset, Structure* structure, const ObjectPropertyConditionSet& conditionSet, JSFunction* intrinsicFunction, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain)
     42Ref<AccessCase> IntrinsicGetterAccessCase::create(VM& vm, JSCell* owner, CacheableIdentifier identifier, PropertyOffset offset, Structure* structure, const ObjectPropertyConditionSet& conditionSet, JSFunction* intrinsicFunction, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain)
    4343{
    44     return std::unique_ptr<AccessCase>(new IntrinsicGetterAccessCase(vm, owner, identifier, offset, structure, conditionSet, intrinsicFunction, WTFMove(prototypeAccessChain)));
     44    return adoptRef(*new IntrinsicGetterAccessCase(vm, owner, identifier, offset, structure, conditionSet, intrinsicFunction, WTFMove(prototypeAccessChain)));
    4545}
    4646
    … …  
    4949}
    5050
    51 std::unique_ptr<AccessCase> IntrinsicGetterAccessCase::clone() const
     51Ref<AccessCase> IntrinsicGetterAccessCase::clone() const
    5252{
    53     std::unique_ptr<IntrinsicGetterAccessCase> result(new IntrinsicGetterAccessCase(*this));
     53    auto result = adoptRef(*new IntrinsicGetterAccessCase(*this));
    5454    result->resetState();
    5555    return result;

  • trunk/Source/JavaScriptCore/bytecode/IntrinsicGetterAccessCase.h

    r275490 r278656  
    4343    void emitIntrinsicGetter(AccessGenerationState&);
    4444
    45     static std::unique_ptr<AccessCase> create(VM&, JSCell*, CacheableIdentifier, PropertyOffset, Structure*, const ObjectPropertyConditionSet&, JSFunction* intrinsicFunction, RefPtr<PolyProtoAccessChain>&&);
     45    static Ref<AccessCase> create(VM&, JSCell*, CacheableIdentifier, PropertyOffset, Structure*, const ObjectPropertyConditionSet&, JSFunction* intrinsicFunction, RefPtr<PolyProtoAccessChain>&&);
    4646
    47     std::unique_ptr<AccessCase> clone() const final;
     47    Ref<AccessCase> clone() const final;
    4848
    4949    ~IntrinsicGetterAccessCase() final;

  • trunk/Source/JavaScriptCore/bytecode/ModuleNamespaceAccessCase.cpp

    r261755 r278656  
    4646}
    4747
    48 std::unique_ptr<AccessCase> ModuleNamespaceAccessCase::create(VM& vm, JSCell* owner, CacheableIdentifier identifier, JSModuleNamespaceObject* moduleNamespaceObject, JSModuleEnvironment* moduleEnvironment, ScopeOffset scopeOffset)
     48Ref<AccessCase> ModuleNamespaceAccessCase::create(VM& vm, JSCell* owner, CacheableIdentifier identifier, JSModuleNamespaceObject* moduleNamespaceObject, JSModuleEnvironment* moduleEnvironment, ScopeOffset scopeOffset)
    4949{
    50     return std::unique_ptr<AccessCase>(new ModuleNamespaceAccessCase(vm, owner, identifier, moduleNamespaceObject, moduleEnvironment, scopeOffset));
     50    return adoptRef(*new ModuleNamespaceAccessCase(vm, owner, identifier, moduleNamespaceObject, moduleEnvironment, scopeOffset));
    5151}
    5252
    … …  
    5555}
    5656
    57 std::unique_ptr<AccessCase> ModuleNamespaceAccessCase::clone() const
     57Ref<AccessCase> ModuleNamespaceAccessCase::clone() const
    5858{
    59     std::unique_ptr<ModuleNamespaceAccessCase> result(new ModuleNamespaceAccessCase(*this));
     59    auto result = adoptRef(*new ModuleNamespaceAccessCase(*this));
    6060    result->resetState();
    6161    return result;

  • trunk/Source/JavaScriptCore/bytecode/ModuleNamespaceAccessCase.h

    r261567 r278656  
    4545    ScopeOffset scopeOffset() const { return m_scopeOffset; }
    4646
    47     static std::unique_ptr<AccessCase> create(VM&, JSCell* owner, CacheableIdentifier, JSModuleNamespaceObject*, JSModuleEnvironment*, ScopeOffset);
     47    static Ref<AccessCase> create(VM&, JSCell* owner, CacheableIdentifier, JSModuleNamespaceObject*, JSModuleEnvironment*, ScopeOffset);
    4848
    49     std::unique_ptr<AccessCase> clone() const final;
     49    Ref<AccessCase> clone() const final;
    5050
    5151    void emit(AccessGenerationState&, MacroAssembler::JumpList& fallThrough);

  • trunk/Source/JavaScriptCore/bytecode/ObjectPropertyConditionSet.h

    r278253 r278656  
    2929#include <wtf/FastMalloc.h>
    3030#include <wtf/FixedVector.h>
     31#include <wtf/Hasher.h>
    3132#include <wtf/Vector.h>
    3233
    … …  
    8990            return nullptr;
    9091        return m_data->m_vector.end();
     92    }
     93
     94    unsigned hash() const
     95    {
     96        Hasher hasher;
     97        for (auto& condition : *this)
     98            add(hasher, condition.hash());
     99        return hasher.hash();
     100    }
     101
     102    friend bool operator==(const ObjectPropertyConditionSet& lhs, const ObjectPropertyConditionSet& rhs)
     103    {
     104        if (lhs.size() != rhs.size())
     105            return false;
     106        auto liter = lhs.begin();
     107        auto riter = rhs.begin();
     108        for (; liter != lhs.end(); ++liter, ++riter) {
     109            if (!(*liter == *riter))
     110                return false;
     111        }
     112        return true;
     113    }
     114
     115    friend bool operator!=(const ObjectPropertyConditionSet& lhs, const ObjectPropertyConditionSet& rhs)
     116    {
     117        return !(lhs == rhs);
    91118    }
    92119   

  • trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp

    r277669 r278656  
    6060}
    6161
    62 void AccessGenerationState::installWatchpoint(const ObjectPropertyCondition& condition)
    63 {
    64     WatchpointsOnStructureStubInfo::ensureReferenceAndInstallWatchpoint(
    65         watchpoints, jit->codeBlock(), stubInfo, condition);
     62void AccessGenerationState::installWatchpoint(CodeBlock* codeBlock, const ObjectPropertyCondition& condition)
     63{
     64    WatchpointsOnStructureStubInfo::ensureReferenceAndInstallWatchpoint(watchpoints, codeBlock, stubInfo, condition);
    6665}
    6766
    … …  
    7473{
    7574    restoreScratch();
    76     success.append(jit->jump());
     75    if (jit->codeBlock()->useDataIC())
     76        jit->ret();
     77    else
     78        success.append(jit->jump());
    7779}
    7880
    … …  
    127129}
    128130
    129 auto AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions() -> SpillState
     131auto AccessGenerationState::preserveLiveRegistersToStackForCallWithoutExceptions(const RegisterSet& extra) -> SpillState
    130132{
    131133    RegisterSet liveRegisters = allocator->usedRegisters();
    132134    liveRegisters.exclude(calleeSaveRegisters());
     135    liveRegisters.merge(extra);
    133136
    134137    constexpr unsigned extraStackPadding = 0;
    … …  
    253256AccessGenerationResult PolymorphicAccess::addCases(
    254257    const GCSafeConcurrentJSLocker& locker, VM& vm, CodeBlock* codeBlock, StructureStubInfo& stubInfo,
    255     Vector<std::unique_ptr<AccessCase>, 2> originalCasesToAdd)
     258    Vector<RefPtr<AccessCase>, 2> originalCasesToAdd)
    256259{
    257260    SuperSamplerScope superSamplerScope(false);
    … …  
    269272   
    270273    // First ensure that the originalCasesToAdd doesn't contain duplicates.
    271     Vector<std::unique_ptr<AccessCase>> casesToAdd;
     274    Vector<RefPtr<AccessCase>> casesToAdd;
    272275    for (unsigned i = 0; i < originalCasesToAdd.size(); ++i) {
    273         std::unique_ptr<AccessCase> myCase = WTFMove(originalCasesToAdd[i]);
     276        RefPtr<AccessCase> myCase = WTFMove(originalCasesToAdd[i]);
    274277
    275278        // Add it only if it is not replaced by the subsequent cases in the list.
    … …  
    288291    }
    289292
    290     if (PolymorphicAccessInternal::verbose)
    291         dataLog("casesToAdd: ", listDump(casesToAdd), "\n");
     293    dataLogLnIf(PolymorphicAccessInternal::verbose, "casesToAdd: ", listDump(casesToAdd));
    292294
    293295    // If there aren't any cases to add, then fail on the grounds that there's no point to generating a
    … …  
    340342    }
    341343   
    342     if (PolymorphicAccessInternal::verbose)
    343         dataLog("After addCases: m_list: ", listDump(m_list), "\n");
     344    dataLogLnIf(PolymorphicAccessInternal::verbose, "After addCases: m_list: ", listDump(m_list));
    344345
    345346    return AccessGenerationResult::Buffered;
    … …  
    347348
    348349AccessGenerationResult PolymorphicAccess::addCase(
    349     const GCSafeConcurrentJSLocker& locker, VM& vm, CodeBlock* codeBlock, StructureStubInfo& stubInfo, std::unique_ptr<AccessCase> newAccess)
    350 {
    351     Vector<std::unique_ptr<AccessCase>, 2> newAccesses;
     350    const GCSafeConcurrentJSLocker& locker, VM& vm, CodeBlock* codeBlock, StructureStubInfo& stubInfo, Ref<AccessCase> newAccess)
     351{
     352    Vector<RefPtr<AccessCase>, 2> newAccesses;
    352353    newAccesses.append(WTFMove(newAccess));
    353354    return addCases(locker, vm, codeBlock, stubInfo, WTFMove(newAccesses));
    … …  
    360361            return false;
    361362    }
    362     for (const WriteBarrier<JSCell>& weakReference : m_weakReferences) {
    363         if (!vm.heap.isMarked(weakReference.get()))
    364             return false;
     363    if (m_stubRoutine) {
     364        for (StructureID weakReference : m_stubRoutine->weakStructures()) {
     365            Structure* structure = vm.getStructure(weakReference);
     366            if (!vm.heap.isMarked(structure))
     367                return false;
     368        }
    365369    }
    366370    return true;
    … …  
    420424    SuperSamplerScope superSamplerScope(false);
    421425   
    422     if (PolymorphicAccessInternal::verbose)
    423         dataLog("Regenerate with m_list: ", listDump(m_list), "\n");
     426    dataLogLnIf(PolymorphicAccessInternal::verbose, "Regenerate with m_list: ", listDump(m_list));
    424427
    425428    AccessGenerationState state(vm, globalObject, ecmaMode);
    … …  
    441444    unsigned dstIndex = 0;
    442445    while (srcIndex < m_list.size()) {
    443         std::unique_ptr<AccessCase> someCase = WTFMove(m_list[srcIndex++]);
     446        RefPtr<AccessCase> someCase = WTFMove(m_list[srcIndex++]);
    444447       
    445448        // If the case had been generated, then we have to keep the original in m_list in case we
    … …  
    492495        allocator.lock(stubInfo.v.thisTagGPR);
    493496#endif
     497    if (stubInfo.m_stubInfoGPR != InvalidGPRReg)
     498        allocator.lock(stubInfo.m_stubInfoGPR);
    494499
    495500    state.scratchGPR = allocator.allocateScratchGPR();
    … …  
    501506        }
    502507    }
    503    
    504     CCallHelpers jit(codeBlock);
    505     state.jit = &jit;
    506 
    507     state.preservedReusedRegisterState =
    508         allocator.preserveReusedRegistersByPushing(jit, ScratchRegisterAllocator::ExtraStackSpace::NoExtraSpace);
    509508
    510509    bool generatedFinalCode = false;
    … …  
    520519    }
    521520
    522     if (PolymorphicAccessInternal::verbose)
    523         dataLog("Optimized cases: ", listDump(cases), "\n");
    524    
     521    dataLogLnIf(PolymorphicAccessInternal::verbose, "Optimized cases: ", listDump(cases));
     522
     523    bool doesCalls = false;
     524    bool doesJSGetterSetterCalls = false;
     525    bool canBeShared = true;
     526    Vector<JSCell*> cellsToMark;
     527    FixedVector<RefPtr<AccessCase>> keys(cases.size());
     528    unsigned index = 0;
     529    for (auto& entry : cases) {
     530        doesCalls |= entry->doesCalls(vm, &cellsToMark);
     531        switch (entry->type()) {
     532        case AccessCase::Getter:
     533        case AccessCase::Setter:
     534            // Getter / Setter relies on stack-pointer adjustment, which is tied to the linked CodeBlock, which makes this code unshareable.
     535            canBeShared = false;
     536            doesJSGetterSetterCalls = true;
     537            break;
     538        case AccessCase::CustomValueGetter:
     539        case AccessCase::CustomAccessorGetter:
     540        case AccessCase::CustomValueSetter:
     541        case AccessCase::CustomAccessorSetter:
     542            // Custom getter / setter emits JSGlobalObject pointer, which is tied to the linked CodeBlock.
     543            canBeShared = false;
     544            break;
     545        default:
     546            break;
     547        }
     548        keys[index] = entry;
     549        ++index;
     550    }
     551    state.m_doesCalls = doesCalls;
     552    state.m_doesJSGetterSetterCalls = doesJSGetterSetterCalls;
     553
    525554    // At this point we're convinced that 'cases' contains the cases that we want to JIT now and we
    526555    // won't change that set anymore.
    527556   
    528557    bool allGuardedByStructureCheck = true;
    529     bool hasJSGetterSetterCall = false;
    530558    bool needsInt32PropertyCheck = false;
    531559    bool needsStringPropertyCheck = false;
    … …  
    543571        commit(locker, vm, state.watchpoints, codeBlock, stubInfo, *newCase);
    544572        allGuardedByStructureCheck &= newCase->guardedByStructureCheck(stubInfo);
    545         if (newCase->type() == AccessCase::Getter || newCase->type() == AccessCase::Setter)
    546             hasJSGetterSetterCall = true;
    547     }
    548 
     573        if (newCase->usesPolyProto())
     574            canBeShared = false;
     575    }
     576    if (needsSymbolPropertyCheck || needsStringPropertyCheck || needsInt32PropertyCheck)
     577        canBeShared = false;
     578
     579    auto finishCodeGeneration = [&](RefPtr<PolymorphicAccessJITStubRoutine>&& stub) {
     580        m_stubRoutine = WTFMove(stub);
     581        m_watchpoints = WTFMove(state.watchpoints);
     582        dataLogLnIf(PolymorphicAccessInternal::verbose, "Returning: ", m_stubRoutine->code());
     583
     584        m_list = WTFMove(cases);
     585        m_list.shrinkToFit();
     586
     587        AccessGenerationResult::Kind resultKind;
     588        if (m_list.size() >= Options::maxAccessVariantListSize() || generatedFinalCode)
     589            resultKind = AccessGenerationResult::GeneratedFinalCode;
     590        else
     591            resultKind = AccessGenerationResult::GeneratedNewCode;
     592
     593        return AccessGenerationResult(resultKind, m_stubRoutine->code().code());
     594    };
     595
     596    CCallHelpers jit(codeBlock);
     597    state.jit = &jit;
     598
     599    if (codeBlock->useDataIC()) {
     600        if (state.m_doesJSGetterSetterCalls) {
     601            // We have no guarantee that stack-pointer is the expected one. This is not a problem if we do not have JS getter / setter calls since stack-pointer is
     602            // a callee-save register in the C calling convension. However, our JS executable call does not save stack-pointer. So we are adjusting stack-pointer after
     603            // JS getter / setter calls. But this could be different from the initial stack-pointer, and makes PAC tagging broken.
     604            // To ensure PAC-tagging work, we first adjust stack-pointer to the appropriate one.
     605            jit.addPtr(CCallHelpers::TrustedImm32(codeBlock->stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, CCallHelpers::stackPointerRegister);
     606            jit.tagReturnAddress();
     607        } else
     608            jit.tagReturnAddress();
     609    }
     610
     611    state.preservedReusedRegisterState =
     612        allocator.preserveReusedRegistersByPushing(jit, ScratchRegisterAllocator::ExtraStackSpace::NoExtraSpace);
     613   
    549614    if (cases.isEmpty()) {
    550615        // This is super unlikely, but we make it legal anyway.
    … …  
    691756    CodeBlock* codeBlockThatOwnsExceptionHandlers = nullptr;
    692757    DisposableCallSiteIndex callSiteIndexForExceptionHandling;
    693     if (state.needsToRestoreRegistersIfException() && hasJSGetterSetterCall) {
     758    if (state.needsToRestoreRegistersIfException() && doesJSGetterSetterCalls) {
    694759        // Emit the exception handler.
    695760        // Note that this code is only reachable when doing genericUnwind from a pure JS getter/setter .
    … …  
    731796    }
    732797
     798    if (codeBlock->useDataIC()) {
     799        failure.link(&jit);
     800        // In ARM64, we do not push anything on stack specially.
     801        // So we can just jump to the slow-path even though this thunk is called (not jumped).
     802        // FIXME: We should tail call to the thunk which calls the slow path function.
     803        // And we should eliminate IC slow-path generation in BaselineJIT.
     804        jit.farJump(CCallHelpers::Address(stubInfo.m_stubInfoGPR, StructureStubInfo::offsetOfSlowPathStartLocation()), JITStubRoutinePtrTag);
     805    }
     806
     807    RefPtr<PolymorphicAccessJITStubRoutine> stub;
     808    FixedVector<StructureID> weakStructures(WTFMove(state.weakStructures));
     809    if (codeBlock->useDataIC() && canBeShared) {
     810        SharedJITStubSet::Searcher searcher {
     811            stubInfo.baseGPR,
     812            stubInfo.valueGPR,
     813            stubInfo.regs.thisGPR,
     814            stubInfo.m_stubInfoGPR,
     815            stubInfo.usedRegisters,
     816            keys,
     817            weakStructures,
     818        };
     819        stub = vm.m_sharedJITStubs->find(searcher);
     820        if (stub) {
     821            dataLogLnIf(PolymorphicAccessInternal::verbose, "Found existing code stub ", stub->code());
     822            return finishCodeGeneration(WTFMove(stub));
     823        }
     824    }
     825
    733826    LinkBuffer linkBuffer(jit, codeBlock, LinkBuffer::Profile::InlineCache, JITCompilationCanFail);
    734827    if (linkBuffer.didFailToAllocate()) {
    735         if (PolymorphicAccessInternal::verbose)
    736             dataLog("Did fail to allocate.\n");
     828        dataLogLnIf(PolymorphicAccessInternal::verbose, "Did fail to allocate.");
    737829        return AccessGenerationResult::GaveUp;
    738830    }
    … …  
    740832    CodeLocationLabel<JSInternalPtrTag> successLabel = stubInfo.doneLocation;
    741833
    742     linkBuffer.link(state.success, successLabel);
    743 
    744     linkBuffer.link(failure, stubInfo.slowPathStartLocation);
     834    if (codeBlock->useDataIC())
     835        ASSERT(state.success.empty());
     836    else {
     837        linkBuffer.link(state.success, successLabel);
     838        linkBuffer.link(failure, stubInfo.slowPathStartLocation);
     839    }
    745840   
    746     if (PolymorphicAccessInternal::verbose)
    747         dataLog(FullCodeOrigin(codeBlock, stubInfo.codeOrigin), ": Generating polymorphic access stub for ", listDump(cases), "\n");
     841    dataLogLnIf(PolymorphicAccessInternal::verbose, FullCodeOrigin(codeBlock, stubInfo.codeOrigin), ": Generating polymorphic access stub for ", listDump(cases));
    748842
    749843    MacroAssemblerCodeRef<JITStubRoutinePtrTag> code = FINALIZE_CODE_FOR(
    750844        codeBlock, linkBuffer, JITStubRoutinePtrTag,
    751845        "%s", toCString("Access stub for ", *codeBlock, " ", stubInfo.codeOrigin, " with return point ", successLabel, ": ", listDump(cases)).data());
    752 
    753     bool doesCalls = false;
    754     Vector<JSCell*> cellsToMark;
    755     for (auto& entry : cases)
    756         doesCalls |= entry->doesCalls(vm, &cellsToMark);
    757846   
    758     m_stubRoutine = createJITStubRoutine(code, vm, codeBlock, doesCalls, cellsToMark, WTFMove(state.m_callLinkInfos), codeBlockThatOwnsExceptionHandlers, callSiteIndexForExceptionHandling);
    759     m_watchpoints = WTFMove(state.watchpoints);
    760     if (!state.weakReferences.isEmpty())
    761         m_weakReferences = FixedVector<WriteBarrier<JSCell>>(WTFMove(state.weakReferences));
    762     if (PolymorphicAccessInternal::verbose)
    763         dataLog("Returning: ", code.code(), "\n");
    764    
    765     m_list = WTFMove(cases);
    766     m_list.shrinkToFit();
    767    
    768     AccessGenerationResult::Kind resultKind;
    769     if (m_list.size() >= Options::maxAccessVariantListSize() || generatedFinalCode)
    770         resultKind = AccessGenerationResult::GeneratedFinalCode;
    771     else
    772         resultKind = AccessGenerationResult::GeneratedNewCode;
    773    
    774     return AccessGenerationResult(resultKind, code.code());
     847    stub = createICJITStubRoutine(code, WTFMove(keys), WTFMove(weakStructures), vm, codeBlock, doesCalls, cellsToMark, WTFMove(state.m_callLinkInfos), codeBlockThatOwnsExceptionHandlers, callSiteIndexForExceptionHandling);
     848
     849    if (codeBlock->useDataIC()) {
     850        if (canBeShared)
     851            vm.m_sharedJITStubs->add(SharedJITStubSet::Hash::Key(stubInfo.baseGPR, stubInfo.valueGPR, stubInfo.regs.thisGPR, stubInfo.m_stubInfoGPR, stubInfo.usedRegisters, stub.get()));
     852    }
     853
     854    return finishCodeGeneration(WTFMove(stub));
    775855}
    776856

  • trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.h

    r277669 r278656  
    142142    // to call this method again for that PolymorphicAccess instance.
    143143    AccessGenerationResult addCases(
    144         const GCSafeConcurrentJSLocker&, VM&, CodeBlock*, StructureStubInfo&, Vector<std::unique_ptr<AccessCase>, 2>);
     144        const GCSafeConcurrentJSLocker&, VM&, CodeBlock*, StructureStubInfo&, Vector<RefPtr<AccessCase>, 2>);
    145145
    146146    AccessGenerationResult addCase(
    147         const GCSafeConcurrentJSLocker&, VM&, CodeBlock*, StructureStubInfo&, std::unique_ptr<AccessCase>);
     147        const GCSafeConcurrentJSLocker&, VM&, CodeBlock*, StructureStubInfo&, Ref<AccessCase>);
    148148   
    149149    AccessGenerationResult regenerate(const GCSafeConcurrentJSLocker&, VM&, JSGlobalObject*, CodeBlock*, ECMAMode, StructureStubInfo&);
    … …  
    180180    friend struct AccessGenerationState;
    181181   
    182     typedef Vector<std::unique_ptr<AccessCase>, 2> ListType;
     182    typedef Vector<RefPtr<AccessCase>, 2> ListType;
    183183   
    184184    void commit(
    … …  
    187187
    188188    ListType m_list;
    189     RefPtr<JITStubRoutine> m_stubRoutine;
     189    RefPtr<PolymorphicAccessJITStubRoutine> m_stubRoutine;
    190190    std::unique_ptr<WatchpointsOnStructureStubInfo> m_watchpoints;
    191     FixedVector<WriteBarrier<JSCell>> m_weakReferences;
    192191};
    193192
    … …  
    197196        , m_globalObject(globalObject)
    198197        , m_ecmaMode(ecmaMode)
     198        , m_doesJSGetterSetterCalls(false)
     199        , m_doesCalls(false)
    199200        , m_calculatedRegistersForCallAndExceptionHandling(false)
    200201        , m_needsToRestoreRegistersIfException(false)
    … …  
    224225    ECMAMode m_ecmaMode { ECMAMode::sloppy() };
    225226    std::unique_ptr<WatchpointsOnStructureStubInfo> watchpoints;
    226     Vector<WriteBarrier<JSCell>> weakReferences;
     227    Vector<StructureID> weakStructures;
    227228    Bag<CallLinkInfo> m_callLinkInfos;
    228 
    229     void installWatchpoint(const ObjectPropertyCondition&);
     229    bool m_doesJSGetterSetterCalls : 1;
     230    bool m_doesCalls : 1;
     231
     232    void installWatchpoint(CodeBlock*, const ObjectPropertyCondition&);
    230233
    231234    void restoreScratch();
    … …  
    249252
    250253    SpillState preserveLiveRegistersToStackForCall(const RegisterSet& extra = { });
    251     SpillState preserveLiveRegistersToStackForCallWithoutExceptions();
     254    SpillState preserveLiveRegistersToStackForCallWithoutExceptions(const RegisterSet& extra = { });
    252255
    253256    void restoreLiveRegistersFromStackForCallWithThrownException(const SpillState&);

  • trunk/Source/JavaScriptCore/bytecode/ProxyableAccessCase.cpp

    r275490 r278656  
    3939}
    4040
    41 std::unique_ptr<AccessCase> ProxyableAccessCase::create(VM& vm, JSCell* owner, AccessType type, CacheableIdentifier identifier, PropertyOffset offset, Structure* structure, const ObjectPropertyConditionSet& conditionSet, bool viaProxy, WatchpointSet* additionalSet, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain)
     41Ref<AccessCase> ProxyableAccessCase::create(VM& vm, JSCell* owner, AccessType type, CacheableIdentifier identifier, PropertyOffset offset, Structure* structure, const ObjectPropertyConditionSet& conditionSet, bool viaProxy, WatchpointSet* additionalSet, RefPtr<PolyProtoAccessChain>&& prototypeAccessChain)
    4242{
    4343    ASSERT(type == Load || type == Miss || type == GetGetter || type == Replace);
    44     return std::unique_ptr<AccessCase>(new ProxyableAccessCase(vm, owner, type, identifier, offset, structure, conditionSet, viaProxy, additionalSet, WTFMove(prototypeAccessChain)));
     44    return adoptRef(*new ProxyableAccessCase(vm, owner, type, identifier, offset, structure, conditionSet, viaProxy, additionalSet, WTFMove(prototypeAccessChain)));
    4545}
    4646
    … …  
    4949}
    5050
    51 std::unique_ptr<AccessCase> ProxyableAccessCase::clone() const
     51Ref<AccessCase> ProxyableAccessCase::clone() const
    5252{
    53     std::unique_ptr<ProxyableAccessCase> result(new ProxyableAccessCase(*this));
     53    auto result = adoptRef(*new ProxyableAccessCase(*this));
    5454    result->resetState();
    5555    return result;

  • trunk/Source/JavaScriptCore/bytecode/ProxyableAccessCase.h

    r275490 r278656  
    3838    WatchpointSet* additionalSet() const override { return m_additionalSet.get(); }
    3939
    40     static std::unique_ptr<AccessCase> create(VM&, JSCell*, AccessType, CacheableIdentifier, PropertyOffset, Structure*, const ObjectPropertyConditionSet& = ObjectPropertyConditionSet(),
     40    static Ref<AccessCase> create(VM&, JSCell*, AccessType, CacheableIdentifier, PropertyOffset, Structure*, const ObjectPropertyConditionSet& = ObjectPropertyConditionSet(),
    4141        bool viaProxy = false, WatchpointSet* additionalSet = nullptr, RefPtr<PolyProtoAccessChain>&& = nullptr);
    4242
    4343    void dumpImpl(PrintStream&, CommaPrinter&) const override;
    44     std::unique_ptr<AccessCase> clone() const override;
     44    Ref<AccessCase> clone() const override;
    4545
    4646    ~ProxyableAccessCase() override;

  • trunk/Source/JavaScriptCore/bytecode/StructureStubInfo.cpp

    r278445 r278656  
    5353    , propertyIsSymbol(false)
    5454{
     55    regs.thisGPR = InvalidGPRReg;
    5556}
    5657
    … …  
    140141
    141142AccessGenerationResult StructureStubInfo::addAccessCase(
    142     const GCSafeConcurrentJSLocker& locker, JSGlobalObject* globalObject, CodeBlock* codeBlock, ECMAMode ecmaMode, CacheableIdentifier ident, std::unique_ptr<AccessCase> accessCase)
     143    const GCSafeConcurrentJSLocker& locker, JSGlobalObject* globalObject, CodeBlock* codeBlock, ECMAMode ecmaMode, CacheableIdentifier ident, RefPtr<AccessCase> accessCase)
    143144{
    144145    checkConsistency();
    … …  
    156157       
    157158        if (m_cacheType == CacheType::Stub) {
    158             result = u.stub->addCase(locker, vm, codeBlock, *this, WTFMove(accessCase));
     159            result = u.stub->addCase(locker, vm, codeBlock, *this, accessCase.releaseNonNull());
    159160           
    160161            if (StructureStubInfoInternal::verbose)
    … …  
    171172            std::unique_ptr<PolymorphicAccess> access = makeUnique<PolymorphicAccess>();
    172173           
    173             Vector<std::unique_ptr<AccessCase>, 2> accessCases;
    174            
    175             std::unique_ptr<AccessCase> previousCase = AccessCase::fromStructureStubInfo(vm, codeBlock, ident, *this);
     174            Vector<RefPtr<AccessCase>, 2> accessCases;
     175           
     176            auto previousCase = AccessCase::fromStructureStubInfo(vm, codeBlock, ident, *this);
    176177            if (previousCase)
    177178                accessCases.append(WTFMove(previousCase));
    … …  
    280281        break;
    281282    case AccessType::InstanceOf:
    282         resetInstanceOf(*this);
     283        resetInstanceOf(codeBlock, *this);
    283284        break;
    284285    case AccessType::DeleteByID:

  • trunk/Source/JavaScriptCore/bytecode/StructureStubInfo.h

    r278445 r278656  
    8989    void initInByIdSelf(const ConcurrentJSLockerBase&, CodeBlock*, Structure* baseObjectStructure, PropertyOffset, CacheableIdentifier);
    9090
    91     AccessGenerationResult addAccessCase(const GCSafeConcurrentJSLocker&, JSGlobalObject*, CodeBlock*, ECMAMode, CacheableIdentifier, std::unique_ptr<AccessCase>);
     91    AccessGenerationResult addAccessCase(const GCSafeConcurrentJSLocker&, JSGlobalObject*, CodeBlock*, ECMAMode, CacheableIdentifier, RefPtr<AccessCase>);
    9292
    9393    void reset(const ConcurrentJSLockerBase&, CodeBlock*);
    … …  
    133133        ASSERT(inlineSize >= 0);
    134134        return inlineSize;
    135     }
    136 
    137     CodeLocationJump<JSInternalPtrTag> patchableJump()
    138     {
    139         ASSERT(accessType == AccessType::InstanceOf);
    140         return start.jumpAtOffset<JSInternalPtrTag>(0);
    141135    }
    142136
    … …  
    346340    CodeLocationLabel<JITStubRoutinePtrTag> start; // This is either the start of the inline IC for *byId caches. or the location of patchable jump for 'instanceof' caches.
    347341    CodeLocationLabel<JSInternalPtrTag> doneLocation;
    348     CodeLocationCall<JSInternalPtrTag> slowPathCallLocation;
    349342    CodeLocationLabel<JITStubRoutinePtrTag> slowPathStartLocation;
    350343
     344    union {
     345        CodeLocationCall<JSInternalPtrTag> m_slowPathCallLocation;
     346        FunctionPtr<OperationPtrTag> m_slowOperation;
     347    };
     348
     349    MacroAssemblerCodePtr<JITStubRoutinePtrTag> m_codePtr;
     350
     351    static ptrdiff_t offsetOfCodePtr() { return OBJECT_OFFSETOF(StructureStubInfo, m_codePtr); }
     352    static ptrdiff_t offsetOfSlowPathStartLocation() { return OBJECT_OFFSETOF(StructureStubInfo, slowPathStartLocation); }
     353    static ptrdiff_t offsetOfSlowOperation() { return OBJECT_OFFSETOF(StructureStubInfo, m_slowOperation); }
     354
    351355    RegisterSet usedRegisters;
    352356
    353     GPRReg baseGPR;
    354     GPRReg valueGPR;
     357    GPRReg baseGPR { InvalidGPRReg };
     358    GPRReg valueGPR { InvalidGPRReg };
    355359    union {
    356360        GPRReg thisGPR;
    … …  
    359363        GPRReg brandGPR;
    360364    } regs;
     365    GPRReg m_stubInfoGPR { InvalidGPRReg };
    361366#if USE(JSVALUE32_64)
    362367    GPRReg valueTagGPR;

  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h

    r278445 r278656  
    155155        return functionCall;
    156156    }
     157
     158    void appendCall(CCallHelpers::Address address)
     159    {
     160        call(address, OperationPtrTag);
     161    }
    157162   
    158163    void exceptionCheck();

  • trunk/Source/JavaScriptCore/dfg/DFGSlowPathGenerator.h

    r278213 r278656  
    265265}
    266266
     267template<typename JumpType, typename FunctionType, typename ResultType, typename... Arguments>
     268class CallResultAndArgumentsSlowPathICGenerator final : public CallSlowPathGenerator<JumpType, ResultType> {
     269public:
     270    CallResultAndArgumentsSlowPathICGenerator(
     271        JumpType from, SpeculativeJIT* jit, StructureStubInfo* stubInfo, GPRReg stubInfoGPR, CCallHelpers::Address slowPathOperationAddress, FunctionType function,
     272        SpillRegistersMode spillMode, ExceptionCheckRequirement requirement, ResultType result, Arguments... arguments)
     273        : CallSlowPathGenerator<JumpType, ResultType>(from, jit, spillMode, requirement, result)
     274        , m_stubInfoGPR(stubInfoGPR)
     275        , m_slowPathOperationAddress(slowPathOperationAddress)
     276        , m_function(function)
     277        , m_arguments(std::forward<Arguments>(arguments)...)
     278        , m_stubInfo(stubInfo)
     279    {
     280    }
     281
     282private:
     283    template<size_t... ArgumentsIndex>
     284    void unpackAndGenerate(SpeculativeJIT* jit, std::index_sequence<ArgumentsIndex...>)
     285    {
     286        ASSERT(JITCode::useDataIC(JITType::DFGJIT));
     287        this->setUp(jit);
     288        m_stubInfo->m_slowOperation = m_function;
     289        jit->m_jit.move(CCallHelpers::TrustedImmPtr(m_stubInfo), m_stubInfoGPR);
     290        if constexpr (std::is_same<ResultType, NoResultTag>::value)
     291            jit->callOperation<FunctionType>(m_slowPathOperationAddress, std::get<ArgumentsIndex>(m_arguments)...);
     292        else
     293            jit->callOperation<FunctionType>(m_slowPathOperationAddress, extractResult(this->m_result), std::get<ArgumentsIndex>(m_arguments)...);
     294        this->tearDown(jit);
     295    }
     296
     297    void generateInternal(SpeculativeJIT* jit) final
     298    {
     299        unpackAndGenerate(jit, std::make_index_sequence<std::tuple_size<std::tuple<Arguments...>>::value>());
     300    }
     301
     302    GPRReg m_stubInfoGPR;
     303    CCallHelpers::Address m_slowPathOperationAddress;
     304    FunctionType m_function;
     305    std::tuple<Arguments...> m_arguments;
     306    StructureStubInfo* m_stubInfo;
     307};
     308
     309template<typename JumpType, typename FunctionType, typename ResultType, typename... Arguments>
     310inline std::unique_ptr<SlowPathGenerator> slowPathICCall(
     311    JumpType from, SpeculativeJIT* jit, StructureStubInfo* stubInfo, GPRReg stubInfoGPR, CCallHelpers::Address slowPathOperationAddress, FunctionType function,
     312    SpillRegistersMode spillMode, ExceptionCheckRequirement requirement,
     313    ResultType result, Arguments... arguments)
     314{
     315    return makeUnique<CallResultAndArgumentsSlowPathICGenerator<JumpType, FunctionType, ResultType, Arguments...>>(from, jit, stubInfo, stubInfoGPR, slowPathOperationAddress, function, spillMode, requirement, result, arguments...);
     316}
     317
     318template<typename JumpType, typename FunctionType, typename ResultType, typename... Arguments>
     319inline std::unique_ptr<SlowPathGenerator> slowPathICCall(
     320    JumpType from, SpeculativeJIT* jit, StructureStubInfo* stubInfo, GPRReg stubInfoGPR, CCallHelpers::Address slowPathOperationAddress, FunctionType function,
     321    ResultType result, Arguments... arguments)
     322{
     323    return slowPathICCall(from, jit, stubInfo, stubInfoGPR, slowPathOperationAddress, function, NeedToSpill, ExceptionCheckRequirement::CheckNeeded, result, arguments...);
     324}
     325
    267326} } // namespace JSC::DFG
    268327

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r278642 r278656  
    10471047    switch (node->child1().useKind()) {
    10481048    case CellUse: {
     1049        std::optional<GPRTemporary> stubInfo;
    10491050        SpeculateCellOperand base(this, node->child1());
    10501051        JSValueRegsTemporary result(this, Reuse, base);
    10511052
     1053        GPRReg stubInfoGPR = InvalidGPRReg;
     1054        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1055            stubInfo.emplace(this);
     1056            stubInfoGPR = stubInfo->gpr();
     1057        }
    10521058        JSValueRegs baseRegs = JSValueRegs::payloadOnly(base.gpr());
    10531059        JSValueRegs resultRegs = result.regs();
    … …  
    10551061        base.use();
    10561062
    1057         cachedGetById(node->origin.semantic, baseRegs, resultRegs, node->cacheableIdentifier(), JITCompiler::Jump(), NeedToSpill, accessType);
     1063        cachedGetById(node->origin.semantic, baseRegs, resultRegs, stubInfoGPR, node->cacheableIdentifier(), JITCompiler::Jump(), NeedToSpill, accessType);
    10581064
    10591065        jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
    … …  
    10621068
    10631069    case UntypedUse: {
     1070        std::optional<GPRTemporary> stubInfo;
    10641071        JSValueOperand base(this, node->child1());
    10651072        JSValueRegsTemporary result(this, Reuse, base);
    10661073
     1074        GPRReg stubInfoGPR = InvalidGPRReg;
     1075        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1076            stubInfo.emplace(this);
     1077            stubInfoGPR = stubInfo->gpr();
     1078        }
    10671079        JSValueRegs baseRegs = base.jsValueRegs();
    10681080        JSValueRegs resultRegs = result.regs();
    … …  
    10721084        JITCompiler::Jump notCell = m_jit.branchIfNotCell(baseRegs);
    10731085
    1074         cachedGetById(node->origin.semantic, baseRegs, resultRegs, node->cacheableIdentifier(), notCell, NeedToSpill, accessType);
     1086        cachedGetById(node->origin.semantic, baseRegs, resultRegs, stubInfoGPR, node->cacheableIdentifier(), notCell, NeedToSpill, accessType);
    10751087
    10761088        jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
    … …  
    10881100    switch (node->child1().useKind()) {
    10891101    case CellUse: {
     1102        std::optional<GPRTemporary> stubInfo;
    10901103        SpeculateCellOperand base(this, node->child1());
     1104        JSValueRegsFlushedCallResult result(this);
     1105
     1106        GPRReg stubInfoGPR = InvalidGPRReg;
     1107        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1108            stubInfo.emplace(this);
     1109            stubInfoGPR = stubInfo->gpr();
     1110        }
    10911111        JSValueRegs baseRegs = JSValueRegs::payloadOnly(base.gpr());
    1092 
     1112        JSValueRegs resultRegs = result.regs();
     1113
     1114        base.use();
     1115
     1116        flushRegisters();
     1117
     1118        cachedGetById(node->origin.semantic, baseRegs, resultRegs, stubInfoGPR, node->cacheableIdentifier(), JITCompiler::Jump(), DontSpill, accessType);
     1119
     1120        jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
     1121        break;
     1122    }
     1123
     1124    case UntypedUse: {
     1125        std::optional<GPRTemporary> stubInfo;
     1126        JSValueOperand base(this, node->child1());
    10931127        JSValueRegsFlushedCallResult result(this);
     1128
     1129        GPRReg stubInfoGPR = InvalidGPRReg;
     1130        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1131            stubInfo.emplace(this);
     1132            stubInfoGPR = stubInfo->gpr();
     1133        }
     1134        JSValueRegs baseRegs = base.jsValueRegs();
    10941135        JSValueRegs resultRegs = result.regs();
    10951136
    … …  
    10981139        flushRegisters();
    10991140
    1100         cachedGetById(node->origin.semantic, baseRegs, resultRegs, node->cacheableIdentifier(), JITCompiler::Jump(), DontSpill, accessType);
    1101 
    1102         jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
    1103         break;
    1104     }
    1105 
    1106     case UntypedUse: {
    1107         JSValueOperand base(this, node->child1());
    1108         JSValueRegs baseRegs = base.jsValueRegs();
    1109 
    1110         JSValueRegsFlushedCallResult result(this);
    1111         JSValueRegs resultRegs = result.regs();
    1112 
    1113         base.use();
    1114 
    1115         flushRegisters();
    1116 
    11171141        JITCompiler::Jump notCell = m_jit.branchIfNotCell(baseRegs);
    11181142
    1119         cachedGetById(node->origin.semantic, baseRegs, resultRegs, node->cacheableIdentifier(), notCell, DontSpill, accessType);
     1143        cachedGetById(node->origin.semantic, baseRegs, resultRegs, stubInfoGPR, node->cacheableIdentifier(), notCell, DontSpill, accessType);
    11201144
    11211145        jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
    … …  
    11321156{
    11331157    if (node->child1().useKind() == CellUse) {
     1158        std::optional<GPRTemporary> stubInfo;
    11341159        SpeculateCellOperand base(this, node->child1());
    11351160        JSValueRegsTemporary result(this);
    … …  
    11381163        JITCompiler::JumpList slowCases;
    11391164
     1165        GPRReg stubInfoGPR = InvalidGPRReg;
     1166        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1167            stubInfo.emplace(this);
     1168            stubInfoGPR = stubInfo->gpr();
     1169        }
    11401170        JSValueRegs resultRegs = result.regs();
    1141 
    11421171        GPRReg baseGPR = base.gpr();
    11431172        GPRReg scratchGPR = scratch.gpr();
    … …  
    11491178
    11501179        JITDelByIdGenerator gen(
    1151             m_jit.codeBlock(), codeOrigin, callSite, usedRegisters, node->cacheableIdentifier(),
    1152             JSValueRegs::payloadOnly(baseGPR), resultRegs, scratchGPR);
     1180            m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters, node->cacheableIdentifier(),
     1181            JSValueRegs::payloadOnly(baseGPR), resultRegs, stubInfoGPR, scratchGPR);
    11531182
    11541183        gen.generateFastPath(m_jit);
    1155         slowCases.append(gen.slowPathJump());
     1184        if (!JITCode::useDataIC(JITType::DFGJIT))
     1185            slowCases.append(gen.slowPathJump());
    11561186
    11571187#if USE(JSVALUE64)
    1158         std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    1159             slowCases, this, operationDeleteByIdOptimize,
    1160             resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), JSValueRegs(baseGPR), node->cacheableIdentifier().rawBits(), TrustedImm32(node->ecmaMode().value()));
     1188        std::unique_ptr<SlowPathGenerator> slowPath;
     1189        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1190            slowPath = slowPathICCall(
     1191                slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationDeleteByIdOptimize,
     1192                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, JSValueRegs(baseGPR), node->cacheableIdentifier().rawBits(), TrustedImm32(node->ecmaMode().value()));
     1193        } else {
     1194            slowPath = slowPathCall(
     1195                slowCases, this, operationDeleteByIdOptimize,
     1196                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), JSValueRegs(baseGPR), node->cacheableIdentifier().rawBits(), TrustedImm32(node->ecmaMode().value()));
     1197        }
    11611198#else
    1162         std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    1163             slowCases, this, operationDeleteByIdOptimize,
    1164             resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), node->cacheableIdentifier().rawBits(), TrustedImm32(node->ecmaMode().value()));
     1199        std::unique_ptr<SlowPathGenerator> slowPath;
     1200        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1201            slowPath = slowPathICCall(
     1202                slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationDeleteByIdOptimize,
     1203                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, CCallHelpers::CellValue(baseGPR), node->cacheableIdentifier().rawBits(), TrustedImm32(node->ecmaMode().value()));
     1204        } else {
     1205            slowPath = slowPathCall(
     1206                slowCases, this, operationDeleteByIdOptimize,
     1207                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), node->cacheableIdentifier().rawBits(), TrustedImm32(node->ecmaMode().value()));
     1208        }
    11651209#endif
    11661210
    … …  
    11911235{
    11921236    if (node->child1().useKind() == CellUse) {
     1237        std::optional<GPRTemporary> stubInfo;
    11931238        SpeculateCellOperand base(this, node->child1());
    1194 
    11951239        JSValueOperand key(this, node->child2(), ManualOperandSpeculation);
    1196         speculate(node, node->child2());
    1197 
    11981240        JSValueRegsTemporary result(this, Reuse, key);
    11991241        GPRTemporary scratch(this);
    … …  
    12011243        JITCompiler::JumpList slowCases;
    12021244
     1245        GPRReg stubInfoGPR = InvalidGPRReg;
     1246        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1247            stubInfo.emplace(this);
     1248            stubInfoGPR = stubInfo->gpr();
     1249        }
    12031250        GPRReg baseGPR = base.gpr();
    12041251        JSValueRegs keyRegs = key.jsValueRegs();
    … …  
    12071254        GPRReg resultGPR = resultRegs.payloadGPR();
    12081255
     1256        speculate(node, node->child2());
     1257
    12091258        if (needsTypeCheck(node->child2(), SpecCell))
    12101259            slowCases.append(m_jit.branchIfNotCell(keyRegs));
    … …  
    12151264
    12161265        JITDelByValGenerator gen(
    1217             m_jit.codeBlock(), codeOrigin, callSite, usedRegisters,
    1218             JSValueRegs::payloadOnly(baseGPR), keyRegs, resultRegs, scratchGPR);
     1266            m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters,
     1267            JSValueRegs::payloadOnly(baseGPR), keyRegs, resultRegs, stubInfoGPR, scratchGPR);
    12191268
    12201269        gen.generateFastPath(m_jit);
    1221         slowCases.append(gen.slowPathJump());
     1270        if (!JITCode::useDataIC(JITType::DFGJIT))
     1271            slowCases.append(gen.slowPathJump());
    12221272
    12231273#if USE(JSVALUE64)
    1224         std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    1225             slowCases, this, operationDeleteByValOptimize,
    1226             resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), JSValueRegs(baseGPR), keyRegs, TrustedImm32(node->ecmaMode().value()));
     1274        std::unique_ptr<SlowPathGenerator> slowPath;
     1275        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1276            slowPath = slowPathICCall(
     1277                slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationDeleteByValOptimize,
     1278                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, JSValueRegs(baseGPR), keyRegs, TrustedImm32(node->ecmaMode().value()));
     1279        } else {
     1280            slowPath = slowPathCall(
     1281                slowCases, this, operationDeleteByValOptimize,
     1282                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), JSValueRegs(baseGPR), keyRegs, TrustedImm32(node->ecmaMode().value()));
     1283        }
    12271284#else
    1228         std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    1229             slowCases, this, operationDeleteByValOptimize,
    1230             resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), keyRegs, TrustedImm32(node->ecmaMode().value()));
     1285        std::unique_ptr<SlowPathGenerator> slowPath;
     1286        if (JITCode::useDataIC(JITType::DFGJIT)) {
     1287            slowPath = slowPathICCall(
     1288                slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationDeleteByValOptimize,
     1289                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, CCallHelpers::CellValue(baseGPR), keyRegs, TrustedImm32(node->ecmaMode().value()));
     1290        } else {
     1291            slowPath = slowPathCall(
     1292                slowCases, this, operationDeleteByValOptimize,
     1293                resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), keyRegs, TrustedImm32(node->ecmaMode().value()));
     1294        }
    12311295#endif
    12321296
    … …  
    12571321void SpeculativeJIT::compileInById(Node* node)
    12581322{
     1323    std::optional<GPRTemporary> stubInfo;
    12591324    SpeculateCellOperand base(this, node->child1());
    12601325    JSValueRegsTemporary result(this, Reuse, base, PayloadWord);
    12611326
     1327    GPRReg stubInfoGPR = InvalidGPRReg;
     1328    if (JITCode::useDataIC(JITType::DFGJIT)) {
     1329        stubInfo.emplace(this);
     1330        stubInfoGPR = stubInfo->gpr();
     1331    }
    12621332    GPRReg baseGPR = base.gpr();
    12631333    JSValueRegs resultRegs = result.regs();
    … …  
    12691339    RegisterSet usedRegisters = this->usedRegisters();
    12701340    JITInByIdGenerator gen(
    1271         m_jit.codeBlock(), codeOrigin, callSite, usedRegisters, node->cacheableIdentifier(),
    1272         JSValueRegs::payloadOnly(baseGPR), resultRegs);
     1341        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters, node->cacheableIdentifier(),
     1342        JSValueRegs::payloadOnly(baseGPR), resultRegs, stubInfoGPR);
    12731343    gen.generateFastPath(m_jit);
    12741344
    1275     auto slowPath = slowPathCall(
    1276         gen.slowPathJump(), this, operationInByIdOptimize,
    1277         NeedToSpill, ExceptionCheckRequirement::CheckNeeded,
    1278         resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), node->cacheableIdentifier().rawBits());
     1345    JITCompiler::JumpList slowCases;
     1346    slowCases.append(gen.slowPathJump());
     1347
     1348    std::unique_ptr<SlowPathGenerator> slowPath;
     1349    if (JITCode::useDataIC(JITType::DFGJIT)) {
     1350        slowPath = slowPathICCall(
     1351            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationInByIdOptimize,
     1352            NeedToSpill, ExceptionCheckRequirement::CheckNeeded,
     1353            resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), stubInfoGPR, CCallHelpers::CellValue(baseGPR), node->cacheableIdentifier().rawBits());
     1354    } else {
     1355        slowPath = slowPathCall(
     1356            slowCases, this, operationInByIdOptimize,
     1357            NeedToSpill, ExceptionCheckRequirement::CheckNeeded,
     1358            resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), node->cacheableIdentifier().rawBits());
     1359    }
    12791360
    12801361    m_jit.addInById(gen, slowPath.get());
    … …  
    12891370    JSValueOperand key(this, node->child2());
    12901371    JSValueRegsTemporary result(this, Reuse, key);
    1291 
     1372    std::optional<GPRTemporary> stubInfo;
     1373
     1374    GPRReg stubInfoGPR = InvalidGPRReg;
     1375    if (JITCode::useDataIC(JITType::DFGJIT)) {
     1376        stubInfo.emplace(this);
     1377        stubInfoGPR = stubInfo->gpr();
     1378    }
    12921379    GPRReg baseGPR = base.gpr();
    12931380    JSValueRegs keyRegs = key.jsValueRegs();
    … …  
    12961383    base.use();
    12971384    key.use();
     1385
     1386    CCallHelpers::JumpList slowCases;
    12981387
    12991388    CodeOrigin codeOrigin = node->origin.semantic;
    … …  
    13011390    RegisterSet usedRegisters = this->usedRegisters();
    13021391    JITInByValGenerator gen(
    1303         m_jit.codeBlock(), codeOrigin, callSite, usedRegisters,
    1304         JSValueRegs::payloadOnly(baseGPR), keyRegs, resultRegs);
     1392        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters,
     1393        JSValueRegs::payloadOnly(baseGPR), keyRegs, resultRegs, stubInfoGPR);
    13051394    gen.generateFastPath(m_jit);
    1306 
    1307     auto slowPath = slowPathCall(
    1308         gen.slowPathJump(), this, operationInByValOptimize,
    1309         NeedToSpill, ExceptionCheckRequirement::CheckNeeded,
    1310         resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), gen.stubInfo(), nullptr, CCallHelpers::CellValue(baseGPR), keyRegs);
     1395    if (!JITCode::useDataIC(JITType::DFGJIT))
     1396        slowCases.append(gen.slowPathJump());
     1397
     1398    std::unique_ptr<SlowPathGenerator> slowPath;
     1399    if (JITCode::useDataIC(JITType::DFGJIT)) {
     1400        slowPath = slowPathICCall(
     1401            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationInByValOptimize,
     1402            NeedToSpill, ExceptionCheckRequirement::CheckNeeded,
     1403            resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), stubInfoGPR, nullptr, CCallHelpers::CellValue(baseGPR), keyRegs);
     1404    } else {
     1405        slowPath = slowPathCall(
     1406            slowCases, this, operationInByValOptimize,
     1407            NeedToSpill, ExceptionCheckRequirement::CheckNeeded,
     1408            resultRegs, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), gen.stubInfo(), nullptr, CCallHelpers::CellValue(baseGPR), keyRegs);
     1409    }
    13111410
    13121411    m_jit.addInByVal(gen, slowPath.get());
    … …  
    36043703    DFG_ASSERT(m_jit.graph(), node, node->op() == GetPrivateName);
    36053704    DFG_ASSERT(m_jit.graph(), node, m_graph.child(node, 1).useKind() == SymbolUse);
     3705
     3706    std::optional<GPRTemporary> stubInfo;
     3707    JSValueRegsTemporary result(this);
     3708
     3709    GPRReg stubInfoGPR = InvalidGPRReg;
     3710    if (JITCode::useDataIC(JITType::DFGJIT)) {
     3711        stubInfo.emplace(this);
     3712        stubInfoGPR = stubInfo->gpr();
     3713    }
     3714    JSValueRegs resultRegs = result.regs();
     3715
    36063716    speculateSymbol(m_graph.child(node, 1));
    36073717
    3608     JSValueRegsTemporary result(this);
    36093718    CodeOrigin codeOrigin = node->origin.semantic;
    36103719    CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(codeOrigin, m_stream->size());
    … …  
    36173726
    36183727    JITGetByValGenerator gen(
    3619         m_jit.codeBlock(), codeOrigin, callSite, AccessType::GetPrivateName, usedRegisters,
    3620         base, property, result.regs());
     3728        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, AccessType::GetPrivateName, usedRegisters,
     3729        base, property, resultRegs, stubInfoGPR);
    36213730    gen.stubInfo()->propertyIsSymbol = true;
    36223731    gen.generateFastPath(m_jit);
    3623 
    3624     slowCases.append(gen.slowPathJump());
    3625 
    3626     auto makeSlowPathCall = [&](auto base) {
     3732    if (!JITCode::useDataIC(JITType::DFGJIT))
     3733        slowCases.append(gen.slowPathJump());
     3734
     3735    auto makeSlowPathICCall = [&](auto base) {
     3736        if (JITCode::useDataIC(JITType::DFGJIT)) {
     3737            return slowPathICCall(
     3738                slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationGetPrivateNameOptimize,
     3739                result.regs(), TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR,
     3740                base, CCallHelpers::CellValue(property.payloadGPR()));
     3741        }
    36273742        return slowPathCall(
    36283743            slowCases, this, operationGetPrivateNameOptimize,
    … …  
    36323747
    36333748    std::unique_ptr<SlowPathGenerator> slowPath = baseIsKnownCell
    3634         ? makeSlowPathCall(CCallHelpers::CellValue(base.payloadGPR()))
    3635         : makeSlowPathCall(base);
     3749        ? makeSlowPathICCall(CCallHelpers::CellValue(base.payloadGPR()))
     3750        : makeSlowPathICCall(base);
    36363751
    36373752    m_jit.addGetByVal(gen, slowPath.get());
    … …  
    36453760    switch (m_graph.child(node, 0).useKind()) {
    36463761    case CellUse: {
     3762        std::optional<GPRTemporary> stubInfo;
    36473763        SpeculateCellOperand base(this, m_graph.child(node, 0));
    36483764        JSValueRegsTemporary result(this, Reuse, base);
    36493765
     3766        GPRReg stubInfoGPR = InvalidGPRReg;
     3767        if (JITCode::useDataIC(JITType::DFGJIT)) {
     3768            stubInfo.emplace(this);
     3769            stubInfoGPR = stubInfo->gpr();
     3770        }
    36503771        JSValueRegs baseRegs = JSValueRegs::payloadOnly(base.gpr());
    36513772        JSValueRegs resultRegs = result.regs();
    36523773
    3653         cachedGetById(node->origin.semantic, baseRegs, resultRegs, node->cacheableIdentifier(), JITCompiler::Jump(), NeedToSpill, AccessType::GetPrivateName);
     3774        cachedGetById(node->origin.semantic, baseRegs, resultRegs, stubInfoGPR, node->cacheableIdentifier(), JITCompiler::Jump(), NeedToSpill, AccessType::GetPrivateName);
    36543775
    36553776        jsValueResult(resultRegs, node, DataFormatJS);
    … …  
    36583779
    36593780    case UntypedUse: {
     3781        std::optional<GPRTemporary> stubInfo;
    36603782        JSValueOperand base(this, m_graph.child(node, 0));
    36613783        JSValueRegsTemporary result(this, Reuse, base);
    36623784
     3785        GPRReg stubInfoGPR = InvalidGPRReg;
     3786        if (JITCode::useDataIC(JITType::DFGJIT)) {
     3787            stubInfo.emplace(this);
     3788            stubInfoGPR = stubInfo->gpr();
     3789        }
    36633790        JSValueRegs baseRegs = base.jsValueRegs();
    36643791        JSValueRegs resultRegs = result.regs();
    … …  
    36663793        JITCompiler::Jump notCell = m_jit.branchIfNotCell(baseRegs);
    36673794
    3668         cachedGetById(node->origin.semantic, baseRegs, resultRegs, node->cacheableIdentifier(), notCell, NeedToSpill, AccessType::GetPrivateName);
     3795        cachedGetById(node->origin.semantic, baseRegs, resultRegs, stubInfoGPR, node->cacheableIdentifier(), notCell, NeedToSpill, AccessType::GetPrivateName);
    36693796
    36703797        jsValueResult(resultRegs, node, DataFormatJS);
    … …  
    37653892void SpeculativeJIT::compilePutPrivateNameById(Node* node)
    37663893{
     3894    std::optional<GPRTemporary> stubInfo;
    37673895    SpeculateCellOperand base(this, node->child1());
    37683896    JSValueOperand value(this, node->child2());
     3897    GPRTemporary scratch(this);
     3898
     3899    GPRReg stubInfoGPR = InvalidGPRReg;
     3900    if (JITCode::useDataIC(JITType::DFGJIT)) {
     3901        stubInfo.emplace(this);
     3902        stubInfoGPR = stubInfo->gpr();
     3903    }
    37693904    JSValueRegs valueRegs = value.jsValueRegs();
    37703905    GPRReg baseGPR = base.gpr();
    3771 
    3772     GPRTemporary scratch(this);
    37733906    GPRReg scratchGPR = scratch.gpr();
    37743907
    … …  
    37763909    // to check it here.
    37773910    auto putKind = node->privateFieldPutKind().isDefine() ? PutKind::DirectPrivateFieldDefine : PutKind::DirectPrivateFieldSet;
    3778     cachedPutById(node->origin.semantic, baseGPR, valueRegs, scratchGPR, node->cacheableIdentifier(), putKind, ECMAMode::strict());
     3911    cachedPutById(node->origin.semantic, baseGPR, valueRegs, stubInfoGPR, scratchGPR, node->cacheableIdentifier(), putKind, ECMAMode::strict());
    37793912
    37803913    noResult(node);
    … …  
    37833916void SpeculativeJIT::compileCheckPrivateBrand(Node* node)
    37843917{
     3918    std::optional<GPRTemporary> stubInfo;
    37853919    JSValueOperand base(this, node->child1());
    37863920    SpeculateCellOperand brandValue(this, node->child2());
    37873921
     3922    GPRReg stubInfoGPR = InvalidGPRReg;
     3923    if (JITCode::useDataIC(JITType::DFGJIT)) {
     3924        stubInfo.emplace(this);
     3925        stubInfoGPR = stubInfo->gpr();
     3926    }
    37883927    JSValueRegs baseRegs = base.jsValueRegs();
    3789 
    37903928    GPRReg brandGPR = brandValue.gpr();
    37913929
    … …  
    38013939
    38023940    JITPrivateBrandAccessGenerator gen(
    3803         m_jit.codeBlock(), codeOrigin, callSite, AccessType::CheckPrivateBrand, usedRegisters,
    3804         baseRegs, JSValueRegs::payloadOnly(brandGPR));
     3941        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, AccessType::CheckPrivateBrand, usedRegisters,
     3942        baseRegs, JSValueRegs::payloadOnly(brandGPR), stubInfoGPR);
    38053943
    38063944    gen.stubInfo()->propertyIsSymbol = true;
    38073945    gen.generateFastPath(m_jit);
    3808     slowCases.append(gen.slowPathJump());
    3809 
    3810     std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    3811         slowCases, this, operationCheckPrivateBrandOptimize, NoResult,
    3812         TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), baseRegs, CCallHelpers::CellValue(brandGPR));
     3946    if (!JITCode::useDataIC(JITType::DFGJIT))
     3947        slowCases.append(gen.slowPathJump());
     3948
     3949    std::unique_ptr<SlowPathGenerator> slowPath;
     3950    if (JITCode::useDataIC(JITType::DFGJIT)) {
     3951        slowPath = slowPathICCall(
     3952            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationCheckPrivateBrandOptimize, NoResult,
     3953            TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, baseRegs, CCallHelpers::CellValue(brandGPR));
     3954    } else {
     3955        slowPath = slowPathCall(
     3956            slowCases, this, operationCheckPrivateBrandOptimize, NoResult,
     3957            TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), baseRegs, CCallHelpers::CellValue(brandGPR));
     3958    }
    38133959
    38143960    m_jit.addPrivateBrandAccess(gen, slowPath.get());
    … …  
    38213967{
    38223968    ASSERT(node->child1().useKind() == CellUse);
     3969    std::optional<GPRTemporary> stubInfo;
    38233970    SpeculateCellOperand base(this, node->child1());
    38243971    SpeculateCellOperand brandValue(this, node->child2());
    38253972
     3973    GPRReg stubInfoGPR = InvalidGPRReg;
     3974    if (JITCode::useDataIC(JITType::DFGJIT)) {
     3975        stubInfo.emplace(this);
     3976        stubInfoGPR = stubInfo->gpr();
     3977    }
    38263978    GPRReg baseGPR = base.gpr();
    38273979    GPRReg brandGPR = brandValue.gpr();
    … …  
    38353987    JITCompiler::JumpList slowCases;
    38363988    JITPrivateBrandAccessGenerator gen(
    3837         m_jit.codeBlock(), codeOrigin, callSite, AccessType::SetPrivateBrand, usedRegisters,
    3838         JSValueRegs::payloadOnly(baseGPR), JSValueRegs::payloadOnly(brandGPR));
     3989        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, AccessType::SetPrivateBrand, usedRegisters,
     3990        JSValueRegs::payloadOnly(baseGPR), JSValueRegs::payloadOnly(brandGPR), stubInfoGPR);
    38393991
    38403992    gen.stubInfo()->propertyIsSymbol = true;
    38413993    gen.generateFastPath(m_jit);
    3842     slowCases.append(gen.slowPathJump());
    3843 
    3844     std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    3845         slowCases, this, operationSetPrivateBrandOptimize, NoResult,
    3846         TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), CCallHelpers::CellValue(brandGPR));
     3994    if (!JITCode::useDataIC(JITType::DFGJIT))
     3995        slowCases.append(gen.slowPathJump());
     3996
     3997    std::unique_ptr<SlowPathGenerator> slowPath;
     3998    if (JITCode::useDataIC(JITType::DFGJIT)) {
     3999        slowPath = slowPathICCall(
     4000            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationSetPrivateBrandOptimize, NoResult,
     4001            TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, CCallHelpers::CellValue(baseGPR), CCallHelpers::CellValue(brandGPR));
     4002    } else {
     4003        slowPath = slowPathCall(
     4004            slowCases, this, operationSetPrivateBrandOptimize, NoResult,
     4005            TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), CCallHelpers::CellValue(baseGPR), CCallHelpers::CellValue(brandGPR));
     4006    }
    38474007
    38484008    m_jit.addPrivateBrandAccess(gen, slowPath.get());
    … …  
    39624122}
    39634123
    3964 void SpeculativeJIT::compileInstanceOfForCells(Node* node, JSValueRegs valueRegs, JSValueRegs prototypeRegs, GPRReg resultGPR, GPRReg scratchGPR, GPRReg scratch2GPR, JITCompiler::Jump slowCase)
     4124void SpeculativeJIT::compileInstanceOfForCells(Node* node, JSValueRegs valueRegs, JSValueRegs prototypeRegs, GPRReg resultGPR, GPRReg stubInfoGPR, GPRReg scratchGPR, GPRReg scratch2GPR, JITCompiler::Jump slowCase)
    39654125{
    39664126    CallSiteIndex callSiteIndex = m_jit.addCallSite(node->origin.semantic);
    39674127   
    39684128    JITInstanceOfGenerator gen(
    3969         m_jit.codeBlock(), node->origin.semantic, callSiteIndex, usedRegisters(), resultGPR,
    3970         valueRegs.payloadGPR(), prototypeRegs.payloadGPR(), scratchGPR, scratch2GPR,
     4129        m_jit.codeBlock(), JITType::DFGJIT, node->origin.semantic, callSiteIndex, usedRegisters(), resultGPR,
     4130        valueRegs.payloadGPR(), prototypeRegs.payloadGPR(), stubInfoGPR, scratchGPR, scratch2GPR,
    39714131        m_state.forNode(node->child2()).isType(SpecObject | ~SpecCell));
    39724132    gen.generateFastPath(m_jit);
    3973    
    39744133    JITCompiler::JumpList slowCases;
     4134    if (!JITCode::useDataIC(JITType::DFGJIT))
     4135        slowCases.append(gen.slowPathJump());
    39754136    slowCases.append(slowCase);
    3976    
    3977     std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    3978         slowCases, this, operationInstanceOfOptimize, resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), gen.stubInfo(), valueRegs,
    3979         prototypeRegs);
     4137
     4138    std::unique_ptr<SlowPathGenerator> slowPath;
     4139    if (JITCode::useDataIC(JITType::DFGJIT)) {
     4140        slowPath = slowPathICCall(
     4141            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationInstanceOfOptimize, resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), stubInfoGPR, valueRegs, prototypeRegs);
     4142    } else {
     4143        slowPath = slowPathCall(
     4144            slowCases, this, operationInstanceOfOptimize, resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(node->origin.semantic)), gen.stubInfo(), valueRegs, prototypeRegs);
     4145    }
    39804146   
    39814147    m_jit.addInstanceOf(gen, slowPath.get());
    … …  
    39884154    if (node->child1().useKind() == CellUse
    39894155        && node->child2().useKind() == CellUse) {
     4156        std::optional<GPRTemporary> stubInfo;
    39904157        SpeculateCellOperand value(this, node->child1());
    39914158        SpeculateCellOperand prototype(this, node->child2());
    … …  
    39954162        GPRTemporary scratch2(this);
    39964163       
     4164        GPRReg stubInfoGPR = InvalidGPRReg;
     4165        if (JITCode::useDataIC(JITType::DFGJIT)) {
     4166            stubInfo.emplace(this);
     4167            stubInfoGPR = stubInfo->gpr();
     4168        }
    39974169        GPRReg valueGPR = value.gpr();
    39984170        GPRReg prototypeGPR = prototype.gpr();
    … …  
    40014173        GPRReg scratch2GPR = scratch2.gpr();
    40024174       
    4003         compileInstanceOfForCells(node, JSValueRegs(valueGPR), JSValueRegs(prototypeGPR), resultGPR, scratchGPR, scratch2GPR);
     4175        compileInstanceOfForCells(node, JSValueRegs(valueGPR), JSValueRegs(prototypeGPR), resultGPR, stubInfoGPR, scratchGPR, scratch2GPR);
    40044176       
    40054177        blessedBooleanResult(resultGPR, node);
    … …  
    40114183    DFG_ASSERT(m_jit.graph(), node, node->child2().useKind() == UntypedUse);
    40124184   
     4185    std::optional<GPRTemporary> stubInfo;
    40134186    JSValueOperand value(this, node->child1());
    40144187    JSValueOperand prototype(this, node->child2());
    … …  
    40174190    GPRTemporary scratch(this);
    40184191   
     4192    GPRReg stubInfoGPR = InvalidGPRReg;
     4193    if (JITCode::useDataIC(JITType::DFGJIT)) {
     4194        stubInfo.emplace(this);
     4195        stubInfoGPR = stubInfo->gpr();
     4196    }
    40194197    JSValueRegs valueRegs = value.jsValueRegs();
    40204198    JSValueRegs prototypeRegs = prototype.jsValueRegs();
    … …  
    40324210    JITCompiler::Jump slowCase = m_jit.branchIfNotCell(prototypeRegs);
    40334211   
    4034     compileInstanceOfForCells(node, valueRegs, prototypeRegs, resultGPR, scratchGPR, InvalidGPRReg, slowCase);
     4212    compileInstanceOfForCells(node, valueRegs, prototypeRegs, resultGPR, stubInfoGPR, scratchGPR, InvalidGPRReg, slowCase);
    40354213   
    40364214    done.link(&m_jit);
    … …  
    1327713455void SpeculativeJIT::compilePutByIdFlush(Node* node)
    1327813456{
     13457    std::optional<GPRTemporary> stubInfo;
    1327913458    SpeculateCellOperand base(this, node->child1());
    1328013459    JSValueOperand value(this, node->child2());
    1328113460    GPRTemporary scratch(this);
    1328213461
     13462    GPRReg stubInfoGPR = InvalidGPRReg;
     13463    if (JITCode::useDataIC(JITType::DFGJIT)) {
     13464        stubInfo.emplace(this);
     13465        stubInfoGPR = stubInfo->gpr();
     13466    }
    1328313467    GPRReg baseGPR = base.gpr();
    1328413468    JSValueRegs valueRegs = value.jsValueRegs();
    … …  
    1328613470    flushRegisters();
    1328713471
    13288     cachedPutById(node->origin.semantic, baseGPR, valueRegs, scratchGPR, node->cacheableIdentifier(), PutKind::NotDirect, node->ecmaMode(), MacroAssembler::Jump(), DontSpill);
     13472    cachedPutById(node->origin.semantic, baseGPR, valueRegs, stubInfoGPR, scratchGPR, node->cacheableIdentifier(), PutKind::NotDirect, node->ecmaMode(), MacroAssembler::Jump(), DontSpill);
    1328913473
    1329013474    noResult(node);
    … …  
    1329313477void SpeculativeJIT::compilePutById(Node* node)
    1329413478{
     13479    std::optional<GPRTemporary> stubInfo;
    1329513480    SpeculateCellOperand base(this, node->child1());
    1329613481    JSValueOperand value(this, node->child2());
    1329713482    GPRTemporary scratch(this);
    1329813483
     13484    GPRReg stubInfoGPR = InvalidGPRReg;
     13485    if (JITCode::useDataIC(JITType::DFGJIT)) {
     13486        stubInfo.emplace(this);
     13487        stubInfoGPR = stubInfo->gpr();
     13488    }
    1329913489    GPRReg baseGPR = base.gpr();
    1330013490    JSValueRegs valueRegs = value.jsValueRegs();
    1330113491    GPRReg scratchGPR = scratch.gpr();
    1330213492
    13303     cachedPutById(node->origin.semantic, baseGPR, valueRegs, scratchGPR, node->cacheableIdentifier(), PutKind::NotDirect, node->ecmaMode());
     13493    cachedPutById(node->origin.semantic, baseGPR, valueRegs, stubInfoGPR, scratchGPR, node->cacheableIdentifier(), PutKind::NotDirect, node->ecmaMode());
    1330413494
    1330513495    noResult(node);
    … …  
    1330813498void SpeculativeJIT::compilePutByIdDirect(Node* node)
    1330913499{
     13500    std::optional<GPRTemporary> stubInfo;
    1331013501    SpeculateCellOperand base(this, node->child1());
    1331113502    JSValueOperand value(this, node->child2());
    1331213503    GPRTemporary scratch(this);
    1331313504
     13505    GPRReg stubInfoGPR = InvalidGPRReg;
     13506    if (JITCode::useDataIC(JITType::DFGJIT)) {
     13507        stubInfo.emplace(this);
     13508        stubInfoGPR = stubInfo->gpr();
     13509    }
    1331413510    GPRReg baseGPR = base.gpr();
    1331513511    JSValueRegs valueRegs = value.jsValueRegs();
    1331613512    GPRReg scratchGPR = scratch.gpr();
    1331713513
    13318     cachedPutById(node->origin.semantic, baseGPR, valueRegs, scratchGPR, node->cacheableIdentifier(), PutKind::Direct, node->ecmaMode());
     13514    cachedPutById(node->origin.semantic, baseGPR, valueRegs, stubInfoGPR, scratchGPR, node->cacheableIdentifier(), PutKind::Direct, node->ecmaMode());
    1331913515
    1332013516    noResult(node);
    … …  
    1494315139}
    1494415140
    14945 void SpeculativeJIT::cachedPutById(CodeOrigin codeOrigin, GPRReg baseGPR, JSValueRegs valueRegs, GPRReg scratchGPR, CacheableIdentifier identifier, PutKind putKind, ECMAMode ecmaMode, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode)
     15141void SpeculativeJIT::cachedPutById(CodeOrigin codeOrigin, GPRReg baseGPR, JSValueRegs valueRegs, GPRReg stubInfoGPR, GPRReg scratchGPR, CacheableIdentifier identifier, PutKind putKind, ECMAMode ecmaMode, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode)
    1494615142{
    1494715143    RegisterSet usedRegisters = this->usedRegisters();
    … …  
    1495015146        usedRegisters.set(baseGPR, false);
    1495115147        usedRegisters.set(valueRegs, false);
     15148        if (stubInfoGPR != InvalidGPRReg)
     15149            usedRegisters.set(stubInfoGPR, false);
    1495215150    }
    1495315151    CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(codeOrigin, m_stream->size());
    1495415152    JITPutByIdGenerator gen(
    14955         m_jit.codeBlock(), codeOrigin, callSite, usedRegisters, identifier,
    14956         JSValueRegs::payloadOnly(baseGPR), valueRegs,
     15153        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters, identifier,
     15154        JSValueRegs::payloadOnly(baseGPR), valueRegs, stubInfoGPR,
    1495715155        scratchGPR, ecmaMode, putKind);
    1495815156
    … …  
    1496415162    slowCases.append(gen.slowPathJump());
    1496515163
    14966     auto slowPath = slowPathCall(
    14967         slowCases, this, gen.slowPathFunction(), NoResult, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), valueRegs,
    14968         CCallHelpers::CellValue(baseGPR), identifier.rawBits());
     15164    std::unique_ptr<SlowPathGenerator> slowPath;
     15165    if (JITCode::useDataIC(JITType::DFGJIT)) {
     15166        slowPath = slowPathICCall(
     15167            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), gen.slowPathFunction(), NoResult, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, valueRegs,
     15168            CCallHelpers::CellValue(baseGPR), identifier.rawBits());
     15169    } else {
     15170        slowPath = slowPathCall(
     15171            slowCases, this, gen.slowPathFunction(), NoResult, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), valueRegs,
     15172            CCallHelpers::CellValue(baseGPR), identifier.rawBits());
     15173    }
    1496915174
    1497015175    m_jit.addPutById(gen, slowPath.get());

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

    r278568 r278656  
    720720    void compileCheckDetached(Node*);
    721721
    722     void cachedGetById(CodeOrigin, JSValueRegs base, JSValueRegs result, CacheableIdentifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode, AccessType);
    723     void cachedPutById(CodeOrigin, GPRReg baseGPR, JSValueRegs valueRegs, GPRReg scratchGPR, CacheableIdentifier, PutKind, ECMAMode, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill);
     722    void cachedGetById(CodeOrigin, JSValueRegs base, JSValueRegs result, GPRReg stubInfoGPR, CacheableIdentifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode, AccessType);
     723    void cachedPutById(CodeOrigin, GPRReg baseGPR, JSValueRegs valueRegs, GPRReg stubInfoGPR, GPRReg scratchGPR, CacheableIdentifier, PutKind, ECMAMode, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill);
    724724    void cachedGetByVal(CodeOrigin, JSValueRegs base, JSValueRegs property, JSValueRegs result, JITCompiler::Jump slowPathTarget);
    725725
    726726#if USE(JSVALUE64)
    727     void cachedGetById(CodeOrigin, GPRReg baseGPR, GPRReg resultGPR, CacheableIdentifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode, AccessType);
    728     void cachedGetByIdWithThis(CodeOrigin, GPRReg baseGPR, GPRReg thisGPR, GPRReg resultGPR, CacheableIdentifier, const JITCompiler::JumpList& slowPathTarget = JITCompiler::JumpList());
     727    void cachedGetById(CodeOrigin, GPRReg baseGPR, GPRReg resultGPR, GPRReg stubInfoGPR, CacheableIdentifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode, AccessType);
     728    void cachedGetByIdWithThis(CodeOrigin, GPRReg baseGPR, GPRReg thisGPR, GPRReg resultGPR, GPRReg stubInfoGPR, CacheableIdentifier, const JITCompiler::JumpList& slowPathTarget = JITCompiler::JumpList());
    729729#elif USE(JSVALUE32_64)
    730     void cachedGetById(CodeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, CacheableIdentifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode, AccessType);
    731     void cachedGetByIdWithThis(CodeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg thisTagGPROrNone, GPRReg thisPayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, CacheableIdentifier, const JITCompiler::JumpList& slowPathTarget = JITCompiler::JumpList());
     730    void cachedGetById(CodeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, GPRReg stubInfoGPR, CacheableIdentifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode, AccessType);
     731    void cachedGetByIdWithThis(CodeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg thisTagGPROrNone, GPRReg thisPayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, GPRReg stubInfoGPR, CacheableIdentifier, const JITCompiler::JumpList& slowPathTarget = JITCompiler::JumpList());
    732732#endif
    733733
    … …  
    752752    bool genericJSValueStrictEq(Node*, bool invert = false);
    753753   
    754     void compileInstanceOfForCells(Node*, JSValueRegs valueGPR, JSValueRegs prototypeGPR, GPRReg resultGPT, GPRReg scratchGPR, GPRReg scratch2GPR, JITCompiler::Jump slowCase = JITCompiler::Jump());
     754    void compileInstanceOfForCells(Node*, JSValueRegs valueGPR, JSValueRegs prototypeGPR, GPRReg resultGPT, GPRReg stubInfoGPR, GPRReg scratchGPR, GPRReg scratch2GPR, JITCompiler::Jump slowCase = JITCompiler::Jump());
    755755    void compileInstanceOf(Node*);
    756756    void compileInstanceOfCustom(Node*);
    … …  
    950950    }
    951951
     952    template<typename OperationType, typename ResultRegType, typename... Args>
     953    std::enable_if_t<
     954        FunctionTraits<OperationType>::hasResult,
     955    void>
     956    callOperation(CCallHelpers::Address address, ResultRegType result, Args... args)
     957    {
     958        m_jit.setupArgumentsForIndirectCall<OperationType>(address, args...);
     959        appendCallSetResult(CCallHelpers::Address(GPRInfo::nonArgGPR0, address.offset), result);
     960    }
     961
     962    template<typename OperationType, typename... Args>
     963    std::enable_if_t<
     964        !FunctionTraits<OperationType>::hasResult,
     965    void>
     966    callOperation(CCallHelpers::Address address, Args... args)
     967    {
     968        m_jit.setupArgumentsForIndirectCall<OperationType>(address, args...);
     969        appendCall(CCallHelpers::Address(GPRInfo::nonArgGPR0, address.offset));
     970    }
     971
    952972    JITCompiler::Call callOperationWithCallFrameRollbackOnException(V_JITOperation_Cb operation, CodeBlock* codeBlock)
    953973    {
    … …  
    9891009    }
    9901010
     1011    void appendCall(CCallHelpers::Address address)
     1012    {
     1013        prepareForExternalCall();
     1014        m_jit.emitStoreCodeOrigin(m_currentNode->origin.semantic);
     1015        m_jit.appendCall(address);
     1016    }
     1017
    9911018    JITCompiler::Call appendOperationCall(const FunctionPtr<OperationPtrTag> function)
    9921019    {
    … …  
    10091036            m_jit.move(GPRInfo::returnValueGPR, result);
    10101037        return call;
     1038    }
     1039
     1040    void appendCallSetResult(CCallHelpers::Address address, GPRReg result)
     1041    {
     1042        appendCall(address);
     1043        if (result != InvalidGPRReg)
     1044            m_jit.move(GPRInfo::returnValueGPR, result);
     1045    }
     1046
     1047    void appendCallSetResult(CCallHelpers::Address address, GPRReg result1, GPRReg result2)
     1048    {
     1049        appendCall(address);
     1050        m_jit.setupResults(result1, result2);
     1051    }
     1052
     1053    void appendCallSetResult(CCallHelpers::Address address, JSValueRegs resultRegs)
     1054    {
     1055#if USE(JSVALUE64)
     1056        appendCallSetResult(address, resultRegs.gpr());
     1057#else
     1058        appendCallSetResult(address, resultRegs.payloadGPR(), resultRegs.tagGPR());
     1059#endif
    10111060    }
    10121061

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp

    r278462 r278656  
    169169}
    170170
    171 void SpeculativeJIT::cachedGetById(CodeOrigin origin, JSValueRegs base, JSValueRegs result, CacheableIdentifier identifier, JITCompiler::Jump slowPathTarget , SpillRegistersMode mode, AccessType type)
     171void SpeculativeJIT::cachedGetById(CodeOrigin origin, JSValueRegs base, JSValueRegs result, GPRReg stubInfoGPR, CacheableIdentifier identifier, JITCompiler::Jump slowPathTarget , SpillRegistersMode mode, AccessType type)
    172172{
    173     cachedGetById(origin, base.tagGPR(), base.payloadGPR(), result.tagGPR(), result.payloadGPR(), identifier, slowPathTarget, mode, type);
     173    cachedGetById(origin, base.tagGPR(), base.payloadGPR(), result.tagGPR(), result.payloadGPR(), stubInfoGPR, identifier, slowPathTarget, mode, type);
    174174}
    175175
    176176void SpeculativeJIT::cachedGetById(
    177     CodeOrigin codeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR,
     177    CodeOrigin codeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, GPRReg stubInfoGPR,
    178178    CacheableIdentifier identifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode, AccessType type)
    179179{
     180    UNUSED_PARAM(stubInfoGPR);
    180181    // This is a hacky fix for when the register allocator decides to alias the base payload with the result tag. This only happens
    181182    // in the case of GetByIdFlush/GetByIdDirectFlush, which has a relatively expensive register allocation story already so we probably don't need to
    … …  
    201202    CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(codeOrigin, m_stream->size());
    202203    JITGetByIdGenerator gen(
    203         m_jit.codeBlock(), codeOrigin, callSite, usedRegisters, identifier,
    204         JSValueRegs(baseTagGPROrNone, basePayloadGPR), JSValueRegs(resultTagGPR, resultPayloadGPR), type);
     204        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters, identifier,
     205        JSValueRegs(baseTagGPROrNone, basePayloadGPR), JSValueRegs(resultTagGPR, resultPayloadGPR), InvalidGPRReg, type);
    205206   
    206207    gen.generateFastPath(m_jit);
    … …  
    229230
    230231void SpeculativeJIT::cachedGetByIdWithThis(
    231     CodeOrigin codeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg thisTagGPR, GPRReg thisPayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR,
     232    CodeOrigin codeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg thisTagGPR, GPRReg thisPayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, GPRReg stubInfoGPR,
    232233    CacheableIdentifier identifier, const JITCompiler::JumpList& slowPathTarget)
    233234{
     235    UNUSED_PARAM(stubInfoGPR);
    234236    RegisterSet usedRegisters = this->usedRegisters();
    235237   
    236238    CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(codeOrigin, m_stream->size());
    237239    JITGetByIdWithThisGenerator gen(
    238         m_jit.codeBlock(), codeOrigin, callSite, usedRegisters, identifier,
    239         JSValueRegs(resultTagGPR, resultPayloadGPR), JSValueRegs(baseTagGPROrNone, basePayloadGPR), JSValueRegs(thisTagGPR, thisPayloadGPR));
     240        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters, identifier,
     241        JSValueRegs(resultTagGPR, resultPayloadGPR), JSValueRegs(baseTagGPROrNone, basePayloadGPR), JSValueRegs(thisTagGPR, thisPayloadGPR), InvalidGPRReg);
    240242   
    241243    gen.generateFastPath(m_jit);
    … …  
    23422344
    23432345            JITGetByValGenerator gen(
    2344                 m_jit.codeBlock(), codeOrigin, callSite, AccessType::GetByVal, usedRegisters,
    2345                 baseRegs, propertyRegs, resultRegs);
     2346                m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, AccessType::GetByVal, usedRegisters,
     2347                baseRegs, propertyRegs, resultRegs, InvalidGPRReg);
    23462348
    23472349            if (m_state.forNode(m_graph.varArgChild(node, 1)).isType(SpecString))
    … …  
    34103412            GPRReg resultPayloadGPR = resultPayload.gpr();
    34113413           
    3412             cachedGetByIdWithThis(node->origin.semantic, InvalidGPRReg, baseGPR, InvalidGPRReg, thisGPR, resultTagGPR, resultPayloadGPR, node->cacheableIdentifier());
     3414            cachedGetByIdWithThis(node->origin.semantic, InvalidGPRReg, baseGPR, InvalidGPRReg, thisGPR, resultTagGPR, resultPayloadGPR, InvalidGPRReg, node->cacheableIdentifier());
    34133415           
    34143416            jsValueResult(resultTagGPR, resultPayloadGPR, node);
    … …  
    34303432            notCellList.append(m_jit.branchIfNotCell(thisValue.jsValueRegs()));
    34313433           
    3432             cachedGetByIdWithThis(node->origin.semantic, baseTagGPR, basePayloadGPR, thisTagGPR, thisPayloadGPR, resultTagGPR, resultPayloadGPR, node->cacheableIdentifier(), notCellList);
     3434            cachedGetByIdWithThis(node->origin.semantic, baseTagGPR, basePayloadGPR, thisTagGPR, thisPayloadGPR, resultTagGPR, resultPayloadGPR, InvalidGPRReg, node->cacheableIdentifier(), notCellList);
    34333435           
    34343436            jsValueResult(resultTagGPR, resultPayloadGPR, node);

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp

    r278462 r278656  
    148148}
    149149
    150 void SpeculativeJIT::cachedGetById(CodeOrigin origin, JSValueRegs base, JSValueRegs result, CacheableIdentifier identifier, JITCompiler::Jump slowPathTarget , SpillRegistersMode mode, AccessType type)
     150void SpeculativeJIT::cachedGetById(CodeOrigin origin, JSValueRegs base, JSValueRegs result, GPRReg stubInfoGPR, CacheableIdentifier identifier, JITCompiler::Jump slowPathTarget , SpillRegistersMode mode, AccessType type)
    151151{
    152     cachedGetById(origin, base.gpr(), result.gpr(), identifier, slowPathTarget, mode, type);
     152    cachedGetById(origin, base.gpr(), result.gpr(), stubInfoGPR, identifier, slowPathTarget, mode, type);
    153153}
    154154
    155 void SpeculativeJIT::cachedGetById(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg resultGPR, CacheableIdentifier identifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode, AccessType type)
     155void SpeculativeJIT::cachedGetById(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg resultGPR, GPRReg stubInfoGPR, CacheableIdentifier identifier, JITCompiler::Jump slowPathTarget, SpillRegistersMode spillMode, AccessType type)
    156156{
    157157    CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(codeOrigin, m_stream->size());
    … …  
    161161        usedRegisters.set(baseGPR, false);
    162162        usedRegisters.set(resultGPR, false);
     163        if (stubInfoGPR != InvalidGPRReg)
     164            usedRegisters.set(stubInfoGPR, false);
    163165    }
    164166    JITGetByIdGenerator gen(
    165         m_jit.codeBlock(), codeOrigin, callSite, usedRegisters, identifier,
    166         JSValueRegs(baseGPR), JSValueRegs(resultGPR), type);
     167        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters, identifier,
     168        JSValueRegs(baseGPR), JSValueRegs(resultGPR), stubInfoGPR, type);
    167169    gen.generateFastPath(m_jit);
    168170   
    … …  
    171173    slowCases.append(gen.slowPathJump());
    172174
    173     std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    174         slowCases, this, appropriateOptimizingGetByIdFunction(type),
    175         spillMode, ExceptionCheckRequirement::CheckNeeded,
    176         resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), baseGPR, identifier.rawBits());
     175    std::unique_ptr<SlowPathGenerator> slowPath;
     176    if (JITCode::useDataIC(JITType::DFGJIT)) {
     177        slowPath = slowPathICCall(
     178            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), appropriateOptimizingGetByIdFunction(type),
     179            spillMode, ExceptionCheckRequirement::CheckNeeded,
     180            resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, baseGPR, identifier.rawBits());
     181    } else {
     182        slowPath = slowPathCall(
     183            slowCases, this, appropriateOptimizingGetByIdFunction(type),
     184            spillMode, ExceptionCheckRequirement::CheckNeeded,
     185            resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), baseGPR, identifier.rawBits());
     186    }
    177187   
    178188    m_jit.addGetById(gen, slowPath.get());
    … …  
    180190}
    181191
    182 void SpeculativeJIT::cachedGetByIdWithThis(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg thisGPR, GPRReg resultGPR, CacheableIdentifier identifier, const JITCompiler::JumpList& slowPathTarget)
     192void SpeculativeJIT::cachedGetByIdWithThis(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg thisGPR, GPRReg resultGPR, GPRReg stubInfoGPR, CacheableIdentifier identifier, const JITCompiler::JumpList& slowPathTarget)
    183193{
    184194    CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(codeOrigin, m_stream->size());
    … …  
    188198    usedRegisters.set(thisGPR, false);
    189199    usedRegisters.set(resultGPR, false);
     200    if (stubInfoGPR != InvalidGPRReg)
     201        usedRegisters.set(stubInfoGPR, false);
    190202   
    191203    JITGetByIdWithThisGenerator gen(
    192         m_jit.codeBlock(), codeOrigin, callSite, usedRegisters, identifier,
    193         JSValueRegs(resultGPR), JSValueRegs(baseGPR), JSValueRegs(thisGPR));
     204        m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, usedRegisters, identifier,
     205        JSValueRegs(resultGPR), JSValueRegs(baseGPR), JSValueRegs(thisGPR), stubInfoGPR);
    194206    gen.generateFastPath(m_jit);
    195207   
    … …  
    198210    slowCases.append(gen.slowPathJump());
    199211   
    200     std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    201         slowCases, this, operationGetByIdWithThisOptimize,
    202         DontSpill, ExceptionCheckRequirement::CheckNeeded,
    203         resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), baseGPR, thisGPR, identifier.rawBits());
     212    std::unique_ptr<SlowPathGenerator> slowPath;
     213    if (JITCode::useDataIC(JITType::DFGJIT)) {
     214        slowPath = slowPathICCall(
     215            slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationGetByIdWithThisOptimize,
     216            DontSpill, ExceptionCheckRequirement::CheckNeeded,
     217            resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, baseGPR, thisGPR, identifier.rawBits());
     218    } else {
     219        slowPath = slowPathCall(
     220            slowCases, this, operationGetByIdWithThisOptimize,
     221            DontSpill, ExceptionCheckRequirement::CheckNeeded,
     222            resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), baseGPR, thisGPR, identifier.rawBits());
     223    }
    204224   
    205225    m_jit.addGetByIdWithThis(gen, slowPath.get());
    … …  
    26822702            }
    26832703
    2684             speculate(node, m_graph.varArgChild(node, 0));
    2685             speculate(node, m_graph.varArgChild(node, 1));
    2686 
     2704            std::optional<GPRTemporary> stubInfo;
    26872705            JSValueOperand base(this, m_graph.varArgChild(node, 0), ManualOperandSpeculation);
    26882706            JSValueOperand property(this, m_graph.varArgChild(node, 1), ManualOperandSpeculation);
    26892707            GPRTemporary result(this, Reuse, property);
     2708
     2709            GPRReg stubInfoGPR = InvalidGPRReg;
     2710            if (JITCode::useDataIC(JITType::DFGJIT)) {
     2711                stubInfo.emplace(this);
     2712                stubInfoGPR = stubInfo->gpr();
     2713            }
    26902714            GPRReg baseGPR = base.gpr();
    26912715            GPRReg propertyGPR = property.gpr();
    26922716            GPRReg resultGPR = result.gpr();
    26932717
     2718            speculate(node, m_graph.varArgChild(node, 0));
     2719            speculate(node, m_graph.varArgChild(node, 1));
     2720
    26942721            CodeOrigin codeOrigin = node->origin.semantic;
    26952722            CallSiteIndex callSite = m_jit.recordCallSiteAndGenerateExceptionHandlingOSRExitIfNeeded(codeOrigin, m_stream->size());
    … …  
    27012728
    27022729            JITGetByValGenerator gen(
    2703                 m_jit.codeBlock(), codeOrigin, callSite, AccessType::GetByVal, usedRegisters,
    2704                 JSValueRegs(baseGPR), JSValueRegs(propertyGPR), JSValueRegs(resultGPR));
     2730                m_jit.codeBlock(), JITType::DFGJIT, codeOrigin, callSite, AccessType::GetByVal, usedRegisters,
     2731                JSValueRegs(baseGPR), JSValueRegs(propertyGPR), JSValueRegs(resultGPR), stubInfoGPR);
    27052732
    27062733            if (m_state.forNode(m_graph.varArgChild(node, 1)).isType(SpecString))
    … …  
    27132740            gen.generateFastPath(m_jit);
    27142741           
    2715             slowCases.append(gen.slowPathJump());
    2716 
    2717             std::unique_ptr<SlowPathGenerator> slowPath = slowPathCall(
    2718                 slowCases, this, operationGetByValOptimize,
    2719                 resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), nullptr, baseGPR, propertyGPR);
     2742            if (!JITCode::useDataIC(JITType::DFGJIT))
     2743                slowCases.append(gen.slowPathJump());
     2744
     2745            std::unique_ptr<SlowPathGenerator> slowPath;
     2746            if (JITCode::useDataIC(JITType::DFGJIT)) {
     2747                slowPath = slowPathICCall(
     2748                    slowCases, this, gen.stubInfo(), stubInfoGPR, CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), operationGetByValOptimize,
     2749                    resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), stubInfoGPR, nullptr, baseGPR, propertyGPR);
     2750            } else {
     2751                slowPath = slowPathCall(
     2752                    slowCases, this, operationGetByValOptimize,
     2753                    resultGPR, TrustedImmPtr::weakPointer(m_graph, m_graph.globalObjectFor(codeOrigin)), gen.stubInfo(), nullptr, baseGPR, propertyGPR);
     2754            }
    27202755           
    27212756            m_jit.addGetByVal(gen, slowPath.get());
    … …  
    39894024    case GetByIdWithThis: {
    39904025        if (node->child1().useKind() == CellUse && node->child2().useKind() == CellUse) {
     4026            std::optional<GPRTemporary> stubInfo;
    39914027            SpeculateCellOperand base(this, node->child1());
     4028            SpeculateCellOperand thisValue(this, node->child2());
     4029
     4030            GPRReg stubInfoGPR = InvalidGPRReg;
     4031            if (JITCode::useDataIC(JITType::DFGJIT)) {
     4032                stubInfo.emplace(this);
     4033                stubInfoGPR = stubInfo->gpr();
     4034            }
    39924035            GPRReg baseGPR = base.gpr();
    3993             SpeculateCellOperand thisValue(this, node->child2());
    39944036            GPRReg thisValueGPR = thisValue.gpr();
    39954037           
    … …  
    39994041            flushRegisters();
    40004042           
    4001             cachedGetByIdWithThis(node->origin.semantic, baseGPR, thisValueGPR, resultGPR, node->cacheableIdentifier(), JITCompiler::JumpList());
     4043            cachedGetByIdWithThis(node->origin.semantic, baseGPR, thisValueGPR, resultGPR, stubInfoGPR, node->cacheableIdentifier(), JITCompiler::JumpList());
    40024044           
    40034045            jsValueResult(resultGPR, node);
    40044046           
    40054047        } else {
     4048            std::optional<GPRTemporary> stubInfo;
    40064049            JSValueOperand base(this, node->child1());
     4050            JSValueOperand thisValue(this, node->child2());
     4051
     4052            GPRReg stubInfoGPR = InvalidGPRReg;
     4053            if (JITCode::useDataIC(JITType::DFGJIT)) {
     4054                stubInfo.emplace(this);
     4055                stubInfoGPR = stubInfo->gpr();
     4056            }
    40074057            GPRReg baseGPR = base.gpr();
    4008             JSValueOperand thisValue(this, node->child2());
    40094058            GPRReg thisValueGPR = thisValue.gpr();
    40104059           
    … …  
    40184067            notCellList.append(m_jit.branchIfNotCell(JSValueRegs(thisValueGPR)));
    40194068           
    4020             cachedGetByIdWithThis(node->origin.semantic, baseGPR, thisValueGPR, resultGPR, node->cacheableIdentifier(), notCellList);
     4069            cachedGetByIdWithThis(node->origin.semantic, baseGPR, thisValueGPR, resultGPR, stubInfoGPR, node->cacheableIdentifier(), notCellList);
    40214070           
    40224071            jsValueResult(resultGPR, node);

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

    r278587 r278656  
    39893989        patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
    39903990        patchpoint->clobber(RegisterSet::macroScratchRegisters());
     3991        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 1 : 0;
    39913992
    39923993        RefPtr<PatchpointExceptionHandle> exceptionHandle = preparePatchpointForExceptions(patchpoint);
    … …  
    40104011                GPRReg baseGPR = params[1].gpr();
    40114012                GPRReg propertyGPR = params[2].gpr();
     4013                GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(0) : InvalidGPRReg;
    40124014
    40134015                auto generator = Box<JITGetByValGenerator>::create(
    4014                     jit.codeBlock(), node->origin.semantic, callSiteIndex, AccessType::GetPrivateName,
    4015                     params.unavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(propertyGPR), JSValueRegs(resultGPR));
     4016                    jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex, AccessType::GetPrivateName,
     4017                    params.unavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(propertyGPR), JSValueRegs(resultGPR), stubInfoGPR);
    40164018
    40174019                CCallHelpers::Jump notCell;
    … …  
    40274029                    if (notCell.isSet())
    40284030                        notCell.link(&jit);
    4029                     generator->slowPathJump().link(&jit);
     4031                    if (!JITCode::useDataIC(JITType::FTLJIT))
     4032                        generator->slowPathJump().link(&jit);
    40304033                    CCallHelpers::Label slowPathBegin = jit.label();
    4031                     CCallHelpers::Call slowPathCall = callOperation(
    4032                         *state, params.unavailableRegisters(), jit, node->origin.semantic,
    4033                         exceptions.get(), operationGetPrivateNameOptimize, resultGPR,
    4034                         jit.codeBlock()->globalObjectFor(node->origin.semantic),
    4035                         CCallHelpers::TrustedImmPtr(generator->stubInfo()), baseGPR, propertyGPR).call();
     4034                    CCallHelpers::Call slowPathCall;
     4035                    if (JITCode::useDataIC(JITType::FTLJIT)) {
     4036                        jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     4037                        generator->stubInfo()->m_slowOperation = operationGetPrivateNameOptimize;
     4038                        slowPathCall = callOperation(
     4039                            *state, params.unavailableRegisters(), jit, node->origin.semantic,
     4040                            exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), resultGPR,
     4041                            jit.codeBlock()->globalObjectFor(node->origin.semantic),
     4042                            stubInfoGPR, baseGPR, propertyGPR).call();
     4043                    } else {
     4044                        slowPathCall = callOperation(
     4045                            *state, params.unavailableRegisters(), jit, node->origin.semantic,
     4046                            exceptions.get(), operationGetPrivateNameOptimize, resultGPR,
     4047                            jit.codeBlock()->globalObjectFor(node->origin.semantic),
     4048                            CCallHelpers::TrustedImmPtr(generator->stubInfo()), baseGPR, propertyGPR).call();
     4049                    }
    40364050                    jit.jump().linkTo(done, &jit);
    40374051
    … …  
    41214135        patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
    41224136        patchpoint->clobber(RegisterSet::macroScratchRegisters());
     4137        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 1 : 0;
    41234138
    41244139        RefPtr<PatchpointExceptionHandle> exceptionHandle = preparePatchpointForExceptions(patchpoint);
    … …  
    41414156            GPRReg baseGPR = params[0].gpr();
    41424157            GPRReg brandGPR = params[1].gpr();
     4158            GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(0) : InvalidGPRReg;
    41434159
    41444160            auto generator = Box<JITPrivateBrandAccessGenerator>::create(
    4145                 jit.codeBlock(), node->origin.semantic, callSiteIndex, accessType,
    4146                 params.unavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(brandGPR));
     4161                jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex, accessType,
     4162                params.unavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(brandGPR), stubInfoGPR);
    41474163
    41484164            CCallHelpers::Jump notCell;
    … …  
    41704186                if (notCell.isSet())
    41714187                    notCell.link(&jit);
    4172                 generator->slowPathJump().link(&jit);
     4188                if (!JITCode::useDataIC(JITType::FTLJIT))
     4189                    generator->slowPathJump().link(&jit);
    41734190                CCallHelpers::Label slowPathBegin = jit.label();
    4174                 CCallHelpers::Call slowPathCall = callOperation(
    4175                     *state, params.unavailableRegisters(), jit, node->origin.semantic,
    4176                     exceptions.get(), appropriatePrivateAccessFunction(accessType), InvalidGPRReg,
    4177                     jit.codeBlock()->globalObjectFor(node->origin.semantic),
    4178                     CCallHelpers::TrustedImmPtr(generator->stubInfo()), baseGPR, brandGPR).call();
     4191                CCallHelpers::Call slowPathCall;
     4192                if (JITCode::useDataIC(JITType::FTLJIT)) {
     4193                    jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     4194                    generator->stubInfo()->m_slowOperation = appropriatePrivateAccessFunction(accessType);
     4195                    slowPathCall = callOperation(
     4196                        *state, params.unavailableRegisters(), jit, node->origin.semantic,
     4197                        exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), InvalidGPRReg,
     4198                        jit.codeBlock()->globalObjectFor(node->origin.semantic),
     4199                        stubInfoGPR, baseGPR, brandGPR).call();
     4200                } else {
     4201                    slowPathCall = callOperation(
     4202                        *state, params.unavailableRegisters(), jit, node->origin.semantic,
     4203                        exceptions.get(), appropriatePrivateAccessFunction(accessType), InvalidGPRReg,
     4204                        jit.codeBlock()->globalObjectFor(node->origin.semantic),
     4205                        CCallHelpers::TrustedImmPtr(generator->stubInfo()), baseGPR, brandGPR).call();
     4206                }
    41794207                jit.jump().linkTo(done, &jit);
    41804208
    … …  
    45274555        patchpoint->append(m_numberTag, ValueRep::reg(GPRInfo::numberTagRegister));
    45284556        patchpoint->clobber(RegisterSet::macroScratchRegisters());
     4557        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 1 : 0;
    45294558
    45304559        // FIXME: If this is a PutByIdFlush, we might want to late-clobber volatile registers.
    … …  
    45494578                exceptionHandle->scheduleExitCreationForUnwind(params, callSiteIndex);
    45504579
     4580                GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(0) : InvalidGPRReg;
     4581
    45514582                auto generator = Box<JITPutByIdGenerator>::create(
    4552                     jit.codeBlock(), node->origin.semantic, callSiteIndex,
     4583                    jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
    45534584                    params.unavailableRegisters(), identifier, JSValueRegs(params[0].gpr()),
    4554                     JSValueRegs(params[1].gpr()), GPRInfo::patchpointScratchRegister, ecmaMode,
     4585                    JSValueRegs(params[1].gpr()), stubInfoGPR, GPRInfo::patchpointScratchRegister, ecmaMode,
    45554586                    putKind);
    45564587
    … …  
    45644595                        generator->slowPathJump().link(&jit);
    45654596                        CCallHelpers::Label slowPathBegin = jit.label();
    4566                         CCallHelpers::Call slowPathCall = callOperation(
    4567                             *state, params.unavailableRegisters(), jit, node->origin.semantic,
    4568                             exceptions.get(), generator->slowPathFunction(), InvalidGPRReg,
    4569                             jit.codeBlock()->globalObjectFor(node->origin.semantic),
    4570                             CCallHelpers::TrustedImmPtr(generator->stubInfo()), params[1].gpr(),
    4571                             params[0].gpr(), identifier.rawBits()).call();
     4597                        CCallHelpers::Call slowPathCall;
     4598                        if (JITCode::useDataIC(JITType::FTLJIT)) {
     4599                            jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     4600                            generator->stubInfo()->m_slowOperation = generator->slowPathFunction();
     4601                            slowPathCall = callOperation(
     4602                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     4603                                exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), InvalidGPRReg,
     4604                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     4605                                stubInfoGPR, params[1].gpr(),
     4606                                params[0].gpr(), identifier.rawBits()).call();
     4607                        } else {
     4608                            slowPathCall = callOperation(
     4609                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     4610                                exceptions.get(), generator->slowPathFunction(), InvalidGPRReg,
     4611                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     4612                                CCallHelpers::TrustedImmPtr(generator->stubInfo()), params[1].gpr(),
     4613                                params[0].gpr(), identifier.rawBits()).call();
     4614                        }
    45724615                        jit.jump().linkTo(done, &jit);
    45734616
    … …  
    52505293            patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
    52515294            patchpoint->clobber(RegisterSet::macroScratchRegisters());
     5295            patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 1 : 0;
    52525296
    52535297            RefPtr<PatchpointExceptionHandle> exceptionHandle = preparePatchpointForExceptions(patchpoint);
    … …  
    52705314                GPRReg baseGPR = params[1].gpr();
    52715315                GPRReg propertyGPR = params[2].gpr();
     5316                GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(0) : InvalidGPRReg;
    52725317
    52735318                auto generator = Box<JITGetByValGenerator>::create(
    5274                     jit.codeBlock(), node->origin.semantic, callSiteIndex, AccessType::GetByVal,
    5275                     params.unavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(propertyGPR), JSValueRegs(resultGPR));
     5319                    jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex, AccessType::GetByVal,
     5320                    params.unavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(propertyGPR), JSValueRegs(resultGPR), stubInfoGPR);
    52765321
    52775322                generator->stubInfo()->propertyIsString = propertyIsString;
    … …  
    52915336                    if (notCell.isSet())
    52925337                        notCell.link(&jit);
    5293                     generator->slowPathJump().link(&jit);
     5338                    if (!JITCode::useDataIC(JITType::FTLJIT))
     5339                        generator->slowPathJump().link(&jit);
    52945340                    CCallHelpers::Label slowPathBegin = jit.label();
    5295                     CCallHelpers::Call slowPathCall = callOperation(
    5296                         *state, params.unavailableRegisters(), jit, node->origin.semantic,
    5297                         exceptions.get(), operationGetByValOptimize, resultGPR,
    5298                         jit.codeBlock()->globalObjectFor(node->origin.semantic),
    5299                         CCallHelpers::TrustedImmPtr(generator->stubInfo()), CCallHelpers::TrustedImmPtr(nullptr), baseGPR, propertyGPR).call();
     5341                    CCallHelpers::Call slowPathCall;
     5342                    if (JITCode::useDataIC(JITType::FTLJIT)) {
     5343                        jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     5344                        generator->stubInfo()->m_slowOperation = operationGetByValOptimize;
     5345                        slowPathCall = callOperation(
     5346                            *state, params.unavailableRegisters(), jit, node->origin.semantic,
     5347                            exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), resultGPR,
     5348                            jit.codeBlock()->globalObjectFor(node->origin.semantic),
     5349                            stubInfoGPR, CCallHelpers::TrustedImmPtr(nullptr), baseGPR, propertyGPR).call();
     5350                    } else {
     5351                        slowPathCall = callOperation(
     5352                            *state, params.unavailableRegisters(), jit, node->origin.semantic,
     5353                            exceptions.get(), operationGetByValOptimize, resultGPR,
     5354                            jit.codeBlock()->globalObjectFor(node->origin.semantic),
     5355                            CCallHelpers::TrustedImmPtr(generator->stubInfo()), CCallHelpers::TrustedImmPtr(nullptr), baseGPR, propertyGPR).call();
     5356                    }
    53005357                    jit.jump().linkTo(done, &jit);
    53015358
    … …  
    58475904        patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
    58485905        patchpoint->clobber(RegisterSet::macroScratchRegisters());
    5849         patchpoint->numGPScratchRegisters = 1;
     5906        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 2 : 1;
    58505907
    58515908        RefPtr<PatchpointExceptionHandle> exceptionHandle =
    … …  
    58675924                auto base = JSValueRegs(params[1].gpr());
    58685925                auto returnGPR = params[0].gpr();
     5926                auto scratchGPR = params.gpScratch(0);
     5927                auto stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(1) : InvalidGPRReg;
    58695928                ASSERT(base.gpr() != returnGPR);
    5870                 ASSERT(base.gpr() != params.gpScratch(0));
    5871                 ASSERT(returnGPR != params.gpScratch(0));
     5929                ASSERT(base.gpr() != scratchGPR);
     5930                ASSERT(returnGPR != scratchGPR);
    58725931
    58735932                if (node->child1().useKind() == UntypedUse)
    … …  
    58855944                        return CCallHelpers::TrustedImmPtr(subscriptValue.rawBits());
    58865945                    else {
    5887                         ASSERT(params.gpScratch(0) != params[2].gpr());
     5946                        ASSERT(scratchGPR != params[2].gpr());
    58885947                        if (node->child2().useKind() == UntypedUse)
    58895948                            slowCases.append(jit.branchIfNotCell(JSValueRegs(params[2].gpr())));
    … …  
    58955954                    if constexpr (kind == DelByKind::Normal) {
    58965955                        return Box<JITDelByIdGenerator>::create(
    5897                             jit.codeBlock(), node->origin.semantic, callSiteIndex,
     5956                            jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
    58985957                            params.unavailableRegisters(), subscriptValue, base,
    5899                             JSValueRegs(returnGPR), params.gpScratch(0));
     5958                            JSValueRegs(returnGPR), stubInfoGPR, scratchGPR);
    59005959                    } else {
    59015960                        return Box<JITDelByValGenerator>::create(
    5902                             jit.codeBlock(), node->origin.semantic, callSiteIndex,
     5961                            jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
    59035962                            params.unavailableRegisters(), base,
    5904                             subscript, JSValueRegs(returnGPR), params.gpScratch(0));
     5963                            subscript, JSValueRegs(returnGPR), stubInfoGPR, scratchGPR);
    59055964                    }
    59065965                }();
    59075966
    59085967                generator->generateFastPath(jit);
    5909                 slowCases.append(generator->slowPathJump());
     5968                if (!JITCode::useDataIC(JITType::FTLJIT))
     5969                    slowCases.append(generator->slowPathJump());
    59105970                CCallHelpers::Label done = jit.label();
    59115971
    … …  
    59165976                        slowCases.link(&jit);
    59175977                        CCallHelpers::Label slowPathBegin = jit.label();
    5918                         CCallHelpers::Call slowPathCall = callOperation(
    5919                             *state, params.unavailableRegisters(), jit, node->origin.semantic,
    5920                             exceptions.get(), optimizationFunction, returnGPR,
    5921                             jit.codeBlock()->globalObjectFor(node->origin.semantic),
    5922                             CCallHelpers::TrustedImmPtr(generator->stubInfo()), base,
    5923                             subscript, CCallHelpers::TrustedImm32(node->ecmaMode().value())).call();
     5978                        CCallHelpers::Call slowPathCall;
     5979                        if (JITCode::useDataIC(JITType::FTLJIT)) {
     5980                            jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     5981                            generator->stubInfo()->m_slowOperation = optimizationFunction;
     5982                            slowPathCall = callOperation(
     5983                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     5984                                exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), returnGPR,
     5985                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     5986                                stubInfoGPR, base,
     5987                                subscript, CCallHelpers::TrustedImm32(node->ecmaMode().value())).call();
     5988                        } else {
     5989                            slowPathCall = callOperation(
     5990                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     5991                                exceptions.get(), optimizationFunction, returnGPR,
     5992                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     5993                                CCallHelpers::TrustedImmPtr(generator->stubInfo()), base,
     5994                                subscript, CCallHelpers::TrustedImm32(node->ecmaMode().value())).call();
     5995                        }
    59245996                        jit.jump().linkTo(done, &jit);
    59255997
    … …  
    1230212374        patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
    1230312375        patchpoint->clobber(RegisterSet::macroScratchRegisters());
     12376        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 1 : 0;
    1230412377
    1230512378        RefPtr<PatchpointExceptionHandle> exceptionHandle = preparePatchpointForExceptions(patchpoint);
    … …  
    1231612389                Box<CCallHelpers::JumpList> exceptions = exceptionHandle->scheduleExitCreation(params)->jumps(jit);
    1231712390
     12391                GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(0) : InvalidGPRReg;
    1231812392                auto returnGPR = params[0].gpr();
    1231912393                auto base = JSValueRegs(params[1].gpr());
    … …  
    1232912403                    if constexpr (kind == InByKind::Normal) {
    1233012404                        return Box<JITInByIdGenerator>::create(
    12331                             jit.codeBlock(), node->origin.semantic, callSiteIndex,
     12405                            jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
    1233212406                            params.unavailableRegisters(), subscriptValue, base,
    12333                             JSValueRegs(returnGPR));
     12407                            JSValueRegs(returnGPR), stubInfoGPR);
    1233412408                    } else {
    1233512409                        return Box<JITInByValGenerator>::create(
    12336                             jit.codeBlock(), node->origin.semantic, callSiteIndex,
     12410                            jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
    1233712411                            params.unavailableRegisters(), base, subscript,
    12338                             JSValueRegs(returnGPR));
     12412                            JSValueRegs(returnGPR), stubInfoGPR);
    1233912413                    }
    1234012414                }();
    1234112415
     12416                CCallHelpers::JumpList slowCases;
    1234212417                generator->generateFastPath(jit);
     12418                if constexpr (kind == InByKind::Normal)
     12419                    slowCases.append(generator->slowPathJump());
     12420                else {
     12421                    if (!JITCode::useDataIC(JITType::FTLJIT))
     12422                        slowCases.append(generator->slowPathJump());
     12423                }
    1234312424                CCallHelpers::Label done = jit.label();
    1234412425
    … …  
    1234712428                        AllowMacroScratchRegisterUsage allowScratch(jit);
    1234812429
    12349                         generator->slowPathJump().link(&jit);
     12430                        slowCases.link(&jit);
    1235012431                        CCallHelpers::Label slowPathBegin = jit.label();
    1235112432                        CCallHelpers::Call slowPathCall;
    1235212433                        if constexpr (kind == InByKind::Normal) {
    12353                             slowPathCall = callOperation(
    12354                                 *state, params.unavailableRegisters(), jit, node->origin.semantic,
    12355                                 exceptions.get(), operationInByIdOptimize, returnGPR,
    12356                                 jit.codeBlock()->globalObjectFor(node->origin.semantic),
    12357                                 CCallHelpers::TrustedImmPtr(generator->stubInfo()),
    12358                                 base, subscript).call();
     12434                            if (JITCode::useDataIC(JITType::FTLJIT)) {
     12435                                jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     12436                                generator->stubInfo()->m_slowOperation = operationInByIdOptimize;
     12437                                slowPathCall = callOperation(
     12438                                    *state, params.unavailableRegisters(), jit, node->origin.semantic,
     12439                                    exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), returnGPR,
     12440                                    jit.codeBlock()->globalObjectFor(node->origin.semantic),
     12441                                    stubInfoGPR, base, subscript).call();
     12442                            } else {
     12443                                slowPathCall = callOperation(
     12444                                    *state, params.unavailableRegisters(), jit, node->origin.semantic,
     12445                                    exceptions.get(), operationInByIdOptimize, returnGPR,
     12446                                    jit.codeBlock()->globalObjectFor(node->origin.semantic),
     12447                                    CCallHelpers::TrustedImmPtr(generator->stubInfo()), base, subscript).call();
     12448                            }
    1235912449                        } else {
    12360                             slowPathCall = callOperation(
    12361                                 *state, params.unavailableRegisters(), jit, node->origin.semantic,
    12362                                 exceptions.get(), operationInByValOptimize, returnGPR,
    12363                                 jit.codeBlock()->globalObjectFor(node->origin.semantic),
    12364                                 CCallHelpers::TrustedImmPtr(generator->stubInfo()),
    12365                                 CCallHelpers::TrustedImmPtr(nullptr), base, subscript).call();
     12450                            if (JITCode::useDataIC(JITType::FTLJIT)) {
     12451                                jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     12452                                generator->stubInfo()->m_slowOperation = operationInByValOptimize;
     12453                                slowPathCall = callOperation(
     12454                                    *state, params.unavailableRegisters(), jit, node->origin.semantic,
     12455                                    exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), returnGPR,
     12456                                    jit.codeBlock()->globalObjectFor(node->origin.semantic),
     12457                                    stubInfoGPR,
     12458                                    CCallHelpers::TrustedImmPtr(nullptr), base, subscript).call();
     12459                            } else {
     12460                                slowPathCall = callOperation(
     12461                                    *state, params.unavailableRegisters(), jit, node->origin.semantic,
     12462                                    exceptions.get(), operationInByValOptimize, returnGPR,
     12463                                    jit.codeBlock()->globalObjectFor(node->origin.semantic),
     12464                                    CCallHelpers::TrustedImmPtr(generator->stubInfo()),
     12465                                    CCallHelpers::TrustedImmPtr(nullptr), base, subscript).call();
     12466                            }
    1236612467                        }
    1236712468                        jit.jump().linkTo(done, &jit);
    … …  
    1258112682        patchpoint->append(m_notCellMask, ValueRep::lateReg(GPRInfo::notCellMaskRegister));
    1258212683        patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
    12583         patchpoint->numGPScratchRegisters = 2;
     12684        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 3 : 2;
    1258412685        patchpoint->resultConstraints = { ValueRep::SomeEarlyRegister };
    1258512686        patchpoint->clobber(RegisterSet::macroScratchRegisters());
    … …  
    1259712698                GPRReg scratchGPR = params.gpScratch(0);
    1259812699                GPRReg scratch2GPR = params.gpScratch(1);
     12700                GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(2) : InvalidGPRReg;
    1259912701               
    1260012702                CCallHelpers::Jump doneJump;
    … …  
    1261812720               
    1261912721                auto generator = Box<JITInstanceOfGenerator>::create(
    12620                     jit.codeBlock(), node->origin.semantic, callSiteIndex,
    12621                     params.unavailableRegisters(), resultGPR, valueGPR, prototypeGPR, scratchGPR,
     12722                    jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
     12723                    params.unavailableRegisters(), resultGPR, valueGPR, prototypeGPR, stubInfoGPR, scratchGPR,
    1262212724                    scratch2GPR, prototypeIsObject);
    1262312725                generator->generateFastPath(jit);
     12726                if (!JITCode::useDataIC(JITType::FTLJIT))
     12727                    slowCases.append(generator->slowPathJump());
    1262412728                CCallHelpers::Label done = jit.label();
    1262512729               
    … …  
    1263212736                        slowCases.link(&jit);
    1263312737                        CCallHelpers::Label slowPathBegin = jit.label();
    12634                         CCallHelpers::Call slowPathCall = callOperation(
    12635                             *state, params.unavailableRegisters(), jit, node->origin.semantic,
    12636                             exceptions.get(), optimizationFunction, resultGPR,
    12637                             jit.codeBlock()->globalObjectFor(node->origin.semantic),
    12638                             CCallHelpers::TrustedImmPtr(generator->stubInfo()), valueGPR,
    12639                             prototypeGPR).call();
     12738                        CCallHelpers::Call slowPathCall;
     12739                        if (JITCode::useDataIC(JITType::FTLJIT)) {
     12740                            jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     12741                            generator->stubInfo()->m_slowOperation = optimizationFunction;
     12742                            slowPathCall = callOperation(
     12743                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     12744                                exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), resultGPR,
     12745                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     12746                                stubInfoGPR, valueGPR,
     12747                                prototypeGPR).call();
     12748                        } else {
     12749                            slowPathCall = callOperation(
     12750                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     12751                                exceptions.get(), optimizationFunction, resultGPR,
     12752                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     12753                                CCallHelpers::TrustedImmPtr(generator->stubInfo()), valueGPR,
     12754                                prototypeGPR).call();
     12755                        }
    1264012756                        jit.jump().linkTo(done, &jit);
    1264112757                       
    … …  
    1408114197        patchpoint->append(m_notCellMask, ValueRep::lateReg(GPRInfo::notCellMaskRegister));
    1408214198        patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
     14199        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 1 : 0;
    1408314200
    1408414201        // FIXME: If this is a GetByIdFlush/GetByIdDirectFlush, we might get some performance boost if we claim that it
    … …  
    1410914226                exceptionHandle->scheduleExitCreationForUnwind(params, callSiteIndex);
    1411014227
     14228                GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(0) : InvalidGPRReg;
     14229
    1411114230                auto generator = Box<JITGetByIdGenerator>::create(
    14112                     jit.codeBlock(), node->origin.semantic, callSiteIndex,
     14231                    jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
    1411314232                    params.unavailableRegisters(), identifier, JSValueRegs(params[1].gpr()),
    14114                     JSValueRegs(params[0].gpr()), type);
     14233                    JSValueRegs(params[0].gpr()), stubInfoGPR, type);
    1411514234
    1411614235                generator->generateFastPath(jit);
    … …  
    1412514244                        generator->slowPathJump().link(&jit);
    1412614245                        CCallHelpers::Label slowPathBegin = jit.label();
    14127                         CCallHelpers::Call slowPathCall = callOperation(
    14128                             *state, params.unavailableRegisters(), jit, node->origin.semantic,
    14129                             exceptions.get(), optimizationFunction, params[0].gpr(),
    14130                             jit.codeBlock()->globalObjectFor(node->origin.semantic),
    14131                             CCallHelpers::TrustedImmPtr(generator->stubInfo()), params[1].gpr(),
    14132                             CCallHelpers::TrustedImmPtr(identifier.rawBits())).call();
     14246                        CCallHelpers::Call slowPathCall;
     14247                        if (JITCode::useDataIC(JITType::FTLJIT)) {
     14248                            jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     14249                            generator->stubInfo()->m_slowOperation = optimizationFunction;
     14250                            slowPathCall = callOperation(
     14251                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     14252                                exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), params[0].gpr(),
     14253                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     14254                                stubInfoGPR, params[1].gpr(),
     14255                                CCallHelpers::TrustedImmPtr(identifier.rawBits())).call();
     14256                        } else {
     14257                            slowPathCall = callOperation(
     14258                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     14259                                exceptions.get(), optimizationFunction, params[0].gpr(),
     14260                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     14261                                CCallHelpers::TrustedImmPtr(generator->stubInfo()), params[1].gpr(),
     14262                                CCallHelpers::TrustedImmPtr(identifier.rawBits())).call();
     14263                        }
    1413314264                        jit.jump().linkTo(done, &jit);
    1413414265
    … …  
    1415514286        patchpoint->append(m_notCellMask, ValueRep::lateReg(GPRInfo::notCellMaskRegister));
    1415614287        patchpoint->append(m_numberTag, ValueRep::lateReg(GPRInfo::numberTagRegister));
    14157 
    1415814288        patchpoint->clobber(RegisterSet::macroScratchRegisters());
     14289        patchpoint->numGPScratchRegisters = JITCode::useDataIC(JITType::FTLJIT) ? 1 : 0;
    1415914290
    1416014291        RefPtr<PatchpointExceptionHandle> exceptionHandle =
    … …  
    1417814309                exceptionHandle->scheduleExitCreationForUnwind(params, callSiteIndex);
    1417914310
     14311                GPRReg stubInfoGPR = JITCode::useDataIC(JITType::FTLJIT) ? params.gpScratch(0) : InvalidGPRReg;
     14312
    1418014313                auto generator = Box<JITGetByIdWithThisGenerator>::create(
    14181                     jit.codeBlock(), node->origin.semantic, callSiteIndex,
     14314                    jit.codeBlock(), JITType::FTLJIT, node->origin.semantic, callSiteIndex,
    1418214315                    params.unavailableRegisters(), identifier, JSValueRegs(params[0].gpr()),
    14183                     JSValueRegs(params[1].gpr()), JSValueRegs(params[2].gpr()));
     14316                    JSValueRegs(params[1].gpr()), JSValueRegs(params[2].gpr()), stubInfoGPR);
    1418414317
    1418514318                generator->generateFastPath(jit);
    … …  
    1419414327                        generator->slowPathJump().link(&jit);
    1419514328                        CCallHelpers::Label slowPathBegin = jit.label();
    14196                         CCallHelpers::Call slowPathCall = callOperation(
    14197                             *state, params.unavailableRegisters(), jit, node->origin.semantic,
    14198                             exceptions.get(), optimizationFunction, params[0].gpr(),
    14199                             jit.codeBlock()->globalObjectFor(node->origin.semantic),
    14200                             CCallHelpers::TrustedImmPtr(generator->stubInfo()), params[1].gpr(),
    14201                             params[2].gpr(), CCallHelpers::TrustedImmPtr(identifier.rawBits())).call();
     14329                        CCallHelpers::Call slowPathCall;
     14330                        if (JITCode::useDataIC(JITType::FTLJIT)) {
     14331                            jit.move(CCallHelpers::TrustedImmPtr(generator->stubInfo()), stubInfoGPR);
     14332                            generator->stubInfo()->m_slowOperation = optimizationFunction;
     14333                            slowPathCall = callOperation(
     14334                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     14335                                exceptions.get(), CCallHelpers::Address(stubInfoGPR, StructureStubInfo::offsetOfSlowOperation()), params[0].gpr(),
     14336                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     14337                                stubInfoGPR, params[1].gpr(),
     14338                                params[2].gpr(), CCallHelpers::TrustedImmPtr(identifier.rawBits())).call();
     14339                        } else {
     14340                            slowPathCall = callOperation(
     14341                                *state, params.unavailableRegisters(), jit, node->origin.semantic,
     14342                                exceptions.get(), optimizationFunction, params[0].gpr(),
     14343                                jit.codeBlock()->globalObjectFor(node->origin.semantic),
     14344                                CCallHelpers::TrustedImmPtr(generator->stubInfo()), params[1].gpr(),
     14345                                params[2].gpr(), CCallHelpers::TrustedImmPtr(identifier.rawBits())).call();
     14346                        }
    1420214347                        jit.jump().linkTo(done, &jit);
    1420314348

  • trunk/Source/JavaScriptCore/ftl/FTLSlowPathCall.cpp

    r268247 r278656  
    4040
    4141SlowPathCallContext::SlowPathCallContext(
    42     RegisterSet usedRegisters, CCallHelpers& jit, unsigned numArgs, GPRReg returnRegister)
     42    RegisterSet usedRegisters, CCallHelpers& jit, unsigned numArgs, GPRReg returnRegister, GPRReg indirectCallTargetRegister)
    4343    : m_jit(jit)
    4444    , m_numArgs(numArgs)
    … …  
    6464    if (returnRegister != InvalidGPRReg)
    6565        m_callingConventionRegisters.set(GPRInfo::returnValueGPR);
     66    if (indirectCallTargetRegister != InvalidGPRReg)
     67        m_callingConventionRegisters.set(indirectCallTargetRegister);
    6668    m_callingConventionRegisters.filter(usedRegisters);
    6769       
    … …  
    115117SlowPathCallKey SlowPathCallContext::keyWithTarget(FunctionPtr<CFunctionPtrTag> callTarget) const
    116118{
    117     return SlowPathCallKey(m_thunkSaveSet, callTarget, m_argumentRegisters, m_offset);
     119    return SlowPathCallKey(m_thunkSaveSet, callTarget, m_argumentRegisters, m_offset, 0);
     120}
     121
     122SlowPathCallKey SlowPathCallContext::keyWithTarget(CCallHelpers::Address address) const
     123{
     124    return SlowPathCallKey(m_thunkSaveSet, nullptr, m_argumentRegisters, m_offset, address.offset);
    118125}
    119126
    … …  
    134141}
    135142
     143SlowPathCall SlowPathCallContext::makeCall(VM& vm, CCallHelpers::Address callTarget)
     144{
     145    SlowPathCallKey key = keyWithTarget(callTarget);
     146    SlowPathCall result = SlowPathCall(m_jit.call(JITThunkPtrTag), key);
     147
     148    m_jit.addLinkTask(
     149        [result, &vm] (LinkBuffer& linkBuffer) {
     150            MacroAssemblerCodeRef<JITThunkPtrTag> thunk =
     151                vm.ftlThunks->getSlowPathCallThunk(vm, result.key());
     152
     153            linkBuffer.link(result.call(), CodeLocationLabel<JITThunkPtrTag>(thunk.code()));
     154        });
     155
     156    return result;
     157}
     158
    136159CallSiteIndex callSiteIndexForCodeOrigin(State& state, CodeOrigin codeOrigin)
    137160{

  • trunk/Source/JavaScriptCore/ftl/FTLSlowPathCall.h

    r254735 r278656  
    5555class SlowPathCallContext {
    5656public:
    57     SlowPathCallContext(RegisterSet usedRegisters, CCallHelpers&, unsigned numArgs, GPRReg returnRegister);
     57    SlowPathCallContext(RegisterSet usedRegisters, CCallHelpers&, unsigned numArgs, GPRReg returnRegister, GPRReg indirectCallTargetRegister);
    5858    ~SlowPathCallContext();
    5959
    … …  
    6161    // so there is no need for you to link it yourself.
    6262    SlowPathCall makeCall(VM&, FunctionPtr<CFunctionPtrTag> callTarget);
     63    SlowPathCall makeCall(VM&, CCallHelpers::Address);
    6364
    6465private:
    6566    SlowPathCallKey keyWithTarget(FunctionPtr<CFunctionPtrTag> callTarget) const;
     67    SlowPathCallKey keyWithTarget(CCallHelpers::Address) const;
    6668   
    6769    RegisterSet m_argumentRegisters;
    … …  
    8385    SlowPathCall call;
    8486    {
    85         SlowPathCallContext context(usedRegisters, jit, sizeof...(ArgumentTypes) + 1, resultGPR);
     87        SlowPathCallContext context(usedRegisters, jit, sizeof...(ArgumentTypes) + 1, resultGPR, InvalidGPRReg);
    8688        jit.setupArguments<void(ArgumentTypes...)>(arguments...);
    8789        call = context.makeCall(vm, function);
    … …  
    118120}
    119121
     122template<typename... ArgumentTypes>
     123SlowPathCall callOperation(
     124    VM& vm, const RegisterSet& usedRegisters, CCallHelpers& jit, CCallHelpers::JumpList* exceptionTarget,
     125    CCallHelpers::Address function, GPRReg resultGPR, ArgumentTypes... arguments)
     126{
     127    SlowPathCall call;
     128    {
     129        SlowPathCallContext context(usedRegisters, jit, sizeof...(ArgumentTypes) + 1, resultGPR, GPRInfo::nonArgGPR0);
     130        jit.setupArgumentsForIndirectCall<void(ArgumentTypes...)>(function, arguments...);
     131        call = context.makeCall(vm, CCallHelpers::Address(GPRInfo::nonArgGPR0, function.offset));
     132    }
     133    if (exceptionTarget)
     134        exceptionTarget->append(jit.emitExceptionCheck(vm));
     135    return call;
     136}
     137
     138template<typename... ArgumentTypes>
     139SlowPathCall callOperation(
     140    VM& vm, const RegisterSet& usedRegisters, CCallHelpers& jit, CallSiteIndex callSiteIndex,
     141    CCallHelpers::JumpList* exceptionTarget, CCallHelpers::Address function, GPRReg resultGPR,
     142    ArgumentTypes... arguments)
     143{
     144    if (callSiteIndex) {
     145        jit.store32(
     146            CCallHelpers::TrustedImm32(callSiteIndex.bits()),
     147            CCallHelpers::tagFor(VirtualRegister(CallFrameSlot::argumentCountIncludingThis)));
     148    }
     149    return callOperation(vm, usedRegisters, jit, exceptionTarget, function, resultGPR, arguments...);
     150}
     151
     152CallSiteIndex callSiteIndexForCodeOrigin(State&, CodeOrigin);
     153
     154template<typename... ArgumentTypes>
     155SlowPathCall callOperation(
     156    State& state, const RegisterSet& usedRegisters, CCallHelpers& jit, CodeOrigin codeOrigin,
     157    CCallHelpers::JumpList* exceptionTarget, CCallHelpers::Address function, GPRReg result, ArgumentTypes... arguments)
     158{
     159    return callOperation(
     160        state.vm(), usedRegisters, jit, callSiteIndexForCodeOrigin(state, codeOrigin), exceptionTarget, function,
     161        result, arguments...);
     162}
     163
    120164} } // namespace JSC::FTL
    121165

  • trunk/Source/JavaScriptCore/ftl/FTLSlowPathCallKey.cpp

    r230748 r278656  
    3333void SlowPathCallKey::dump(PrintStream& out) const
    3434{
    35     out.print("<usedRegisters = ", m_usedRegisters, ", offset = ", m_offset, ", callTarget = ", RawPointer(m_callTarget.executableAddress()), ", argumentRegisters = ", m_argumentRegisters, ">");
     35    out.print("<usedRegisters = ", m_usedRegisters, ", offset = ", m_offset, ", indirectOffset = ", m_indirectOffset, ", callTarget = ", RawPointer(m_callTarget.executableAddress()), ", argumentRegisters = ", m_argumentRegisters, ">");
    3636}
    3737

  • trunk/Source/JavaScriptCore/ftl/FTLSlowPathCallKey.h

    r264488 r278656  
    5252    SlowPathCallKey(
    5353        const RegisterSet& set, FunctionPtr<CFunctionPtrTag> callTarget, const RegisterSet& argumentRegisters,
    54         ptrdiff_t offset)
     54        ptrdiff_t offset, int32_t indirectOffset)
    5555        : m_usedRegisters(set)
    5656        , m_callTarget(callTarget.retagged<OperationPtrTag>())
    5757        , m_argumentRegisters(argumentRegisters)
    5858        , m_offset(offset)
     59        , m_indirectOffset(indirectOffset)
    5960    {
    6061    }
    … …  
    6465    const RegisterSet& argumentRegisters() const { return m_argumentRegisters; }
    6566    ptrdiff_t offset() const { return m_offset; }
     67    int32_t indirectOffset() const { return m_indirectOffset; }
    6668   
    6769    SlowPathCallKey withCallTarget(FunctionPtr<CFunctionPtrTag> callTarget)
    6870    {
    69         return SlowPathCallKey(usedRegisters(), callTarget, argumentRegisters(), offset());
     71        return SlowPathCallKey(usedRegisters(), callTarget, argumentRegisters(), offset(), indirectOffset());
    7072    }
    7173   
    … …  
    7779    SlowPathCallKey(EmptyValueTag)
    7880        : m_usedRegisters(RegisterSet::EmptyValue)
    79         , m_offset(0)
    8081    {
    8182    }
    … …  
    8384    SlowPathCallKey(DeletedValueTag)
    8485        : m_usedRegisters(RegisterSet::DeletedValue)
    85         , m_offset(0)
    8686    {
    8787    }
    … …  
    9494        return m_usedRegisters == other.m_usedRegisters
    9595            && m_callTarget == other.m_callTarget
    96             && m_offset == other.m_offset;
     96            && m_offset == other.m_offset
     97            && m_indirectOffset == other.m_indirectOffset;
    9798    }
    9899    unsigned hash() const
    99100    {
    100         return m_usedRegisters.hash() + PtrHash<void*>::hash(m_callTarget.executableAddress()) + m_offset;
     101        return m_usedRegisters.hash() + PtrHash<void*>::hash(m_callTarget.executableAddress()) + m_offset + m_indirectOffset;
    101102    }
    102103
    … …  
    105106    FunctionPtr<OperationPtrTag> m_callTarget;
    106107    RegisterSet m_argumentRegisters;
    107     ptrdiff_t m_offset;
     108    ptrdiff_t m_offset { 0 };
     109    int32_t m_indirectOffset { 0 };
    108110};
    109111

  • trunk/Source/JavaScriptCore/ftl/FTLThunks.cpp

    r277928 r278656  
    7979   
    8080    // Tell GC mark phase how much of the scratch buffer is active during call.
    81     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::nonArgGPR0);
    82     jit.storePtr(MacroAssembler::TrustedImmPtr(requiredScratchMemorySizeInBytes()), GPRInfo::nonArgGPR0);
     81    jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->addressOfActiveLength()), GPRInfo::nonArgGPR1);
     82    jit.storePtr(MacroAssembler::TrustedImmPtr(requiredScratchMemorySizeInBytes()), GPRInfo::nonArgGPR1);
    8383
    8484    jit.loadPtr(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
    … …  
    201201    }
    202202   
    203     jit.preserveReturnAddressAfterCall(GPRInfo::nonArgGPR0);
    204     jit.storePtr(GPRInfo::nonArgGPR0, AssemblyHelpers::Address(MacroAssembler::stackPointerRegister, key.offset()));
     203    jit.preserveReturnAddressAfterCall(GPRInfo::nonArgGPR1);
     204    jit.storePtr(GPRInfo::nonArgGPR1, AssemblyHelpers::Address(MacroAssembler::stackPointerRegister, key.offset()));
    205205    jit.prepareCallOperation(vm);
    206206   
    207     registerClobberCheck(jit, key.argumentRegisters());
    208 
    209     AssemblyHelpers::Call call = jit.call(OperationPtrTag);
     207    RegisterSet dontClobber = key.argumentRegisters();
     208    if (!key.callTarget())
     209        dontClobber.set(GPRInfo::nonArgGPR0);
     210    registerClobberCheck(jit, WTFMove(dontClobber));
     211
     212    AssemblyHelpers::Call call;
     213    if (key.callTarget())
     214        call = jit.call(OperationPtrTag);
     215    else
     216        jit.call(CCallHelpers::Address(GPRInfo::nonArgGPR0, key.indirectOffset()), OperationPtrTag);
    210217
    211218    jit.loadPtr(AssemblyHelpers::Address(MacroAssembler::stackPointerRegister, key.offset()), GPRInfo::nonPreservedNonReturnGPR);
    … …  
    233240
    234241    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::FTLThunk);
    235     patchBuffer.link(call, key.callTarget());
     242    if (key.callTarget())
     243        patchBuffer.link(call, key.callTarget());
    236244    return FINALIZE_CODE(patchBuffer, JITThunkPtrTag, "FTL slow path call thunk for %s", toCString(key).data());
    237245}

  • trunk/Source/JavaScriptCore/jit/AssemblyHelpers.cpp

    r278030 r278656  
    2929#if ENABLE(JIT)
    3030
     31#include "AccessCase.h"
    3132#include "JITOperations.h"
    3233#include "JSArrayBufferView.h"
    … …  
    653654        linkBuffer.addMainThreadFinalizationTask([=, &vm] () {
    654655            MacroAssemblerCodeRef<JITStubRoutinePtrTag> virtualThunk = virtualThunkFor(vm, *info);
    655             info->setSlowStub(GCAwareJITStubRoutine::create(virtualThunk, vm));
     656            info->setSlowStub(GCAwareJITStubRoutine::create(vm, virtualThunk));
    656657            MacroAssembler::repatchNearCall(callLocation, CodeLocationLabel<JITStubRoutinePtrTag>(virtualThunk.code()));
    657658        });

  • trunk/Source/JavaScriptCore/jit/CCallHelpers.cpp

    r277858 r278656  
    7373}
    7474
    75 void CCallHelpers::emitJITCodeOver(MacroAssemblerCodePtr<JSInternalPtrTag> where, WTF::Function<void(CCallHelpers&)> emitCode, const char* description)
     75void CCallHelpers::emitJITCodeOver(MacroAssemblerCodePtr<JSInternalPtrTag> where, ScopedLambda<void(CCallHelpers&)> emitCode, const char* description)
    7676{
    7777    CCallHelpers jit;

  • trunk/Source/JavaScriptCore/jit/CCallHelpers.h

    r277680 r278656  
    3333#include "StackAlignment.h"
    3434#include <wtf/FunctionTraits.h>
     35#include <wtf/ScopedLambda.h>
    3536
    3637namespace JSC {
    … …  
    208209    // extraGPRArgs is used to track 64-bit argument types passed in register on 32-bit architectures.
    209210    // extraPoke is used to track 64-bit argument types passed on the stack.
    210     template<unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke>
     211    template<unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke>
    211212    struct ArgCollection {
    212213        ArgCollection()
    … …  
    220221        }
    221222
    222         template<unsigned a, unsigned b, unsigned c, unsigned d, unsigned e, unsigned f, unsigned g>
    223         ArgCollection(ArgCollection<a, b, c, d, e, f, g>& other)
     223        template<unsigned a, unsigned b, unsigned c, unsigned d, unsigned e, unsigned f, unsigned g, unsigned h>
     224        ArgCollection(ArgCollection<a, b, c, d, e, f, g, h>& other)
    224225        {
    225226            gprSources = other.gprSources;
    … …  
    231232        }
    232233
    233         ArgCollection<numGPRArgs + 1, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> pushRegArg(GPRReg argument, GPRReg destination)
    234         {
    235             ArgCollection<numGPRArgs + 1, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> result(*this);
     234        ArgCollection<numGPRArgs + 1, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> pushRegArg(GPRReg argument, GPRReg destination)
     235        {
     236            ArgCollection<numGPRArgs + 1, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> result(*this);
    236237
    237238            result.gprSources[numGPRSources] = argument;
    … …  
    240241        }
    241242
    242         ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources + 1, numCrossSources, extraGPRArgs, extraPoke> pushRegArg(FPRReg argument, FPRReg destination)
    243         {
    244             ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources + 1, numCrossSources, extraGPRArgs, extraPoke> result(*this);
     243        ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources + 1, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> pushRegArg(FPRReg argument, FPRReg destination)
     244        {
     245            ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources + 1, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> result(*this);
    245246
    246247            result.fprSources[numFPRSources] = argument;
    … …  
    249250        }
    250251
    251         ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources + 1, extraGPRArgs, extraPoke> pushRegArg(FPRReg argument, GPRReg destination)
    252         {
    253             ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources + 1, extraGPRArgs, extraPoke> result(*this);
     252        ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources + 1, extraGPRArgs, nonArgGPRs, extraPoke> pushRegArg(FPRReg argument, GPRReg destination)
     253        {
     254            ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources + 1, extraGPRArgs, nonArgGPRs, extraPoke> result(*this);
    254255
    255256            result.crossSources[numCrossSources] = argument;
    … …  
    258259        }
    259260
    260         ArgCollection<numGPRArgs, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, extraPoke> pushExtraRegArg(GPRReg argument, GPRReg destination)
    261         {
    262             ArgCollection<numGPRArgs, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, extraPoke> result(*this);
     261        ArgCollection<numGPRArgs, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, nonArgGPRs, extraPoke> pushExtraRegArg(GPRReg argument, GPRReg destination)
     262        {
     263            ArgCollection<numGPRArgs, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, nonArgGPRs, extraPoke> result(*this);
    263264
    264265            result.gprSources[numGPRSources] = argument;
    … …  
    267268        }
    268269
    269         ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> addGPRArg()
    270         {
    271             return ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke>(*this);
    272         }
    273 
    274         ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, extraPoke> addGPRExtraArg()
    275         {
    276             return ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, extraPoke>(*this);
    277         }
    278 
    279         ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> addStackArg(GPRReg)
    280         {
    281             return ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke>(*this);
    282         }
    283 
    284         ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> addStackArg(FPRReg)
    285         {
    286             return ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources, extraGPRArgs, extraPoke>(*this);
    287         }
    288 
    289         ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke + 1> addPoke()
    290         {
    291             return ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke + 1>(*this);
     270        ArgCollection<numGPRArgs, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs + 1, extraPoke> pushNonArg(GPRReg argument, GPRReg destination)
     271        {
     272            ArgCollection<numGPRArgs, numGPRSources + 1, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs + 1, extraPoke> result(*this);
     273
     274            result.gprSources[numGPRSources] = argument;
     275            result.gprDestinations[numGPRSources] = destination;
     276            return result;
     277        }
     278
     279        ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> addGPRArg()
     280        {
     281            return ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke>(*this);
     282        }
     283
     284        ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, nonArgGPRs, extraPoke> addGPRExtraArg()
     285        {
     286            return ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs + 1, nonArgGPRs, extraPoke>(*this);
     287        }
     288
     289        ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> addStackArg(GPRReg)
     290        {
     291            return ArgCollection<numGPRArgs + 1, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke>(*this);
     292        }
     293
     294        ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> addStackArg(FPRReg)
     295        {
     296            return ArgCollection<numGPRArgs, numGPRSources, numFPRArgs + 1, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke>(*this);
     297        }
     298
     299        ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke + 1> addPoke()
     300        {
     301            return ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke + 1>(*this);
    292302        }
    293303
    … …  
    332342    }
    333343
    334     ALWAYS_INLINE unsigned calculatePokeOffset(unsigned currentGPRArgument, unsigned currentFPRArgument, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke)
     344    ALWAYS_INLINE unsigned calculatePokeOffset(unsigned currentGPRArgument, unsigned currentFPRArgument, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke)
    335345    {
    336346        // Clang claims that it cannot find the symbol for FPRReg/GPRReg::numberOfArgumentRegisters when they are passed directly to std::max... seems like a bug
    337347        unsigned numberOfFPArgumentRegisters = FPRInfo::numberOfArgumentRegisters;
    338348        unsigned numberOfGPArgumentRegisters = GPRInfo::numberOfArgumentRegisters;
     349
     350        UNUSED_PARAM(nonArgGPRs);
    339351
    340352        currentGPRArgument += extraGPRArgs;
    … …  
    351363
    352364    template<typename ArgType>
    353     ALWAYS_INLINE void pokeForArgument(ArgType arg, unsigned currentGPRArgument, unsigned currentFPRArgument, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke)
    354     {
    355         unsigned pokeOffset = calculatePokeOffset(currentGPRArgument, currentFPRArgument, numCrossSources, extraGPRArgs, extraPoke);
     365    ALWAYS_INLINE void pokeForArgument(ArgType arg, unsigned currentGPRArgument, unsigned currentFPRArgument, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke)
     366    {
     367        unsigned pokeOffset = calculatePokeOffset(currentGPRArgument, currentFPRArgument, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke);
    356368        poke(arg, pokeOffset);
    357369    }
    358370
    359     ALWAYS_INLINE bool stackAligned(unsigned currentGPRArgument, unsigned currentFPRArgument, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke)
    360     {
    361         unsigned pokeOffset = calculatePokeOffset(currentGPRArgument, currentFPRArgument, numCrossSources, extraGPRArgs, extraPoke);
     371    ALWAYS_INLINE bool stackAligned(unsigned currentGPRArgument, unsigned currentFPRArgument, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke)
     372    {
     373        unsigned pokeOffset = calculatePokeOffset(currentGPRArgument, currentFPRArgument, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke);
    362374        return !(pokeOffset & 1);
    363375    }
    … …  
    377389
    378390    // Avoid MSVC optimization time explosion associated with __forceinline in recursive templates.
    379     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename RegType, typename... Args>
    380     ALWAYS_INLINE_EXCEPT_MSVC void marshallArgumentRegister(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, RegType arg, Args... args)
     391    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename RegType, typename... Args>
     392    ALWAYS_INLINE_EXCEPT_MSVC void marshallArgumentRegister(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, RegType arg, Args... args)
    381393    {
    382394        using InfoType = InfoTypeForReg<RegType>;
    … …  
    393405        }
    394406
    395         pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke);
     407        pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke);
    396408        setupArgumentsImpl<OperationType>(argSourceRegs.addStackArg(arg), args...);
    397409    }
    398410
    399     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
    400     ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, FPRReg arg, Args... args)
     411    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
     412    ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, FPRReg arg, Args... args)
    401413    {
    402414        static_assert(std::is_same<CURRENT_ARGUMENT_TYPE, double>::value, "We should only be passing FPRRegs to a double");
    … …  
    404416    }
    405417
    406     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
    407     ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, GPRReg arg, Args... args)
     418    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
     419    ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, GPRReg arg, Args... args)
    408420    {
    409421        marshallArgumentRegister<OperationType>(argSourceRegs, arg, args...);
    410422    }
    411423
    412     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
    413     ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, JSValueRegs arg, Args... args)
     424    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
     425    ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, JSValueRegs arg, Args... args)
    414426    {
    415427        marshallArgumentRegister<OperationType>(argSourceRegs, arg.gpr(), args...);
    416428    }
    417429
    418     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
    419     ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, CellValue arg, Args... args)
     430    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
     431    ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, CellValue arg, Args... args)
    420432    {
    421433        marshallArgumentRegister<OperationType>(argSourceRegs, arg.gpr(), args...);
    … …  
    425437#if CPU(ARM_THUMB2) || CPU(MIPS)
    426438
    427     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
    428     void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, FPRReg arg, Args... args)
     439    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
     440    void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, FPRReg arg, Args... args)
    429441    {
    430442        static_assert(std::is_same<CURRENT_ARGUMENT_TYPE, double>::value, "We should only be passing FPRRegs to a double");
    … …  
    470482
    471483        if (currentGPArgCount < numberOfGPArgumentRegisters) {
    472             pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs + 1, extraPoke);
     484            pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs + 1, nonArgGPRs, extraPoke);
    473485            setupArgumentsImpl<OperationType>(argSourceRegs.addGPRExtraArg().addStackArg(arg).addPoke(), args...);
    474486            return;
    … …  
    477489
    478490        // Otherwise pass FP argument on stack.
    479         if (stackAligned(numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke)) {
    480             pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke);
     491        if (stackAligned(numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke)) {
     492            pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke);
    481493            setupArgumentsImpl<OperationType>(argSourceRegs.addStackArg(arg).addPoke(), args...);
    482494        } else {
    483             pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke + 1);
     495            pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke + 1);
    484496            setupArgumentsImpl<OperationType>(argSourceRegs.addStackArg(arg).addPoke().addPoke(), args...);
    485497        }
    486498    }
    487499
    488     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
     500    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
    489501    std::enable_if_t<sizeof(CURRENT_ARGUMENT_TYPE) <= 4>
    490     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, GPRReg arg, Args... args)
     502    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, GPRReg arg, Args... args)
    491503    {
    492504        unsigned numArgRegisters = GPRInfo::numberOfArgumentRegisters;
    … …  
    498510        }
    499511
    500         pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke);
     512        pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke);
    501513        setupArgumentsImpl<OperationType>(argSourceRegs.addStackArg(arg), args...);
    502514    }
    503515
    504     template<typename OperationType, typename Arg1, typename Arg2, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
    505     void pokeArgumentsAligned(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, Arg1 arg1, Arg2 arg2, Args... args)
     516    template<typename OperationType, typename Arg1, typename Arg2, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
     517    void pokeArgumentsAligned(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, Arg1 arg1, Arg2 arg2, Args... args)
    506518    {
    507519        unsigned numArgRegisters = GPRInfo::numberOfArgumentRegisters;
    … …  
    509521
    510522        if (currentArgCount + 1 == numArgRegisters) {
    511             pokeForArgument(arg1, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs + 1, extraPoke);
    512             pokeForArgument(arg2, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs + 1, extraPoke + 1);
     523            pokeForArgument(arg1, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs + 1, nonArgGPRs, extraPoke);
     524            pokeForArgument(arg2, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs + 1, nonArgGPRs, extraPoke + 1);
    513525            setupArgumentsImpl<OperationType>(argSourceRegs.addGPRExtraArg().addGPRArg().addPoke(), args...);
    514         } else if (stackAligned(numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke)) {
    515             pokeForArgument(arg1, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke);
    516             pokeForArgument(arg2, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke + 1);
     526        } else if (stackAligned(numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke)) {
     527            pokeForArgument(arg1, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke);
     528            pokeForArgument(arg2, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke + 1);
    517529            setupArgumentsImpl<OperationType>(argSourceRegs.addGPRArg().addPoke(), args...);
    518530        } else {
    519             pokeForArgument(arg1, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke + 1);
    520             pokeForArgument(arg2, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke + 2);
     531            pokeForArgument(arg1, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke + 1);
     532            pokeForArgument(arg2, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke + 2);
    521533            setupArgumentsImpl<OperationType>(argSourceRegs.addGPRArg().addPoke().addPoke(), args...);
    522534        }
    523535    }
    524536
    525     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
     537    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
    526538    std::enable_if_t<std::is_same<CURRENT_ARGUMENT_TYPE, EncodedJSValue>::value>
    527     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, CellValue payload, Args... args)
     539    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, CellValue payload, Args... args)
    528540    {
    529541        unsigned numArgRegisters = GPRInfo::numberOfArgumentRegisters;
    … …  
    545557    }
    546558
    547     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename... Args>
     559    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
    548560    std::enable_if_t<std::is_same<CURRENT_ARGUMENT_TYPE, EncodedJSValue>::value>
    549     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, JSValueRegs arg, Args... args)
     561    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, JSValueRegs arg, Args... args)
    550562    {
    551563        unsigned numArgRegisters = GPRInfo::numberOfArgumentRegisters;
    … …  
    570582#endif // USE(JSVALUE64)
    571583
    572     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename Arg, typename... Args>
     584    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename Arg, typename... Args>
    573585    ALWAYS_INLINE std::enable_if_t<
    574586        std::is_base_of<TrustedImm, Arg>::value
    575587        || std::is_convertible<Arg, TrustedImm>::value> // We have this since DFGSpeculativeJIT has it's own implementation of TrustedImmPtr
    576     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, Arg arg, Args... args)
     588    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, Arg arg, Args... args)
    577589    {
    578590        // Right now this only supports non-floating point immediate arguments since we never call operations with non-register values.
    … …  
    592604        }
    593605
    594         pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, extraPoke);
     606        pokeForArgument(arg, numGPRArgs, numFPRArgs, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke);
    595607        setupArgumentsImpl<OperationType>(argSourceRegs.addGPRArg(), args...);
    596608    }
    597609
    598     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename Arg, typename... Args>
     610    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename Arg, typename... Args>
    599611    ALWAYS_INLINE std::enable_if_t<
    600612        std::is_same<CURRENT_ARGUMENT_TYPE, Arg>::value
    601613        && std::is_integral<CURRENT_ARGUMENT_TYPE>::value
    602614        && (sizeof(CURRENT_ARGUMENT_TYPE) <= 4)>
    603     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, Arg arg, Args... args)
     615    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, Arg arg, Args... args)
    604616    {
    605617        setupArgumentsImpl<OperationType>(argSourceRegs, TrustedImm32(arg), args...);
    606618    }
    607619
    608     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename Arg, typename... Args>
     620    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename Arg, typename... Args>
    609621    ALWAYS_INLINE std::enable_if_t<
    610622        std::is_same<CURRENT_ARGUMENT_TYPE, Arg>::value
    611623        && std::is_integral<CURRENT_ARGUMENT_TYPE>::value
    612624        && (sizeof(CURRENT_ARGUMENT_TYPE) == 8)>
    613     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, Arg arg, Args... args)
     625    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, Arg arg, Args... args)
    614626    {
    615627        setupArgumentsImpl<OperationType>(argSourceRegs, TrustedImm64(arg), args...);
    616628    }
    617629
    618     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename Arg, typename... Args>
     630    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename Arg, typename... Args>
    619631    ALWAYS_INLINE std::enable_if_t<
    620632        std::is_pointer<CURRENT_ARGUMENT_TYPE>::value
    621633        && ((std::is_pointer<Arg>::value && std::is_convertible<std::remove_const_t<std::remove_pointer_t<Arg>>*, CURRENT_ARGUMENT_TYPE>::value)
    622634            || std::is_same<Arg, std::nullptr_t>::value)>
    623     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, Arg arg, Args... args)
     635    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, Arg arg, Args... args)
    624636    {
    625637        setupArgumentsImpl<OperationType>(argSourceRegs, TrustedImmPtr(arg), args...);
    … …  
    627639
    628640    // Special case DFG::RegisteredStructure because it's really annoying to deal with otherwise...
    629     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke, typename Arg, typename... Args>
     641    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename Arg, typename... Args>
    630642    ALWAYS_INLINE std::enable_if_t<
    631643        std::is_same<CURRENT_ARGUMENT_TYPE, Structure*>::value
    632644        && std::is_same<Arg, DFG::RegisteredStructure>::value>
    633     setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs, Arg arg, Args... args)
     645    setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, Arg arg, Args... args)
    634646    {
    635647        setupArgumentsImpl<OperationType>(argSourceRegs, TrustedImmPtr(arg.get()), args...);
    … …  
    640652
    641653    // Base case; set up the argument registers.
    642     template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned extraPoke>
    643     ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, extraPoke> argSourceRegs)
     654    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke>
     655    ALWAYS_INLINE void setupArgumentsImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs)
    644656    {
    645657        static_assert(FunctionTraits<OperationType>::arity == numGPRArgs + numFPRArgs, "One last sanity check");
    … …  
    656668    }
    657669
     670    template<typename OperationType, unsigned numGPRArgs, unsigned numGPRSources, unsigned numFPRArgs, unsigned numFPRSources, unsigned numCrossSources, unsigned extraGPRArgs, unsigned nonArgGPRs, unsigned extraPoke, typename... Args>
     671    ALWAYS_INLINE void setupArgumentsEntryImpl(ArgCollection<numGPRArgs, numGPRSources, numFPRArgs, numFPRSources, numCrossSources, extraGPRArgs, nonArgGPRs, extraPoke> argSourceRegs, Args... args)
     672    {
     673        using FirstArgumentType = typename FunctionTraits<OperationType>::template ArgumentType<0>;
     674        if constexpr (std::is_same<FirstArgumentType, CallFrame*>::value) {
     675#if USE(JSVALUE64)
     676            // This only really works for 64-bit since jsvalue regs mess things up for 32-bit...
     677            static_assert(FunctionTraits<OperationType>::cCallArity() == sizeof...(Args) + 1, "Basic sanity check");
     678#endif
     679            setupArgumentsImpl<OperationType>(argSourceRegs, GPRInfo::callFrameRegister, args...);
     680        } else {
     681#if USE(JSVALUE64)
     682            // This only really works for 64-bit since jsvalue regs mess things up for 32-bit...
     683            static_assert(FunctionTraits<OperationType>::cCallArity() == sizeof...(Args), "Basic sanity check");
     684#endif
     685            setupArgumentsImpl<OperationType>(argSourceRegs, args...);
     686        }
     687    }
     688
    658689public:
    659690
    660 #define FIRST_ARGUMENT_TYPE typename FunctionTraits<OperationType>::template ArgumentType<0>
    661 
    662691    template<typename OperationType, typename... Args>
    663     ALWAYS_INLINE std::enable_if_t<std::is_same<FIRST_ARGUMENT_TYPE, CallFrame*>::value> setupArguments(Args... args)
    664     {
    665 #if USE(JSVALUE64)
    666         // This only really works for 64-bit since jsvalue regs mess things up for 32-bit...
    667         static_assert(FunctionTraits<OperationType>::cCallArity() == sizeof...(Args) + 1, "Basic sanity check");
    668 #endif
    669         setupArgumentsImpl<OperationType, 0, 0, 0, 0, 0, 0, 0>(ArgCollection<0, 0, 0, 0, 0, 0, 0>(), GPRInfo::callFrameRegister, args...);
     692    ALWAYS_INLINE void setupArguments(Args... args)
     693    {
     694        setupArgumentsEntryImpl<OperationType>(ArgCollection<0, 0, 0, 0, 0, 0, 0, 0>(), args...);
    670695    }
    671696
    672697    template<typename OperationType, typename... Args>
    673     ALWAYS_INLINE std::enable_if_t<!std::is_same<FIRST_ARGUMENT_TYPE, CallFrame*>::value> setupArguments(Args... args)
    674     {
    675 #if USE(JSVALUE64)
    676         // This only really works for 64-bit since jsvalue regs mess things up for 32-bit...
    677         static_assert(FunctionTraits<OperationType>::cCallArity() == sizeof...(Args), "Basic sanity check");
    678 #endif
    679         setupArgumentsImpl<OperationType, 0, 0, 0, 0, 0, 0, 0>(ArgCollection<0, 0, 0, 0, 0, 0, 0>(), args...);
    680     }
    681 
    682 #undef FIRST_ARGUMENT_TYPE
     698    ALWAYS_INLINE void setupArgumentsForIndirectCall(GPRReg functionGPR, Args... args)
     699    {
     700        setupArgumentsEntryImpl<OperationType>(ArgCollection<0, 0, 0, 0, 0, 0, 0, 0>().pushNonArg(functionGPR, GPRInfo::nonArgGPR0), args...);
     701    }
     702
     703    template<typename OperationType, typename... Args>
     704    ALWAYS_INLINE void setupArgumentsForIndirectCall(Address address, Args... args)
     705    {
     706        setupArgumentsEntryImpl<OperationType>(ArgCollection<0, 0, 0, 0, 0, 0, 0, 0>().pushNonArg(address.base, GPRInfo::nonArgGPR0), args...);
     707    }
    683708
    684709    void setupResults(GPRReg destA, GPRReg destB)
    … …  
    820845    void ensureShadowChickenPacket(VM&, GPRReg shadowPacket, GPRReg scratch1NonArgGPR, GPRReg scratch2);
    821846
    822     static void emitJITCodeOver(MacroAssemblerCodePtr<JSInternalPtrTag> where, WTF::Function<void(CCallHelpers&)>, const char*);
     847    static void emitJITCodeOver(MacroAssemblerCodePtr<JSInternalPtrTag> where, ScopedLambda<void(CCallHelpers&)>, const char*);
    823848};
    824849

  • trunk/Source/JavaScriptCore/jit/GCAwareJITStubRoutine.cpp

    r277027 r278656  
    2929#if ENABLE(JIT)
    3030
     31#include "AccessCase.h"
    3132#include "CodeBlock.h"
    3233#include "DFGCommonData.h"
    … …  
    3940namespace JSC {
    4041
    41 GCAwareJITStubRoutine::GCAwareJITStubRoutine(
    42     const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code, VM& vm)
     42GCAwareJITStubRoutine::GCAwareJITStubRoutine(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code)
    4343    : JITStubRoutine(code)
    4444{
     45}
     46
     47GCAwareJITStubRoutine::~GCAwareJITStubRoutine() { }
     48
     49void GCAwareJITStubRoutine::makeGCAware(VM& vm)
     50{
    4551    vm.heap.m_jitStubRoutines->add(this);
    46 }
    47 
    48 GCAwareJITStubRoutine::~GCAwareJITStubRoutine() { }
     52    m_isGCAware = true;
     53}
    4954
    5055void GCAwareJITStubRoutine::observeZeroRefCount()
    5156{
    52     if (m_isJettisoned) {
     57    if (m_isJettisoned || !m_isGCAware) {
    5358        // This case is needed for when the system shuts down. It may be that
    5459        // the JIT stub routine set gets deleted before we get around to deleting
    … …  
    7580}
    7681
     82PolymorphicAccessJITStubRoutine::PolymorphicAccessJITStubRoutine(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code, VM& vm, FixedVector<RefPtr<AccessCase>>&& cases, FixedVector<StructureID>&& weakStructures)
     83    : GCAwareJITStubRoutine(code)
     84    , m_vm(vm)
     85    , m_cases(WTFMove(cases))
     86    , m_weakStructures(WTFMove(weakStructures))
     87{
     88}
     89
     90void PolymorphicAccessJITStubRoutine::observeZeroRefCount()
     91{
     92    if (m_vm.m_sharedJITStubs)
     93        m_vm.m_sharedJITStubs->remove(this);
     94    Base::observeZeroRefCount();
     95}
     96
     97unsigned PolymorphicAccessJITStubRoutine::computeHash(const FixedVector<RefPtr<AccessCase>>& cases, const FixedVector<StructureID>& weakStructures)
     98{
     99    Hasher hasher;
     100    for (auto& key : cases)
     101        WTF::add(hasher, key->hash());
     102    for (auto& structureID : weakStructures)
     103        WTF::add(hasher, structureID);
     104    return hasher.hash();
     105}
     106
    77107MarkingGCAwareJITStubRoutine::MarkingGCAwareJITStubRoutine(
    78     const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code, VM& vm, const JSCell* owner,
     108    const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code, VM& vm, FixedVector<RefPtr<AccessCase>>&& cases, FixedVector<StructureID>&& weakStructures, const JSCell* owner,
    79109    const Vector<JSCell*>& cells, Bag<CallLinkInfo>&& callLinkInfos)
    80     : GCAwareJITStubRoutine(code, vm)
     110    : PolymorphicAccessJITStubRoutine(code, vm, WTFMove(cases), WTFMove(weakStructures))
    81111    , m_cells(cells.size())
    82112    , m_callLinkInfos(WTFMove(callLinkInfos))
    … …  
    107137
    108138GCAwareJITStubRoutineWithExceptionHandler::GCAwareJITStubRoutineWithExceptionHandler(
    109     const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code, VM& vm, const JSCell* owner, const Vector<JSCell*>& cells, Bag<CallLinkInfo>&& callLinkInfos,
     139    const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code, VM& vm, FixedVector<RefPtr<AccessCase>>&& cases, FixedVector<StructureID>&& weakStructures, const JSCell* owner, const Vector<JSCell*>& cells, Bag<CallLinkInfo>&& callLinkInfos,
    110140    CodeBlock* codeBlockForExceptionHandlers, DisposableCallSiteIndex exceptionHandlerCallSiteIndex)
    111     : MarkingGCAwareJITStubRoutine(code, vm, owner, cells, WTFMove(callLinkInfos))
     141    : MarkingGCAwareJITStubRoutine(code, vm, WTFMove(cases), WTFMove(weakStructures), owner, cells, WTFMove(callLinkInfos))
    112142    , m_codeBlockWithExceptionHandler(codeBlockForExceptionHandlers)
    113143#if ENABLE(DFG_JIT)
    … …  
    153183
    154184
    155 Ref<JITStubRoutine> createJITStubRoutine(
     185Ref<PolymorphicAccessJITStubRoutine> createICJITStubRoutine(
    156186    const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code,
     187    FixedVector<RefPtr<AccessCase>>&& cases,
     188    FixedVector<StructureID>&& weakStructures,
    157189    VM& vm,
    158190    const JSCell* owner,
    … …  
    166198        // Allocating CallLinkInfos means we should have calls.
    167199        ASSERT(callLinkInfos.isEmpty());
    168         return adoptRef(*new JITStubRoutine(code));
     200        return adoptRef(*new PolymorphicAccessJITStubRoutine(code, vm, WTFMove(cases), WTFMove(weakStructures)));
    169201    }
    170202   
    171203    if (codeBlockForExceptionHandlers) {
    172204        RELEASE_ASSERT(JITCode::isOptimizingJIT(codeBlockForExceptionHandlers->jitType()));
    173         return adoptRef(*new GCAwareJITStubRoutineWithExceptionHandler(code, vm, owner, cells, WTFMove(callLinkInfos), codeBlockForExceptionHandlers, exceptionHandlerCallSiteIndex));
    174     }
    175 
    176     if (cells.isEmpty() && callLinkInfos.isEmpty())
    177         return GCAwareJITStubRoutine::create(code, vm);
    178    
    179     return adoptRef(*new MarkingGCAwareJITStubRoutine(code, vm, owner, cells, WTFMove(callLinkInfos)));
     205        auto stub = adoptRef(*new GCAwareJITStubRoutineWithExceptionHandler(code, vm, WTFMove(cases), WTFMove(weakStructures), owner, cells, WTFMove(callLinkInfos), codeBlockForExceptionHandlers, exceptionHandlerCallSiteIndex));
     206        stub->makeGCAware(vm);
     207        return stub;
     208    }
     209
     210    if (cells.isEmpty() && callLinkInfos.isEmpty()) {
     211        auto stub = adoptRef(*new PolymorphicAccessJITStubRoutine(code, vm, WTFMove(cases), WTFMove(weakStructures)));
     212        stub->makeGCAware(vm);
     213        return stub;
     214    }
     215   
     216    auto stub = adoptRef(*new MarkingGCAwareJITStubRoutine(code, vm, WTFMove(cases), WTFMove(weakStructures), owner, cells, WTFMove(callLinkInfos)));
     217    stub->makeGCAware(vm);
     218    return stub;
    180219}
    181220

  • trunk/Source/JavaScriptCore/jit/GCAwareJITStubRoutine.h

    r275542 r278656  
    3232#include "WriteBarrier.h"
    3333#include <wtf/FixedVector.h>
     34#include <wtf/Hasher.h>
    3435#include <wtf/Vector.h>
    3536
    … …  
    3940}
    4041
     42class AccessCase;
    4143class CallLinkInfo;
    4244class JITStubRoutineSet;
    … …  
    5557class GCAwareJITStubRoutine : public JITStubRoutine {
    5658public:
    57     GCAwareJITStubRoutine(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, VM&);
     59    GCAwareJITStubRoutine(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&);
    5860    ~GCAwareJITStubRoutine() override;
    5961
    60     static Ref<JITStubRoutine> create(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code, VM& vm)
     62    static Ref<JITStubRoutine> create(VM& vm, const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code)
    6163    {
    62         return adoptRef(*new GCAwareJITStubRoutine(code, vm));
     64        auto stub = adoptRef(*new GCAwareJITStubRoutine(code));
     65        stub->makeGCAware(vm);
     66        return stub;
    6367    }
    6468
    … …  
    7074   
    7175    void deleteFromGC();
     76
     77    void makeGCAware(VM&);
    7278   
    7379protected:
    … …  
    8288    bool m_mayBeExecuting { false };
    8389    bool m_isJettisoned { false };
     90    bool m_isGCAware { false };
     91};
     92
     93class PolymorphicAccessJITStubRoutine : public GCAwareJITStubRoutine {
     94public:
     95    using Base = GCAwareJITStubRoutine;
     96
     97    PolymorphicAccessJITStubRoutine(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, VM&, FixedVector<RefPtr<AccessCase>>&&, FixedVector<StructureID>&&);
     98
     99    const FixedVector<RefPtr<AccessCase>>& cases() const { return m_cases; }
     100    const FixedVector<StructureID>& weakStructures() const { return m_weakStructures; }
     101
     102    unsigned hash() const
     103    {
     104        if (!m_hash)
     105            m_hash = computeHash(m_cases, m_weakStructures);
     106        return m_hash;
     107    }
     108
     109    static unsigned computeHash(const FixedVector<RefPtr<AccessCase>>&, const FixedVector<StructureID>&);
     110
     111protected:
     112    void observeZeroRefCount() override;
     113
     114private:
     115    VM& m_vm;
     116    FixedVector<RefPtr<AccessCase>> m_cases;
     117    FixedVector<StructureID> m_weakStructures;
    84118};
    85119
    86120// Use this if you want to mark one additional object during GC if your stub
    87121// routine is known to be executing.
    88 class MarkingGCAwareJITStubRoutine : public GCAwareJITStubRoutine {
     122class MarkingGCAwareJITStubRoutine : public PolymorphicAccessJITStubRoutine {
    89123public:
     124    using Base = PolymorphicAccessJITStubRoutine;
     125
    90126    MarkingGCAwareJITStubRoutine(
    91         const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, VM&, const JSCell* owner, const Vector<JSCell*>&, Bag<CallLinkInfo>&&);
     127        const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, VM&, FixedVector<RefPtr<AccessCase>>&&, FixedVector<StructureID>&&, const JSCell* owner, const Vector<JSCell*>&, Bag<CallLinkInfo>&&);
    92128    ~MarkingGCAwareJITStubRoutine() override;
    93129   
    … …  
    108144class GCAwareJITStubRoutineWithExceptionHandler final : public MarkingGCAwareJITStubRoutine {
    109145public:
    110     typedef GCAwareJITStubRoutine Base;
     146    using Base = MarkingGCAwareJITStubRoutine;
    111147
    112     GCAwareJITStubRoutineWithExceptionHandler(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, VM&, const JSCell* owner, const Vector<JSCell*>&, Bag<CallLinkInfo>&&, CodeBlock*, DisposableCallSiteIndex);
     148    GCAwareJITStubRoutineWithExceptionHandler(const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, VM&, FixedVector<RefPtr<AccessCase>>&&, FixedVector<StructureID>&&, const JSCell* owner, const Vector<JSCell*>&, Bag<CallLinkInfo>&&, CodeBlock*, DisposableCallSiteIndex);
    113149    ~GCAwareJITStubRoutineWithExceptionHandler() final;
    114150
    … …  
    130166// after the first call to C++ or JS.
    131167//
    132 // Ref<JITStubRoutine> createJITStubRoutine(
     168// Ref<PolymorphicAccessJITStubRoutine> createICJITStubRoutine(
    133169//    const MacroAssemblerCodeRef<JITStubRoutinePtrTag>& code,
    134170//    VM& vm,
     171//    FixedVector<RefPtr<AccessCase>>&& cases,
    135172//    const JSCell* owner,
    136173//    bool makesCalls,
    … …  
    143180// way.
    144181
    145 Ref<JITStubRoutine> createJITStubRoutine(
    146     const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, VM&, const JSCell* owner, bool makesCalls,
     182Ref<PolymorphicAccessJITStubRoutine> createICJITStubRoutine(
     183    const MacroAssemblerCodeRef<JITStubRoutinePtrTag>&, FixedVector<RefPtr<AccessCase>>&& cases, FixedVector<StructureID>&& weakStructures, VM&, const JSCell* owner, bool makesCalls,
    147184    const Vector<JSCell*>&, Bag<CallLinkInfo>&& callLinkInfos,
    148185    CodeBlock* codeBlockForExceptionHandlers, DisposableCallSiteIndex exceptionHandlingCallSiteIndex);

  • trunk/Source/JavaScriptCore/jit/GPRInfo.h

    r269349 r278656  
    461461#endif
    462462    static constexpr GPRReg nonArgGPR0 = X86Registers::r10; // regT5 (regT4 on Windows)
     463    static constexpr GPRReg nonArgGPR1 = X86Registers::eax; // regT0
    463464    static constexpr GPRReg returnValueGPR = X86Registers::eax; // regT0
    464465    static constexpr GPRReg returnValueGPR2 = X86Registers::edx; // regT1 or regT2
    … …  
    653654    static constexpr GPRReg argumentGPR7 = ARM64Registers::x7; // regT7
    654655    static constexpr GPRReg nonArgGPR0 = ARM64Registers::x8; // regT8
     656    static constexpr GPRReg nonArgGPR1 = ARM64Registers::x9; // regT9
    655657    static constexpr GPRReg returnValueGPR = ARM64Registers::x0; // regT0
    656658    static constexpr GPRReg returnValueGPR2 = ARM64Registers::x1; // regT1

  • trunk/Source/JavaScriptCore/jit/JIT.cpp

    r278576 r278656  
    12661266        for (const auto& byValCompilationInfo : m_byValCompilationInfo) {
    12671267            PatchableJump patchableNotIndexJump = byValCompilationInfo.notIndexJump;
    1268             auto notIndexJump = CodeLocationJump<JSInternalPtrTag>();
     1268            CodeLocationJump<JSInternalPtrTag> notIndexJump;
    12691269            if (Jump(patchableNotIndexJump).isSet())
    12701270                notIndexJump = CodeLocationJump<JSInternalPtrTag>(patchBuffer.locationOf<JSInternalPtrTag>(patchableNotIndexJump));
    12711271
    12721272            PatchableJump patchableBadTypeJump = byValCompilationInfo.badTypeJump;
    1273             auto badTypeJump = CodeLocationJump<JSInternalPtrTag>();
     1273            CodeLocationJump<JSInternalPtrTag> badTypeJump;
    12741274            if (Jump(patchableBadTypeJump).isSet())
    12751275                badTypeJump = CodeLocationJump<JSInternalPtrTag>(patchBuffer.locationOf<JSInternalPtrTag>(byValCompilationInfo.badTypeJump));
    … …  
    12801280
    12811281            byValCompilationInfo.byValInfo->setUp(
    1282                 notIndexJump,
    1283                 badTypeJump,
    12841282                exceptionHandler,
    12851283                byValCompilationInfo.arrayMode,
    … …  
    12881286                nextHotPathTarget,
    12891287                slowPathTarget);
     1288            if (JITCode::useDataIC(JITType::BaselineJIT)) {
     1289                byValCompilationInfo.byValInfo->m_notIndexJumpTarget = slowPathTarget.retagged<JITStubRoutinePtrTag>();
     1290                byValCompilationInfo.byValInfo->m_badTypeJumpTarget = slowPathTarget.retagged<JITStubRoutinePtrTag>();
     1291            } else {
     1292                byValCompilationInfo.byValInfo->m_notIndexJump = notIndexJump;
     1293                byValCompilationInfo.byValInfo->m_badTypeJump = badTypeJump;
     1294            }
    12901295        }
    12911296    }

  • trunk/Source/JavaScriptCore/jit/JIT.h

    r278591 r278656  
    295295        }
    296296
     297        void appendCall(Address function)
     298        {
     299            call(function, OperationPtrTag);
     300        }
     301
    297302#if OS(WINDOWS) && CPU(X86_64)
    298303        Call appendCallWithSlowPathReturnType(const FunctionPtr<CFunctionPtrTag> function)
    … …  
    398403        // Structure is already profiled. Returns the slow cases. Fall-through
    399404        // case contains result in regT0, and it is not yet profiled.
    400         JumpList emitInt32Load(const Instruction* instruction, PatchableJump& badType) { return emitContiguousLoad(instruction, badType, Int32Shape); }
    401         JumpList emitDoubleLoad(const Instruction*, PatchableJump& badType);
    402         JumpList emitContiguousLoad(const Instruction*, PatchableJump& badType, IndexingType expectedShape = ContiguousShape);
    403         JumpList emitArrayStorageLoad(const Instruction*, PatchableJump& badType);
    404         JumpList emitLoadForArrayMode(const Instruction*, JITArrayMode, PatchableJump& badType);
     405        JumpList emitInt32Load(const Instruction* instruction, PatchableJump& badType, ByValInfo* byValInfo) { return emitContiguousLoad(instruction, badType, byValInfo, Int32Shape); }
     406        JumpList emitDoubleLoad(const Instruction*, PatchableJump& badType, ByValInfo*);
     407        JumpList emitContiguousLoad(const Instruction*, PatchableJump& badType, ByValInfo*, IndexingType expectedShape = ContiguousShape);
     408        JumpList emitArrayStorageLoad(const Instruction*, PatchableJump& badType, ByValInfo*);
     409        JumpList emitLoadForArrayMode(const Instruction*, JITArrayMode, PatchableJump& badType, ByValInfo*);
    405410
    406411        // Property is in regT1, base is in regT0. regT2 contains indecing type.
    … …  
    409414        // returns the slow cases.
    410415        template<typename Op>
    411         JumpList emitInt32PutByVal(Op bytecode, PatchableJump& badType)
    412         {
    413             return emitGenericContiguousPutByVal(bytecode, badType, Int32Shape);
    414         }
    415         template<typename Op>
    416         JumpList emitDoublePutByVal(Op bytecode, PatchableJump& badType)
    417         {
    418             return emitGenericContiguousPutByVal(bytecode, badType, DoubleShape);
    419         }
    420         template<typename Op>
    421         JumpList emitContiguousPutByVal(Op bytecode, PatchableJump& badType)
    422         {
    423             return emitGenericContiguousPutByVal(bytecode, badType);
    424         }
    425         template<typename Op>
    426         JumpList emitGenericContiguousPutByVal(Op, PatchableJump& badType, IndexingType indexingShape = ContiguousShape);
    427         template<typename Op>
    428         JumpList emitArrayStoragePutByVal(Op, PatchableJump& badType);
    429         template<typename Op>
    430         JumpList emitIntTypedArrayPutByVal(Op, PatchableJump& badType, TypedArrayType);
    431         template<typename Op>
    432         JumpList emitFloatTypedArrayPutByVal(Op, PatchableJump& badType, TypedArrayType);
     416        JumpList emitInt32PutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo)
     417        {
     418            return emitGenericContiguousPutByVal(bytecode, badType, byValInfo, Int32Shape);
     419        }
     420        template<typename Op>
     421        JumpList emitDoublePutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo)
     422        {
     423            return emitGenericContiguousPutByVal(bytecode, badType, byValInfo, DoubleShape);
     424        }
     425        template<typename Op>
     426        JumpList emitContiguousPutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo)
     427        {
     428            return emitGenericContiguousPutByVal(bytecode, badType, byValInfo);
     429        }
     430        template<typename Op>
     431        JumpList emitGenericContiguousPutByVal(Op, PatchableJump& badType, ByValInfo*, IndexingType indexingShape = ContiguousShape);
     432        template<typename Op>
     433        JumpList emitArrayStoragePutByVal(Op, PatchableJump& badType, ByValInfo*);
     434        template<typename Op>
     435        JumpList emitIntTypedArrayPutByVal(Op, PatchableJump& badType, ByValInfo*, TypedArrayType);
     436        template<typename Op>
     437        JumpList emitFloatTypedArrayPutByVal(Op, PatchableJump& badType, ByValInfo*, TypedArrayType);
    433438
    434439        template<typename Op>
    … …  
    902907
    903908        MacroAssembler::Call appendCallWithExceptionCheck(const FunctionPtr<CFunctionPtrTag>);
     909        void appendCallWithExceptionCheck(Address);
    904910#if OS(WINDOWS) && CPU(X86_64)
    905911        MacroAssembler::Call appendCallWithExceptionCheckAndSlowPathReturnType(const FunctionPtr<CFunctionPtrTag>);
    … …  
    907913        MacroAssembler::Call appendCallWithCallFrameRollbackOnException(const FunctionPtr<CFunctionPtrTag>);
    908914        MacroAssembler::Call appendCallWithExceptionCheckSetJSValueResult(const FunctionPtr<CFunctionPtrTag>, VirtualRegister result);
     915        void appendCallWithExceptionCheckSetJSValueResult(Address, VirtualRegister result);
    909916        template<typename Metadata>
    910917        MacroAssembler::Call appendCallWithExceptionCheckSetJSValueResultWithProfile(Metadata&, const FunctionPtr<CFunctionPtrTag>, VirtualRegister result);
     918        template<typename Metadata>
     919        void appendCallWithExceptionCheckSetJSValueResultWithProfile(Metadata&, Address, VirtualRegister result);
    911920       
    912921        template<typename OperationType, typename... Args>
    … …  
    916925            setupArguments<OperationType>(args...);
    917926            return appendCallWithExceptionCheckSetJSValueResult(operation, result);
     927        }
     928
     929        template<typename OperationType, typename... Args>
     930        std::enable_if_t<FunctionTraits<OperationType>::hasResult, void>
     931        callOperation(Address target, VirtualRegister result, Args... args)
     932        {
     933            setupArgumentsForIndirectCall<OperationType>(target, args...);
     934            return appendCallWithExceptionCheckSetJSValueResult(Address(GPRInfo::nonArgGPR0, target.offset), result);
    918935        }
    919936
    … …  
    938955            return appendCallWithExceptionCheckAndSlowPathReturnType(operation);
    939956        }
     957
     958        template<typename OperationType, typename... Args>
     959        void callOperation(Address target, Args... args)
     960        {
     961            setupArgumentsForIndirectCall<OperationType>(target, args...);
     962            // x64 Windows cannot use standard call when the return type is larger than 64 bits.
     963            if constexpr (is64BitType<typename FunctionTraits<OperationType>::ResultType>::value)
     964                appendCallWithExceptionCheck(Address(GPRInfo::nonArgGPR0, target.offset));
     965            else
     966                appendCallWithExceptionCheckAndSlowPathReturnType(Address(GPRInfo::nonArgGPR0, target.offset));
     967        }
    940968#else // OS(WINDOWS) && CPU(X86_64)
    941969        template<typename OperationType, typename... Args>
    … …  
    945973            return appendCallWithExceptionCheck(operation);
    946974        }
     975
     976        template<typename OperationType, typename... Args>
     977        void callOperation(Address target, Args... args)
     978        {
     979            setupArgumentsForIndirectCall<OperationType>(target, args...);
     980            appendCallWithExceptionCheck(Address(GPRInfo::nonArgGPR0, target.offset));
     981        }
    947982#endif // OS(WINDOWS) && CPU(X86_64)
    948983
    … …  
    953988            setupArguments<OperationType>(args...);
    954989            return appendCallWithExceptionCheckSetJSValueResultWithProfile(metadata, operation, result);
     990        }
     991
     992        template<typename OperationType, typename Metadata, typename... Args>
     993        std::enable_if_t<FunctionTraits<OperationType>::hasResult, void>
     994        callOperationWithProfile(Metadata& metadata, Address target, VirtualRegister result, Args... args)
     995        {
     996            setupArgumentsForIndirectCall<OperationType>(target, args...);
     997            return appendCallWithExceptionCheckSetJSValueResultWithProfile(metadata, Address(GPRInfo::nonArgGPR0, target.offset), result);
    955998        }
    956999

  • trunk/Source/JavaScriptCore/jit/JITCall.cpp

    r278030 r278656  
    399399
    400400    JITGetByIdGenerator gen(
    401         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())), RegisterSet::stubUnavailableRegisters(),
    402         CacheableIdentifier::createFromImmortalIdentifier(ident->impl()), JSValueRegs(regT0), JSValueRegs(regT0), AccessType::GetById);
     401        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())), RegisterSet::stubUnavailableRegisters(),
     402        CacheableIdentifier::createFromImmortalIdentifier(ident->impl()), JSValueRegs(regT0), JSValueRegs(regT0), regT1, AccessType::GetById);
    403403    gen.generateFastPath(*this);
    404404    addSlowCase(gen.slowPathJump());
    … …  
    433433    Label coldPathBegin = label();
    434434
    435     Call call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, nextVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), iteratorGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
     435    Call call;
     436    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     437        gen.stubInfo()->m_slowOperation = operationGetByIdOptimize;
     438        move(TrustedImmPtr(gen.stubInfo()), GPRInfo::nonArgGPR0);
     439        callOperationWithProfile<decltype(operationGetByIdOptimize)>(bytecode.metadata(m_codeBlock), Address(GPRInfo::nonArgGPR0, StructureStubInfo::offsetOfSlowOperation()), nextVReg, TrustedImmPtr(m_codeBlock->globalObject()), GPRInfo::nonArgGPR0, iteratorGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
     440    } else
     441        call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, nextVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), iteratorGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
    436442    gen.reportSlowPathCall(coldPathBegin, call);
     443
    437444    auto done = jump();
    438445
    … …  
    481488        preservedRegs.add(valueGPR);
    482489        JITGetByIdGenerator gen(
    483             m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())), preservedRegs,
    484             CacheableIdentifier::createFromImmortalIdentifier(vm().propertyNames->done.impl()), JSValueRegs(iterResultGPR), JSValueRegs(doneGPR), AccessType::GetById);
     490            m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())), preservedRegs,
     491            CacheableIdentifier::createFromImmortalIdentifier(vm().propertyNames->done.impl()), JSValueRegs(iterResultGPR), JSValueRegs(doneGPR), regT3, AccessType::GetById);
    485492        gen.generateFastPath(*this);
    486493        addSlowCase(gen.slowPathJump());
    … …  
    500507
    501508        JITGetByIdGenerator gen(
    502             m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())), RegisterSet::stubUnavailableRegisters(),
    503             CacheableIdentifier::createFromImmortalIdentifier(vm().propertyNames->value.impl()), JSValueRegs(valueGPR), JSValueRegs(valueGPR), AccessType::GetById);
     509            m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())), RegisterSet::stubUnavailableRegisters(),
     510            CacheableIdentifier::createFromImmortalIdentifier(vm().propertyNames->value.impl()), JSValueRegs(valueGPR), JSValueRegs(valueGPR), regT4, AccessType::GetById);
    504511        gen.generateFastPath(*this);
    505512        addSlowCase(gen.slowPathJump());
    … …  
    529536        JumpList notObject;
    530537        notObject.append(branchIfNotCell(iterResultGPR));
    531         notObject.append(branchIfNotObject(iterResultGPR));
    532538
    533539        UniquedStringImpl* ident = vm().propertyNames->done.impl();
    … …  
    536542        Label coldPathBegin = label();
    537543
    538         Call call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, doneVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), iterResultGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
    539 
     544        notObject.append(branchIfNotObject(iterResultGPR));
     545
     546        Call call;
     547        if (JITCode::useDataIC(JITType::BaselineJIT)) {
     548            gen.stubInfo()->m_slowOperation = operationGetByIdOptimize;
     549            move(TrustedImmPtr(gen.stubInfo()), GPRInfo::nonArgGPR0);
     550            callOperationWithProfile<decltype(operationGetByIdOptimize)>(bytecode.metadata(m_codeBlock), Address(GPRInfo::nonArgGPR0, StructureStubInfo::offsetOfSlowOperation()), doneVReg, TrustedImmPtr(m_codeBlock->globalObject()), GPRInfo::nonArgGPR0, iterResultGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
     551        } else
     552            call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, doneVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), iterResultGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
    540553        gen.reportSlowPathCall(coldPathBegin, call);
     554
    541555        emitGetVirtualRegister(doneVReg, regT1);
    542556        emitGetVirtualRegister(bytecode.m_value, regT0);
    … …  
    557571        Label coldPathBegin = label();
    558572
    559         Call call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, valueVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), iterResultGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
    560 
     573        Call call;
     574        if (JITCode::useDataIC(JITType::BaselineJIT)) {
     575            gen.stubInfo()->m_slowOperation = operationGetByIdOptimize;
     576            move(TrustedImmPtr(gen.stubInfo()), GPRInfo::nonArgGPR0);
     577            callOperationWithProfile<decltype(operationGetByIdOptimize)>(bytecode.metadata(m_codeBlock), Address(GPRInfo::nonArgGPR0, StructureStubInfo::offsetOfSlowOperation()), valueVReg, TrustedImmPtr(m_codeBlock->globalObject()), GPRInfo::nonArgGPR0, iterResultGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
     578        } else
     579            call = callOperationWithProfile(bytecode.metadata(m_codeBlock), operationGetByIdOptimize, valueVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), iterResultGPR, CacheableIdentifier::createFromImmortalIdentifier(ident).rawBits());
    561580        gen.reportSlowPathCall(coldPathBegin, call);
    562581    }

  • trunk/Source/JavaScriptCore/jit/JITCall32_64.cpp

    r278030 r278656  
    397397    JITGetByIdGenerator gen(
    398398        m_codeBlock,
     399        JITType::BaselineJIT,
    399400        CodeOrigin(m_bytecodeIndex),
    400401        CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())),
    … …  
    403404        JSValueRegs(tagIteratorGPR, payloadIteratorGPR),
    404405        nextRegs,
     406        InvalidGPRReg,
    405407        AccessType::GetById);
    406408   
    … …  
    506508        JITGetByIdGenerator gen(
    507509            m_codeBlock,
     510            JITType::BaselineJIT,
    508511            CodeOrigin(m_bytecodeIndex),
    509512            CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())),
    … …  
    512515            JSValueRegs(tagIterResultGPR, payloadIterResultGPR),
    513516            doneRegs,
     517            InvalidGPRReg,
    514518            AccessType::GetById);
    515519        gen.generateFastPath(*this);
    … …  
    534538        JITGetByIdGenerator gen(
    535539            m_codeBlock,
     540            JITType::BaselineJIT,
    536541            CodeOrigin(m_bytecodeIndex),
    537542            CallSiteIndex(BytecodeIndex(m_bytecodeIndex.offset())),
    … …  
    540545            JSValueRegs(tagIterResultGPR, payloadIterResultGPR),
    541546            resultRegs,
     547            InvalidGPRReg,
    542548            AccessType::GetById);
    543549        gen.generateFastPath(*this);

  • trunk/Source/JavaScriptCore/jit/JITCode.h

    r278340 r278656  
    5252
    5353enum class JITType : uint8_t {
    54     None,
    55     HostCallThunk,
    56     InterpreterThunk,
    57     BaselineJIT,
    58     DFGJIT,
    59     FTLJIT
    60 };
     54    None = 0b000,
     55    HostCallThunk = 0b001,
     56    InterpreterThunk = 0b010,
     57    BaselineJIT = 0b011,
     58    DFGJIT = 0b100,
     59    FTLJIT = 0b101,
     60};
     61static constexpr unsigned widthOfJITType = 3;
     62static_assert(WTF::getMSBSetConstexpr(static_cast<std::underlying_type_t<JITType>>(JITType::FTLJIT)) + 1 == widthOfJITType);
    6163
    6264class JITCode : public ThreadSafeRefCounted<JITCode> {
    … …  
    154156    {
    155157        return jitType == JITType::InterpreterThunk || jitType == JITType::BaselineJIT;
     158    }
     159
     160    static bool useDataIC(JITType jitType)
     161    {
     162        if (!Options::useDataIC())
     163            return false;
     164        if (JITCode::isBaselineCode(jitType))
     165            return true;
     166        return Options::useDataICInOptimizingJIT();
    156167    }
    157168

  • trunk/Source/JavaScriptCore/jit/JITInlineCacheGenerator.cpp

    r278445 r278656  
    4444
    4545JITInlineCacheGenerator::JITInlineCacheGenerator(
    46     CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSite, AccessType accessType,
     46    CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSite, AccessType accessType,
    4747    const RegisterSet& usedRegisters)
    4848    : m_codeBlock(codeBlock)
     49    , m_jitType(jitType)
    4950{
    5051    m_stubInfo = m_codeBlock ? m_codeBlock->addStubInfo(accessType, codeOrigin) : garbageStubInfo();
    … …  
    6162    m_stubInfo->doneLocation = fastPath.locationOf<JSInternalPtrTag>(m_done);
    6263
    63     m_stubInfo->slowPathCallLocation = slowPath.locationOf<JSInternalPtrTag>(m_slowPathCall);
     64    if (!JITCode::useDataIC(m_jitType))
     65        m_stubInfo->m_slowPathCallLocation = slowPath.locationOf<JSInternalPtrTag>(m_slowPathCall);
    6466    m_stubInfo->slowPathStartLocation = slowPath.locationOf<JITStubRoutinePtrTag>(m_slowPathBegin);
    6567}
    6668
    6769JITByIdGenerator::JITByIdGenerator(
    68     CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSite, AccessType accessType,
    69     const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs value)
    70     : JITInlineCacheGenerator(codeBlock, codeOrigin, callSite, accessType, usedRegisters)
     70    CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSite, AccessType accessType,
     71    const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs value, GPRReg stubInfoGPR)
     72    : JITInlineCacheGenerator(codeBlock, jitType, codeOrigin, callSite, accessType, usedRegisters)
    7173    , m_base(base)
    7274    , m_value(value)
    … …  
    7577    m_stubInfo->valueGPR = value.payloadGPR();
    7678    m_stubInfo->regs.thisGPR = InvalidGPRReg;
     79    m_stubInfo->m_stubInfoGPR = stubInfoGPR;
    7780#if USE(JSVALUE32_64)
    7881    m_stubInfo->baseTagGPR = base.tagGPR();
    … …  
    8487void JITByIdGenerator::finalize(LinkBuffer& fastPath, LinkBuffer& slowPath)
    8588{
    86     ASSERT(m_start.isSet());
    87     JITInlineCacheGenerator::finalize(
    88         fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     89    JITInlineCacheGenerator::finalize(fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     90    if (JITCode::useDataIC(m_jitType))
     91        m_stubInfo->m_codePtr = m_stubInfo->slowPathStartLocation;
    8992}
    9093
    9194void JITByIdGenerator::generateFastCommon(MacroAssembler& jit, size_t inlineICSize)
    9295{
     96    // We generate the same code regardless of whether SharedIC is enabled because we still need to use InlineAccess
     97    // for the performance reason.
    9398    m_start = jit.label();
    9499    size_t startSize = jit.m_assembler.buffer().codeSize();
    … …  
    102107
    103108JITGetByIdGenerator::JITGetByIdGenerator(
    104     CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters,
    105     CacheableIdentifier propertyName, JSValueRegs base, JSValueRegs value, AccessType accessType)
    106     : JITByIdGenerator(codeBlock, codeOrigin, callSite, accessType, usedRegisters, base, value)
     109    CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters,
     110    CacheableIdentifier propertyName, JSValueRegs base, JSValueRegs value, GPRReg stubInfoGPR, AccessType accessType)
     111    : JITByIdGenerator(codeBlock, jitType, codeOrigin, callSite, accessType, usedRegisters, base, value, stubInfoGPR)
    107112    , m_isLengthAccess(propertyName.uid() == codeBlock->vm().propertyNames->length.impl())
    108113{
    … …  
    116121
    117122JITGetByIdWithThisGenerator::JITGetByIdWithThisGenerator(
    118     CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters,
    119     CacheableIdentifier, JSValueRegs value, JSValueRegs base, JSValueRegs thisRegs)
    120     : JITByIdGenerator(codeBlock, codeOrigin, callSite, AccessType::GetByIdWithThis, usedRegisters, base, value)
     123    CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters,
     124    CacheableIdentifier, JSValueRegs value, JSValueRegs base, JSValueRegs thisRegs, GPRReg stubInfoGPR)
     125    : JITByIdGenerator(codeBlock, jitType, codeOrigin, callSite, AccessType::GetByIdWithThis, usedRegisters, base, value, stubInfoGPR)
    121126{
    122127    RELEASE_ASSERT(thisRegs.payloadGPR() != thisRegs.tagGPR());
    … …  
    134139
    135140JITPutByIdGenerator::JITPutByIdGenerator(
    136     CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters, CacheableIdentifier,
    137     JSValueRegs base, JSValueRegs value, GPRReg scratch,
     141    CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters, CacheableIdentifier,
     142    JSValueRegs base, JSValueRegs value, GPRReg stubInfoGPR, GPRReg scratch,
    138143    ECMAMode ecmaMode, PutKind putKind)
    139         : JITByIdGenerator(codeBlock, codeOrigin, callSite, AccessType::Put, usedRegisters, base, value)
     144        : JITByIdGenerator(codeBlock, jitType, codeOrigin, callSite, AccessType::Put, usedRegisters, base, value, stubInfoGPR)
    140145        , m_ecmaMode(ecmaMode)
    141146        , m_putKind(putKind)
    … …  
    172177}
    173178
    174 JITDelByValGenerator::JITDelByValGenerator(CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg scratch)
    175     : Base(codeBlock, codeOrigin, callSiteIndex, AccessType::DeleteByVal, usedRegisters)
     179JITDelByValGenerator::JITDelByValGenerator(CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg stubInfoGPR, GPRReg scratch)
     180    : Base(codeBlock, jitType, codeOrigin, callSiteIndex, AccessType::DeleteByVal, usedRegisters)
    176181{
    177182    m_stubInfo->hasConstantIdentifier = false;
    … …  
    180185    m_stubInfo->regs.propertyGPR = property.payloadGPR();
    181186    m_stubInfo->valueGPR = result.payloadGPR();
     187    m_stubInfo->m_stubInfoGPR = stubInfoGPR;
    182188#if USE(JSVALUE32_64)
    183189    ASSERT(base.tagGPR() != result.tagGPR());
    … …  
    192198{
    193199    m_start = jit.label();
    194     m_slowPathJump = jit.patchableJump();
    195     m_done = jit.label();
    196 }
    197 
    198 void JITDelByValGenerator::finalize(
    199     LinkBuffer& fastPath, LinkBuffer& slowPath)
    200 {
    201     ASSERT(m_slowPathJump.m_jump.isSet());
    202     Base::finalize(
    203         fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
    204 }
    205 
    206 JITDelByIdGenerator::JITDelByIdGenerator(CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier, JSValueRegs base, JSValueRegs result, GPRReg scratch)
    207     : Base(codeBlock, codeOrigin, callSiteIndex, AccessType::DeleteByID, usedRegisters)
     200    if (JITCode::useDataIC(m_jitType)) {
     201        jit.move(CCallHelpers::TrustedImmPtr(m_stubInfo), m_stubInfo->m_stubInfoGPR);
     202        jit.call(CCallHelpers::Address(m_stubInfo->m_stubInfoGPR, StructureStubInfo::offsetOfCodePtr()), JITStubRoutinePtrTag);
     203    } else
     204        m_slowPathJump = jit.patchableJump();
     205    m_done = jit.label();
     206}
     207
     208void JITDelByValGenerator::finalize(LinkBuffer& fastPath, LinkBuffer& slowPath)
     209{
     210    Base::finalize(fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     211    if (JITCode::useDataIC(m_jitType))
     212        m_stubInfo->m_codePtr = m_stubInfo->slowPathStartLocation;
     213}
     214
     215JITDelByIdGenerator::JITDelByIdGenerator(CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier, JSValueRegs base, JSValueRegs result, GPRReg stubInfoGPR, GPRReg scratch)
     216    : Base(codeBlock, jitType, codeOrigin, callSiteIndex, AccessType::DeleteByID, usedRegisters)
    208217{
    209218    m_stubInfo->hasConstantIdentifier = true;
    … …  
    212221    m_stubInfo->regs.propertyGPR = InvalidGPRReg;
    213222    m_stubInfo->valueGPR = result.payloadGPR();
     223    m_stubInfo->m_stubInfoGPR = stubInfoGPR;
    214224#if USE(JSVALUE32_64)
    215225    ASSERT(base.tagGPR() != result.tagGPR());
    … …  
    224234{
    225235    m_start = jit.label();
    226     m_slowPathJump = jit.patchableJump();
    227     m_done = jit.label();
    228 }
    229 
    230 void JITDelByIdGenerator::finalize(
    231     LinkBuffer& fastPath, LinkBuffer& slowPath)
    232 {
    233     ASSERT(m_slowPathJump.m_jump.isSet());
    234     Base::finalize(
    235         fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
    236 }
    237 
    238 JITInByValGenerator::JITInByValGenerator(CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs property, JSValueRegs result)
    239     : Base(codeBlock, codeOrigin, callSiteIndex, AccessType::InByVal, usedRegisters)
     236    if (JITCode::useDataIC(m_jitType)) {
     237        jit.move(CCallHelpers::TrustedImmPtr(m_stubInfo), m_stubInfo->m_stubInfoGPR);
     238        jit.call(CCallHelpers::Address(m_stubInfo->m_stubInfoGPR, StructureStubInfo::offsetOfCodePtr()), JITStubRoutinePtrTag);
     239    } else
     240        m_slowPathJump = jit.patchableJump();
     241    m_done = jit.label();
     242}
     243
     244void JITDelByIdGenerator::finalize(LinkBuffer& fastPath, LinkBuffer& slowPath)
     245{
     246    Base::finalize(fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     247    if (JITCode::useDataIC(m_jitType))
     248        m_stubInfo->m_codePtr = m_stubInfo->slowPathStartLocation;
     249}
     250
     251JITInByValGenerator::JITInByValGenerator(CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg stubInfoGPR)
     252    : Base(codeBlock, jitType, codeOrigin, callSiteIndex, AccessType::InByVal, usedRegisters)
    240253{
    241254    m_stubInfo->hasConstantIdentifier = false;
    … …  
    244257    m_stubInfo->regs.propertyGPR = property.payloadGPR();
    245258    m_stubInfo->valueGPR = result.payloadGPR();
     259    m_stubInfo->m_stubInfoGPR = stubInfoGPR;
    246260#if USE(JSVALUE32_64)
    247261    m_stubInfo->baseTagGPR = base.tagGPR();
    … …  
    254268{
    255269    m_start = jit.label();
    256     m_slowPathJump = jit.patchableJump();
     270    if (JITCode::useDataIC(m_jitType)) {
     271        jit.move(CCallHelpers::TrustedImmPtr(m_stubInfo), m_stubInfo->m_stubInfoGPR);
     272        jit.call(CCallHelpers::Address(m_stubInfo->m_stubInfoGPR, StructureStubInfo::offsetOfCodePtr()), JITStubRoutinePtrTag);
     273    } else
     274        m_slowPathJump = jit.patchableJump();
    257275    m_done = jit.label();
    258276}
    … …  
    262280{
    263281    ASSERT(m_start.isSet());
    264     Base::finalize(
    265         fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
    266 }
    267 
    268 JITInByIdGenerator::JITInByIdGenerator(CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters, CacheableIdentifier propertyName, JSValueRegs base, JSValueRegs value)
    269     : JITByIdGenerator(codeBlock, codeOrigin, callSite, AccessType::InById, usedRegisters, base, value)
     282    Base::finalize(fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     283    if (JITCode::useDataIC(m_jitType))
     284        m_stubInfo->m_codePtr = m_stubInfo->slowPathStartLocation;
     285}
     286
     287JITInByIdGenerator::JITInByIdGenerator(
     288    CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSite, const RegisterSet& usedRegisters,
     289    CacheableIdentifier propertyName, JSValueRegs base, JSValueRegs value, GPRReg stubInfoGPR)
     290    : JITByIdGenerator(codeBlock, jitType, codeOrigin, callSite, AccessType::InById, usedRegisters, base, value, stubInfoGPR)
    270291{
    271292    // FIXME: We are not supporting fast path for "length" property.
    … …  
    280301
    281302JITInstanceOfGenerator::JITInstanceOfGenerator(
    282     CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex,
    283     const RegisterSet& usedRegisters, GPRReg result, GPRReg value, GPRReg prototype,
     303    CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex,
     304    const RegisterSet& usedRegisters, GPRReg result, GPRReg value, GPRReg prototype, GPRReg stubInfoGPR,
    284305    GPRReg scratch1, GPRReg scratch2, bool prototypeIsKnownObject)
    285     : JITInlineCacheGenerator(
    286         codeBlock, codeOrigin, callSiteIndex, AccessType::InstanceOf, usedRegisters)
     306    : JITInlineCacheGenerator(codeBlock, jitType, codeOrigin, callSiteIndex, AccessType::InstanceOf, usedRegisters)
    287307{
    288308    m_stubInfo->baseGPR = value;
    289309    m_stubInfo->valueGPR = result;
    290310    m_stubInfo->regs.prototypeGPR = prototype;
     311    m_stubInfo->m_stubInfoGPR = stubInfoGPR;
    291312#if USE(JSVALUE32_64)
    292313    m_stubInfo->baseTagGPR = InvalidGPRReg;
    … …  
    308329void JITInstanceOfGenerator::generateFastPath(MacroAssembler& jit)
    309330{
    310     m_jump = jit.patchableJump();
     331    m_start = jit.label();
     332    if (JITCode::useDataIC(m_jitType)) {
     333        jit.move(CCallHelpers::TrustedImmPtr(m_stubInfo), m_stubInfo->m_stubInfoGPR);
     334        jit.call(CCallHelpers::Address(m_stubInfo->m_stubInfoGPR, StructureStubInfo::offsetOfCodePtr()), JITStubRoutinePtrTag);
     335    } else
     336        m_slowPathJump = jit.patchableJump();
    311337    m_done = jit.label();
    312338}
    … …  
    314340void JITInstanceOfGenerator::finalize(LinkBuffer& fastPath, LinkBuffer& slowPath)
    315341{
    316     JITInlineCacheGenerator::finalize(
    317         fastPath, slowPath,
    318         fastPath.locationOf<JITStubRoutinePtrTag>(m_jump));
    319    
    320     fastPath.link(m_jump.m_jump, slowPath.locationOf<NoPtrTag>(m_slowPathBegin));
    321 }
    322 
    323 JITGetByValGenerator::JITGetByValGenerator(CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, AccessType accessType, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs property, JSValueRegs result)
    324     : Base(codeBlock, codeOrigin, callSiteIndex, accessType, usedRegisters)
     342    Base::finalize(fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     343    if (JITCode::useDataIC(m_jitType))
     344        m_stubInfo->m_codePtr = m_stubInfo->slowPathStartLocation;
     345}
     346
     347JITGetByValGenerator::JITGetByValGenerator(CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, AccessType accessType, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg stubInfoGPR)
     348    : Base(codeBlock, jitType, codeOrigin, callSiteIndex, accessType, usedRegisters)
    325349    , m_base(base)
    326350    , m_result(result)
    … …  
    331355    m_stubInfo->regs.propertyGPR = property.payloadGPR();
    332356    m_stubInfo->valueGPR = result.payloadGPR();
     357    m_stubInfo->m_stubInfoGPR = stubInfoGPR;
    333358#if USE(JSVALUE32_64)
    334359    m_stubInfo->baseTagGPR = base.tagGPR();
    … …  
    341366{
    342367    m_start = jit.label();
    343     m_slowPathJump = jit.patchableJump();
    344     m_done = jit.label();
    345 }
    346 
    347 void JITGetByValGenerator::finalize(
    348     LinkBuffer& fastPath, LinkBuffer& slowPath)
    349 {
    350     ASSERT(m_start.isSet());
    351     Base::finalize(
    352         fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
    353 }
    354 
    355 JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator(CodeBlock* codeBlock, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, AccessType accessType, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs brand)
    356     : Base(codeBlock, codeOrigin, callSiteIndex, accessType, usedRegisters)
     368    if (JITCode::useDataIC(m_jitType)) {
     369        jit.move(CCallHelpers::TrustedImmPtr(m_stubInfo), m_stubInfo->m_stubInfoGPR);
     370        jit.call(CCallHelpers::Address(m_stubInfo->m_stubInfoGPR, StructureStubInfo::offsetOfCodePtr()), JITStubRoutinePtrTag);
     371    } else
     372        m_slowPathJump = jit.patchableJump();
     373    m_done = jit.label();
     374}
     375
     376void JITGetByValGenerator::finalize(LinkBuffer& fastPath, LinkBuffer& slowPath)
     377{
     378    Base::finalize(fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     379    if (JITCode::useDataIC(m_jitType))
     380        m_stubInfo->m_codePtr = m_stubInfo->slowPathStartLocation;
     381}
     382
     383JITPrivateBrandAccessGenerator::JITPrivateBrandAccessGenerator(CodeBlock* codeBlock, JITType jitType, CodeOrigin codeOrigin, CallSiteIndex callSiteIndex, AccessType accessType, const RegisterSet& usedRegisters, JSValueRegs base, JSValueRegs brand, GPRReg stubInfoGPR)
     384    : Base(codeBlock, jitType, codeOrigin, callSiteIndex, accessType, usedRegisters)
    357385{
    358386    ASSERT(accessType == AccessType::CheckPrivateBrand || accessType == AccessType::SetPrivateBrand);
    … …  
    362390    m_stubInfo->regs.brandGPR = brand.payloadGPR();
    363391    m_stubInfo->valueGPR = InvalidGPRReg;
     392    m_stubInfo->m_stubInfoGPR = stubInfoGPR;
    364393#if USE(JSVALUE32_64)
    365394    m_stubInfo->baseTagGPR = base.tagGPR();
    … …  
    372401{
    373402    m_start = jit.label();
    374     m_slowPathJump = jit.patchableJump();
    375     m_done = jit.label();
    376 }
    377 
    378 void JITPrivateBrandAccessGenerator::finalize(
    379     LinkBuffer& fastPath, LinkBuffer& slowPath)
    380 {
    381     ASSERT(m_start.isSet());
    382     Base::finalize(
    383         fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     403    if (JITCode::useDataIC(m_jitType)) {
     404        jit.move(CCallHelpers::TrustedImmPtr(m_stubInfo), m_stubInfo->m_stubInfoGPR);
     405        jit.call(CCallHelpers::Address(m_stubInfo->m_stubInfoGPR, StructureStubInfo::offsetOfCodePtr()), JITStubRoutinePtrTag);
     406    } else
     407        m_slowPathJump = jit.patchableJump();
     408    m_done = jit.label();
     409}
     410
     411void JITPrivateBrandAccessGenerator::finalize(LinkBuffer& fastPath, LinkBuffer& slowPath)
     412{
     413    Base::finalize(fastPath, slowPath, fastPath.locationOf<JITStubRoutinePtrTag>(m_start));
     414    if (JITCode::useDataIC(m_jitType))
     415        m_stubInfo->m_codePtr = m_stubInfo->slowPathStartLocation;
    384416}
    385417

  • trunk/Source/JavaScriptCore/jit/JITInlineCacheGenerator.h

    r278445 r278656  
    4242
    4343enum class AccessType : int8_t;
     44enum class JITType : uint8_t;
    4445
    4546class JITInlineCacheGenerator {
    4647protected:
    4748    JITInlineCacheGenerator() { }
    48     JITInlineCacheGenerator(
    49         CodeBlock*, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters);
     49    JITInlineCacheGenerator(CodeBlock*, JITType, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters);
    5050   
    5151public:
    … …  
    6666protected:
    6767    CodeBlock* m_codeBlock;
     68    JITType m_jitType;
    6869    StructureStubInfo* m_stubInfo;
    6970
    … …  
    7879
    7980    JITByIdGenerator(
    80         CodeBlock*, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters,
    81         JSValueRegs base, JSValueRegs value);
     81        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters,
     82        JSValueRegs base, JSValueRegs value, GPRReg stubInfoGPR);
    8283
    8384public:
    … …  
    102103};
    103104
    104 class JITGetByIdGenerator : public JITByIdGenerator {
     105class JITGetByIdGenerator final : public JITByIdGenerator {
    105106public:
    106107    JITGetByIdGenerator() { }
    107108
    108109    JITGetByIdGenerator(
    109         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
    110         JSValueRegs base, JSValueRegs value, AccessType);
     110        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
     111        JSValueRegs base, JSValueRegs value, GPRReg stubInfoGPR, AccessType);
    111112   
    112113    void generateFastPath(MacroAssembler&);
    … …  
    116117};
    117118
    118 class JITGetByIdWithThisGenerator : public JITByIdGenerator {
     119class JITGetByIdWithThisGenerator final : public JITByIdGenerator {
    119120public:
    120121    JITGetByIdWithThisGenerator() { }
    121122
    122123    JITGetByIdWithThisGenerator(
    123         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
    124         JSValueRegs value, JSValueRegs base, JSValueRegs thisRegs);
    125 
    126     void generateFastPath(MacroAssembler&);
    127 };
    128 
    129 class JITPutByIdGenerator : public JITByIdGenerator {
     124        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
     125        JSValueRegs value, JSValueRegs base, JSValueRegs thisRegs, GPRReg stubInfoGPR);
     126
     127    void generateFastPath(MacroAssembler&);
     128};
     129
     130class JITPutByIdGenerator final : public JITByIdGenerator {
    130131public:
    131132    JITPutByIdGenerator()
    … …  
    134135
    135136    JITPutByIdGenerator(
    136         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier, JSValueRegs base,
    137         JSValueRegs value, GPRReg scratch, ECMAMode, PutKind);
     137        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier, JSValueRegs base,
     138        JSValueRegs value, GPRReg stubInfoGPR, GPRReg scratch, ECMAMode, PutKind);
    138139   
    139140    void generateFastPath(MacroAssembler&);
    … …  
    146147};
    147148
    148 class JITDelByValGenerator : public JITInlineCacheGenerator {
     149class JITDelByValGenerator final : public JITInlineCacheGenerator {
    149150    using Base = JITInlineCacheGenerator;
    150151public:
    … …  
    152153
    153154    JITDelByValGenerator(
    154         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters,
    155         JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg scratch);
    156 
    157     MacroAssembler::Jump slowPathJump() const
    158     {
    159         ASSERT(m_slowPathJump.m_jump.isSet());
    160         return m_slowPathJump.m_jump;
    161     }
    162 
    163     void finalize(
    164         LinkBuffer& fastPathLinkBuffer, LinkBuffer& slowPathLinkBuffer);
    165 
    166     void generateFastPath(MacroAssembler&);
    167 
    168 private:
    169     MacroAssembler::Label m_start;
    170     MacroAssembler::PatchableJump m_slowPathJump;
    171 };
    172 
    173 class JITDelByIdGenerator : public JITInlineCacheGenerator {
     155        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters,
     156        JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg stubInfoGPR, GPRReg scratch);
     157
     158    MacroAssembler::Jump slowPathJump() const
     159    {
     160        ASSERT(m_slowPathJump.m_jump.isSet());
     161        return m_slowPathJump.m_jump;
     162    }
     163
     164    void finalize(
     165        LinkBuffer& fastPathLinkBuffer, LinkBuffer& slowPathLinkBuffer);
     166
     167    void generateFastPath(MacroAssembler&);
     168
     169private:
     170    MacroAssembler::Label m_start;
     171    MacroAssembler::PatchableJump m_slowPathJump;
     172};
     173
     174class JITDelByIdGenerator final : public JITInlineCacheGenerator {
    174175    using Base = JITInlineCacheGenerator;
    175176public:
    … …  
    177178
    178179    JITDelByIdGenerator(
    179         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
    180         JSValueRegs base, JSValueRegs result, GPRReg scratch);
     180        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
     181        JSValueRegs base, JSValueRegs result, GPRReg stubInfoGPR, GPRReg scratch);
    181182
    182183    MacroAssembler::Jump slowPathJump() const
    … …  
    202203
    203204    JITInByValGenerator(
    204         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters,
    205         JSValueRegs base, JSValueRegs property, JSValueRegs result);
    206 
    207     MacroAssembler::Jump slowPathJump() const
    208     {
    209         ASSERT(m_slowPathJump.m_jump.isSet());
    210         return m_slowPathJump.m_jump;
    211     }
    212 
    213     void finalize(
    214         LinkBuffer& fastPathLinkBuffer, LinkBuffer& slowPathLinkBuffer);
    215 
    216     void generateFastPath(MacroAssembler&);
    217 
    218 private:
    219     MacroAssembler::Label m_start;
    220     MacroAssembler::PatchableJump m_slowPathJump;
    221 };
    222 
    223 class JITInByIdGenerator : public JITByIdGenerator {
     205        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters,
     206        JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg stubInfoGPR);
     207
     208    MacroAssembler::Jump slowPathJump() const
     209    {
     210        ASSERT(m_slowPathJump.m_jump.isSet());
     211        return m_slowPathJump.m_jump;
     212    }
     213
     214    void finalize(
     215        LinkBuffer& fastPathLinkBuffer, LinkBuffer& slowPathLinkBuffer);
     216
     217    void generateFastPath(MacroAssembler&);
     218
     219private:
     220    MacroAssembler::Label m_start;
     221    MacroAssembler::PatchableJump m_slowPathJump;
     222};
     223
     224class JITInByIdGenerator final : public JITByIdGenerator {
    224225public:
    225226    JITInByIdGenerator() { }
    226227
    227228    JITInByIdGenerator(
    228         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
    229         JSValueRegs base, JSValueRegs value);
    230 
    231     void generateFastPath(MacroAssembler&);
    232 };
    233 
    234 class JITInstanceOfGenerator : public JITInlineCacheGenerator {
    235 public:
     229        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, CacheableIdentifier,
     230        JSValueRegs base, JSValueRegs value, GPRReg stubInfoGPR);
     231
     232    void generateFastPath(MacroAssembler&);
     233};
     234
     235class JITInstanceOfGenerator final : public JITInlineCacheGenerator {
     236public:
     237    using Base = JITInlineCacheGenerator;
    236238    JITInstanceOfGenerator() { }
    237239   
    238240    JITInstanceOfGenerator(
    239         CodeBlock*, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, GPRReg result,
    240         GPRReg value, GPRReg prototype, GPRReg scratch1, GPRReg scratch2,
     241        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, const RegisterSet& usedRegisters, GPRReg result,
     242        GPRReg value, GPRReg prototype, GPRReg stubInfoGPR, GPRReg scratch1, GPRReg scratch2,
    241243        bool prototypeIsKnownObject = false);
    242244   
    243245    void generateFastPath(MacroAssembler&);
    244246
     247    MacroAssembler::Jump slowPathJump() const
     248    {
     249        ASSERT(m_slowPathJump.m_jump.isSet());
     250        return m_slowPathJump.m_jump;
     251    }
     252
    245253    void finalize(LinkBuffer& fastPathLinkBuffer, LinkBuffer& slowPathLinkBuffer);
    246254
    247255private:
    248     MacroAssembler::PatchableJump m_jump;
    249 };
    250 
    251 class JITGetByValGenerator : public JITInlineCacheGenerator {
     256    MacroAssembler::Label m_start;
     257    MacroAssembler::PatchableJump m_slowPathJump;
     258};
     259
     260class JITGetByValGenerator final : public JITInlineCacheGenerator {
    252261    using Base = JITInlineCacheGenerator;
    253262public:
    … …  
    255264
    256265    JITGetByValGenerator(
    257         CodeBlock*, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters,
    258         JSValueRegs base, JSValueRegs property, JSValueRegs result);
     266        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters,
     267        JSValueRegs base, JSValueRegs property, JSValueRegs result, GPRReg stubInfoGPR);
    259268
    260269    MacroAssembler::Jump slowPathJump() const
    … …  
    277286};
    278287
    279 class JITPrivateBrandAccessGenerator : public JITInlineCacheGenerator {
     288class JITPrivateBrandAccessGenerator final : public JITInlineCacheGenerator {
    280289    using Base = JITInlineCacheGenerator;
    281290public:
    … …  
    283292
    284293    JITPrivateBrandAccessGenerator(
    285         CodeBlock*, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters,
    286         JSValueRegs base, JSValueRegs brand);
     294        CodeBlock*, JITType, CodeOrigin, CallSiteIndex, AccessType, const RegisterSet& usedRegisters,
     295        JSValueRegs base, JSValueRegs brand, GPRReg stubInfoGPR);
    287296
    288297    MacroAssembler::Jump slowPathJump() const

  • trunk/Source/JavaScriptCore/jit/JITInlines.h

    r278576 r278656  
    3434namespace JSC {
    3535
    36 ALWAYS_INLINE MacroAssembler::JumpList JIT::emitLoadForArrayMode(const Instruction* currentInstruction, JITArrayMode arrayMode, PatchableJump& badType)
     36ALWAYS_INLINE MacroAssembler::JumpList JIT::emitLoadForArrayMode(const Instruction* currentInstruction, JITArrayMode arrayMode, PatchableJump& badType, ByValInfo* byValInfo)
    3737{
    3838    switch (arrayMode) {
    3939    case JITInt32:
    40         return emitInt32Load(currentInstruction, badType);
     40        return emitInt32Load(currentInstruction, badType, byValInfo);
    4141    case JITDouble:
    42         return emitDoubleLoad(currentInstruction, badType);
     42        return emitDoubleLoad(currentInstruction, badType, byValInfo);
    4343    case JITContiguous:
    44         return emitContiguousLoad(currentInstruction, badType);
     44        return emitContiguousLoad(currentInstruction, badType, byValInfo);
    4545    case JITArrayStorage:
    46         return emitArrayStorageLoad(currentInstruction, badType);
     46        return emitArrayStorageLoad(currentInstruction, badType, byValInfo);
    4747    default:
    4848        break;
    … …  
    128128}
    129129
     130ALWAYS_INLINE void JIT::appendCallWithExceptionCheck(Address function)
     131{
     132    updateTopCallFrame();
     133    appendCall(function);
     134    exceptionCheck();
     135}
     136
    130137#if OS(WINDOWS) && CPU(X86_64)
    131138ALWAYS_INLINE MacroAssembler::Call JIT::appendCallWithExceptionCheckAndSlowPathReturnType(const FunctionPtr<CFunctionPtrTag> function)
    … …  
    155162#endif
    156163    return call;
     164}
     165
     166ALWAYS_INLINE void JIT::appendCallWithExceptionCheckSetJSValueResult(Address function, VirtualRegister dst)
     167{
     168    appendCallWithExceptionCheck(function);
     169#if USE(JSVALUE64)
     170    emitPutVirtualRegister(dst, returnValueGPR);
     171#else
     172    emitStore(dst, returnValueGPR2, returnValueGPR);
     173#endif
    157174}
    158175
    … …  
    169186#endif
    170187    return call;
     188}
     189
     190template<typename Metadata>
     191ALWAYS_INLINE void JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile(Metadata& metadata, Address function, VirtualRegister dst)
     192{
     193    appendCallWithExceptionCheck(function);
     194#if USE(JSVALUE64)
     195    emitValueProfilingSite(metadata, returnValueGPR);
     196    emitPutVirtualRegister(dst, returnValueGPR);
     197#else
     198    emitValueProfilingSite(metadata, JSValueRegs(returnValueGPR2, returnValueGPR));
     199    emitStore(dst, returnValueGPR2, returnValueGPR);
     200#endif
    171201}
    172202

  • trunk/Source/JavaScriptCore/jit/JITOpcodes.cpp

    r278576 r278656  
    165165
    166166    JITInstanceOfGenerator gen(
    167         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex),
     167        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex),
    168168        RegisterSet::stubUnavailableRegisters(),
    169169        regT0, // result
    170170        regT2, // value
    171171        regT1, // proto
     172        regT5,
    172173        regT3, regT4); // scratch
    173174    gen.generateFastPath(*this);
     175    if (!JITCode::useDataIC(JITType::BaselineJIT))
     176        addSlowCase(gen.slowPathJump());
     177    else
     178        addSlowCase();
    174179    m_instanceOfs.append(gen);
    175180   
    … …  
    187192   
    188193    Label coldPathBegin = label();
    189     Call call = callOperation(operationInstanceOfOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT2, regT1);
     194
     195    Call call;
     196    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     197        gen.stubInfo()->m_slowOperation = operationInstanceOfOptimize;
     198        move(TrustedImmPtr(gen.stubInfo()), GPRInfo::nonArgGPR0);
     199        callOperation<decltype(operationInstanceOfOptimize)>(Address(GPRInfo::nonArgGPR0, StructureStubInfo::offsetOfSlowOperation()), resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), GPRInfo::nonArgGPR0, regT2, regT1);
     200    } else
     201        call = callOperation(operationInstanceOfOptimize, resultVReg, TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT2, regT1);
    190202    gen.reportSlowPathCall(coldPathBegin, call);
    191203}
    … …  
    18051817    // FIXME: Add support for other types like TypedArrays and Arguments.
    18061818    // See https://bugs.webkit.org/show_bug.cgi?id=135033 and https://bugs.webkit.org/show_bug.cgi?id=135034.
    1807     JumpList slowCases = emitLoadForArrayMode(currentInstruction, arrayMode, badType);
     1819    JumpList slowCases = emitLoadForArrayMode(currentInstruction, arrayMode, badType, nullptr);
    18081820    move(TrustedImm64(JSValue::encode(jsBoolean(true))), regT0);
    18091821    Jump done = jump();
    … …  
    18201832        "Baseline has_indexed_property stub for %s, return point %p", toCString(*m_codeBlock).data(), returnAddress.untaggedValue());
    18211833   
    1822     MacroAssembler::repatchJump(byValInfo->badTypeJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
    1823     MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(operationHasIndexedPropertyGeneric));
     1834    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     1835        byValInfo->m_badTypeJumpTarget = CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code());
     1836        byValInfo->m_slowOperation = operationHasIndexedPropertyGeneric;
     1837    } else {
     1838        MacroAssembler::repatchJump(byValInfo->m_badTypeJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
     1839        MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(operationHasIndexedPropertyGeneric));
     1840    }
    18241841}
    18251842
    … …  
    18551872    // FIXME: Add support for other types like TypedArrays and Arguments.
    18561873    // See https://bugs.webkit.org/show_bug.cgi?id=135033 and https://bugs.webkit.org/show_bug.cgi?id=135034.
    1857     JumpList slowCases = emitLoadForArrayMode(currentInstruction, mode, badType);
     1874    JumpList slowCases = emitLoadForArrayMode(currentInstruction, mode, badType, byValInfo);
    18581875   
    18591876    move(TrustedImm64(JSValue::encode(jsBoolean(true))), regT0);
    18601877
    1861     addSlowCase(badType);
     1878    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1879        addSlowCase(badType);
    18621880    addSlowCase(slowCases);
    18631881   
    … …  
    18851903    emitGetVirtualRegister(base, regT0);
    18861904    emitGetVirtualRegister(property, regT1);
    1887     Call call = callOperation(operationHasIndexedPropertyDefault, dst, TrustedImmPtr(m_codeBlock->globalObject()), regT0, regT1, byValInfo);
     1905
     1906    Call call;
     1907    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     1908        byValInfo->m_slowOperation = operationHasIndexedPropertyDefault;
     1909        move(TrustedImmPtr(byValInfo), GPRInfo::nonArgGPR0);
     1910        callOperation<decltype(operationHasIndexedPropertyDefault)>(Address(GPRInfo::nonArgGPR0, ByValInfo::offsetOfSlowOperation()), dst, TrustedImmPtr(m_codeBlock->globalObject()), regT0, regT1, GPRInfo::nonArgGPR0);
     1911    } else
     1912        call = callOperation(operationHasIndexedPropertyDefault, dst, TrustedImmPtr(m_codeBlock->globalObject()), regT0, regT1, byValInfo);
    18881913
    18891914    m_byValCompilationInfo[m_byValInstructionIndex].slowPathTarget = slowPath;

  • trunk/Source/JavaScriptCore/jit/JITOpcodes32_64.cpp

    r278576 r278656  
    163163   
    164164    JITInstanceOfGenerator gen(
    165         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex),
     165        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex),
    166166        RegisterSet::stubUnavailableRegisters(),
    167167        regT0, // result
    168168        regT2, // value
    169169        regT1, // proto
     170        InvalidGPRReg,
    170171        regT3, regT4); // scratch
    171172    gen.generateFastPath(*this);
     173    addSlowCase(gen.slowPathJump());
    172174    m_instanceOfs.append(gen);
    173175   
    … …  
    11841186    // FIXME: Add support for other types like TypedArrays and Arguments.
    11851187    // See https://bugs.webkit.org/show_bug.cgi?id=135033 and https://bugs.webkit.org/show_bug.cgi?id=135034.
    1186     JumpList slowCases = emitLoadForArrayMode(currentInstruction, arrayMode, badType);
     1188    JumpList slowCases = emitLoadForArrayMode(currentInstruction, arrayMode, badType, nullptr);
    11871189    move(TrustedImm32(1), regT0);
    11881190    Jump done = jump();
    … …  
    11991201        "Baseline has_indexed_property stub for %s, return point %p", toCString(*m_codeBlock).data(), returnAddress.untaggedValue());
    12001202   
    1201     MacroAssembler::repatchJump(byValInfo->badTypeJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
     1203    MacroAssembler::repatchJump(byValInfo->m_badTypeJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
    12021204    MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(operationHasIndexedPropertyGeneric));
    12031205}
    … …  
    12351237    // FIXME: Add support for other types like TypedArrays and Arguments.
    12361238    // See https://bugs.webkit.org/show_bug.cgi?id=135033 and https://bugs.webkit.org/show_bug.cgi?id=135034.
    1237     JumpList slowCases = emitLoadForArrayMode(currentInstruction, mode, badType);
     1239    JumpList slowCases = emitLoadForArrayMode(currentInstruction, mode, badType, byValInfo);
    12381240    move(TrustedImm32(1), regT0);
    12391241

  • trunk/Source/JavaScriptCore/jit/JITOperations.cpp

    r278591 r278656  
    10481048        // Don't ever try to optimize.
    10491049        byValInfo->tookSlowPath = true;
    1050         ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationPutByValGeneric);
     1050        if (codeBlock->useDataIC())
     1051            byValInfo->m_slowOperation = operationPutByValGeneric;
     1052        else
     1053            ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationPutByValGeneric);
    10511054    }
    10521055    RELEASE_AND_RETURN(scope, putByVal(globalObject, baseValue, subscript, value, byValInfo, ecmaMode));
    … …  
    11401143        // Don't ever try to optimize.
    11411144        byValInfo->tookSlowPath = true;
    1142         ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationDirectPutByValGeneric);
     1145        if (codeBlock->useDataIC())
     1146            byValInfo->m_slowOperation = operationDirectPutByValGeneric;
     1147        else
     1148            ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationDirectPutByValGeneric);
    11431149    }
    11441150
    … …  
    13241330        // Don't ever try to optimize.
    13251331        byValInfo->tookSlowPath = true;
    1326         ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationPutPrivateNameGeneric);
     1332        if (codeBlock->useDataIC())
     1333            byValInfo->m_slowOperation = operationPutPrivateNameGeneric;
     1334        else
     1335            ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationPutPrivateNameGeneric);
    13271336    }
    13281337
    … …  
    26132622            || object->structure(vm)->typeInfo().interceptsGetOwnPropertySlotByIndexEvenWhenLengthIsNotZero()) {
    26142623            // Don't ever try to optimize.
    2615             ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationHasIndexedPropertyGeneric);
     2624            CodeBlock* codeBlock = callFrame->codeBlock();
     2625            if (codeBlock->useDataIC())
     2626                byValInfo->m_slowOperation = operationHasIndexedPropertyGeneric;
     2627            else
     2628                ctiPatchCallByReturnAddress(ReturnAddressPtr(OUR_RETURN_ADDRESS), operationHasIndexedPropertyGeneric);
    26162629        }
    26172630    }

  • trunk/Source/JavaScriptCore/jit/JITPropertyAccess.cpp

    r278591 r278656  
    7070
    7171        JITGetByValGenerator gen(
    72             m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetByVal, RegisterSet::stubUnavailableRegisters(),
    73             JSValueRegs(regT0), JSValueRegs(regT1), resultRegs);
     72            m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetByVal, RegisterSet::stubUnavailableRegisters(),
     73            JSValueRegs(regT0), JSValueRegs(regT1), resultRegs, regT2);
    7474        if (isOperandConstantInt(property))
    7575            gen.stubInfo()->propertyIsInt32 = true;
    7676        gen.generateFastPath(*this);
    77         addSlowCase(gen.slowPathJump());
     77        if (!JITCode::useDataIC(JITType::BaselineJIT))
     78            addSlowCase(gen.slowPathJump());
     79        else
     80            addSlowCase();
    7881        m_getByVals.append(gen);
    7982
    … …  
    9497        linkAllSlowCases(iter);
    9598
    96         JITGetByValGenerator& gen = m_getByVals[m_getByValIndex];
    97         ++m_getByValIndex;
     99        JITGetByValGenerator& gen = m_getByVals[m_getByValIndex++];
     100
    98101        Label coldPathBegin = label();
    99102
    … …  
    119122        emitNakedNearCall(vm.getCTIStub(slow_op_get_by_val_prepareCallGenerator).retaggedCode<NoPtrTag>());
    120123
    121         Call call = appendCall(operationGetByValOptimize);
     124        Call call;
     125        if (JITCode::useDataIC(JITType::BaselineJIT))
     126            gen.stubInfo()->m_slowOperation = operationGetByValOptimize;
     127        else
     128            call = appendCall(operationGetByValOptimize);
    122129        emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    123130
    … …  
    139146    JIT jit(vm);
    140147
    141     jit.tagReturnAddress();
     148    if (!JITCode::useDataIC(JITType::BaselineJIT))
     149        jit.tagReturnAddress();
    142150
    143151    constexpr GPRReg bytecodeOffsetGPR = argumentGPR4;
    … …  
    158166    jit.prepareCallOperation(vm);
    159167
    160     // FIXME: Consider making this a tail call directly to the target operation instead
    161     // of returning to the client to call it.
    162     jit.ret();
     168    if (JITCode::useDataIC(JITType::BaselineJIT))
     169        jit.farJump(Address(argumentGPR1, StructureStubInfo::offsetOfSlowOperation()), OperationPtrTag);
     170    else
     171        jit.ret();
    163172
    164173    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    183192
    184193    JITGetByValGenerator gen(
    185         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetPrivateName,
    186         RegisterSet::stubUnavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(propertyGPR), resultRegs);
     194        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetPrivateName,
     195        RegisterSet::stubUnavailableRegisters(), JSValueRegs(baseGPR), JSValueRegs(propertyGPR), resultRegs, regT2);
    187196    gen.generateFastPath(*this);
    188     addSlowCase(gen.slowPathJump());
     197    if (!JITCode::useDataIC(JITType::BaselineJIT))
     198        addSlowCase(gen.slowPathJump());
     199    else
     200        addSlowCase();
    189201    m_getByVals.append(gen);
    190202
    … …  
    202214    linkAllSlowCases(iter);
    203215
    204     JITGetByValGenerator& gen = m_getByVals[m_getByValIndex];
    205     ++m_getByValIndex;
     216    JITGetByValGenerator& gen = m_getByVals[m_getByValIndex++];
    206217    Label coldPathBegin = label();
    207218
    … …  
    227238    emitNakedNearCall(vm.getCTIStub(slow_op_get_private_name_prepareCallGenerator).retaggedCode<NoPtrTag>());
    228239
    229     Call call = appendCall(operationGetPrivateNameOptimize);
     240    Call call;
     241    if (JITCode::useDataIC(JITType::BaselineJIT))
     242        gen.stubInfo()->m_slowOperation = operationGetPrivateNameOptimize;
     243    else
     244        call = appendCall(operationGetPrivateNameOptimize);
    230245    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    231246
    … …  
    246261    JIT jit(vm);
    247262
    248     jit.tagReturnAddress();
     263    if (!JITCode::useDataIC(JITType::BaselineJIT))
     264        jit.tagReturnAddress();
    249265
    250266    constexpr GPRReg bytecodeOffsetGPR = argumentGPR3;
    … …  
    264280    jit.prepareCallOperation(vm);
    265281
    266     // FIXME: Consider making this a tail call directly to the target operation instead
    267     // of returning to the client to call it.
    268     jit.ret();
     282    if (JITCode::useDataIC(JITType::BaselineJIT))
     283        jit.farJump(Address(argumentGPR1, StructureStubInfo::offsetOfSlowOperation()), OperationPtrTag);
     284    else
     285        jit.ret();
    269286
    270287    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    286303
    287304    JITPrivateBrandAccessGenerator gen(
    288         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::SetPrivateBrand, RegisterSet::stubUnavailableRegisters(),
    289         JSValueRegs(baseGPR), JSValueRegs(brandGPR));
     305        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::SetPrivateBrand, RegisterSet::stubUnavailableRegisters(),
     306        JSValueRegs(baseGPR), JSValueRegs(brandGPR), regT2);
    290307    gen.generateFastPath(*this);
    291     addSlowCase(gen.slowPathJump());
     308    if (!JITCode::useDataIC(JITType::BaselineJIT))
     309        addSlowCase(gen.slowPathJump());
     310    else
     311        addSlowCase();
    292312    m_privateBrandAccesses.append(gen);
    293313
    … …  
    303323    linkAllSlowCases(iter);
    304324
    305     JITPrivateBrandAccessGenerator& gen = m_privateBrandAccesses[m_privateBrandAccessIndex];
    306     ++m_privateBrandAccessIndex;
     325    JITPrivateBrandAccessGenerator& gen = m_privateBrandAccesses[m_privateBrandAccessIndex++];
    307326    Label coldPathBegin = label();
    308327
    … …  
    329348    emitNakedNearCall(vm.getCTIStub(slow_op_get_private_name_prepareCallGenerator).retaggedCode<NoPtrTag>());
    330349
    331     Call call = appendCall(operationSetPrivateBrandOptimize);
     350    Call call;
     351    if (JITCode::useDataIC(JITType::BaselineJIT))
     352        gen.stubInfo()->m_slowOperation = operationSetPrivateBrandOptimize;
     353    else
     354        call = appendCall(operationSetPrivateBrandOptimize);
    332355    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    333356#endif
     357
    334358    gen.reportSlowPathCall(coldPathBegin, call);
    335359}
    … …  
    347371
    348372    JITPrivateBrandAccessGenerator gen(
    349         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::CheckPrivateBrand, RegisterSet::stubUnavailableRegisters(),
    350         JSValueRegs(regT0), JSValueRegs(regT1));
     373        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::CheckPrivateBrand, RegisterSet::stubUnavailableRegisters(),
     374        JSValueRegs(regT0), JSValueRegs(regT1), regT2);
    351375    gen.generateFastPath(*this);
    352     addSlowCase(gen.slowPathJump());
     376    if (!JITCode::useDataIC(JITType::BaselineJIT))
     377        addSlowCase(gen.slowPathJump());
     378    else
     379        addSlowCase();
    353380    m_privateBrandAccesses.append(gen);
    354381}
    … …  
    358385    linkAllSlowCases(iter);
    359386
    360     JITPrivateBrandAccessGenerator& gen = m_privateBrandAccesses[m_privateBrandAccessIndex];
    361     ++m_privateBrandAccessIndex;
     387    JITPrivateBrandAccessGenerator& gen = m_privateBrandAccesses[m_privateBrandAccessIndex++];
    362388    Label coldPathBegin = label();
    363389
    … …  
    384410    emitNakedNearCall(vm.getCTIStub(slow_op_get_private_name_prepareCallGenerator).retaggedCode<NoPtrTag>());
    385411
    386     Call call = appendCall(operationCheckPrivateBrandOptimize);
     412    Call call;
     413    if (JITCode::useDataIC(JITType::BaselineJIT))
     414        gen.stubInfo()->m_slowOperation = operationCheckPrivateBrandOptimize;
     415    else
     416        call = appendCall(operationCheckPrivateBrandOptimize);
    387417    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    388418#endif // ENABLE(EXTRA_CTI_THUNKS)
    … …  
    416446    PatchableJump notIndex;
    417447    if (!propertyNameIsIntegerConstant) {
    418         notIndex = emitPatchableJumpIfNotInt(regT1);
    419         addSlowCase(notIndex);
     448        if (JITCode::useDataIC(JITType::BaselineJIT)) {
     449            auto isInt32 = branchIfInt32(regT1);
     450            farJump(AbsoluteAddress(&byValInfo->m_notIndexJumpTarget), JITStubRoutinePtrTag);
     451            isInt32.link(this);
     452        } else {
     453            notIndex = emitPatchableJumpIfNotInt(regT1);
     454            addSlowCase(notIndex);
     455        }
    420456        // See comment in op_get_by_val.
    421457        zeroExtend32ToWord(regT1, regT1);
    … …  
    433469    switch (mode) {
    434470    case JITInt32:
    435         slowCases = emitInt32PutByVal(bytecode, badType);
     471        slowCases = emitInt32PutByVal(bytecode, badType, byValInfo);
    436472        break;
    437473    case JITDouble:
    438         slowCases = emitDoublePutByVal(bytecode, badType);
     474        slowCases = emitDoublePutByVal(bytecode, badType, byValInfo);
    439475        break;
    440476    case JITContiguous:
    441         slowCases = emitContiguousPutByVal(bytecode, badType);
     477        slowCases = emitContiguousPutByVal(bytecode, badType, byValInfo);
    442478        break;
    443479    case JITArrayStorage:
    444         slowCases = emitArrayStoragePutByVal(bytecode, badType);
     480        slowCases = emitArrayStoragePutByVal(bytecode, badType, byValInfo);
    445481        break;
    446482    default:
    … …  
    449485    }
    450486   
    451     addSlowCase(badType);
     487    if (!JITCode::useDataIC(JITType::BaselineJIT))
     488        addSlowCase(badType);
    452489    addSlowCase(slowCases);
    453490   
    … …  
    458495
    459496template<typename Op>
    460 JIT::JumpList JIT::emitGenericContiguousPutByVal(Op bytecode, PatchableJump& badType, IndexingType indexingShape)
     497JIT::JumpList JIT::emitGenericContiguousPutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo, IndexingType indexingShape)
    461498{
    462499    auto& metadata = bytecode.metadata(m_codeBlock);
    … …  
    466503    JumpList slowCases;
    467504
    468     badType = patchableBranch32(NotEqual, regT2, TrustedImm32(indexingShape));
     505    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     506        if (byValInfo) {
     507            auto isCorrectType = branch32(Equal, regT2, TrustedImm32(indexingShape));
     508            farJump(AbsoluteAddress(&byValInfo->m_badTypeJumpTarget), JITStubRoutinePtrTag);
     509            isCorrectType.link(this);
     510        } else
     511            badType = patchableBranch32(NotEqual, regT2, TrustedImm32(indexingShape));
     512    } else
     513        badType = patchableBranch32(NotEqual, regT2, TrustedImm32(indexingShape));
     514
    469515   
    470516    loadPtr(Address(regT0, JSObject::butterflyOffset()), regT2);
    … …  
    516562
    517563template<typename Op>
    518 JIT::JumpList JIT::emitArrayStoragePutByVal(Op bytecode, PatchableJump& badType)
     564JIT::JumpList JIT::emitArrayStoragePutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo)
    519565{
    520566    auto& metadata = bytecode.metadata(m_codeBlock);
    … …  
    524570    JumpList slowCases;
    525571   
    526     badType = patchableBranch32(NotEqual, regT2, TrustedImm32(ArrayStorageShape));
     572    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     573        if (byValInfo) {
     574            auto isCorrectType = branch32(Equal, regT2, TrustedImm32(ArrayStorageShape));
     575            farJump(AbsoluteAddress(&byValInfo->m_badTypeJumpTarget), JITStubRoutinePtrTag);
     576            isCorrectType.link(this);
     577        } else
     578            badType = patchableBranch32(NotEqual, regT2, TrustedImm32(ArrayStorageShape));
     579    } else
     580        badType = patchableBranch32(NotEqual, regT2, TrustedImm32(ArrayStorageShape));
     581
    527582    loadPtr(Address(regT0, JSObject::butterflyOffset()), regT2);
    528583    slowCases.append(branch32(AboveOrEqual, regT1, Address(regT2, ArrayStorage::vectorLengthOffset())));
    … …  
    569624
    570625    JITPutByIdGenerator gen(
    571         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(), propertyName,
    572         JSValueRegs(regT0), JSValueRegs(regT1), regT2, ecmaMode(bytecode), putKind);
     626        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(), propertyName,
     627        JSValueRegs(regT0), JSValueRegs(regT1), regT3, regT2, ecmaMode(bytecode), putKind);
    573628    gen.generateFastPath(*this);
    574629    // IC can write new Structure without write-barrier if a base is cell.
    … …  
    581636    gen.slowPathJump().link(this);
    582637
    583     Call call = callOperation(gen.slowPathFunction(), TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT1, regT0, propertyName.rawBits());
     638    Call call;
     639    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     640        gen.stubInfo()->m_slowOperation = gen.slowPathFunction();
     641        move(TrustedImmPtr(gen.stubInfo()), GPRInfo::nonArgGPR0);
     642        callOperation<decltype(gen.slowPathFunction())>(Address(GPRInfo::nonArgGPR0, StructureStubInfo::offsetOfSlowOperation()), TrustedImmPtr(m_codeBlock->globalObject()), GPRInfo::nonArgGPR0, regT1, regT0, propertyName.rawBits());
     643    } else
     644        call = callOperation(gen.slowPathFunction(), TrustedImmPtr(m_codeBlock->globalObject()), gen.stubInfo(), regT1, regT0, propertyName.rawBits());
    584645    gen.reportSlowPathCall(coldPathBegin, call);
    585646    doneCases.append(jump());
    … …  
    639700    emitNakedNearCall(vm.getCTIStub(slow_op_put_by_val_prepareCallGenerator).retaggedCode<NoPtrTag>());
    640701
    641     auto* operation = isDirect ? operationDirectPutByValOptimize : operationPutByValOptimize;
    642     Call call = appendCall(operation);
     702    Call call;
     703    if (JITCode::useDataIC(JITType::BaselineJIT))
     704        byValInfo->m_slowOperation = isDirect ? operationDirectPutByValOptimize : operationPutByValOptimize;
     705    else
     706        call = appendCall(isDirect ? operationDirectPutByValOptimize : operationPutByValOptimize);
    643707    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    644708#endif // ENABLE(EXTRA_CTI_THUNKS)
    … …  
    658722    JIT jit(vm);
    659723
    660     jit.tagReturnAddress();
     724    if (!JITCode::useDataIC(JITType::BaselineJIT))
     725        jit.tagReturnAddress();
    661726
    662727    constexpr GPRReg bytecodeOffsetGPR = argumentGPR0;
    … …  
    676741    jit.prepareCallOperation(vm);
    677742
    678     // FIXME: Consider making this a tail call directly to the target operation instead
    679     // of returning to the client to call it.
    680     jit.ret();
     743    if (JITCode::useDataIC(JITType::BaselineJIT))
     744        jit.farJump(Address(argumentGPR4, ByValInfo::offsetOfSlowOperation()), OperationPtrTag);
     745    else
     746        jit.ret();
    681747
    682748    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    697763    emitJumpSlowCaseIfNotJSCell(regT0, base);
    698764
    699     PatchableJump fastPathJmp = patchableJump();
    700     addSlowCase(fastPathJmp);
     765    PatchableJump fastPathJmp;
     766    if (JITCode::useDataIC(JITType::BaselineJIT))
     767        farJump(AbsoluteAddress(&byValInfo->m_notIndexJumpTarget), JITStubRoutinePtrTag);
     768    else {
     769        fastPathJmp = patchableJump();
     770        addSlowCase(fastPathJmp);
     771    }
    701772   
    702773    Label done = label();
    … …  
    744815    emitNakedNearCall(vm.getCTIStub(slow_op_put_private_name_prepareCallGenerator).retaggedCode<NoPtrTag>());
    745816
    746     Call call = appendCall(operationPutPrivateNameOptimize);
     817    Call call;
     818    if (JITCode::useDataIC(JITType::BaselineJIT))
     819        byValInfo->m_slowOperation = operationPutPrivateNameOptimize;
     820    else
     821        call = appendCall(operationPutPrivateNameOptimize);
    747822    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    748823#endif // ENABLE(EXTRA_CTI_THUNKS)
    … …  
    762837    JIT jit(vm);
    763838
    764     jit.tagReturnAddress();
     839    if (!JITCode::useDataIC(JITType::BaselineJIT))
     840        jit.tagReturnAddress();
    765841
    766842    constexpr GPRReg bytecodeOffsetGPR = argumentGPR0;
    … …  
    780856    jit.prepareCallOperation(vm);
    781857
    782     // FIXME: Consider making this a tail call directly to the target operation instead
    783     // of returning to the client to call it.
    784     jit.ret();
     858    if (JITCode::useDataIC(JITType::BaselineJIT))
     859        jit.farJump(Address(argumentGPR4, ByValInfo::offsetOfSlowOperation()), OperationPtrTag);
     860    else
     861        jit.ret();
    785862
    786863    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    847924    emitJumpSlowCaseIfNotJSCell(regT1, base);
    848925    JITDelByIdGenerator gen(
    849         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     926        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    850927        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident),
    851         JSValueRegs(regT1), JSValueRegs(regT0), regT2);
     928        JSValueRegs(regT1), JSValueRegs(regT0), regT3, regT2);
    852929    gen.generateFastPath(*this);
    853     addSlowCase(gen.slowPathJump());
     930    if (!JITCode::useDataIC(JITType::BaselineJIT))
     931        addSlowCase(gen.slowPathJump());
     932    else
     933        addSlowCase();
    854934    m_delByIds.append(gen);
    855935
    … …  
    899979    emitNakedNearCall(vm.getCTIStub(slow_op_del_by_id_prepareCallGenerator).retaggedCode<NoPtrTag>());
    900980
    901     Call call = appendCall(operationDeleteByIdOptimize);
     981    Call call;
     982    if (JITCode::useDataIC(JITType::BaselineJIT))
     983        gen.stubInfo()->m_slowOperation = operationDeleteByIdOptimize;
     984    else
     985        call = appendCall(operationDeleteByIdOptimize);
    902986    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    903987    static_assert(returnValueGPR == regT0);
    904988#endif // ENABLE(EXTRA_CTI_THUNKS)
    905989
    906     gen.reportSlowPathCall(coldPathBegin, call);
    907 
    908990    boxBoolean(regT0, JSValueRegs(regT0));
    909991    emitPutVirtualRegister(dst, JSValueRegs(regT0));
     992    gen.reportSlowPathCall(coldPathBegin, call);
    910993}
    911994
    … …  
    9191002    JIT jit(vm);
    9201003
    921     jit.tagReturnAddress();
     1004    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1005        jit.tagReturnAddress();
    9221006
    9231007    constexpr GPRReg bytecodeOffsetGPR = argumentGPR0;
    … …  
    9361020    jit.prepareCallOperation(vm);
    9371021
    938     // FIXME: Consider making this a tail call directly to the target operation instead
    939     // of returning to the client to call it.
    940     jit.ret();
     1022    if (JITCode::useDataIC(JITType::BaselineJIT))
     1023        jit.farJump(Address(argumentGPR1, StructureStubInfo::offsetOfSlowOperation()), OperationPtrTag);
     1024    else
     1025        jit.ret();
    9411026
    9421027    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    9571042    emitJumpSlowCaseIfNotJSCell(regT0, property);
    9581043    JITDelByValGenerator gen(
    959         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    960         JSValueRegs(regT1), JSValueRegs(regT0), JSValueRegs(regT0), regT2);
     1044        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1045        JSValueRegs(regT1), JSValueRegs(regT0), JSValueRegs(regT0), regT3, regT2);
    9611046    gen.generateFastPath(*this);
    962     addSlowCase(gen.slowPathJump());
     1047    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1048        addSlowCase(gen.slowPathJump());
     1049    else
     1050        addSlowCase();
    9631051    m_delByVals.append(gen);
    9641052
    … …  
    10091097    emitNakedNearCall(vm.getCTIStub(slow_op_del_by_val_prepareCallGenerator).retaggedCode<NoPtrTag>());
    10101098
    1011     Call call = appendCall(operationDeleteByValOptimize);
     1099    Call call;
     1100    if (JITCode::useDataIC(JITType::BaselineJIT))
     1101        gen.stubInfo()->m_slowOperation = operationDeleteByValOptimize;
     1102    else
     1103        call = appendCall(operationDeleteByValOptimize);
    10121104    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    10131105    static_assert(returnValueGPR == regT0);
    10141106#endif // ENABLE(EXTRA_CTI_THUNKS)
    10151107
    1016     gen.reportSlowPathCall(coldPathBegin, call);
    1017 
    10181108    boxBoolean(regT0, JSValueRegs(regT0));
    10191109    emitPutVirtualRegister(dst, JSValueRegs(regT0));
     1110    gen.reportSlowPathCall(coldPathBegin, call);
    10201111}
    10211112
    … …  
    10291120    JIT jit(vm);
    10301121
    1031     jit.tagReturnAddress();
     1122    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1123        jit.tagReturnAddress();
    10321124
    10331125    constexpr GPRReg bytecodeOffsetGPR = argumentGPR0;
    … …  
    10461138    jit.prepareCallOperation(vm);
    10471139
    1048     // FIXME: Consider making this a tail call directly to the target operation instead
    1049     // of returning to the client to call it.
    1050     jit.ret();
     1140    if (JITCode::useDataIC(JITType::BaselineJIT))
     1141        jit.farJump(Address(argumentGPR1, StructureStubInfo::offsetOfSlowOperation()), OperationPtrTag);
     1142    else
     1143        jit.ret();
    10511144
    10521145    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    10691162
    10701163    JITGetByIdGenerator gen(
    1071         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    1072         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), resultRegs, AccessType::TryGetById);
     1164        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1165        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), resultRegs, regT1, AccessType::TryGetById);
    10731166    gen.generateFastPath(*this);
    10741167    addSlowCase(gen.slowPathJump());
    … …  
    11111204    emitNakedNearCall(vm.getCTIStub(slow_op_get_by_id_prepareCallGenerator).retaggedCode<NoPtrTag>());
    11121205
    1113     Call call = appendCall(operationTryGetByIdOptimize);
     1206    Call call;
     1207    if (JITCode::useDataIC(JITType::BaselineJIT))
     1208        gen.stubInfo()->m_slowOperation = operationTryGetByIdOptimize;
     1209    else
     1210        call = appendCall(operationTryGetByIdOptimize);
    11141211    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    11151212
    … …  
    11341231
    11351232    JITGetByIdGenerator gen(
    1136         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    1137         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), resultRegs, AccessType::GetByIdDirect);
     1233        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1234        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), resultRegs, regT1, AccessType::GetByIdDirect);
    11381235    gen.generateFastPath(*this);
    11391236    addSlowCase(gen.slowPathJump());
    … …  
    11771274    emitNakedNearCall(vm.getCTIStub(slow_op_get_by_id_prepareCallGenerator).retaggedCode<NoPtrTag>());
    11781275
    1179     Call call = appendCall(operationGetByIdDirectOptimize);
     1276    Call call;
     1277    if (JITCode::useDataIC(JITType::BaselineJIT))
     1278        gen.stubInfo()->m_slowOperation = operationGetByIdDirectOptimize;
     1279    else
     1280        call = appendCall(operationGetByIdDirectOptimize);
    11801281    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    11811282
    … …  
    12081309
    12091310    JITGetByIdGenerator gen(
    1210         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    1211         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), resultRegs, AccessType::GetById);
     1311        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1312        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), resultRegs, regT1, AccessType::GetById);
    12121313    gen.generateFastPath(*this);
    12131314    addSlowCase(gen.slowPathJump());
    … …  
    12341335
    12351336    JITGetByIdWithThisGenerator gen(
    1236         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    1237         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), resultRegs, JSValueRegs(regT0), JSValueRegs(regT1));
     1337        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1338        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), resultRegs, JSValueRegs(regT0), JSValueRegs(regT1), regT2);
    12381339    gen.generateFastPath(*this);
    12391340    addSlowCase(gen.slowPathJump());
    … …  
    12761377    emitNakedNearCall(vm.getCTIStub(slow_op_get_by_id_prepareCallGenerator).retaggedCode<NoPtrTag>());
    12771378
    1278     Call call = appendCall(operationGetByIdOptimize);
     1379    Call call;
     1380    if (JITCode::useDataIC(JITType::BaselineJIT))
     1381        gen.stubInfo()->m_slowOperation = operationGetByIdOptimize;
     1382    else
     1383        call = appendCall(operationGetByIdOptimize);
    12791384    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    12801385
    … …  
    12951400    JIT jit(vm);
    12961401
    1297     jit.tagReturnAddress();
     1402    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1403        jit.tagReturnAddress();
    12981404
    12991405    constexpr GPRReg bytecodeOffsetGPR = argumentGPR2;
    … …  
    13121418    jit.prepareCallOperation(vm);
    13131419
    1314     // FIXME: Consider making this a tail call directly to the target operation instead
    1315     // of returning to the client to call it.
    1316     jit.ret();
     1420    if (JITCode::useDataIC(JITType::BaselineJIT))
     1421        jit.farJump(Address(argumentGPR1, StructureStubInfo::offsetOfSlowOperation()), OperationPtrTag);
     1422    else
     1423        jit.ret();
    13171424
    13181425    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    13551462    emitNakedNearCall(vm.getCTIStub(slow_op_get_by_id_with_this_prepareCallGenerator).retaggedCode<NoPtrTag>());
    13561463
    1357     Call call = appendCall(operationGetByIdWithThisOptimize);
     1464    Call call;
     1465    if (JITCode::useDataIC(JITType::BaselineJIT))
     1466        gen.stubInfo()->m_slowOperation = operationGetByIdWithThisOptimize;
     1467    else
     1468        call = appendCall(operationGetByIdWithThisOptimize);
    13581469    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    13591470
    … …  
    13741485    JIT jit(vm);
    13751486
    1376     jit.tagReturnAddress();
     1487    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1488        jit.tagReturnAddress();
    13771489
    13781490    constexpr GPRReg bytecodeOffsetGPR = argumentGPR3;
    … …  
    13931505    jit.prepareCallOperation(vm);
    13941506
    1395     // FIXME: Consider making this a tail call directly to the target operation instead
    1396     // of returning to the client to call it.
    1397     jit.ret();
     1507    if (JITCode::useDataIC(JITType::BaselineJIT))
     1508        jit.farJump(Address(argumentGPR1, StructureStubInfo::offsetOfSlowOperation()), OperationPtrTag);
     1509    else
     1510        jit.ret();
    13981511
    13991512    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    14191532
    14201533    JITPutByIdGenerator gen(
    1421         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1534        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    14221535        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident),
    1423         JSValueRegs(regT0), JSValueRegs(regT1), regT2, ecmaMode(bytecode),
     1536        JSValueRegs(regT0), JSValueRegs(regT1), regT3, regT2, ecmaMode(bytecode),
    14241537        direct ? PutKind::Direct : PutKind::NotDirect);
    1425    
    14261538    gen.generateFastPath(*this);
    14271539    addSlowCase(gen.slowPathJump());
    … …  
    14661578    emitNakedNearCall(vm.getCTIStub(slow_op_put_by_id_prepareCallGenerator).retaggedCode<NoPtrTag>());
    14671579
    1468     Call call = appendCall(gen.slowPathFunction());
     1580    Call call;
     1581    if (JITCode::useDataIC(JITType::BaselineJIT))
     1582        gen.stubInfo()->m_slowOperation = gen.slowPathFunction();
     1583    else
     1584        call = appendCall(gen.slowPathFunction());
    14691585    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    14701586#endif // ENABLE(EXTRA_CTI_THUNKS)
    … …  
    14821598    JIT jit(vm);
    14831599
    1484     jit.tagReturnAddress();
     1600    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1601        jit.tagReturnAddress();
    14851602
    14861603    constexpr GPRReg bytecodeOffsetGPR = argumentGPR2;
    … …  
    15011618    jit.prepareCallOperation(vm);
    15021619
    1503     // FIXME: Consider making this a tail call directly to the target operation instead
    1504     // of returning to the client to call it.
    1505     jit.ret();
     1620    if (JITCode::useDataIC(JITType::BaselineJIT))
     1621        jit.farJump(Address(argumentGPR1, StructureStubInfo::offsetOfSlowOperation()), OperationPtrTag);
     1622    else
     1623        jit.ret();
    15061624
    15071625    LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::ExtraCTIThunk);
    … …  
    15221640
    15231641    JITInByIdGenerator gen(
    1524         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    1525         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), JSValueRegs(regT0));
     1642        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1643        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs(regT0), JSValueRegs(regT0), regT1);
    15261644    gen.generateFastPath(*this);
    15271645    addSlowCase(gen.slowPathJump());
    … …  
    15651683    emitNakedNearCall(vm.getCTIStub(slow_op_get_by_id_prepareCallGenerator).retaggedCode<NoPtrTag>());
    15661684
    1567     Call call = appendCall(operationInByIdOptimize);
     1685    Call call;
     1686    if (JITCode::useDataIC(JITType::BaselineJIT))
     1687        gen.stubInfo()->m_slowOperation = operationInByIdOptimize;
     1688    else
     1689        call = appendCall(operationInByIdOptimize);
    15681690    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    15691691
    … …  
    15891711
    15901712    JITInByValGenerator gen(
    1591         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    1592         JSValueRegs(regT0), JSValueRegs(regT1), JSValueRegs(regT0));
     1713        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     1714        JSValueRegs(regT0), JSValueRegs(regT1), JSValueRegs(regT0), regT2);
    15931715    gen.generateFastPath(*this);
    1594     addSlowCase(gen.slowPathJump());
     1716    if (!JITCode::useDataIC(JITType::BaselineJIT))
     1717        addSlowCase(gen.slowPathJump());
     1718    else
     1719        addSlowCase();
    15951720    m_inByVals.append(gen);
    15961721
    … …  
    16351760    emitNakedNearCall(vm.getCTIStub(slow_op_get_by_val_prepareCallGenerator).retaggedCode<NoPtrTag>());
    16361761
    1637     Call call = appendCall(operationInByValOptimize);
     1762    Call call;
     1763    if (JITCode::useDataIC(JITType::BaselineJIT))
     1764        gen.stubInfo()->m_slowOperation = operationInByValOptimize;
     1765    else
     1766        call = appendCall(operationInByValOptimize);
    16381767    emitNakedNearCall(vm.getCTIStub(checkExceptionGenerator).retaggedCode<NoPtrTag>());
    16391768
    … …  
    28252954
    28262955    JITPutByIdGenerator gen(
    2827         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(), propertyName,
    2828         JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2), regT1, ecmaMode(bytecode), putKind);
     2956        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(), propertyName,
     2957        JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2), InvalidGPRReg, regT1, ecmaMode(bytecode), putKind);
    28292958    gen.generateFastPath(*this);
    28302959    doneCases.append(jump());
    … …  
    28362965    emitLoadTag(base, regT1);
    28372966
    2838     Call call = callOperation(gen.slowPathFunction(), m_codeBlock->globalObject(), gen.stubInfo(), JSValueRegs(regT3, regT2), JSValueRegs(regT1, regT0), propertyName.rawBits());
     2967    Call call;
     2968    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     2969        gen.stubInfo()->m_slowOperation = gen.slowPathFunction();
     2970        move(TrustedImmPtr(gen.stubInfo()), GPRInfo::nonArgGPR0);
     2971        callOperation<decltype(gen.slowPathFunction())>(Address(GPRInfo::nonArgGPR0, StructureStubInfo::offsetOfSlowOperation()), m_codeBlock->globalObject(), GPRInfo::nonArgGPR0, JSValueRegs(regT3, regT2), JSValueRegs(regT1, regT0), propertyName.rawBits());
     2972    } else
     2973        call = callOperation(gen.slowPathFunction(), m_codeBlock->globalObject(), gen.stubInfo(), JSValueRegs(regT3, regT2), JSValueRegs(regT1, regT0), propertyName.rawBits());
    28392974    gen.reportSlowPathCall(coldPathBegin, call);
    28402975    doneCases.append(jump());
    … …  
    28883023    switch (arrayMode) {
    28893024    case JITInt32:
    2890         slowCases = emitInt32PutByVal(bytecode, badType);
     3025        slowCases = emitInt32PutByVal(bytecode, badType, nullptr);
    28913026        break;
    28923027    case JITDouble:
    2893         slowCases = emitDoublePutByVal(bytecode, badType);
     3028        slowCases = emitDoublePutByVal(bytecode, badType, nullptr);
    28943029        break;
    28953030    case JITContiguous:
    2896         slowCases = emitContiguousPutByVal(bytecode, badType);
     3031        slowCases = emitContiguousPutByVal(bytecode, badType, nullptr);
    28973032        needsLinkForWriteBarrier = true;
    28983033        break;
    28993034    case JITArrayStorage:
    2900         slowCases = emitArrayStoragePutByVal(bytecode, badType);
     3035        slowCases = emitArrayStoragePutByVal(bytecode, badType, nullptr);
    29013036        needsLinkForWriteBarrier = true;
    29023037        break;
    … …  
    29043039        TypedArrayType type = typedArrayTypeForJITArrayMode(arrayMode);
    29053040        if (isInt(type))
    2906             slowCases = emitIntTypedArrayPutByVal(bytecode, badType, type);
     3041            slowCases = emitIntTypedArrayPutByVal(bytecode, badType, nullptr, type);
    29073042        else {
    29083043            // FIXME: Optimize BigInt64Array / BigUint64Array in IC
    … …  
    29103045            // https://bugs.webkit.org/show_bug.cgi?id=221183
    29113046            ASSERT(isFloat(type));
    2912             slowCases = emitFloatTypedArrayPutByVal(bytecode, badType, type);
     3047            slowCases = emitFloatTypedArrayPutByVal(bytecode, badType, nullptr, type);
    29133048        }
    29143049        break;
    … …  
    29373072            "Baseline put_by_val_direct stub for %s, return point %p", toCString(*m_codeBlock).data(), returnAddress.untaggedValue());
    29383073    }
    2939     MacroAssembler::repatchJump(byValInfo->badTypeJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
    2940     MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(isDirect ? operationDirectPutByValGeneric : operationPutByValGeneric));
     3074
     3075    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3076        byValInfo->m_badTypeJumpTarget = CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code());
     3077        byValInfo->m_slowOperation = isDirect ? operationDirectPutByValGeneric : operationPutByValGeneric;
     3078    } else {
     3079        MacroAssembler::repatchJump(byValInfo->m_badTypeJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
     3080        MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(isDirect ? operationDirectPutByValGeneric : operationPutByValGeneric));
     3081    }
    29413082}
    29423083// This function is only consumed from another translation unit (JITOperations.cpp),
    … …  
    29773118    byValInfo->stubInfo = gen.stubInfo();
    29783119
    2979     MacroAssembler::repatchJump(byValInfo->notIndexJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
    2980     MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(operationPutPrivateNameGeneric));
     3120    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3121        byValInfo->m_notIndexJumpTarget = CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code());
     3122        byValInfo->m_slowOperation = operationPutPrivateNameGeneric;
     3123    } else {
     3124        MacroAssembler::repatchJump(byValInfo->m_notIndexJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
     3125        MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(operationPutPrivateNameGeneric));
     3126    }
    29813127}
    29823128
    … …  
    30153161    byValInfo->stubInfo = gen.stubInfo();
    30163162
    3017     MacroAssembler::repatchJump(byValInfo->notIndexJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
    3018     MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(putKind == PutKind::Direct ? operationDirectPutByValGeneric : operationPutByValGeneric));
     3163    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3164        byValInfo->m_notIndexJumpTarget = CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code());
     3165        byValInfo->m_slowOperation = putKind == PutKind::Direct ? operationDirectPutByValGeneric : operationPutByValGeneric;
     3166    } else {
     3167        MacroAssembler::repatchJump(byValInfo->m_notIndexJump, CodeLocationLabel<JITStubRoutinePtrTag>(byValInfo->stubRoutine->code().code()));
     3168        MacroAssembler::repatchCall(CodeLocationCall<ReturnAddressPtrTag>(MacroAssemblerCodePtr<ReturnAddressPtrTag>(returnAddress)), FunctionPtr<OperationPtrTag>(putKind == PutKind::Direct ? operationDirectPutByValGeneric : operationPutByValGeneric));
     3169    }
    30193170}
    30203171// This function is only consumed from another translation unit (JITOperations.cpp),
    … …  
    30233174template void JIT::privateCompilePutByValWithCachedId<OpPutByValDirect>(ByValInfo*, ReturnAddressPtr, PutKind, CacheableIdentifier);
    30243175
    3025 JIT::JumpList JIT::emitDoubleLoad(const Instruction*, PatchableJump& badType)
     3176JIT::JumpList JIT::emitDoubleLoad(const Instruction*, PatchableJump& badType, ByValInfo* byValInfo)
    30263177{
    30273178#if USE(JSVALUE64)
    … …  
    30393190    JumpList slowCases;
    30403191
    3041     badType = patchableBranch32(NotEqual, indexing, TrustedImm32(DoubleShape));
     3192    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3193        if (byValInfo) {
     3194            auto isCorrectType = branch32(Equal, indexing, TrustedImm32(DoubleShape));
     3195            farJump(AbsoluteAddress(&byValInfo->m_badTypeJumpTarget), JITStubRoutinePtrTag);
     3196            isCorrectType.link(this);
     3197        } else
     3198            badType = patchableBranch32(NotEqual, indexing, TrustedImm32(DoubleShape));
     3199    } else
     3200        badType = patchableBranch32(NotEqual, indexing, TrustedImm32(DoubleShape));
     3201
    30423202    loadPtr(Address(base, JSObject::butterflyOffset()), scratch);
    30433203    slowCases.append(branch32(AboveOrEqual, property, Address(scratch, Butterfly::offsetOfPublicLength())));
    … …  
    30483208}
    30493209
    3050 JIT::JumpList JIT::emitContiguousLoad(const Instruction*, PatchableJump& badType, IndexingType expectedShape)
     3210JIT::JumpList JIT::emitContiguousLoad(const Instruction*, PatchableJump& badType, ByValInfo* byValInfo, IndexingType expectedShape)
    30513211{
    30523212#if USE(JSVALUE64)
    … …  
    30663226    JumpList slowCases;
    30673227
    3068     badType = patchableBranch32(NotEqual, indexing, TrustedImm32(expectedShape));
     3228    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3229        if (byValInfo) {
     3230            auto isCorrectType = branch32(Equal, indexing, TrustedImm32(expectedShape));
     3231            farJump(AbsoluteAddress(&byValInfo->m_badTypeJumpTarget), JITStubRoutinePtrTag);
     3232            isCorrectType.link(this);
     3233        } else
     3234            badType = patchableBranch32(NotEqual, indexing, TrustedImm32(expectedShape));
     3235    } else
     3236        badType = patchableBranch32(NotEqual, indexing, TrustedImm32(expectedShape));
     3237
    30693238    loadPtr(Address(base, JSObject::butterflyOffset()), scratch);
    30703239    slowCases.append(branch32(AboveOrEqual, property, Address(scratch, Butterfly::offsetOfPublicLength())));
    … …  
    30753244}
    30763245
    3077 JIT::JumpList JIT::emitArrayStorageLoad(const Instruction*, PatchableJump& badType)
     3246JIT::JumpList JIT::emitArrayStorageLoad(const Instruction*, PatchableJump& badType, ByValInfo* byValInfo)
    30783247{
    30793248#if USE(JSVALUE64)
    … …  
    30943263
    30953264    add32(TrustedImm32(-ArrayStorageShape), indexing, scratch);
    3096     badType = patchableBranch32(Above, scratch, TrustedImm32(SlowPutArrayStorageShape - ArrayStorageShape));
     3265
     3266    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3267        if (byValInfo) {
     3268            auto isCorrectType = branch32(BelowOrEqual, scratch, TrustedImm32(SlowPutArrayStorageShape - ArrayStorageShape));
     3269            farJump(AbsoluteAddress(&byValInfo->m_badTypeJumpTarget), JITStubRoutinePtrTag);
     3270            isCorrectType.link(this);
     3271        } else
     3272            badType = patchableBranch32(Above, scratch, TrustedImm32(SlowPutArrayStorageShape - ArrayStorageShape));
     3273    } else
     3274        badType = patchableBranch32(Above, scratch, TrustedImm32(SlowPutArrayStorageShape - ArrayStorageShape));
    30973275
    30983276    loadPtr(Address(base, JSObject::butterflyOffset()), scratch);
    … …  
    31063284
    31073285template<typename Op>
    3108 JIT::JumpList JIT::emitIntTypedArrayPutByVal(Op bytecode, PatchableJump& badType, TypedArrayType type)
     3286JIT::JumpList JIT::emitIntTypedArrayPutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo, TypedArrayType type)
    31093287{
    31103288    auto& metadata = bytecode.metadata(m_codeBlock);
    … …  
    31313309   
    31323310    load8(Address(base, JSCell::typeInfoTypeOffset()), earlyScratch);
    3133     badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3311
     3312    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3313        if (byValInfo) {
     3314            auto isCorrectType = branch32(Equal, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3315            farJump(AbsoluteAddress(&byValInfo->m_badTypeJumpTarget), JITStubRoutinePtrTag);
     3316            isCorrectType.link(this);
     3317        } else
     3318            badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3319    } else
     3320        badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3321
    31343322    load32(Address(base, JSArrayBufferView::offsetOfLength()), lateScratch2);
    31353323    Jump inBounds = branch32(Below, property, lateScratch2);
    … …  
    31823370
    31833371template<typename Op>
    3184 JIT::JumpList JIT::emitFloatTypedArrayPutByVal(Op bytecode, PatchableJump& badType, TypedArrayType type)
     3372JIT::JumpList JIT::emitFloatTypedArrayPutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo, TypedArrayType type)
    31853373{
    31863374    auto& metadata = bytecode.metadata(m_codeBlock);
    … …  
    32073395   
    32083396    load8(Address(base, JSCell::typeInfoTypeOffset()), earlyScratch);
    3209     badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3397
     3398    if (JITCode::useDataIC(JITType::BaselineJIT)) {
     3399        if (byValInfo) {
     3400            auto isCorrectType = branch32(Equal, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3401            farJump(AbsoluteAddress(&byValInfo->m_badTypeJumpTarget), JITStubRoutinePtrTag);
     3402            isCorrectType.link(this);
     3403        } else
     3404            badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3405    } else
     3406        badType = patchableBranch32(NotEqual, earlyScratch, TrustedImm32(typeForTypedArrayType(type)));
     3407
    32103408    load32(Address(base, JSArrayBufferView::offsetOfLength()), lateScratch2);
    32113409    Jump inBounds = branch32(Below, property, lateScratch2);

  • trunk/Source/JavaScriptCore/jit/JITPropertyAccess32_64.cpp

    r278591 r278656  
    132132    emitJumpSlowCaseIfNotJSCell(base, baseRegs.tagGPR());
    133133    JITDelByIdGenerator gen(
    134         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     134        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    135135        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident),
    136         baseRegs, resultRegs, regT4);
     136        baseRegs, resultRegs, InvalidGPRReg, regT4);
    137137    gen.generateFastPath(*this);
    138138    addSlowCase(gen.slowPathJump());
    … …  
    167167
    168168    JITDelByValGenerator gen(
    169         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    170         baseRegs, propertyRegs, resultRegs, regT4);
     169        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     170        baseRegs, propertyRegs, resultRegs, InvalidGPRReg, regT4);
    171171
    172172    gen.generateFastPath(*this);
    … …  
    258258
    259259        JITGetByValGenerator gen(
    260             m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetByVal, RegisterSet::stubUnavailableRegisters(),
    261             JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2), resultRegs);
     260            m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetByVal, RegisterSet::stubUnavailableRegisters(),
     261            JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2), resultRegs, InvalidGPRReg);
    262262        if (isOperandConstantInt(property))
    263263            gen.stubInfo()->propertyIsInt32 = true;
    … …  
    306306
    307307    JITGetByValGenerator gen(
    308         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetPrivateName,
    309         RegisterSet::stubUnavailableRegisters(), baseGPR, propertyGPR, resultRegs);
     308        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::GetPrivateName,
     309        RegisterSet::stubUnavailableRegisters(), baseGPR, propertyGPR, resultRegs, InvalidGPRReg);
    310310    gen.generateFastPath(*this);
    311311    addSlowCase(gen.slowPathJump());
    … …  
    394394
    395395    JITPrivateBrandAccessGenerator gen(
    396         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::SetPrivateBrand, RegisterSet::stubUnavailableRegisters(),
    397         baseRegs, brandRegs);
     396        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::SetPrivateBrand, RegisterSet::stubUnavailableRegisters(),
     397        baseRegs, brandRegs, InvalidGPRReg);
    398398    gen.generateFastPath(*this);
    399399    addSlowCase(gen.slowPathJump());
    … …  
    434434
    435435    JITPrivateBrandAccessGenerator gen(
    436         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::CheckPrivateBrand, RegisterSet::stubUnavailableRegisters(),
    437         baseRegs, brandRegs);
     436        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), AccessType::CheckPrivateBrand, RegisterSet::stubUnavailableRegisters(),
     437        baseRegs, brandRegs, InvalidGPRReg);
    438438    gen.generateFastPath(*this);
    439439    addSlowCase(gen.slowPathJump());
    … …  
    487487    switch (mode) {
    488488    case JITInt32:
    489         slowCases = emitInt32PutByVal(bytecode, badType);
     489        slowCases = emitInt32PutByVal(bytecode, badType, byValInfo);
    490490        break;
    491491    case JITDouble:
    492         slowCases = emitDoublePutByVal(bytecode, badType);
     492        slowCases = emitDoublePutByVal(bytecode, badType, byValInfo);
    493493        break;
    494494    case JITContiguous:
    495         slowCases = emitContiguousPutByVal(bytecode, badType);
     495        slowCases = emitContiguousPutByVal(bytecode, badType, byValInfo);
    496496        break;
    497497    case JITArrayStorage:
    498         slowCases = emitArrayStoragePutByVal(bytecode, badType);
     498        slowCases = emitArrayStoragePutByVal(bytecode, badType, byValInfo);
    499499        break;
    500500    default:
    … …  
    512512
    513513template <typename Op>
    514 JIT::JumpList JIT::emitGenericContiguousPutByVal(Op bytecode, PatchableJump& badType, IndexingType indexingShape)
     514JIT::JumpList JIT::emitGenericContiguousPutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo, IndexingType indexingShape)
    515515{
    516516    auto& metadata = bytecode.metadata(m_codeBlock);
    … …  
    521521    JumpList slowCases;
    522522   
     523    UNUSED_PARAM(byValInfo);
    523524    badType = patchableBranch32(NotEqual, regT1, TrustedImm32(ContiguousShape));
    524525   
    … …  
    573574
    574575template <typename Op>
    575 JIT::JumpList JIT::emitArrayStoragePutByVal(Op bytecode, PatchableJump& badType)
     576JIT::JumpList JIT::emitArrayStoragePutByVal(Op bytecode, PatchableJump& badType, ByValInfo* byValInfo)
    576577{
    577578    auto& metadata = bytecode.metadata(m_codeBlock);
    … …  
    582583    JumpList slowCases;
    583584   
     585    UNUSED_PARAM(byValInfo);
    584586    badType = patchableBranch32(NotEqual, regT1, TrustedImm32(ArrayStorageShape));
    585587   
    … …  
    661663
    662664    JITGetByIdGenerator gen(
    663         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    664         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), resultRegs, AccessType::TryGetById);
     665        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     666        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), resultRegs, InvalidGPRReg, AccessType::TryGetById);
    665667    gen.generateFastPath(*this);
    666668    addSlowCase(gen.slowPathJump());
    … …  
    702704
    703705    JITGetByIdGenerator gen(
    704         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    705         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), resultRegs, AccessType::GetByIdDirect);
     706        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     707        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), resultRegs, InvalidGPRReg, AccessType::GetByIdDirect);
    706708    gen.generateFastPath(*this);
    707709    addSlowCase(gen.slowPathJump());
    … …  
    749751    JSValueRegs resultRegs = JSValueRegs(regT1, regT0);
    750752    JITGetByIdGenerator gen(
    751         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    752         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), resultRegs, AccessType::GetById);
     753        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     754        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), resultRegs, InvalidGPRReg, AccessType::GetById);
    753755    gen.generateFastPath(*this);
    754756    addSlowCase(gen.slowPathJump());
    … …  
    792794
    793795    JITGetByIdWithThisGenerator gen(
    794         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    795         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), resultRegs, JSValueRegs::payloadOnly(regT0), JSValueRegs(regT4, regT3));
     796        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     797        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), resultRegs, JSValueRegs::payloadOnly(regT0), JSValueRegs(regT4, regT3), InvalidGPRReg);
    796798    gen.generateFastPath(*this);
    797799    addSlowCase(gen.slowPathJump());
    … …  
    836838
    837839    JITPutByIdGenerator gen(
    838         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     840        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    839841        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident),
    840         JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2),
     842        JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2), InvalidGPRReg,
    841843        regT1, bytecode.m_flags.ecmaMode(), direct ? PutKind::Direct : PutKind::NotDirect);
    842844   
    … …  
    883885
    884886    JITInByIdGenerator gen(
    885         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    886         CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), JSValueRegs(regT1, regT0));
     887        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     888        CacheableIdentifier::createFromIdentifierOwnedByCodeBlock(m_codeBlock, *ident), JSValueRegs::payloadOnly(regT0), JSValueRegs(regT1, regT0), InvalidGPRReg);
    887889    gen.generateFastPath(*this);
    888890    addSlowCase(gen.slowPathJump());
    … …  
    923925
    924926    JITInByValGenerator gen(
    925         m_codeBlock, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
    926         JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2), JSValueRegs(regT1, regT0));
     927        m_codeBlock, JITType::BaselineJIT, CodeOrigin(m_bytecodeIndex), CallSiteIndex(m_bytecodeIndex), RegisterSet::stubUnavailableRegisters(),
     928        JSValueRegs::payloadOnly(regT0), JSValueRegs(regT3, regT2), JSValueRegs(regT1, regT0), InvalidGPRReg);
    927929    gen.generateFastPath(*this);
    928930    addSlowCase(gen.slowPathJump());

  • trunk/Source/JavaScriptCore/jit/JITStubRoutine.h

    r277027 r278656  
    3535class JITStubRoutineSet;
    3636class VM;
     37
     38#if USE(JSVALUE64)
     39using StructureID = uint32_t;
     40#else
     41using StructureID = Structure*;
     42#endif
     43
     44class AccessCase;
    3745
    3846// This is a base-class for JIT stub routines, and also the class you want
    … …  
    104112    MacroAssemblerCodeRef<JITStubRoutinePtrTag> m_code;
    105113    unsigned m_refCount;
     114    mutable unsigned m_hash { 0 };
    106115};
    107116

  • trunk/Source/JavaScriptCore/jit/PolymorphicCallStubRoutine.cpp

    r277680 r278656  
    2929#if ENABLE(JIT)
    3030
     31#include "AccessCase.h"
    3132#include "CallLinkInfo.h"
    3233#include "CodeBlock.h"
    … …  
    6869    CallLinkInfo& info, const Vector<PolymorphicCallCase>& cases,
    6970    UniqueArray<uint32_t>&& fastCounts)
    70     : GCAwareJITStubRoutine(codeRef, vm)
     71    : GCAwareJITStubRoutine(codeRef)
    7172    , m_variants(cases.size())
    7273    , m_fastCounts(WTFMove(fastCounts))
    … …  
    8182    }
    8283    WTF::storeStoreFence();
     84    makeGCAware(vm);
    8385}
    8486

  • trunk/Source/JavaScriptCore/jit/Repatch.cpp

    r278445 r278656  
    7272namespace JSC {
    7373
    74 static FunctionPtr<CFunctionPtrTag> readPutICCallTarget(CodeBlock* codeBlock, CodeLocationCall<JSInternalPtrTag> call)
    75 {
     74static FunctionPtr<CFunctionPtrTag> readPutICCallTarget(CodeBlock* codeBlock, StructureStubInfo& stubInfo)
     75{
     76    if (codeBlock->useDataIC())
     77        return stubInfo.m_slowOperation.retagged<CFunctionPtrTag>();
     78    CodeLocationCall<JSInternalPtrTag> call = stubInfo.m_slowPathCallLocation;
    7679#if ENABLE(FTL_JIT)
    7780    if (codeBlock->jitType() == JITType::FTLJIT) {
    … …  
    104107#endif // ENABLE(FTL_JIT)
    105108    MacroAssembler::repatchCall(call, newCalleeFunction.retagged<OperationPtrTag>());
     109}
     110
     111static void repatchSlowPathCall(CodeBlock* codeBlock, StructureStubInfo& stubInfo, FunctionPtr<CFunctionPtrTag> newCalleeFunction)
     112{
     113    if (codeBlock->useDataIC()) {
     114        stubInfo.m_slowOperation = newCalleeFunction.retagged<OperationPtrTag>();
     115        return;
     116    }
     117    ftlThunkAwareRepatchCall(codeBlock, stubInfo.m_slowPathCallLocation, newCalleeFunction);
    106118}
    107119
    … …  
    210222        const bool isPrivate = kind == GetByKind::PrivateName || kind == GetByKind::PrivateNameById;
    211223
    212         std::unique_ptr<AccessCase> newCase;
     224        RefPtr<AccessCase> newCase;
    213225
    214226        if (propertyName == vm.propertyNames->length) {
    … …  
    220232                    bool generatedCodeInline = InlineAccess::generateArrayLength(stubInfo, jsCast<JSArray*>(baseCell));
    221233                    if (generatedCodeInline) {
    222                         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, appropriateOptimizingGetByFunction(kind));
     234                        repatchSlowPathCall(codeBlock, stubInfo, appropriateOptimizingGetByFunction(kind));
    223235                        stubInfo.initArrayLength(locker);
    224236                        return RetryCacheLater;
    … …  
    228240                newCase = AccessCase::create(vm, codeBlock, AccessCase::ArrayLength, propertyName);
    229241            } else if (isJSString(baseCell)) {
    230                 if (stubInfo.cacheType() == CacheType::Unset && InlineAccess::isCacheableStringLength(stubInfo)) {
     242                if (stubInfo.cacheType() == CacheType::Unset
     243                    && InlineAccess::isCacheableStringLength(stubInfo)) {
    231244                    bool generatedCodeInline = InlineAccess::generateStringLength(stubInfo);
    232245                    if (generatedCodeInline) {
    233                         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, appropriateOptimizingGetByFunction(kind));
     246                        repatchSlowPathCall(codeBlock, stubInfo, appropriateOptimizingGetByFunction(kind));
    234247                        stubInfo.initStringLength(locker);
    235248                        return RetryCacheLater;
    … …  
    283296                && !structure->needImpurePropertyWatchpoint()
    284297                && !loadTargetFromProxy) {
    285 
    286298                bool generatedCodeInline = InlineAccess::generateSelfPropertyAccess(stubInfo, structure, slot.cachedOffset());
    287299                if (generatedCodeInline) {
    288300                    LOG_IC((ICEvent::GetBySelfPatch, structure->classInfo(), Identifier::fromUid(vm, propertyName.uid()), slot.slotBase() == baseValue));
    289301                    structure->startWatchingPropertyForReplacements(vm, slot.cachedOffset());
    290                     ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, appropriateOptimizingGetByFunction(kind));
     302                    repatchSlowPathCall(codeBlock, stubInfo, appropriateOptimizingGetByFunction(kind));
    291303                    stubInfo.initGetByIdSelf(locker, codeBlock, structure, slot.cachedOffset(), propertyName);
    292304                    return RetryCacheLater;
    … …  
    426438           
    427439            RELEASE_ASSERT(result.code());
    428             InlineAccess::rewireStubAsJump(stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     440            switch (kind) {
     441            case GetByKind::Normal:
     442            case GetByKind::WithThis:
     443            case GetByKind::Try:
     444            case GetByKind::Direct:
     445            case GetByKind::PrivateNameById:
     446                InlineAccess::rewireStubAsJumpInAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     447                break;
     448            case GetByKind::NormalByVal:
     449            case GetByKind::PrivateName:
     450                InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     451                break;
     452            }
     453
    429454        }
    430455    }
    … …  
    440465   
    441466    if (tryCacheGetBy(globalObject, codeBlock, baseValue, propertyName, slot, stubInfo, kind) == GiveUpOnCache)
    442         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, appropriateGetByFunction(kind));
     467        repatchSlowPathCall(codeBlock, stubInfo, appropriateGetByFunction(kind));
    443468}
    444469
    … …  
    530555           
    531556            RELEASE_ASSERT(result.code());
    532             InlineAccess::rewireStubAsJump(stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     557            InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
    533558        }
    534559    }
    … …  
    541566{
    542567    if (tryCacheArrayGetByVal(globalObject, codeBlock, base, index, stubInfo) == GiveUpOnCache)
    543         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationGetByValGeneric);
     568        repatchSlowPathCall(codeBlock, stubInfo, operationGetByValGeneric);
    544569}
    545570
    … …  
    637662            return GiveUpOnCache;
    638663
    639         std::unique_ptr<AccessCase> newCase;
     664        RefPtr<AccessCase> newCase;
    640665
    641666        if (slot.base() == baseValue && slot.isCacheablePut()) {
    … …  
    659684                    if (generatedCodeInline) {
    660685                        LOG_IC((ICEvent::PutByIdSelfPatch, oldStructure->classInfo(), ident, slot.base() == baseValue));
    661                         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, appropriateOptimizingPutByIdFunction(slot, putKind));
     686                        repatchSlowPathCall(codeBlock, stubInfo, appropriateOptimizingPutByIdFunction(slot, putKind));
    662687                        stubInfo.initPutByIdReplace(locker, codeBlock, oldStructure, slot.cachedOffset(), propertyName);
    663688                        return RetryCacheLater;
    … …  
    802827            RELEASE_ASSERT(result.code());
    803828
    804             InlineAccess::rewireStubAsJump(stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     829            InlineAccess::rewireStubAsJumpInAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
    805830        }
    806831    }
    … …  
    816841   
    817842    if (tryCachePutByID(globalObject, codeBlock, baseValue, oldStructure, propertyName, slot, stubInfo, putKind) == GiveUpOnCache)
    818         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, appropriateGenericPutByIdFunction(slot, putKind));
     843        repatchSlowPathCall(codeBlock, stubInfo, appropriateGenericPutByIdFunction(slot, putKind));
    819844}
    820845
    … …  
    847872            return RetryCacheLater;
    848873
    849         std::unique_ptr<AccessCase> newCase;
     874        RefPtr<AccessCase> newCase;
    850875
    851876        if (slot.isDeleteHit()) {
    … …  
    875900            RELEASE_ASSERT(result.code());
    876901            LOG_IC((ICEvent::DelByReplaceWithJump, oldStructure->classInfo(), Identifier::fromUid(vm, propertyName.uid())));
    877             InlineAccess::rewireStubAsJump(stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     902            InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
    878903        }
    879904    }
    … …  
    892917        LOG_IC((ICEvent::DelByReplaceWithGeneric, baseValue.classInfoOrNull(globalObject->vm()), Identifier::fromUid(vm, propertyName.uid())));
    893918        if (kind == DelByKind::Normal)
    894             ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationDeleteByIdGeneric);
     919            repatchSlowPathCall(codeBlock, stubInfo, operationDeleteByIdGeneric);
    895920        else
    896             ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationDeleteByValGeneric);
     921            repatchSlowPathCall(codeBlock, stubInfo, operationDeleteByValGeneric);
    897922    }
    898923}
    … …  
    900925static InlineCacheAction tryCacheInBy(
    901926    JSGlobalObject* globalObject, CodeBlock* codeBlock, JSObject* base, CacheableIdentifier propertyName,
    902     bool wasFound, const PropertySlot& slot, StructureStubInfo& stubInfo)
     927    bool wasFound, const PropertySlot& slot, StructureStubInfo& stubInfo, InByKind kind)
    903928{
    904929    VM& vm = globalObject->vm();
    … …  
    938963                    LOG_IC((ICEvent::InByIdSelfPatch, structure->classInfo(), ident, slot.slotBase() == base));
    939964                    structure->startWatchingPropertyForReplacements(vm, slot.cachedOffset());
    940                     ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationInByIdOptimize);
     965                    repatchSlowPathCall(codeBlock, stubInfo, operationInByIdOptimize);
    941966                    stubInfo.initInByIdSelf(locker, codeBlock, structure, slot.cachedOffset(), propertyName);
    942967                    return RetryCacheLater;
    … …  
    9851010        LOG_IC((ICEvent::InAddAccessCase, structure->classInfo(), ident, slot.slotBase() == base));
    9861011
    987         std::unique_ptr<AccessCase> newCase = AccessCase::create(
     1012        Ref<AccessCase> newCase = AccessCase::create(
    9881013            vm, codeBlock, wasFound ? AccessCase::InHit : AccessCase::InMiss, propertyName, wasFound ? slot.cachedOffset() : invalidOffset, structure, conditionSet, WTFMove(prototypeAccessChain));
    9891014
    … …  
    9941019           
    9951020            RELEASE_ASSERT(result.code());
    996             InlineAccess::rewireStubAsJump(stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     1021
     1022            switch (kind) {
     1023            case InByKind::Normal:
     1024                InlineAccess::rewireStubAsJumpInAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     1025                break;
     1026            case InByKind::NormalByVal:
     1027                InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     1028                break;
     1029            }
    9971030        }
    9981031    }
    … …  
    10081041    VM& vm = globalObject->vm();
    10091042
    1010     if (tryCacheInBy(globalObject, codeBlock, baseObject, propertyName, wasFound, slot, stubInfo) == GiveUpOnCache) {
     1043    if (tryCacheInBy(globalObject, codeBlock, baseObject, propertyName, wasFound, slot, stubInfo, kind) == GiveUpOnCache) {
    10111044        LOG_IC((ICEvent::InReplaceWithGeneric, baseObject->classInfo(globalObject->vm()), Identifier::fromUid(vm, propertyName.uid())));
    10121045        if (kind == InByKind::Normal)
    1013             ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationInByIdGeneric);
     1046            repatchSlowPathCall(codeBlock, stubInfo, operationInByIdGeneric);
    10141047        else
    1015             ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationInByValGeneric);
     1048            repatchSlowPathCall(codeBlock, stubInfo, operationInByValGeneric);
    10161049    }
    10171050}
    … …  
    10391072        LOG_IC((ICEvent::CheckPrivateBrandAddAccessCase, structure->classInfo(), ident, isBaseProperty));
    10401073
    1041         std::unique_ptr<AccessCase> newCase = AccessCase::createCheckPrivateBrand(vm, codeBlock, brandID, structure);
     1074        Ref<AccessCase> newCase = AccessCase::createCheckPrivateBrand(vm, codeBlock, brandID, structure);
    10421075
    10431076        result = stubInfo.addAccessCase(locker, globalObject, codeBlock, ECMAMode::strict(), brandID, WTFMove(newCase));
    … …  
    10471080
    10481081            RELEASE_ASSERT(result.code());
    1049             InlineAccess::rewireStubAsJump(stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     1082            InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
    10501083        }
    10511084    }
    … …  
    10611094
    10621095    if (tryCacheCheckPrivateBrand(globalObject, codeBlock, baseObject, brandID, stubInfo) == GiveUpOnCache)
    1063         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationCheckPrivateBrandGeneric);
     1096        repatchSlowPathCall(codeBlock, stubInfo, operationCheckPrivateBrandGeneric);
    10641097}
    10651098
    … …  
    10981131        LOG_IC((ICEvent::SetPrivateBrandAddAccessCase, oldStructure->classInfo(), ident, isBaseProperty));
    10991132
    1100         std::unique_ptr<AccessCase> newCase = AccessCase::createSetPrivateBrand(vm, codeBlock, brandID, oldStructure, newStructure);
     1133        Ref<AccessCase> newCase = AccessCase::createSetPrivateBrand(vm, codeBlock, brandID, oldStructure, newStructure);
    11011134
    11021135        result = stubInfo.addAccessCase(locker, globalObject, codeBlock, ECMAMode::strict(), brandID, WTFMove(newCase));
    … …  
    11061139           
    11071140            RELEASE_ASSERT(result.code());
    1108             InlineAccess::rewireStubAsJump(stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     1141            InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
    11091142        }
    11101143    }
    … …  
    11201153
    11211154    if (tryCacheSetPrivateBrand(globalObject, codeBlock, baseObject, oldStructure,  brandID, stubInfo) == GiveUpOnCache)
    1122         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationSetPrivateBrandGeneric);
     1155        repatchSlowPathCall(codeBlock, stubInfo, operationSetPrivateBrandGeneric);
    11231156}
    11241157
    … …  
    11401173        JSCell* value = valueValue.asCell();
    11411174        Structure* structure = value->structure(vm);
    1142         std::unique_ptr<AccessCase> newCase;
     1175        RefPtr<AccessCase> newCase;
    11431176        JSObject* prototype = jsDynamicCast<JSObject*>(vm, prototypeValue);
    11441177        if (prototype) {
    … …  
    11741207           
    11751208            RELEASE_ASSERT(result.code());
    1176 
    1177             MacroAssembler::repatchJump(
    1178                 stubInfo.patchableJump(),
    1179                 CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
     1209            InlineAccess::rewireStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo, CodeLocationLabel<JITStubRoutinePtrTag>(result.code()));
    11801210        }
    11811211    }
    … …  
    11921222    SuperSamplerScope superSamplerScope(false);
    11931223    if (tryCacheInstanceOf(globalObject, codeBlock, valueValue, prototypeValue, stubInfo, wasFound) == GiveUpOnCache)
    1194         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationInstanceOfGeneric);
     1224        repatchSlowPathCall(codeBlock, stubInfo, operationInstanceOfGeneric);
    11951225}
    11961226
    … …  
    12091239    MacroAssemblerCodeRef<JITStubRoutinePtrTag> virtualThunk = virtualThunkFor(vm, callLinkInfo);
    12101240    linkSlowPathTo(vm, callLinkInfo, virtualThunk);
    1211     callLinkInfo.setSlowStub(GCAwareJITStubRoutine::create(virtualThunk, vm));
     1241    callLinkInfo.setSlowStub(GCAwareJITStubRoutine::create(vm, virtualThunk));
    12121242}
    12131243
    … …  
    13271357    MacroAssemblerCodeRef<JITStubRoutinePtrTag> virtualThunk = virtualThunkFor(vm, callLinkInfo);
    13281358    revertCall(vm, callLinkInfo, virtualThunk);
    1329     callLinkInfo.setSlowStub(GCAwareJITStubRoutine::create(virtualThunk, vm));
     1359    callLinkInfo.setSlowStub(GCAwareJITStubRoutine::create(vm, virtualThunk));
    13301360    callLinkInfo.setClearedByVirtual();
    13311361}
    … …  
    16481678void resetGetBy(CodeBlock* codeBlock, StructureStubInfo& stubInfo, GetByKind kind)
    16491679{
    1650     ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, appropriateOptimizingGetByFunction(kind));
    1651     InlineAccess::rewireStubAsJump(stubInfo, stubInfo.slowPathStartLocation);
     1680    repatchSlowPathCall(codeBlock, stubInfo, appropriateOptimizingGetByFunction(kind));
     1681    switch (kind) {
     1682    case GetByKind::Normal:
     1683    case GetByKind::WithThis:
     1684    case GetByKind::Try:
     1685    case GetByKind::Direct:
     1686    case GetByKind::PrivateNameById:
     1687        InlineAccess::resetStubAsJumpInAccess(codeBlock, stubInfo);
     1688        break;
     1689    case GetByKind::NormalByVal:
     1690    case GetByKind::PrivateName:
     1691        InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo);
     1692        break;
     1693    }
    16521694}
    16531695
    … …  
    16551697{
    16561698    using FunctionType = decltype(&operationPutByIdDirectStrictOptimize);
    1657     FunctionType unoptimizedFunction = reinterpret_cast<FunctionType>(readPutICCallTarget(codeBlock, stubInfo.slowPathCallLocation).executableAddress());
     1699    FunctionType unoptimizedFunction = reinterpret_cast<FunctionType>(readPutICCallTarget(codeBlock, stubInfo).executableAddress());
    16581700    FunctionType optimizedFunction;
    16591701    if (unoptimizedFunction == operationPutByIdStrict || unoptimizedFunction == operationPutByIdStrictOptimize)
    … …  
    16721714    }
    16731715
    1674     ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, optimizedFunction);
    1675     InlineAccess::rewireStubAsJump(stubInfo, stubInfo.slowPathStartLocation);
     1716    repatchSlowPathCall(codeBlock, stubInfo, optimizedFunction);
     1717    InlineAccess::resetStubAsJumpInAccess(codeBlock, stubInfo);
    16761718}
    16771719
    … …  
    16791721{
    16801722    if (kind == DelByKind::Normal)
    1681         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationDeleteByIdOptimize);
     1723        repatchSlowPathCall(codeBlock, stubInfo, operationDeleteByIdOptimize);
    16821724    else
    1683         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationDeleteByValOptimize);
    1684     InlineAccess::rewireStubAsJump(stubInfo, stubInfo.slowPathStartLocation);
    1685 }
    1686 
    1687 static void resetPatchableJump(StructureStubInfo& stubInfo)
    1688 {
    1689     MacroAssembler::repatchJump(stubInfo.patchableJump(), stubInfo.slowPathStartLocation);
     1725        repatchSlowPathCall(codeBlock, stubInfo, operationDeleteByValOptimize);
     1726    InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo);
    16901727}
    16911728
    16921729void resetInBy(CodeBlock* codeBlock, StructureStubInfo& stubInfo, InByKind kind)
    16931730{
    1694     if (kind == InByKind::Normal)
    1695         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationInByIdOptimize);
    1696     else
    1697         ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationInByValOptimize);
    1698     InlineAccess::rewireStubAsJump(stubInfo, stubInfo.slowPathStartLocation);
    1699 }
    1700 
    1701 void resetInstanceOf(StructureStubInfo& stubInfo)
    1702 {
    1703     resetPatchableJump(stubInfo);
     1731    switch (kind) {
     1732    case InByKind::Normal:
     1733        repatchSlowPathCall(codeBlock, stubInfo, operationInByIdOptimize);
     1734        InlineAccess::resetStubAsJumpInAccess(codeBlock, stubInfo);
     1735        break;
     1736    case InByKind::NormalByVal:
     1737        repatchSlowPathCall(codeBlock, stubInfo, operationInByValOptimize);
     1738        InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo);
     1739        break;
     1740    }
     1741}
     1742
     1743void resetInstanceOf(CodeBlock* codeBlock, StructureStubInfo& stubInfo)
     1744{
     1745    repatchSlowPathCall(codeBlock, stubInfo, operationInstanceOfOptimize);
     1746    InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo);
    17041747}
    17051748
    17061749void resetCheckPrivateBrand(CodeBlock* codeBlock, StructureStubInfo& stubInfo)
    17071750{
    1708     ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationCheckPrivateBrandOptimize);
    1709     InlineAccess::rewireStubAsJump(stubInfo, stubInfo.slowPathStartLocation);
     1751    repatchSlowPathCall(codeBlock, stubInfo, operationCheckPrivateBrandOptimize);
     1752    InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo);
    17101753}
    17111754
    17121755void resetSetPrivateBrand(CodeBlock* codeBlock, StructureStubInfo& stubInfo)
    17131756{
    1714     ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation, operationSetPrivateBrandOptimize);
    1715     InlineAccess::rewireStubAsJump(stubInfo, stubInfo.slowPathStartLocation);
     1757    repatchSlowPathCall(codeBlock, stubInfo, operationSetPrivateBrandOptimize);
     1758    InlineAccess::resetStubAsJumpInAccessNotUsingInlineAccess(codeBlock, stubInfo);
    17161759}
    17171760

  • trunk/Source/JavaScriptCore/jit/Repatch.h

    r278445 r278656  
    7171void resetDelBy(CodeBlock*, StructureStubInfo&, DelByKind);
    7272void resetInBy(CodeBlock*, StructureStubInfo&, InByKind);
    73 void resetInstanceOf(StructureStubInfo&);
     73void resetInstanceOf(CodeBlock*, StructureStubInfo&);
    7474void resetCheckPrivateBrand(CodeBlock*, StructureStubInfo&);
    7575void resetSetPrivateBrand(CodeBlock*, StructureStubInfo&);

  • trunk/Source/JavaScriptCore/runtime/Options.cpp

    r278340 r278656  
    433433    Options::useConcurrentGC() = false;
    434434#endif
     435
     436    if (!isARM64())
     437        Options::useDataIC() = false;
     438    if (!Options::useDataIC())
     439        Options::useDataICInOptimizingJIT() = false;
    435440
    436441    // At initialization time, we may decide that useJIT should be false for any

  • trunk/Source/JavaScriptCore/runtime/OptionsList.h

    r278213 r278656  
    539539    v(Bool, useTopLevelAwait, true, Normal, "allow the await keyword at the top level of a module.") \
    540540    v(Bool, verboseExecutablePoolAllocation, false, Normal, nullptr) \
     541    v(Bool, useDataIC, isARM64(), Normal, nullptr) \
     542    v(Bool, useDataICInOptimizingJIT, false, Normal, nullptr) \
    541543
    542544

  • trunk/Source/JavaScriptCore/runtime/StructureIDTable.h

    r267192 r278656  
    3737
    3838#if USE(JSVALUE64)
    39 typedef uint32_t StructureID;
     39using StructureID = uint32_t;
    4040
    4141inline StructureID nukedStructureIDBit()
    … …  
    5959}
    6060#else // not USE(JSVALUE64)
    61 typedef Structure* StructureID;
     61using StructureID = Structure*;
    6262
    6363inline StructureID nukedStructureIDBit()

  • trunk/Source/JavaScriptCore/runtime/VM.cpp

    r278213 r278656  
    3030#include "VM.h"
    3131
     32#include "AccessCase.h"
    3233#include "AggregateError.h"
    3334#include "ArgList.h"
    … …  
    568569        getCTIInternalFunctionTrampolineFor(CodeForCall);
    569570        getCTIInternalFunctionTrampolineFor(CodeForConstruct);
     571        m_sharedJITStubs = makeUnique<SharedJITStubSet>();
    570572    }
    571573#endif // ENABLE(JIT)
    … …  
    655657    for (unsigned i = 0; i < m_scratchBuffers.size(); ++i)
    656658        VMMalloc::free(m_scratchBuffers[i]);
     659#endif
     660
     661#if ENABLE(JIT)
     662    m_sharedJITStubs = nullptr;
    657663#endif
    658664}

  • trunk/Source/JavaScriptCore/runtime/VM.h

    r278338 r278656  
    176176#endif
    177177class ShadowChicken;
     178class SharedJITStubSet;
    178179class ScriptExecutable;
    179180class SourceProvider;
    … …  
    847848    std::unique_ptr<JITThunks> jitStubs;
    848849    MacroAssemblerCodeRef<JITThunkPtrTag> getCTIStub(ThunkGenerator);
     850    std::unique_ptr<SharedJITStubSet> m_sharedJITStubs;
    849851
    850852#endif // ENABLE(JIT)
    … …  
    852854    std::unique_ptr<FTL::Thunks> ftlThunks;
    853855#endif
     856
    854857    NativeExecutable* getHostFunction(NativeFunction, NativeFunction constructor, const String& name);
    855858    NativeExecutable* getHostFunction(NativeFunction, Intrinsic, NativeFunction constructor, const DOMJIT::Signature*, const String& name);

  • trunk/Tools/ChangeLog

    r278638 r278656  
     12021-06-08  Yusuke Suzuki  <ysuzuki@apple.com>
     2
     3        [JSC] Use DataIC for AccessCase
     4        https://bugs.webkit.org/show_bug.cgi?id=226072
     5
     6        Reviewed by Saam Barati and Filip Pizlo.
     7
     8        * Scripts/run-jsc-stress-tests:
     9
    1102021-06-08  Dewei Zhu  <dewei_zhu@apple.com>
    211

  • trunk/Tools/Scripts/run-jsc-stress-tests

    r278425 r278656  
    553553# NOTE: Tests rely on this using scribbleFreeCells.
    554554NO_CJIT_OPTIONS = ["--useConcurrentJIT=false", "--thresholdForJITAfterWarmUp=100", "--scribbleFreeCells=true"]
    555 B3O1_OPTIONS = ["--defaultB3OptLevel=1"]
     555B3O1_OPTIONS = ["--defaultB3OptLevel=1", "--useDataICInOptimizingJIT=1"]
    556556B3O0_OPTIONS = ["--maxDFGNodesInBasicBlockForPreciseAnalysis=100", "--defaultB3OptLevel=0"]
    557557FTL_OPTIONS = ["--useFTLJIT=true"]
    … …  
    16201620
    16211621def runNoisyTestNoCJITB3O1
    1622     runNoisyTest("ftl-no-cjit", "--validateBytecode=true", "--validateGraphAtEachPhase=true", *(FTL_OPTIONS + NO_CJIT_OPTIONS + B3O1_OPTIONS))
     1622    runNoisyTest("ftl-no-cjit-b3o1", "--validateBytecode=true", "--validateGraphAtEachPhase=true", *(FTL_OPTIONS + NO_CJIT_OPTIONS + B3O1_OPTIONS))
    16231623end
    16241624
Note: See TracChangeset for help on using the changeset viewer.