Changeset 157209 in webkit


Ignore:
Timestamp:
Oct 9, 2013 9:24:57 PM (11 years ago)
Author:
fpizlo@apple.com
Message:

FTL should be able to do simple OSR exits using llvm.webkit.stackmap
​https://bugs.webkit.org/show_bug.cgi?id=122538

Reviewed by Oliver Hunt.

This gives the FTL the ability to OSR exit using the llvm.webkit.stackmap intrinsic.

  • The FTL compiles all OSR exit calls as calls to llvm.webkit.stackmap with a unique ID, passing a requested size that is big enough for own jump replacement.


  • After LLVM compilation, we parse the new LLVM stackmap section.


  • For all llvm.webkit.stackmaps that we used for OSR exits, we do a jumpReplacement, which targets exit thunks that we generate.


  • If an exit thunk fires, it causes JSC to compile an exit off-ramp that uses a combination of the JSC-internal OSR exit accounting (FTL::ExitValue and friends) and LLVM stackmap's accounting of where data actually ended up (register, indirect, constant) to reconstruct bytecode state.


This still has shortcomings; for example it cannot handle XMM or YMM registers. Handling
YMM registers will require adding some basic YMM support to our assemblers - really we
just need the ability to move a YMM's value into a GPR.

This patch preserves all of the old, intrinsic-less, FTL OSR exit support. Hence it
manages to pass all existing FTL tests even despite its incompleteness. I think that's
the right way to go since this is already a big patch, and anyway it would be great to
keep the intrinsic-less FTL OSR exit support so long as the LLVM side of this hasn't
landed.

  • JavaScriptCore.xcodeproj/project.pbxproj:
  • assembler/AbstractMacroAssembler.h:

(JSC::AbstractMacroAssembler::firstRegister):
(JSC::AbstractMacroAssembler::lastRegister):

  • assembler/MacroAssembler.h:

(JSC::MacroAssembler::isStackRelated):
(JSC::MacroAssembler::firstRealRegister):
(JSC::MacroAssembler::nextRegister):
(JSC::MacroAssembler::secondRealRegister):

  • assembler/MacroAssemblerX86Common.h:
  • assembler/X86Assembler.h:

(JSC::X86Assembler::firstRegister):
(JSC::X86Assembler::lastRegister):

  • dfg/DFGPlan.cpp:

(JSC::DFG::Plan::compileInThreadImpl):

  • ftl/FTLCArgumentGetter.cpp:

(JSC::FTL::CArgumentGetter::loadNextAndBox):

  • ftl/FTLCArgumentGetter.h:

(JSC::FTL::CArgumentGetter::loadNextDoubleIntoGPR):

  • ftl/FTLCompile.cpp:

(JSC::FTL::mmAllocateCodeSection):
(JSC::FTL::mmAllocateDataSection):
(JSC::FTL::dumpDataSection):
(JSC::FTL::fixFunctionBasedOnStackMaps):
(JSC::FTL::compile):

  • ftl/FTLExitThunkGenerator.cpp:

(JSC::FTL::ExitThunkGenerator::emitThunk):
(JSC::FTL::ExitThunkGenerator::emitThunks):

  • ftl/FTLExitThunkGenerator.h:
  • ftl/FTLExitValue.h:

(JSC::FTL::ExitValue::isInJSStackSomehow):
(JSC::FTL::ExitValue::valueFormat):

  • ftl/FTLFail.cpp:

(JSC::FTL::fail):

  • ftl/FTLIntrinsicRepository.h:
  • ftl/FTLJITCode.h:
  • ftl/FTLLowerDFGToLLVM.cpp:

(JSC::FTL::generateExitThunks):
(JSC::FTL::LowerDFGToLLVM::LowerDFGToLLVM):
(JSC::FTL::LowerDFGToLLVM::appendOSRExit):
(JSC::FTL::LowerDFGToLLVM::emitOSRExitCall):
(JSC::FTL::LowerDFGToLLVM::linkOSRExitsAndCompleteInitializationBlocks):

  • ftl/FTLOSRExit.h:
  • ftl/FTLOSRExitCompilationInfo.h:

(JSC::FTL::OSRExitCompilationInfo::OSRExitCompilationInfo):

  • ftl/FTLOSRExitCompiler.cpp:

(JSC::FTL::compileStubWithOSRExitStackmap):
(JSC::FTL::compileStubWithoutOSRExitStackmap):
(JSC::FTL::compileFTLOSRExit):

  • ftl/FTLSaveRestore.cpp: Added.

(JSC::FTL::bytesForGPRs):
(JSC::FTL::requiredScratchMemorySizeInBytes):
(JSC::FTL::offsetOfGPR):
(JSC::FTL::saveAllRegisters):
(JSC::FTL::restoreAllRegisters):

  • ftl/FTLSaveRestore.h: Added.
  • ftl/FTLStackMaps.cpp: Added.

(JSC::FTL::readObject):
(JSC::FTL::StackMaps::Constant::parse):
(JSC::FTL::StackMaps::Constant::dump):
(JSC::FTL::StackMaps::Location::parse):
(JSC::FTL::StackMaps::Location::dump):
(JSC::FTL::StackMaps::Location::involvesGPR):
(JSC::FTL::StackMaps::Location::isGPR):
(JSC::FTL::StackMaps::Location::gpr):
(JSC::FTL::StackMaps::Location::restoreInto):
(JSC::FTL::StackMaps::Record::parse):
(JSC::FTL::StackMaps::Record::dump):
(JSC::FTL::StackMaps::parse):
(JSC::FTL::StackMaps::dump):
(JSC::FTL::StackMaps::dumpMultiline):
(JSC::FTL::StackMaps::getRecordMap):
(WTF::printInternal):

  • ftl/FTLStackMaps.h: Added.
  • ftl/FTLState.h:
  • ftl/FTLThunks.cpp:

(JSC::FTL::osrExitGenerationThunkGenerator):

  • ftl/FTLValueFormat.cpp:

(JSC::FTL::reboxAccordingToFormat):

  • ftl/FTLValueFormat.h:
  • runtime/DataView.cpp:

(JSC::DataView::create):

  • runtime/DataView.h:

(JSC::DataView::read):

  • runtime/Options.h:
Location:
trunk/Source/JavaScriptCore
Files:
4 added
27 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/ChangeLog

    r157193 r157209  
     12013-10-08  Filip Pizlo  <fpizlo@apple.com>
     2
     3        FTL should be able to do simple OSR exits using llvm.webkit.stackmap
     4        https://bugs.webkit.org/show_bug.cgi?id=122538
     5
     6        Reviewed by Oliver Hunt.
     7       
     8        This gives the FTL the ability to OSR exit using the llvm.webkit.stackmap intrinsic.
     9       
     10        - The FTL compiles all OSR exit calls as calls to llvm.webkit.stackmap with a unique
     11          ID, passing a requested size that is big enough for own jump replacement.
     12       
     13        - After LLVM compilation, we parse the new LLVM stackmap section.
     14       
     15        - For all llvm.webkit.stackmaps that we used for OSR exits, we do a jumpReplacement,
     16          which targets exit thunks that we generate.
     17       
     18        - If an exit thunk fires, it causes JSC to compile an exit off-ramp that uses a
     19          combination of the JSC-internal OSR exit accounting (FTL::ExitValue and friends) and
     20          LLVM stackmap's accounting of where data actually ended up (register, indirect,
     21          constant) to reconstruct bytecode state.
     22       
     23        This still has shortcomings; for example it cannot handle XMM or YMM registers. Handling
     24        YMM registers will require adding some basic YMM support to our assemblers - really we
     25        just need the ability to move a YMM's value into a GPR.
     26       
     27        This patch preserves all of the old, intrinsic-less, FTL OSR exit support. Hence it
     28        manages to pass all existing FTL tests even despite its incompleteness. I think that's
     29        the right way to go since this is already a big patch, and anyway it would be great to
     30        keep the intrinsic-less FTL OSR exit support so long as the LLVM side of this hasn't
     31        landed.
     32
     33        * JavaScriptCore.xcodeproj/project.pbxproj:
     34        * assembler/AbstractMacroAssembler.h:
     35        (JSC::AbstractMacroAssembler::firstRegister):
     36        (JSC::AbstractMacroAssembler::lastRegister):
     37        * assembler/MacroAssembler.h:
     38        (JSC::MacroAssembler::isStackRelated):
     39        (JSC::MacroAssembler::firstRealRegister):
     40        (JSC::MacroAssembler::nextRegister):
     41        (JSC::MacroAssembler::secondRealRegister):
     42        * assembler/MacroAssemblerX86Common.h:
     43        * assembler/X86Assembler.h:
     44        (JSC::X86Assembler::firstRegister):
     45        (JSC::X86Assembler::lastRegister):
     46        * dfg/DFGPlan.cpp:
     47        (JSC::DFG::Plan::compileInThreadImpl):
     48        * ftl/FTLCArgumentGetter.cpp:
     49        (JSC::FTL::CArgumentGetter::loadNextAndBox):
     50        * ftl/FTLCArgumentGetter.h:
     51        (JSC::FTL::CArgumentGetter::loadNextDoubleIntoGPR):
     52        * ftl/FTLCompile.cpp:
     53        (JSC::FTL::mmAllocateCodeSection):
     54        (JSC::FTL::mmAllocateDataSection):
     55        (JSC::FTL::dumpDataSection):
     56        (JSC::FTL::fixFunctionBasedOnStackMaps):
     57        (JSC::FTL::compile):
     58        * ftl/FTLExitThunkGenerator.cpp:
     59        (JSC::FTL::ExitThunkGenerator::emitThunk):
     60        (JSC::FTL::ExitThunkGenerator::emitThunks):
     61        * ftl/FTLExitThunkGenerator.h:
     62        * ftl/FTLExitValue.h:
     63        (JSC::FTL::ExitValue::isInJSStackSomehow):
     64        (JSC::FTL::ExitValue::valueFormat):
     65        * ftl/FTLFail.cpp:
     66        (JSC::FTL::fail):
     67        * ftl/FTLIntrinsicRepository.h:
     68        * ftl/FTLJITCode.h:
     69        * ftl/FTLLowerDFGToLLVM.cpp:
     70        (JSC::FTL::generateExitThunks):
     71        (JSC::FTL::LowerDFGToLLVM::LowerDFGToLLVM):
     72        (JSC::FTL::LowerDFGToLLVM::appendOSRExit):
     73        (JSC::FTL::LowerDFGToLLVM::emitOSRExitCall):
     74        (JSC::FTL::LowerDFGToLLVM::linkOSRExitsAndCompleteInitializationBlocks):
     75        * ftl/FTLOSRExit.h:
     76        * ftl/FTLOSRExitCompilationInfo.h:
     77        (JSC::FTL::OSRExitCompilationInfo::OSRExitCompilationInfo):
     78        * ftl/FTLOSRExitCompiler.cpp:
     79        (JSC::FTL::compileStubWithOSRExitStackmap):
     80        (JSC::FTL::compileStubWithoutOSRExitStackmap):
     81        (JSC::FTL::compileFTLOSRExit):
     82        * ftl/FTLSaveRestore.cpp: Added.
     83        (JSC::FTL::bytesForGPRs):
     84        (JSC::FTL::requiredScratchMemorySizeInBytes):
     85        (JSC::FTL::offsetOfGPR):
     86        (JSC::FTL::saveAllRegisters):
     87        (JSC::FTL::restoreAllRegisters):
     88        * ftl/FTLSaveRestore.h: Added.
     89        * ftl/FTLStackMaps.cpp: Added.
     90        (JSC::FTL::readObject):
     91        (JSC::FTL::StackMaps::Constant::parse):
     92        (JSC::FTL::StackMaps::Constant::dump):
     93        (JSC::FTL::StackMaps::Location::parse):
     94        (JSC::FTL::StackMaps::Location::dump):
     95        (JSC::FTL::StackMaps::Location::involvesGPR):
     96        (JSC::FTL::StackMaps::Location::isGPR):
     97        (JSC::FTL::StackMaps::Location::gpr):
     98        (JSC::FTL::StackMaps::Location::restoreInto):
     99        (JSC::FTL::StackMaps::Record::parse):
     100        (JSC::FTL::StackMaps::Record::dump):
     101        (JSC::FTL::StackMaps::parse):
     102        (JSC::FTL::StackMaps::dump):
     103        (JSC::FTL::StackMaps::dumpMultiline):
     104        (JSC::FTL::StackMaps::getRecordMap):
     105        (WTF::printInternal):
     106        * ftl/FTLStackMaps.h: Added.
     107        * ftl/FTLState.h:
     108        * ftl/FTLThunks.cpp:
     109        (JSC::FTL::osrExitGenerationThunkGenerator):
     110        * ftl/FTLValueFormat.cpp:
     111        (JSC::FTL::reboxAccordingToFormat):
     112        * ftl/FTLValueFormat.h:
     113        * runtime/DataView.cpp:
     114        (JSC::DataView::create):
     115        * runtime/DataView.h:
     116        (JSC::DataView::read):
     117        * runtime/Options.h:
     118
    11192013-10-09  Filip Pizlo  <fpizlo@apple.com>
    2120

  • trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

    r157193 r157209  
    315315                0F9D339617FFC4E60073C2BC /* DFGFlushedAt.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D339417FFC4E60073C2BC /* DFGFlushedAt.cpp */; };
    316316                0F9D339717FFC4E60073C2BC /* DFGFlushedAt.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9D339517FFC4E60073C2BC /* DFGFlushedAt.h */; settings = {ATTRIBUTES = (Private, ); }; };
     317                0F9D339A1803ADB70073C2BC /* FTLStackMaps.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D33981803ADB70073C2BC /* FTLStackMaps.cpp */; };
     318                0F9D339B1803ADB70073C2BC /* FTLStackMaps.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9D33991803ADB70073C2BC /* FTLStackMaps.h */; settings = {ATTRIBUTES = (Private, ); }; };
    317319                0F9FB4F417FCB91700CB67F8 /* DFGStackLayoutPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */; };
    318320                0F9FB4F517FCB91700CB67F8 /* DFGStackLayoutPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    373375                0FC81516140511B500CFA603 /* VTableSpectrum.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FC815121405118600CFA603 /* VTableSpectrum.cpp */; };
    374376                0FCCAE4516D0CF7400D0C65B /* ParserError.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FCCAE4316D0CF6E00D0C65B /* ParserError.h */; settings = {ATTRIBUTES = (Private, ); }; };
     377                0FCEFAAB1804C13E00472CE4 /* FTLSaveRestore.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */; };
     378                0FCEFAAC1804C13E00472CE4 /* FTLSaveRestore.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */; settings = {ATTRIBUTES = (Private, ); }; };
    375379                0FD2C92416D01EE900C7803F /* StructureInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD2C92316D01EE900C7803F /* StructureInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
    376380                0FD3C82614115D4000FD81CB /* DFGDriver.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD3C82014115CF800FD81CB /* DFGDriver.cpp */; };
    … …  
    15291533                0F9D339417FFC4E60073C2BC /* DFGFlushedAt.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGFlushedAt.cpp; path = dfg/DFGFlushedAt.cpp; sourceTree = "<group>"; };
    15301534                0F9D339517FFC4E60073C2BC /* DFGFlushedAt.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGFlushedAt.h; path = dfg/DFGFlushedAt.h; sourceTree = "<group>"; };
     1535                0F9D33981803ADB70073C2BC /* FTLStackMaps.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLStackMaps.cpp; path = ftl/FTLStackMaps.cpp; sourceTree = "<group>"; };
     1536                0F9D33991803ADB70073C2BC /* FTLStackMaps.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLStackMaps.h; path = ftl/FTLStackMaps.h; sourceTree = "<group>"; };
    15311537                0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStackLayoutPhase.cpp; path = dfg/DFGStackLayoutPhase.cpp; sourceTree = "<group>"; };
    15321538                0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStackLayoutPhase.h; path = dfg/DFGStackLayoutPhase.h; sourceTree = "<group>"; };
    … …  
    15991605                0FCB408515C0A3C30048932B /* SlotVisitorInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SlotVisitorInlines.h; sourceTree = "<group>"; };
    16001606                0FCCAE4316D0CF6E00D0C65B /* ParserError.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ParserError.h; sourceTree = "<group>"; };
     1607                0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLSaveRestore.cpp; path = ftl/FTLSaveRestore.cpp; sourceTree = "<group>"; };
     1608                0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLSaveRestore.h; path = ftl/FTLSaveRestore.h; sourceTree = "<group>"; };
    16011609                0FD2C92316D01EE900C7803F /* StructureInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StructureInlines.h; sourceTree = "<group>"; };
    16021610                0FD3C82014115CF800FD81CB /* DFGDriver.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGDriver.cpp; path = dfg/DFGDriver.cpp; sourceTree = "<group>"; };
    … …  
    26412649                                0FEA0A291709629600BB722C /* FTLOutput.cpp */,
    26422650                                0FEA0A06170513DB00BB722C /* FTLOutput.h */,
     2651                                0FCEFAA91804C13E00472CE4 /* FTLSaveRestore.cpp */,
     2652                                0FCEFAAA1804C13E00472CE4 /* FTLSaveRestore.h */,
     2653                                0F9D33981803ADB70073C2BC /* FTLStackMaps.cpp */,
     2654                                0F9D33991803ADB70073C2BC /* FTLStackMaps.h */,
    26432655                                0FEA0A151706BB9000BB722C /* FTLState.cpp */,
    26442656                                0FEA0A07170513DB00BB722C /* FTLState.h */,
    … …  
    43054317                                0F2B66FB17B6B5AB00A7AE3F /* JSTypedArrayConstructors.h in Headers */,
    43064318                                0F2B66FD17B6B5AB00A7AE3F /* JSTypedArrayPrototypes.h in Headers */,
     4319                                0FCEFAAC1804C13E00472CE4 /* FTLSaveRestore.h in Headers */,
    43074320                                0F2B66FF17B6B5AB00A7AE3F /* JSTypedArrays.h in Headers */,
    43084321                                6507D29E0E871E5E00D7D896 /* JSTypeInfo.h in Headers */,
    … …  
    45004513                                C2DF44301707AC0100A5CA96 /* SuperRegion.h in Headers */,
    45014514                                BC18C46B0E16F5CD00B34460 /* SymbolTable.h in Headers */,
     4515                                0F9D339B1803ADB70073C2BC /* FTLStackMaps.h in Headers */,
    45024516                                A784A26411D16622005776AC /* SyntaxChecker.h in Headers */,
    45034517                                0F572D4F16879FDD00E57FBD /* ThunkGenerator.h in Headers */,
    … …  
    52505264                                14280850107EC0D70013E7B2 /* Operations.cpp in Sources */,
    52515265                                A7BDAEC817F4EA1400F6140C /* ArrayIteratorPrototype.cpp in Sources */,
     5266                                0FCEFAAB1804C13E00472CE4 /* FTLSaveRestore.cpp in Sources */,
    52525267                                0FE228EE1436AB2C00196C48 /* Options.cpp in Sources */,
    52535268                                148F21BC107EC54D0042EC2C /* Parser.cpp in Sources */,
    … …  
    53475362                                86704B8612DBA33700A9FE7B /* YarrJIT.cpp in Sources */,
    53485363                                86704B8912DBA33700A9FE7B /* YarrPattern.cpp in Sources */,
     5364                                0F9D339A1803ADB70073C2BC /* FTLStackMaps.cpp in Sources */,
    53495365                                86704B4212DB8A8100A9FE7B /* YarrSyntaxChecker.cpp in Sources */,
    53505366                        );

  • trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h

    r156780 r157209  
    7575
    7676    typedef typename AssemblerType::RegisterID RegisterID;
     77   
     78    static RegisterID firstRegister() { return AssemblerType::firstRegister(); }
     79    static RegisterID lastRegister() { return AssemblerType::lastRegister(); }
    7780
    7881    // Section 1: MacroAssembler operand types

  • trunk/Source/JavaScriptCore/assembler/MacroAssembler.h

    r157050 r157209  
    6767class MacroAssembler : public MacroAssemblerBase {
    6868public:
     69
     70    static bool isStackRelated(RegisterID reg)
     71    {
     72        return reg == stackPointerRegister || reg == framePointerRegister;
     73    }
     74   
     75    static RegisterID firstRealRegister()
     76    {
     77        RegisterID firstRegister = MacroAssembler::firstRegister();
     78        while (MacroAssembler::isStackRelated(firstRegister))
     79            firstRegister = static_cast<RegisterID>(firstRegister + 1);
     80        return firstRegister;
     81    }
     82   
     83    static RegisterID nextRegister(RegisterID reg)
     84    {
     85        RegisterID result = static_cast<RegisterID>(reg + 1);
     86        while (MacroAssembler::isStackRelated(result))
     87            result = static_cast<RegisterID>(result + 1);
     88        return result;
     89    }
     90   
     91    static RegisterID secondRealRegister()
     92    {
     93        return nextRegister(firstRealRegister());
     94    }
    6995
    7096    using MacroAssemblerBase::pop;

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.h

    r156780 r157209  
    9898    static const RegisterID stackPointerRegister = X86Registers::esp;
    9999    static const RegisterID framePointerRegister = X86Registers::ebp;
    100 
     100   
    101101    static bool shouldBlindForSpecificArch(uint32_t value) { return value >= 0x00ffffff; }
    102102#if CPU(X86_64)

  • trunk/Source/JavaScriptCore/assembler/X86Assembler.h

    r156047 r157209  
    127127public:
    128128    typedef X86Registers::RegisterID RegisterID;
     129    static RegisterID firstRegister() { return X86Registers::eax; }
     130    static RegisterID lastRegister()
     131    {
     132#if CPU(X86_64)
     133        return X86Registers::r15;
     134#else
     135        return X86Registers::edi;
     136#endif
     137    }
     138   
    129139    typedef X86Registers::XMMRegisterID XMMRegisterID;
    130140    typedef XMMRegisterID FPRegisterID;

  • trunk/Source/JavaScriptCore/dfg/DFGPlan.cpp

    r156984 r157209  
    265265            beforeFTL = currentTimeMS();
    266266       
    267         if (Options::llvmAlwaysFails()) {
     267        if (Options::llvmAlwaysFailsBeforeCompile()) {
    268268            FTL::fail(state);
    269269            return FTLPath;
    … …  
    271271       
    272272        FTL::compile(state);
     273
     274        if (Options::llvmAlwaysFailsBeforeLink()) {
     275            FTL::fail(state);
     276            return FTLPath;
     277        }
     278       
    273279        FTL::link(state);
    274280        return FTLPath;

  • trunk/Source/JavaScriptCore/ftl/FTLCArgumentGetter.cpp

    r156047 r157209  
    5353   
    5454    switch (format) {
    55     case ValueFormatInt32: {
     55    case ValueFormatInt32:
     56    case ValueFormatUInt32:
    5657        loadNext32(destination);
    57         m_jit.or64(GPRInfo::tagTypeNumberRegister, destination);
    5858        break;
    59     }
    60            
    61     case ValueFormatUInt32: {
    62         loadNext32(destination);
    63         m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
    64         m_jit.boxInt52(destination, destination, scratch1, FPRInfo::fpRegT0);
    65         m_jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
    66         break;
    67     }
    6859       
    69     case ValueFormatInt52: {
    70         loadNext64(destination);
    71         m_jit.rshift64(AssemblyHelpers::TrustedImm32(JSValue::int52ShiftAmount), destination);
    72         m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
    73         m_jit.boxInt52(destination, destination, scratch1, FPRInfo::fpRegT0);
    74         m_jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
    75         break;
    76     }
    77            
    78     case ValueFormatStrictInt52: {
    79         loadNext64(destination);
    80         m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
    81         m_jit.boxInt52(destination, destination, scratch1, FPRInfo::fpRegT0);
    82         m_jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
    83         break;
    84     }
    85            
    86     case ValueFormatBoolean: {
    87         loadNext8(destination);
    88         m_jit.or32(MacroAssembler::TrustedImm32(ValueFalse), destination);
    89         break;
    90     }
    91            
    92     case ValueFormatJSValue: {
     60    case ValueFormatInt52:
     61    case ValueFormatStrictInt52:
     62    case ValueFormatJSValue:
    9363        loadNext64(destination);
    9464        break;
    95     }
    9665           
    97     case ValueFormatDouble: {
    98         m_jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch1);
    99         loadNextDouble(FPRInfo::fpRegT0);
    100         m_jit.boxDouble(FPRInfo::fpRegT0, destination);
    101         m_jit.move64ToDouble(scratch1, FPRInfo::fpRegT0);
     66    case ValueFormatBoolean:
     67        loadNext8(destination);
    10268        break;
    103     }
     69           
     70    case ValueFormatDouble:
     71        loadNextDoubleIntoGPR(destination);
     72        break;
    10473           
    10574    default:
    … …  
    10776        break;
    10877    }
     78   
     79    reboxAccordingToFormat(format, m_jit, destination, scratch1, scratch2);
    10980}
    11081

  • trunk/Source/JavaScriptCore/ftl/FTLCArgumentGetter.h

    r156184 r157209  
    101101    }
    102102   
     103    void loadNextDoubleIntoGPR(GPRReg destination)
     104    {
     105        if (m_fprArgumentIndex < FPRInfo::numberOfArgumentRegisters) {
     106            m_jit.moveDoubleTo64(FPRInfo::toArgumentRegister(m_fprArgumentIndex++), destination);
     107            return;
     108        }
     109       
     110        m_jit.load64(nextAddress(), destination);
     111    }
     112   
    103113    void loadNextDouble(FPRReg destination)
    104114    {

  • trunk/Source/JavaScriptCore/ftl/FTLCompile.cpp

    r157062 r157209  
    3232#include "CCallHelpers.h"
    3333#include "DFGCommon.h"
     34#include "DataView.h"
    3435#include "Disassembler.h"
     36#include "FTLExitThunkGenerator.h"
    3537#include "FTLJITCode.h"
     38#include "FTLThunks.h"
    3639#include "JITStubs.h"
    3740#include "LinkBuffer.h"
     41#include "RepatchBuffer.h"
    3842#include <wtf/LLVMHeaders.h>
    3943
    … …  
    4347
    4448static uint8_t* mmAllocateCodeSection(
    45     void* opaqueState, uintptr_t size, unsigned alignment, unsigned sectionID,
    46     const char* sectionName)
    47 {
    48     UNUSED_PARAM(sectionID);
    49     UNUSED_PARAM(sectionName);
     49    void* opaqueState, uintptr_t size, unsigned alignment, unsigned, const char* sectionName)
     50{
    5051   
    5152    State& state = *static_cast<State*>(opaqueState);
    … …  
    5859   
    5960    state.jitCode->addHandle(result);
     61    state.codeSectionNames.append(sectionName);
    6062   
    6163    return static_cast<uint8_t*>(result->start());
    … …  
    6769{
    6870    UNUSED_PARAM(sectionID);
    69     UNUSED_PARAM(sectionName);
    7071    UNUSED_PARAM(isReadOnly);
    7172
    … …  
    7778        (size + sizeof(LSectionWord) - 1) / sizeof(LSectionWord));
    7879   
    79     state.jitCode->addDataSection(section);
     80    if (!strcmp(sectionName, "__js_stackmaps"))
     81        state.stackmapsSection = section;
     82    else {
     83        state.jitCode->addDataSection(section);
     84        state.dataSectionNames.append(sectionName);
     85    }
    8086   
    8187    return bitwise_cast<uint8_t*>(section.data());
    … …  
    8995static void mmDestroy(void*)
    9096{
     97}
     98
     99static void dumpDataSection(RefCountedArray<LSectionWord> section, const char* prefix)
     100{
     101    for (unsigned j = 0; j < section.size(); ++j) {
     102        char buf[32];
     103        snprintf(buf, sizeof(buf), "0x%lx", static_cast<unsigned long>(bitwise_cast<uintptr_t>(section.data() + j)));
     104        dataLogF("%s%16s: 0x%016llx\n", prefix, buf, static_cast<long long>(section[j]));
     105    }
     106}
     107
     108static void fixFunctionBasedOnStackMaps(
     109    State& state, CodeBlock* codeBlock, JITCode* jitCode, GeneratedFunction generatedFunction,
     110    StackMaps::RecordMap& recordMap)
     111{
     112    VM& vm = state.graph.m_vm;
     113    MacroAssemblerCodeRef osrExitThunk =
     114        vm.getCTIStub(osrExitGenerationThunkGenerator);
     115    CodeLocationLabel target = CodeLocationLabel(osrExitThunk.code());
     116
     117    ExitThunkGenerator exitThunkGenerator(state);
     118    exitThunkGenerator.emitThunks();
     119    if (exitThunkGenerator.didThings()) {
     120        OwnPtr<LinkBuffer> linkBuffer = adoptPtr(new LinkBuffer(
     121            vm, &exitThunkGenerator, codeBlock, JITCompilationMustSucceed));
     122       
     123        ASSERT(state.osrExit.size() == state.jitCode->osrExit.size());
     124       
     125        for (unsigned i = 0; i < state.osrExit.size(); ++i) {
     126            OSRExitCompilationInfo& info = state.osrExit[i];
     127            OSRExit& exit = jitCode->osrExit[i];
     128           
     129            linkBuffer->link(info.m_thunkJump, target);
     130           
     131            info.m_thunkAddress = linkBuffer->locationOf(info.m_thunkLabel);
     132           
     133            exit.m_patchableCodeOffset = linkBuffer->offsetOf(info.m_thunkJump);
     134        }
     135       
     136        state.finalizer->initializeExitThunksLinkBuffer(linkBuffer.release());
     137    }
     138
     139    RepatchBuffer repatchBuffer(codeBlock);
     140   
     141    for (unsigned exitIndex = jitCode->osrExit.size(); exitIndex--;) {
     142        OSRExitCompilationInfo& info = state.osrExit[exitIndex];
     143        OSRExit& exit = jitCode->osrExit[exitIndex];
     144        StackMaps::Record& record = recordMap.find(exit.m_stackmapID)->value;
     145       
     146        repatchBuffer.replaceWithJump(
     147            CodeLocationLabel(
     148                bitwise_cast<char*>(generatedFunction) + record.instructionOffset),
     149            info.m_thunkAddress);
     150    }
    91151}
    92152
    … …  
    144204
    145205    if (shouldShowDisassembly()) {
    146         // FIXME: fourthTier: FTL memory allocator should be able to tell us which of
    147         // these things is actually code or data.
    148         // https://bugs.webkit.org/show_bug.cgi?id=116189
    149206        for (unsigned i = 0; i < state.jitCode->handles().size(); ++i) {
    150207            ExecutableMemoryHandle* handle = state.jitCode->handles()[i].get();
    … …  
    152209                "Generated LLVM code for ",
    153210                CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT),
    154                 " #", i, ":\n");
     211                " #", i, ", ", state.codeSectionNames[i], ":\n");
    155212            disassemble(
    156213                MacroAssemblerCodePtr(handle->start()), handle->sizeInBytes(),
    … …  
    163220                "Generated LLVM data section for ",
    164221                CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT),
    165                 " #", i, ":\n");
    166             for (unsigned j = 0; j < section.size(); ++j) {
    167                 char buf[32];
    168                 snprintf(buf, sizeof(buf), "0x%lx", static_cast<unsigned long>(bitwise_cast<uintptr_t>(section.data() + j)));
    169                 dataLogF("    %16s: 0x%016llx\n", buf, static_cast<long long>(section[j]));
     222                " #", i, ", ", state.dataSectionNames[i], ":\n");
     223            dumpDataSection(section, "    ");
     224        }
     225    }
     226   
     227    if (state.stackmapsSection.size()) {
     228        if (shouldShowDisassembly()) {
     229            dataLog(
     230                "Generated LLVM stackmaps section for ",
     231                CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT), ":\n");
     232            dataLog("    Raw data:\n");
     233            dumpDataSection(state.stackmapsSection, "    ");
     234        }
     235       
     236        RefPtr<DataView> stackmapsData = DataView::create(
     237            ArrayBuffer::create(state.stackmapsSection.data(), state.stackmapsSection.byteSize()));
     238        state.jitCode->stackmaps.parse(stackmapsData.get());
     239   
     240        if (shouldShowDisassembly()) {
     241            dataLog("    Structured data:\n");
     242            state.jitCode->stackmaps.dumpMultiline(WTF::dataFile(), "        ");
     243        }
     244       
     245        StackMaps::RecordMap recordMap = state.jitCode->stackmaps.getRecordMap();
     246        fixFunctionBasedOnStackMaps(
     247            state, state.graph.m_codeBlock, state.jitCode.get(), state.generatedFunction,
     248            recordMap);
     249       
     250        if (shouldShowDisassembly()) {
     251            for (unsigned i = 0; i < state.jitCode->handles().size(); ++i) {
     252                if (state.codeSectionNames[i] != "__text")
     253                    continue;
     254               
     255                ExecutableMemoryHandle* handle = state.jitCode->handles()[i].get();
     256                dataLog(
     257                    "Generated LLVM code after stackmap-based fix-up for ",
     258                    CodeBlockWithJITType(state.graph.m_codeBlock, JITCode::DFGJIT),
     259                    " #", i, ", ", state.codeSectionNames[i], ":\n");
     260                disassemble(
     261                    MacroAssemblerCodePtr(handle->start()), handle->sizeInBytes(),
     262                    "    ", WTF::dataFile(), LLVMSubset);
    170263            }
    171264        }
    172265    }
     266   
     267    state.module = 0; // We no longer own the module.
    173268}
    174269

  • trunk/Source/JavaScriptCore/ftl/FTLExitThunkGenerator.cpp

    r153121 r157209  
    5252   
    5353    info.m_thunkLabel = label();
    54     move(TrustedImm32(index), GPRInfo::nonArgGPR0);
     54    if (Options::ftlOSRExitUsesStackmap())
     55        push(TrustedImm32(index));
     56    else
     57        move(TrustedImm32(index), GPRInfo::nonArgGPR0);
    5558    info.m_thunkJump = patchableJump();
    5659   
    5760    m_didThings = true;
     61}
     62
     63void ExitThunkGenerator::emitThunks()
     64{
     65    for (unsigned i = 0; i < m_state.osrExit.size(); ++i)
     66        emitThunk(i);
    5867}
    5968

  • trunk/Source/JavaScriptCore/ftl/FTLExitThunkGenerator.h

    r156184 r157209  
    4444   
    4545    void emitThunk(unsigned index);
     46    void emitThunks();
    4647   
    4748    bool didThings() const { return m_didThings; }

  • trunk/Source/JavaScriptCore/ftl/FTLExitValue.h

    r156984 r157209  
    127127        case ExitValueInJSStack:
    128128        case ExitValueInJSStackAsInt32:
     129        case ExitValueInJSStackAsInt52:
    129130        case ExitValueInJSStackAsDouble:
    130131            return true;
    … …  
    153154        return VirtualRegister(u.virtualRegister);
    154155    }
    155    
     156
     157    // If it's in the JSStack somehow, this will tell you what format it's in, in a manner
     158    // that is compatible with exitArgument().format(). If it's a constant or it's dead, it
     159    // will claim to be a JSValue. If it's an argument then it will tell you the argument's
     160    // format.
     161    ValueFormat valueFormat() const
     162    {
     163        switch (kind()) {
     164        case InvalidExitValue:
     165            RELEASE_ASSERT_NOT_REACHED();
     166            return InvalidValueFormat;
     167           
     168        case ExitValueDead:
     169        case ExitValueConstant:
     170        case ExitValueInJSStack:
     171            return ValueFormatJSValue;
     172           
     173        case ExitValueArgument:
     174            return exitArgument().format();
     175           
     176        case ExitValueInJSStackAsInt32:
     177            return ValueFormatInt32;
     178           
     179        case ExitValueInJSStackAsInt52:
     180            return ValueFormatInt52;
     181           
     182        case ExitValueInJSStackAsDouble:
     183            return ValueFormatDouble;
     184        }
     185       
     186        RELEASE_ASSERT_NOT_REACHED();
     187        return InvalidValueFormat;
     188    }
     189
    156190    void dump(PrintStream&) const;
    157191    void dumpInContext(PrintStream&, DumpContext*) const;

  • trunk/Source/JavaScriptCore/ftl/FTLFail.cpp

    r153289 r157209  
    4141    state.graph.m_plan.finalizer = adoptPtr(new FailedFinalizer(state.graph.m_plan));
    4242   
    43     LLVMDisposeModule(state.module);
     43    if (state.module)
     44        LLVMDisposeModule(state.module);
    4445}
    4546

  • trunk/Source/JavaScriptCore/ftl/FTLIntrinsicRepository.h

    r156490 r157209  
    4545    macro(subWithOverflow32, "llvm.ssub.with.overflow.i32", functionType(structType(m_context, int32, boolean), int32, int32)) \
    4646    macro(subWithOverflow64, "llvm.ssub.with.overflow.i64", functionType(structType(m_context, int64, boolean), int64, int64)) \
     47    macro(webkitStackmap, "llvm.webkit.stackmap", functionType(voidType, int32, int32, Variadic)) \
    4748    macro(trap, "llvm.trap", functionType(voidType)) \
    4849    macro(osrExit, "webkit_osr_exit", functionType(voidType, boolean, int32, Variadic))

  • trunk/Source/JavaScriptCore/ftl/FTLJITCode.h

    r157062 r157209  
    3333#include "DFGCommonData.h"
    3434#include "FTLOSRExit.h"
     35#include "FTLStackMaps.h"
    3536#include "JITCode.h"
    3637#include <wtf/LLVMHeaders.h>
    … …  
    6869    DFG::CommonData common;
    6970    SegmentedVector<OSRExit, 8> osrExit;
     71    StackMaps stackmaps;
    7072   
    7173private:

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToLLVM.cpp

    r156984 r157209  
    5353static int compileCounter;
    5454
     55static bool generateExitThunks()
     56{
     57    return !Options::useLLVMOSRExitIntrinsic() && !Options::ftlOSRExitUsesStackmap();
     58}
     59
    5560// Using this instead of typeCheck() helps to reduce the load on LLVM, by creating
    5661// significantly less dead code.
    … …  
    7681        , m_state(state.graph)
    7782        , m_interpreter(state.graph, m_state)
     83        , m_stackmapIDs(0)
    7884    {
    7985    }
    … …  
    32573263            m_out.branch(failCondition, failCase, continuation);
    32583264
    3259             m_out.appendTo(m_prologue);
    3260             info.m_thunkAddress = buildAlloca(m_out.m_builder, m_out.intPtr);
     3265            if (generateExitThunks()) {
     3266                m_out.appendTo(m_prologue);
     3267                info.m_thunkAddressValue = buildAlloca(m_out.m_builder, m_out.intPtr);
     3268            }
    32613269       
    32623270            lastNext = m_out.appendTo(failCase, continuation);
    … …  
    32663274            m_out.call(
    32673275                m_out.intToPtr(
    3268                     m_out.get(info.m_thunkAddress),
     3276                    m_out.get(info.m_thunkAddressValue),
    32693277                    pointerType(functionType(m_out.voidType))));
    32703278        } else
    … …  
    32763284            m_out.appendTo(continuation, lastNext);
    32773285       
    3278             m_exitThunkGenerator.emitThunk(index);
     3286            if (generateExitThunks())
     3287                m_exitThunkGenerator.emitThunk(index);
    32793288        }
    32803289    }
    … …  
    33423351        }
    33433352       
     3353        if (Options::ftlOSRExitUsesStackmap()) {
     3354            exit.m_stackmapID = m_stackmapIDs++;
     3355            arguments.insert(0, m_out.constInt32(MacroAssembler::maxJumpReplacementSize()));
     3356            arguments.insert(0, m_out.constInt32(exit.m_stackmapID));
     3357       
     3358            m_out.call(m_out.webkitStackmapIntrinsic(), arguments);
     3359            return;
     3360        }
     3361       
    33443362        m_out.call(
    33453363            m_out.intToPtr(
    3346                 m_out.get(info.m_thunkAddress),
     3364                m_out.get(info.m_thunkAddressValue),
    33473365                pointerType(functionType(m_out.voidType, argumentTypes))),
    33483366            arguments);
    … …  
    35003518                    m_out.constIntPtr(
    35013519                        linkBuffer->locationOf(info.m_thunkLabel).executableAddress()),
    3502                     info.m_thunkAddress);
     3520                    info.m_thunkAddressValue);
    35033521           
    35043522                exit.m_patchableCodeOffset = linkBuffer->offsetOf(info.m_thunkJump);
    … …  
    36973715    Node* m_node;
    36983716    SpeculationDirection m_direction;
     3717   
     3718    uint32_t m_stackmapIDs;
    36993719};
    37003720

  • trunk/Source/JavaScriptCore/ftl/FTLOSRExit.h

    r156511 r157209  
    166166    Operands<ExitValue> m_values;
    167167   
     168    uint32_t m_stackmapID;
     169   
    168170    CodeLocationJump codeLocationForRepatch(CodeBlock* ftlCodeBlock) const;
    169171   

  • trunk/Source/JavaScriptCore/ftl/FTLOSRExitCompilationInfo.h

    r153121 r157209  
    3838struct OSRExitCompilationInfo {
    3939    OSRExitCompilationInfo()
    40         : m_thunkAddress(0)
     40        : m_thunkAddressValue(0)
    4141    {
    4242    }
    … …  
    4444    MacroAssembler::Label m_thunkLabel;
    4545    MacroAssembler::PatchableJump m_thunkJump;
    46     LValue m_thunkAddress;
     46    CodeLocationLabel m_thunkAddress;
     47    LValue m_thunkAddressValue;
    4748};
    4849

  • trunk/Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp

    r156984 r157209  
    3535#include "FTLJITCode.h"
    3636#include "FTLOSRExit.h"
     37#include "FTLSaveRestore.h"
    3738#include "Operations.h"
    3839#include "RepatchBuffer.h"
    … …  
    4243using namespace DFG;
    4344
    44 static void compileStub(
     45// This implements two flavors of OSR exit: one that involves having LLVM intrinsics to help
     46// OSR exit, and one that doesn't. The one that doesn't will get killed off, so we don't attempt
     47// to share code between the two.
     48
     49static void compileStubWithOSRExitStackmap(
     50    unsigned exitID, JITCode* jitCode, OSRExit& exit, VM* vm, CodeBlock* codeBlock)
     51{
     52    StackMaps::Record* record;
     53   
     54    for (unsigned i = jitCode->stackmaps.records.size(); i--;) {
     55        record = &jitCode->stackmaps.records[i];
     56        if (record->patchpointID == exit.m_stackmapID)
     57            break;
     58    }
     59   
     60    RELEASE_ASSERT(record->patchpointID == exit.m_stackmapID);
     61   
     62    CCallHelpers jit(vm, codeBlock);
     63   
     64    // We need scratch space to save all registers and to build up the JSStack.
     65    // Use a scratch buffer to transfer all values.
     66    ScratchBuffer* scratchBuffer = vm->scratchBufferForSize(sizeof(EncodedJSValue) * exit.m_values.size() + requiredScratchMemorySizeInBytes());
     67    EncodedJSValue* scratch = scratchBuffer ? static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer()) : 0;
     68    char* registerScratch = bitwise_cast<char*>(scratch + exit.m_values.size());
     69   
     70    saveAllRegisters(jit, registerScratch);
     71   
     72    // Bring the stack back into a sane form.
     73    jit.pop(GPRInfo::regT0);
     74   
     75    // Get the call frame and tag thingies.
     76    record->locations[0].restoreInto(jit, jitCode->stackmaps, registerScratch, GPRInfo::callFrameRegister);
     77    jit.move(MacroAssembler::TrustedImm64(TagTypeNumber), GPRInfo::tagTypeNumberRegister);
     78    jit.move(MacroAssembler::TrustedImm64(TagMask), GPRInfo::tagMaskRegister);
     79   
     80    // Do some value profiling.
     81    if (exit.m_profileValueFormat != InvalidValueFormat) {
     82        record->locations[1].restoreInto(jit, jitCode->stackmaps, registerScratch, GPRInfo::regT0);
     83        reboxAccordingToFormat(
     84            exit.m_profileValueFormat, jit, GPRInfo::regT0, GPRInfo::regT1, GPRInfo::regT2);
     85       
     86        if (exit.m_kind == BadCache || exit.m_kind == BadIndexingType) {
     87            CodeOrigin codeOrigin = exit.m_codeOriginForExitProfile;
     88            if (ArrayProfile* arrayProfile = jit.baselineCodeBlockFor(codeOrigin)->getArrayProfile(codeOrigin.bytecodeIndex)) {
     89                jit.loadPtr(MacroAssembler::Address(GPRInfo::regT0, JSCell::structureOffset()), GPRInfo::regT1);
     90                jit.storePtr(GPRInfo::regT1, arrayProfile->addressOfLastSeenStructure());
     91                jit.load8(MacroAssembler::Address(GPRInfo::regT1, Structure::indexingTypeOffset()), GPRInfo::regT1);
     92                jit.move(MacroAssembler::TrustedImm32(1), GPRInfo::regT2);
     93                jit.lshift32(GPRInfo::regT1, GPRInfo::regT2);
     94                jit.or32(GPRInfo::regT2, MacroAssembler::AbsoluteAddress(arrayProfile->addressOfArrayModes()));
     95            }
     96        }
     97       
     98        if (!!exit.m_valueProfile)
     99            jit.store64(GPRInfo::regT0, exit.m_valueProfile.getSpecFailBucket(0));
     100    }
     101
     102    // Save all state from wherever the exit data tells us it was, into the appropriate place in
     103    // the scratch buffer. This doesn't rebox any values yet.
     104   
     105    for (unsigned index = exit.m_values.size(); index--;) {
     106        ExitValue value = exit.m_values[index];
     107       
     108        switch (value.kind()) {
     109        case ExitValueDead:
     110            jit.move(MacroAssembler::TrustedImm64(JSValue::encode(jsUndefined())), GPRInfo::regT0);
     111            break;
     112           
     113        case ExitValueConstant:
     114            jit.move(MacroAssembler::TrustedImm64(JSValue::encode(value.constant())), GPRInfo::regT0);
     115            break;
     116           
     117        case ExitValueArgument:
     118            record->locations[value.exitArgument().argument()].restoreInto(
     119                jit, jitCode->stackmaps, registerScratch, GPRInfo::regT0);
     120            break;
     121           
     122        case ExitValueInJSStack:
     123        case ExitValueInJSStackAsInt32:
     124        case ExitValueInJSStackAsInt52:
     125        case ExitValueInJSStackAsDouble:
     126            jit.load64(AssemblyHelpers::addressFor(value.virtualRegister()), GPRInfo::regT0);
     127            break;
     128           
     129        default:
     130            RELEASE_ASSERT_NOT_REACHED();
     131            break;
     132        }
     133       
     134        jit.store64(GPRInfo::regT0, scratch + index);
     135    }
     136   
     137    // Now get state out of the scratch buffer and place it back into the stack. This part does
     138    // all reboxing.
     139    for (unsigned index = exit.m_values.size(); index--;) {
     140        int operand = exit.m_values.operandForIndex(index);
     141        ExitValue value = exit.m_values[index];
     142       
     143        jit.load64(scratch + index, GPRInfo::regT0);
     144        reboxAccordingToFormat(
     145            value.valueFormat(), jit, GPRInfo::regT0, GPRInfo::regT1, GPRInfo::regT2);
     146        jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor(operand));
     147    }
     148   
     149    handleExitCounts(jit, exit);
     150    reifyInlinedCallFrames(jit, exit);
     151   
     152    jit.move(MacroAssembler::framePointerRegister, MacroAssembler::stackPointerRegister);
     153    jit.pop(MacroAssembler::framePointerRegister);
     154    jit.pop(GPRInfo::nonArgGPR0); // ignore the result.
     155   
     156    if (exit.m_lastSetOperand.isValid()) {
     157        jit.load64(
     158            AssemblyHelpers::addressFor(exit.m_lastSetOperand), GPRInfo::cachedResultRegister);
     159    }
     160   
     161    adjustAndJumpToTarget(jit, exit);
     162   
     163    LinkBuffer patchBuffer(*vm, &jit, codeBlock);
     164    exit.m_code = FINALIZE_CODE_IF(
     165        shouldShowDisassembly(),
     166        patchBuffer,
     167        ("FTL OSR exit #%u (bc#%u, %s) from %s, with operands = %s",
     168            exitID, exit.m_codeOrigin.bytecodeIndex,
     169            exitKindToString(exit.m_kind), toCString(*codeBlock).data(),
     170            toCString(ignoringContext<DumpContext>(exit.m_values)).data()));
     171}
     172
     173static void compileStubWithoutOSRExitStackmap(
    45174    unsigned exitID, OSRExit& exit, VM* vm, CodeBlock* codeBlock)
    46175{
    … …  
    220349    DeferGCForAWhile deferGC(vm->heap);
    221350
    222     OSRExit& exit = codeBlock->jitCode()->ftl()->osrExit[exitID];
     351    JITCode* jitCode = codeBlock->jitCode()->ftl();
     352    OSRExit& exit = jitCode->osrExit[exitID];
    223353   
    224354    prepareCodeOriginForOSRExit(exec, exit.m_codeOrigin);
    225355   
    226     compileStub(exitID, exit, vm, codeBlock);
     356    if (Options::ftlOSRExitUsesStackmap())
     357        compileStubWithOSRExitStackmap(exitID, jitCode, exit, vm, codeBlock);
     358    else
     359        compileStubWithoutOSRExitStackmap(exitID, exit, vm, codeBlock);
    227360   
    228361    RepatchBuffer repatchBuffer(codeBlock);

  • trunk/Source/JavaScriptCore/ftl/FTLState.h

    r153174 r157209  
    3737#include "FTLJITFinalizer.h"
    3838#include "FTLOSRExitCompilationInfo.h"
     39#include "FTLStackMaps.h"
    3940#include <wtf/Noncopyable.h>
    4041
    … …  
    5859    GeneratedFunction generatedFunction;
    5960    JITFinalizer* finalizer;
     61    Vector<CString> codeSectionNames;
     62    Vector<CString> dataSectionNames;
     63    RefCountedArray<LSectionWord> stackmapsSection;
    6064   
    6165    void dumpState(const char* when);

  • trunk/Source/JavaScriptCore/ftl/FTLThunks.cpp

    r156184 r157209  
    2929#if ENABLE(FTL_JIT)
    3030
     31#include "AssemblyHelpers.h"
    3132#include "FPRInfo.h"
    3233#include "FTLOSRExitCompiler.h"
     34#include "FTLSaveRestore.h"
    3335#include "GPRInfo.h"
    3436#include "LinkBuffer.h"
    35 #include "MacroAssembler.h"
    3637
    3738namespace JSC { namespace FTL {
    … …  
    4142MacroAssemblerCodeRef osrExitGenerationThunkGenerator(VM* vm)
    4243{
    43     MacroAssembler jit;
     44    AssemblyHelpers jit(vm, 0);
     45   
     46    // Note that in the ftlOSRExitUsesStackmap() case, the "return address" will be the
     47    // OSR exit ID.
    4448   
    4549    // Pretend that we're a C call frame.
    … …  
    4953    jit.push(GPRInfo::regT0);
    5054   
    51     size_t scratchSize = sizeof(EncodedJSValue) * (GPRInfo::numberOfArgumentRegisters + FPRInfo::numberOfArgumentRegisters);
    52     ScratchBuffer* scratchBuffer = vm->scratchBufferForSize(scratchSize);
    53     EncodedJSValue* buffer = static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer());
     55    if (!Options::ftlOSRExitUsesStackmap())
     56        jit.poke(GPRInfo::nonArgGPR0, 1);
    5457   
    55     for (unsigned i = 0; i < GPRInfo::numberOfArgumentRegisters; ++i)
    56         jit.store64(GPRInfo::toArgumentRegister(i), buffer + i);
    57     for (unsigned i = 0; i < FPRInfo::numberOfArgumentRegisters; ++i) {
    58         jit.move(MacroAssembler::TrustedImmPtr(buffer + GPRInfo::numberOfArgumentRegisters + i), GPRInfo::nonArgGPR1);
    59         jit.storeDouble(FPRInfo::toArgumentRegister(i), GPRInfo::nonArgGPR1);
    60     }
     58    ScratchBuffer* scratchBuffer = vm->scratchBufferForSize(requiredScratchMemorySizeInBytes());
     59    char* buffer = static_cast<char*>(scratchBuffer->dataBuffer());
     60   
     61    saveAllRegisters(jit, buffer);
    6162   
    6263    // Tell GC mark phase how much of the scratch buffer is active during call.
    6364    jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->activeLengthPtr()), GPRInfo::nonArgGPR1);
    64     jit.storePtr(MacroAssembler::TrustedImmPtr(scratchSize), GPRInfo::nonArgGPR1);
     65    jit.storePtr(MacroAssembler::TrustedImmPtr(requiredScratchMemorySizeInBytes()), GPRInfo::nonArgGPR1);
    6566
    6667    // argument 0 is already the call frame.
    67     jit.move(GPRInfo::nonArgGPR0, GPRInfo::argumentGPR1);
     68    if (Options::ftlOSRExitUsesStackmap())
     69        jit.peek(GPRInfo::argumentGPR1, 3);
     70    else
     71        jit.peek(GPRInfo::argumentGPR1, 1);
    6872    MacroAssembler::Call functionCall = jit.call();
    6973   
    70     // Make sure that we're not using the return register if it's an argument register.
    71     jit.move(GPRInfo::returnValueGPR, GPRInfo::nonArgGPR0);
     74    // At this point we want to make a tail call to what was returned to us in the
     75    // returnValueGPR. But at the same time as we do this, we must restore all registers.
     76    // The way we will accomplish this is by arranging to have the tail call target in the
     77    // return address "slot" (be it a register or the stack).
     78   
     79    jit.move(GPRInfo::returnValueGPR, GPRInfo::regT0);
    7280   
    7381    // Prepare for tail call.
    74     jit.pop(GPRInfo::nonArgGPR1);
    75     jit.pop(GPRInfo::nonArgGPR1);
     82    jit.pop(GPRInfo::regT1);
     83    jit.pop(GPRInfo::regT1);
    7684    jit.pop(MacroAssembler::framePointerRegister);
    7785   
    78     jit.move(MacroAssembler::TrustedImmPtr(scratchBuffer->activeLengthPtr()), GPRInfo::nonArgGPR1);
    79     jit.storePtr(MacroAssembler::TrustedImmPtr(0), GPRInfo::nonArgGPR1);
     86    // At this point we're sitting on the return address - so if we did a jump right now, the
     87    // tail-callee would be happy. Instead we'll stash the callee in the return address and then
     88    // restore all registers.
    8089   
    81     for (unsigned i = 0; i < FPRInfo::numberOfArgumentRegisters; ++i) {
    82         jit.move(MacroAssembler::TrustedImmPtr(buffer + GPRInfo::numberOfArgumentRegisters + i), GPRInfo::nonArgGPR1);
    83         jit.loadDouble(GPRInfo::nonArgGPR1, FPRInfo::toArgumentRegister(i));
    84     }
    85     for (unsigned i = 0; i < GPRInfo::numberOfArgumentRegisters; ++i)
    86         jit.load64(buffer + i, GPRInfo::toArgumentRegister(i));
     90    jit.restoreReturnAddressBeforeReturn(GPRInfo::regT0);
    8791   
    88     jit.jump(GPRInfo::nonArgGPR0);
     92    restoreAllRegisters(jit, buffer);
     93
     94    jit.ret();
    8995   
    9096    LinkBuffer patchBuffer(*vm, &jit, GLOBAL_THUNK_ID);

  • trunk/Source/JavaScriptCore/ftl/FTLValueFormat.cpp

    r156047 r157209  
    2828
    2929#if ENABLE(FTL_JIT)
     30
     31#include "AssemblyHelpers.h"
     32
     33namespace JSC { namespace FTL {
     34
     35void reboxAccordingToFormat(
     36    ValueFormat format, AssemblyHelpers& jit, GPRReg value, GPRReg scratch1, GPRReg scratch2)
     37{
     38    switch (format) {
     39    case ValueFormatInt32: {
     40        jit.or64(GPRInfo::tagTypeNumberRegister, value);
     41        break;
     42    }
     43   
     44    case ValueFormatUInt32: {
     45        jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
     46        jit.boxInt52(value, value, scratch1, FPRInfo::fpRegT0);
     47        jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
     48        break;
     49    }
     50   
     51    case ValueFormatInt52: {
     52        jit.rshift64(AssemblyHelpers::TrustedImm32(JSValue::int52ShiftAmount), value);
     53        jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
     54        jit.boxInt52(value, value, scratch1, FPRInfo::fpRegT0);
     55        jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
     56        break;
     57    }
     58   
     59    case ValueFormatStrictInt52: {
     60        jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch2);
     61        jit.boxInt52(value, value, scratch1, FPRInfo::fpRegT0);
     62        jit.move64ToDouble(scratch2, FPRInfo::fpRegT0);
     63        break;
     64    }
     65   
     66    case ValueFormatBoolean: {
     67        jit.or32(MacroAssembler::TrustedImm32(ValueFalse), value);
     68        break;
     69    }
     70   
     71    case ValueFormatJSValue: {
     72        // Done already!
     73        break;
     74    }
     75   
     76    case ValueFormatDouble: {
     77        jit.moveDoubleTo64(FPRInfo::fpRegT0, scratch1);
     78        jit.move64ToDouble(value, FPRInfo::fpRegT0);
     79        jit.boxDouble(FPRInfo::fpRegT0, value);
     80        jit.move64ToDouble(scratch1, FPRInfo::fpRegT0);
     81        break;
     82    }
     83   
     84    default:
     85        RELEASE_ASSERT_NOT_REACHED();
     86        break;
     87    }
     88}
     89
     90} } // namespace JSC::FTL
    3091
    3192namespace WTF {

  • trunk/Source/JavaScriptCore/ftl/FTLValueFormat.h

    r156047 r157209  
    3131#if ENABLE(FTL_JIT)
    3232
     33#include "GPRInfo.h"
    3334#include <wtf/PrintStream.h>
    3435
    35 namespace JSC { namespace FTL {
     36namespace JSC {
     37
     38class AssemblyHelpers;
     39
     40namespace FTL {
    3641
    3742// Note that this is awkwardly similar to DataFormat in other parts of JSC, except that
    … …  
    4954};
    5055
     56void reboxAccordingToFormat(
     57    ValueFormat, AssemblyHelpers&, GPRReg value, GPRReg scratch1, GPRReg scratch2);
     58
    5159} } // namespace JSC::FTL
    5260

  • trunk/Source/JavaScriptCore/runtime/DataView.cpp

    r154127 r157209  
    4444}
    4545
     46PassRefPtr<DataView> DataView::create(PassRefPtr<ArrayBuffer> passedBuffer)
     47{
     48    RefPtr<ArrayBuffer> buffer = passedBuffer;
     49    return create(buffer, 0, buffer->byteLength());
     50}
     51
    4652JSArrayBufferView* DataView::wrap(ExecState* exec, JSGlobalObject* globalObject)
    4753{

  • trunk/Source/JavaScriptCore/runtime/DataView.h

    r157044 r157209  
    3939public:
    4040    JS_EXPORT_PRIVATE static PassRefPtr<DataView> create(PassRefPtr<ArrayBuffer>, unsigned byteOffset, unsigned length);
     41    static PassRefPtr<DataView> create(PassRefPtr<ArrayBuffer>);
    4142   
    4243    virtual unsigned byteLength() const OVERRIDE
    … …  
    6970   
    7071    template<typename T>
     72    T read(unsigned& offset, bool littleEndian, bool* status = 0)
     73    {
     74        T result = this->template get<T>(offset, littleEndian, status);
     75        if (!status || *status)
     76            offset += sizeof(T);
     77        return result;
     78    }
     79   
     80    template<typename T>
    7181    void set(unsigned offset, T value, bool littleEndian, bool* status = 0)
    7282    {

  • trunk/Source/JavaScriptCore/runtime/Options.h

    r156064 r157209  
    124124    v(bool, ftlTrapsOnOSRExit, false) \
    125125    v(bool, ftlOSRExitOmitsMarshalling, false) \
     126    v(bool, ftlOSRExitUsesStackmap, false) \
    126127    v(bool, useLLVMOSRExitIntrinsic, false) \
    127128    v(bool, dumpLLVMIR, false) \
    128     v(bool, llvmAlwaysFails, false) \
     129    v(bool, llvmAlwaysFailsBeforeCompile, false) \
     130    v(bool, llvmAlwaysFailsBeforeLink, false) \
    129131    v(unsigned, llvmBackendOptimizationLevel, 2) \
    130132    v(unsigned, llvmOptimizationLevel, 2) \
Note: See TracChangeset for help on using the changeset viewer.