Changeset 206289 in webkit


Ignore:
Timestamp:
Sep 22, 2016 8:50:22 PM (8 years ago)
Author:
commit-queue@webkit.org
Message:

[JSC] Use an inline cache to generate op_negate
​https://bugs.webkit.org/show_bug.cgi?id=162371

Patch by Benjamin Poulain <​bpoulain@apple.com> on 2016-09-22
Reviewed by Saam Barati.

JSTests:

  • stress/op-negate-inline-cache.js: Added.

Source/JavaScriptCore:

Use an inline cache to reduce the amount of code
required to implement op_negate.

For pure integer negate, the generated asm shrinks
from 147 bytes to 125 bytes (14%).
For double negate, the generated asm shrinks
to 130 bytes (11%).
The average size on Sunspider is 100bytes, this is due
to the op_negates that are never executed and do not
generate much.

  • bytecode/ArithProfile.h:

(JSC::ArithProfile::ArithProfile):
(JSC::ArithProfile::observeLHS):
(JSC::ArithProfile::observeLHSAndRHS):

  • bytecode/BytecodeList.json:
  • bytecode/CodeBlock.cpp:

(JSC::CodeBlock::addJITNegIC):

  • bytecode/CodeBlock.h:
  • bytecompiler/BytecodeGenerator.cpp:

(JSC::BytecodeGenerator::emitUnaryOp):

  • bytecompiler/BytecodeGenerator.h:
  • bytecompiler/NodesCodegen.cpp:

(JSC::UnaryOpNode::emitBytecode):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::compileMathIC):

  • dfg/DFGSpeculativeJIT.h:
  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileMathIC):

  • jit/CCallHelpers.h:

(JSC::CCallHelpers::setupArgumentsWithExecState):

  • jit/JIT.h:
  • jit/JITArithmetic.cpp:

(JSC::JIT::emit_op_negate):
(JSC::JIT::emitSlow_op_negate):
(JSC::JIT::emitMathICFast):
(JSC::JIT::emitMathICSlow):

  • jit/JITInlines.h:

(JSC::JIT::callOperation):

  • jit/JITMathIC.h:

(JSC::JITMathIC::generateInline):
(JSC::canGenerateWithBinaryProfile):
(JSC::canGenerateWithUnaryProfile):

  • jit/JITMathICForwards.h:
  • jit/JITNegGenerator.cpp:

(JSC::JITNegGenerator::generateInline):
(JSC::JITNegGenerator::generateFastPath):

  • jit/JITNegGenerator.h:

(JSC::JITNegGenerator::JITNegGenerator):
(JSC::JITNegGenerator::arithProfile):
(JSC::JITNegGenerator::didEmitFastPath): Deleted.
(JSC::JITNegGenerator::endJumpList): Deleted.
(JSC::JITNegGenerator::slowPathJumpList): Deleted.

  • jit/JITOperations.cpp:
  • jit/JITOperations.h:
  • llint/LowLevelInterpreter.asm:
  • llint/LowLevelInterpreter32_64.asm:
  • llint/LowLevelInterpreter64.asm:
  • runtime/CommonSlowPaths.cpp:

(JSC::updateArithProfileForUnaryArithOp):
(JSC::SLOW_PATH_DECL):

Location:
trunk
Files:
1 added
26 edited

Legend:

Unmodified
Added
Removed

  • trunk/JSTests/ChangeLog

    r206281 r206289  
     12016-09-22  Benjamin Poulain  <bpoulain@apple.com>
     2
     3        [JSC] Use an inline cache to generate op_negate
     4        https://bugs.webkit.org/show_bug.cgi?id=162371
     5
     6        Reviewed by Saam Barati.
     7
     8        * stress/op-negate-inline-cache.js: Added.
     9
    1102016-09-22  Mark Lam  <mark.lam@apple.com>
    211

  • trunk/Source/JavaScriptCore/ChangeLog

    r206284 r206289  
     12016-09-22  Benjamin Poulain  <bpoulain@apple.com>
     2
     3        [JSC] Use an inline cache to generate op_negate
     4        https://bugs.webkit.org/show_bug.cgi?id=162371
     5
     6        Reviewed by Saam Barati.
     7
     8        Use an inline cache to reduce the amount of code
     9        required to implement op_negate.
     10
     11        For pure integer negate, the generated asm shrinks
     12        from 147 bytes to 125 bytes (14%).
     13        For double negate, the generated asm shrinks
     14        to 130 bytes (11%).
     15        The average size on Sunspider is 100bytes, this is due
     16        to the op_negates that are never executed and do not
     17        generate much.
     18
     19        * bytecode/ArithProfile.h:
     20        (JSC::ArithProfile::ArithProfile):
     21        (JSC::ArithProfile::observeLHS):
     22        (JSC::ArithProfile::observeLHSAndRHS):
     23        * bytecode/BytecodeList.json:
     24        * bytecode/CodeBlock.cpp:
     25        (JSC::CodeBlock::addJITNegIC):
     26        * bytecode/CodeBlock.h:
     27        * bytecompiler/BytecodeGenerator.cpp:
     28        (JSC::BytecodeGenerator::emitUnaryOp):
     29        * bytecompiler/BytecodeGenerator.h:
     30        * bytecompiler/NodesCodegen.cpp:
     31        (JSC::UnaryOpNode::emitBytecode):
     32        * dfg/DFGSpeculativeJIT.cpp:
     33        (JSC::DFG::SpeculativeJIT::compileMathIC):
     34        * dfg/DFGSpeculativeJIT.h:
     35        * ftl/FTLLowerDFGToB3.cpp:
     36        (JSC::FTL::DFG::LowerDFGToB3::compileMathIC):
     37        * jit/CCallHelpers.h:
     38        (JSC::CCallHelpers::setupArgumentsWithExecState):
     39        * jit/JIT.h:
     40        * jit/JITArithmetic.cpp:
     41        (JSC::JIT::emit_op_negate):
     42        (JSC::JIT::emitSlow_op_negate):
     43        (JSC::JIT::emitMathICFast):
     44        (JSC::JIT::emitMathICSlow):
     45        * jit/JITInlines.h:
     46        (JSC::JIT::callOperation):
     47        * jit/JITMathIC.h:
     48        (JSC::JITMathIC::generateInline):
     49        (JSC::canGenerateWithBinaryProfile):
     50        (JSC::canGenerateWithUnaryProfile):
     51        * jit/JITMathICForwards.h:
     52        * jit/JITNegGenerator.cpp:
     53        (JSC::JITNegGenerator::generateInline):
     54        (JSC::JITNegGenerator::generateFastPath):
     55        * jit/JITNegGenerator.h:
     56        (JSC::JITNegGenerator::JITNegGenerator):
     57        (JSC::JITNegGenerator::arithProfile):
     58        (JSC::JITNegGenerator::didEmitFastPath): Deleted.
     59        (JSC::JITNegGenerator::endJumpList): Deleted.
     60        (JSC::JITNegGenerator::slowPathJumpList): Deleted.
     61        * jit/JITOperations.cpp:
     62        * jit/JITOperations.h:
     63        * llint/LowLevelInterpreter.asm:
     64        * llint/LowLevelInterpreter32_64.asm:
     65        * llint/LowLevelInterpreter64.asm:
     66        * runtime/CommonSlowPaths.cpp:
     67        (JSC::updateArithProfileForUnaryArithOp):
     68        (JSC::SLOW_PATH_DECL):
     69
    1702016-09-22  Mark Lam  <mark.lam@apple.com>
    271

  • trunk/Source/JavaScriptCore/bytecode/ArithProfile.h

    r204373 r206289  
    8888    static_assert(specialFastPathBit > ~clearLhsObservedTypeBitMask, "These bits should not intersect and specialFastPathBit should be a higher bit.");
    8989
     90    ArithProfile(ResultType arg)
     91    {
     92        m_bits = (arg.bits() << lhsResultTypeShift);
     93        ASSERT(lhsResultType().bits() == arg.bits());
     94        ASSERT(lhsObservedType().isEmpty());
     95        ASSERT(rhsObservedType().isEmpty());
     96    }
     97
    9098    ArithProfile(ResultType lhs, ResultType rhs)
    9199    {
    … …  
    95103        ASSERT(rhsObservedType().isEmpty());
    96104    }
    97     ArithProfile() { }
     105    ArithProfile() = default;
    98106
    99107    static ArithProfile fromInt(uint32_t bits)
    … …  
    171179    void rhsSawNonNumber() { setRhsObservedType(rhsObservedType().withNonNumber()); }
    172180
    173     void observeLHSAndRHS(JSValue lhs, JSValue rhs)
     181    void observeLHS(JSValue lhs)
    174182    {
    175183        ArithProfile newProfile = *this;
    … …  
    182190            newProfile.lhsSawNonNumber();
    183191
     192        m_bits = newProfile.bits();
     193    }
     194
     195    void observeLHSAndRHS(JSValue lhs, JSValue rhs)
     196    {
     197        observeLHS(lhs);
     198
     199        ArithProfile newProfile = *this;
    184200        if (rhs.isNumber()) {
    185201            if (rhs.isInt32())

  • trunk/Source/JavaScriptCore/bytecode/BytecodeList.json

    r206098 r206289  
    3434            { "name" : "op_to_number", "length" : 4 },
    3535            { "name" : "op_to_string", "length" : 3 },
    36             { "name" : "op_negate", "length" : 3 },
     36            { "name" : "op_negate", "length" : 4 },
    3737            { "name" : "op_add", "length" : 5 },
    3838            { "name" : "op_mul", "length" : 5 },

  • trunk/Source/JavaScriptCore/bytecode/CodeBlock.cpp

    r206098 r206289  
    30143014{
    30153015    return m_subICs.add();
     3016}
     3017
     3018JITNegIC* CodeBlock::addJITNegIC()
     3019{
     3020    return m_negICs.add();
    30163021}
    30173022
    … …  
    43554360ArithProfile* CodeBlock::arithProfileForBytecodeOffset(int bytecodeOffset)
    43564361{
    4357     auto opcodeID = vm()->interpreter->getOpcodeID(instructions()[bytecodeOffset].u.opcode);
     4362    return arithProfileForPC(instructions().begin() + bytecodeOffset);
     4363}
     4364
     4365ArithProfile* CodeBlock::arithProfileForPC(Instruction* pc)
     4366{
     4367    auto opcodeID = vm()->interpreter->getOpcodeID(pc[0].u.opcode);
    43584368    switch (opcodeID) {
     4369    case op_negate:
     4370        return bitwise_cast<ArithProfile*>(&pc[3].u.operand);
    43594371    case op_bitor:
    43604372    case op_bitand:
    … …  
    43644376    case op_sub:
    43654377    case op_div:
     4378        return bitwise_cast<ArithProfile*>(&pc[4].u.operand);
     4379    default:
    43664380        break;
    4367     default:
    4368         return nullptr;
    4369     }
    4370 
    4371     return &arithProfileForPC(instructions().begin() + bytecodeOffset);
    4372 }
    4373 
    4374 ArithProfile& CodeBlock::arithProfileForPC(Instruction* pc)
    4375 {
    4376     if (!ASSERT_DISABLED) {
    4377         ASSERT(pc >= instructions().begin() && pc < instructions().end());
    4378         auto opcodeID = vm()->interpreter->getOpcodeID(pc[0].u.opcode);
    4379         switch (opcodeID) {
    4380         case op_bitor:
    4381         case op_bitand:
    4382         case op_bitxor:
    4383         case op_add:
    4384         case op_mul:
    4385         case op_sub:
    4386         case op_div:
    4387             break;
    4388         default:
    4389             ASSERT_NOT_REACHED();
    4390         }
    4391     }
    4392 
    4393     return *bitwise_cast<ArithProfile*>(&pc[4].u.operand);
     4381    }
     4382
     4383    return nullptr;
    43944384}
    43954385
    … …  
    45634553    double numMuls = 0.0;
    45644554    double totalMulSize = 0.0;
     4555    double numNegs = 0.0;
     4556    double totalNegSize = 0.0;
    45654557    double numSubs = 0.0;
    45664558    double totalSubSize = 0.0;
    … …  
    45754567            numMuls++;
    45764568            totalMulSize += mulIC->codeSize();
     4569        }
     4570
     4571        for (JITNegIC* negIC : codeBlock->m_negICs) {
     4572            numNegs++;
     4573            totalNegSize += negIC->codeSize();
    45774574        }
    45784575
    … …  
    45944591    dataLog("Average Mul size: ", totalMulSize / numMuls, "\n");
    45954592    dataLog("\n");
     4593    dataLog("Num Negs: ", numNegs, "\n");
     4594    dataLog("Total Neg size in bytes: ", totalNegSize, "\n");
     4595    dataLog("Average Neg size: ", totalNegSize / numNegs, "\n");
     4596    dataLog("\n");
    45964597    dataLog("Num Subs: ", numSubs, "\n");
    45974598    dataLog("Total Sub size in bytes: ", totalSubSize, "\n");

  • trunk/Source/JavaScriptCore/bytecode/CodeBlock.h

    r206267 r206289  
    246246    JITAddIC* addJITAddIC();
    247247    JITMulIC* addJITMulIC();
     248    JITNegIC* addJITNegIC();
    248249    JITSubIC* addJITSubIC();
    249250    Bag<StructureStubInfo>::iterator stubInfoBegin() { return m_stubInfos.begin(); }
    … …  
    448449
    449450    ArithProfile* arithProfileForBytecodeOffset(int bytecodeOffset);
    450     ArithProfile& arithProfileForPC(Instruction*);
     451    ArithProfile* arithProfileForPC(Instruction*);
    451452
    452453    bool couldTakeSpecialFastCase(int bytecodeOffset);
    … …  
    998999    Bag<JITAddIC> m_addICs;
    9991000    Bag<JITMulIC> m_mulICs;
     1001    Bag<JITNegIC> m_negICs;
    10001002    Bag<JITSubIC> m_subICs;
    10011003    Bag<ByValInfo> m_byValInfos;

  • trunk/Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp

    r206147 r206289  
    15801580RegisterID* BytecodeGenerator::emitUnaryOp(OpcodeID opcodeID, RegisterID* dst, RegisterID* src)
    15811581{
    1582     ASSERT_WITH_MESSAGE(op_to_number != opcodeID, "op_to_number is profiled.");
     1582    ASSERT_WITH_MESSAGE(op_to_number != opcodeID, "op_to_number has a Value Profile.");
     1583    ASSERT_WITH_MESSAGE(op_negate != opcodeID, "op_negate has an Arith Profile.");
    15831584    emitOpcode(opcodeID);
    15841585    instructions().append(dst->index());
    15851586    instructions().append(src->index());
     1587
     1588    return dst;
     1589}
     1590
     1591RegisterID* BytecodeGenerator::emitUnaryOp(OpcodeID opcodeID, RegisterID* dst, RegisterID* src, OperandTypes types)
     1592{
     1593    ASSERT_WITH_MESSAGE(op_to_number != opcodeID, "op_to_number has a Value Profile.");
     1594    emitOpcode(opcodeID);
     1595    instructions().append(dst->index());
     1596    instructions().append(src->index());
     1597
     1598    if (opcodeID == op_negate)
     1599        instructions().append(ArithProfile(types.first()).bits());
    15861600    return dst;
    15871601}

  • trunk/Source/JavaScriptCore/bytecompiler/BytecodeGenerator.h

    r206104 r206289  
    512512
    513513        RegisterID* emitUnaryOp(OpcodeID, RegisterID* dst, RegisterID* src);
     514        RegisterID* emitUnaryOp(OpcodeID, RegisterID* dst, RegisterID* src, OperandTypes);
    514515        RegisterID* emitUnaryOpProfiled(OpcodeID, RegisterID* dst, RegisterID* src);
    515516        RegisterID* emitBinaryOp(OpcodeID, RegisterID* dst, RegisterID* src1, RegisterID* src2, OperandTypes);

  • trunk/Source/JavaScriptCore/bytecompiler/NodesCodegen.cpp

    r206267 r206289  
    16071607    RefPtr<RegisterID> src = generator.emitNode(m_expr);
    16081608    generator.emitExpressionInfo(position(), position(), position());
    1609     return generator.emitUnaryOp(opcodeID(), generator.finalDestination(dst), src.get());
     1609    return generator.emitUnaryOp(opcodeID(), generator.finalDestination(dst), src.get(), OperandTypes(m_expr->resultDescriptor()));
    16101610}
    16111611

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r206134 r206289  
    34333433
    34343434template <typename Generator, typename RepatchingFunction, typename NonRepatchingFunction>
    3435 void SpeculativeJIT::compileMathIC(Node* node, JITMathIC<Generator>* mathIC, bool needsScratchGPRReg, bool needsScratchFPRReg, RepatchingFunction repatchingFunction, NonRepatchingFunction nonRepatchingFunction)
     3435void SpeculativeJIT::compileMathIC(Node* node, JITBinaryMathIC<Generator>* mathIC, bool needsScratchGPRReg, bool needsScratchFPRReg, RepatchingFunction repatchingFunction, NonRepatchingFunction nonRepatchingFunction)
    34363436{
    34373437    Edge& leftChild = node->child1();

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

    r206136 r206289  
    25162516
    25172517    template <typename Generator, typename RepatchingFunction, typename NonRepatchingFunction>
    2518     void compileMathIC(Node*, JITMathIC<Generator>*, bool needsScratchGPRReg, bool needsScratchFPRReg, RepatchingFunction, NonRepatchingFunction);
     2518    void compileMathIC(Node*, JITBinaryMathIC<Generator>*, bool needsScratchGPRReg, bool needsScratchFPRReg, RepatchingFunction, NonRepatchingFunction);
    25192519
    25202520    void compileArithDoubleUnaryOp(Node*, double (*doubleFunction)(double), double (*operation)(ExecState*, EncodedJSValue));

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

    r206212 r206289  
    15611561
    15621562    template <typename Generator>
    1563     void compileMathIC(JITMathIC<Generator>* mathIC, FunctionPtr repatchingFunction, FunctionPtr nonRepatchingFunction)
     1563    void compileMathIC(JITBinaryMathIC<Generator>* mathIC, FunctionPtr repatchingFunction, FunctionPtr nonRepatchingFunction)
    15641564    {
    15651565        Node* node = m_node;

  • trunk/Source/JavaScriptCore/jit/CCallHelpers.h

    r205462 r206289  
    353353
    354354    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, TrustedImmPtr arg3, TrustedImm32 arg4)
     355    {
     356        resetCallArguments();
     357        addCallArgument(GPRInfo::callFrameRegister);
     358        addCallArgument(arg1);
     359        addCallArgument(arg2);
     360        addCallArgument(arg3);
     361        addCallArgument(arg4);
     362    }
     363
     364    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, TrustedImmPtr arg3, TrustedImmPtr arg4)
    355365    {
    356366        resetCallArguments();
    … …  
    22672277    }
    22682278#endif
     2279
     2280    void setupArgumentsWithExecState(JSValueRegs arg)
     2281    {
     2282#if USE(JSVALUE64)
     2283        setupArgumentsWithExecState(arg.gpr());
     2284#else
     2285        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg.payloadGPR(), arg.tagGPR());
     2286#endif
     2287    }
    22692288   
    22702289    void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2)
    … …  
    22772296    }
    22782297
     2298    void setupArgumentsWithExecState(JSValueRegs arg1, TrustedImmPtr arg2)
     2299    {
     2300#if USE(JSVALUE64)
     2301        setupArgumentsWithExecState(arg1.gpr(), arg2);
     2302#else
     2303        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2);
     2304#endif
     2305    }
     2306
    22792307    void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3)
    22802308    {
    … …  
    22922320#else
    22932321        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3, arg4);
     2322#endif
     2323    }
     2324
     2325    void setupArgumentsWithExecState(JSValueRegs arg1, TrustedImmPtr arg2, TrustedImmPtr arg3)
     2326    {
     2327#if USE(JSVALUE64)
     2328        setupArgumentsWithExecState(arg1.gpr(), arg2, arg3);
     2329#else
     2330        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2, arg3);
    22942331#endif
    22952332    }

  • trunk/Source/JavaScriptCore/jit/JIT.h

    r206267 r206289  
    698698
    699699        template <typename Generator, typename ProfiledFunction, typename NonProfiledFunction>
    700         void emitMathICFast(JITMathIC<Generator>*, Instruction*, ProfiledFunction, NonProfiledFunction);
     700        void emitMathICFast(JITUnaryMathIC<Generator>*, Instruction*, ProfiledFunction, NonProfiledFunction);
     701        template <typename Generator, typename ProfiledFunction, typename NonProfiledFunction>
     702        void emitMathICFast(JITBinaryMathIC<Generator>*, Instruction*, ProfiledFunction, NonProfiledFunction);
    701703
    702704        template <typename Generator, typename ProfiledRepatchFunction, typename ProfiledFunction, typename RepatchFunction>
    703         void emitMathICSlow(JITMathIC<Generator>*, Instruction*, ProfiledRepatchFunction, ProfiledFunction, RepatchFunction);
     705        void emitMathICSlow(JITBinaryMathIC<Generator>*, Instruction*, ProfiledRepatchFunction, ProfiledFunction, RepatchFunction);
     706        template <typename Generator, typename ProfiledRepatchFunction, typename ProfiledFunction, typename RepatchFunction>
     707        void emitMathICSlow(JITUnaryMathIC<Generator>*, Instruction*, ProfiledRepatchFunction, ProfiledFunction, RepatchFunction);
    704708
    705709        Jump getSlowCase(Vector<SlowCaseEntry>::iterator& iter)
    … …  
    746750        MacroAssembler::Call callOperation(V_JITOperation_EC, JSCell*);
    747751        MacroAssembler::Call callOperation(J_JITOperation_EJ, int, GPRReg);
     752        MacroAssembler::Call callOperation(J_JITOperation_EJ, JSValueRegs, JSValueRegs);
    748753#if USE(JSVALUE64)
    749754        MacroAssembler::Call callOperation(J_JITOperation_ESsiJI, int, StructureStubInfo*, GPRReg, UniquedStringImpl*);
    … …  
    755760        MacroAssembler::Call callOperation(J_JITOperation_EJI, int, GPRReg, UniquedStringImpl*);
    756761        MacroAssembler::Call callOperation(J_JITOperation_EJJ, int, GPRReg, GPRReg);
     762        MacroAssembler::Call callOperation(J_JITOperation_EJArp, JSValueRegs, JSValueRegs, ArithProfile*);
    757763        MacroAssembler::Call callOperation(J_JITOperation_EJJArp, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*);
    758764        MacroAssembler::Call callOperation(J_JITOperation_EJJ, JSValueRegs, JSValueRegs, JSValueRegs);
    759765        MacroAssembler::Call callOperation(J_JITOperation_EJJArpMic, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*, TrustedImmPtr);
     766        MacroAssembler::Call callOperation(J_JITOperation_EJMic, JSValueRegs, JSValueRegs, TrustedImmPtr);
    760767        MacroAssembler::Call callOperation(J_JITOperation_EJJMic, JSValueRegs, JSValueRegs, JSValueRegs, TrustedImmPtr);
    761768        MacroAssembler::Call callOperation(J_JITOperation_EJJAp, int, GPRReg, GPRReg, ArrayProfile*);

  • trunk/Source/JavaScriptCore/jit/JITArithmetic.cpp

    r205364 r206289  
    470470void JIT::emit_op_negate(Instruction* currentInstruction)
    471471{
    472     int result = currentInstruction[1].u.operand;
    473     int src = currentInstruction[2].u.operand;
    474 
    475 #if USE(JSVALUE64)
    476     JSValueRegs srcRegs = JSValueRegs(regT0);
    477     JSValueRegs resultRegs = srcRegs;
    478     GPRReg scratchGPR = regT2;
    479 #else
    480     JSValueRegs srcRegs = JSValueRegs(regT1, regT0);
    481     JSValueRegs resultRegs = srcRegs;
    482     GPRReg scratchGPR = regT4;
    483 #endif
    484 
    485     emitGetVirtualRegister(src, srcRegs);
    486 
    487     JITNegGenerator gen(resultRegs, srcRegs, scratchGPR);
    488     gen.generateFastPath(*this);
    489 
    490     ASSERT(gen.didEmitFastPath());
    491     gen.endJumpList().link(this);
    492     emitPutVirtualRegister(result, resultRegs);
    493 
    494     addSlowCase(gen.slowPathJumpList());
     472    JITNegIC* negateIC = m_codeBlock->addJITNegIC();
     473    m_instructionToMathIC.add(currentInstruction, negateIC);
     474    emitMathICFast(negateIC, currentInstruction, operationArithNegateProfiled, operationArithNegate);
    495475}
    496476
    … …  
    499479    linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
    500480
    501     JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_negate);
    502     slowPathCall.call();
     481    JITNegIC* negIC = bitwise_cast<JITNegIC*>(m_instructionToMathIC.get(currentInstruction));
     482    emitMathICSlow(negIC, currentInstruction, operationArithNegateProfiledOptimize, operationArithNegateProfiled, operationArithNegateOptimize);
    503483}
    504484
    … …  
    699679
    700680template <typename Generator, typename ProfiledFunction, typename NonProfiledFunction>
    701 void JIT::emitMathICFast(JITMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledFunction profiledFunction, NonProfiledFunction nonProfiledFunction)
     681void JIT::emitMathICFast(JITUnaryMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledFunction profiledFunction, NonProfiledFunction nonProfiledFunction)
     682{
     683    int result = currentInstruction[1].u.operand;
     684    int operand = currentInstruction[2].u.operand;
     685
     686#if USE(JSVALUE64)
     687    // ArithNegate benefits from using the same register as src and dst.
     688    // Since regT1==argumentGPR1, using regT1 avoid shuffling register to call the slow path.
     689    JSValueRegs srcRegs = JSValueRegs(regT1);
     690    JSValueRegs resultRegs = JSValueRegs(regT1);
     691    GPRReg scratchGPR = regT2;
     692#else
     693    JSValueRegs srcRegs = JSValueRegs(regT1, regT0);
     694    JSValueRegs resultRegs = JSValueRegs(regT3, regT2);
     695    GPRReg scratchGPR = regT4;
     696#endif
     697
     698#if ENABLE(MATH_IC_STATS)
     699    auto inlineStart = label();
     700#endif
     701
     702    ArithProfile& arithProfile = *bitwise_cast<ArithProfile*>(&currentInstruction[3].u.operand);
     703    mathIC->m_generator = Generator(resultRegs, srcRegs, scratchGPR, arithProfile);
     704
     705    emitGetVirtualRegister(operand, srcRegs);
     706
     707    MathICGenerationState& mathICGenerationState = m_instructionToMathICGenerationState.add(currentInstruction, MathICGenerationState()).iterator->value;
     708
     709    bool generatedInlineCode = mathIC->generateInline(*this, mathICGenerationState);
     710    if (!generatedInlineCode) {
     711        if (shouldEmitProfiling())
     712            callOperation(profiledFunction, resultRegs, srcRegs, &arithProfile);
     713        else
     714            callOperation(nonProfiledFunction, resultRegs, srcRegs);
     715    } else
     716        addSlowCase(mathICGenerationState.slowPathJumps);
     717
     718#if ENABLE(MATH_IC_STATS)
     719    auto inlineEnd = label();
     720    addLinkTask([=] (LinkBuffer& linkBuffer) {
     721        size_t size = static_cast<char*>(linkBuffer.locationOf(inlineEnd).executableAddress()) - static_cast<char*>(linkBuffer.locationOf(inlineStart).executableAddress());
     722        mathIC->m_generatedCodeSize += size;
     723    });
     724#endif
     725
     726    emitPutVirtualRegister(result, resultRegs);
     727}
     728
     729template <typename Generator, typename ProfiledFunction, typename NonProfiledFunction>
     730void JIT::emitMathICFast(JITBinaryMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledFunction profiledFunction, NonProfiledFunction nonProfiledFunction)
    702731{
    703732    int result = currentInstruction[1].u.operand;
    … …  
    723752    ArithProfile* arithProfile = nullptr;
    724753    if (shouldEmitProfiling())
    725         arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
     754        arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
    726755
    727756    SnippetOperand leftOperand(types.first());
    … …  
    775804
    776805template <typename Generator, typename ProfiledRepatchFunction, typename ProfiledFunction, typename RepatchFunction>
    777 void JIT::emitMathICSlow(JITMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledRepatchFunction profiledRepatchFunction, ProfiledFunction profiledFunction, RepatchFunction repatchFunction)
     806void JIT::emitMathICSlow(JITUnaryMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledRepatchFunction profiledRepatchFunction, ProfiledFunction profiledFunction, RepatchFunction repatchFunction)
     807{
     808    MathICGenerationState& mathICGenerationState = m_instructionToMathICGenerationState.find(currentInstruction)->value;
     809    mathICGenerationState.slowPathStart = label();
     810
     811    int result = currentInstruction[1].u.operand;
     812
     813#if USE(JSVALUE64)
     814    JSValueRegs srcRegs = JSValueRegs(regT1);
     815    JSValueRegs resultRegs = JSValueRegs(regT0);
     816#else
     817    JSValueRegs srcRegs = JSValueRegs(regT1, regT0);
     818    JSValueRegs resultRegs = JSValueRegs(regT3, regT2);
     819#endif
     820
     821#if ENABLE(MATH_IC_STATS)
     822    auto slowPathStart = label();
     823#endif
     824
     825    if (shouldEmitProfiling()) {
     826        ArithProfile* arithProfile = bitwise_cast<ArithProfile*>(&currentInstruction[3].u.operand);
     827        if (mathICGenerationState.shouldSlowPathRepatch)
     828            mathICGenerationState.slowPathCall = callOperation(reinterpret_cast<J_JITOperation_EJMic>(profiledRepatchFunction), resultRegs, srcRegs, TrustedImmPtr(mathIC));
     829        else
     830            mathICGenerationState.slowPathCall = callOperation(profiledFunction, resultRegs, srcRegs, arithProfile);
     831    } else
     832        mathICGenerationState.slowPathCall = callOperation(reinterpret_cast<J_JITOperation_EJMic>(repatchFunction), resultRegs, srcRegs, TrustedImmPtr(mathIC));
     833
     834#if ENABLE(MATH_IC_STATS)
     835    auto slowPathEnd = label();
     836    addLinkTask([=] (LinkBuffer& linkBuffer) {
     837        size_t size = static_cast<char*>(linkBuffer.locationOf(slowPathEnd).executableAddress()) - static_cast<char*>(linkBuffer.locationOf(slowPathStart).executableAddress());
     838        mathIC->m_generatedCodeSize += size;
     839    });
     840#endif
     841
     842    emitPutVirtualRegister(result, resultRegs);
     843
     844    addLinkTask([=] (LinkBuffer& linkBuffer) {
     845        MathICGenerationState& mathICGenerationState = m_instructionToMathICGenerationState.find(currentInstruction)->value;
     846        mathIC->finalizeInlineCode(mathICGenerationState, linkBuffer);
     847    });
     848}
     849
     850template <typename Generator, typename ProfiledRepatchFunction, typename ProfiledFunction, typename RepatchFunction>
     851void JIT::emitMathICSlow(JITBinaryMathIC<Generator>* mathIC, Instruction* currentInstruction, ProfiledRepatchFunction profiledRepatchFunction, ProfiledFunction profiledFunction, RepatchFunction repatchFunction)
    778852{
    779853    MathICGenerationState& mathICGenerationState = m_instructionToMathICGenerationState.find(currentInstruction)->value;
    … …  
    816890
    817891    if (shouldEmitProfiling()) {
    818         ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
     892        ArithProfile& arithProfile = *m_codeBlock->arithProfileForPC(currentInstruction);
    819893        if (mathICGenerationState.shouldSlowPathRepatch)
    820894            mathICGenerationState.slowPathCall = callOperation(bitwise_cast<J_JITOperation_EJJArpMic>(profiledRepatchFunction), resultRegs, leftRegs, rightRegs, &arithProfile, TrustedImmPtr(mathIC));
    … …  
    863937    ArithProfile* arithProfile = nullptr;
    864938    if (shouldEmitProfiling())
    865         arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
     939        arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
    866940
    867941    SnippetOperand leftOperand(types.first());

  • trunk/Source/JavaScriptCore/jit/JITInlines.h

    r204840 r206289  
    417417}
    418418
     419ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJ operation, JSValueRegs result, JSValueRegs arg)
     420{
     421    setupArgumentsWithExecState(arg);
     422    Call call = appendCallWithExceptionCheck(operation);
     423    setupResults(result);
     424    return call;
     425}
     426
    419427ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJ operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2)
    420428{
    … …  
    425433}
    426434
     435ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJArp operation, JSValueRegs result, JSValueRegs operand, ArithProfile* arithProfile)
     436{
     437    setupArgumentsWithExecState(operand, TrustedImmPtr(arithProfile));
     438    Call call = appendCallWithExceptionCheck(operation);
     439    setupResults(result);
     440    return call;
     441}
     442
    427443ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile)
    428444{
    … …  
    436452{
    437453    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile), mathIC);
     454    Call call = appendCallWithExceptionCheck(operation);
     455    setupResults(result);
     456    return call;
     457}
     458
     459ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJMic operation, JSValueRegs result, JSValueRegs arg, TrustedImmPtr mathIC)
     460{
     461    setupArgumentsWithExecState(arg, mathIC);
    438462    Call call = appendCallWithExceptionCheck(operation);
    439463    setupResults(result);

  • trunk/Source/JavaScriptCore/jit/JITMathIC.h

    r205364 r206289  
    3333#include "JITMathICInlineResult.h"
    3434#include "JITMulGenerator.h"
     35#include "JITNegGenerator.h"
    3536#include "JITSubGenerator.h"
    3637#include "LinkBuffer.h"
    … …  
    5354#define ENABLE_MATH_IC_STATS 0
    5455
    55 template <typename GeneratorType>
     56template <typename GeneratorType, bool(*isProfileEmpty)(ArithProfile*)>
    5657class JITMathIC {
    5758public:
    … …  
    7374
    7475        if (ArithProfile* arithProfile = m_generator.arithProfile()) {
    75             if (arithProfile->lhsObservedType().isEmpty() || arithProfile->rhsObservedType().isEmpty()) {
     76            if (!isProfileEmpty(arithProfile)) {
    7677                // It looks like the MathIC has yet to execute. We don't want to emit code in this
    7778                // case for a couple reasons. First, the operation may never execute, so if we don't emit
    7879                // code, it's a win. Second, if the operation does execute, we can emit better code
    79                 // once we have an idea about the types of lhs and rhs.
     80                // once we have an idea about the types.
    8081                state.slowPathJumps.append(jit.patchableJump());
    8182                size_t inlineSize = jit.m_assembler.buffer().codeSize() - startSize;
    … …  
    8384                state.shouldSlowPathRepatch = true;
    8485                state.fastPathEnd = jit.label();
    85                 ASSERT(!m_generateFastPathOnRepatch); // We should have gathered some observed type info for lhs and rhs before trying to regenerate again.
     86                ASSERT(!m_generateFastPathOnRepatch); // We should have gathered some observed type info about the types before trying to regenerate again.
    8687                m_generateFastPathOnRepatch = true;
    8788                return true;
    … …  
    244245};
    245246
    246 typedef JITMathIC<JITAddGenerator> JITAddIC;
    247 typedef JITMathIC<JITMulGenerator> JITMulIC;
    248 typedef JITMathIC<JITSubGenerator> JITSubIC;
     247inline bool canGenerateWithBinaryProfile(ArithProfile* arithProfile)
     248{
     249    return !arithProfile->lhsObservedType().isEmpty() && !arithProfile->rhsObservedType().isEmpty();
     250}
     251template <typename GeneratorType>
     252class JITBinaryMathIC : public JITMathIC<GeneratorType, canGenerateWithBinaryProfile> { };
     253
     254typedef JITBinaryMathIC<JITAddGenerator> JITAddIC;
     255typedef JITBinaryMathIC<JITMulGenerator> JITMulIC;
     256typedef JITBinaryMathIC<JITSubGenerator> JITSubIC;
     257
     258
     259inline bool canGenerateWithUnaryProfile(ArithProfile* arithProfile)
     260{
     261    return !arithProfile->lhsObservedType().isEmpty();
     262}
     263template <typename GeneratorType>
     264class JITUnaryMathIC : public JITMathIC<GeneratorType, canGenerateWithUnaryProfile> { };
     265
     266typedef JITUnaryMathIC<JITNegGenerator> JITNegIC;
    249267
    250268} // namespace JSC

  • trunk/Source/JavaScriptCore/jit/JITMathICForwards.h

    r203979 r206289  
    3030namespace JSC {
    3131
    32 template <typename Generator> class JITMathIC;
     32template <typename Generator> class JITBinaryMathIC;
     33template <typename Generator> class JITUnaryMathIC;
    3334class JITAddGenerator;
    3435class JITMulGenerator;
     36class JITNegGenerator;
    3537class JITSubGenerator;
    3638
    37 typedef JITMathIC<JITAddGenerator> JITAddIC;
    38 typedef JITMathIC<JITMulGenerator> JITMulIC;
    39 typedef JITMathIC<JITSubGenerator> JITSubIC;
     39typedef JITBinaryMathIC<JITAddGenerator> JITAddIC;
     40typedef JITBinaryMathIC<JITMulGenerator> JITMulIC;
     41typedef JITUnaryMathIC<JITNegGenerator> JITNegIC;
     42typedef JITBinaryMathIC<JITSubGenerator> JITSubIC;
    4043
    4144} // namespace JSC

  • trunk/Source/JavaScriptCore/jit/JITNegGenerator.cpp

    r194363 r206289  
    11/*
    2  * Copyright (C) 2015 Apple Inc. All rights reserved.
     2 * Copyright (C) 2015, 2016 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2727#include "JITNegGenerator.h"
    2828
     29#include "ArithProfile.h"
     30
    2931#if ENABLE(JIT)
    3032
    3133namespace JSC {
    3234
    33 void JITNegGenerator::generateFastPath(CCallHelpers& jit)
     35JITMathICInlineResult JITNegGenerator::generateInline(CCallHelpers& jit, MathICGenerationState& state)
     36{
     37    ASSERT(m_scratchGPR != InvalidGPRReg);
     38    ASSERT(m_scratchGPR != m_src.payloadGPR());
     39    ASSERT(m_scratchGPR != m_result.payloadGPR());
     40#if USE(JSVALUE32_64)
     41    ASSERT(m_scratchGPR != m_src.tagGPR());
     42    ASSERT(m_scratchGPR != m_result.tagGPR());
     43#endif
     44
     45    ObservedType observedTypes = m_arithProfile->lhsObservedType();
     46    ASSERT_WITH_MESSAGE(!observedTypes.isEmpty(), "We should not attempt to generate anything if we do not have a profile.");
     47
     48    if (observedTypes.isOnlyNonNumber())
     49        return JITMathICInlineResult::DontGenerate;
     50
     51    if (observedTypes.isOnlyInt32()) {
     52        jit.moveValueRegs(m_src, m_result);
     53        state.slowPathJumps.append(jit.branchIfNotInt32(m_src));
     54        state.slowPathJumps.append(jit.branchTest32(CCallHelpers::Zero, m_src.payloadGPR(), CCallHelpers::TrustedImm32(0x7fffffff)));
     55        jit.neg32(m_result.payloadGPR());
     56#if USE(JSVALUE64)
     57        jit.boxInt32(m_result.payloadGPR(), m_result);
     58#endif
     59
     60        return JITMathICInlineResult::GeneratedFastPath;
     61    }
     62    if (observedTypes.isOnlyNumber()) {
     63        state.slowPathJumps.append(jit.branchIfInt32(m_src));
     64        state.slowPathJumps.append(jit.branchIfNotNumber(m_src, m_scratchGPR));
     65#if USE(JSVALUE64)
     66        if (m_src.payloadGPR() != m_result.payloadGPR()) {
     67            jit.move(CCallHelpers::TrustedImm64(static_cast<int64_t>(1ull << 63)), m_result.payloadGPR());
     68            jit.xor64(m_src.payloadGPR(), m_result.payloadGPR());
     69        } else {
     70            jit.move(CCallHelpers::TrustedImm64(static_cast<int64_t>(1ull << 63)), m_scratchGPR);
     71            jit.xor64(m_scratchGPR, m_result.payloadGPR());
     72        }
     73#else
     74        jit.moveValueRegs(m_src, m_result);
     75        jit.xor32(CCallHelpers::TrustedImm32(1 << 31), m_result.tagGPR());
     76#endif
     77        return JITMathICInlineResult::GeneratedFastPath;
     78    }
     79    return JITMathICInlineResult::GenerateFullSnippet;
     80}
     81
     82bool JITNegGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList, bool)
    3483{
    3584    ASSERT(m_scratchGPR != m_src.payloadGPR());
    … …  
    4190#endif
    4291
    43     m_didEmitFastPath = true;
    44 
    4592    jit.moveValueRegs(m_src, m_result);
    4693    CCallHelpers::Jump srcNotInt = jit.branchIfNotInt32(m_src);
    … …  
    4895    // -0 should produce a double, and hence cannot be negated as an int.
    4996    // The negative int32 0x80000000 doesn't have a positive int32 representation, and hence cannot be negated as an int.
    50     m_slowPathJumpList.append(jit.branchTest32(CCallHelpers::Zero, m_src.payloadGPR(), CCallHelpers::TrustedImm32(0x7fffffff)));
     97    slowPathJumpList.append(jit.branchTest32(CCallHelpers::Zero, m_src.payloadGPR(), CCallHelpers::TrustedImm32(0x7fffffff)));
    5198
    5299    jit.neg32(m_result.payloadGPR());
    … …  
    54101    jit.boxInt32(m_result.payloadGPR(), m_result);
    55102#endif
    56     m_endJumpList.append(jit.jump());
     103    endJumpList.append(jit.jump());
    57104
    58105    srcNotInt.link(&jit);
    59     m_slowPathJumpList.append(jit.branchIfNotNumber(m_src, m_scratchGPR));
     106    slowPathJumpList.append(jit.branchIfNotNumber(m_src, m_scratchGPR));
    60107
    61108    // For a double, all we need to do is to invert the sign bit.
    … …  
    66113    jit.xor32(CCallHelpers::TrustedImm32(1 << 31), m_result.tagGPR());
    67114#endif
     115    return true;
    68116}
    69117

  • trunk/Source/JavaScriptCore/jit/JITNegGenerator.h

    r194363 r206289  
    11/*
    2  * Copyright (C) 2015 Apple Inc. All rights reserved.
     2 * Copyright (C) 2015, 2016 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    3030
    3131#include "CCallHelpers.h"
    32 #include "SnippetOperand.h"
     32#include "JITMathIC.h"
     33#include "JITMathICInlineResult.h"
    3334
    3435namespace JSC {
    … …  
    3637class JITNegGenerator {
    3738public:
    38     JITNegGenerator(JSValueRegs result, JSValueRegs src, GPRReg scratchGPR)
    39         : m_result(result)
     39    JITNegGenerator() = default;
     40
     41    JITNegGenerator(JSValueRegs result, JSValueRegs src, GPRReg scratchGPR, ArithProfile& arithProfile)
     42        : m_arithProfile(&arithProfile)
     43        , m_result(result)
    4044        , m_src(src)
    4145        , m_scratchGPR(scratchGPR)
    4246    { }
    4347
    44     void generateFastPath(CCallHelpers&);
     48    JITMathICInlineResult generateInline(CCallHelpers&, MathICGenerationState&);
     49    bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList, bool shouldEmitProfiling);
    4550
    46     bool didEmitFastPath() const { return m_didEmitFastPath; }
    47     CCallHelpers::JumpList& endJumpList() { return m_endJumpList; }
    48     CCallHelpers::JumpList& slowPathJumpList() { return m_slowPathJumpList; }
     51    ArithProfile* arithProfile() const { return m_arithProfile; }
    4952
    5053private:
     54    ArithProfile* m_arithProfile { nullptr };
    5155    JSValueRegs m_result;
    5256    JSValueRegs m_src;
    5357    GPRReg m_scratchGPR;
    54     bool m_didEmitFastPath { false };
    55 
    56     CCallHelpers::JumpList m_endJumpList;
    57     CCallHelpers::JumpList m_slowPathJumpList;
    5858};
    5959

  • trunk/Source/JavaScriptCore/jit/JITOperations.cpp

    r206267 r206289  
    24702470}
    24712471
     2472ALWAYS_INLINE static EncodedJSValue unprofiledNegate(ExecState* exec, EncodedJSValue encodedOperand)
     2473{
     2474    VM& vm = exec->vm();
     2475    auto scope = DECLARE_THROW_SCOPE(vm);
     2476    NativeCallFrameTracer tracer(&vm, exec);
     2477   
     2478    JSValue operand = JSValue::decode(encodedOperand);
     2479    double number = operand.toNumber(exec);
     2480    if (UNLIKELY(scope.exception()))
     2481        return JSValue::encode(JSValue());
     2482    return JSValue::encode(jsNumber(-number));
     2483}
     2484
     2485ALWAYS_INLINE static EncodedJSValue profiledNegate(ExecState* exec, EncodedJSValue encodedOperand, ArithProfile* arithProfile)
     2486{
     2487    ASSERT(arithProfile);
     2488    VM& vm = exec->vm();
     2489    auto scope = DECLARE_THROW_SCOPE(vm);
     2490    NativeCallFrameTracer tracer(&vm, exec);
     2491
     2492    JSValue operand = JSValue::decode(encodedOperand);
     2493    arithProfile->observeLHS(operand);
     2494    double number = operand.toNumber(exec);
     2495    if (UNLIKELY(scope.exception()))
     2496        return JSValue::encode(JSValue());
     2497    return JSValue::encode(jsNumber(-number));
     2498}
     2499
     2500EncodedJSValue JIT_OPERATION operationArithNegate(ExecState* exec, EncodedJSValue operand)
     2501{
     2502    return unprofiledNegate(exec, operand);
     2503}
     2504
     2505EncodedJSValue JIT_OPERATION operationArithNegateProfiled(ExecState* exec, EncodedJSValue operand, ArithProfile* arithProfile)
     2506{
     2507    return profiledNegate(exec, operand, arithProfile);
     2508}
     2509
     2510EncodedJSValue JIT_OPERATION operationArithNegateProfiledOptimize(ExecState* exec, EncodedJSValue encodedOperand, JITNegIC* negIC)
     2511{
     2512    VM& vm = exec->vm();
     2513    auto scope = DECLARE_THROW_SCOPE(vm);
     2514    NativeCallFrameTracer tracer(&vm, exec);
     2515   
     2516    JSValue operand = JSValue::decode(encodedOperand);
     2517
     2518    ArithProfile* arithProfile = negIC->m_generator.arithProfile();
     2519    ASSERT(arithProfile);
     2520    arithProfile->observeLHS(operand);
     2521    negIC->generateOutOfLine(vm, exec->codeBlock(), operationArithNegateProfiled);
     2522
     2523#if ENABLE(MATH_IC_STATS)
     2524    exec->codeBlock()->dumpMathICStats();
     2525#endif
     2526   
     2527    double number = operand.toNumber(exec);
     2528    if (UNLIKELY(scope.exception()))
     2529        return JSValue::encode(JSValue());
     2530    return JSValue::encode(jsNumber(-number));
     2531}
     2532
     2533EncodedJSValue JIT_OPERATION operationArithNegateOptimize(ExecState* exec, EncodedJSValue encodedOperand, JITNegIC* negIC)
     2534{
     2535    VM& vm = exec->vm();
     2536    auto scope = DECLARE_THROW_SCOPE(vm);
     2537    NativeCallFrameTracer tracer(&vm, exec);
     2538
     2539    JSValue operand = JSValue::decode(encodedOperand);
     2540
     2541    ArithProfile* arithProfile = negIC->m_generator.arithProfile();
     2542    ASSERT(arithProfile);
     2543    arithProfile->observeLHS(operand);
     2544    negIC->generateOutOfLine(vm, exec->codeBlock(), operationArithNegate);
     2545
     2546#if ENABLE(MATH_IC_STATS)
     2547    exec->codeBlock()->dumpMathICStats();
     2548#endif
     2549
     2550    double number = operand.toNumber(exec);
     2551    if (UNLIKELY(scope.exception()))
     2552        return JSValue::encode(JSValue());
     2553    return JSValue::encode(jsNumber(-number));
     2554}
     2555
    24722556ALWAYS_INLINE static EncodedJSValue unprofiledSub(VM& vm, ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
    24732557{

  • trunk/Source/JavaScriptCore/jit/JITOperations.h

    r206136 r206289  
    141141typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJC)(ExecState*, EncodedJSValue, JSCell*);
    142142typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJA)(ExecState*, EncodedJSValue, JSArray*);
     143typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJArp)(ExecState*, EncodedJSValue, ArithProfile*);
    143144typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJI)(ExecState*, EncodedJSValue, UniquedStringImpl*);
    144145typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJ)(ExecState*, EncodedJSValue, EncodedJSValue);
    … …  
    150151typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArpMic)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*, void*);
    151152typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJMic)(ExecState*, EncodedJSValue, EncodedJSValue, void*);
     153typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJMic)(ExecState*, EncodedJSValue, void*);
    152154typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssZ)(ExecState*, JSString*, int32_t);
    153155typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReo)(ExecState*, JSString*, RegExpObject*);
    … …  
    438440EncodedJSValue JIT_OPERATION operationValueMulProfiledNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITMulIC*) WTF_INTERNAL;
    439441EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
     442EncodedJSValue JIT_OPERATION operationArithNegate(ExecState*, EncodedJSValue operand);
     443EncodedJSValue JIT_OPERATION operationArithNegateProfiled(ExecState*, EncodedJSValue operand, ArithProfile*);
     444EncodedJSValue JIT_OPERATION operationArithNegateProfiledOptimize(ExecState*, EncodedJSValue encodedOperand, JITNegIC*);
     445EncodedJSValue JIT_OPERATION operationArithNegateOptimize(ExecState*, EncodedJSValue encodedOperand, JITNegIC*);
    440446EncodedJSValue JIT_OPERATION operationValueSub(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
    441447EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;

  • trunk/Source/JavaScriptCore/llint/LowLevelInterpreter.asm

    r206065 r206289  
    254254
    255255# ArithProfile data
     256const ArithProfileInt = 0x100000
    256257const ArithProfileIntInt = 0x120000
     258const ArithProfileNumber = 0x200000
    257259const ArithProfileNumberInt = 0x220000
    258260const ArithProfileNumberNumber = 0x240000

  • trunk/Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm

    r206098 r206289  
    949949    loadi 4[PC], t3
    950950    loadConstantOrVariable(t0, t1, t2)
     951    loadisFromInstruction(3, t0)
    951952    bineq t1, Int32Tag, .opNegateSrcNotInt
    952953    btiz t2, 0x7fffffff, .opNegateSlow
    953954    negi t2
     955    ori ArithProfileInt, t0
    954956    storei Int32Tag, TagOffset[cfr, t3, 8]
     957    storeisToInstruction(t0, 3)
    955958    storei t2, PayloadOffset[cfr, t3, 8]
    956     dispatch(3)
     959    dispatch(4)
    957960.opNegateSrcNotInt:
    958961    bia t1, LowestTag, .opNegateSlow
    959962    xori 0x80000000, t1
     963    ori ArithProfileNumber, t0
     964    storei t2, PayloadOffset[cfr, t3, 8]
     965    storeisToInstruction(t0, 3)
    960966    storei t1, TagOffset[cfr, t3, 8]
    961     storei t2, PayloadOffset[cfr, t3, 8]
    962     dispatch(3)
     967    dispatch(4)
    963968
    964969.opNegateSlow:
    965970    callOpcodeSlowPath(_slow_path_negate)
    966     dispatch(3)
     971    dispatch(4)
    967972
    968973

  • trunk/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm

    r206128 r206289  
    828828    loadisFromInstruction(2, t0)
    829829    loadisFromInstruction(1, t1)
    830     loadConstantOrVariable(t0, t2)
    831     bqb t2, tagTypeNumber, .opNegateNotInt
    832     btiz t2, 0x7fffffff, .opNegateSlow
    833     negi t2
    834     orq tagTypeNumber, t2
    835     storeq t2, [cfr, t1, 8]
    836     dispatch(3)
     830    loadConstantOrVariable(t0, t3)
     831    loadisFromInstruction(3, t2)
     832    bqb t3, tagTypeNumber, .opNegateNotInt
     833    btiz t3, 0x7fffffff, .opNegateSlow
     834    negi t3
     835    ori ArithProfileInt, t2
     836    orq tagTypeNumber, t3
     837    storeisToInstruction(t2, 3)
     838    storeq t3, [cfr, t1, 8]
     839    dispatch(4)
    837840.opNegateNotInt:
    838     btqz t2, tagTypeNumber, .opNegateSlow
    839     xorq 0x8000000000000000, t2
    840     storeq t2, [cfr, t1, 8]
    841     dispatch(3)
     841    btqz t3, tagTypeNumber, .opNegateSlow
     842    xorq 0x8000000000000000, t3
     843    ori ArithProfileNumber, t2
     844    storeq t3, [cfr, t1, 8]
     845    storeisToInstruction(t2, 3)
     846    dispatch(4)
    842847
    843848.opNegateSlow:
    844849    callOpcodeSlowPath(_slow_path_negate)
    845     dispatch(3)
     850    dispatch(4)
    846851
    847852

  • trunk/Source/JavaScriptCore/runtime/CommonSlowPaths.cpp

    r205569 r206289  
    351351}
    352352
     353#if ENABLE(JIT)
     354static void updateArithProfileForUnaryArithOp(Instruction* pc, JSValue result, JSValue operand)
     355{
     356    ArithProfile& profile = *bitwise_cast<ArithProfile*>(&pc[3].u.operand);
     357    profile.observeLHS(operand);
     358    ASSERT(result.isNumber());
     359    if (!result.isInt32()) {
     360        if (operand.isInt32())
     361            profile.setObservedInt32Overflow();
     362
     363        double doubleVal = result.asNumber();
     364        if (!doubleVal && std::signbit(doubleVal))
     365            profile.setObservedNegZeroDouble();
     366        else {
     367            profile.setObservedNonNegZeroDouble();
     368
     369            // The Int52 overflow check here intentionally omits 1ll << 51 as a valid negative Int52 value.
     370            // Therefore, we will get a false positive if the result is that value. This is intentionally
     371            // done to simplify the checking algorithm.
     372            static const int64_t int52OverflowPoint = (1ll << 51);
     373            int64_t int64Val = static_cast<int64_t>(std::abs(doubleVal));
     374            if (int64Val >= int52OverflowPoint)
     375                profile.setObservedInt52Overflow();
     376        }
     377    }
     378}
     379#else
     380static void updateArithProfileForUnaryArithOp(Instruction*, JSValue, JSValue) { }
     381#endif
     382
    353383SLOW_PATH_DECL(slow_path_negate)
    354384{
    355385    BEGIN();
    356     RETURN(jsNumber(-OP_C(2).jsValue().toNumber(exec)));
     386    JSValue operand = OP_C(2).jsValue();
     387    JSValue result = jsNumber(-operand.toNumber(exec));
     388    RETURN_WITH_PROFILING(result, {
     389        updateArithProfileForUnaryArithOp(pc, result, operand);
     390    });
    357391}
    358392
    … …  
    361395{
    362396    CodeBlock* codeBlock = exec->codeBlock();
    363     ArithProfile& profile = codeBlock->arithProfileForPC(pc);
     397    ArithProfile& profile = *codeBlock->arithProfileForPC(pc);
    364398
    365399    if (result.isNumber()) {
    … …  
    405439    JSValue result;
    406440
    407     ArithProfile& arithProfile = exec->codeBlock()->arithProfileForPC(pc);
     441    ArithProfile& arithProfile = *exec->codeBlock()->arithProfileForPC(pc);
    408442    arithProfile.observeLHSAndRHS(v1, v2);
    409443
Note: See TracChangeset for help on using the changeset viewer.