Changeset 203537 in webkit


Ignore:
Timestamp:
Jul 21, 2016 4:41:44 PM (8 years ago)
Author:
sbarati@apple.com
Message:

op_add/ValueAdd should be an IC in all JIT tiers
https://bugs.webkit.org/show_bug.cgi?id=159649

Reviewed by Benjamin Poulain.

Source/JavaScriptCore:

This patch makes Add an IC inside all JIT tiers. It does so in a
simple, but effective, way. We will try to generate an int+int add
that will repatch itself if its type checks fail. Sometimes though,
we have runtime type data saying that the add won't be int+int.
In those cases, we will just generate a full snippet that doesn't patch itself.
Other times, we may generate no inline code and defer to making a C call. A lot
of this patch is just refactoring ResultProfile into what we're now calling ArithProfile.
ArithProfile does everything ResultProfile used to do, and more. It records simple type
data about the LHS/RHS operands it sees. This allows us to determine if an op_add
has only seen int+int operands, etc. ArithProfile will also contain the ResultType
for the LHS/RHS that the parser feeds into op_add. ArithProfile now fits into 32-bits.
This means instead of having a side table like we did for ResultProfile, we just
inject the ArithProfile into the bytecode instruction stream. This makes asking
for ArithProfile faster; we no longer need to lock around this operation.

The size of an Add has gone down on average, but we can still do better.
We still generate a lot of code because we generate calls to the slow path.
I think we can make this better by moving the slow path to a shared thunk
system. This patch mostly lays the foundation for future improvements to Add,
and a framework to move all other arithmetic operations to be typed-based ICs.

Here is some data I took on the average op_add/ValueAdd size on various benchmarks:

| JetStream | Speedometer | Unity 3D |

------| -------------

Old | 189 bytes | 169 bytes | 192 bytes |

------| -------------

New | 148 bytes | 124 bytes | 143 bytes |

---------------------------------------------------

Making an arithmetic IC is now easy. The JITMathIC class will hold a snippet
generator as a member variable. To make a snippet an IC, you need to implement
a generateInline(.) method, which generates the inline IC. Then, you need to
generate the IC where you used to generate the snippet. When generating the
IC, we need to inform JITMathIC of various data like we do with StructureStubInfo.
We need to tell it about where the slow path starts, where the slow path call is, etc.
When generating a JITMathIC, it may tell you that it didn't generate any code inline.
This is a request to the user of JITMathIC to just generate a C call along the
fast path. JITMathIC may also have the snippet tell it to just generate the full
snippet instead of the int+int path along the fast path.

In subsequent patches, we can improve upon how we decide to generate int+int or
the full snippet. I tried to get clever by having double+double, double+int, int+double,
fast paths, but they didn't work out nearly as well as the int+int fast path. I ended up
generating a lot of code when I did this and ended up using more memory than just generating
the full snippet. There is probably some way we can be clever and generate specialized fast
paths that are more successful than what I tried implementing, but I think that's worth deferring
this to follow up patches once the JITMathIC foundation has landed.

This patch also fixes a bug inside the slow path lambdas in the DFG.
Before, it was not legal to emit an exception check inside them. Now,
it is. So it's now easy to define arbitrary late paths using the DFG
slow path lambda API.

  • CMakeLists.txt:
  • JavaScriptCore.xcodeproj/project.pbxproj:
  • bytecode/ArithProfile.cpp: Added.

(JSC::ArithProfile::emitObserveResult):
(JSC::ArithProfile::shouldEmitSetDouble):
(JSC::ArithProfile::emitSetDouble):
(JSC::ArithProfile::shouldEmitSetNonNumber):
(JSC::ArithProfile::emitSetNonNumber):
(WTF::printInternal):

  • bytecode/ArithProfile.h: Added.

(JSC::ObservedType::ObservedType):
(JSC::ObservedType::sawInt32):
(JSC::ObservedType::isOnlyInt32):
(JSC::ObservedType::sawNumber):
(JSC::ObservedType::isOnlyNumber):
(JSC::ObservedType::sawNonNumber):
(JSC::ObservedType::isOnlyNonNumber):
(JSC::ObservedType::isEmpty):
(JSC::ObservedType::bits):
(JSC::ObservedType::withInt32):
(JSC::ObservedType::withNumber):
(JSC::ObservedType::withNonNumber):
(JSC::ObservedType::withoutNonNumber):
(JSC::ObservedType::operator==):
(JSC::ArithProfile::ArithProfile):
(JSC::ArithProfile::fromInt):
(JSC::ArithProfile::lhsResultType):
(JSC::ArithProfile::rhsResultType):
(JSC::ArithProfile::lhsObservedType):
(JSC::ArithProfile::rhsObservedType):
(JSC::ArithProfile::setLhsObservedType):
(JSC::ArithProfile::setRhsObservedType):
(JSC::ArithProfile::tookSpecialFastPath):
(JSC::ArithProfile::didObserveNonInt32):
(JSC::ArithProfile::didObserveDouble):
(JSC::ArithProfile::didObserveNonNegZeroDouble):
(JSC::ArithProfile::didObserveNegZeroDouble):
(JSC::ArithProfile::didObserveNonNumber):
(JSC::ArithProfile::didObserveInt32Overflow):
(JSC::ArithProfile::didObserveInt52Overflow):
(JSC::ArithProfile::setObservedNonNegZeroDouble):
(JSC::ArithProfile::setObservedNegZeroDouble):
(JSC::ArithProfile::setObservedNonNumber):
(JSC::ArithProfile::setObservedInt32Overflow):
(JSC::ArithProfile::setObservedInt52Overflow):
(JSC::ArithProfile::addressOfBits):
(JSC::ArithProfile::observeResult):
(JSC::ArithProfile::lhsSawInt32):
(JSC::ArithProfile::lhsSawNumber):
(JSC::ArithProfile::lhsSawNonNumber):
(JSC::ArithProfile::rhsSawInt32):
(JSC::ArithProfile::rhsSawNumber):
(JSC::ArithProfile::rhsSawNonNumber):
(JSC::ArithProfile::observeLHSAndRHS):
(JSC::ArithProfile::bits):
(JSC::ArithProfile::hasBits):
(JSC::ArithProfile::setBit):

  • bytecode/CodeBlock.cpp:

(JSC::CodeBlock::dumpRareCaseProfile):
(JSC::CodeBlock::dumpArithProfile):
(JSC::CodeBlock::dumpBytecode):
(JSC::CodeBlock::addStubInfo):
(JSC::CodeBlock::addJITAddIC):
(JSC::CodeBlock::findStubInfo):
(JSC::CodeBlock::resetJITData):
(JSC::CodeBlock::shrinkToFit):
(JSC::CodeBlock::dumpValueProfiles):
(JSC::CodeBlock::rareCaseProfileCountForBytecodeOffset):
(JSC::CodeBlock::arithProfileForBytecodeOffset):
(JSC::CodeBlock::arithProfileForPC):
(JSC::CodeBlock::couldTakeSpecialFastCase):
(JSC::CodeBlock::dumpResultProfile): Deleted.
(JSC::CodeBlock::resultProfileForBytecodeOffset): Deleted.
(JSC::CodeBlock::specialFastCaseProfileCountForBytecodeOffset): Deleted.
(JSC::CodeBlock::ensureResultProfile): Deleted.

  • bytecode/CodeBlock.h:

(JSC::CodeBlock::stubInfoBegin):
(JSC::CodeBlock::stubInfoEnd):
(JSC::CodeBlock::couldTakeSlowCase):
(JSC::CodeBlock::numberOfResultProfiles): Deleted.

  • bytecode/MethodOfGettingAValueProfile.cpp:

(JSC::MethodOfGettingAValueProfile::emitReportValue):

  • bytecode/MethodOfGettingAValueProfile.h:

(JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile):

  • bytecode/ValueProfile.cpp:

(JSC::ResultProfile::emitDetectNumericness): Deleted.
(JSC::ResultProfile::emitSetDouble): Deleted.
(JSC::ResultProfile::emitSetNonNumber): Deleted.
(WTF::printInternal): Deleted.

  • bytecode/ValueProfile.h:

(JSC::getRareCaseProfileBytecodeOffset):
(JSC::ResultProfile::ResultProfile): Deleted.
(JSC::ResultProfile::bytecodeOffset): Deleted.
(JSC::ResultProfile::specialFastPathCount): Deleted.
(JSC::ResultProfile::didObserveNonInt32): Deleted.
(JSC::ResultProfile::didObserveDouble): Deleted.
(JSC::ResultProfile::didObserveNonNegZeroDouble): Deleted.
(JSC::ResultProfile::didObserveNegZeroDouble): Deleted.
(JSC::ResultProfile::didObserveNonNumber): Deleted.
(JSC::ResultProfile::didObserveInt32Overflow): Deleted.
(JSC::ResultProfile::didObserveInt52Overflow): Deleted.
(JSC::ResultProfile::setObservedNonNegZeroDouble): Deleted.
(JSC::ResultProfile::setObservedNegZeroDouble): Deleted.
(JSC::ResultProfile::setObservedNonNumber): Deleted.
(JSC::ResultProfile::setObservedInt32Overflow): Deleted.
(JSC::ResultProfile::setObservedInt52Overflow): Deleted.
(JSC::ResultProfile::addressOfFlags): Deleted.
(JSC::ResultProfile::addressOfSpecialFastPathCount): Deleted.
(JSC::ResultProfile::detectNumericness): Deleted.
(JSC::ResultProfile::hasBits): Deleted.
(JSC::ResultProfile::setBit): Deleted.
(JSC::getResultProfileBytecodeOffset): Deleted.

  • bytecompiler/BytecodeGenerator.cpp:

(JSC::BytecodeGenerator::emitBinaryOp):

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::makeSafe):

  • dfg/DFGGraph.cpp:

(JSC::DFG::Graph::methodOfGettingAValueProfileFor):

  • dfg/DFGJITCompiler.cpp:

(JSC::DFG::JITCompiler::exceptionCheck):

  • dfg/DFGSlowPathGenerator.h:

(JSC::DFG::SlowPathGenerator::generate):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::addSlowPathGenerator):
(JSC::DFG::SpeculativeJIT::runSlowPathGenerators):
(JSC::DFG::SpeculativeJIT::compileValueAdd):

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::silentSpillAllRegistersImpl):
(JSC::DFG::SpeculativeJIT::silentSpillAllRegisters):
(JSC::DFG::SpeculativeJIT::callOperation):

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileValueAdd):
(JSC::FTL::DFG::LowerDFGToB3::compileStrCat):

  • jit/CCallHelpers.h:

(JSC::CCallHelpers::setupArgumentsWithExecState):
(JSC::CCallHelpers::setupArguments):

  • jit/JIT.h:
  • jit/JITAddGenerator.cpp:

(JSC::JITAddGenerator::generateInline):
(JSC::JITAddGenerator::generateFastPath):

  • jit/JITAddGenerator.h:

(JSC::JITAddGenerator::JITAddGenerator):
(JSC::JITAddGenerator::didEmitFastPath): Deleted.
(JSC::JITAddGenerator::endJumpList): Deleted.
(JSC::JITAddGenerator::slowPathJumpList): Deleted.

  • jit/JITArithmetic.cpp:

(JSC::JIT::emit_op_jless):
(JSC::JIT::emitSlow_op_urshift):
(JSC::getOperandTypes):
(JSC::JIT::emit_op_add):
(JSC::JIT::emitSlow_op_add):
(JSC::JIT::emit_op_div):
(JSC::JIT::emit_op_mul):
(JSC::JIT::emitSlow_op_mul):
(JSC::JIT::emit_op_sub):
(JSC::JIT::emitSlow_op_sub):

  • jit/JITDivGenerator.cpp:

(JSC::JITDivGenerator::generateFastPath):

  • jit/JITDivGenerator.h:

(JSC::JITDivGenerator::JITDivGenerator):

  • jit/JITInlines.h:

(JSC::JIT::callOperation):

  • jit/JITMathIC.h: Added.

(JSC::JITMathIC::doneLocation):
(JSC::JITMathIC::slowPathStartLocation):
(JSC::JITMathIC::slowPathCallLocation):
(JSC::JITMathIC::generateInline):
(JSC::JITMathIC::generateOutOfLine):
(JSC::JITMathIC::finalizeInlineCode):

  • jit/JITMathICForwards.h: Added.
  • jit/JITMathICInlineResult.h: Added.
  • jit/JITMulGenerator.cpp:

(JSC::JITMulGenerator::generateFastPath):

  • jit/JITMulGenerator.h:

(JSC::JITMulGenerator::JITMulGenerator):

  • jit/JITOperations.cpp:
  • jit/JITOperations.h:
  • jit/JITSubGenerator.cpp:

(JSC::JITSubGenerator::generateFastPath):

  • jit/JITSubGenerator.h:

(JSC::JITSubGenerator::JITSubGenerator):

  • jit/Repatch.cpp:

(JSC::readCallTarget):
(JSC::ftlThunkAwareRepatchCall):
(JSC::tryCacheGetByID):
(JSC::repatchGetByID):
(JSC::appropriateGenericPutByIdFunction):
(JSC::tryCachePutByID):
(JSC::repatchPutByID):
(JSC::tryRepatchIn):
(JSC::repatchIn):
(JSC::linkSlowFor):
(JSC::resetGetByID):
(JSC::resetPutByID):
(JSC::repatchCall): Deleted.

  • jit/Repatch.h:
  • llint/LLIntData.cpp:

(JSC::LLInt::Data::performAssertions):

  • llint/LowLevelInterpreter.asm:
  • llint/LowLevelInterpreter32_64.asm:
  • llint/LowLevelInterpreter64.asm:
  • parser/ResultType.h:

(JSC::ResultType::ResultType):
(JSC::ResultType::isInt32):
(JSC::ResultType::definitelyIsNumber):
(JSC::ResultType::definitelyIsString):
(JSC::ResultType::definitelyIsBoolean):
(JSC::ResultType::mightBeNumber):
(JSC::ResultType::isNotNumber):
(JSC::ResultType::forBitOp):
(JSC::ResultType::bits):
(JSC::OperandTypes::OperandTypes):

  • runtime/CommonSlowPaths.cpp:

(JSC::SLOW_PATH_DECL):
(JSC::updateArithProfileForBinaryArithOp):
(JSC::updateResultProfileForBinaryArithOp): Deleted.

  • tests/stress/op-add-exceptions.js: Added.

(assert):
(f1):
(f2):
(f3):
(let.oException.valueOf):
(foo):
(ident):
(bar):

Source/WebCore:

  • ForwardingHeaders/jit/JITMathICForwards.h: Added.
Location:
trunk/Source
Files:
7 added
40 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/JavaScriptCore/CMakeLists.txt

    r203491 r203537  
    177177
    178178    bytecode/AdaptiveInferredPropertyValueWatchpointBase.cpp
     179    bytecode/ArithProfile.cpp
    179180    bytecode/ArrayAllocationProfile.cpp
    180181    bytecode/ArrayProfile.cpp
  • trunk/Source/JavaScriptCore/ChangeLog

    r203512 r203537  
     12016-07-21  Saam Barati  <sbarati@apple.com>
     2
     3        op_add/ValueAdd should be an IC in all JIT tiers
     4        https://bugs.webkit.org/show_bug.cgi?id=159649
     5
     6        Reviewed by Benjamin Poulain.
     7
     8        This patch makes Add an IC inside all JIT tiers. It does so in a
     9        simple, but effective, way. We will try to generate an int+int add
     10        that will repatch itself if its type checks fail. Sometimes though,
     11        we have runtime type data saying that the add won't be int+int.
     12        In those cases, we will just generate a full snippet that doesn't patch itself.
     13        Other times, we may generate no inline code and defer to making a C call. A lot
     14        of this patch is just refactoring ResultProfile into what we're now calling ArithProfile.
     15        ArithProfile does everything ResultProfile used to do, and more. It records simple type
     16        data about the LHS/RHS operands it sees. This allows us to determine if an op_add
     17        has only seen int+int operands, etc. ArithProfile will also contain the ResultType
     18        for the LHS/RHS that the parser feeds into op_add. ArithProfile now fits into 32-bits.
     19        This means instead of having a side table like we did for ResultProfile, we just
     20        inject the ArithProfile into the bytecode instruction stream. This makes asking
     21        for ArithProfile faster; we no longer need to lock around this operation.
     22
     23        The size of an Add has gone down on average, but we can still do better.
     24        We still generate a lot of code because we generate calls to the slow path.
     25        I think we can make this better by moving the slow path to a shared thunk
     26        system. This patch mostly lays the foundation for future improvements to Add,
     27        and a framework to move all other arithmetic operations to be typed-based ICs.
     28
     29        Here is some data I took on the average op_add/ValueAdd size on various benchmarks:
     30                   |   JetStream  |  Speedometer |  Unity 3D  |
     31             ------| -------------|-----------------------------
     32              Old  |  189 bytes   |  169 bytes   |  192 bytes |
     33             ------| -------------|-----------------------------
     34              New  |  148 bytes   |  124 bytes   |  143 bytes |
     35             ---------------------------------------------------
     36
     37        Making an arithmetic IC is now easy. The JITMathIC class will hold a snippet
     38        generator as a member variable. To make a snippet an IC, you need to implement
     39        a generateInline(.) method, which generates the inline IC. Then, you need to
     40        generate the IC where you used to generate the snippet. When generating the
     41        IC, we need to inform JITMathIC of various data like we do with StructureStubInfo.
     42        We need to tell it about where the slow path starts, where the slow path call is, etc.
     43        When generating a JITMathIC, it may tell you that it didn't generate any code inline.
     44        This is a request to the user of JITMathIC to just generate a C call along the
     45        fast path. JITMathIC may also have the snippet tell it to just generate the full
     46        snippet instead of the int+int path along the fast path.
     47
     48        In subsequent patches, we can improve upon how we decide to generate int+int or
     49        the full snippet. I tried to get clever by having double+double, double+int, int+double,
     50        fast paths, but they didn't work out nearly as well as the int+int fast path. I ended up
     51        generating a lot of code when I did this and ended up using more memory than just generating
     52        the full snippet. There is probably some way we can be clever and generate specialized fast
     53        paths that are more successful than what I tried implementing, but I think that's worth deferring
     54        this to follow up patches once the JITMathIC foundation has landed.
     55
     56        This patch also fixes a bug inside the slow path lambdas in the DFG.
     57        Before, it was not legal to emit an exception check inside them. Now,
     58        it is. So it's now easy to define arbitrary late paths using the DFG
     59        slow path lambda API.
     60
     61        * CMakeLists.txt:
     62        * JavaScriptCore.xcodeproj/project.pbxproj:
     63        * bytecode/ArithProfile.cpp: Added.
     64        (JSC::ArithProfile::emitObserveResult):
     65        (JSC::ArithProfile::shouldEmitSetDouble):
     66        (JSC::ArithProfile::emitSetDouble):
     67        (JSC::ArithProfile::shouldEmitSetNonNumber):
     68        (JSC::ArithProfile::emitSetNonNumber):
     69        (WTF::printInternal):
     70        * bytecode/ArithProfile.h: Added.
     71        (JSC::ObservedType::ObservedType):
     72        (JSC::ObservedType::sawInt32):
     73        (JSC::ObservedType::isOnlyInt32):
     74        (JSC::ObservedType::sawNumber):
     75        (JSC::ObservedType::isOnlyNumber):
     76        (JSC::ObservedType::sawNonNumber):
     77        (JSC::ObservedType::isOnlyNonNumber):
     78        (JSC::ObservedType::isEmpty):
     79        (JSC::ObservedType::bits):
     80        (JSC::ObservedType::withInt32):
     81        (JSC::ObservedType::withNumber):
     82        (JSC::ObservedType::withNonNumber):
     83        (JSC::ObservedType::withoutNonNumber):
     84        (JSC::ObservedType::operator==):
     85        (JSC::ArithProfile::ArithProfile):
     86        (JSC::ArithProfile::fromInt):
     87        (JSC::ArithProfile::lhsResultType):
     88        (JSC::ArithProfile::rhsResultType):
     89        (JSC::ArithProfile::lhsObservedType):
     90        (JSC::ArithProfile::rhsObservedType):
     91        (JSC::ArithProfile::setLhsObservedType):
     92        (JSC::ArithProfile::setRhsObservedType):
     93        (JSC::ArithProfile::tookSpecialFastPath):
     94        (JSC::ArithProfile::didObserveNonInt32):
     95        (JSC::ArithProfile::didObserveDouble):
     96        (JSC::ArithProfile::didObserveNonNegZeroDouble):
     97        (JSC::ArithProfile::didObserveNegZeroDouble):
     98        (JSC::ArithProfile::didObserveNonNumber):
     99        (JSC::ArithProfile::didObserveInt32Overflow):
     100        (JSC::ArithProfile::didObserveInt52Overflow):
     101        (JSC::ArithProfile::setObservedNonNegZeroDouble):
     102        (JSC::ArithProfile::setObservedNegZeroDouble):
     103        (JSC::ArithProfile::setObservedNonNumber):
     104        (JSC::ArithProfile::setObservedInt32Overflow):
     105        (JSC::ArithProfile::setObservedInt52Overflow):
     106        (JSC::ArithProfile::addressOfBits):
     107        (JSC::ArithProfile::observeResult):
     108        (JSC::ArithProfile::lhsSawInt32):
     109        (JSC::ArithProfile::lhsSawNumber):
     110        (JSC::ArithProfile::lhsSawNonNumber):
     111        (JSC::ArithProfile::rhsSawInt32):
     112        (JSC::ArithProfile::rhsSawNumber):
     113        (JSC::ArithProfile::rhsSawNonNumber):
     114        (JSC::ArithProfile::observeLHSAndRHS):
     115        (JSC::ArithProfile::bits):
     116        (JSC::ArithProfile::hasBits):
     117        (JSC::ArithProfile::setBit):
     118        * bytecode/CodeBlock.cpp:
     119        (JSC::CodeBlock::dumpRareCaseProfile):
     120        (JSC::CodeBlock::dumpArithProfile):
     121        (JSC::CodeBlock::dumpBytecode):
     122        (JSC::CodeBlock::addStubInfo):
     123        (JSC::CodeBlock::addJITAddIC):
     124        (JSC::CodeBlock::findStubInfo):
     125        (JSC::CodeBlock::resetJITData):
     126        (JSC::CodeBlock::shrinkToFit):
     127        (JSC::CodeBlock::dumpValueProfiles):
     128        (JSC::CodeBlock::rareCaseProfileCountForBytecodeOffset):
     129        (JSC::CodeBlock::arithProfileForBytecodeOffset):
     130        (JSC::CodeBlock::arithProfileForPC):
     131        (JSC::CodeBlock::couldTakeSpecialFastCase):
     132        (JSC::CodeBlock::dumpResultProfile): Deleted.
     133        (JSC::CodeBlock::resultProfileForBytecodeOffset): Deleted.
     134        (JSC::CodeBlock::specialFastCaseProfileCountForBytecodeOffset): Deleted.
     135        (JSC::CodeBlock::ensureResultProfile): Deleted.
     136        * bytecode/CodeBlock.h:
     137        (JSC::CodeBlock::stubInfoBegin):
     138        (JSC::CodeBlock::stubInfoEnd):
     139        (JSC::CodeBlock::couldTakeSlowCase):
     140        (JSC::CodeBlock::numberOfResultProfiles): Deleted.
     141        * bytecode/MethodOfGettingAValueProfile.cpp:
     142        (JSC::MethodOfGettingAValueProfile::emitReportValue):
     143        * bytecode/MethodOfGettingAValueProfile.h:
     144        (JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile):
     145        * bytecode/ValueProfile.cpp:
     146        (JSC::ResultProfile::emitDetectNumericness): Deleted.
     147        (JSC::ResultProfile::emitSetDouble): Deleted.
     148        (JSC::ResultProfile::emitSetNonNumber): Deleted.
     149        (WTF::printInternal): Deleted.
     150        * bytecode/ValueProfile.h:
     151        (JSC::getRareCaseProfileBytecodeOffset):
     152        (JSC::ResultProfile::ResultProfile): Deleted.
     153        (JSC::ResultProfile::bytecodeOffset): Deleted.
     154        (JSC::ResultProfile::specialFastPathCount): Deleted.
     155        (JSC::ResultProfile::didObserveNonInt32): Deleted.
     156        (JSC::ResultProfile::didObserveDouble): Deleted.
     157        (JSC::ResultProfile::didObserveNonNegZeroDouble): Deleted.
     158        (JSC::ResultProfile::didObserveNegZeroDouble): Deleted.
     159        (JSC::ResultProfile::didObserveNonNumber): Deleted.
     160        (JSC::ResultProfile::didObserveInt32Overflow): Deleted.
     161        (JSC::ResultProfile::didObserveInt52Overflow): Deleted.
     162        (JSC::ResultProfile::setObservedNonNegZeroDouble): Deleted.
     163        (JSC::ResultProfile::setObservedNegZeroDouble): Deleted.
     164        (JSC::ResultProfile::setObservedNonNumber): Deleted.
     165        (JSC::ResultProfile::setObservedInt32Overflow): Deleted.
     166        (JSC::ResultProfile::setObservedInt52Overflow): Deleted.
     167        (JSC::ResultProfile::addressOfFlags): Deleted.
     168        (JSC::ResultProfile::addressOfSpecialFastPathCount): Deleted.
     169        (JSC::ResultProfile::detectNumericness): Deleted.
     170        (JSC::ResultProfile::hasBits): Deleted.
     171        (JSC::ResultProfile::setBit): Deleted.
     172        (JSC::getResultProfileBytecodeOffset): Deleted.
     173        * bytecompiler/BytecodeGenerator.cpp:
     174        (JSC::BytecodeGenerator::emitBinaryOp):
     175        * dfg/DFGByteCodeParser.cpp:
     176        (JSC::DFG::ByteCodeParser::makeSafe):
     177        * dfg/DFGGraph.cpp:
     178        (JSC::DFG::Graph::methodOfGettingAValueProfileFor):
     179        * dfg/DFGJITCompiler.cpp:
     180        (JSC::DFG::JITCompiler::exceptionCheck):
     181        * dfg/DFGSlowPathGenerator.h:
     182        (JSC::DFG::SlowPathGenerator::generate):
     183        * dfg/DFGSpeculativeJIT.cpp:
     184        (JSC::DFG::SpeculativeJIT::addSlowPathGenerator):
     185        (JSC::DFG::SpeculativeJIT::runSlowPathGenerators):
     186        (JSC::DFG::SpeculativeJIT::compileValueAdd):
     187        * dfg/DFGSpeculativeJIT.h:
     188        (JSC::DFG::SpeculativeJIT::silentSpillAllRegistersImpl):
     189        (JSC::DFG::SpeculativeJIT::silentSpillAllRegisters):
     190        (JSC::DFG::SpeculativeJIT::callOperation):
     191        * ftl/FTLLowerDFGToB3.cpp:
     192        (JSC::FTL::DFG::LowerDFGToB3::compileValueAdd):
     193        (JSC::FTL::DFG::LowerDFGToB3::compileStrCat):
     194        * jit/CCallHelpers.h:
     195        (JSC::CCallHelpers::setupArgumentsWithExecState):
     196        (JSC::CCallHelpers::setupArguments):
     197        * jit/JIT.h:
     198        * jit/JITAddGenerator.cpp:
     199        (JSC::JITAddGenerator::generateInline):
     200        (JSC::JITAddGenerator::generateFastPath):
     201        * jit/JITAddGenerator.h:
     202        (JSC::JITAddGenerator::JITAddGenerator):
     203        (JSC::JITAddGenerator::didEmitFastPath): Deleted.
     204        (JSC::JITAddGenerator::endJumpList): Deleted.
     205        (JSC::JITAddGenerator::slowPathJumpList): Deleted.
     206        * jit/JITArithmetic.cpp:
     207        (JSC::JIT::emit_op_jless):
     208        (JSC::JIT::emitSlow_op_urshift):
     209        (JSC::getOperandTypes):
     210        (JSC::JIT::emit_op_add):
     211        (JSC::JIT::emitSlow_op_add):
     212        (JSC::JIT::emit_op_div):
     213        (JSC::JIT::emit_op_mul):
     214        (JSC::JIT::emitSlow_op_mul):
     215        (JSC::JIT::emit_op_sub):
     216        (JSC::JIT::emitSlow_op_sub):
     217        * jit/JITDivGenerator.cpp:
     218        (JSC::JITDivGenerator::generateFastPath):
     219        * jit/JITDivGenerator.h:
     220        (JSC::JITDivGenerator::JITDivGenerator):
     221        * jit/JITInlines.h:
     222        (JSC::JIT::callOperation):
     223        * jit/JITMathIC.h: Added.
     224        (JSC::JITMathIC::doneLocation):
     225        (JSC::JITMathIC::slowPathStartLocation):
     226        (JSC::JITMathIC::slowPathCallLocation):
     227        (JSC::JITMathIC::generateInline):
     228        (JSC::JITMathIC::generateOutOfLine):
     229        (JSC::JITMathIC::finalizeInlineCode):
     230        * jit/JITMathICForwards.h: Added.
     231        * jit/JITMathICInlineResult.h: Added.
     232        * jit/JITMulGenerator.cpp:
     233        (JSC::JITMulGenerator::generateFastPath):
     234        * jit/JITMulGenerator.h:
     235        (JSC::JITMulGenerator::JITMulGenerator):
     236        * jit/JITOperations.cpp:
     237        * jit/JITOperations.h:
     238        * jit/JITSubGenerator.cpp:
     239        (JSC::JITSubGenerator::generateFastPath):
     240        * jit/JITSubGenerator.h:
     241        (JSC::JITSubGenerator::JITSubGenerator):
     242        * jit/Repatch.cpp:
     243        (JSC::readCallTarget):
     244        (JSC::ftlThunkAwareRepatchCall):
     245        (JSC::tryCacheGetByID):
     246        (JSC::repatchGetByID):
     247        (JSC::appropriateGenericPutByIdFunction):
     248        (JSC::tryCachePutByID):
     249        (JSC::repatchPutByID):
     250        (JSC::tryRepatchIn):
     251        (JSC::repatchIn):
     252        (JSC::linkSlowFor):
     253        (JSC::resetGetByID):
     254        (JSC::resetPutByID):
     255        (JSC::repatchCall): Deleted.
     256        * jit/Repatch.h:
     257        * llint/LLIntData.cpp:
     258        (JSC::LLInt::Data::performAssertions):
     259        * llint/LowLevelInterpreter.asm:
     260        * llint/LowLevelInterpreter32_64.asm:
     261        * llint/LowLevelInterpreter64.asm:
     262        * parser/ResultType.h:
     263        (JSC::ResultType::ResultType):
     264        (JSC::ResultType::isInt32):
     265        (JSC::ResultType::definitelyIsNumber):
     266        (JSC::ResultType::definitelyIsString):
     267        (JSC::ResultType::definitelyIsBoolean):
     268        (JSC::ResultType::mightBeNumber):
     269        (JSC::ResultType::isNotNumber):
     270        (JSC::ResultType::forBitOp):
     271        (JSC::ResultType::bits):
     272        (JSC::OperandTypes::OperandTypes):
     273        * runtime/CommonSlowPaths.cpp:
     274        (JSC::SLOW_PATH_DECL):
     275        (JSC::updateArithProfileForBinaryArithOp):
     276        (JSC::updateResultProfileForBinaryArithOp): Deleted.
     277        * tests/stress/op-add-exceptions.js: Added.
     278        (assert):
     279        (f1):
     280        (f2):
     281        (f3):
     282        (let.oException.valueOf):
     283        (foo):
     284        (ident):
     285        (bar):
     286
    12872016-07-21  Csaba Osztrogonác  <ossy@webkit.org>
    2288
  • trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

    r203491 r203537  
    12801280                79160DBD1C8E3EC8008C085A /* ProxyRevoke.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */; };
    12811281                79160DBE1C8E3EC8008C085A /* ProxyRevoke.h in Headers */ = {isa = PBXBuildFile; fileRef = 79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1282                79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */ = {isa = PBXBuildFile; fileRef = 79233C291D34715700C5A834 /* JITMathIC.h */; settings = {ATTRIBUTES = (Private, ); }; };
    12821283                792CB3491C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */; };
    12831284                792CB34A1C4EED5C00D13AF3 /* PCToCodeOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
     
    12861287                797E07AA1B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h in Headers */ = {isa = PBXBuildFile; fileRef = 797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */; settings = {ATTRIBUTES = (Private, ); }; };
    12871288                799EF7C41C56ED96002B0534 /* B3PCToOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1289                79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79A228331D35D71E00D8E067 /* ArithProfile.cpp */; };
     1290                79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A228341D35D71E00D8E067 /* ArithProfile.h */; };
     1291                79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */ = {isa = PBXBuildFile; fileRef = 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */; settings = {ATTRIBUTES = (Private, ); }; };
    12881292                79B00CBC1C6AB07E0088C65D /* ProxyConstructor.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */; };
    12891293                79B00CBD1C6AB07E0088C65D /* ProxyConstructor.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */; settings = {ATTRIBUTES = (Private, ); }; };
    12901294                79B00CBE1C6AB07E0088C65D /* ProxyObject.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CBA1C6AB07E0088C65D /* ProxyObject.cpp */; settings = {COMPILER_FLAGS = "-fno-optimize-sibling-calls"; }; };
    12911295                79B00CBF1C6AB07E0088C65D /* ProxyObject.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CBB1C6AB07E0088C65D /* ProxyObject.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1296                79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */; settings = {ATTRIBUTES = (Private, ); }; };
    12921297                79C4B15D1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79C4B15B1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp */; };
    12931298                79C4B15E1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 79C4B15C1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
     
    34803485                79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyRevoke.cpp; sourceTree = "<group>"; };
    34813486                79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyRevoke.h; sourceTree = "<group>"; };
     3487                79233C291D34715700C5A834 /* JITMathIC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathIC.h; sourceTree = "<group>"; };
    34823488                792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PCToCodeOriginMap.cpp; sourceTree = "<group>"; };
    34833489                792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PCToCodeOriginMap.h; sourceTree = "<group>"; };
     
    34863492                797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSGlobalLexicalEnvironment.h; sourceTree = "<group>"; };
    34873493                799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3PCToOriginMap.h; path = b3/B3PCToOriginMap.h; sourceTree = "<group>"; };
     3494                79A228331D35D71E00D8E067 /* ArithProfile.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ArithProfile.cpp; sourceTree = "<group>"; };
     3495                79A228341D35D71E00D8E067 /* ArithProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ArithProfile.h; sourceTree = "<group>"; };
     3496                79A899FE1D38612E00D18C73 /* JITMathICForwards.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICForwards.h; sourceTree = "<group>"; };
     3497                79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICInlineResult.h; sourceTree = "<group>"; };
    34883498                79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyConstructor.cpp; sourceTree = "<group>"; };
    34893499                79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyConstructor.h; sourceTree = "<group>"; };
     
    50715081                                1429D92D0ED22D7000B89619 /* JIT.cpp */,
    50725082                                1429D92E0ED22D7000B89619 /* JIT.h */,
     5083                                79233C291D34715700C5A834 /* JITMathIC.h */,
     5084                                79A899FE1D38612E00D18C73 /* JITMathICForwards.h */,
     5085                                79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */,
    50735086                                FE1220251BE7F5640039E6F2 /* JITAddGenerator.cpp */,
    50745087                                FE1220261BE7F5640039E6F2 /* JITAddGenerator.h */,
     
    65726585                                0F63945115D07051006A597C /* ArrayProfile.cpp */,
    65736586                                0F63945215D07051006A597C /* ArrayProfile.h */,
     6587                                79A228331D35D71E00D8E067 /* ArithProfile.cpp */,
     6588                                79A228341D35D71E00D8E067 /* ArithProfile.h */,
    65746589                                C2FCAE0C17A9C24E0034C735 /* BytecodeBasicBlock.cpp */,
    65756590                                C2FCAE0D17A9C24E0034C735 /* BytecodeBasicBlock.h */,
     
    70847099                                0FEC84FF1BDACDAC0080FF74 /* B3ArgumentRegValue.h in Headers */,
    70857100                                0FEC85011BDACDAC0080FF74 /* B3BasicBlock.h in Headers */,
     7101                                79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */,
    70867102                                0FEC85021BDACDAC0080FF74 /* B3BasicBlockInlines.h in Headers */,
    70877103                                0FEC85031BDACDAC0080FF74 /* B3BasicBlockUtils.h in Headers */,
     
    76647680                                BC11667B0E199C05008066DD /* InternalFunction.h in Headers */,
    76657681                                1429D77C0ED20D7300B89619 /* Interpreter.h in Headers */,
     7682                                79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */,
    76667683                                A1B9E23A1B4E0D6700BC7FED /* IntlCollator.h in Headers */,
    76677684                                A1B9E23C1B4E0D6700BC7FED /* IntlCollatorConstructor.h in Headers */,
     
    77737790                                A50E4B6418809DD50068A46D /* JSGlobalObjectRuntimeAgent.h in Headers */,
    77747791                                A503FA2A188F105900110F14 /* JSGlobalObjectScriptDebugServer.h in Headers */,
     7792                                79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */,
    77757793                                A513E5C0185BFACC007E95AD /* JSInjectedScriptHost.h in Headers */,
    77767794                                A513E5C2185BFACC007E95AD /* JSInjectedScriptHostPrototype.h in Headers */,
     
    81988216                                9688CB160ED12B4E001D649F /* X86Assembler.h in Headers */,
    81998217                                9959E92E1BD17FA4001AA413 /* xxd.pl in Headers */,
     8218                                79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */,
    82008219                                451539B912DC994500EF7AC4 /* Yarr.h in Headers */,
    82018220                                86704B8512DBA33700A9FE7B /* YarrInterpreter.h in Headers */,
     
    92839302                                86C568E011A213EE0007F7F0 /* MacroAssemblerARM.cpp in Sources */,
    92849303                                FEB137571BB11EF900CD5100 /* MacroAssemblerARM64.cpp in Sources */,
     9304                                79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */,
    92859305                                A729009C17976C6000317298 /* MacroAssemblerARMv7.cpp in Sources */,
    92869306                                FE68C6381B90DE0B0042BCB3 /* MacroAssemblerPrinter.cpp in Sources */,
  • trunk/Source/JavaScriptCore/bytecode/CodeBlock.cpp

    r203499 r203537  
    3131#include "CodeBlock.h"
    3232
     33#include "ArithProfile.h"
    3334#include "BasicBlockLocation.h"
    3435#include "BytecodeGenerator.h"
     
    4647#include "Interpreter.h"
    4748#include "JIT.h"
     49#include "JITMathIC.h"
    4850#include "JSCJSValue.h"
    4951#include "JSFunction.h"
     
    770772}
    771773
    772 void CodeBlock::dumpResultProfile(PrintStream& out, ResultProfile* profile, bool& hasPrintedProfiling)
     774void CodeBlock::dumpArithProfile(PrintStream& out, ArithProfile* profile, bool& hasPrintedProfiling)
    773775{
    774776    if (!profile)
     
    17741776    dumpRareCaseProfile(out, "rare case: ", rareCaseProfileForBytecodeOffset(location), hasPrintedProfiling);
    17751777    {
    1776         ConcurrentJITLocker locker(m_lock);
    1777         dumpResultProfile(out, resultProfileForBytecodeOffset(locker, location), hasPrintedProfiling);
     1778        dumpArithProfile(out, arithProfileForBytecodeOffset(location), hasPrintedProfiling);
    17781779    }
    17791780   
     
    30083009}
    30093010
     3011JITAddIC* CodeBlock::addJITAddIC()
     3012{
     3013    return m_addICs.add();
     3014}
     3015
    30103016StructureStubInfo* CodeBlock::findStubInfo(CodeOrigin codeOrigin)
    30113017{
     
    30543060    // there is JIT code.
    30553061    m_rareCaseProfiles.clear();
    3056    
    3057     // We can clear these because the DFG only accesses members of this data structure when
    3058     // holding the lock or after querying whether we have JIT code.
    3059     m_resultProfiles.clear();
    3060     m_bytecodeOffsetToResultProfileIndexMap = nullptr;
    30613062}
    30623063#endif
     
    32883289
    32893290    m_rareCaseProfiles.shrinkToFit();
    3290     m_resultProfiles.shrinkToFit();
    32913291   
    32923292    if (shrinkMode == EarlyShrink) {
     
    41644164        dataLogF("   bc = %d: %u\n", profile->m_bytecodeOffset, profile->m_counter);
    41654165    }
    4166     dataLog("ResultProfile for ", *this, ":\n");
    4167     for (unsigned i = 0; i < numberOfResultProfiles(); ++i) {
    4168         const ResultProfile& profile = *resultProfile(i);
    4169         dataLog("   bc = ", profile.bytecodeOffset(), ": ", profile, "\n");
    4170     }
    41714166}
    41724167#endif // ENABLE(VERBOSE_VALUE_PROFILE)
     
    43714366}
    43724367
    4373 ResultProfile* CodeBlock::resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset)
    4374 {
    4375     if (!m_bytecodeOffsetToResultProfileIndexMap)
     4368ArithProfile* CodeBlock::arithProfileForBytecodeOffset(int bytecodeOffset)
     4369{
     4370    auto opcodeID = vm()->interpreter->getOpcodeID(instructions()[bytecodeOffset].u.opcode);
     4371    switch (opcodeID) {
     4372    case op_bitor:
     4373    case op_bitand:
     4374    case op_bitxor:
     4375    case op_add:
     4376    case op_mul:
     4377    case op_sub:
     4378    case op_div:
     4379        break;
     4380    default:
    43764381        return nullptr;
    4377     auto iterator = m_bytecodeOffsetToResultProfileIndexMap->find(bytecodeOffset);
    4378     if (iterator == m_bytecodeOffsetToResultProfileIndexMap->end())
    4379         return nullptr;
    4380     return &m_resultProfiles[iterator->value];
    4381 }
    4382 
    4383 unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset)
    4384 {
    4385     ConcurrentJITLocker locker(m_lock);
    4386     return specialFastCaseProfileCountForBytecodeOffset(locker, bytecodeOffset);
    4387 }
    4388 
    4389 unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker& locker, int bytecodeOffset)
    4390 {
    4391     ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset);
    4392     if (!profile)
    4393         return 0;
    4394     return profile->specialFastPathCount();
     4382    }
     4383
     4384    return &arithProfileForPC(instructions().begin() + bytecodeOffset);
     4385}
     4386
     4387ArithProfile& CodeBlock::arithProfileForPC(Instruction* pc)
     4388{
     4389    if (!ASSERT_DISABLED) {
     4390        ASSERT(pc >= instructions().begin() && pc < instructions().end());
     4391        auto opcodeID = vm()->interpreter->getOpcodeID(pc[0].u.opcode);
     4392        switch (opcodeID) {
     4393        case op_bitor:
     4394        case op_bitand:
     4395        case op_bitxor:
     4396        case op_add:
     4397        case op_mul:
     4398        case op_sub:
     4399        case op_div:
     4400            break;
     4401        default:
     4402            ASSERT_NOT_REACHED();
     4403        }
     4404    }
     4405
     4406    return *bitwise_cast<ArithProfile*>(&pc[4].u.operand);
    43954407}
    43964408
     
    43994411    if (!hasBaselineJITProfiling())
    44004412        return false;
    4401     unsigned specialFastCaseCount = specialFastCaseProfileCountForBytecodeOffset(bytecodeOffset);
    4402     return specialFastCaseCount >= Options::couldTakeSlowCaseMinimumCount();
    4403 }
    4404 
    4405 ResultProfile* CodeBlock::ensureResultProfile(int bytecodeOffset)
    4406 {
    4407     ConcurrentJITLocker locker(m_lock);
    4408     return ensureResultProfile(locker, bytecodeOffset);
    4409 }
    4410 
    4411 ResultProfile* CodeBlock::ensureResultProfile(const ConcurrentJITLocker& locker, int bytecodeOffset)
    4412 {
    4413     ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset);
    4414     if (!profile) {
    4415         m_resultProfiles.append(ResultProfile(bytecodeOffset));
    4416         profile = &m_resultProfiles.last();
    4417         ASSERT(&m_resultProfiles.last() == &m_resultProfiles[m_resultProfiles.size() - 1]);
    4418         if (!m_bytecodeOffsetToResultProfileIndexMap)
    4419             m_bytecodeOffsetToResultProfileIndexMap = std::make_unique<BytecodeOffsetToResultProfileIndexMap>();
    4420         m_bytecodeOffsetToResultProfileIndexMap->add(bytecodeOffset, m_resultProfiles.size() - 1);
    4421     }
    4422     return profile;
     4413    ArithProfile* profile = arithProfileForBytecodeOffset(bytecodeOffset);
     4414    if (!profile)
     4415        return false;
     4416    return profile->tookSpecialFastPath();
    44234417}
    44244418
  • trunk/Source/JavaScriptCore/bytecode/CodeBlock.h

    r202778 r203537  
    5252#include "Instruction.h"
    5353#include "JITCode.h"
     54#include "JITMathICForwards.h"
    5455#include "JITWriteBarrier.h"
    5556#include "JSCell.h"
     
    8081
    8182class ExecState;
     83class JITAddGenerator;
    8284class JSModuleEnvironment;
    8385class LLIntOffsetsExtractor;
     
    8890
    8991enum class AccessType : int8_t;
     92
     93struct ArithProfile;
    9094
    9195typedef HashMap<CodeOrigin, StructureStubInfo*, CodeOriginApproximateHash> StubInfoMap;
     
    246250#if ENABLE(JIT)
    247251    StructureStubInfo* addStubInfo(AccessType);
     252    JITAddIC* addJITAddIC();
    248253    Bag<StructureStubInfo>::iterator stubInfoBegin() { return m_stubInfos.begin(); }
    249254    Bag<StructureStubInfo>::iterator stubInfoEnd() { return m_stubInfos.end(); }
     
    441446    }
    442447
    443     ResultProfile* ensureResultProfile(int bytecodeOffset);
    444     ResultProfile* ensureResultProfile(const ConcurrentJITLocker&, int bytecodeOffset);
    445     unsigned numberOfResultProfiles() { return m_resultProfiles.size(); }
    446     ResultProfile* resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset);
    447 
    448     unsigned specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset);
    449     unsigned specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset);
     448    ArithProfile* arithProfileForBytecodeOffset(int bytecodeOffset);
     449    ArithProfile& arithProfileForPC(Instruction*);
    450450
    451451    bool couldTakeSpecialFastCase(int bytecodeOffset);
     
    958958    void dumpArrayProfiling(PrintStream&, const Instruction*&, bool& hasPrintedProfiling);
    959959    void dumpRareCaseProfile(PrintStream&, const char* name, RareCaseProfile*, bool& hasPrintedProfiling);
    960     void dumpResultProfile(PrintStream&, ResultProfile*, bool& hasPrintedProfiling);
     960    void dumpArithProfile(PrintStream&, ArithProfile*, bool& hasPrintedProfiling);
    961961
    962962    bool shouldVisitStrongly();
     
    10141014    std::unique_ptr<RegisterAtOffsetList> m_calleeSaveRegisters;
    10151015    Bag<StructureStubInfo> m_stubInfos;
     1016    Bag<JITAddIC> m_addICs;
    10161017    Bag<ByValInfo> m_byValInfos;
    10171018    Bag<CallLinkInfo> m_callLinkInfos;
     
    10301031    RefCountedArray<ValueProfile> m_valueProfiles;
    10311032    SegmentedVector<RareCaseProfile, 8> m_rareCaseProfiles;
    1032     SegmentedVector<ResultProfile, 8> m_resultProfiles;
    1033     typedef HashMap<unsigned, unsigned, IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> BytecodeOffsetToResultProfileIndexMap;
    1034     std::unique_ptr<BytecodeOffsetToResultProfileIndexMap> m_bytecodeOffsetToResultProfileIndexMap;
    10351033    RefCountedArray<ArrayAllocationProfile> m_arrayAllocationProfiles;
    10361034    ArrayProfileVector m_arrayProfiles;
  • trunk/Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.cpp

    r200606 r203537  
    2929#if ENABLE(DFG_JIT)
    3030
     31#include "ArithProfile.h"
    3132#include "CCallHelpers.h"
    3233#include "CodeBlock.h"
     
    6667    }
    6768       
    68     case ResultProfileReady: {
    69         u.resultProfile->emitDetectNumericness(jit, regs, DoNotHaveTagRegisters);
     69    case ArithProfileReady: {
     70        u.arithProfile->emitObserveResult(jit, regs, DoNotHaveTagRegisters);
    7071        return;
    7172    } }
  • trunk/Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.h

    r200606 r203537  
    4141class CodeBlock;
    4242class LazyOperandValueProfileKey;
    43 struct ResultProfile;
     43struct ArithProfile;
    4444struct ValueProfile;
    4545
     
    6060    }
    6161   
    62     MethodOfGettingAValueProfile(ResultProfile* profile)
     62    MethodOfGettingAValueProfile(ArithProfile* profile)
    6363    {
    6464        if (profile) {
    65             m_kind = ResultProfileReady;
    66             u.resultProfile = profile;
     65            m_kind = ArithProfileReady;
     66            u.arithProfile = profile;
    6767        } else
    6868            m_kind = None;
     
    8080        None,
    8181        Ready,
    82         ResultProfileReady,
     82        ArithProfileReady,
    8383        LazyOperand
    8484    };
     
    8787    union {
    8888        ValueProfile* profile;
    89         ResultProfile* resultProfile;
     89        ArithProfile* arithProfile;
    9090        struct {
    9191            CodeBlock* codeBlock;
  • trunk/Source/JavaScriptCore/bytecode/ValueProfile.cpp

    r200613 r203537  
    3232namespace JSC {
    3333
    34 #if ENABLE(JIT)
    35 void ResultProfile::emitDetectNumericness(CCallHelpers& jit, JSValueRegs regs, TagRegistersMode mode)
    36 {
    37     CCallHelpers::Jump isInt32 = jit.branchIfInt32(regs, mode);
    38     CCallHelpers::Jump notDouble = jit.branchIfNotDoubleKnownNotInt32(regs, mode);
    39     // FIXME: We could be more precise here.
    40     emitSetDouble(jit);
    41     CCallHelpers::Jump done = jit.jump();
    42     notDouble.link(&jit);
    43     emitSetNonNumber(jit);
    44     done.link(&jit);
    45     isInt32.link(&jit);
    46 }
    47 
    48 void ResultProfile::emitSetDouble(CCallHelpers& jit)
    49 {
    50     jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int32Overflow | ResultProfile::Int52Overflow | ResultProfile::NegZeroDouble | ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(addressOfFlags()));
    51 }
    52 
    53 void ResultProfile::emitSetNonNumber(CCallHelpers& jit)
    54 {
    55     jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNumber), CCallHelpers::AbsoluteAddress(addressOfFlags()));
    56 }
    57 #endif // ENABLE(JIT)
    58 
    5934} // namespace JSC
    60 
    61 namespace WTF {
    62    
    63 using namespace JSC;
    64 
    65 void printInternal(PrintStream& out, const ResultProfile& profile)
    66 {
    67     const char* separator = "";
    68 
    69     if (!profile.didObserveNonInt32()) {
    70         out.print("Int32");
    71         separator = "|";
    72     } else {
    73         if (profile.didObserveNegZeroDouble()) {
    74             out.print(separator, "NegZeroDouble");
    75             separator = "|";
    76         }
    77         if (profile.didObserveNonNegZeroDouble()) {
    78             out.print("NonNegZeroDouble");
    79             separator = "|";
    80         }
    81         if (profile.didObserveNonNumber()) {
    82             out.print("NonNumber");
    83             separator = "|";
    84         }
    85         if (profile.didObserveInt32Overflow()) {
    86             out.print("Int32Overflow");
    87             separator = "|";
    88         }
    89         if (profile.didObserveInt52Overflow()) {
    90             out.print("Int52Overflow");
    91             separator = "|";
    92         }
    93     }
    94     if (profile.specialFastPathCount()) {
    95         out.print(" special fast path: ");
    96         out.print(profile.specialFastPathCount());
    97     }
    98 }
    99 
    100 } // namespace WTF
  • trunk/Source/JavaScriptCore/bytecode/ValueProfile.h

    r200613 r203537  
    210210}
    211211
    212 struct ResultProfile {
    213 private:
    214     static const int numberOfFlagBits = 5;
    215 
    216 public:
    217     ResultProfile(int bytecodeOffset)
    218         : m_bytecodeOffsetAndFlags(bytecodeOffset << numberOfFlagBits)
    219     {
    220         ASSERT(((bytecodeOffset << numberOfFlagBits) >> numberOfFlagBits) == bytecodeOffset);
    221     }
    222 
    223     enum ObservedResults {
    224         NonNegZeroDouble = 1 << 0,
    225         NegZeroDouble    = 1 << 1,
    226         NonNumber        = 1 << 2,
    227         Int32Overflow    = 1 << 3,
    228         Int52Overflow    = 1 << 4,
    229     };
    230 
    231     int bytecodeOffset() const { return m_bytecodeOffsetAndFlags >> numberOfFlagBits; }
    232     unsigned specialFastPathCount() const { return m_specialFastPathCount; }
    233 
    234     bool didObserveNonInt32() const { return hasBits(NonNegZeroDouble | NegZeroDouble | NonNumber); }
    235     bool didObserveDouble() const { return hasBits(NonNegZeroDouble | NegZeroDouble); }
    236     bool didObserveNonNegZeroDouble() const { return hasBits(NonNegZeroDouble); }
    237     bool didObserveNegZeroDouble() const { return hasBits(NegZeroDouble); }
    238     bool didObserveNonNumber() const { return hasBits(NonNumber); }
    239     bool didObserveInt32Overflow() const { return hasBits(Int32Overflow); }
    240     bool didObserveInt52Overflow() const { return hasBits(Int52Overflow); }
    241 
    242     void setObservedNonNegZeroDouble() { setBit(NonNegZeroDouble); }
    243     void setObservedNegZeroDouble() { setBit(NegZeroDouble); }
    244     void setObservedNonNumber() { setBit(NonNumber); }
    245     void setObservedInt32Overflow() { setBit(Int32Overflow); }
    246     void setObservedInt52Overflow() { setBit(Int52Overflow); }
    247 
    248     void* addressOfFlags() { return &m_bytecodeOffsetAndFlags; }
    249     void* addressOfSpecialFastPathCount() { return &m_specialFastPathCount; }
    250    
    251     void detectNumericness(JSValue value)
    252     {
    253         if (value.isInt32())
    254             return;
    255         if (value.isNumber()) {
    256             m_bytecodeOffsetAndFlags |= Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble;
    257             return;
    258         }
    259         m_bytecodeOffsetAndFlags |= NonNumber;
    260     }
    261 
    262 #if ENABLE(JIT)   
    263     // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble) if it sees a
    264     // double. Sets NonNumber if it sees a non-number.
    265     void emitDetectNumericness(CCallHelpers&, JSValueRegs, TagRegistersMode = HaveTagRegisters);
    266    
    267     // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble).
    268     void emitSetDouble(CCallHelpers&);
    269    
    270     // Sets NonNumber.
    271     void emitSetNonNumber(CCallHelpers&);
    272 #endif // ENABLE(JIT)
    273 
    274 private:
    275     bool hasBits(int mask) const { return m_bytecodeOffsetAndFlags & mask; }
    276     void setBit(int mask) { m_bytecodeOffsetAndFlags |= mask; }
    277 
    278     int m_bytecodeOffsetAndFlags;
    279     unsigned m_specialFastPathCount { 0 };
    280 };
    281 
    282 inline int getResultProfileBytecodeOffset(ResultProfile* profile)
    283 {
    284     return profile->bytecodeOffset();
    285 }
    286 
    287212} // namespace JSC
    288213
    289 namespace WTF {
    290 
    291 void printInternal(PrintStream&, const JSC::ResultProfile&);
    292 
    293 } // namespace WTF
    294 
    295214#endif // ValueProfile_h
    296 
  • trunk/Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp

    r203361 r203537  
    3232#include "BytecodeGenerator.h"
    3333
     34#include "ArithProfile.h"
    3435#include "BuiltinExecutables.h"
    3536#include "BytecodeLivenessAnalysis.h"
     
    16011602    if (opcodeID == op_bitor || opcodeID == op_bitand || opcodeID == op_bitxor ||
    16021603        opcodeID == op_add || opcodeID == op_mul || opcodeID == op_sub || opcodeID == op_div)
    1603         instructions().append(types.toInt());
     1604        instructions().append(ArithProfile(types.first(), types.second()).bits());
    16041605
    16051606    return dst;
  • trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

    r203499 r203537  
    2929#if ENABLE(DFG_JIT)
    3030
     31#include "ArithProfile.h"
    3132#include "ArrayConstructor.h"
    3233#include "BasicBlockLocation.h"
     
    915916
    916917        {
    917             ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
    918             ResultProfile* resultProfile = m_inlineStackTop->m_profiledBlock->resultProfileForBytecodeOffset(locker, m_currentIndex);
    919             if (resultProfile) {
     918            ArithProfile* arithProfile = m_inlineStackTop->m_profiledBlock->arithProfileForBytecodeOffset(m_currentIndex);
     919            if (arithProfile) {
    920920                switch (node->op()) {
    921921                case ArithAdd:
    922922                case ArithSub:
    923923                case ValueAdd:
    924                     if (resultProfile->didObserveDouble())
     924                    if (arithProfile->didObserveDouble())
    925925                        node->mergeFlags(NodeMayHaveDoubleResult);
    926                     if (resultProfile->didObserveNonNumber())
     926                    if (arithProfile->didObserveNonNumber())
    927927                        node->mergeFlags(NodeMayHaveNonNumberResult);
    928928                    break;
    929929               
    930930                case ArithMul: {
    931                     if (resultProfile->didObserveInt52Overflow())
     931                    if (arithProfile->didObserveInt52Overflow())
    932932                        node->mergeFlags(NodeMayOverflowInt52);
    933                     if (resultProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow))
     933                    if (arithProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow))
    934934                        node->mergeFlags(NodeMayOverflowInt32InBaseline);
    935                     if (resultProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero))
     935                    if (arithProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero))
    936936                        node->mergeFlags(NodeMayNegZeroInBaseline);
    937                     if (resultProfile->didObserveDouble())
     937                    if (arithProfile->didObserveDouble())
    938938                        node->mergeFlags(NodeMayHaveDoubleResult);
    939                     if (resultProfile->didObserveNonNumber())
     939                    if (arithProfile->didObserveNonNumber())
    940940                        node->mergeFlags(NodeMayHaveNonNumberResult);
    941941                    break;
  • trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp

    r203006 r203537  
    15491549       
    15501550        {
    1551             ConcurrentJITLocker locker(profiledBlock->m_lock);
    15521551            if (profiledBlock->hasBaselineJITProfiling()) {
    1553                 if (ResultProfile* result = profiledBlock->resultProfileForBytecodeOffset(locker, node->origin.semantic.bytecodeIndex))
     1552                if (ArithProfile* result = profiledBlock->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex))
    15541553                    return result;
    15551554            }
  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp

    r203130 r203537  
    605605    bool willCatchException = m_graph.willCatchExceptionInMachineFrame(m_speculative->m_currentNode->origin.forExit, opCatchOrigin, exceptionHandler);
    606606    if (willCatchException) {
    607         unsigned streamIndex = m_speculative->m_outOfLineStreamIndex != UINT_MAX ? m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size();
     607        unsigned streamIndex = m_speculative->m_outOfLineStreamIndex ? *m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size();
    608608        MacroAssembler::Jump hadException = emitNonPatchableExceptionCheck();
    609609        // We assume here that this is called after callOpeartion()/appendCall() is called.
  • trunk/Source/JavaScriptCore/dfg/DFGSlowPathGenerator.h

    r197563 r203537  
    5353        jit->m_origin = m_origin;
    5454        generateInternal(jit);
    55         jit->m_outOfLineStreamIndex = UINT_MAX;
     55        jit->m_outOfLineStreamIndex = Nullopt;
    5656        if (!ASSERT_DISABLED)
    5757            jit->m_jit.abortWithReason(DFGSlowPathGeneratorFellThrough);
  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r203491 r203537  
    4545#include "JITDivGenerator.h"
    4646#include "JITLeftShiftGenerator.h"
     47#include "JITMathIC.h"
    4748#include "JITMulGenerator.h"
    4849#include "JITRightShiftGenerator.h"
     
    5758#include "ScratchRegisterAllocator.h"
    5859#include "WriteBarrierBuffer.h"
     60#include <wtf/Box.h>
    5961#include <wtf/MathExtras.h>
    6062
     
    363365void SpeculativeJIT::addSlowPathGenerator(std::function<void()> lambda)
    364366{
    365     m_slowPathLambdas.append(std::make_pair(lambda, m_currentNode));
     367    m_slowPathLambdas.append(SlowPathLambda{ lambda, m_currentNode, static_cast<unsigned>(m_stream->size()) });
    366368}
    367369
     
    372374        slowPathGenerator->generate(this);
    373375    }
    374     for (auto& generatorPair : m_slowPathLambdas) {
    375         Node* currentNode = generatorPair.second;
     376    for (auto& slowPathLambda : m_slowPathLambdas) {
     377        Node* currentNode = slowPathLambda.currentNode;
    376378        m_currentNode = currentNode;
     379        m_outOfLineStreamIndex = slowPathLambda.streamIndex;
    377380        pcToCodeOriginMapBuilder.appendItem(m_jit.label(), currentNode->origin.semantic);
    378         generatorPair.first();
     381        slowPathLambda.generator();
     382        m_outOfLineStreamIndex = Nullopt;
    379383    }
    380384}
     
    33803384    }
    33813385
    3382     JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
    3383         leftFPR, rightFPR, scratchGPR, scratchFPR);
    3384     gen.generateFastPath(m_jit);
    3385 
    3386     ASSERT(gen.didEmitFastPath());
    3387     gen.endJumpList().append(m_jit.jump());
    3388 
    3389     gen.slowPathJumpList().link(&m_jit);
    3390 
    3391     silentSpillAllRegisters(resultRegs);
    3392 
    3393     if (leftOperand.isConst()) {
    3394         leftRegs = resultRegs;
    3395         m_jit.moveValue(leftChild->asJSValue(), leftRegs);
    3396     } else if (rightOperand.isConst()) {
    3397         rightRegs = resultRegs;
    3398         m_jit.moveValue(rightChild->asJSValue(), rightRegs);
    3399     }
    3400 
    3401     callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
    3402 
    3403     silentFillAllRegisters(resultRegs);
    3404     m_jit.exceptionCheck();
    3405 
    3406     gen.endJumpList().link(&m_jit);
     3386    JITAddIC* addIC = m_jit.codeBlock()->addJITAddIC();
     3387    Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
     3388    ArithProfile* arithProfile = m_jit.graph().baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
     3389    addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, leftFPR, rightFPR, scratchGPR, scratchFPR, arithProfile);
     3390
     3391    bool generatedInline = addIC->generateInline(m_jit, *addICGenerationState);
     3392
     3393    if (generatedInline) {
     3394        ASSERT(!addICGenerationState->slowPathJumps.empty());
     3395
     3396        Vector<SilentRegisterSavePlan> savePlans;
     3397        silentSpillAllRegistersImpl(false, savePlans, resultRegs);
     3398
     3399        auto done = m_jit.label();
     3400
     3401        addSlowPathGenerator([=, savePlans = WTFMove(savePlans)] () {
     3402            addICGenerationState->slowPathJumps.link(&m_jit);
     3403            addICGenerationState->slowPathStart = m_jit.label();
     3404
     3405            silentSpill(savePlans);
     3406
     3407            auto innerLeftRegs = leftRegs;
     3408            auto innerRightRegs = rightRegs;
     3409            if (leftOperand.isConst()) {
     3410                innerLeftRegs = resultRegs;
     3411                m_jit.moveValue(leftChild->asJSValue(), innerLeftRegs);
     3412            } else if (rightOperand.isConst()) {
     3413                innerRightRegs = resultRegs;
     3414                m_jit.moveValue(rightChild->asJSValue(), innerRightRegs);
     3415            }
     3416
     3417            if (addICGenerationState->shouldSlowPathRepatch)
     3418                addICGenerationState->slowPathCall = callOperation(operationValueAddOptimize, resultRegs, innerLeftRegs, innerRightRegs, addIC);
     3419            else
     3420                addICGenerationState->slowPathCall = callOperation(operationValueAdd, resultRegs, innerLeftRegs, innerRightRegs);
     3421
     3422            silentFill(savePlans);
     3423            m_jit.exceptionCheck();
     3424            m_jit.jump().linkTo(done, &m_jit);
     3425
     3426            m_jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
     3427                addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
     3428            });
     3429        });
     3430    } else {
     3431        if (leftOperand.isConst()) {
     3432            left = JSValueOperand(this, leftChild);
     3433            leftRegs = left->jsValueRegs();
     3434        } else if (rightOperand.isConst()) {
     3435            right = JSValueOperand(this, rightChild);
     3436            rightRegs = right->jsValueRegs();
     3437        }
     3438
     3439        flushRegisters();
     3440        callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
     3441        m_jit.exceptionCheck();
     3442    }
     3443
    34073444    jsValueResult(resultRegs, node);
    34083445    return;
  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

    r203491 r203537  
    389389        silentSpillAllRegistersImpl(doSpill, plans, InvalidGPRReg, InvalidGPRReg, exclude);
    390390    }
    391 #if USE(JSVALUE32_64)
    392391    template<typename CollectionType>
    393392    void silentSpillAllRegistersImpl(bool doSpill, CollectionType& plans, JSValueRegs exclude)
    394393    {
     394#if USE(JSVALUE32_64)
    395395        silentSpillAllRegistersImpl(doSpill, plans, exclude.tagGPR(), exclude.payloadGPR());
    396     }
    397 #endif
     396#else
     397        silentSpillAllRegistersImpl(doSpill, plans, exclude.gpr());
     398#endif
     399    }
    398400   
    399401    void silentSpillAllRegisters(GPRReg exclude, GPRReg exclude2 = InvalidGPRReg, FPRReg fprExclude = InvalidFPRReg)
     
    12861288
    12871289#if USE(JSVALUE64)
     1290
     1291    JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
     1292    {
     1293        m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), TrustedImmPtr(addIC));
     1294        return appendCallSetResult(operation, result.gpr());
     1295    }
     1296
    12881297    JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg result, GPRReg arg1, GPRReg arg2, UniquedStringImpl* uid)
    12891298    {
     
    17021711    }
    17031712#else // USE(JSVALUE32_64)
     1713
     1714    JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
     1715    {
     1716        m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), TrustedImmPtr(addIC));
     1717        return appendCallSetResult(operation, result.payloadGPR(), result.tagGPR());
     1718    }
    17041719
    17051720    JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg resultTag, GPRReg resultPayload, GPRReg arg1Tag, GPRReg arg1Payload, GPRReg arg2Tag, GPRReg arg2Payload, UniquedStringImpl* uid)
     
    28332848   
    28342849    Vector<std::unique_ptr<SlowPathGenerator>, 8> m_slowPathGenerators;
    2835     Vector<std::pair<std::function<void()>, Node*>, 8> m_slowPathLambdas;
     2850    struct SlowPathLambda {
     2851        std::function<void()> generator;
     2852        Node* currentNode;
     2853        unsigned streamIndex;
     2854    };
     2855    Vector<SlowPathLambda> m_slowPathLambdas;
    28362856    Vector<SilentRegisterSavePlan> m_plans;
    2837     unsigned m_outOfLineStreamIndex { UINT_MAX };
     2857    Optional<unsigned> m_outOfLineStreamIndex;
    28382858};
    28392859
  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

    r203491 r203537  
    6363#include "JITInlineCacheGenerator.h"
    6464#include "JITLeftShiftGenerator.h"
     65#include "JITMathIC.h"
    6566#include "JITMulGenerator.h"
    6667#include "JITRightShiftGenerator.h"
     
    15391540    void compileValueAdd()
    15401541    {
    1541         emitBinarySnippet<JITAddGenerator>(operationValueAdd);
     1542        Node* node = m_node;
     1543       
     1544        LValue left = lowJSValue(node->child1());
     1545        LValue right = lowJSValue(node->child2());
     1546
     1547        SnippetOperand leftOperand(m_state.forNode(node->child1()).resultType());
     1548        SnippetOperand rightOperand(m_state.forNode(node->child2()).resultType());
     1549           
     1550        PatchpointValue* patchpoint = m_out.patchpoint(Int64);
     1551        patchpoint->appendSomeRegister(left);
     1552        patchpoint->appendSomeRegister(right);
     1553        patchpoint->append(m_tagMask, ValueRep::lateReg(GPRInfo::tagMaskRegister));
     1554        patchpoint->append(m_tagTypeNumber, ValueRep::lateReg(GPRInfo::tagTypeNumberRegister));
     1555        RefPtr<PatchpointExceptionHandle> exceptionHandle =
     1556            preparePatchpointForExceptions(patchpoint);
     1557        patchpoint->numGPScratchRegisters = 1;
     1558        patchpoint->numFPScratchRegisters = 2;
     1559        patchpoint->clobber(RegisterSet::macroScratchRegisters());
     1560        State* state = &m_ftlState;
     1561        patchpoint->setGenerator(
     1562            [=] (CCallHelpers& jit, const StackmapGenerationParams& params) {
     1563                AllowMacroScratchRegisterUsage allowScratch(jit);
     1564
     1565                Box<CCallHelpers::JumpList> exceptions =
     1566                    exceptionHandle->scheduleExitCreation(params)->jumps(jit);
     1567
     1568                JITAddIC* addIC = jit.codeBlock()->addJITAddIC();
     1569                Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create();
     1570                ArithProfile* arithProfile = state->graph.baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex);
     1571                addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, JSValueRegs(params[0].gpr()),
     1572                    JSValueRegs(params[1].gpr()), JSValueRegs(params[2].gpr()), params.fpScratch(0),
     1573                    params.fpScratch(1), params.gpScratch(0), InvalidFPRReg, arithProfile);
     1574
     1575                bool generatedInline = addIC->generateInline(jit, *addICGenerationState);
     1576
     1577                if (generatedInline) {
     1578                    ASSERT(!addICGenerationState->slowPathJumps.empty());
     1579                    auto done = jit.label();
     1580                    params.addLatePath([=] (CCallHelpers& jit) {
     1581                        AllowMacroScratchRegisterUsage allowScratch(jit);
     1582                        addICGenerationState->slowPathJumps.link(&jit);
     1583                        addICGenerationState->slowPathStart = jit.label();
     1584
     1585                        if (addICGenerationState->shouldSlowPathRepatch) {
     1586                            SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
     1587                                operationValueAddOptimize, params[0].gpr(), params[1].gpr(), params[2].gpr(), CCallHelpers::TrustedImmPtr(addIC));
     1588                            addICGenerationState->slowPathCall = call.call();
     1589                        } else {
     1590                            SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic,
     1591                                exceptions.get(), operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
     1592                            addICGenerationState->slowPathCall = call.call();
     1593                        }
     1594                        jit.jump().linkTo(done, &jit);
     1595
     1596                        jit.addLinkTask([=] (LinkBuffer& linkBuffer) {
     1597                            addIC->finalizeInlineCode(*addICGenerationState, linkBuffer);
     1598                        });
     1599                    });
     1600                } else {
     1601                    callOperation(
     1602                        *state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(),
     1603                        operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr());
     1604                }
     1605            });
     1606
     1607        setJSValue(patchpoint);
    15421608    }
    15431609   
  • trunk/Source/JavaScriptCore/jit/CCallHelpers.h

    r203006 r203537  
    816816   
    817817    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, GPRReg arg5, TrustedImm32 arg6)
     818    {
     819        resetCallArguments();
     820        addCallArgument(GPRInfo::callFrameRegister);
     821        addCallArgument(arg1);
     822        addCallArgument(arg2);
     823        addCallArgument(arg3);
     824        addCallArgument(arg4);
     825        addCallArgument(arg5);
     826        addCallArgument(arg6);
     827    }
     828
     829    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6)
    818830    {
    819831        resetCallArguments();
     
    15951607    }
    15961608
     1609    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6)
     1610    {
     1611        poke(arg6, POKE_ARGUMENT_OFFSET + 2);
     1612        poke(arg5, POKE_ARGUMENT_OFFSET + 1);
     1613        poke(arg4, POKE_ARGUMENT_OFFSET);
     1614        setupArgumentsWithExecState(arg1, arg2, arg3);
     1615    }
     1616
    15971617    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5)
    15981618    {
     
    21002120    }
    21012121
     2122    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, TrustedImmPtr arg3, TrustedImmPtr arg4)
     2123    {
     2124        setupTwoStubArgsGPR<GPRInfo::argumentGPR1, GPRInfo::argumentGPR2>(arg1, arg2);
     2125        move(arg3, GPRInfo::argumentGPR3);
     2126        move(arg4, GPRInfo::argumentGPR4);
     2127        move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
     2128    }
     2129
    21022130    ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4, GPRReg arg5)
    21032131    {
     
    22082236#endif
    22092237   
     2238    void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2)
     2239    {
     2240#if USE(JSVALUE64)
     2241        setupArgumentsWithExecState(arg1.gpr(), arg2.gpr());
     2242#else
     2243        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR());
     2244#endif
     2245    }
     2246
    22102247    void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3)
    22112248    {
     
    22142251#else
    22152252        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3);
     2253#endif
     2254    }
     2255
     2256    void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3, TrustedImmPtr arg4)
     2257    {
     2258#if USE(JSVALUE64)
     2259        setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), arg3, arg4);
     2260#else
     2261        setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3, arg4);
    22162262#endif
    22172263    }
  • trunk/Source/JavaScriptCore/jit/JIT.h

    r203499 r203537  
    4444#include "JITDisassembler.h"
    4545#include "JITInlineCacheGenerator.h"
     46#include "JITMathIC.h"
    4647#include "JSInterfaceJIT.h"
    4748#include "Opcode.h"
    4849#include "PCToCodeOriginMap.h"
    49 #include "ResultType.h"
    5050#include "UnusedPointer.h"
    5151
     
    436436
    437437        // Arithmetic opcode helpers
    438         void emitSub32Constant(int dst, int op, int32_t constant, ResultType opType);
    439438        void emitBinaryDoubleOp(OpcodeID, int dst, int op1, int op2, OperandTypes, JumpList& notInt32Op1, JumpList& notInt32Op2, bool op1IsInRegisters = true, bool op2IsInRegisters = true);
    440439
     
    765764        MacroAssembler::Call callOperation(J_JITOperation_EJI, int, GPRReg, UniquedStringImpl*);
    766765        MacroAssembler::Call callOperation(J_JITOperation_EJJ, int, GPRReg, GPRReg);
    767         MacroAssembler::Call callOperation(J_JITOperation_EJJRp, JSValueRegs, JSValueRegs, JSValueRegs, ResultProfile*);
     766        MacroAssembler::Call callOperation(J_JITOperation_EJJArp, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*);
     767        MacroAssembler::Call callOperation(J_JITOperation_EJJ, JSValueRegs, JSValueRegs, JSValueRegs);
     768        MacroAssembler::Call callOperation(J_JITOperation_EJJArpJaic, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*, JITAddIC*);
     769        MacroAssembler::Call callOperation(J_JITOperation_EJJJaic, JSValueRegs, JSValueRegs, JSValueRegs, JITAddIC*);
    768770        MacroAssembler::Call callOperation(J_JITOperation_EJJAp, int, GPRReg, GPRReg, ArrayProfile*);
    769771        MacroAssembler::Call callOperation(J_JITOperation_EJJBy, int, GPRReg, GPRReg, ByValInfo*);
     
    955957        PCToCodeOriginMapBuilder m_pcToCodeOriginMapBuilder;
    956958
     959        HashMap<Instruction*, JITAddIC*> m_instructionToJITAddIC;
     960        HashMap<Instruction*, MathICGenerationState> m_instructionToJITAddICGenerationState;
     961
    957962        bool m_canBeOptimized;
    958963        bool m_canBeOptimizedOrInlined;
  • trunk/Source/JavaScriptCore/jit/JITAddGenerator.cpp

    r200606 r203537  
    2727#include "JITAddGenerator.h"
    2828
     29#include "ArithProfile.h"
     30#include "JITMathIC.h"
     31#include "LinkBuffer.h"
     32
    2933#if ENABLE(JIT)
    3034
    3135namespace JSC {
    3236
    33 void JITAddGenerator::generateFastPath(CCallHelpers& jit)
     37JITMathICInlineResult JITAddGenerator::generateInline(CCallHelpers& jit, MathICGenerationState& state)
     38{
     39    // We default to speculating int32.
     40    ObservedType lhs = ObservedType().withInt32();
     41    ObservedType rhs = ObservedType().withInt32();
     42    if (m_arithProfile) {
     43        lhs = m_arithProfile->lhsObservedType();
     44        rhs = m_arithProfile->rhsObservedType();
     45        if (lhs.isEmpty() || rhs.isEmpty()) {
     46            lhs = ObservedType().withInt32();
     47            rhs = ObservedType().withInt32();
     48        }
     49    }
     50
     51    if (lhs.isOnlyNonNumber() && rhs.isOnlyNonNumber())
     52        return JITMathICInlineResult::DontGenerate;
     53
     54    if ((lhs.isOnlyInt32() || m_leftOperand.isConstInt32()) && (rhs.isOnlyInt32() || m_rightOperand.isConstInt32())) {
     55        ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
     56        if (!m_leftOperand.isConstInt32())
     57            state.slowPathJumps.append(jit.branchIfNotInt32(m_left));
     58        if (!m_rightOperand.isConstInt32())
     59            state.slowPathJumps.append(jit.branchIfNotInt32(m_right));
     60
     61        if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) {
     62            JSValueRegs var = m_leftOperand.isConstInt32() ? m_right : m_left;
     63            int32_t constValue = m_leftOperand.isConstInt32() ? m_leftOperand.asConstInt32() : m_rightOperand.asConstInt32();
     64            state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constValue), m_scratchGPR));
     65        } else
     66            state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
     67        jit.boxInt32(m_scratchGPR, m_result);
     68        return JITMathICInlineResult::GeneratedFastPath;
     69    }
     70
     71    return JITMathICInlineResult::GenerateFullSnippet;
     72}
     73
     74bool JITAddGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList)
    3475{
    3576    ASSERT(m_scratchGPR != InvalidGPRReg);
     
    4485    ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
    4586   
    46     if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber()) {
    47         ASSERT(!m_didEmitFastPath);
    48         return;
    49     }
    50 
    51     m_didEmitFastPath = true;
     87    if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber())
     88        return false;
    5289
    5390    if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) {
     
    5996        CCallHelpers::Jump notInt32 = jit.branchIfNotInt32(var);
    6097
    61         m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR));
     98        slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR));
    6299
    63100        jit.boxInt32(m_scratchGPR, m_result);
    64         m_endJumpList.append(jit.jump());
     101        endJumpList.append(jit.jump());
    65102
    66103        if (!jit.supportsFloatingPoint()) {
    67             m_slowPathJumpList.append(notInt32);
    68             return;
     104            slowPathJumpList.append(notInt32);
     105            return true;
    69106        }
    70107
     
    72109        notInt32.link(&jit);
    73110        if (!varOpr.definitelyIsNumber())
    74             m_slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
     111            slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));
    75112
    76113        jit.unboxDoubleNonDestructive(var, m_leftFPR, m_scratchGPR, m_scratchFPR);
     
    90127        rightNotInt = jit.branchIfNotInt32(m_right);
    91128
    92         m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
     129        slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));
    93130
    94131        jit.boxInt32(m_scratchGPR, m_result);
    95         m_endJumpList.append(jit.jump());
     132        endJumpList.append(jit.jump());
     133
    96134
    97135        if (!jit.supportsFloatingPoint()) {
    98             m_slowPathJumpList.append(leftNotInt);
    99             m_slowPathJumpList.append(rightNotInt);
    100             return;
     136            slowPathJumpList.append(leftNotInt);
     137            slowPathJumpList.append(rightNotInt);
     138            return true;
    101139        }
    102140
    103141        leftNotInt.link(&jit);
    104142        if (!m_leftOperand.definitelyIsNumber())
    105             m_slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
     143            slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));
    106144        if (!m_rightOperand.definitelyIsNumber())
    107             m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
     145            slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
    108146
    109147        jit.unboxDoubleNonDestructive(m_left, m_leftFPR, m_scratchGPR, m_scratchFPR);
     
    115153        rightNotInt.link(&jit);
    116154        if (!m_rightOperand.definitelyIsNumber())
    117             m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
     155            slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));
    118156
    119157        jit.convertInt32ToDouble(m_left.payloadGPR(), m_leftFPR);
     
    129167    // Do doubleVar + doubleVar.
    130168    jit.addDouble(m_rightFPR, m_leftFPR);
    131     if (m_resultProfile)
    132         m_resultProfile->emitSetDouble(jit);
     169    if (m_arithProfile)
     170        m_arithProfile->emitSetDouble(jit);
    133171       
    134172    jit.boxDouble(m_leftFPR, m_result);
     173
     174    return true;
    135175}
    136176
  • trunk/Source/JavaScriptCore/jit/JITAddGenerator.h

    r200606 r203537  
    3030
    3131#include "CCallHelpers.h"
     32#include "JITMathICInlineResult.h"
     33#include "JITOperations.h"
    3234#include "SnippetOperand.h"
    3335
    3436namespace JSC {
    3537
     38struct MathICGenerationState;
     39
    3640class JITAddGenerator {
    3741public:
     42    JITAddGenerator()
     43    { }
     44
    3845    JITAddGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand,
    3946        JSValueRegs result, JSValueRegs left, JSValueRegs right,
    4047        FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
    41         ResultProfile* resultProfile = nullptr)
     48        ArithProfile* arithProfile = nullptr)
    4249        : m_leftOperand(leftOperand)
    4350        , m_rightOperand(rightOperand)
     
    4956        , m_scratchGPR(scratchGPR)
    5057        , m_scratchFPR(scratchFPR)
    51         , m_resultProfile(resultProfile)
     58        , m_arithProfile(arithProfile)
    5259    {
    5360        ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
    5461    }
    5562
    56     void generateFastPath(CCallHelpers&);
    57 
    58     bool didEmitFastPath() const { return m_didEmitFastPath; }
    59     CCallHelpers::JumpList& endJumpList() { return m_endJumpList; }
    60     CCallHelpers::JumpList& slowPathJumpList() { return m_slowPathJumpList; }
     63    JITMathICInlineResult generateInline(CCallHelpers&, MathICGenerationState&);
     64    bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList);
    6165
    6266private:
     
    7074    GPRReg m_scratchGPR;
    7175    FPRReg m_scratchFPR;
    72     ResultProfile* m_resultProfile;
    73     bool m_didEmitFastPath { false };
    74 
    75     CCallHelpers::JumpList m_endJumpList;
    76     CCallHelpers::JumpList m_slowPathJumpList;
     76    ArithProfile* m_arithProfile;
    7777};
    7878
  • trunk/Source/JavaScriptCore/jit/JITArithmetic.cpp

    r203499 r203537  
    2929#include "JIT.h"
    3030
     31#include "ArithProfile.h"
    3132#include "CodeBlock.h"
    3233#include "JITAddGenerator.h"
     
    3738#include "JITInlines.h"
    3839#include "JITLeftShiftGenerator.h"
     40#include "JITMathIC.h"
    3941#include "JITMulGenerator.h"
    4042#include "JITNegGenerator.h"
     
    4648#include "Interpreter.h"
    4749#include "JSCInlines.h"
     50#include "LinkBuffer.h"
    4851#include "ResultType.h"
    4952#include "SlowPathCall.h"
    50 
    5153
    5254namespace JSC {
     
    676678}
    677679
     680ALWAYS_INLINE static OperandTypes getOperandTypes(Instruction* instruction)
     681{
     682    return OperandTypes(ArithProfile::fromInt(instruction[4].u.operand).lhsResultType(), ArithProfile::fromInt(instruction[4].u.operand).rhsResultType());
     683}
     684
    678685void JIT::emit_op_add(Instruction* currentInstruction)
    679686{
     
    681688    int op1 = currentInstruction[2].u.operand;
    682689    int op2 = currentInstruction[3].u.operand;
    683     OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
    684 
    685 #if USE(JSVALUE64)
     690
     691#if USE(JSVALUE64)
     692    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
    686693    JSValueRegs leftRegs = JSValueRegs(regT0);
    687694    JSValueRegs rightRegs = JSValueRegs(regT1);
     
    690697    FPRReg scratchFPR = InvalidFPRReg;
    691698#else
     699    OperandTypes types = getOperandTypes(currentInstruction);
    692700    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
    693701    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     
    697705#endif
    698706
    699     ResultProfile* resultProfile = nullptr;
     707    ArithProfile* arithProfile = nullptr;
    700708    if (shouldEmitProfiling())
    701         resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
     709        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
    702710
    703711    SnippetOperand leftOperand(types.first());
     
    716724        emitGetVirtualRegister(op2, rightRegs);
    717725
    718     JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
    719         fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
    720 
    721     gen.generateFastPath(*this);
    722 
    723     if (gen.didEmitFastPath()) {
    724         gen.endJumpList().link(this);
    725         emitPutVirtualRegister(result, resultRegs);
    726        
    727         addSlowCase(gen.slowPathJumpList());
    728     } else {
    729         ASSERT(gen.endJumpList().empty());
    730         ASSERT(gen.slowPathJumpList().empty());
    731         if (resultProfile) {
    732             if (leftOperand.isConst())
    733                 emitGetVirtualRegister(op1, leftRegs);
    734             if (rightOperand.isConst())
    735                 emitGetVirtualRegister(op2, rightRegs);
    736             callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
    737             emitPutVirtualRegister(result, resultRegs);
    738         } else {
    739             JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add);
    740             slowPathCall.call();
    741         }
    742     }
     726    JITAddIC* addIC = m_codeBlock->addJITAddIC();
     727
     728    m_instructionToJITAddIC.add(currentInstruction, addIC);
     729    MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.add(currentInstruction, MathICGenerationState()).iterator->value;
     730
     731    addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
     732
     733    bool generatedInlineCode = addIC->generateInline(*this, addICGenerationState);
     734    if (!generatedInlineCode) {
     735        if (leftOperand.isConst())
     736            emitGetVirtualRegister(op1, leftRegs);
     737        else if (rightOperand.isConst())
     738            emitGetVirtualRegister(op2, rightRegs);
     739
     740        if (arithProfile)
     741            callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
     742        else
     743            callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs);
     744    } else
     745        addSlowCase(addICGenerationState.slowPathJumps);
     746    emitPutVirtualRegister(result, resultRegs);
    743747}
    744748
     
    746750{
    747751    linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
     752
     753    MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
     754    addICGenerationState.slowPathStart = label();
    748755
    749756    int result = currentInstruction[1].u.operand;
    750757    int op1 = currentInstruction[2].u.operand;
    751758    int op2 = currentInstruction[3].u.operand;
    752     OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
    753 
    754 #if USE(JSVALUE64)
     759
     760#if USE(JSVALUE64)
     761    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
    755762    JSValueRegs leftRegs = JSValueRegs(regT0);
    756763    JSValueRegs rightRegs = JSValueRegs(regT1);
    757764    JSValueRegs resultRegs = leftRegs;
    758765#else
     766    OperandTypes types = getOperandTypes(currentInstruction);
    759767    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
    760768    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     
    770778        rightOperand.setConstInt32(getOperandConstantInt(op2));
    771779
     780    if (leftOperand.isConst())
     781        emitGetVirtualRegister(op1, leftRegs);
     782    if (rightOperand.isConst())
     783        emitGetVirtualRegister(op2, rightRegs);
     784
     785    JITAddIC* addIC = m_instructionToJITAddIC.get(currentInstruction);
    772786    if (shouldEmitProfiling()) {
    773         if (leftOperand.isConst())
    774             emitGetVirtualRegister(op1, leftRegs);
    775         if (rightOperand.isConst())
    776             emitGetVirtualRegister(op2, rightRegs);
    777         ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
    778         callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
    779         emitPutVirtualRegister(result, resultRegs);
    780     } else {
    781         JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add);
    782         slowPathCall.call();
    783     }
     787        ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
     788        if (addICGenerationState.shouldSlowPathRepatch)
     789            addICGenerationState.slowPathCall = callOperation(operationValueAddProfiledOptimize, resultRegs, leftRegs, rightRegs, &arithProfile, addIC);
     790        else
     791            addICGenerationState.slowPathCall = callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
     792    } else
     793        addICGenerationState.slowPathCall = callOperation(operationValueAddOptimize, resultRegs, leftRegs, rightRegs, addIC);
     794    emitPutVirtualRegister(result, resultRegs);
     795
     796    addLinkTask([=] (LinkBuffer& linkBuffer) {
     797        MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value;
     798        addIC->finalizeInlineCode(addICGenerationState, linkBuffer);
     799    });
    784800}
    785801
     
    789805    int op1 = currentInstruction[2].u.operand;
    790806    int op2 = currentInstruction[3].u.operand;
    791     OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
    792 
    793 #if USE(JSVALUE64)
     807
     808#if USE(JSVALUE64)
     809    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
    794810    JSValueRegs leftRegs = JSValueRegs(regT0);
    795811    JSValueRegs rightRegs = JSValueRegs(regT1);
     
    797813    GPRReg scratchGPR = regT2;
    798814#else
     815    OperandTypes types = getOperandTypes(currentInstruction);
    799816    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
    800817    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     
    804821    FPRReg scratchFPR = fpRegT2;
    805822
    806     ResultProfile* resultProfile = nullptr;
     823    ArithProfile* arithProfile = nullptr;
    807824    if (shouldEmitProfiling())
    808         resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
     825        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
    809826
    810827    SnippetOperand leftOperand(types.first());
     
    832849
    833850    JITDivGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
    834         fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
     851        fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
    835852
    836853    gen.generateFastPath(*this);
     
    862879    int op1 = currentInstruction[2].u.operand;
    863880    int op2 = currentInstruction[3].u.operand;
    864     OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
    865 
    866 #if USE(JSVALUE64)
     881
     882#if USE(JSVALUE64)
     883    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
    867884    JSValueRegs leftRegs = JSValueRegs(regT0);
    868885    JSValueRegs rightRegs = JSValueRegs(regT1);
     
    871888    FPRReg scratchFPR = InvalidFPRReg;
    872889#else
     890    OperandTypes types = getOperandTypes(currentInstruction);
    873891    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
    874892    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     
    878896#endif
    879897
    880     ResultProfile* resultProfile = nullptr;
     898    ArithProfile* arithProfile = nullptr;
    881899    if (shouldEmitProfiling())
    882         resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
     900        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
    883901
    884902    SnippetOperand leftOperand(types.first());
     
    898916
    899917    JITMulGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
    900         fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
     918        fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
    901919
    902920    gen.generateFastPath(*this);
     
    910928        ASSERT(gen.endJumpList().empty());
    911929        ASSERT(gen.slowPathJumpList().empty());
    912         if (resultProfile) {
     930        if (arithProfile) {
    913931            if (leftOperand.isPositiveConstInt32())
    914932                emitGetVirtualRegister(op1, leftRegs);
    915933            if (rightOperand.isPositiveConstInt32())
    916934                emitGetVirtualRegister(op2, rightRegs);
    917             callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
     935            callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, arithProfile);
    918936            emitPutVirtualRegister(result, resultRegs);
    919937        } else {
     
    931949    int op1 = currentInstruction[2].u.operand;
    932950    int op2 = currentInstruction[3].u.operand;
    933     OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
    934 
    935 #if USE(JSVALUE64)
     951
     952#if USE(JSVALUE64)
     953    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
    936954    JSValueRegs leftRegs = JSValueRegs(regT0);
    937955    JSValueRegs rightRegs = JSValueRegs(regT1);
    938956    JSValueRegs resultRegs = leftRegs;
    939957#else
     958    OperandTypes types = getOperandTypes(currentInstruction);
    940959    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
    941960    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     
    956975        if (rightOperand.isPositiveConstInt32())
    957976            emitGetVirtualRegister(op2, rightRegs);
    958         ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
    959         callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
     977        ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
     978        callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
    960979        emitPutVirtualRegister(result, resultRegs);
    961980    } else {
     
    970989    int op1 = currentInstruction[2].u.operand;
    971990    int op2 = currentInstruction[3].u.operand;
    972     OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
    973 
    974 #if USE(JSVALUE64)
     991
     992#if USE(JSVALUE64)
     993    OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction));
    975994    JSValueRegs leftRegs = JSValueRegs(regT0);
    976995    JSValueRegs rightRegs = JSValueRegs(regT1);
     
    979998    FPRReg scratchFPR = InvalidFPRReg;
    980999#else
     1000    OperandTypes types = getOperandTypes(currentInstruction);
    9811001    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
    9821002    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
     
    9861006#endif
    9871007
    988     ResultProfile* resultProfile = nullptr;
     1008    ArithProfile* arithProfile = nullptr;
    9891009    if (shouldEmitProfiling())
    990         resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
     1010        arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction);
    9911011
    9921012    SnippetOperand leftOperand(types.first());
     
    9971017
    9981018    JITSubGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs,
    999         fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);
     1019        fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile);
    10001020
    10011021    gen.generateFastPath(*this);
     
    10241044
    10251045    if (shouldEmitProfiling()) {
    1026         ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);
    1027         callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, resultProfile);
     1046        ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction);
     1047        callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, &arithProfile);
    10281048        emitPutVirtualRegister(result, resultRegs);
    10291049    } else {
  • trunk/Source/JavaScriptCore/jit/JITDivGenerator.cpp

    r199866 r203537  
    2929#if ENABLE(JIT)
    3030
     31#include "ArithProfile.h"
    3132#include "JSCJSValueInlines.h"
    3233#include "MathCommon.h"
     
    102103
    103104    // Is the result actually an integer? The DFG JIT would really like to know. If it's
    104     // not an integer, we increment a count. If this together with the slow case counter
    105     // are below threshold then the DFG JIT will compile this division with a speculation
    106     // that the remainder is zero.
     105    // not an integer, we set a bit. If this together with the slow case counter are below
     106    // threshold then the DFG JIT will compile this division with a speculation that the
     107    // remainder is zero.
    107108
    108109    // As well, there are cases where a double result here would cause an important field
     
    129130    notDoubleZero.link(&jit);
    130131#endif
    131     if (m_resultProfile)
    132         jit.add32(CCallHelpers::TrustedImm32(1), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfSpecialFastPathCount()));
     132    if (m_arithProfile)
     133        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::specialFastPathBit), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
    133134    jit.boxDouble(m_leftFPR, m_result);
    134135}
  • trunk/Source/JavaScriptCore/jit/JITDivGenerator.h

    r194294 r203537  
    3939        JSValueRegs result, JSValueRegs left, JSValueRegs right,
    4040        FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
    41         ResultProfile* resultProfile = nullptr)
     41        ArithProfile* arithProfile = nullptr)
    4242        : m_leftOperand(leftOperand)
    4343        , m_rightOperand(rightOperand)
     
    4949        , m_scratchGPR(scratchGPR)
    5050        , m_scratchFPR(scratchFPR)
    51         , m_resultProfile(resultProfile)
     51        , m_arithProfile(arithProfile)
    5252    {
    5353        ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32());
     
    7272    GPRReg m_scratchGPR;
    7373    FPRReg m_scratchFPR;
    74     ResultProfile* m_resultProfile;
     74    ArithProfile* m_arithProfile;
    7575    bool m_didEmitFastPath { false };
    7676
  • trunk/Source/JavaScriptCore/jit/JITInlines.h

    r203006 r203537  
    417417}
    418418
    419 inline MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJRp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ResultProfile* resultProfile)
    420 {
    421     setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(resultProfile));
     419ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJ operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2)
     420{
     421    setupArgumentsWithExecState(arg1, arg2);
     422    Call call = appendCallWithExceptionCheck(operation);
     423    setupResults(result);
     424    return call;
     425}
     426
     427ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile)
     428{
     429    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile));
     430    Call call = appendCallWithExceptionCheck(operation);
     431    setupResults(result);
     432    return call;
     433}
     434
     435ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArpJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile, JITAddIC* addIC)
     436{
     437    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile), TrustedImmPtr(addIC));
     438    Call call = appendCallWithExceptionCheck(operation);
     439    setupResults(result);
     440    return call;
     441}
     442
     443ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC)
     444{
     445    setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(addIC));
    422446    Call call = appendCallWithExceptionCheck(operation);
    423447    setupResults(result);
  • trunk/Source/JavaScriptCore/jit/JITMulGenerator.cpp

    r200606 r203537  
    2929#if ENABLE(JIT)
    3030
     31#include "ArithProfile.h"
     32
    3133namespace JSC {
    3234
     
    138140    jit.mulDouble(m_rightFPR, m_leftFPR);
    139141
    140     if (!m_resultProfile)
     142    if (!m_arithProfile)
    141143        jit.boxDouble(m_leftFPR, m_result);
    142144    else {
     
    150152        CCallHelpers::Jump notNegativeZero = jit.branch64(CCallHelpers::NotEqual, m_result.payloadGPR(), CCallHelpers::TrustedImm64(negativeZeroBits));
    151153
    152         jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
     154        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
    153155        CCallHelpers::Jump done = jit.jump();
    154156
    155157        notNegativeZero.link(&jit);
    156         jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
     158        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
    157159
    158160        jit.move(m_result.payloadGPR(), m_scratchGPR);
     
    161163        CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431));
    162164
    163         jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
     165        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
    164166        noInt52Overflow.link(&jit);
    165167
     
    172174        notNegativeZero.append(jit.branch32(CCallHelpers::NotEqual, m_result.tagGPR(), CCallHelpers::TrustedImm32(negativeZeroBits >> 32)));
    173175
    174         jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
     176        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
    175177        CCallHelpers::Jump done = jit.jump();
    176178
    177179        notNegativeZero.link(&jit);
    178         jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
     180        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
    179181
    180182        jit.move(m_result.tagGPR(), m_scratchGPR);
     
    183185        CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431));
    184186       
    185         jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));
     187        jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits()));
    186188
    187189        m_endJumpList.append(noInt52Overflow);
  • trunk/Source/JavaScriptCore/jit/JITMulGenerator.h

    r194294 r203537  
    3939        JSValueRegs result, JSValueRegs left, JSValueRegs right,
    4040        FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
    41         ResultProfile* resultProfile = nullptr)
     41        ArithProfile* arithProfile = nullptr)
    4242        : m_leftOperand(leftOperand)
    4343        , m_rightOperand(rightOperand)
     
    4949        , m_scratchGPR(scratchGPR)
    5050        , m_scratchFPR(scratchFPR)
    51         , m_resultProfile(resultProfile)
     51        , m_arithProfile(arithProfile)
    5252    {
    5353        ASSERT(!m_leftOperand.isPositiveConstInt32() || !m_rightOperand.isPositiveConstInt32());
     
    7070    GPRReg m_scratchGPR;
    7171    FPRReg m_scratchFPR;
    72     ResultProfile* m_resultProfile;
     72    ArithProfile* m_arithProfile;
    7373    bool m_didEmitFastPath { false };
    7474
  • trunk/Source/JavaScriptCore/jit/JITOperations.cpp

    r203452 r203537  
    2929#if ENABLE(JIT)
    3030
     31#include "ArithProfile.h"
    3132#include "ArrayConstructor.h"
    3233#include "CommonSlowPaths.h"
     
    22482249}
    22492250
    2250 EncodedJSValue JIT_OPERATION operationValueAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
     2251ALWAYS_INLINE static EncodedJSValue unprofiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
    22512252{
    22522253    VM* vm = &exec->vm();
     
    22592260}
    22602261
    2261 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
     2262ALWAYS_INLINE static EncodedJSValue profiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
    22622263{
    22632264    VM* vm = &exec->vm();
     
    22662267    JSValue op1 = JSValue::decode(encodedOp1);
    22672268    JSValue op2 = JSValue::decode(encodedOp2);
    2268    
     2269
     2270    ASSERT(arithProfile);
     2271    arithProfile->observeLHSAndRHS(op1, op2);
     2272
    22692273    JSValue result = jsAdd(exec, op1, op2);
    2270     resultProfile->detectNumericness(result);
     2274    arithProfile->observeResult(result);
     2275
     2276    return JSValue::encode(result);
     2277}
     2278
     2279EncodedJSValue JIT_OPERATION operationValueAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)
     2280{
     2281    return unprofiledAdd(exec, encodedOp1, encodedOp2);
     2282}
     2283
     2284EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
     2285{
     2286    return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile);
     2287}
     2288
     2289EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC* addIC)
     2290{
     2291    VM* vm = &exec->vm();
     2292    NativeCallFrameTracer tracer(vm, exec);
     2293   
     2294    JSValue op1 = JSValue::decode(encodedOp1);
     2295    JSValue op2 = JSValue::decode(encodedOp2);
     2296
     2297    ASSERT(arithProfile);
     2298    arithProfile->observeLHSAndRHS(op1, op2);
     2299    auto nonOptimizeVariant = operationValueAddProfiledNoOptimize;
     2300    addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
     2301   
     2302    JSValue result = jsAdd(exec, op1, op2);
     2303    arithProfile->observeResult(result);
     2304
     2305    return JSValue::encode(result);
     2306}
     2307
     2308EncodedJSValue JIT_OPERATION operationValueAddProfiledNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC*)
     2309{
     2310    return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile);
     2311}
     2312
     2313EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC* addIC)
     2314{
     2315    VM* vm = &exec->vm();
     2316    NativeCallFrameTracer tracer(vm, exec);
     2317
     2318    auto nonOptimizeVariant = operationValueAddNoOptimize;
     2319    addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant);
     2320
     2321    JSValue op1 = JSValue::decode(encodedOp1);
     2322    JSValue op2 = JSValue::decode(encodedOp2);
     2323    return JSValue::encode(jsAdd(exec, op1, op2));
     2324}
     2325
     2326EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*)
     2327{
     2328    VM* vm = &exec->vm();
     2329    NativeCallFrameTracer tracer(vm, exec);
     2330   
     2331    JSValue op1 = JSValue::decode(encodedOp1);
     2332    JSValue op2 = JSValue::decode(encodedOp2);
     2333   
     2334    JSValue result = jsAdd(exec, op1, op2);
     2335
    22712336    return JSValue::encode(result);
    22722337}
     
    22852350}
    22862351
    2287 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
     2352EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
    22882353{
    22892354    VM* vm = &exec->vm();
     
    22972362   
    22982363    JSValue result = jsNumber(a * b);
    2299     resultProfile->detectNumericness(result);
     2364    arithProfile->observeResult(result);
    23002365    return JSValue::encode(result);
    23012366}
     
    23142379}
    23152380
    2316 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)
     2381EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile)
    23172382{
    23182383    VM* vm = &exec->vm();
     
    23262391   
    23272392    JSValue result = jsNumber(a - b);
    2328     resultProfile->detectNumericness(result);
     2393    arithProfile->observeResult(result);
    23292394    return JSValue::encode(result);
    23302395}
  • trunk/Source/JavaScriptCore/jit/JITOperations.h

    r202125 r203537  
    2929#if ENABLE(JIT)
    3030
     31#include "JITMathICForwards.h"
    3132#include "MacroAssemblerCodeRef.h"
    3233#include "PropertyOffset.h"
     
    4344class CodeBlock;
    4445class ExecState;
     46class JITAddGenerator;
    4547class JSArray;
    4648class JSFunction;
     
    5557struct ByValInfo;
    5658struct InlineCallFrame;
    57 struct ResultProfile;
     59struct ArithProfile;
    5860
    5961typedef ExecState CallFrame;
     
    7072    Aap: ArrayAllocationProfile*
    7173    Ap: ArrayProfile*
     74    Arp: ArithProfile*
    7275    By: ByValInfo*
    7376    C: JSCell*
     
    8285    Idc: const Identifier*
    8386    J: EncodedJSValue
     87    Jaic: JITAddIC*
    8488    Jcp: const JSValue*
    8589    Jsc: JSScope*
     
    9397    R: Register
    9498    Reo: RegExpObject*
    95     Rp: ResultProfile*
    9699    S: size_t
    97100    Sprt: SlowPathReturnType
     
    134137typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJBy)(ExecState*, EncodedJSValue, EncodedJSValue, ByValInfo*);
    135138typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJ)(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue);
    136 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJRp)(ExecState*, EncodedJSValue, EncodedJSValue, ResultProfile*);
     139typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArp)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*);
     140typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArpJaic)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*, JITAddIC*);
     141typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJaic)(ExecState*, EncodedJSValue, EncodedJSValue, JITAddIC*);
    137142typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssZ)(ExecState*, JSString*, int32_t);
    138143typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReo)(ExecState*, JSString*, RegExpObject*);
     
    409414
    410415EncodedJSValue JIT_OPERATION operationValueAdd(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
    411 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
     416EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
     417EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL;
     418EncodedJSValue JIT_OPERATION operationValueAddProfiledNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL;
     419EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
     420EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL;
    412421EncodedJSValue JIT_OPERATION operationValueMul(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
    413 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
     422EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
    414423EncodedJSValue JIT_OPERATION operationValueSub(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL;
    415 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;
     424EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL;
    416425
    417426void JIT_OPERATION operationProcessTypeProfilerLog(ExecState*) WTF_INTERNAL;
  • trunk/Source/JavaScriptCore/jit/JITSubGenerator.cpp

    r200606 r203537  
    2626#include "config.h"
    2727#include "JITSubGenerator.h"
     28
     29#include "ArithProfile.h"
    2830
    2931#if ENABLE(JIT)
     
    8486
    8587    jit.subDouble(m_rightFPR, m_leftFPR);
    86     if (m_resultProfile)
    87         m_resultProfile->emitSetDouble(jit);
     88    if (m_arithProfile)
     89        m_arithProfile->emitSetDouble(jit);
    8890
    8991    jit.boxDouble(m_leftFPR, m_result);
  • trunk/Source/JavaScriptCore/jit/JITSubGenerator.h

    r200606 r203537  
    3939        JSValueRegs result, JSValueRegs left, JSValueRegs right,
    4040        FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR,
    41         ResultProfile* resultProfile = nullptr)
     41        ArithProfile* arithProfile = nullptr)
    4242        : m_leftOperand(leftOperand)
    4343        , m_rightOperand(rightOperand)
     
    4949        , m_scratchGPR(scratchGPR)
    5050        , m_scratchFPR(scratchFPR)
    51         , m_resultProfile(resultProfile)
     51        , m_arithProfile(arithProfile)
    5252    { }
    5353
     
    6868    GPRReg m_scratchGPR;
    6969    FPRReg m_scratchFPR;
    70     ResultProfile* m_resultProfile;
     70    ArithProfile* m_arithProfile;
    7171    bool m_didEmitFastPath { false };
    7272
  • trunk/Source/JavaScriptCore/jit/Repatch.cpp

    r203272 r203537  
    7373}
    7474
    75 static void repatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction)
     75void ftlThunkAwareRepatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction)
    7676{
    7777#if ENABLE(FTL_JIT)
     
    168168                bool generatedCodeInline = InlineAccess::generateArrayLength(*codeBlock->vm(), stubInfo, jsCast<JSArray*>(baseValue));
    169169                if (generatedCodeInline) {
    170                     repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
     170                    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
    171171                    stubInfo.initArrayLength();
    172172                    return RetryCacheLater;
     
    221221                LOG_IC((ICEvent::GetByIdSelfPatch, structure->classInfo(), propertyName));
    222222                structure->startWatchingPropertyForReplacements(vm, slot.cachedOffset());
    223                 repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
     223                ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
    224224                stubInfo.initGetByIdSelf(codeBlock, structure, slot.cachedOffset());
    225225                return RetryCacheLater;
     
    315315   
    316316    if (tryCacheGetByID(exec, baseValue, propertyName, slot, stubInfo, kind) == GiveUpOnCache)
    317         repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind));
     317        ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind));
    318318}
    319319
     
    373373                if (generatedCodeInline) {
    374374                    LOG_IC((ICEvent::PutByIdSelfPatch, structure->classInfo(), ident));
    375                     repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind));
     375                    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind));
    376376                    stubInfo.initPutByIdReplace(codeBlock, structure, slot.cachedOffset());
    377377                    return RetryCacheLater;
     
    469469   
    470470    if (tryCachePutByID(exec, baseValue, structure, propertyName, slot, stubInfo, putKind) == GiveUpOnCache)
    471         repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind));
     471        ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind));
    472472}
    473473
     
    530530    SuperSamplerScope superSamplerScope(false);
    531531    if (tryRepatchIn(exec, base, ident, wasFound, slot, stubInfo) == GiveUpOnCache)
    532         repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn);
     532        ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn);
    533533}
    534534
     
    902902void resetGetByID(CodeBlock* codeBlock, StructureStubInfo& stubInfo, GetByIDKind kind)
    903903{
    904     repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
     904    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));
    905905    InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation());
    906906}
     
    921921    }
    922922
    923     repatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction);
     923    ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction);
    924924    InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation());
    925925}
  • trunk/Source/JavaScriptCore/jit/Repatch.h

    r199170 r203537  
    5555void resetPutByID(CodeBlock*, StructureStubInfo&);
    5656void resetIn(CodeBlock*, StructureStubInfo&);
     57void ftlThunkAwareRepatchCall(CodeBlock*, CodeLocationCall, FunctionPtr newCalleeFunction);
    5758
    5859} // namespace JSC
  • trunk/Source/JavaScriptCore/llint/LLIntData.cpp

    r203006 r203537  
    2727#include "LLIntData.h"
    2828
     29#include "ArithProfile.h"
    2930#include "BytecodeConventions.h"
    3031#include "CodeBlock.h"
     
    224225
    225226    ASSERT(StringImpl::s_hashFlag8BitBuffer == 8);
     227
     228    {
     229        uint32_t bits = 0x120000;
     230        UNUSED_PARAM(bits);
     231        ArithProfile arithProfile;
     232        arithProfile.lhsSawInt32();
     233        arithProfile.rhsSawInt32();
     234        ASSERT(arithProfile.bits() == bits);
     235        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32());
     236        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32());
     237    }
     238    {
     239        uint32_t bits = 0x220000;
     240        UNUSED_PARAM(bits);
     241        ArithProfile arithProfile;
     242        arithProfile.lhsSawNumber();
     243        arithProfile.rhsSawInt32();
     244        ASSERT(arithProfile.bits() == bits);
     245        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber());
     246        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32());
     247    }
     248    {
     249        uint32_t bits = 0x240000;
     250        UNUSED_PARAM(bits);
     251        ArithProfile arithProfile;
     252        arithProfile.lhsSawNumber();
     253        arithProfile.rhsSawNumber();
     254        ASSERT(arithProfile.bits() == bits);
     255        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber());
     256        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber());
     257    }
     258    {
     259        uint32_t bits = 0x140000;
     260        UNUSED_PARAM(bits);
     261        ArithProfile arithProfile;
     262        arithProfile.lhsSawInt32();
     263        arithProfile.rhsSawNumber();
     264        ASSERT(arithProfile.bits() == bits);
     265        ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32());
     266        ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber());
     267    }
    226268}
    227269#if COMPILER(CLANG)
  • trunk/Source/JavaScriptCore/llint/LowLevelInterpreter.asm

    r203499 r203537  
    252252# ShadowChicken data
    253253const ShadowChickenTailMarker = 0x7a11
     254
     255# ArithProfile data
     256const ArithProfileIntInt = 0x120000
     257const ArithProfileNumberInt = 0x220000
     258const ArithProfileNumberNumber = 0x240000
     259const ArithProfileIntNumber = 0x140000
    254260
    255261# Some register conventions.
     
    287293    end
    288294   
     295    macro storeisToInstruction(value, offset)
     296        storei value, offset * 8[PB, PC, 8]
     297    end
     298
    289299    macro storepToInstruction(value, offset)
    290300        storep value, offset * 8[PB, PC, 8]
     
    299309    macro loadpFromInstruction(offset, dest)
    300310        loadp offset * 4[PC], dest
     311    end
     312
     313    macro storeisToInstruction(value, offset)
     314        storei value, offset * 4[PC]
    301315    end
    302316end
  • trunk/Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm

    r203361 r203537  
    10151015    bineq t2, Int32Tag, .op1NotInt
    10161016    bineq t3, Int32Tag, .op2NotInt
     1017    loadisFromInstruction(4, t5)
     1018    ori ArithProfileIntInt, t5
     1019    storeisToInstruction(t5, 4)
    10171020    loadi 4[PC], t2
    10181021    integerOperationAndStore(t3, t1, t0, .slow, t2)
     
    10241027    bib t3, LowestTag, .op1NotIntOp2Double
    10251028    bineq t3, Int32Tag, .slow
     1029    loadisFromInstruction(4, t5)
     1030    ori ArithProfileNumberInt, t5
     1031    storeisToInstruction(t5, 4)
    10261032    ci2d t1, ft1
    10271033    jmp .op1NotIntReady
    10281034.op1NotIntOp2Double:
    10291035    fii2d t1, t3, ft1
     1036    loadisFromInstruction(4, t5)
     1037    ori ArithProfileNumberNumber, t5
     1038    storeisToInstruction(t5, 4)
    10301039.op1NotIntReady:
    10311040    loadi 4[PC], t1
     
    10391048    loadi 4[PC], t2
    10401049    bia t3, LowestTag, .slow
     1050    loadisFromInstruction(4, t5)
     1051    ori ArithProfileIntNumber, t5
     1052    storeisToInstruction(t5, 4)
    10411053    ci2d t0, ft0
    10421054    fii2d t1, t3, ft1
  • trunk/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm

    r203361 r203537  
    894894    loadisFromInstruction(1, t2)
    895895    integerOperationAndStore(t1, t0, .slow, t2)
     896    loadisFromInstruction(4, t1)
     897    ori ArithProfileIntInt, t1
     898    storeisToInstruction(t1, 4)
    896899    dispatch(5)
    897900
     
    903906    addq tagTypeNumber, t1
    904907    fq2d t1, ft1
     908    loadisFromInstruction(4, t2)
     909    ori ArithProfileNumberNumber, t2
     910    storeisToInstruction(t2, 4)
    905911    jmp .op1NotIntReady
    906912.op1NotIntOp2Int:
     913    loadisFromInstruction(4, t2)
     914    ori ArithProfileNumberInt, t2
     915    storeisToInstruction(t2, 4)
    907916    ci2d t1, ft1
    908917.op1NotIntReady:
     
    920929    loadisFromInstruction(1, t2)
    921930    btqz t1, tagTypeNumber, .slow
     931    loadisFromInstruction(4, t3)
     932    ori ArithProfileIntNumber, t3
     933    storeisToInstruction(t3, 4)
    922934    ci2d t0, ft0
    923935    addq tagTypeNumber, t1
  • trunk/Source/JavaScriptCore/parser/ResultType.h

    r192836 r203537  
    3333        friend struct OperandTypes;
    3434
    35         typedef char Type;
     35        typedef uint8_t Type;
    3636        static const Type TypeInt32 = 1;
    37        
    38         static const Type TypeMaybeNumber = 0x04;
    39         static const Type TypeMaybeString = 0x08;
    40         static const Type TypeMaybeNull   = 0x10;
    41         static const Type TypeMaybeBool   = 0x20;
    42         static const Type TypeMaybeOther  = 0x40;
     37        static const Type TypeMaybeNumber = 0x02;
     38        static const Type TypeMaybeString = 0x04;
     39        static const Type TypeMaybeNull   = 0x08;
     40        static const Type TypeMaybeBool   = 0x10;
     41        static const Type TypeMaybeOther  = 0x20;
    4342
    4443        static const Type TypeBits = TypeMaybeNumber | TypeMaybeString | TypeMaybeNull | TypeMaybeBool | TypeMaybeOther;
    4544
     45    public:
     46        static const int numBitsNeeded = 6;
     47        static_assert((TypeBits & ((1 << numBitsNeeded) - 1)) == TypeBits, "This is necessary for correctness.");
     48
    4649        explicit ResultType(Type type)
    47             : m_type(type)
     50            : m_bits(type)
    4851        {
    4952        }
    5053
    51     public:
    5254        bool isInt32() const
    5355        {
    54             return m_type & TypeInt32;
     56            return m_bits & TypeInt32;
    5557        }
    5658
    5759        bool definitelyIsNumber() const
    5860        {
    59             return (m_type & TypeBits) == TypeMaybeNumber;
     61            return (m_bits & TypeBits) == TypeMaybeNumber;
    6062        }
    6163       
    6264        bool definitelyIsString() const
    6365        {
    64             return (m_type & TypeBits) == TypeMaybeString;
     66            return (m_bits & TypeBits) == TypeMaybeString;
    6567        }
    6668
    6769        bool definitelyIsBoolean() const
    6870        {
    69             return (m_type & TypeBits) == TypeMaybeBool;
     71            return (m_bits & TypeBits) == TypeMaybeBool;
    7072        }
    7173
    7274        bool mightBeNumber() const
    7375        {
    74             return m_type & TypeMaybeNumber;
     76            return m_bits & TypeMaybeNumber;
    7577        }
    7678
     
    142144        }
    143145
     146        Type bits() const { return m_bits; }
     147
    144148    private:
    145         Type m_type;
     149        Type m_bits;
    146150    };
    147151   
     
    153157            // the entire struct is initialized.
    154158            m_u.i = 0;
    155             m_u.rds.first = first.m_type;
    156             m_u.rds.second = second.m_type;
     159            m_u.rds.first = first.m_bits;
     160            m_u.rds.second = second.m_bits;
    157161        }
    158162       
  • trunk/Source/JavaScriptCore/runtime/CommonSlowPaths.cpp

    r203499 r203537  
    2727#include "CommonSlowPaths.h"
    2828
     29#include "ArithProfile.h"
    2930#include "ArrayConstructor.h"
    3031#include "BuiltinNames.h"
     
    360361
    361362#if ENABLE(DFG_JIT)
    362 static void updateResultProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right)
     363static void updateArithProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right)
    363364{
    364365    CodeBlock* codeBlock = exec->codeBlock();
    365     unsigned bytecodeOffset = codeBlock->bytecodeOffset(pc);
    366     ResultProfile* profile = codeBlock->ensureResultProfile(bytecodeOffset);
     366    ArithProfile& profile = codeBlock->arithProfileForPC(pc);
    367367
    368368    if (result.isNumber()) {
    369369        if (!result.isInt32()) {
    370370            if (left.isInt32() && right.isInt32())
    371                 profile->setObservedInt32Overflow();
     371                profile.setObservedInt32Overflow();
    372372
    373373            double doubleVal = result.asNumber();
    374374            if (!doubleVal && std::signbit(doubleVal))
    375                 profile->setObservedNegZeroDouble();
     375                profile.setObservedNegZeroDouble();
    376376            else {
    377                 profile->setObservedNonNegZeroDouble();
     377                profile.setObservedNonNegZeroDouble();
    378378
    379379                // The Int52 overflow check here intentionally omits 1ll << 51 as a valid negative Int52 value.
     
    383383                int64_t int64Val = static_cast<int64_t>(std::abs(doubleVal));
    384384                if (int64Val >= int52OverflowPoint)
    385                     profile->setObservedInt52Overflow();
     385                    profile.setObservedInt52Overflow();
    386386            }
    387387        }
    388388    } else
    389         profile->setObservedNonNumber();
     389        profile.setObservedNonNumber();
    390390}
    391391#else
    392 static void updateResultProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { }
     392static void updateArithProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { }
    393393#endif
    394394
     
    407407    JSValue v2 = OP_C(3).jsValue();
    408408    JSValue result;
     409
     410    ArithProfile& arithProfile = exec->codeBlock()->arithProfileForPC(pc);
     411    arithProfile.observeLHSAndRHS(v1, v2);
    409412
    410413    if (v1.isString() && !v2.isObject())
     
    416419
    417420    RETURN_WITH_PROFILING(result, {
    418         updateResultProfileForBinaryArithOp(exec, pc, result, v1, v2);
     421        updateArithProfileForBinaryArithOp(exec, pc, result, v1, v2);
    419422    });
    420423}
     
    433436    JSValue result = jsNumber(a * b);
    434437    RETURN_WITH_PROFILING(result, {
    435         updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
     438        updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
    436439    });
    437440}
     
    446449    JSValue result = jsNumber(a - b);
    447450    RETURN_WITH_PROFILING(result, {
    448         updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
     451        updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
    449452    });
    450453}
     
    459462    JSValue result = jsNumber(a / b);
    460463    RETURN_WITH_PROFILING(result, {
    461         updateResultProfileForBinaryArithOp(exec, pc, result, left, right);
     464        updateArithProfileForBinaryArithOp(exec, pc, result, left, right);
    462465    });
    463466}
  • trunk/Source/WebCore/ChangeLog

    r203535 r203537  
     12016-07-21  Saam Barati  <sbarati@apple.com>
     2
     3        op_add/ValueAdd should be an IC in all JIT tiers
     4        https://bugs.webkit.org/show_bug.cgi?id=159649
     5
     6        Reviewed by Benjamin Poulain.
     7
     8        * ForwardingHeaders/jit/JITMathICForwards.h: Added.
     9
    1102016-07-21  Chris Dumez  <cdumez@apple.com>
    211
Note: See TracChangeset for help on using the changeset viewer.