Changeset 98912 in webkit


Ignore:
Timestamp:
Oct 31, 2011 4:50:57 PM (13 years ago)
Author:
fpizlo@apple.com
Message:

DFG OSR exits should add to value profiles
​https://bugs.webkit.org/show_bug.cgi?id=71202

Reviewed by Oliver Hunt.

Value profiles now have an extra special slot not used by the old JIT's
profiling, which is reserved for OSR exits.

The DFG's OSR exit code now knows which register, node index, and value
profiling site was responsible for the (possibly flawed) information that
led to the OSR failure. This is somewhat opportunistic and imperfect;
if there's a lot of control flow between the value profiling site and the
OSR failure point, then this mechanism simply gives up. It also gives up
if the OSR failure is caused by either known deficiencies in the DFG
(like that we always assume that the index in a strict charCodeAt access
is within bounds) or where the OSR failure would be catalogues and
profiled through other means (like slow case counters).

This patch also adds the notion of a JSValueRegs, which is either a
single register in JSVALUE64 or a pair in JSVALUE32_64. We should
probably move the 32_64 DFG towards using this, since it often makes it
easier to share code between 64 and 32_64.

Also fixed a number of pathologies that this uncovered. op_method_check
didn't have a value profiling site on the slow path. GetById should not
always force OSR exit if it never executed in the old JIT; we may be
able to infer its type if it's a array or string length get. Finally,
these changes benefit from a slight tweak to optimization delay
heuristics (profile fullness is now 0.35 instead of 0.25).

3.8% speed-up on Kraken, mostly due to ~35% on both stanford-crypto-aes
and imaging-darkroom.

  • bytecode/ValueProfile.cpp:

(JSC::ValueProfile::computeStatistics):
(JSC::ValueProfile::computeUpdatedPrediction):

  • bytecode/ValueProfile.h:

(JSC::ValueProfile::ValueProfile):
(JSC::ValueProfile::specFailBucket):
(JSC::ValueProfile::numberOfSamples):
(JSC::ValueProfile::isLive):
(JSC::ValueProfile::numberOfInt32s):
(JSC::ValueProfile::numberOfDoubles):
(JSC::ValueProfile::numberOfCells):
(JSC::ValueProfile::numberOfObjects):
(JSC::ValueProfile::numberOfFinalObjects):
(JSC::ValueProfile::numberOfStrings):
(JSC::ValueProfile::numberOfArrays):
(JSC::ValueProfile::numberOfBooleans):
(JSC::ValueProfile::dump):

  • dfg/DFGAbstractState.cpp:

(JSC::DFG::AbstractState::execute):

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::getPredictionWithoutOSRExit):
(JSC::DFG::ByteCodeParser::getPrediction):
(JSC::DFG::ByteCodeParser::parseBlock):

  • dfg/DFGGPRInfo.h:

(JSC::DFG::JSValueRegs::JSValueRegs):
(JSC::DFG::JSValueRegs::operator!):
(JSC::DFG::JSValueRegs::gpr):
(JSC::DFG::JSValueSource::JSValueSource):
(JSC::DFG::JSValueSource::unboxedCell):
(JSC::DFG::JSValueSource::operator!):
(JSC::DFG::JSValueSource::isAddress):
(JSC::DFG::JSValueSource::offset):
(JSC::DFG::JSValueSource::base):
(JSC::DFG::JSValueSource::gpr):
(JSC::DFG::JSValueSource::asAddress):
(JSC::DFG::JSValueSource::notAddress):
(JSC::DFG::JSValueRegs::tagGPR):
(JSC::DFG::JSValueRegs::payloadGPR):
(JSC::DFG::JSValueSource::tagGPR):
(JSC::DFG::JSValueSource::payloadGPR):
(JSC::DFG::JSValueSource::hasKnownTag):
(JSC::DFG::JSValueSource::tag):

  • dfg/DFGGenerationInfo.h:

(JSC::DFG::GenerationInfo::jsValueRegs):

  • dfg/DFGGraph.h:

(JSC::DFG::Graph::valueProfileFor):

  • dfg/DFGJITCodeGenerator.h:

(JSC::JSValueOperand::jsValueRegs):

  • dfg/DFGJITCompiler.cpp:

(JSC::DFG::JITCompiler::exitSpeculativeWithOSR):

  • dfg/DFGJITCompiler.h:

(JSC::DFG::JITCompiler::valueProfileFor):

  • dfg/DFGJITCompiler32_64.cpp:

(JSC::DFG::JITCompiler::exitSpeculativeWithOSR):

  • dfg/DFGPropagator.cpp:

(JSC::DFG::Propagator::propagateNodePredictions):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::OSRExit::OSRExit):
(JSC::DFG::SpeculativeJIT::compilePeepHoleObjectEquality):
(JSC::DFG::SpeculativeJIT::checkArgumentTypes):
(JSC::DFG::SpeculativeJIT::compileGetCharCodeAt):
(JSC::DFG::SpeculativeJIT::compileGetByValOnString):
(JSC::DFG::SpeculativeJIT::compilePutByValForByteArray):
(JSC::DFG::SpeculativeJIT::compileGetByValOnByteArray):

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::speculationCheck):
(JSC::DFG::SpeculativeJIT::terminateSpeculativeExecution):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::fillSpeculateIntInternal):
(JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
(JSC::DFG::SpeculativeJIT::fillSpeculateCell):
(JSC::DFG::SpeculativeJIT::fillSpeculateBoolean):
(JSC::DFG::SpeculativeJIT::compileObjectEquality):
(JSC::DFG::SpeculativeJIT::compileObjectOrOtherLogicalNot):
(JSC::DFG::SpeculativeJIT::compileLogicalNot):
(JSC::DFG::SpeculativeJIT::emitObjectOrOtherBranch):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::fillSpeculateIntInternal):
(JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
(JSC::DFG::SpeculativeJIT::fillSpeculateCell):
(JSC::DFG::SpeculativeJIT::fillSpeculateBoolean):
(JSC::DFG::SpeculativeJIT::compileObjectEquality):
(JSC::DFG::SpeculativeJIT::compileObjectOrOtherLogicalNot):
(JSC::DFG::SpeculativeJIT::compileLogicalNot):
(JSC::DFG::SpeculativeJIT::emitObjectOrOtherBranch):
(JSC::DFG::SpeculativeJIT::emitBranch):
(JSC::DFG::SpeculativeJIT::compile):

  • jit/JITPropertyAccess.cpp:

(JSC::JIT::emitSlow_op_method_check):

  • jit/JITPropertyAccess32_64.cpp:

(JSC::JIT::emitSlow_op_method_check):

  • runtime/Heuristics.cpp:

(JSC::Heuristics::initializeHeuristics):

  • runtime/JSValue.h:
Location:
trunk/Source/JavaScriptCore
Files:
21 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/ChangeLog

    r98909 r98912  
     12011-10-31  Filip Pizlo  <fpizlo@apple.com>
     2
     3        DFG OSR exits should add to value profiles
     4        https://bugs.webkit.org/show_bug.cgi?id=71202
     5
     6        Reviewed by Oliver Hunt.
     7       
     8        Value profiles now have an extra special slot not used by the old JIT's
     9        profiling, which is reserved for OSR exits.
     10       
     11        The DFG's OSR exit code now knows which register, node index, and value
     12        profiling site was responsible for the (possibly flawed) information that
     13        led to the OSR failure. This is somewhat opportunistic and imperfect;
     14        if there's a lot of control flow between the value profiling site and the
     15        OSR failure point, then this mechanism simply gives up. It also gives up
     16        if the OSR failure is caused by either known deficiencies in the DFG
     17        (like that we always assume that the index in a strict charCodeAt access
     18        is within bounds) or where the OSR failure would be catalogues and
     19        profiled through other means (like slow case counters).
     20       
     21        This patch also adds the notion of a JSValueRegs, which is either a
     22        single register in JSVALUE64 or a pair in JSVALUE32_64. We should
     23        probably move the 32_64 DFG towards using this, since it often makes it
     24        easier to share code between 64 and 32_64.
     25       
     26        Also fixed a number of pathologies that this uncovered. op_method_check
     27        didn't have a value profiling site on the slow path. GetById should not
     28        always force OSR exit if it never executed in the old JIT; we may be
     29        able to infer its type if it's a array or string length get. Finally,
     30        these changes benefit from a slight tweak to optimization delay
     31        heuristics (profile fullness is now 0.35 instead of 0.25).
     32       
     33        3.8% speed-up on Kraken, mostly due to ~35% on both stanford-crypto-aes
     34        and imaging-darkroom.
     35
     36        * bytecode/ValueProfile.cpp:
     37        (JSC::ValueProfile::computeStatistics):
     38        (JSC::ValueProfile::computeUpdatedPrediction):
     39        * bytecode/ValueProfile.h:
     40        (JSC::ValueProfile::ValueProfile):
     41        (JSC::ValueProfile::specFailBucket):
     42        (JSC::ValueProfile::numberOfSamples):
     43        (JSC::ValueProfile::isLive):
     44        (JSC::ValueProfile::numberOfInt32s):
     45        (JSC::ValueProfile::numberOfDoubles):
     46        (JSC::ValueProfile::numberOfCells):
     47        (JSC::ValueProfile::numberOfObjects):
     48        (JSC::ValueProfile::numberOfFinalObjects):
     49        (JSC::ValueProfile::numberOfStrings):
     50        (JSC::ValueProfile::numberOfArrays):
     51        (JSC::ValueProfile::numberOfBooleans):
     52        (JSC::ValueProfile::dump):
     53        * dfg/DFGAbstractState.cpp:
     54        (JSC::DFG::AbstractState::execute):
     55        * dfg/DFGByteCodeParser.cpp:
     56        (JSC::DFG::ByteCodeParser::getPredictionWithoutOSRExit):
     57        (JSC::DFG::ByteCodeParser::getPrediction):
     58        (JSC::DFG::ByteCodeParser::parseBlock):
     59        * dfg/DFGGPRInfo.h:
     60        (JSC::DFG::JSValueRegs::JSValueRegs):
     61        (JSC::DFG::JSValueRegs::operator!):
     62        (JSC::DFG::JSValueRegs::gpr):
     63        (JSC::DFG::JSValueSource::JSValueSource):
     64        (JSC::DFG::JSValueSource::unboxedCell):
     65        (JSC::DFG::JSValueSource::operator!):
     66        (JSC::DFG::JSValueSource::isAddress):
     67        (JSC::DFG::JSValueSource::offset):
     68        (JSC::DFG::JSValueSource::base):
     69        (JSC::DFG::JSValueSource::gpr):
     70        (JSC::DFG::JSValueSource::asAddress):
     71        (JSC::DFG::JSValueSource::notAddress):
     72        (JSC::DFG::JSValueRegs::tagGPR):
     73        (JSC::DFG::JSValueRegs::payloadGPR):
     74        (JSC::DFG::JSValueSource::tagGPR):
     75        (JSC::DFG::JSValueSource::payloadGPR):
     76        (JSC::DFG::JSValueSource::hasKnownTag):
     77        (JSC::DFG::JSValueSource::tag):
     78        * dfg/DFGGenerationInfo.h:
     79        (JSC::DFG::GenerationInfo::jsValueRegs):
     80        * dfg/DFGGraph.h:
     81        (JSC::DFG::Graph::valueProfileFor):
     82        * dfg/DFGJITCodeGenerator.h:
     83        (JSC::JSValueOperand::jsValueRegs):
     84        * dfg/DFGJITCompiler.cpp:
     85        (JSC::DFG::JITCompiler::exitSpeculativeWithOSR):
     86        * dfg/DFGJITCompiler.h:
     87        (JSC::DFG::JITCompiler::valueProfileFor):
     88        * dfg/DFGJITCompiler32_64.cpp:
     89        (JSC::DFG::JITCompiler::exitSpeculativeWithOSR):
     90        * dfg/DFGPropagator.cpp:
     91        (JSC::DFG::Propagator::propagateNodePredictions):
     92        * dfg/DFGSpeculativeJIT.cpp:
     93        (JSC::DFG::OSRExit::OSRExit):
     94        (JSC::DFG::SpeculativeJIT::compilePeepHoleObjectEquality):
     95        (JSC::DFG::SpeculativeJIT::checkArgumentTypes):
     96        (JSC::DFG::SpeculativeJIT::compileGetCharCodeAt):
     97        (JSC::DFG::SpeculativeJIT::compileGetByValOnString):
     98        (JSC::DFG::SpeculativeJIT::compilePutByValForByteArray):
     99        (JSC::DFG::SpeculativeJIT::compileGetByValOnByteArray):
     100        * dfg/DFGSpeculativeJIT.h:
     101        (JSC::DFG::SpeculativeJIT::speculationCheck):
     102        (JSC::DFG::SpeculativeJIT::terminateSpeculativeExecution):
     103        * dfg/DFGSpeculativeJIT32_64.cpp:
     104        (JSC::DFG::SpeculativeJIT::fillSpeculateIntInternal):
     105        (JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
     106        (JSC::DFG::SpeculativeJIT::fillSpeculateCell):
     107        (JSC::DFG::SpeculativeJIT::fillSpeculateBoolean):
     108        (JSC::DFG::SpeculativeJIT::compileObjectEquality):
     109        (JSC::DFG::SpeculativeJIT::compileObjectOrOtherLogicalNot):
     110        (JSC::DFG::SpeculativeJIT::compileLogicalNot):
     111        (JSC::DFG::SpeculativeJIT::emitObjectOrOtherBranch):
     112        (JSC::DFG::SpeculativeJIT::compile):
     113        * dfg/DFGSpeculativeJIT64.cpp:
     114        (JSC::DFG::SpeculativeJIT::fillSpeculateIntInternal):
     115        (JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
     116        (JSC::DFG::SpeculativeJIT::fillSpeculateCell):
     117        (JSC::DFG::SpeculativeJIT::fillSpeculateBoolean):
     118        (JSC::DFG::SpeculativeJIT::compileObjectEquality):
     119        (JSC::DFG::SpeculativeJIT::compileObjectOrOtherLogicalNot):
     120        (JSC::DFG::SpeculativeJIT::compileLogicalNot):
     121        (JSC::DFG::SpeculativeJIT::emitObjectOrOtherBranch):
     122        (JSC::DFG::SpeculativeJIT::emitBranch):
     123        (JSC::DFG::SpeculativeJIT::compile):
     124        * jit/JITPropertyAccess.cpp:
     125        (JSC::JIT::emitSlow_op_method_check):
     126        * jit/JITPropertyAccess32_64.cpp:
     127        (JSC::JIT::emitSlow_op_method_check):
     128        * runtime/Heuristics.cpp:
     129        (JSC::Heuristics::initializeHeuristics):
     130        * runtime/JSValue.h:
     131
    11322011-10-31  Sam Weinig  <sam@webkit.org>
    2133

  • trunk/Source/JavaScriptCore/bytecode/ValueProfile.cpp

    r97294 r98912  
    6060void ValueProfile::computeStatistics(Statistics& statistics) const
    6161{
    62     for (unsigned i = 0; i < numberOfBuckets; ++i) {
     62    for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    6363        JSValue value = JSValue::decode(m_buckets[i]);
    6464        if (!value)
    … …  
    8080PredictedType ValueProfile::computeUpdatedPrediction()
    8181{
    82     for (unsigned i = 0; i < numberOfBuckets; ++i) {
     82    for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    8383        JSValue value = JSValue::decode(m_buckets[i]);
    8484        if (!value)

  • trunk/Source/JavaScriptCore/bytecode/ValueProfile.h

    r97294 r98912  
    4141    static const unsigned logNumberOfBuckets = 3; // 8 buckets
    4242    static const unsigned numberOfBuckets = 1 << logNumberOfBuckets;
     43    static const unsigned numberOfSpecFailBuckets = 1;
    4344    static const unsigned bucketIndexMask = numberOfBuckets - 1;
    44     static const unsigned certainty = numberOfBuckets * numberOfBuckets;
     45    static const unsigned totalNumberOfBuckets = numberOfBuckets + numberOfSpecFailBuckets;
     46    static const unsigned certainty = totalNumberOfBuckets * totalNumberOfBuckets;
    4547    static const unsigned majority = certainty / 2;
    4648   
    … …  
    5052        , m_numberOfSamplesInPrediction(0)
    5153    {
    52         for (unsigned i = 0; i < numberOfBuckets; ++i)
     54        for (unsigned i = 0; i < totalNumberOfBuckets; ++i)
    5355            m_buckets[i] = JSValue::encode(JSValue());
     56    }
     57   
     58    EncodedJSValue* specFailBucket(unsigned i)
     59    {
     60        ASSERT(numberOfBuckets + i < totalNumberOfBuckets);
     61        return m_buckets + numberOfBuckets + i;
    5462    }
    5563   
    … …  
    6876    {
    6977        unsigned result = 0;
    70         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     78        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    7179            if (!!JSValue::decode(m_buckets[i]))
    7280                result++;
    … …  
    8290    bool isLive() const
    8391    {
    84         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     92        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    8593            if (!!JSValue::decode(m_buckets[i]))
    8694                return true;
    … …  
    99107    {
    100108        unsigned result = 0;
    101         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     109        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    102110            if (JSValue::decode(m_buckets[i]).isInt32())
    103111                result++;
    … …  
    109117    {
    110118        unsigned result = 0;
    111         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     119        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    112120            if (JSValue::decode(m_buckets[i]).isDouble())
    113121                result++;
    … …  
    119127    {
    120128        unsigned result = 0;
    121         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     129        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    122130            if (!!classInfo(i))
    123131                result++;
    … …  
    129137    {
    130138        unsigned result = 0;
    131         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     139        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    132140            const ClassInfo* ci = classInfo(i);
    133141            if (!!ci && ci->isSubClassOf(&JSObject::s_info))
    … …  
    140148    {
    141149        unsigned result = 0;
    142         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     150        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    143151            if (classInfo(i) == &JSFinalObject::s_info)
    144152                result++;
    … …  
    150158    {
    151159        unsigned result = 0;
    152         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     160        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    153161            if (classInfo(i) == &JSString::s_info)
    154162                result++;
    … …  
    160168    {
    161169        unsigned result = 0;
    162         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     170        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    163171            if (classInfo(i) == &JSArray::s_info)
    164172                result++;
    … …  
    170178    {
    171179        unsigned result = 0;
    172         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     180        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    173181            if (JSValue::decode(m_buckets[i]).isBoolean())
    174182                result++;
    … …  
    238246                predictionToString(m_prediction), m_numberOfSamplesInPrediction);
    239247        bool first = true;
    240         for (unsigned i = 0; i < numberOfBuckets; ++i) {
     248        for (unsigned i = 0; i < totalNumberOfBuckets; ++i) {
    241249            JSValue value = JSValue::decode(m_buckets[i]);
    242250            if (!!value) {
    … …  
    285293    unsigned m_numberOfSamplesInPrediction;
    286294   
    287     EncodedJSValue m_buckets[numberOfBuckets];
     295    EncodedJSValue m_buckets[totalNumberOfBuckets];
    288296};
    289297

  • trunk/Source/JavaScriptCore/dfg/DFGAbstractState.cpp

    r98214 r98912  
    552552    case GetById:
    553553    case GetMethod:
     554        if (!node.prediction()) {
     555            m_isValid = false;
     556            break;
     557        }
    554558        forNode(node.child1()).filter(PredictCell);
    555559        clobberStructures(nodeIndex);

  • trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

    r98831 r98912  
    595595        return call;
    596596    }
    597 
    598     PredictedType getPrediction(NodeIndex nodeIndex, unsigned bytecodeIndex)
     597   
     598    PredictedType getPredictionWithoutOSRExit(NodeIndex nodeIndex, unsigned bytecodeIndex)
    599599    {
    600600        UNUSED_PARAM(nodeIndex);
    … …  
    607607#endif
    608608       
     609        return prediction;
     610    }
     611
     612    PredictedType getPrediction(NodeIndex nodeIndex, unsigned bytecodeIndex)
     613    {
     614        PredictedType prediction = getPredictionWithoutOSRExit(nodeIndex, bytecodeIndex);
     615       
    609616        if (prediction == PredictNone) {
    610617            // We have no information about what values this node generates. Give up
    … …  
    614621       
    615622        return prediction;
     623    }
     624   
     625    PredictedType getPredictionWithoutOSRExit()
     626    {
     627        return getPredictionWithoutOSRExit(m_graph.size(), m_currentIndex);
    616628    }
    617629   
    … …  
    16521664        }
    16531665        case op_get_by_id: {
    1654             PredictedType prediction = getPrediction();
     1666            PredictedType prediction = getPredictionWithoutOSRExit();
    16551667           
    16561668            NodeIndex base = get(currentInstruction[2].u.operand);
    … …  
    17191731            if (offset != notFound) {
    17201732                ASSERT(structureSet.size());
     1733               
     1734                // The implementation of GetByOffset does not know to terminate speculative
     1735                // execution if it doesn't have a prediction, so we do it manually.
     1736                if (prediction == PredictNone)
     1737                    addToGraph(ForceOSRExit);
     1738               
    17211739                addToGraph(CheckStructure, OpInfo(m_graph.addStructureSet(structureSet)), base);
    17221740                set(currentInstruction[1].u.operand, addToGraph(GetByOffset, OpInfo(m_graph.m_storageAccessData.size()), OpInfo(prediction), addToGraph(GetPropertyStorage, base)));

  • trunk/Source/JavaScriptCore/dfg/DFGGPRInfo.h

    r95902 r98912  
    3737#define InvalidGPRReg ((GPRReg)-1)
    3838
     39#if USE(JSVALUE64)
     40class JSValueRegs {
     41public:
     42    JSValueRegs()
     43        : m_gpr(InvalidGPRReg)
     44    {
     45    }
     46   
     47    explicit JSValueRegs(GPRReg gpr)
     48        : m_gpr(gpr)
     49    {
     50    }
     51   
     52    bool operator!() const { return m_gpr == InvalidGPRReg; }
     53   
     54    GPRReg gpr() const { return m_gpr; }
     55   
     56private:
     57    GPRReg m_gpr;
     58};
     59
     60class JSValueSource {
     61public:
     62    JSValueSource()
     63        : m_offset(notAddress())
     64        , m_base(InvalidGPRReg)
     65    {
     66    }
     67   
     68    JSValueSource(JSValueRegs regs)
     69        : m_offset(notAddress())
     70        , m_base(regs.gpr())
     71    {
     72    }
     73   
     74    explicit JSValueSource(GPRReg gpr)
     75        : m_offset(notAddress())
     76        , m_base(gpr)
     77    {
     78    }
     79   
     80    JSValueSource(MacroAssembler::Address address)
     81        : m_offset(address.offset)
     82        , m_base(address.base)
     83    {
     84        ASSERT(m_offset != notAddress());
     85        ASSERT(m_base != InvalidGPRReg);
     86    }
     87   
     88    static JSValueSource unboxedCell(GPRReg payloadGPR)
     89    {
     90        return JSValueSource(payloadGPR);
     91    }
     92   
     93    bool operator!() const { return m_base == InvalidGPRReg; }
     94   
     95    bool isAddress() const { return m_offset != notAddress(); }
     96   
     97    int32_t offset() const
     98    {
     99        ASSERT(isAddress());
     100        return m_offset;
     101    }
     102   
     103    GPRReg base() const
     104    {
     105        ASSERT(isAddress());
     106        return m_base;
     107    }
     108   
     109    GPRReg gpr() const
     110    {
     111        ASSERT(!isAddress());
     112        return m_base;
     113    }
     114   
     115    MacroAssembler::Address asAddress() const { return MacroAssembler::Address(base(), offset()); }
     116   
     117private:
     118    static inline int32_t notAddress() { return 0x80000000; }     
     119         
     120    int32_t m_offset;
     121    GPRReg m_base;
     122};
     123#endif
     124
     125#if USE(JSVALUE32_64)
     126class JSValueRegs {
     127public:
     128    JSValueRegs()
     129        : m_tagGPR(static_cast<int8_t>(InvalidGPRReg))
     130        , m_payloadGPR(static_cast<int8_t>(InvalidGPRReg))
     131    {
     132    }
     133   
     134    JSValueRegs(GPRReg tagGPR, GPRReg payloadGPR)
     135        : m_tagGPR(tagGPR)
     136        , m_payloadGPR(payloadGPR)
     137    {
     138        ASSERT((static_cast<GPRReg>(m_tagGPR) == InvalidGPRReg) == (static_cast<GPRReg>(payloadGPR) == InvalidGPRReg));
     139    }
     140   
     141    bool operator!() const { return static_cast<GPRReg>(m_tagGPR) == InvalidGPRReg; }
     142   
     143    GPRReg tagGPR() const { return static_cast<GPRReg>(m_tagGPR); }
     144    GPRReg payloadGPR() const { return static_cast<GPRReg>(m_payloadGPR); }
     145
     146private:
     147    int8_t m_tagGPR;
     148    int8_t m_payloadGPR;
     149};
     150
     151class JSValueSource {
     152public:
     153    JSValueSource()
     154        : m_offset(notAddress())
     155        , m_baseOrTag(static_cast<int8_t>(InvalidGPRReg))
     156        , m_payload(static_cast<int8_t>(InvalidGPRReg))
     157        , m_tagType(0)
     158    {
     159    }
     160   
     161    JSValueSource(JSValueRegs regs)
     162        : m_offset(notAddress())
     163        , m_baseOrTag(regs.tagGPR())
     164        , m_payload(regs.payloadGPR())
     165        , m_tagType(0)
     166    {
     167    }
     168   
     169    JSValueSource(GPRReg tagGPR, GPRReg payloadGPR)
     170        : m_offset(notAddress())
     171        , m_baseOrTag(static_cast<int8_t>(tagGPR))
     172        , m_payload(static_cast<int8_t>(payloadGPR))
     173        , m_tagType(0)
     174    {
     175    }
     176   
     177    JSValueSource(MacroAssembler::Address address)
     178        : m_offset(address.offset)
     179        , m_baseOrTag(static_cast<int8_t>(address.base))
     180        , m_payload(static_cast<int8_t>(InvalidGPRReg))
     181        , m_tagType(0)
     182    {
     183        ASSERT(m_offset != notAddress());
     184        ASSERT(static_cast<GPRReg>(m_baseOrTag) != InvalidGPRReg);
     185    }
     186   
     187    static JSValueSource unboxedCell(GPRReg payloadGPR)
     188    {
     189        JSValueSource result;
     190        result.m_offset = notAddress();
     191        result.m_baseOrTag = static_cast<int8_t>(InvalidGPRReg);
     192        result.m_payload = static_cast<int8_t>(payloadGPR);
     193        result.m_tagType = static_cast<int8_t>(JSValue::CellTag);
     194        return result;
     195    }
     196   
     197    bool operator!() const { return static_cast<GPRReg>(m_baseOrTag) == InvalidGPRReg && static_cast<GPRReg>(m_payload) == InvalidGPRReg; }
     198   
     199    bool isAddress() const
     200    {
     201        ASSERT(!!*this);
     202        return m_offset != notAddress();
     203    }
     204   
     205    int32_t offset() const
     206    {
     207        ASSERT(isAddress());
     208        return m_offset;
     209    }
     210   
     211    GPRReg base() const
     212    {
     213        ASSERT(isAddress());
     214        return static_cast<GPRReg>(m_baseOrTag);
     215    }
     216   
     217    GPRReg tagGPR() const
     218    {
     219        ASSERT(!isAddress() && m_baseOrTag != InvalidGPRReg);
     220        return static_cast<GPRReg>(m_baseOrTag);
     221    }
     222   
     223    GPRReg payloadGPR() const
     224    {
     225        ASSERT(!isAddress());
     226        return static_cast<GPRReg>(m_payload);
     227    }
     228   
     229    bool hasKnownTag() const
     230    {
     231        ASSERT(!!*this);
     232        ASSERT(!isAddress());
     233        return static_cast<GPRReg>(m_baseOrTag) == InvalidGPRReg;
     234    }
     235   
     236    uint32_t tag() const
     237    {
     238        return static_cast<int32_t>(m_tagType);
     239    }
     240   
     241    MacroAssembler::Address asAddress(unsigned additionalOffset = 0) const { return MacroAssembler::Address(base(), offset() + additionalOffset); }
     242   
     243private:
     244    static inline int32_t notAddress() { return 0x80000000; }     
     245         
     246    int32_t m_offset;
     247    int8_t m_baseOrTag;
     248    int8_t m_payload;
     249    int8_t m_tagType; // Contains the low bits of the tag.
     250};
     251#endif
     252
    39253#if CPU(X86)
    40254

  • trunk/Source/JavaScriptCore/dfg/DFGGenerationInfo.h

    r98291 r98912  
    340340    GPRReg gpr() { ASSERT(m_registerFormat && m_registerFormat != DataFormatDouble); return u.gpr; }
    341341    FPRReg fpr() { ASSERT(m_registerFormat == DataFormatDouble); return u.fpr; }
     342    JSValueRegs jsValueRegs() { ASSERT(m_registerFormat & DataFormatJS); return JSValueRegs(u.gpr); }
    342343#elif USE(JSVALUE32_64)
    343344    GPRReg gpr() { ASSERT(!(m_registerFormat & DataFormatJS) && m_registerFormat != DataFormatDouble); return u.gpr; }
    … …  
    345346    GPRReg payloadGPR() { ASSERT(m_registerFormat & DataFormatJS); return u.v.payloadGPR; }
    346347    FPRReg fpr() { ASSERT(m_registerFormat == DataFormatDouble || m_registerFormat == DataFormatJSDouble); return u.fpr; }
     348    JSValueRegs jsValueRegs() { ASSERT(m_registerFormat & DataFormatJS); return JSValueRegs(u.v.tagGPR, u.v.payloadGPR); }
    347349#endif
    348350

  • trunk/Source/JavaScriptCore/dfg/DFGGraph.h

    r98179 r98912  
    220220        return &m_structureTransitionData.last();
    221221    }
     222   
     223    ValueProfile* valueProfileFor(NodeIndex nodeIndex, CodeBlock* profiledBlock)
     224    {
     225        if (nodeIndex == NoNode)
     226            return 0;
     227       
     228        Node& node = at(nodeIndex);
     229       
     230        switch (node.op) {
     231        case GetLocal: {
     232            if (!operandIsArgument(node.local()))
     233                return 0;
     234            int argument = node.local() + m_arguments.size() + RegisterFile::CallFrameHeaderSize;
     235            if (node.variableAccessData() != at(m_arguments[argument]).variableAccessData())
     236                return 0;
     237            return profiledBlock->valueProfileForArgument(argument);
     238        }
     239       
     240        // Nodes derives from calls need special handling because the value profile is
     241        // associated with the op_call_put_result instruction.
     242        case Call:
     243        case Construct:
     244        case ArrayPop:
     245        case ArrayPush: {
     246            ASSERT(OPCODE_LENGTH(op_call) == OPCODE_LENGTH(op_construct));
     247            return profiledBlock->valueProfileForBytecodeOffset(node.codeOrigin.bytecodeIndex + OPCODE_LENGTH(op_call));
     248        }
     249
     250        default:
     251            if (node.hasHeapPrediction())
     252                return profiledBlock->valueProfileForBytecodeOffset(node.codeOrigin.bytecodeIndex);
     253            return 0;
     254        }
     255    }
    222256
    223257    Vector< OwnPtr<BasicBlock> , 8> m_blocks;

  • trunk/Source/JavaScriptCore/dfg/DFGJITCodeGenerator.h

    r98878 r98912  
    18901890        return m_gprOrInvalid;
    18911891    }
     1892    JSValueRegs jsValueRegs()
     1893    {
     1894        return JSValueRegs(gpr());
     1895    }
    18921896#elif USE(JSVALUE32_64)
    18931897    bool isDouble() { return m_isDouble; }
    … …  
    19111915        ASSERT(!m_isDouble);
    19121916        return m_register.pair.payloadGPR;
     1917    }
     1918
     1919    JSValueRegs jsValueRegs()
     1920    {
     1921        return JSValueRegs(tagGPR(), payloadGPR());
    19131922    }
    19141923

  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp

    r98831 r98912  
    9999    emitCount(counter);
    100100#endif
    101 
     101   
    102102    // 2) Perform speculation recovery. This only comes into play when an operation
    103103    //    starts mutating state before verifying the speculation it has already made.
    … …  
    122122    }
    123123
    124     // 3) Figure out how many scratch slots we'll need. We need one for every GPR/FPR
     124    // 3) Refine some value profile, if appropriate.
     125   
     126    if (!!exit.m_jsValueSource && !!exit.m_valueProfile) {
     127        if (exit.m_jsValueSource.isAddress()) {
     128            // We can't be sure that we have a spare register. So use the tagTypeNumberRegister,
     129            // since we know how to restore it.
     130            loadPtr(Address(exit.m_jsValueSource.asAddress()), GPRInfo::tagTypeNumberRegister);
     131            storePtr(GPRInfo::tagTypeNumberRegister, exit.m_valueProfile->specFailBucket(0));
     132            move(TrustedImmPtr(bitwise_cast<void*>(TagTypeNumber)), GPRInfo::tagTypeNumberRegister);
     133        } else
     134            storePtr(exit.m_jsValueSource.gpr(), exit.m_valueProfile->specFailBucket(0));
     135    }
     136
     137    // 4) Figure out how many scratch slots we'll need. We need one for every GPR/FPR
    125138    //    whose destination is now occupied by a DFG virtual register, and we need
    126139    //    one for every displaced virtual register if there are more than
    … …  
    218231    // between when something is computed and when it is stored.
    219232   
    220     // 4) Perform all reboxing of integers.
     233    // 5) Perform all reboxing of integers.
    221234   
    222235    if (haveUnboxedInt32s) {
    … …  
    239252    }
    240253   
    241     // 5) Dump all non-poisoned GPRs. For poisoned GPRs, save them into the scratch storage.
     254    // 6) Dump all non-poisoned GPRs. For poisoned GPRs, save them into the scratch storage.
    242255    //    Note that GPRs do not have a fast change (like haveFPRs) because we expect that
    243256    //    most OSR failure points will have at least one GPR that needs to be dumped.
    … …  
    263276   
    264277    if (haveFPRs) {
    265         // 6) Box all doubles (relies on there being more GPRs than FPRs)
     278        // 7) Box all doubles (relies on there being more GPRs than FPRs)
    266279       
    267280        for (int index = 0; index < exit.numberOfRecoveries(); ++index) {
    … …  
    274287        }
    275288       
    276         // 7) Dump all doubles into the register file, or to the scratch storage if
     289        // 8) Dump all doubles into the register file, or to the scratch storage if
    277290        //    the destination virtual register is poisoned.
    278291       
    … …  
    291304    ASSERT(scratchIndex == numberOfPoisonedVirtualRegisters);
    292305   
    293     // 8) Reshuffle displaced virtual registers. Optimize for the case that
     306    // 9) Reshuffle displaced virtual registers. Optimize for the case that
    294307    //    the number of displaced virtual registers is not more than the number
    295308    //    of available physical registers.
    … …  
    407420    }
    408421   
    409     // 9) Dump all poisoned virtual registers.
     422    // 10) Dump all poisoned virtual registers.
    410423   
    411424    scratchIndex = 0;
    … …  
    431444    ASSERT(scratchIndex == numberOfPoisonedVirtualRegisters);
    432445   
    433     // 10) Dump all constants. Optimize for Undefined, since that's a constant we see
     446    // 11) Dump all constants. Optimize for Undefined, since that's a constant we see
    434447    //     often.
    435448
    … …  
    449462    }
    450463   
    451     // 11) Adjust the old JIT's execute counter. Since we are exiting OSR, we know
     464    // 12) Adjust the old JIT's execute counter. Since we are exiting OSR, we know
    452465    //     that all new calls into this code will go to the new JIT, so the execute
    453466    //     counter only affects call frames that performed OSR exit and call frames
    … …  
    512525    doneAdjusting.link(this);
    513526   
    514     // 12) Load the result of the last bytecode operation into regT0.
     527    // 13) Load the result of the last bytecode operation into regT0.
    515528   
    516529    if (exit.m_lastSetOperand != std::numeric_limits<int>::max())
    517530        loadPtr(addressFor((VirtualRegister)exit.m_lastSetOperand), GPRInfo::cachedResultRegister);
    518531   
    519     // 13) Fix call frame(s).
     532    // 14) Fix call frame(s).
    520533   
    521534    ASSERT(codeBlock()->alternative()->getJITType() == JITCode::BaselineJIT);
    … …  
    553566        addPtr(Imm32(exit.m_codeOrigin.inlineCallFrame->stackOffset * sizeof(EncodedJSValue)), GPRInfo::callFrameRegister);
    554567   
    555     // 14) Jump into the corresponding baseline JIT code.
     568    // 15) Jump into the corresponding baseline JIT code.
    556569   
    557570    CodeBlock* baselineCodeBlock = baselineCodeBlockFor(exit.m_codeOrigin);

  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h

    r98405 r98912  
    425425    }
    426426
     427    ValueProfile* valueProfileFor(NodeIndex nodeIndex)
     428    {
     429        if (nodeIndex == NoNode)
     430            return 0;
     431       
     432        return m_graph.valueProfileFor(nodeIndex, baselineCodeBlockFor(m_graph[nodeIndex].codeOrigin));
     433    }
     434   
    427435private:
    428436    // Internal implementation to compile.

  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler32_64.cpp

    r98831 r98912  
    9292    }
    9393
    94     // 3) Figure out how many scratch slots we'll need. We need one for every GPR/FPR
     94    // 3) Refine some value profile, if appropriate.
     95   
     96    if (!!exit.m_jsValueSource && !!exit.m_valueProfile) {
     97        if (exit.m_jsValueSource.isAddress()) {
     98            // Save a register so we can use it.
     99            GPRReg scratch = GPRInfo::regT0;
     100            if (scratch == exit.m_jsValueSource.base())
     101                scratch = GPRInfo::regT1;
     102            EncodedJSValue* scratchBuffer = static_cast<EncodedJSValue*>(globalData()->scratchBufferForSize(sizeof(uint32_t)));
     103            store32(scratch, scratchBuffer);
     104            load32(exit.m_jsValueSource.asAddress(OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag)), scratch);
     105            store32(scratch, &bitwise_cast<EncodedValueDescriptor*>(exit.m_valueProfile->specFailBucket(0))->asBits.tag);
     106            load32(exit.m_jsValueSource.asAddress(OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload)), scratch);
     107            store32(scratch, &bitwise_cast<EncodedValueDescriptor*>(exit.m_valueProfile->specFailBucket(0))->asBits.payload);
     108            load32(scratchBuffer, scratch);
     109        } else if (exit.m_jsValueSource.hasKnownTag()) {
     110            store32(Imm32(exit.m_jsValueSource.tag()), &bitwise_cast<EncodedValueDescriptor*>(exit.m_valueProfile->specFailBucket(0))->asBits.tag);
     111            store32(exit.m_jsValueSource.payloadGPR(), &bitwise_cast<EncodedValueDescriptor*>(exit.m_valueProfile->specFailBucket(0))->asBits.payload);
     112        } else {
     113            store32(exit.m_jsValueSource.tagGPR(), &bitwise_cast<EncodedValueDescriptor*>(exit.m_valueProfile->specFailBucket(0))->asBits.tag);
     114            store32(exit.m_jsValueSource.payloadGPR(), &bitwise_cast<EncodedValueDescriptor*>(exit.m_valueProfile->specFailBucket(0))->asBits.payload);
     115        }
     116    }
     117   
     118    // 4) Figure out how many scratch slots we'll need. We need one for every GPR/FPR
    95119    //    whose destination is now occupied by a DFG virtual register, and we need
    96120    //    one for every displaced virtual register if there are more than
    … …  
    180204    // between when something is computed and when it is stored.
    181205   
    182     // 4) Perform all reboxing of integers and cells, except for those in registers.
     206    // 5) Perform all reboxing of integers and cells, except for those in registers.
    183207
    184208    if (haveUnboxedInt32InRegisterFile || haveUnboxedCellInRegisterFile || haveUnboxedBooleanInRegisterFile) {
    … …  
    204228    }
    205229
    206     // 5) Dump all non-poisoned GPRs. For poisoned GPRs, save them into the scratch storage.
     230    // 6) Dump all non-poisoned GPRs. For poisoned GPRs, save them into the scratch storage.
    207231    //    Note that GPRs do not have a fast change (like haveFPRs) because we expect that
    208232    //    most OSR failure points will have at least one GPR that needs to be dumped.
    … …  
    261285   
    262286    if (haveFPRs) {
    263         // 6) Box all doubles (relies on there being more GPRs than FPRs)
     287        // 7) Box all doubles (relies on there being more GPRs than FPRs)
    264288        //    For JSValue32_64, no need to box doubles.
    265289       
    266         // 7) Dump all doubles into the register file, or to the scratch storage if
     290        // 8) Dump all doubles into the register file, or to the scratch storage if
    267291        //    the destination virtual register is poisoned.
    268292       
    … …  
    280304    ASSERT(scratchIndex == numberOfPoisonedVirtualRegisters);
    281305   
    282     // 8) Reshuffle displaced virtual registers. Optimize for the case that
     306    // 9) Reshuffle displaced virtual registers. Optimize for the case that
    283307    //    the number of displaced virtual registers is not more than the number
    284308    //    of available physical registers.
    … …  
    351375    }
    352376   
    353     // 9) Dump all poisoned virtual registers.
     377    // 10) Dump all poisoned virtual registers.
    354378   
    355379    scratchIndex = 0;
    … …  
    380404    ASSERT(scratchIndex == numberOfPoisonedVirtualRegisters);
    381405   
    382     // 10) Dump all constants. Optimize for Undefined, since that's a constant we see
     406    // 11) Dump all constants. Optimize for Undefined, since that's a constant we see
    383407    //     often.
    384408
    … …  
    403427    }
    404428   
    405     // 11) Adjust the old JIT's execute counter. Since we are exiting OSR, we know
     429    // 12) Adjust the old JIT's execute counter. Since we are exiting OSR, we know
    406430    //     that all new calls into this code will go to the new JIT, so the execute
    407431    //     counter only affects call frames that performed OSR exit and call frames
    … …  
    466490    doneAdjusting.link(this);
    467491   
    468     // 12) Load the result of the last bytecode operation into regT0.
     492    // 13) Load the result of the last bytecode operation into regT0.
    469493   
    470494    if (exit.m_lastSetOperand != std::numeric_limits<int>::max()) {
    … …  
    473497    }
    474498   
    475     // 13) Fix call frame (s).
     499    // 14) Fix call frame (s).
    476500   
    477501    ASSERT(codeBlock()->alternative()->getJITType() == JITCode::BaselineJIT);
    … …  
    513537        addPtr(Imm32(exit.m_codeOrigin.inlineCallFrame->stackOffset * sizeof(EncodedJSValue)), GPRInfo::callFrameRegister);
    514538
    515     // 14) Jump into the corresponding baseline JIT code.
     539    // 15) Jump into the corresponding baseline JIT code.
    516540   
    517541    CodeBlock* baselineCodeBlock = baselineCodeBlockFor(exit.m_codeOrigin);

  • trunk/Source/JavaScriptCore/dfg/DFGPropagator.cpp

    r98179 r98912  
    441441        }
    442442           
    443         case GetById:
     443        case GetById: {
     444            if (node.getHeapPrediction())
     445                changed |= mergePrediction(node.getHeapPrediction());
     446            else if (m_codeBlock->identifier(node.identifierNumber()) == m_globalData.propertyNames->length) {
     447                // If there is no prediction from value profiles, check if we might be
     448                // able to infer the type ourselves.
     449                bool isArray = isArrayPrediction(m_graph[node.child1()].prediction());
     450                bool isString = isStringPrediction(m_graph[node.child1()].prediction());
     451                bool isByteArray = m_graph[node.child1()].shouldSpeculateByteArray();
     452                if (isArray || isString || isByteArray)
     453                    changed |= mergePrediction(PredictInt32);
     454            }
     455            break;
     456        }
     457           
    444458        case GetMethod:
    445459        case GetByVal: {

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r98674 r98912  
    116116#endif
    117117
    118 OSRExit::OSRExit(MacroAssembler::Jump check, SpeculativeJIT* jit, unsigned recoveryIndex)
    119     : m_check(check)
     118OSRExit::OSRExit(JSValueSource jsValueSource, ValueProfile* valueProfile, MacroAssembler::Jump check, SpeculativeJIT* jit, unsigned recoveryIndex)
     119    : m_jsValueSource(jsValueSource)
     120    , m_valueProfile(valueProfile)
     121    , m_check(check)
    120122    , m_nodeIndex(jit->m_compileIndex)
    121123    , m_codeOrigin(jit->m_codeOriginForOSR)
    … …  
    180182   
    181183    if (!predictionCheck(m_state.forNode(node.child1()).m_type))
    182         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op1GPR), MacroAssembler::TrustedImmPtr(vptr)));
     184        speculationCheck(JSValueSource::unboxedCell(op1GPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op1GPR), MacroAssembler::TrustedImmPtr(vptr)));
    183185    if (!predictionCheck(m_state.forNode(node.child2()).m_type))
    184         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op2GPR), MacroAssembler::TrustedImmPtr(vptr)));
     186        speculationCheck(JSValueSource::unboxedCell(op2GPR), node.child2(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op2GPR), MacroAssembler::TrustedImmPtr(vptr)));
    185187   
    186188    addBranch(m_jit.branchPtr(condition, op1GPR, op2GPR), taken);
    … …  
    393395#if USE(JSVALUE64)
    394396        if (isInt32Prediction(predictedType))
    395             speculationCheck(m_jit.branchPtr(MacroAssembler::Below, JITCompiler::addressFor(virtualRegister), GPRInfo::tagTypeNumberRegister));
     397            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(MacroAssembler::Below, JITCompiler::addressFor(virtualRegister), GPRInfo::tagTypeNumberRegister));
    396398        else if (isArrayPrediction(predictedType)) {
    397399            GPRTemporary temp(this);
    398400            m_jit.loadPtr(JITCompiler::addressFor(virtualRegister), temp.gpr());
    399             speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, temp.gpr(), GPRInfo::tagMaskRegister));
    400             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     401            speculationCheck(JSValueRegs(), NoNode, m_jit.branchTestPtr(MacroAssembler::NonZero, temp.gpr(), GPRInfo::tagMaskRegister));
     402            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    401403        } else if (isByteArrayPrediction(predictedType)) {
    402404            GPRTemporary temp(this);
    403405            m_jit.loadPtr(JITCompiler::addressFor(virtualRegister), temp.gpr());
    404             speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, temp.gpr(), GPRInfo::tagMaskRegister));
    405             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     406            speculationCheck(JSValueRegs(), NoNode, m_jit.branchTestPtr(MacroAssembler::NonZero, temp.gpr(), GPRInfo::tagMaskRegister));
     407            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    406408        } else if (isBooleanPrediction(predictedType)) {
    407409            GPRTemporary temp(this);
    408410            m_jit.loadPtr(JITCompiler::addressFor(virtualRegister), temp.gpr());
    409411            m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), temp.gpr());
    410             speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, temp.gpr(), TrustedImm32(static_cast<int32_t>(~1))));
     412            speculationCheck(JSValueRegs(), NoNode, m_jit.branchTestPtr(MacroAssembler::NonZero, temp.gpr(), TrustedImm32(static_cast<int32_t>(~1))));
    411413        }
    412414#else
    413415        if (isInt32Prediction(predictedType))
    414             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::Int32Tag)));
     416            speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::Int32Tag)));
    415417        else if (isArrayPrediction(predictedType)) {
    416418            GPRTemporary temp(this);
    417419            m_jit.load32(JITCompiler::tagFor(virtualRegister), temp.gpr());
    418             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, temp.gpr(), TrustedImm32(JSValue::CellTag)));
     420            speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::NotEqual, temp.gpr(), TrustedImm32(JSValue::CellTag)));
    419421            m_jit.load32(JITCompiler::payloadFor(virtualRegister), temp.gpr());
    420             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     422            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    421423        } else if (isByteArrayPrediction(predictedType)) {
    422424            GPRTemporary temp(this);
    423425            m_jit.load32(JITCompiler::tagFor(virtualRegister), temp.gpr());
    424             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, temp.gpr(), TrustedImm32(JSValue::CellTag)));
     426            speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::NotEqual, temp.gpr(), TrustedImm32(JSValue::CellTag)));
    425427            m_jit.load32(JITCompiler::payloadFor(virtualRegister), temp.gpr());
    426             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     428            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(temp.gpr()), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    427429        } else if (isBooleanPrediction(predictedType))
    428             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::BooleanTag)));
     430            speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::BooleanTag)));
    429431#endif
    430432    }
    … …  
    587589
    588590    if (!isStringPrediction(m_state.forNode(node.child1()).m_type))
    589         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(stringReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
     591        speculationCheck(JSValueSource::unboxedCell(stringReg), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(stringReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
    590592
    591593    // unsigned comparison so we can filter out negative indices and indices that are too large
    592     speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, indexReg, MacroAssembler::Address(stringReg, JSString::offsetOfLength())));
     594    speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::AboveOrEqual, indexReg, MacroAssembler::Address(stringReg, JSString::offsetOfLength())));
    593595
    594596    GPRTemporary scratch(this);
    … …  
    598600
    599601    // Speculate that we're not accessing a rope
    600     speculationCheck(m_jit.branchTest32(MacroAssembler::Zero, scratchReg));
     602    speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(MacroAssembler::Zero, scratchReg));
    601603
    602604    // Load the character into scratchReg
    … …  
    627629
    628630    if (!isStringPrediction(m_state.forNode(node.child1()).m_type))
    629         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
     631        speculationCheck(JSValueSource::unboxedCell(baseReg), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
    630632
    631633    // unsigned comparison so we can filter out negative indices and indices that are too large
    632     speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(baseReg, JSString::offsetOfLength())));
     634    speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(baseReg, JSString::offsetOfLength())));
    633635
    634636    GPRTemporary scratch(this);
    … …  
    638640
    639641    // Speculate that we're not accessing a rope
    640     speculationCheck(m_jit.branchTest32(MacroAssembler::Zero, scratchReg));
     642    speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(MacroAssembler::Zero, scratchReg));
    641643
    642644    // Load the character into scratchReg
    … …  
    653655
    654656    // We only support ascii characters
    655     speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, scratchReg, TrustedImm32(0x100)));
     657    speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::AboveOrEqual, scratchReg, TrustedImm32(0x100)));
    656658
    657659    // 8 bit string values don't need the isASCII check.
    … …  
    662664    m_jit.move(MacroAssembler::TrustedImmPtr(m_jit.globalData()->smallStrings.singleCharacterStrings()), smallStringsReg);
    663665    m_jit.loadPtr(MacroAssembler::BaseIndex(smallStringsReg, scratchReg, MacroAssembler::ScalePtr, 0), scratchReg);
    664     speculationCheck(m_jit.branchTest32(MacroAssembler::Zero, scratchReg));
     666    speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(MacroAssembler::Zero, scratchReg));
    665667    cellResult(scratchReg, m_compileIndex);
    666668}
    … …  
    729731   
    730732    if (!isByteArrayPrediction(m_state.forNode(baseIndex).m_type))
    731         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(base), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     733        speculationCheck(JSValueSource::unboxedCell(base), baseIndex, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(base), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    732734    GPRTemporary value;
    733735    GPRReg valueGPR;
    … …  
    736738        JSValue jsValue = valueOfJSConstant(valueIndex);
    737739        if (!jsValue.isNumber()) {
    738             terminateSpeculativeExecution();
     740            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    739741            noResult(m_compileIndex);
    740742            return;
    … …  
    797799   
    798800    if (!isByteArrayPrediction(m_state.forNode(node.child1()).m_type))
    799         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     801        speculationCheck(JSValueSource::unboxedCell(baseReg), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    800802
    801803    // Load the character into scratchReg
    … …  
    805807   
    806808    // unsigned comparison so we can filter out negative indices and indices that are too large
    807     speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(storageReg, ByteArray::offsetOfSize())));
     809    speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(storageReg, ByteArray::offsetOfSize())));
    808810
    809811    m_jit.load8(MacroAssembler::BaseIndex(storageReg, propertyReg, MacroAssembler::TimesOne, ByteArray::offsetOfData()), storageReg);

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

    r98291 r98912  
    339339// going into baseline code.
    340340struct OSRExit {
    341     OSRExit(MacroAssembler::Jump, SpeculativeJIT*, unsigned recoveryIndex = 0);
     341    OSRExit(JSValueSource, ValueProfile*, MacroAssembler::Jump, SpeculativeJIT*, unsigned recoveryIndex = 0);
     342   
     343    JSValueSource m_jsValueSource;
     344    ValueProfile* m_valueProfile;
    342345   
    343346    MacroAssembler::Jump m_check;
    … …  
    506509
    507510    // Add a speculation check without additional recovery.
    508     void speculationCheck(MacroAssembler::Jump jumpToFail)
     511    void speculationCheck(JSValueSource jsValueSource, NodeIndex nodeIndex, MacroAssembler::Jump jumpToFail)
    509512    {
    510513        if (!m_compileOkay)
    511514            return;
    512         m_osrExits.append(OSRExit(jumpToFail, this));
     515        m_osrExits.append(OSRExit(jsValueSource, m_jit.valueProfileFor(nodeIndex), jumpToFail, this));
    513516    }
    514517    // Add a speculation check with additional recovery.
    515     void speculationCheck(MacroAssembler::Jump jumpToFail, const SpeculationRecovery& recovery)
     518    void speculationCheck(JSValueSource jsValueSource, NodeIndex nodeIndex, MacroAssembler::Jump jumpToFail, const SpeculationRecovery& recovery)
    516519    {
    517520        if (!m_compileOkay)
    518521            return;
    519522        m_speculationRecoveryList.append(recovery);
    520         m_osrExits.append(OSRExit(jumpToFail, this, m_speculationRecoveryList.size()));
     523        m_osrExits.append(OSRExit(jsValueSource, m_jit.valueProfileFor(nodeIndex), jumpToFail, this, m_speculationRecoveryList.size()));
    521524    }
    522525
    523526    // Called when we statically determine that a speculation will fail.
    524     void terminateSpeculativeExecution()
     527    void terminateSpeculativeExecution(JSValueRegs jsValueRegs, NodeIndex nodeIndex)
    525528    {
    526529#if DFG_ENABLE(DEBUG_VERBOSE)
    … …  
    529532        if (!m_compileOkay)
    530533            return;
    531         speculationCheck(m_jit.jump());
     534        speculationCheck(jsValueRegs, nodeIndex, m_jit.jump());
    532535        m_compileOkay = false;
    533536    }

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp

    r98291 r98912  
    6969                return gpr;
    7070            }
    71             terminateSpeculativeExecution();
     71            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    7272            returnFormat = DataFormatInteger;
    7373            return allocate();
    … …  
    8181        // If we know this was spilled as an integer we can fill without checking.
    8282        if (spillFormat != DataFormatJSInteger)
    83             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::Int32Tag)));
     83            speculationCheck(JSValueSource(JITCompiler::addressFor(virtualRegister)), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::Int32Tag)));
    8484
    8585        m_jit.load32(JITCompiler::payloadFor(virtualRegister), gpr);
    … …  
    9797        m_gprs.lock(payloadGPR);
    9898        if (info.registerFormat() != DataFormatJSInteger)
    99             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, tagGPR, TrustedImm32(JSValue::Int32Tag)));
     99            speculationCheck(JSValueRegs(tagGPR, payloadGPR), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, tagGPR, TrustedImm32(JSValue::Int32Tag)));
    100100        m_gprs.unlock(tagGPR);
    101101        m_gprs.release(tagGPR);
    … …  
    121121    case DataFormatJSCell:
    122122    case DataFormatJSBoolean: {
    123         terminateSpeculativeExecution();
     123        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    124124        returnFormat = DataFormatInteger;
    125125        return allocate();
    … …  
    172172                return fpr;
    173173            } else {
    174                 terminateSpeculativeExecution();
     174                terminateSpeculativeExecution(JSValueRegs(), NoNode);
    175175                return fprAllocate();
    176176            }
    … …  
    188188            FPRReg fpr = fprAllocate();
    189189            JITCompiler::Jump isInteger = m_jit.branch32(MacroAssembler::Equal, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::Int32Tag));
    190             speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::LowestTag)));
     190            speculationCheck(JSValueSource(JITCompiler::addressFor(virtualRegister)), nodeIndex, m_jit.branch32(MacroAssembler::AboveOrEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::LowestTag)));
    191191            m_jit.loadDouble(JITCompiler::addressFor(virtualRegister), fpr);
    192192            JITCompiler::Jump hasUnboxedDouble = m_jit.jump();
    … …  
    210210
    211211    case DataFormatCell:
    212         terminateSpeculativeExecution();
     212        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    213213        return fprAllocate();
    214214
    … …  
    229229            FPRTemporary scratch(this);
    230230            JITCompiler::Jump isInteger = m_jit.branch32(MacroAssembler::Equal, tagGPR, TrustedImm32(JSValue::Int32Tag));
    231             speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, tagGPR, TrustedImm32(JSValue::LowestTag)));
     231            speculationCheck(JSValueRegs(tagGPR, payloadGPR), nodeIndex, m_jit.branch32(MacroAssembler::AboveOrEqual, tagGPR, TrustedImm32(JSValue::LowestTag)));
    232232            unboxDouble(tagGPR, payloadGPR, fpr, scratch.fpr());
    233233            hasUnboxedDouble = m_jit.jump();
    … …  
    292292                return gpr;
    293293            }
    294             terminateSpeculativeExecution();
     294            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    295295            return gpr;
    296296        }
    … …  
    298298        m_jit.load32(JITCompiler::tagFor(virtualRegister), gpr);
    299299        if (info.spillFormat() != DataFormatJSCell)
    300             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, gpr, TrustedImm32(JSValue::CellTag)));
     300            speculationCheck(JSValueSource(JITCompiler::addressFor(virtualRegister)), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, gpr, TrustedImm32(JSValue::CellTag)));
    301301        m_jit.load32(JITCompiler::payloadFor(virtualRegister), gpr);
    302302        m_gprs.retain(gpr, virtualRegister, SpillOrderSpilled);
    … …  
    318318        m_gprs.lock(payloadGPR);
    319319        if (info.spillFormat() != DataFormatJSCell)
    320             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, tagGPR, TrustedImm32(JSValue::CellTag)));
     320            speculationCheck(JSValueRegs(tagGPR, payloadGPR), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, tagGPR, TrustedImm32(JSValue::CellTag)));
    321321        m_gprs.unlock(tagGPR);
    322322        m_gprs.release(tagGPR);
    … …  
    333333    case DataFormatJSBoolean:
    334334    case DataFormatBoolean: {
    335         terminateSpeculativeExecution();
     335        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    336336        return allocate();
    337337    }
    … …  
    366366                return gpr;
    367367            }
    368             terminateSpeculativeExecution();
     368            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    369369            return gpr;
    370370        }
    … …  
    373373
    374374        if (info.spillFormat() != DataFormatJSBoolean)
    375             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::BooleanTag)));
     375            speculationCheck(JSValueSource(JITCompiler::addressFor(virtualRegister)), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::BooleanTag)));
    376376
    377377        m_jit.load32(JITCompiler::payloadFor(virtualRegister), gpr);
    … …  
    393393        m_gprs.lock(payloadGPR);
    394394        if (info.registerFormat() != DataFormatJSBoolean)
    395             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, tagGPR, TrustedImm32(JSValue::BooleanTag)));
     395            speculationCheck(JSValueRegs(tagGPR, payloadGPR), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, tagGPR, TrustedImm32(JSValue::BooleanTag)));
    396396
    397397        m_gprs.unlock(tagGPR);
    … …  
    409409    case DataFormatJSCell:
    410410    case DataFormatCell: {
    411         terminateSpeculativeExecution();
     411        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    412412        return allocate();
    413413    }
    … …  
    450450   
    451451    if (!predictionCheck(m_state.forNode(node.child1()).m_type))
    452         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op1GPR), MacroAssembler::TrustedImmPtr(vptr)));
     452        speculationCheck(JSValueSource::unboxedCell(op1GPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op1GPR), MacroAssembler::TrustedImmPtr(vptr)));
    453453    if (!predictionCheck(m_state.forNode(node.child2()).m_type))
    454         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op2GPR), MacroAssembler::TrustedImmPtr(vptr)));
     454        speculationCheck(JSValueSource::unboxedCell(op2GPR), node.child2(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op2GPR), MacroAssembler::TrustedImmPtr(vptr)));
    455455   
    456456    MacroAssembler::Jump falseCase = m_jit.branchPtr(MacroAssembler::NotEqual, op1GPR, op2GPR);
    … …  
    535535    MacroAssembler::Jump notCell = m_jit.branch32(MacroAssembler::NotEqual, valueTagGPR, TrustedImm32(JSValue::CellTag));
    536536    if (needSpeculationCheck)
    537         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valuePayloadGPR), MacroAssembler::TrustedImmPtr(vptr)));
     537        speculationCheck(JSValueRegs(valueTagGPR, valuePayloadGPR), nodeIndex, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valuePayloadGPR), MacroAssembler::TrustedImmPtr(vptr)));
    538538    m_jit.move(TrustedImm32(0), resultPayloadGPR);
    539539    MacroAssembler::Jump done = m_jit.jump();
    … …  
    545545        m_jit.move(valueTagGPR, resultPayloadGPR);
    546546        m_jit.or32(TrustedImm32(1), resultPayloadGPR);
    547         speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, resultPayloadGPR, TrustedImm32(JSValue::NullTag)));
     547        speculationCheck(JSValueRegs(valueTagGPR, valuePayloadGPR), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, resultPayloadGPR, TrustedImm32(JSValue::NullTag)));
    548548    }
    549549    m_jit.move(TrustedImm32(1), resultPayloadGPR);
    … …  
    592592    JSValueOperand value(this, node.child1());
    593593    GPRTemporary resultPayload(this, value, false);
    594     speculationCheck(m_jit.branch32(JITCompiler::NotEqual, value.tagGPR(), TrustedImm32(JSValue::BooleanTag)));
     594    speculationCheck(JSValueRegs(value.tagGPR(), value.payloadGPR()), node.child1(), m_jit.branch32(JITCompiler::NotEqual, value.tagGPR(), TrustedImm32(JSValue::BooleanTag)));
    595595    m_jit.move(value.payloadGPR(), resultPayload.gpr());
    596596    m_jit.xor32(TrustedImm32(1), resultPayload.gpr());
    … …  
    636636    MacroAssembler::Jump notCell = m_jit.branch32(MacroAssembler::NotEqual, valueTagGPR, TrustedImm32(JSValue::CellTag));
    637637    if (needSpeculationCheck)
    638         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valuePayloadGPR), MacroAssembler::TrustedImmPtr(vptr)));
     638        speculationCheck(JSValueRegs(valueTagGPR, valuePayloadGPR), nodeIndex, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valuePayloadGPR), MacroAssembler::TrustedImmPtr(vptr)));
    639639    addBranch(m_jit.jump(), taken);
    640640   
    … …  
    645645        m_jit.move(valueTagGPR, scratchGPR);
    646646        m_jit.or32(TrustedImm32(1), scratchGPR);
    647         speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, scratchGPR, TrustedImm32(JSValue::NullTag)));
     647        speculationCheck(JSValueRegs(valueTagGPR, valuePayloadGPR), nodeIndex, m_jit.branch32(MacroAssembler::NotEqual, scratchGPR, TrustedImm32(JSValue::NullTag)));
    648648    }
    649649
    … …  
    748748        // If we have no prediction for this local, then don't attempt to compile.
    749749        if (prediction == PredictNone) {
    750             terminateSpeculativeExecution();
     750            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    751751            break;
    752752        }
    … …  
    826826            GPRReg cellGPR = cell.gpr();
    827827            if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    828                 speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     828                speculationCheck(JSValueSource::unboxedCell(cellGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    829829            m_jit.storePtr(cellGPR, JITCompiler::payloadFor(node.local()));
    830830            noResult(m_compileIndex);
    … …  
    833833            GPRReg cellGPR = cell.gpr();
    834834            if (!isByteArrayPrediction(m_state.forNode(node.child1()).m_type))
    835                 speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     835                speculationCheck(JSValueSource::unboxedCell(cellGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    836836            m_jit.storePtr(cellGPR, JITCompiler::payloadFor(node.local()));
    837837            noResult(m_compileIndex);
    … …  
    939939
    940940        // Test the operand is positive.
    941         speculationCheck(m_jit.branch32(MacroAssembler::LessThan, op1.gpr(), TrustedImm32(0)));
     941        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, op1.gpr(), TrustedImm32(0)));
    942942
    943943        m_jit.move(op1.gpr(), result.gpr());
    … …  
    987987                    m_jit.add32(Imm32(imm1), result.gpr());
    988988                } else
    989                     speculationCheck(m_jit.branchAdd32(MacroAssembler::Overflow, op2.gpr(), Imm32(imm1), result.gpr()));
     989                    speculationCheck(JSValueRegs(), NoNode, m_jit.branchAdd32(MacroAssembler::Overflow, op2.gpr(), Imm32(imm1), result.gpr()));
    990990
    991991                integerResult(result.gpr(), m_compileIndex);
    … …  
    10021002                    m_jit.add32(Imm32(imm2), result.gpr());
    10031003                } else
    1004                     speculationCheck(m_jit.branchAdd32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
     1004                    speculationCheck(JSValueRegs(), NoNode, m_jit.branchAdd32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
    10051005
    10061006                integerResult(result.gpr(), m_compileIndex);
    … …  
    10271027               
    10281028                if (gpr1 == gprResult)
    1029                     speculationCheck(check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr2));
     1029                    speculationCheck(JSValueRegs(), NoNode, check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr2));
    10301030                else if (gpr2 == gprResult)
    1031                     speculationCheck(check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr1));
     1031                    speculationCheck(JSValueRegs(), NoNode, check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr1));
    10321032                else
    1033                     speculationCheck(check);
     1033                    speculationCheck(JSValueRegs(), NoNode, check);
    10341034            }
    10351035
    … …  
    10671067                    m_jit.sub32(Imm32(imm2), result.gpr());
    10681068                } else
    1069                     speculationCheck(m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
     1069                    speculationCheck(JSValueRegs(), NoNode, m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
    10701070
    10711071                integerResult(result.gpr(), m_compileIndex);
    … …  
    10811081                m_jit.sub32(op2.gpr(), result.gpr());
    10821082            } else
    1083                 speculationCheck(m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), op2.gpr(), result.gpr()));
     1083                speculationCheck(JSValueRegs(), NoNode, m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), op2.gpr(), result.gpr()));
    10841084
    10851085            integerResult(result.gpr(), m_compileIndex);
    … …  
    11161116            // domain.
    11171117           
    1118             speculationCheck(m_jit.branchMul32(MacroAssembler::Overflow, reg1, reg2, result.gpr()));
     1118            speculationCheck(JSValueRegs(), NoNode, m_jit.branchMul32(MacroAssembler::Overflow, reg1, reg2, result.gpr()));
    11191119           
    11201120            // Check for negative zero, if the users of this node care about such things.
    11211121            if (!nodeCanIgnoreNegativeZero(node.arithNodeFlags())) {
    11221122                MacroAssembler::Jump resultNonZero = m_jit.branchTest32(MacroAssembler::NonZero, result.gpr());
    1123                 speculationCheck(m_jit.branch32(MacroAssembler::LessThan, reg1, TrustedImm32(0)));
    1124                 speculationCheck(m_jit.branch32(MacroAssembler::LessThan, reg2, TrustedImm32(0)));
     1123                speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, reg1, TrustedImm32(0)));
     1124                speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, reg2, TrustedImm32(0)));
    11251125                resultNonZero.link(&m_jit);
    11261126            }
    … …  
    11521152            GPRReg op2GPR = op2.gpr();
    11531153           
    1154             speculationCheck(m_jit.branchTest32(JITCompiler::Zero, op2GPR));
     1154            speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(JITCompiler::Zero, op2GPR));
    11551155           
    11561156            // If the user cares about negative zero, then speculate that we're not about
    … …  
    11581158            if (!nodeCanIgnoreNegativeZero(node.arithNodeFlags())) {
    11591159                MacroAssembler::Jump numeratorNonZero = m_jit.branchTest32(MacroAssembler::NonZero, op1GPR);
    1160                 speculationCheck(m_jit.branch32(MacroAssembler::LessThan, op2GPR, TrustedImm32(0)));
     1160                speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, op2GPR, TrustedImm32(0)));
    11611161                numeratorNonZero.link(&m_jit);
    11621162            }
    … …  
    11781178            // Check that there was no remainder. If there had been, then we'd be obligated to
    11791179            // produce a double result instead.
    1180             speculationCheck(m_jit.branchTest32(JITCompiler::NonZero, edx.gpr()));
     1180            speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(JITCompiler::NonZero, edx.gpr()));
    11811181           
    11821182            integerResult(eax.gpr(), m_compileIndex);
    … …  
    12221222        GPRReg op2Gpr = op2.gpr();
    12231223
    1224         speculationCheck(m_jit.branchTest32(JITCompiler::Zero, op2Gpr));
     1224        speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(JITCompiler::Zero, op2Gpr));
    12251225
    12261226        GPRReg temp2 = InvalidGPRReg;
    … …  
    12521252            m_jit.add32(scratch.gpr(), result.gpr());
    12531253            m_jit.xor32(scratch.gpr(), result.gpr());
    1254             speculationCheck(m_jit.branch32(MacroAssembler::Equal, result.gpr(), MacroAssembler::TrustedImm32(1 << 31)));
     1254            speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::Equal, result.gpr(), MacroAssembler::TrustedImm32(1 << 31)));
    12551255            integerResult(result.gpr(), m_compileIndex);
    12561256            break;
    … …  
    14391439        // If we have predicted the base to be type array, we can skip the check.
    14401440        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1441             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    1442         speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(baseReg, JSArray::vectorLengthOffset())));
     1441            speculationCheck(JSValueSource::unboxedCell(baseReg), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1442        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(baseReg, JSArray::vectorLengthOffset())));
    14431443
    14441444        // FIXME: In cases where there are subsequent by_val accesses to the same base it might help to cache
    … …  
    14461446        // then we'll need to allocate a new temporary for result.
    14471447        m_jit.load32(MacroAssembler::BaseIndex(storageReg, propertyReg, MacroAssembler::TimesEight, OBJECT_OFFSETOF(ArrayStorage, m_vector[0]) + OBJECT_OFFSETOF(JSValue, u.asBits.tag)), resultTag.gpr());
    1448         speculationCheck(m_jit.branch32(MacroAssembler::Equal, resultTag.gpr(), TrustedImm32(JSValue::EmptyValueTag)));
     1448        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::Equal, resultTag.gpr(), TrustedImm32(JSValue::EmptyValueTag)));
    14491449        m_jit.load32(MacroAssembler::BaseIndex(storageReg, propertyReg, MacroAssembler::TimesEight, OBJECT_OFFSETOF(ArrayStorage, m_vector[0]) + OBJECT_OFFSETOF(JSValue, u.asBits.payload)), storageReg);
    14501450
    … …  
    14971497        // If we have predicted the base to be type array, we can skip the check.
    14981498        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1499             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1499            speculationCheck(JSValueSource::unboxedCell(baseReg), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    15001500
    15011501        base.use();
    … …  
    15851585
    15861586        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1587             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1587            speculationCheck(JSValueSource::unboxedCell(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    15881588       
    15891589        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSArray::storageOffset()), storageGPR);
    … …  
    15911591       
    15921592        // Refuse to handle bizarre lengths.
    1593         speculationCheck(m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(0x7ffffffe)));
     1593        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(0x7ffffffe)));
    15941594       
    15951595        MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(baseGPR, JSArray::vectorLengthOffset()));
    … …  
    16311631       
    16321632        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1633             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1633            speculationCheck(JSValueSource::unboxedCell(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    16341634       
    16351635        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSArray::storageOffset()), storageGPR);
    … …  
    17521752        // We expect that throw statements are rare and are intended to exit the code block
    17531753        // anyway, so we just OSR back to the old JIT for now.
    1754         terminateSpeculativeExecution();
     1754        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    17551755        break;
    17561756    }
    … …  
    18981898            m_jit.move(thisValueTagGPR, scratchGPR);
    18991899            m_jit.or32(TrustedImm32(1), scratchGPR);
    1900             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, scratchGPR, TrustedImm32(JSValue::NullTag)));
     1900            // This is hard. It would be better to save the value, but we can't quite do it,
     1901            // since this operation does not otherwise get the payload.
     1902            speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::NotEqual, scratchGPR, TrustedImm32(JSValue::NullTag)));
    19011903           
    19021904            m_jit.move(MacroAssembler::TrustedImmPtr(m_jit.codeBlock()->globalObject()), scratchGPR);
    … …  
    19121914           
    19131915            if (!isObjectPrediction(m_state.forNode(node.child1()).m_type))
    1914                 speculationCheck(m_jit.branchPtr(JITCompiler::Equal, JITCompiler::Address(thisValueGPR), JITCompiler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
     1916                speculationCheck(JSValueSource::unboxedCell(thisValueGPR), node.child1(), m_jit.branchPtr(JITCompiler::Equal, JITCompiler::Address(thisValueGPR), JITCompiler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
    19151917           
    19161918            m_jit.move(thisValueGPR, resultGPR);
    … …  
    19571959        if (at(node.child1()).shouldSpeculateFinalObject()) {
    19581960            if (!isFinalObjectPrediction(m_state.forNode(node.child1()).m_type))
    1959                 speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(protoGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsFinalObjectVPtr)));
     1961                speculationCheck(JSValueSource::unboxedCell(protoGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(protoGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsFinalObjectVPtr)));
    19601962        } else {
    19611963            m_jit.loadPtr(MacroAssembler::Address(protoGPR, JSCell::structureOffset()), scratchGPR);
    … …  
    20652067       
    20662068    case GetById: {
     2069        if (!node.prediction()) {
     2070            terminateSpeculativeExecution(JSValueRegs(), NoNode);
     2071            break;
     2072        }
     2073       
    20672074        SpeculateCellOperand base(this, node.child1());
    20682075        GPRTemporary resultTag(this, base);
    … …  
    20952102       
    20962103        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    2097             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     2104            speculationCheck(JSValueSource::unboxedCell(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    20982105       
    20992106        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSArray::storageOffset()), resultGPR);
    21002107        m_jit.load32(MacroAssembler::Address(resultGPR, OBJECT_OFFSETOF(ArrayStorage, m_length)), resultGPR);
    21012108       
    2102         speculationCheck(m_jit.branch32(MacroAssembler::LessThan, resultGPR, MacroAssembler::TrustedImm32(0)));
     2109        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, resultGPR, MacroAssembler::TrustedImm32(0)));
    21032110       
    21042111        integerResult(resultGPR, m_compileIndex);
    … …  
    21142121       
    21152122        if (!isStringPrediction(m_state.forNode(node.child1()).m_type))
    2116             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
     2123            speculationCheck(JSValueSource::unboxedCell(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
    21172124       
    21182125        m_jit.load32(MacroAssembler::Address(baseGPR, JSString::offsetOfLength()), resultGPR);
    … …  
    21302137       
    21312138        if (!isByteArrayPrediction(m_state.forNode(node.child1()).m_type))
    2132             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     2139            speculationCheck(JSValueSource::unboxedCell(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    21332140       
    21342141        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSByteArray::offsetOfStorage()), resultGPR);
    … …  
    21412148    case CheckFunction: {
    21422149        SpeculateCellOperand function(this, node.child1());
    2143         speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, function.gpr(), JITCompiler::TrustedImmPtr(node.function())));
     2150        speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, function.gpr(), JITCompiler::TrustedImmPtr(node.function())));
    21442151        noResult(m_compileIndex);
    21452152        break;
    … …  
    21572164       
    21582165        if (node.structureSet().size() == 1)
    2159             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(base.gpr(), JSCell::structureOffset()), JITCompiler::TrustedImmPtr(node.structureSet()[0])));
     2166            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(base.gpr(), JSCell::structureOffset()), JITCompiler::TrustedImmPtr(node.structureSet()[0])));
    21602167        else {
    21612168            GPRTemporary structure(this);
    … …  
    21682175                done.append(m_jit.branchPtr(JITCompiler::Equal, structure.gpr(), JITCompiler::TrustedImmPtr(node.structureSet()[i])));
    21692176           
    2170             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, structure.gpr(), JITCompiler::TrustedImmPtr(node.structureSet().last())));
     2177            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, structure.gpr(), JITCompiler::TrustedImmPtr(node.structureSet().last())));
    21712178           
    21722179            done.link(&m_jit);
    … …  
    22792286       
    22802287        if (!m_state.forNode(node.child1()).m_structure.doesNotContainAnyOtherThan(methodCheckData.structure))
    2281             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(baseGPR, JSCell::structureOffset()), JITCompiler::TrustedImmPtr(methodCheckData.structure)));
     2288            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(baseGPR, JSCell::structureOffset()), JITCompiler::TrustedImmPtr(methodCheckData.structure)));
    22822289        if (methodCheckData.prototype != m_jit.codeBlock()->globalObject()->methodCallDummy()) {
    22832290            m_jit.move(JITCompiler::TrustedImmPtr(methodCheckData.prototype->structureAddress()), scratchGPR);
    2284             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(scratchGPR), JITCompiler::TrustedImmPtr(methodCheckData.prototypeStructure)));
     2291            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(scratchGPR), JITCompiler::TrustedImmPtr(methodCheckData.prototypeStructure)));
    22852292        }
    22862293       
    … …  
    23672374        // Speculate that base 'ImplementsDefaultHasInstance'.
    23682375        m_jit.loadPtr(MacroAssembler::Address(base.gpr(), JSCell::structureOffset()), structure.gpr());
    2369         speculationCheck(m_jit.branchTest8(MacroAssembler::Zero, MacroAssembler::Address(structure.gpr(), Structure::typeInfoFlagsOffset()), MacroAssembler::TrustedImm32(ImplementsDefaultHasInstance)));
     2376        speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest8(MacroAssembler::Zero, MacroAssembler::Address(structure.gpr(), Structure::typeInfoFlagsOffset()), MacroAssembler::TrustedImm32(ImplementsDefaultHasInstance)));
    23702377
    23712378        noResult(m_compileIndex);
    … …  
    23862393        // Check that prototype is an object.
    23872394        m_jit.loadPtr(MacroAssembler::Address(prototypeReg, JSCell::structureOffset()), scratchReg);
    2388         speculationCheck(m_jit.branch8(MacroAssembler::NotEqual, MacroAssembler::Address(scratchReg, Structure::typeInfoTypeOffset()), MacroAssembler::TrustedImm32(ObjectType)));
     2395        speculationCheck(JSValueRegs(), NoNode, m_jit.branch8(MacroAssembler::NotEqual, MacroAssembler::Address(scratchReg, Structure::typeInfoTypeOffset()), MacroAssembler::TrustedImm32(ObjectType)));
    23892396
    23902397        // Initialize scratchReg with the value being checked.
    … …  
    24952502
    24962503    case ForceOSRExit: {
    2497         terminateSpeculativeExecution();
     2504        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    24982505        break;
    24992506    }

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp

    r98299 r98912  
    4848    case DataFormatNone: {
    4949        if ((node.hasConstant() && !isInt32Constant(nodeIndex)) || info.spillFormat() == DataFormatDouble) {
    50             terminateSpeculativeExecution();
     50            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    5151            returnFormat = DataFormatInteger;
    5252            return allocate();
    … …  
    9898        GPRReg gpr = info.gpr();
    9999        m_gprs.lock(gpr);
    100         speculationCheck(m_jit.branchPtr(MacroAssembler::Below, gpr, GPRInfo::tagTypeNumberRegister));
     100        speculationCheck(JSValueRegs(gpr), nodeIndex, m_jit.branchPtr(MacroAssembler::Below, gpr, GPRInfo::tagTypeNumberRegister));
    101101        info.fillJSValue(gpr, DataFormatJSInteger);
    102102        // If !strict we're done, return.
    … …  
    147147    case DataFormatJSCell:
    148148    case DataFormatJSBoolean: {
    149         terminateSpeculativeExecution();
     149        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    150150        returnFormat = DataFormatInteger;
    151151        return allocate();
    … …  
    206206                return fpr;
    207207            }
    208             terminateSpeculativeExecution();
     208            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    209209            return fprAllocate();
    210210        }
    … …  
    249249
    250250    case DataFormatCell:
    251         terminateSpeculativeExecution();
     251        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    252252        return fprAllocate();
    253253
    … …  
    262262        JITCompiler::Jump isInteger = m_jit.branchPtr(MacroAssembler::AboveOrEqual, jsValueGpr, GPRInfo::tagTypeNumberRegister);
    263263
    264         speculationCheck(m_jit.branchTestPtr(MacroAssembler::Zero, jsValueGpr, GPRInfo::tagTypeNumberRegister));
     264        speculationCheck(JSValueRegs(jsValueGpr), nodeIndex, m_jit.branchTestPtr(MacroAssembler::Zero, jsValueGpr, GPRInfo::tagTypeNumberRegister));
    265265
    266266        // First, if we get here we have a double encoded as a JSValue
    … …  
    336336    case DataFormatNone: {
    337337        if (info.spillFormat() == DataFormatInteger || info.spillFormat() == DataFormatDouble) {
    338             terminateSpeculativeExecution();
     338            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    339339            return allocate();
    340340        }
    … …  
    350350                return gpr;
    351351            }
    352             terminateSpeculativeExecution();
     352            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    353353            return gpr;
    354354        }
    … …  
    359359        info.fillJSValue(gpr, DataFormatJS);
    360360        if (info.spillFormat() != DataFormatJSCell)
    361             speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, GPRInfo::tagMaskRegister));
     361            speculationCheck(JSValueRegs(gpr), nodeIndex, m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, GPRInfo::tagMaskRegister));
    362362        info.fillJSValue(gpr, DataFormatJSCell);
    363363        return gpr;
    … …  
    374374        GPRReg gpr = info.gpr();
    375375        m_gprs.lock(gpr);
    376         speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, GPRInfo::tagMaskRegister));
     376        speculationCheck(JSValueRegs(gpr), nodeIndex, m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, GPRInfo::tagMaskRegister));
    377377        info.fillJSValue(gpr, DataFormatJSCell);
    378378        return gpr;
    … …  
    385385    case DataFormatJSBoolean:
    386386    case DataFormatBoolean: {
    387         terminateSpeculativeExecution();
     387        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    388388        return allocate();
    389389    }
    … …  
    409409    case DataFormatNone: {
    410410        if (info.spillFormat() == DataFormatInteger || info.spillFormat() == DataFormatDouble) {
    411             terminateSpeculativeExecution();
     411            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    412412            return allocate();
    413413        }
    … …  
    423423                return gpr;
    424424            }
    425             terminateSpeculativeExecution();
     425            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    426426            return gpr;
    427427        }
    … …  
    433433        if (info.spillFormat() != DataFormatJSBoolean) {
    434434            m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
    435             speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, TrustedImm32(static_cast<int32_t>(~1))), SpeculationRecovery(BooleanSpeculationCheck, gpr, InvalidGPRReg));
     435            speculationCheck(JSValueRegs(gpr), nodeIndex, m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, TrustedImm32(static_cast<int32_t>(~1))), SpeculationRecovery(BooleanSpeculationCheck, gpr, InvalidGPRReg));
    436436            m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
    437437        }
    … …  
    451451        m_gprs.lock(gpr);
    452452        m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
    453         speculationCheck(m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, TrustedImm32(static_cast<int32_t>(~1))), SpeculationRecovery(BooleanSpeculationCheck, gpr, InvalidGPRReg));
     453        speculationCheck(JSValueRegs(gpr), nodeIndex, m_jit.branchTestPtr(MacroAssembler::NonZero, gpr, TrustedImm32(static_cast<int32_t>(~1))), SpeculationRecovery(BooleanSpeculationCheck, gpr, InvalidGPRReg));
    454454        m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), gpr);
    455455        info.fillJSValue(gpr, DataFormatJSBoolean);
    … …  
    463463    case DataFormatJSCell:
    464464    case DataFormatCell: {
    465         terminateSpeculativeExecution();
     465        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    466466        return allocate();
    467467    }
    … …  
    506506   
    507507    if (!predictionCheck(m_state.forNode(node.child1()).m_type))
    508         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op1GPR), MacroAssembler::TrustedImmPtr(vptr)));
     508        speculationCheck(JSValueRegs(op1GPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op1GPR), MacroAssembler::TrustedImmPtr(vptr)));
    509509    if (!predictionCheck(m_state.forNode(node.child2()).m_type))
    510         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op2GPR), MacroAssembler::TrustedImmPtr(vptr)));
     510        speculationCheck(JSValueRegs(op2GPR), node.child2(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(op2GPR), MacroAssembler::TrustedImmPtr(vptr)));
    511511   
    512512    MacroAssembler::Jump falseCase = m_jit.branchPtr(MacroAssembler::NotEqual, op1GPR, op2GPR);
    … …  
    585585    MacroAssembler::Jump notCell = m_jit.branchTestPtr(MacroAssembler::NonZero, valueGPR, GPRInfo::tagMaskRegister);
    586586    if (needSpeculationCheck)
    587         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valueGPR), MacroAssembler::TrustedImmPtr(vptr)));
     587        speculationCheck(JSValueRegs(valueGPR), nodeIndex, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valueGPR), MacroAssembler::TrustedImmPtr(vptr)));
    588588    m_jit.move(TrustedImm32(static_cast<int32_t>(ValueFalse)), resultGPR);
    589589    MacroAssembler::Jump done = m_jit.jump();
    … …  
    594594        m_jit.move(valueGPR, resultGPR);
    595595        m_jit.andPtr(MacroAssembler::TrustedImm32(~TagBitUndefined), resultGPR);
    596         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, resultGPR, MacroAssembler::TrustedImmPtr(reinterpret_cast<void*>(ValueNull))));
     596        speculationCheck(JSValueRegs(valueGPR), nodeIndex, m_jit.branchPtr(MacroAssembler::NotEqual, resultGPR, MacroAssembler::TrustedImmPtr(reinterpret_cast<void*>(ValueNull))));
    597597    }
    598598    m_jit.move(TrustedImm32(static_cast<int32_t>(ValueTrue)), resultGPR);
    … …  
    650650        m_jit.move(value.gpr(), result.gpr());
    651651        m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueFalse)), result.gpr());
    652         speculationCheck(m_jit.branchTestPtr(JITCompiler::NonZero, result.gpr(), TrustedImm32(static_cast<int32_t>(~1))));
     652        speculationCheck(JSValueRegs(value.gpr()), node.child1(), m_jit.branchTestPtr(JITCompiler::NonZero, result.gpr(), TrustedImm32(static_cast<int32_t>(~1))));
    653653        m_jit.xorPtr(TrustedImm32(static_cast<int32_t>(ValueTrue)), result.gpr());
    654654       
    … …  
    689689    MacroAssembler::Jump notCell = m_jit.branchTestPtr(MacroAssembler::NonZero, valueGPR, GPRInfo::tagMaskRegister);
    690690    if (needSpeculationCheck)
    691         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valueGPR), MacroAssembler::TrustedImmPtr(vptr)));
     691        speculationCheck(JSValueRegs(valueGPR), nodeIndex, m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(valueGPR), MacroAssembler::TrustedImmPtr(vptr)));
    692692    addBranch(m_jit.jump(), taken);
    693693   
    … …  
    697697        m_jit.move(valueGPR, scratchGPR);
    698698        m_jit.andPtr(MacroAssembler::TrustedImm32(~TagBitUndefined), scratchGPR);
    699         speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, scratchGPR, MacroAssembler::TrustedImmPtr(reinterpret_cast<void*>(ValueNull))));
     699        speculationCheck(JSValueRegs(valueGPR), nodeIndex, m_jit.branchPtr(MacroAssembler::NotEqual, scratchGPR, MacroAssembler::TrustedImmPtr(reinterpret_cast<void*>(ValueNull))));
    700700    }
    701701    if (notTaken != (m_block + 1))
    … …  
    765765            addBranch(m_jit.branchPtr(MacroAssembler::Equal, valueGPR, MacroAssembler::ImmPtr(JSValue::encode(jsBoolean(true)))), taken);
    766766
    767             speculationCheck(m_jit.jump());
     767            speculationCheck(JSValueRegs(valueGPR), node.child1(), m_jit.jump());
    768768            value.use();
    769769        } else {
    … …  
    806806        // If we have no prediction for this local, then don't attempt to compile.
    807807        if (prediction == PredictNone || value.isClear()) {
    808             terminateSpeculativeExecution();
     808            terminateSpeculativeExecution(JSValueRegs(), NoNode);
    809809            break;
    810810        }
    … …  
    876876            GPRReg cellGPR = cell.gpr();
    877877            if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    878                 speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     878                speculationCheck(JSValueRegs(cellGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    879879            m_jit.storePtr(cellGPR, JITCompiler::addressFor(node.local()));
    880880            noResult(m_compileIndex);
    … …  
    883883            GPRReg cellGPR = cell.gpr();
    884884            if (!isByteArrayPrediction(m_state.forNode(node.child1()).m_type))
    885                 speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     885                speculationCheck(JSValueRegs(cellGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(cellGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    886886            m_jit.storePtr(cellGPR, JITCompiler::addressFor(node.local()));
    887887            noResult(m_compileIndex);
    … …  
    988988
    989989        // Test the operand is positive.
    990         speculationCheck(m_jit.branch32(MacroAssembler::LessThan, op1.gpr(), TrustedImm32(0)));
     990        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, op1.gpr(), TrustedImm32(0)));
    991991
    992992        m_jit.move(op1.gpr(), result.gpr());
    … …  
    10361036                    m_jit.add32(Imm32(imm1), result.gpr());
    10371037                } else
    1038                     speculationCheck(m_jit.branchAdd32(MacroAssembler::Overflow, op2.gpr(), Imm32(imm1), result.gpr()));
     1038                    speculationCheck(JSValueRegs(), NoNode, m_jit.branchAdd32(MacroAssembler::Overflow, op2.gpr(), Imm32(imm1), result.gpr()));
    10391039
    10401040                integerResult(result.gpr(), m_compileIndex);
    … …  
    10511051                    m_jit.add32(Imm32(imm2), result.gpr());
    10521052                } else
    1053                     speculationCheck(m_jit.branchAdd32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
     1053                    speculationCheck(JSValueRegs(), NoNode, m_jit.branchAdd32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
    10541054
    10551055                integerResult(result.gpr(), m_compileIndex);
    … …  
    10761076               
    10771077                if (gpr1 == gprResult)
    1078                     speculationCheck(check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr2));
     1078                    speculationCheck(JSValueRegs(), NoNode, check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr2));
    10791079                else if (gpr2 == gprResult)
    1080                     speculationCheck(check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr1));
     1080                    speculationCheck(JSValueRegs(), NoNode, check, SpeculationRecovery(SpeculativeAdd, gprResult, gpr1));
    10811081                else
    1082                     speculationCheck(check);
     1082                    speculationCheck(JSValueRegs(), NoNode, check);
    10831083            }
    10841084
    … …  
    11161116                    m_jit.sub32(Imm32(imm2), result.gpr());
    11171117                } else
    1118                     speculationCheck(m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
     1118                    speculationCheck(JSValueRegs(), NoNode, m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), Imm32(imm2), result.gpr()));
    11191119
    11201120                integerResult(result.gpr(), m_compileIndex);
    … …  
    11301130                m_jit.sub32(op2.gpr(), result.gpr());
    11311131            } else
    1132                 speculationCheck(m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), op2.gpr(), result.gpr()));
     1132                speculationCheck(JSValueRegs(), NoNode, m_jit.branchSub32(MacroAssembler::Overflow, op1.gpr(), op2.gpr(), result.gpr()));
    11331133
    11341134            integerResult(result.gpr(), m_compileIndex);
    … …  
    11651165            // domain.
    11661166           
    1167             speculationCheck(m_jit.branchMul32(MacroAssembler::Overflow, reg1, reg2, result.gpr()));
     1167            speculationCheck(JSValueRegs(), NoNode, m_jit.branchMul32(MacroAssembler::Overflow, reg1, reg2, result.gpr()));
    11681168           
    11691169            // Check for negative zero, if the users of this node care about such things.
    11701170            if (!nodeCanIgnoreNegativeZero(node.arithNodeFlags())) {
    11711171                MacroAssembler::Jump resultNonZero = m_jit.branchTest32(MacroAssembler::NonZero, result.gpr());
    1172                 speculationCheck(m_jit.branch32(MacroAssembler::LessThan, reg1, TrustedImm32(0)));
    1173                 speculationCheck(m_jit.branch32(MacroAssembler::LessThan, reg2, TrustedImm32(0)));
     1172                speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, reg1, TrustedImm32(0)));
     1173                speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, reg2, TrustedImm32(0)));
    11741174                resultNonZero.link(&m_jit);
    11751175            }
    … …  
    12011201            GPRReg op2GPR = op2.gpr();
    12021202           
    1203             speculationCheck(m_jit.branchTest32(JITCompiler::Zero, op2GPR));
     1203            speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(JITCompiler::Zero, op2GPR));
    12041204           
    12051205            // If the user cares about negative zero, then speculate that we're not about
    … …  
    12071207            if (!nodeCanIgnoreNegativeZero(node.arithNodeFlags())) {
    12081208                MacroAssembler::Jump numeratorNonZero = m_jit.branchTest32(MacroAssembler::NonZero, op1GPR);
    1209                 speculationCheck(m_jit.branch32(MacroAssembler::LessThan, op2GPR, TrustedImm32(0)));
     1209                speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, op2GPR, TrustedImm32(0)));
    12101210                numeratorNonZero.link(&m_jit);
    12111211            }
    … …  
    12271227            // Check that there was no remainder. If there had been, then we'd be obligated to
    12281228            // produce a double result instead.
    1229             speculationCheck(m_jit.branchTest32(JITCompiler::NonZero, edx.gpr()));
     1229            speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(JITCompiler::NonZero, edx.gpr()));
    12301230           
    12311231            integerResult(eax.gpr(), m_compileIndex);
    … …  
    12711271        GPRReg op2Gpr = op2.gpr();
    12721272
    1273         speculationCheck(m_jit.branchTest32(JITCompiler::Zero, op2Gpr));
     1273        speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest32(JITCompiler::Zero, op2Gpr));
    12741274
    12751275        GPRReg temp2 = InvalidGPRReg;
    … …  
    13011301            m_jit.add32(scratch.gpr(), result.gpr());
    13021302            m_jit.xor32(scratch.gpr(), result.gpr());
    1303             speculationCheck(m_jit.branch32(MacroAssembler::Equal, result.gpr(), MacroAssembler::TrustedImm32(1 << 31)));
     1303            speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::Equal, result.gpr(), MacroAssembler::TrustedImm32(1 << 31)));
    13041304            integerResult(result.gpr(), m_compileIndex);
    13051305            break;
    … …  
    14851485        // If we have predicted the base to be type array, we can skip the check.
    14861486        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1487             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    1488         speculationCheck(m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(baseReg, JSArray::vectorLengthOffset())));
     1487            speculationCheck(JSValueRegs(baseReg), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1488        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::AboveOrEqual, propertyReg, MacroAssembler::Address(baseReg, JSArray::vectorLengthOffset())));
    14891489
    14901490        // FIXME: In cases where there are subsequent by_val accesses to the same base it might help to cache
    … …  
    14931493        GPRTemporary& result = storage;
    14941494        m_jit.loadPtr(MacroAssembler::BaseIndex(storageReg, propertyReg, MacroAssembler::ScalePtr, OBJECT_OFFSETOF(ArrayStorage, m_vector[0])), result.gpr());
    1495         speculationCheck(m_jit.branchTestPtr(MacroAssembler::Zero, result.gpr()));
     1495        speculationCheck(JSValueRegs(), NoNode, m_jit.branchTestPtr(MacroAssembler::Zero, result.gpr()));
    14961496
    14971497        jsValueResult(result.gpr(), m_compileIndex);
    … …  
    15401540        // If we have predicted the base to be type array, we can skip the check.
    15411541        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1542             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1542            speculationCheck(JSValueRegs(baseReg), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseReg), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    15431543
    15441544        base.use();
    … …  
    16281628
    16291629        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1630             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1630            speculationCheck(JSValueRegs(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    16311631       
    16321632        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSArray::storageOffset()), storageGPR);
    … …  
    16341634       
    16351635        // Refuse to handle bizarre lengths.
    1636         speculationCheck(m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(0x7ffffffe)));
     1636        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(0x7ffffffe)));
    16371637       
    16381638        MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(baseGPR, JSArray::vectorLengthOffset()));
    … …  
    16711671       
    16721672        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    1673             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     1673            speculationCheck(JSValueRegs(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    16741674       
    16751675        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSArray::storageOffset()), storageGPR);
    … …  
    17761776        // We expect that throw statements are rare and are intended to exit the code block
    17771777        // anyway, so we just OSR back to the old JIT for now.
    1778         terminateSpeculativeExecution();
     1778        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    17791779        break;
    17801780    }
    … …  
    19091909                m_jit.move(thisValueGPR, scratchGPR);
    19101910                m_jit.andPtr(MacroAssembler::TrustedImm32(~TagBitUndefined), scratchGPR);
    1911                 speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, scratchGPR, MacroAssembler::TrustedImmPtr(reinterpret_cast<void*>(ValueNull))));
     1911                speculationCheck(JSValueRegs(thisValueGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, scratchGPR, MacroAssembler::TrustedImmPtr(reinterpret_cast<void*>(ValueNull))));
    19121912            }
    19131913           
    … …  
    19241924           
    19251925            if (!isObjectPrediction(m_state.forNode(node.child1()).m_type))
    1926                 speculationCheck(m_jit.branchPtr(JITCompiler::Equal, JITCompiler::Address(thisValueGPR), JITCompiler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
     1926                speculationCheck(JSValueRegs(thisValueGPR), node.child1(), m_jit.branchPtr(JITCompiler::Equal, JITCompiler::Address(thisValueGPR), JITCompiler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
    19271927           
    19281928            m_jit.move(thisValueGPR, resultGPR);
    … …  
    19681968        if (at(node.child1()).shouldSpeculateFinalObject()) {
    19691969            if (!isFinalObjectPrediction(m_state.forNode(node.child1()).m_type))
    1970                 speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(protoGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsFinalObjectVPtr)));
     1970                speculationCheck(JSValueRegs(protoGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(protoGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsFinalObjectVPtr)));
    19711971        } else {
    19721972            m_jit.loadPtr(MacroAssembler::Address(protoGPR, JSCell::structureOffset()), scratchGPR);
    … …  
    20712071    }
    20722072    case GetById: {
     2073        if (!node.prediction()) {
     2074            terminateSpeculativeExecution(JSValueRegs(), NoNode);
     2075            break;
     2076        }
     2077       
    20732078        SpeculateCellOperand base(this, node.child1());
    20742079        GPRTemporary result(this, base);
    … …  
    20992104       
    21002105        if (!isArrayPrediction(m_state.forNode(node.child1()).m_type))
    2101             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
     2106            speculationCheck(JSValueRegs(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsArrayVPtr)));
    21022107       
    21032108        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSArray::storageOffset()), resultGPR);
    21042109        m_jit.load32(MacroAssembler::Address(resultGPR, OBJECT_OFFSETOF(ArrayStorage, m_length)), resultGPR);
    21052110       
    2106         speculationCheck(m_jit.branch32(MacroAssembler::LessThan, resultGPR, MacroAssembler::TrustedImm32(0)));
     2111        speculationCheck(JSValueRegs(), NoNode, m_jit.branch32(MacroAssembler::LessThan, resultGPR, MacroAssembler::TrustedImm32(0)));
    21072112       
    21082113        integerResult(resultGPR, m_compileIndex);
    … …  
    21182123       
    21192124        if (!isStringPrediction(m_state.forNode(node.child1()).m_type))
    2120             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
     2125            speculationCheck(JSValueRegs(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsStringVPtr)));
    21212126       
    21222127        m_jit.load32(MacroAssembler::Address(baseGPR, JSString::offsetOfLength()), resultGPR);
    … …  
    21342139       
    21352140        if (!isByteArrayPrediction(m_state.forNode(node.child1()).m_type))
    2136             speculationCheck(m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
     2141            speculationCheck(JSValueRegs(baseGPR), node.child1(), m_jit.branchPtr(MacroAssembler::NotEqual, MacroAssembler::Address(baseGPR), MacroAssembler::TrustedImmPtr(m_jit.globalData()->jsByteArrayVPtr)));
    21372142       
    21382143        m_jit.loadPtr(MacroAssembler::Address(baseGPR, JSByteArray::offsetOfStorage()), resultGPR);
    … …  
    21452150    case CheckFunction: {
    21462151        SpeculateCellOperand function(this, node.child1());
    2147         speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, function.gpr(), JITCompiler::TrustedImmPtr(node.function())));
     2152        speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, function.gpr(), JITCompiler::TrustedImmPtr(node.function())));
    21482153        noResult(m_compileIndex);
    21492154        break;
    … …  
    21602165       
    21612166        if (node.structureSet().size() == 1)
    2162             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(base.gpr(), JSCell::structureOffset()), JITCompiler::TrustedImmPtr(node.structureSet()[0])));
     2167            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(base.gpr(), JSCell::structureOffset()), JITCompiler::TrustedImmPtr(node.structureSet()[0])));
    21632168        else {
    21642169            GPRTemporary structure(this);
    … …  
    21712176                done.append(m_jit.branchPtr(JITCompiler::Equal, structure.gpr(), JITCompiler::TrustedImmPtr(node.structureSet()[i])));
    21722177           
    2173             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, structure.gpr(), JITCompiler::TrustedImmPtr(node.structureSet().last())));
     2178            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, structure.gpr(), JITCompiler::TrustedImmPtr(node.structureSet().last())));
    21742179           
    21752180            done.link(&m_jit);
    … …  
    22752280       
    22762281        if (!m_state.forNode(node.child1()).m_structure.doesNotContainAnyOtherThan(methodCheckData.structure))
    2277             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(baseGPR, JSCell::structureOffset()), JITCompiler::TrustedImmPtr(methodCheckData.structure)));
     2282            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(baseGPR, JSCell::structureOffset()), JITCompiler::TrustedImmPtr(methodCheckData.structure)));
    22782283        if (methodCheckData.prototype != m_jit.codeBlock()->globalObject()->methodCallDummy()) {
    22792284            m_jit.move(JITCompiler::TrustedImmPtr(methodCheckData.prototype->structureAddress()), scratchGPR);
    2280             speculationCheck(m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(scratchGPR), JITCompiler::TrustedImmPtr(methodCheckData.prototypeStructure)));
     2285            speculationCheck(JSValueRegs(), NoNode, m_jit.branchPtr(JITCompiler::NotEqual, JITCompiler::Address(scratchGPR), JITCompiler::TrustedImmPtr(methodCheckData.prototypeStructure)));
    22812286        }
    22822287       
    … …  
    23582363        // Speculate that base 'ImplementsDefaultHasInstance'.
    23592364        m_jit.loadPtr(MacroAssembler::Address(base.gpr(), JSCell::structureOffset()), structure.gpr());
    2360         speculationCheck(m_jit.branchTest8(MacroAssembler::Zero, MacroAssembler::Address(structure.gpr(), Structure::typeInfoFlagsOffset()), MacroAssembler::TrustedImm32(ImplementsDefaultHasInstance)));
     2365        speculationCheck(JSValueRegs(), NoNode, m_jit.branchTest8(MacroAssembler::Zero, MacroAssembler::Address(structure.gpr(), Structure::typeInfoFlagsOffset()), MacroAssembler::TrustedImm32(ImplementsDefaultHasInstance)));
    23612366
    23622367        noResult(m_compileIndex);
    … …  
    23772382        // Check that prototype is an object.
    23782383        m_jit.loadPtr(MacroAssembler::Address(prototypeReg, JSCell::structureOffset()), scratchReg);
    2379         speculationCheck(m_jit.branchIfNotObject(scratchReg));
     2384        speculationCheck(JSValueRegs(), NoNode, m_jit.branchIfNotObject(scratchReg));
    23802385
    23812386        // Initialize scratchReg with the value being checked.
    … …  
    24792484
    24802485    case ForceOSRExit: {
    2481         terminateSpeculativeExecution();
     2486        terminateSpeculativeExecution(JSValueRegs(), NoNode);
    24822487        break;
    24832488    }

  • trunk/Source/JavaScriptCore/jit/JITPropertyAccess.cpp

    r98674 r98912  
    348348
    349349    compileGetByIdSlowCase(resultVReg, baseVReg, ident, iter, true);
     350    emitValueProfilingSite(SubsequentProfilingSite);
    350351
    351352    // We've already generated the following get_by_id, so make sure it's skipped over.

  • trunk/Source/JavaScriptCore/jit/JITPropertyAccess32_64.cpp

    r98674 r98912  
    162162   
    163163    compileGetByIdSlowCase(dst, base, &(m_codeBlock->identifier(ident)), iter, true);
     164    emitValueProfilingSite(SubsequentProfilingSite);
    164165   
    165166    // We've already generated the following get_by_id, so make sure it's skipped over.

  • trunk/Source/JavaScriptCore/runtime/Heuristics.cpp

    r98214 r98912  
    147147    SET(maximumOptimizationDelay,   5);
    148148    SET(desiredProfileLivenessRate, 0.75);
    149     SET(desiredProfileFullnessRate, 0.25);
     149    SET(desiredProfileFullnessRate, 0.35);
    150150   
    151151    ASSERT(executionCounterValueForDontOptimizeAnytimeSoon <= executionCounterValueForOptimizeAfterLongWarmUp);

  • trunk/Source/JavaScriptCore/runtime/JSValue.h

    r98679 r98912  
    5050        class JITCompiler;
    5151        class JITCodeGenerator;
     52        class JSValueSource;
    5253        class SpeculativeJIT;
    5354    }
    … …  
    111112        friend class DFG::JITCompiler;
    112113        friend class DFG::JITCodeGenerator;
     114        friend class DFG::JSValueSource;
    113115        friend class DFG::SpeculativeJIT;
    114116#endif
Note: See TracChangeset for help on using the changeset viewer.