Changeset 222675 in webkit


Ignore:
Timestamp:
Sep 29, 2017 6:16:52 PM (7 years ago)
Author:
Yusuke Suzuki
Message:

[DFG] Support ArrayPush with multiple args
​https://bugs.webkit.org/show_bug.cgi?id=175823

Reviewed by Saam Barati.

JSTests:

  • microbenchmarks/array-push-0.js: Added.

(arrayPush0):

  • microbenchmarks/array-push-1.js: Added.

(arrayPush1):

  • microbenchmarks/array-push-2.js: Added.

(arrayPush2):

  • microbenchmarks/array-push-3.js: Added.

(arrayPush3):

  • stress/array-push-multiple-contiguous.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-double-nan.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-double.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-int32.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-many-contiguous.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-many-double.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-many-int32.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-many-storage.js: Added.

(shouldBe):
(test):

  • stress/array-push-multiple-storage.js: Added.

(shouldBe):
(test):

  • stress/array-push-with-force-exit.js: Added.

(target.createBuiltin):

Source/JavaScriptCore:

Reviewed by Saam Barati.

This patch implements ArrayPush(with multiple arguments) in DFG and FTL. Previously, they are not handled
by ArrayPush. Then they go to generic direct call to Array#push and it does in slow path. This patch
extends ArrayPush to push multiple arguments in a bulk push manner.

The problem of ArrayPush is that we need to perform ArrayPush atomically: If OSR exit occurs in the middle
of ArrayPush, we incorrectly push pushed elements twice. Once we start pushing values, we should not exit.
But we do not want to iterate elements twice, once for type checks and once for actually pushing it. It
could move elements between registers and memory back and forth.

This patch achieves the above goal by separating type checks from ArrayPush. When starting ArrayPush, type
checks for elements are already done by separately emitted Check nodes.

We also add JSArray::pushInline for DFG operations just calling JSArray::push. And we also use it in
arrayProtoFuncPush's fast path.

This patch significantly improves performance of push(multiple args).

baseline patched

Microbenchmarks:

array-push-0 461.8455+-28.9995 151.3438+-6.5653 definitely 3.0516x faster
array-push-1 133.8845+-7.0349 ? 136.1775+-5.8327 ? might be 1.0171x slower
array-push-2 675.6555+-13.4645 145.8747+-6.4621 definitely 4.6318x faster
array-push-3 849.5284+-15.2540 253.4421+-9.1249 definitely 3.3520x faster

baseline patched

SixSpeed:

spread-literal.es5 90.3482+-6.6514 24.8123+-2.3304 definitely 3.6413x faster

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::handleIntrinsicCall):

  • dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::fixupNode):

  • dfg/DFGNodeType.h:
  • dfg/DFGOperations.cpp:
  • dfg/DFGOperations.h:
  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::compileArrayPush):

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::callOperation):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGStoreBarrierInsertionPhase.cpp:
  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileArrayPush):

  • jit/JITOperations.h:
  • runtime/ArrayPrototype.cpp:

(JSC::arrayProtoFuncPush):

  • runtime/JSArray.cpp:

(JSC::JSArray::push):

  • runtime/JSArray.h:
  • runtime/JSArrayInlines.h:

(JSC::JSArray::pushInline):

Location:
trunk
Files:
14 added
18 edited

Legend:

Unmodified
Added
Removed

  • trunk/JSTests/ChangeLog

    r222671 r222675  
     12017-09-29  Yusuke Suzuki  <utatane.tea@gmail.com>
     2
     3        [DFG] Support ArrayPush with multiple args
     4        https://bugs.webkit.org/show_bug.cgi?id=175823
     5
     6        Reviewed by Saam Barati.
     7
     8        * microbenchmarks/array-push-0.js: Added.
     9        (arrayPush0):
     10        * microbenchmarks/array-push-1.js: Added.
     11        (arrayPush1):
     12        * microbenchmarks/array-push-2.js: Added.
     13        (arrayPush2):
     14        * microbenchmarks/array-push-3.js: Added.
     15        (arrayPush3):
     16        * stress/array-push-multiple-contiguous.js: Added.
     17        (shouldBe):
     18        (test):
     19        * stress/array-push-multiple-double-nan.js: Added.
     20        (shouldBe):
     21        (test):
     22        * stress/array-push-multiple-double.js: Added.
     23        (shouldBe):
     24        (test):
     25        * stress/array-push-multiple-int32.js: Added.
     26        (shouldBe):
     27        (test):
     28        * stress/array-push-multiple-many-contiguous.js: Added.
     29        (shouldBe):
     30        (test):
     31        * stress/array-push-multiple-many-double.js: Added.
     32        (shouldBe):
     33        (test):
     34        * stress/array-push-multiple-many-int32.js: Added.
     35        (shouldBe):
     36        (test):
     37        * stress/array-push-multiple-many-storage.js: Added.
     38        (shouldBe):
     39        (test):
     40        * stress/array-push-multiple-storage.js: Added.
     41        (shouldBe):
     42        (test):
     43        * stress/array-push-with-force-exit.js: Added.
     44        (target.createBuiltin):
     45
    1462017-09-29  Saam Barati  <sbarati@apple.com>
    247

  • trunk/Source/JavaScriptCore/ChangeLog

    r222671 r222675  
     12017-09-29  Yusuke Suzuki  <utatane.tea@gmail.com>
     2
     3        [DFG] Support ArrayPush with multiple args
     4        https://bugs.webkit.org/show_bug.cgi?id=175823
     5
     6        Reviewed by Saam Barati.
     7
     8        Reviewed by Saam Barati.
     9
     10        This patch implements ArrayPush(with multiple arguments) in DFG and FTL. Previously, they are not handled
     11        by ArrayPush. Then they go to generic direct call to Array#push and it does in slow path. This patch
     12        extends ArrayPush to push multiple arguments in a bulk push manner.
     13
     14        The problem of ArrayPush is that we need to perform ArrayPush atomically: If OSR exit occurs in the middle
     15        of ArrayPush, we incorrectly push pushed elements twice. Once we start pushing values, we should not exit.
     16        But we do not want to iterate elements twice, once for type checks and once for actually pushing it. It
     17        could move elements between registers and memory back and forth.
     18
     19        This patch achieves the above goal by separating type checks from ArrayPush. When starting ArrayPush, type
     20        checks for elements are already done by separately emitted Check nodes.
     21
     22        We also add JSArray::pushInline for DFG operations just calling JSArray::push. And we also use it in
     23        arrayProtoFuncPush's fast path.
     24
     25        This patch significantly improves performance of `push(multiple args)`.
     26
     27                                            baseline                  patched
     28            Microbenchmarks:
     29                array-push-0            461.8455+-28.9995    ^    151.3438+-6.5653        ^ definitely 3.0516x faster
     30                array-push-1            133.8845+-7.0349     ?    136.1775+-5.8327        ? might be 1.0171x slower
     31                array-push-2            675.6555+-13.4645    ^    145.8747+-6.4621        ^ definitely 4.6318x faster
     32                array-push-3            849.5284+-15.2540    ^    253.4421+-9.1249        ^ definitely 3.3520x faster
     33
     34                                            baseline                  patched
     35            SixSpeed:
     36                spread-literal.es5       90.3482+-6.6514     ^     24.8123+-2.3304        ^ definitely 3.6413x faster
     37
     38        * dfg/DFGByteCodeParser.cpp:
     39        (JSC::DFG::ByteCodeParser::handleIntrinsicCall):
     40        * dfg/DFGFixupPhase.cpp:
     41        (JSC::DFG::FixupPhase::fixupNode):
     42        * dfg/DFGNodeType.h:
     43        * dfg/DFGOperations.cpp:
     44        * dfg/DFGOperations.h:
     45        * dfg/DFGSpeculativeJIT.cpp:
     46        (JSC::DFG::SpeculativeJIT::compileArrayPush):
     47        * dfg/DFGSpeculativeJIT.h:
     48        (JSC::DFG::SpeculativeJIT::callOperation):
     49        * dfg/DFGSpeculativeJIT32_64.cpp:
     50        (JSC::DFG::SpeculativeJIT::compile):
     51        * dfg/DFGSpeculativeJIT64.cpp:
     52        (JSC::DFG::SpeculativeJIT::compile):
     53        * dfg/DFGStoreBarrierInsertionPhase.cpp:
     54        * ftl/FTLLowerDFGToB3.cpp:
     55        (JSC::FTL::DFG::LowerDFGToB3::compileArrayPush):
     56        * jit/JITOperations.h:
     57        * runtime/ArrayPrototype.cpp:
     58        (JSC::arrayProtoFuncPush):
     59        * runtime/JSArray.cpp:
     60        (JSC::JSArray::push):
     61        * runtime/JSArray.h:
     62        * runtime/JSArrayInlines.h:
     63        (JSC::JSArray::pushInline):
     64
    1652017-09-29  Saam Barati  <sbarati@apple.com>
    266

  • trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

    r222658 r222675  
    23352335       
    23362336    case ArrayPushIntrinsic: {
    2337         if (argumentCountIncludingThis != 2)
     2337#if USE(JSVALUE32_64)
     2338        if (isX86() || isMIPS()) {
     2339            if (argumentCountIncludingThis > 2)
     2340                return false;
     2341        }
     2342#endif
     2343
     2344        if (static_cast<unsigned>(argumentCountIncludingThis) >= MIN_SPARSE_ARRAY_INDEX)
    23382345            return false;
    23392346       
    … …  
    23472354        case Array::ArrayStorage: {
    23482355            insertChecks();
    2349             Node* arrayPush = addToGraph(ArrayPush, OpInfo(arrayMode.asWord()), OpInfo(prediction), get(virtualRegisterForArgument(0, registerOffset)), get(virtualRegisterForArgument(1, registerOffset)));
     2356
     2357            addVarArgChild(nullptr); // For storage.
     2358            for (int i = 0; i < argumentCountIncludingThis; ++i)
     2359                addVarArgChild(get(virtualRegisterForArgument(i, registerOffset)));
     2360            Node* arrayPush = addToGraph(Node::VarArg, ArrayPush, OpInfo(arrayMode.asWord()), OpInfo(prediction));
    23502361            set(VirtualRegister(resultOperand), arrayPush);
    23512362           

  • trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp

    r222658 r222675  
    10091009            // array-related - so if refine() turned this into a "Generic" ArrayPush then
    10101010            // that would break things.
    1011             node->setArrayMode(
    1012                 node->arrayMode().refine(
    1013                     m_graph, node,
    1014                     node->child1()->prediction() & SpecCell,
    1015                     SpecInt32Only,
    1016                     node->child2()->prediction()));
    1017             blessArrayOperation(node->child1(), Edge(), node->child3());
    1018             fixEdge<KnownCellUse>(node->child1());
    1019            
    1020             switch (node->arrayMode().type()) {
    1021             case Array::Int32:
    1022                 fixEdge<Int32Use>(node->child2());
    1023                 break;
    1024             case Array::Double:
    1025                 fixEdge<DoubleRepRealUse>(node->child2());
    1026                 break;
    1027             case Array::Contiguous:
    1028             case Array::ArrayStorage:
    1029                 speculateForBarrier(node->child2());
    1030                 break;
    1031             default:
    1032                 break;
     1011            Edge& storageEdge = m_graph.varArgChild(node, 0);
     1012            Edge& arrayEdge = m_graph.varArgChild(node, 1);
     1013            unsigned elementOffset = 2;
     1014            unsigned elementCount = node->numChildren() - elementOffset;
     1015            for (unsigned i = 0; i < elementCount; ++i) {
     1016                Edge& element = m_graph.varArgChild(node, i + elementOffset);
     1017                node->setArrayMode(
     1018                    node->arrayMode().refine(
     1019                        m_graph, node,
     1020                        arrayEdge->prediction() & SpecCell,
     1021                        SpecInt32Only,
     1022                        element->prediction()));
     1023            }
     1024            blessArrayOperation(arrayEdge, Edge(), storageEdge);
     1025            fixEdge<KnownCellUse>(arrayEdge);
     1026
     1027            // Convert `array.push()` to GetArrayLength.
     1028            if (!elementCount && node->arrayMode().supportsSelfLength()) {
     1029                node->setOpAndDefaultFlags(GetArrayLength);
     1030                node->child1() = arrayEdge;
     1031                node->child2() = storageEdge;
     1032                fixEdge<KnownCellUse>(node->child1());
     1033                break;
     1034            }
     1035
     1036            // We do not want to perform osr exit and retry for ArrayPush. We insert Check with appropriate type,
     1037            // and ArrayPush uses the edge as known typed edge. Therefore, ArrayPush do not need to perform type checks.
     1038            for (unsigned i = 0; i < elementCount; ++i) {
     1039                Edge& element = m_graph.varArgChild(node, i + elementOffset);
     1040                switch (node->arrayMode().type()) {
     1041                case Array::Int32:
     1042                    insertCheck<Int32Use>(element.node());
     1043                    fixEdge<KnownInt32Use>(element);
     1044                    break;
     1045                case Array::Double:
     1046                    insertCheck<DoubleRepRealUse>(element.node());
     1047                    fixEdge<DoubleRepUse>(element);
     1048                    break;
     1049                case Array::Contiguous:
     1050                case Array::ArrayStorage:
     1051                    speculateForBarrier(element);
     1052                    break;
     1053                default:
     1054                    break;
     1055                }
     1056                ASSERT(shouldNotHaveTypeCheck(element.useKind()));
    10331057            }
    10341058            break;

  • trunk/Source/JavaScriptCore/dfg/DFGNodeType.h

    r222658 r222675  
    266266    \
    267267    /* Optimizations for array mutation. */\
    268     macro(ArrayPush, NodeResultJS | NodeMustGenerate) \
     268    macro(ArrayPush, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
    269269    macro(ArrayPop, NodeResultJS | NodeMustGenerate) \
    270270    macro(ArraySlice, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \

  • trunk/Source/JavaScriptCore/dfg/DFGOperations.cpp

    r222658 r222675  
    870870    NativeCallFrameTracer tracer(vm, exec);
    871871   
    872     array->push(exec, JSValue::decode(encodedValue));
     872    array->pushInline(exec, JSValue::decode(encodedValue));
    873873    return JSValue::encode(jsNumber(array->length()));
    874874}
    … …  
    879879    NativeCallFrameTracer tracer(vm, exec);
    880880   
    881     array->push(exec, JSValue(JSValue::EncodeAsDouble, value));
     881    array->pushInline(exec, JSValue(JSValue::EncodeAsDouble, value));
     882    return JSValue::encode(jsNumber(array->length()));
     883}
     884
     885EncodedJSValue JIT_OPERATION operationArrayPushMultiple(ExecState* exec, JSArray* array, void* buffer, int32_t elementCount)
     886{
     887    VM& vm = exec->vm();
     888    NativeCallFrameTracer tracer(&vm, exec);
     889    auto scope = DECLARE_THROW_SCOPE(vm);
     890
     891    // We assume that multiple JSArray::push calls with ArrayWithInt32/ArrayWithContiguous do not cause JS traps.
     892    // If it can cause any JS interactions, we can call the caller JS function of this function and overwrite the
     893    // content of ScratchBuffer. If the IndexingType is now ArrayWithInt32/ArrayWithContiguous, we can ensure
     894    // that there is no indexed accessors in this object and its prototype chain.
     895    //
     896    // ArrayWithArrayStorage is also OK. It can have indexed accessors. But if you define an indexed accessor, the array's length
     897    // becomes larger than that index. So Array#push never overlaps with this accessor. So accessors are never called unless
     898    // the IndexingType is ArrayWithSlowPutArrayStorage which could have an indexed accessor in a prototype chain.
     899    RELEASE_ASSERT(!shouldUseSlowPut(array->indexingType()));
     900
     901    EncodedJSValue* values = static_cast<EncodedJSValue*>(buffer);
     902    for (int32_t i = 0; i < elementCount; ++i) {
     903        array->pushInline(exec, JSValue::decode(values[i]));
     904        RETURN_IF_EXCEPTION(scope, encodedJSValue());
     905    }
     906    return JSValue::encode(jsNumber(array->length()));
     907}
     908
     909EncodedJSValue JIT_OPERATION operationArrayPushDoubleMultiple(ExecState* exec, JSArray* array, void* buffer, int32_t elementCount)
     910{
     911    VM& vm = exec->vm();
     912    NativeCallFrameTracer tracer(&vm, exec);
     913    auto scope = DECLARE_THROW_SCOPE(vm);
     914
     915    // We assume that multiple JSArray::push calls with ArrayWithDouble do not cause JS traps.
     916    // If it can cause any JS interactions, we can call the caller JS function of this function and overwrite the
     917    // content of ScratchBuffer. If the IndexingType is now ArrayWithDouble, we can ensure
     918    // that there is no indexed accessors in this object and its prototype chain.
     919    ASSERT(array->indexingType() == ArrayWithDouble);
     920
     921    double* values = static_cast<double*>(buffer);
     922    for (int32_t i = 0; i < elementCount; ++i) {
     923        array->pushInline(exec, JSValue(JSValue::EncodeAsDouble, values[i]));
     924        RETURN_IF_EXCEPTION(scope, encodedJSValue());
     925    }
    882926    return JSValue::encode(jsNumber(array->length()));
    883927}

  • trunk/Source/JavaScriptCore/dfg/DFGOperations.h

    r222658 r222675  
    137137void JIT_OPERATION operationDefineAccessorPropertySymbol(ExecState*, JSObject*, Symbol*, JSObject*, JSObject*, int32_t) WTF_INTERNAL;
    138138EncodedJSValue JIT_OPERATION operationArrayPush(ExecState*, EncodedJSValue encodedValue, JSArray*) WTF_INTERNAL;
     139EncodedJSValue JIT_OPERATION operationArrayPushMultiple(ExecState*, JSArray*, void* buffer, int32_t elementCount) WTF_INTERNAL;
    139140EncodedJSValue JIT_OPERATION operationArrayPushDouble(ExecState*, double value, JSArray*) WTF_INTERNAL;
     141EncodedJSValue JIT_OPERATION operationArrayPushDoubleMultiple(ExecState*, JSArray*, void* buffer, int32_t elementCount) WTF_INTERNAL;
    140142EncodedJSValue JIT_OPERATION operationArrayPop(ExecState*, JSArray*) WTF_INTERNAL;
    141143EncodedJSValue JIT_OPERATION operationArrayPopAndRecoverLength(ExecState*, JSArray*) WTF_INTERNAL;

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r222658 r222675  
    78727872}
    78737873
     7874void SpeculativeJIT::compileArrayPush(Node* node)
     7875{
     7876    ASSERT(node->arrayMode().isJSArray());
     7877
     7878    Edge& storageEdge = m_jit.graph().varArgChild(node, 0);
     7879    Edge& arrayEdge = m_jit.graph().varArgChild(node, 1);
     7880
     7881    SpeculateCellOperand base(this, arrayEdge);
     7882    GPRTemporary storageLength(this);
     7883
     7884    GPRReg baseGPR = base.gpr();
     7885    GPRReg storageLengthGPR = storageLength.gpr();
     7886
     7887    StorageOperand storage(this, storageEdge);
     7888    GPRReg storageGPR = storage.gpr();
     7889    unsigned elementOffset = 2;
     7890    unsigned elementCount = node->numChildren() - elementOffset;
     7891
     7892#if USE(JSVALUE32_64)
     7893    GPRTemporary tag(this);
     7894    GPRReg tagGPR = tag.gpr();
     7895    JSValueRegs resultRegs { tagGPR, storageLengthGPR };
     7896#else
     7897    JSValueRegs resultRegs { storageLengthGPR };
     7898#endif
     7899
     7900    auto getStorageBufferAddress = [&] (GPRReg storageGPR, GPRReg indexGPR, int32_t offset, GPRReg bufferGPR) {
     7901#if USE(JSVALUE32_64)
     7902        static_assert(sizeof(JSValue) == 8 && 1 << 3 == 8, "This is strongly assumed in the code below.");
     7903        m_jit.move(indexGPR, bufferGPR);
     7904        m_jit.lshift32(TrustedImm32(3), bufferGPR);
     7905        m_jit.add32(storageGPR, bufferGPR);
     7906        if (offset)
     7907            m_jit.add32(TrustedImm32(offset), bufferGPR);
     7908#else
     7909        m_jit.getEffectiveAddress64(MacroAssembler::BaseIndex(storageGPR, indexGPR, MacroAssembler::TimesEight, offset), bufferGPR);
     7910#endif
     7911    };
     7912
     7913    switch (node->arrayMode().type()) {
     7914    case Array::Int32:
     7915    case Array::Contiguous: {
     7916        if (elementCount == 1) {
     7917            Edge& element = m_jit.graph().varArgChild(node, elementOffset);
     7918            JSValueOperand value(this, element, ManualOperandSpeculation);
     7919            JSValueRegs valueRegs = value.jsValueRegs();
     7920
     7921            if (node->arrayMode().type() == Array::Int32)
     7922                RELEASE_ASSERT(!needsTypeCheck(element, SpecInt32Only));
     7923
     7924            m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
     7925            MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
     7926            m_jit.storeValue(valueRegs, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight));
     7927            m_jit.add32(TrustedImm32(1), storageLengthGPR);
     7928            m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
     7929            m_jit.boxInt32(storageLengthGPR, resultRegs);
     7930
     7931            addSlowPathGenerator(
     7932                slowPathCall(slowPath, this, operationArrayPush, resultRegs, valueRegs, baseGPR));
     7933
     7934            jsValueResult(resultRegs, node);
     7935            return;
     7936        }
     7937
     7938        GPRTemporary buffer(this);
     7939        GPRReg bufferGPR = buffer.gpr();
     7940
     7941        m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
     7942        m_jit.move(storageLengthGPR, bufferGPR);
     7943        m_jit.add32(TrustedImm32(elementCount), bufferGPR);
     7944        MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::Above, bufferGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
     7945
     7946        m_jit.store32(bufferGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
     7947        getStorageBufferAddress(storageGPR, storageLengthGPR, 0, bufferGPR);
     7948        m_jit.add32(TrustedImm32(elementCount), storageLengthGPR);
     7949        m_jit.boxInt32(storageLengthGPR, resultRegs);
     7950        auto storageDone = m_jit.jump();
     7951
     7952        slowPath.link(&m_jit);
     7953
     7954        size_t scratchSize = sizeof(EncodedJSValue) * elementCount;
     7955        ScratchBuffer* scratchBuffer = m_jit.vm()->scratchBufferForSize(scratchSize);
     7956        m_jit.move(TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())), bufferGPR);
     7957        m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), storageLengthGPR);
     7958        m_jit.storePtr(TrustedImmPtr(scratchSize), MacroAssembler::Address(storageLengthGPR));
     7959
     7960        storageDone.link(&m_jit);
     7961        for (unsigned elementIndex = 0; elementIndex < elementCount; ++elementIndex) {
     7962            Edge& element = m_jit.graph().varArgChild(node, elementIndex + elementOffset);
     7963            JSValueOperand value(this, element, ManualOperandSpeculation);
     7964            JSValueRegs valueRegs = value.jsValueRegs();
     7965
     7966            if (node->arrayMode().type() == Array::Int32)
     7967                RELEASE_ASSERT(!needsTypeCheck(element, SpecInt32Only));
     7968
     7969            m_jit.storeValue(valueRegs, MacroAssembler::Address(bufferGPR, sizeof(EncodedJSValue) * elementIndex));
     7970            value.use();
     7971        }
     7972
     7973        MacroAssembler::Jump fastPath = m_jit.branchPtr(MacroAssembler::NotEqual, bufferGPR, TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())));
     7974
     7975        addSlowPathGenerator(slowPathCall(m_jit.jump(), this, operationArrayPushMultiple, resultRegs, baseGPR, bufferGPR, TrustedImm32(elementCount)));
     7976
     7977        m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), bufferGPR);
     7978        m_jit.storePtr(TrustedImmPtr(nullptr), MacroAssembler::Address(bufferGPR));
     7979
     7980        base.use();
     7981        storage.use();
     7982
     7983        fastPath.link(&m_jit);
     7984        jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
     7985        return;
     7986    }
     7987
     7988    case Array::Double: {
     7989        if (elementCount == 1) {
     7990            Edge& element = m_jit.graph().varArgChild(node, elementOffset);
     7991            SpeculateDoubleOperand value(this, element);
     7992            FPRReg valueFPR = value.fpr();
     7993
     7994            RELEASE_ASSERT(!needsTypeCheck(element, SpecDoubleReal));
     7995
     7996            m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
     7997            MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
     7998            m_jit.storeDouble(valueFPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight));
     7999            m_jit.add32(TrustedImm32(1), storageLengthGPR);
     8000            m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
     8001            m_jit.boxInt32(storageLengthGPR, resultRegs);
     8002
     8003            addSlowPathGenerator(
     8004                slowPathCall(slowPath, this, operationArrayPushDouble, resultRegs, valueFPR, baseGPR));
     8005
     8006            jsValueResult(resultRegs, node);
     8007            return;
     8008        }
     8009
     8010        GPRTemporary buffer(this);
     8011        GPRReg bufferGPR = buffer.gpr();
     8012
     8013        m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
     8014        m_jit.move(storageLengthGPR, bufferGPR);
     8015        m_jit.add32(TrustedImm32(elementCount), bufferGPR);
     8016        MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::Above, bufferGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
     8017
     8018        m_jit.store32(bufferGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
     8019        getStorageBufferAddress(storageGPR, storageLengthGPR, 0, bufferGPR);
     8020        m_jit.add32(TrustedImm32(elementCount), storageLengthGPR);
     8021        m_jit.boxInt32(storageLengthGPR, resultRegs);
     8022        auto storageDone = m_jit.jump();
     8023
     8024        slowPath.link(&m_jit);
     8025
     8026        size_t scratchSize = sizeof(double) * elementCount;
     8027        ScratchBuffer* scratchBuffer = m_jit.vm()->scratchBufferForSize(scratchSize);
     8028        m_jit.move(TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())), bufferGPR);
     8029        m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), storageLengthGPR);
     8030        m_jit.storePtr(TrustedImmPtr(scratchSize), MacroAssembler::Address(storageLengthGPR));
     8031
     8032        storageDone.link(&m_jit);
     8033        for (unsigned elementIndex = 0; elementIndex < elementCount; ++elementIndex) {
     8034            Edge& element = m_jit.graph().varArgChild(node, elementIndex + elementOffset);
     8035            SpeculateDoubleOperand value(this, element);
     8036            FPRReg valueFPR = value.fpr();
     8037
     8038            RELEASE_ASSERT(!needsTypeCheck(element, SpecDoubleReal));
     8039
     8040            m_jit.storeDouble(valueFPR, MacroAssembler::Address(bufferGPR, sizeof(double) * elementIndex));
     8041            value.use();
     8042        }
     8043
     8044        MacroAssembler::Jump fastPath = m_jit.branchPtr(MacroAssembler::NotEqual, bufferGPR, TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())));
     8045
     8046        addSlowPathGenerator(slowPathCall(m_jit.jump(), this, operationArrayPushDoubleMultiple, resultRegs, baseGPR, bufferGPR, TrustedImm32(elementCount)));
     8047
     8048        m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), bufferGPR);
     8049        m_jit.storePtr(TrustedImmPtr(nullptr), MacroAssembler::Address(bufferGPR));
     8050
     8051        base.use();
     8052        storage.use();
     8053
     8054        fastPath.link(&m_jit);
     8055        jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
     8056        return;
     8057    }
     8058
     8059    case Array::ArrayStorage: {
     8060        // This ensures that the result of ArrayPush is Int32 in AI.
     8061        int32_t largestPositiveInt32Length = 0x7fffffff - elementCount;
     8062        if (elementCount == 1) {
     8063            Edge& element = m_jit.graph().varArgChild(node, elementOffset);
     8064            JSValueOperand value(this, element);
     8065            JSValueRegs valueRegs = value.jsValueRegs();
     8066
     8067            m_jit.load32(MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()), storageLengthGPR);
     8068
     8069            // Refuse to handle bizarre lengths.
     8070            speculationCheck(Uncountable, JSValueRegs(), 0, m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(largestPositiveInt32Length)));
     8071
     8072            MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, ArrayStorage::vectorLengthOffset()));
     8073
     8074            m_jit.storeValue(valueRegs, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, ArrayStorage::vectorOffset()));
     8075
     8076            m_jit.add32(TrustedImm32(1), storageLengthGPR);
     8077            m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()));
     8078            m_jit.add32(TrustedImm32(1), MacroAssembler::Address(storageGPR, OBJECT_OFFSETOF(ArrayStorage, m_numValuesInVector)));
     8079            m_jit.boxInt32(storageLengthGPR, resultRegs);
     8080
     8081            addSlowPathGenerator(
     8082                slowPathCall(slowPath, this, operationArrayPush, resultRegs, valueRegs, baseGPR));
     8083
     8084            jsValueResult(resultRegs, node);
     8085            return;
     8086        }
     8087
     8088        GPRTemporary buffer(this);
     8089        GPRReg bufferGPR = buffer.gpr();
     8090
     8091        m_jit.load32(MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()), storageLengthGPR);
     8092
     8093        // Refuse to handle bizarre lengths.
     8094        speculationCheck(Uncountable, JSValueRegs(), 0, m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(largestPositiveInt32Length)));
     8095
     8096        m_jit.move(storageLengthGPR, bufferGPR);
     8097        m_jit.add32(TrustedImm32(elementCount), bufferGPR);
     8098        MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::Above, bufferGPR, MacroAssembler::Address(storageGPR, ArrayStorage::vectorLengthOffset()));
     8099
     8100        m_jit.store32(bufferGPR, MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()));
     8101        getStorageBufferAddress(storageGPR, storageLengthGPR, ArrayStorage::vectorOffset(), bufferGPR);
     8102        m_jit.add32(TrustedImm32(elementCount), MacroAssembler::Address(storageGPR, OBJECT_OFFSETOF(ArrayStorage, m_numValuesInVector)));
     8103        m_jit.add32(TrustedImm32(elementCount), storageLengthGPR);
     8104        m_jit.boxInt32(storageLengthGPR, resultRegs);
     8105        auto storageDone = m_jit.jump();
     8106
     8107        slowPath.link(&m_jit);
     8108
     8109        size_t scratchSize = sizeof(EncodedJSValue) * elementCount;
     8110        ScratchBuffer* scratchBuffer = m_jit.vm()->scratchBufferForSize(scratchSize);
     8111        m_jit.move(TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())), bufferGPR);
     8112        m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), storageLengthGPR);
     8113        m_jit.storePtr(TrustedImmPtr(scratchSize), MacroAssembler::Address(storageLengthGPR));
     8114
     8115        storageDone.link(&m_jit);
     8116        for (unsigned elementIndex = 0; elementIndex < elementCount; ++elementIndex) {
     8117            Edge& element = m_jit.graph().varArgChild(node, elementIndex + elementOffset);
     8118            JSValueOperand value(this, element);
     8119            JSValueRegs valueRegs = value.jsValueRegs();
     8120
     8121            m_jit.storeValue(valueRegs, MacroAssembler::Address(bufferGPR, sizeof(EncodedJSValue) * elementIndex));
     8122            value.use();
     8123        }
     8124
     8125        MacroAssembler::Jump fastPath = m_jit.branchPtr(MacroAssembler::NotEqual, bufferGPR, TrustedImmPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())));
     8126
     8127        addSlowPathGenerator(
     8128            slowPathCall(m_jit.jump(), this, operationArrayPushMultiple, resultRegs, baseGPR, bufferGPR, TrustedImm32(elementCount)));
     8129
     8130        m_jit.move(TrustedImmPtr(scratchBuffer->addressOfActiveLength()), bufferGPR);
     8131        m_jit.storePtr(TrustedImmPtr(nullptr), MacroAssembler::Address(bufferGPR));
     8132
     8133        base.use();
     8134        storage.use();
     8135
     8136        fastPath.link(&m_jit);
     8137        jsValueResult(resultRegs, node, DataFormatJS, UseChildrenCalledExplicitly);
     8138        return;
     8139    }
     8140
     8141    default:
     8142        RELEASE_ASSERT_NOT_REACHED();
     8143    }
     8144}
     8145
    78748146void SpeculativeJIT::compileNotifyWrite(Node* node)
    78758147{

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

    r222658 r222675  
    16141614        return appendCallSetResult(operation, result);
    16151615    }
    1616     JITCompiler::Call callOperation(J_JITOperation_EJA operation, GPRReg result, GPRReg arg1, GPRReg arg2)
     1616    JITCompiler::Call callOperation(J_JITOperation_EJA operation, JSValueRegs result, JSValueRegs arg1, GPRReg arg2)
     1617    {
     1618        m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2);
     1619        return appendCallSetResult(operation, result.payloadGPR());
     1620    }
     1621    JITCompiler::Call callOperation(J_JITOperation_EJA operation, GPRReg result, JSValueRegs arg1, GPRReg arg2)
     1622    {
     1623        m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2);
     1624        return appendCallSetResult(operation, result);
     1625    }
     1626    JITCompiler::Call callOperation(J_JITOperation_EP operation, GPRReg result, GPRReg arg1)
     1627    {
     1628        m_jit.setupArgumentsWithExecState(arg1);
     1629        return appendCallSetResult(operation, result);
     1630    }
     1631    JITCompiler::Call callOperation(J_JITOperation_EZ operation, GPRReg result, GPRReg arg1)
     1632    {
     1633        m_jit.setupArgumentsWithExecState(arg1);
     1634        return appendCallSetResult(operation, result);
     1635    }
     1636    JITCompiler::Call callOperation(J_JITOperation_EZ operation, GPRReg result, int32_t arg1)
     1637    {
     1638        m_jit.setupArgumentsWithExecState(TrustedImm32(arg1));
     1639        return appendCallSetResult(operation, result);
     1640    }
     1641    JITCompiler::Call callOperation(J_JITOperation_EZZ operation, GPRReg result, int32_t arg1, GPRReg arg2)
     1642    {
     1643        m_jit.setupArgumentsWithExecState(TrustedImm32(arg1), arg2);
     1644        return appendCallSetResult(operation, result);
     1645    }
     1646    JITCompiler::Call callOperation(P_JITOperation_EQZ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    16171647    {
    16181648        m_jit.setupArgumentsWithExecState(arg1, arg2);
    16191649        return appendCallSetResult(operation, result);
    16201650    }
    1621     JITCompiler::Call callOperation(J_JITOperation_EP operation, GPRReg result, GPRReg arg1)
     1651    JITCompiler::Call callOperation(P_JITOperation_EQZ operation, GPRReg result, GPRReg arg1, TrustedImm32 arg2)
     1652    {
     1653        m_jit.setupArgumentsWithExecState(arg1, arg2);
     1654        return appendCallSetResult(operation, result);
     1655    }
     1656
     1657    JITCompiler::Call callOperation(J_JITOperation_EZIcfZ operation, GPRReg result, int32_t arg1, InlineCallFrame* inlineCallFrame, GPRReg arg2)
     1658    {
     1659        m_jit.setupArgumentsWithExecState(TrustedImm32(arg1), TrustedImmPtr(inlineCallFrame), arg2);
     1660        return appendCallSetResult(operation, result);
     1661    }
     1662
     1663    JITCompiler::Call callOperation(P_JITOperation_EJS operation, GPRReg result, GPRReg value, size_t index)
     1664    {
     1665        m_jit.setupArgumentsWithExecState(value, TrustedImmPtr(index));
     1666        return appendCallSetResult(operation, result);
     1667    }
     1668
     1669    JITCompiler::Call callOperation(P_JITOperation_EStJ operation, GPRReg result, RegisteredStructure structure, GPRReg arg2)
     1670    {
     1671        m_jit.setupArgumentsWithExecState(TrustedImmPtr(structure), arg2);
     1672        return appendCallSetResult(operation, result);
     1673    }
     1674
     1675    JITCompiler::Call callOperation(C_JITOperation_EGJ operation, GPRReg result, JSGlobalObject* globalObject, GPRReg arg1)
     1676    {
     1677        m_jit.setupArgumentsWithExecState(TrustedImmPtr::weakPointer(m_jit.graph(), globalObject), arg1);
     1678        return appendCallSetResult(operation, result);
     1679    }
     1680
     1681    JITCompiler::Call callOperation(C_JITOperation_EGJ operation, GPRReg result, JSGlobalObject* globalObject, JSValueRegs arg1)
     1682    {
     1683        return callOperation(operation, result, globalObject, arg1.gpr());
     1684    }
     1685
     1686    JITCompiler::Call callOperation(C_JITOperation_EJ operation, GPRReg result, GPRReg arg1)
    16221687    {
    16231688        m_jit.setupArgumentsWithExecState(arg1);
    16241689        return appendCallSetResult(operation, result);
    16251690    }
    1626     JITCompiler::Call callOperation(J_JITOperation_EZ operation, GPRReg result, GPRReg arg1)
     1691    JITCompiler::Call callOperation(C_JITOperation_EJ operation, GPRReg result, JSValueRegs arg1)
     1692    {
     1693        m_jit.setupArgumentsWithExecState(arg1.gpr());
     1694        return appendCallSetResult(operation, result);
     1695    }
     1696    JITCompiler::Call callOperation(C_JITOperation_EJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
     1697    {
     1698        m_jit.setupArgumentsWithExecState(arg1, arg2);
     1699        return appendCallSetResult(operation, result);
     1700    }
     1701    JITCompiler::Call callOperation(C_JITOperation_EJJC operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1702    {
     1703        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
     1704        return appendCallSetResult(operation, result);
     1705    }
     1706    JITCompiler::Call callOperation(C_JITOperation_EJJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1707    {
     1708        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
     1709        return appendCallSetResult(operation, result);
     1710    }
     1711    JITCompiler::Call callOperation(C_JITOperation_EJZ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
     1712    {
     1713        m_jit.setupArgumentsWithExecState(arg1, arg2);
     1714        return appendCallSetResult(operation, result);
     1715    }
     1716    JITCompiler::Call callOperation(C_JITOperation_EJZC operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1717    {
     1718        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
     1719        return appendCallSetResult(operation, result);
     1720    }
     1721    JITCompiler::Call callOperation(S_JITOperation_J operation, GPRReg result, GPRReg arg1)
     1722    {
     1723        m_jit.setupArguments(arg1);
     1724        return appendCallSetResult(operation, result);
     1725    }
     1726    JITCompiler::Call callOperation(S_JITOperation_EJ operation, GPRReg result, GPRReg arg1)
    16271727    {
    16281728        m_jit.setupArgumentsWithExecState(arg1);
    16291729        return appendCallSetResult(operation, result);
    16301730    }
    1631     JITCompiler::Call callOperation(J_JITOperation_EZ operation, GPRReg result, int32_t arg1)
    1632     {
    1633         m_jit.setupArgumentsWithExecState(TrustedImm32(arg1));
    1634         return appendCallSetResult(operation, result);
    1635     }
    1636     JITCompiler::Call callOperation(J_JITOperation_EZZ operation, GPRReg result, int32_t arg1, GPRReg arg2)
    1637     {
    1638         m_jit.setupArgumentsWithExecState(TrustedImm32(arg1), arg2);
    1639         return appendCallSetResult(operation, result);
    1640     }
    1641     JITCompiler::Call callOperation(P_JITOperation_EQZ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
     1731    JITCompiler::Call callOperation(S_JITOperation_EJ operation, GPRReg result, JSValueRegs arg1)
     1732    {
     1733        return callOperation(operation, result, arg1.gpr());
     1734    }
     1735    JITCompiler::Call callOperation(J_JITOperation_EJ operation, JSValueRegs result, JSValueRegs arg1)
     1736    {
     1737        return callOperation(operation, result.payloadGPR(), arg1.payloadGPR());
     1738    }
     1739    JITCompiler::Call callOperation(J_JITOperation_EJ operation, GPRReg result, JSValueRegs arg1)
     1740    {
     1741        return callOperation(operation, result, arg1.payloadGPR());
     1742    }
     1743    JITCompiler::Call callOperation(J_JITOperation_EJ operation, GPRReg result, GPRReg arg1)
     1744    {
     1745        m_jit.setupArgumentsWithExecState(arg1);
     1746        return appendCallSetResult(operation, result);
     1747    }
     1748    JITCompiler::Call callOperation(S_JITOperation_EJI operation, GPRReg result, GPRReg arg1, UniquedStringImpl* uid)
     1749    {
     1750        m_jit.setupArgumentsWithExecState(arg1, TrustedImmPtr(uid));
     1751        return appendCallSetResult(operation, result);
     1752    }
     1753    JITCompiler::Call callOperation(S_JITOperation_EJI operation, GPRReg result, JSValueRegs arg1, UniquedStringImpl* uid)
     1754    {
     1755        return callOperation(operation, result, arg1.gpr(), uid);
     1756    }
     1757    JITCompiler::Call callOperation(S_JITOperation_EJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    16421758    {
    16431759        m_jit.setupArgumentsWithExecState(arg1, arg2);
    16441760        return appendCallSetResult(operation, result);
    16451761    }
    1646     JITCompiler::Call callOperation(P_JITOperation_EQZ operation, GPRReg result, GPRReg arg1, TrustedImm32 arg2)
     1762    JITCompiler::Call callOperation(S_JITOperation_EJJ operation, GPRReg result,  JSValueRegs arg1, JSValueRegs arg2)
     1763    {
     1764        return callOperation(operation, result, arg1.gpr(), arg2.gpr());
     1765    }
     1766    JITCompiler::Call callOperation(S_JITOperation_EGJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1767    {
     1768        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
     1769        return appendCallSetResult(operation, result);
     1770    }
     1771    JITCompiler::Call callOperation(S_JITOperation_EGReoJ operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1772    {
     1773        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
     1774        return appendCallSetResult(operation, result);
     1775    }
     1776    JITCompiler::Call callOperation(S_JITOperation_EGReoJss operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1777    {
     1778        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
     1779        return appendCallSetResult(operation, result);
     1780    }
     1781
     1782    JITCompiler::Call callOperation(J_JITOperation_EPP operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    16471783    {
    16481784        m_jit.setupArgumentsWithExecState(arg1, arg2);
    16491785        return appendCallSetResult(operation, result);
    16501786    }
    1651 
    1652     JITCompiler::Call callOperation(J_JITOperation_EZIcfZ operation, GPRReg result, int32_t arg1, InlineCallFrame* inlineCallFrame, GPRReg arg2)
    1653     {
    1654         m_jit.setupArgumentsWithExecState(TrustedImm32(arg1), TrustedImmPtr(inlineCallFrame), arg2);
    1655         return appendCallSetResult(operation, result);
    1656     }
    1657 
    1658     JITCompiler::Call callOperation(P_JITOperation_EJS operation, GPRReg result, GPRReg value, size_t index)
    1659     {
    1660         m_jit.setupArgumentsWithExecState(value, TrustedImmPtr(index));
    1661         return appendCallSetResult(operation, result);
    1662     }
    1663 
    1664     JITCompiler::Call callOperation(P_JITOperation_EStJ operation, GPRReg result, RegisteredStructure structure, GPRReg arg2)
    1665     {
    1666         m_jit.setupArgumentsWithExecState(TrustedImmPtr(structure), arg2);
    1667         return appendCallSetResult(operation, result);
    1668     }
    1669 
    1670     JITCompiler::Call callOperation(C_JITOperation_EGJ operation, GPRReg result, JSGlobalObject* globalObject, GPRReg arg1)
    1671     {
    1672         m_jit.setupArgumentsWithExecState(TrustedImmPtr::weakPointer(m_jit.graph(), globalObject), arg1);
    1673         return appendCallSetResult(operation, result);
    1674     }
    1675 
    1676     JITCompiler::Call callOperation(C_JITOperation_EGJ operation, GPRReg result, JSGlobalObject* globalObject, JSValueRegs arg1)
    1677     {
    1678         return callOperation(operation, result, globalObject, arg1.gpr());
    1679     }
    1680 
    1681     JITCompiler::Call callOperation(C_JITOperation_EJ operation, GPRReg result, GPRReg arg1)
    1682     {
    1683         m_jit.setupArgumentsWithExecState(arg1);
    1684         return appendCallSetResult(operation, result);
    1685     }
    1686     JITCompiler::Call callOperation(C_JITOperation_EJ operation, GPRReg result, JSValueRegs arg1)
    1687     {
    1688         m_jit.setupArgumentsWithExecState(arg1.gpr());
    1689         return appendCallSetResult(operation, result);
    1690     }
    1691     JITCompiler::Call callOperation(C_JITOperation_EJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
     1787    JITCompiler::Call callOperation(J_JITOperation_EPPP operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1788    {
     1789        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
     1790        return appendCallSetResult(operation, result);
     1791    }
     1792    JITCompiler::Call callOperation(J_JITOperation_EGP operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    16921793    {
    16931794        m_jit.setupArgumentsWithExecState(arg1, arg2);
    16941795        return appendCallSetResult(operation, result);
    16951796    }
    1696     JITCompiler::Call callOperation(C_JITOperation_EJJC operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1797    JITCompiler::Call callOperation(J_JITOperation_EAPZ operation, JSValueRegs result, GPRReg arg1, GPRReg arg2, TrustedImm32 arg3)
    16971798    {
    16981799        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    1699         return appendCallSetResult(operation, result);
    1700     }
    1701     JITCompiler::Call callOperation(C_JITOperation_EJJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
     1800        return appendCallSetResult(operation, result.payloadGPR());
     1801    }
     1802    JITCompiler::Call callOperation(J_JITOperation_EAPZ operation, GPRReg result, GPRReg arg1, GPRReg arg2, TrustedImm32 arg3)
    17021803    {
    17031804        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    1704         return appendCallSetResult(operation, result);
    1705     }
    1706     JITCompiler::Call callOperation(C_JITOperation_EJZ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    1707     {
    1708         m_jit.setupArgumentsWithExecState(arg1, arg2);
    1709         return appendCallSetResult(operation, result);
    1710     }
    1711     JITCompiler::Call callOperation(C_JITOperation_EJZC operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
    1712     {
    1713         m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    1714         return appendCallSetResult(operation, result);
    1715     }
    1716     JITCompiler::Call callOperation(S_JITOperation_J operation, GPRReg result, GPRReg arg1)
    1717     {
    1718         m_jit.setupArguments(arg1);
    1719         return appendCallSetResult(operation, result);
    1720     }
    1721     JITCompiler::Call callOperation(S_JITOperation_EJ operation, GPRReg result, GPRReg arg1)
    1722     {
    1723         m_jit.setupArgumentsWithExecState(arg1);
    1724         return appendCallSetResult(operation, result);
    1725     }
    1726     JITCompiler::Call callOperation(S_JITOperation_EJ operation, GPRReg result, JSValueRegs arg1)
    1727     {
    1728         return callOperation(operation, result, arg1.gpr());
    1729     }
    1730     JITCompiler::Call callOperation(J_JITOperation_EJ operation, JSValueRegs result, JSValueRegs arg1)
    1731     {
    1732         return callOperation(operation, result.payloadGPR(), arg1.payloadGPR());
    1733     }
    1734     JITCompiler::Call callOperation(J_JITOperation_EJ operation, GPRReg result, JSValueRegs arg1)
    1735     {
    1736         return callOperation(operation, result, arg1.payloadGPR());
    1737     }
    1738     JITCompiler::Call callOperation(J_JITOperation_EJ operation, GPRReg result, GPRReg arg1)
    1739     {
    1740         m_jit.setupArgumentsWithExecState(arg1);
    1741         return appendCallSetResult(operation, result);
    1742     }
    1743     JITCompiler::Call callOperation(S_JITOperation_EJI operation, GPRReg result, GPRReg arg1, UniquedStringImpl* uid)
    1744     {
    1745         m_jit.setupArgumentsWithExecState(arg1, TrustedImmPtr(uid));
    1746         return appendCallSetResult(operation, result);
    1747     }
    1748     JITCompiler::Call callOperation(S_JITOperation_EJI operation, GPRReg result, JSValueRegs arg1, UniquedStringImpl* uid)
    1749     {
    1750         return callOperation(operation, result, arg1.gpr(), uid);
    1751     }
    1752     JITCompiler::Call callOperation(S_JITOperation_EJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    1753     {
    1754         m_jit.setupArgumentsWithExecState(arg1, arg2);
    1755         return appendCallSetResult(operation, result);
    1756     }
    1757     JITCompiler::Call callOperation(S_JITOperation_EJJ operation, GPRReg result,  JSValueRegs arg1, JSValueRegs arg2)
    1758     {
    1759         return callOperation(operation, result, arg1.gpr(), arg2.gpr());
    1760     }
    1761     JITCompiler::Call callOperation(S_JITOperation_EGJJ operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
    1762     {
    1763         m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    1764         return appendCallSetResult(operation, result);
    1765     }
    1766     JITCompiler::Call callOperation(S_JITOperation_EGReoJ operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
    1767     {
    1768         m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    1769         return appendCallSetResult(operation, result);
    1770     }
    1771     JITCompiler::Call callOperation(S_JITOperation_EGReoJss operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
    1772     {
    1773         m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    1774         return appendCallSetResult(operation, result);
    1775     }
    1776 
    1777     JITCompiler::Call callOperation(J_JITOperation_EPP operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    1778     {
    1779         m_jit.setupArgumentsWithExecState(arg1, arg2);
    1780         return appendCallSetResult(operation, result);
    1781     }
    1782     JITCompiler::Call callOperation(J_JITOperation_EPPP operation, GPRReg result, GPRReg arg1, GPRReg arg2, GPRReg arg3)
    1783     {
    1784         m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    1785         return appendCallSetResult(operation, result);
    1786     }
    1787     JITCompiler::Call callOperation(J_JITOperation_EGP operation, GPRReg result, GPRReg arg1, GPRReg arg2)
    1788     {
    1789         m_jit.setupArgumentsWithExecState(arg1, arg2);
    17901805        return appendCallSetResult(operation, result);
    17911806    }
    … …  
    21132128    {
    21142129        m_jit.setupArgumentsWithExecState(arg1, arg2);
     2130        return appendCallSetResult(operation, result.payloadGPR(), result.tagGPR());
     2131    }
     2132    JITCompiler::Call callOperation(J_JITOperation_EAPZ operation, JSValueRegs result, GPRReg arg1, GPRReg arg2, TrustedImm32 arg3)
     2133    {
     2134        m_jit.setupArgumentsWithExecState(arg1, arg2, arg3);
    21152135        return appendCallSetResult(operation, result.payloadGPR(), result.tagGPR());
    21162136    }
    … …  
    29292949    void compileArraySlice(Node*);
    29302950    void compileArrayIndexOf(Node*);
     2951    void compileArrayPush(Node*);
    29312952    void compileNotifyWrite(Node*);
    29322953    bool compileRegExpExec(Node*);

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp

    r222658 r222675  
    33853385       
    33863386    case ArrayPush: {
    3387         ASSERT(node->arrayMode().isJSArray());
    3388        
    3389         SpeculateCellOperand base(this, node->child1());
    3390         GPRTemporary storageLength(this);
    3391        
    3392         GPRReg baseGPR = base.gpr();
    3393         GPRReg storageLengthGPR = storageLength.gpr();
    3394        
    3395         StorageOperand storage(this, node->child3());
    3396         GPRReg storageGPR = storage.gpr();
    3397        
    3398         switch (node->arrayMode().type()) {
    3399         case Array::Int32: {
    3400             SpeculateInt32Operand value(this, node->child2());
    3401             GPRReg valuePayloadGPR = value.gpr();
    3402            
    3403             m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
    3404             MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
    3405             m_jit.store32(TrustedImm32(JSValue::Int32Tag), MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.tag)));
    3406             m_jit.store32(valuePayloadGPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.payload)));
    3407             m_jit.add32(TrustedImm32(1), storageLengthGPR);
    3408             m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
    3409             m_jit.move(TrustedImm32(JSValue::Int32Tag), storageGPR);
    3410            
    3411             addSlowPathGenerator(
    3412                 slowPathCall(
    3413                     slowPath, this, operationArrayPush,
    3414                     JSValueRegs(storageGPR, storageLengthGPR),
    3415                     TrustedImm32(JSValue::Int32Tag), valuePayloadGPR, baseGPR));
    3416        
    3417             jsValueResult(storageGPR, storageLengthGPR, node);
    3418             break;
    3419         }
    3420            
    3421         case Array::Contiguous: {
    3422             JSValueOperand value(this, node->child2());
    3423             GPRReg valueTagGPR = value.tagGPR();
    3424             GPRReg valuePayloadGPR = value.payloadGPR();
    3425 
    3426             m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
    3427             MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
    3428             m_jit.store32(valueTagGPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.tag)));
    3429             m_jit.store32(valuePayloadGPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.payload)));
    3430             m_jit.add32(TrustedImm32(1), storageLengthGPR);
    3431             m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
    3432             m_jit.move(TrustedImm32(JSValue::Int32Tag), storageGPR);
    3433            
    3434             addSlowPathGenerator(
    3435                 slowPathCall(
    3436                     slowPath, this, operationArrayPush,
    3437                     JSValueRegs(storageGPR, storageLengthGPR),
    3438                     JSValueRegs(valueTagGPR, valuePayloadGPR), baseGPR));
    3439        
    3440             jsValueResult(storageGPR, storageLengthGPR, node);
    3441             break;
    3442         }
    3443            
    3444         case Array::Double: {
    3445             SpeculateDoubleOperand value(this, node->child2());
    3446             FPRReg valueFPR = value.fpr();
    3447 
    3448             DFG_TYPE_CHECK(
    3449                 JSValueRegs(), node->child2(), SpecDoubleReal,
    3450                 m_jit.branchDouble(MacroAssembler::DoubleNotEqualOrUnordered, valueFPR, valueFPR));
    3451            
    3452             m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
    3453             MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
    3454             m_jit.storeDouble(valueFPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight));
    3455             m_jit.add32(TrustedImm32(1), storageLengthGPR);
    3456             m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
    3457             m_jit.move(TrustedImm32(JSValue::Int32Tag), storageGPR);
    3458            
    3459             addSlowPathGenerator(
    3460                 slowPathCall(
    3461                     slowPath, this, operationArrayPushDouble,
    3462                     JSValueRegs(storageGPR, storageLengthGPR),
    3463                     valueFPR, baseGPR));
    3464        
    3465             jsValueResult(storageGPR, storageLengthGPR, node);
    3466             break;
    3467         }
    3468            
    3469         case Array::ArrayStorage: {
    3470             JSValueOperand value(this, node->child2());
    3471             GPRReg valueTagGPR = value.tagGPR();
    3472             GPRReg valuePayloadGPR = value.payloadGPR();
    3473 
    3474             m_jit.load32(MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()), storageLengthGPR);
    3475        
    3476             // Refuse to handle bizarre lengths.
    3477             speculationCheck(Uncountable, JSValueRegs(), 0, m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(0x7ffffffe)));
    3478        
    3479             MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, ArrayStorage::vectorLengthOffset()));
    3480        
    3481             m_jit.store32(valueTagGPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, ArrayStorage::vectorOffset() + OBJECT_OFFSETOF(JSValue, u.asBits.tag)));
    3482             m_jit.store32(valuePayloadGPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, ArrayStorage::vectorOffset() + OBJECT_OFFSETOF(JSValue, u.asBits.payload)));
    3483        
    3484             m_jit.add32(TrustedImm32(1), storageLengthGPR);
    3485             m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()));
    3486             m_jit.add32(TrustedImm32(1), MacroAssembler::Address(storageGPR, OBJECT_OFFSETOF(ArrayStorage, m_numValuesInVector)));
    3487             m_jit.move(TrustedImm32(JSValue::Int32Tag), storageGPR);
    3488        
    3489             addSlowPathGenerator(slowPathCall(slowPath, this, operationArrayPush, JSValueRegs(storageGPR, storageLengthGPR),
    3490                 JSValueRegs(valueTagGPR, valuePayloadGPR), baseGPR));
    3491        
    3492             jsValueResult(storageGPR, storageLengthGPR, node);
    3493             break;
    3494         }
    3495            
    3496         default:
    3497             CRASH();
    3498             break;
    3499         }
     3387        compileArrayPush(node);
    35003388        break;
    35013389    }

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp

    r222658 r222675  
    36643664       
    36653665    case ArrayPush: {
    3666         ASSERT(node->arrayMode().isJSArray());
    3667        
    3668         SpeculateCellOperand base(this, node->child1());
    3669         GPRTemporary storageLength(this);
    3670        
    3671         GPRReg baseGPR = base.gpr();
    3672         GPRReg storageLengthGPR = storageLength.gpr();
    3673        
    3674         StorageOperand storage(this, node->child3());
    3675         GPRReg storageGPR = storage.gpr();
    3676 
    3677         switch (node->arrayMode().type()) {
    3678         case Array::Int32:
    3679         case Array::Contiguous: {
    3680             JSValueOperand value(this, node->child2(), ManualOperandSpeculation);
    3681             GPRReg valueGPR = value.gpr();
    3682 
    3683             if (node->arrayMode().type() == Array::Int32) {
    3684                 DFG_TYPE_CHECK(
    3685                     JSValueRegs(valueGPR), node->child2(), SpecInt32Only,
    3686                     m_jit.branch64(
    3687                         MacroAssembler::Below, valueGPR, GPRInfo::tagTypeNumberRegister));
    3688             }
    3689 
    3690             m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
    3691             MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
    3692             m_jit.store64(valueGPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight));
    3693             m_jit.add32(TrustedImm32(1), storageLengthGPR);
    3694             m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
    3695             m_jit.or64(GPRInfo::tagTypeNumberRegister, storageLengthGPR);
    3696            
    3697             addSlowPathGenerator(
    3698                 slowPathCall(
    3699                     slowPath, this, operationArrayPush, storageLengthGPR,
    3700                     valueGPR, baseGPR));
    3701        
    3702             jsValueResult(storageLengthGPR, node);
    3703             break;
    3704         }
    3705            
    3706         case Array::Double: {
    3707             SpeculateDoubleOperand value(this, node->child2());
    3708             FPRReg valueFPR = value.fpr();
    3709 
    3710             DFG_TYPE_CHECK(
    3711                 JSValueRegs(), node->child2(), SpecDoubleReal,
    3712                 m_jit.branchDouble(MacroAssembler::DoubleNotEqualOrUnordered, valueFPR, valueFPR));
    3713            
    3714             m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR);
    3715             MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
    3716             m_jit.storeDouble(valueFPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight));
    3717             m_jit.add32(TrustedImm32(1), storageLengthGPR);
    3718             m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
    3719             m_jit.or64(GPRInfo::tagTypeNumberRegister, storageLengthGPR);
    3720            
    3721             addSlowPathGenerator(
    3722                 slowPathCall(
    3723                     slowPath, this, operationArrayPushDouble, storageLengthGPR,
    3724                     valueFPR, baseGPR));
    3725        
    3726             jsValueResult(storageLengthGPR, node);
    3727             break;
    3728         }
    3729            
    3730         case Array::ArrayStorage: {
    3731             JSValueOperand value(this, node->child2());
    3732             GPRReg valueGPR = value.gpr();
    3733 
    3734             m_jit.load32(MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()), storageLengthGPR);
    3735        
    3736             // Refuse to handle bizarre lengths.
    3737             speculationCheck(Uncountable, JSValueRegs(), 0, m_jit.branch32(MacroAssembler::Above, storageLengthGPR, TrustedImm32(0x7ffffffe)));
    3738        
    3739             MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, ArrayStorage::vectorLengthOffset()));
    3740        
    3741             m_jit.store64(valueGPR, MacroAssembler::BaseIndex(storageGPR, storageLengthGPR, MacroAssembler::TimesEight, ArrayStorage::vectorOffset()));
    3742        
    3743             m_jit.add32(TrustedImm32(1), storageLengthGPR);
    3744             m_jit.store32(storageLengthGPR, MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()));
    3745             m_jit.add32(TrustedImm32(1), MacroAssembler::Address(storageGPR, OBJECT_OFFSETOF(ArrayStorage, m_numValuesInVector)));
    3746             m_jit.or64(GPRInfo::tagTypeNumberRegister, storageLengthGPR);
    3747        
    3748             addSlowPathGenerator(
    3749                 slowPathCall(
    3750                     slowPath, this, operationArrayPush, NoResult, storageLengthGPR,
    3751                     valueGPR, baseGPR));
    3752        
    3753             jsValueResult(storageLengthGPR, node);
    3754             break;
    3755         }
    3756            
    3757         default:
    3758             CRASH();
    3759             break;
    3760         }
     3666        compileArrayPush(node);
    37613667        break;
    37623668    }

  • trunk/Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.cpp

    r222658 r222675  
    251251                switch (m_node->arrayMode().type()) {
    252252                case Array::Contiguous:
    253                 case Array::ArrayStorage:
    254                     considerBarrier(m_node->child1(), m_node->child2());
     253                case Array::ArrayStorage: {
     254                    unsigned elementOffset = 2;
     255                    unsigned elementCount = m_node->numChildren() - elementOffset;
     256                    Edge& arrayEdge = m_graph.varArgChild(m_node, 1);
     257                    for (unsigned i = 0; i < elementCount; ++i) {
     258                        Edge& element = m_graph.varArgChild(m_node, i + elementOffset);
     259                        considerBarrier(arrayEdge, element);
     260                    }
    255261                    break;
     262                }
    256263                default:
    257264                    break;

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

    r222658 r222675  
    40894089    void compileArrayPush()
    40904090    {
    4091         LValue base = lowCell(m_node->child1());
    4092         LValue storage = lowStorage(m_node->child3());
    4093        
     4091        LValue base = lowCell(m_graph.varArgChild(m_node, 1));
     4092        LValue storage = lowStorage(m_graph.varArgChild(m_node, 0));
     4093        unsigned elementOffset = 2;
     4094        unsigned elementCount = m_node->numChildren() - elementOffset;
     4095
    40944096        switch (m_node->arrayMode().type()) {
    40954097        case Array::Int32:
    40964098        case Array::Contiguous:
    40974099        case Array::Double: {
    4098             LValue value;
    4099             Output::StoreType storeType;
    4100            
    4101             if (m_node->arrayMode().type() != Array::Double) {
    4102                 value = lowJSValue(m_node->child2(), ManualOperandSpeculation);
    4103                 if (m_node->arrayMode().type() == Array::Int32) {
    4104                     FTL_TYPE_CHECK(
    4105                         jsValueValue(value), m_node->child2(), SpecInt32Only, isNotInt32(value));
     4100            IndexedAbstractHeap& heap = m_heaps.forArrayType(m_node->arrayMode().type());
     4101
     4102            if (elementCount == 1) {
     4103                LValue value;
     4104                Output::StoreType storeType;
     4105               
     4106                Edge& element = m_graph.varArgChild(m_node, elementOffset);
     4107                if (m_node->arrayMode().type() != Array::Double) {
     4108                    value = lowJSValue(element, ManualOperandSpeculation);
     4109                    if (m_node->arrayMode().type() == Array::Int32)
     4110                        RELEASE_ASSERT(!m_interpreter.needsTypeCheck(element, SpecInt32Only));
     4111                    storeType = Output::Store64;
     4112                } else {
     4113                    value = lowDouble(element);
     4114                    RELEASE_ASSERT(!m_interpreter.needsTypeCheck(element, SpecDoubleReal));
     4115                    storeType = Output::StoreDouble;
    41064116                }
    4107                 storeType = Output::Store64;
    4108             } else {
    4109                 value = lowDouble(m_node->child2());
    4110                 FTL_TYPE_CHECK(
    4111                     doubleValue(value), m_node->child2(), SpecDoubleReal,
    4112                     m_out.doubleNotEqualOrUnordered(value, value));
    4113                 storeType = Output::StoreDouble;
     4117
     4118                LValue prevLength = m_out.load32(storage, m_heaps.Butterfly_publicLength);
     4119               
     4120                LBasicBlock fastPath = m_out.newBlock();
     4121                LBasicBlock slowPath = m_out.newBlock();
     4122                LBasicBlock continuation = m_out.newBlock();
     4123               
     4124                m_out.branch(
     4125                    m_out.aboveOrEqual(
     4126                        prevLength, m_out.load32(storage, m_heaps.Butterfly_vectorLength)),
     4127                    unsure(slowPath), unsure(fastPath));
     4128               
     4129                LBasicBlock lastNext = m_out.appendTo(fastPath, slowPath);
     4130                m_out.store(
     4131                    value, m_out.baseIndex(heap, storage, m_out.zeroExtPtr(prevLength)), storeType);
     4132                LValue newLength = m_out.add(prevLength, m_out.int32One);
     4133                m_out.store32(newLength, storage, m_heaps.Butterfly_publicLength);
     4134               
     4135                ValueFromBlock fastResult = m_out.anchor(boxInt32(newLength));
     4136                m_out.jump(continuation);
     4137               
     4138                m_out.appendTo(slowPath, continuation);
     4139                LValue operation;
     4140                if (m_node->arrayMode().type() != Array::Double)
     4141                    operation = m_out.operation(operationArrayPush);
     4142                else
     4143                    operation = m_out.operation(operationArrayPushDouble);
     4144                ValueFromBlock slowResult = m_out.anchor(
     4145                    vmCall(Int64, operation, m_callFrame, value, base));
     4146                m_out.jump(continuation);
     4147               
     4148                m_out.appendTo(continuation, lastNext);
     4149                setJSValue(m_out.phi(Int64, fastResult, slowResult));
     4150                return;
    41144151            }
    4115            
    4116             IndexedAbstractHeap& heap = m_heaps.forArrayType(m_node->arrayMode().type());
    41174152
    41184153            LValue prevLength = m_out.load32(storage, m_heaps.Butterfly_publicLength);
    4119            
     4154            LValue newLength = m_out.add(prevLength, m_out.constInt32(elementCount));
     4155
    41204156            LBasicBlock fastPath = m_out.newBlock();
    41214157            LBasicBlock slowPath = m_out.newBlock();
     4158            LBasicBlock setup = m_out.newBlock();
     4159            LBasicBlock slowCallPath = m_out.newBlock();
    41224160            LBasicBlock continuation = m_out.newBlock();
    4123            
    4124             m_out.branch(
    4125                 m_out.aboveOrEqual(
    4126                     prevLength, m_out.load32(storage, m_heaps.Butterfly_vectorLength)),
    4127                 unsure(slowPath), unsure(fastPath));
    4128            
     4161
     4162            LValue beyondVectorLength = m_out.above(newLength, m_out.load32(storage, m_heaps.Butterfly_vectorLength));
     4163
     4164            m_out.branch(beyondVectorLength, unsure(slowPath), unsure(fastPath));
     4165
    41294166            LBasicBlock lastNext = m_out.appendTo(fastPath, slowPath);
    4130             m_out.store(
    4131                 value, m_out.baseIndex(heap, storage, m_out.zeroExtPtr(prevLength)), storeType);
    4132             LValue newLength = m_out.add(prevLength, m_out.int32One);
    41334167            m_out.store32(newLength, storage, m_heaps.Butterfly_publicLength);
    4134            
     4168            ValueFromBlock fastBufferResult = m_out.anchor(m_out.baseIndex(storage, m_out.zeroExtPtr(prevLength), ScaleEight));
     4169            m_out.jump(setup);
     4170
     4171            m_out.appendTo(slowPath, setup);
     4172            size_t scratchSize = sizeof(EncodedJSValue) * elementCount;
     4173            static_assert(sizeof(EncodedJSValue) == sizeof(double), "");
     4174            ASSERT(scratchSize);
     4175            ScratchBuffer* scratchBuffer = vm().scratchBufferForSize(scratchSize);
     4176            m_out.storePtr(m_out.constIntPtr(scratchSize), m_out.absolute(scratchBuffer->addressOfActiveLength()));
     4177            ValueFromBlock slowBufferResult = m_out.anchor(m_out.constIntPtr(static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer())));
     4178            m_out.jump(setup);
     4179
     4180            m_out.appendTo(setup, slowCallPath);
     4181            LValue buffer = m_out.phi(pointerType(), fastBufferResult, slowBufferResult);
     4182            for (unsigned elementIndex = 0; elementIndex < elementCount; ++elementIndex) {
     4183                Edge& element = m_graph.varArgChild(m_node, elementIndex + elementOffset);
     4184
     4185                LValue value;
     4186                Output::StoreType storeType;
     4187                if (m_node->arrayMode().type() != Array::Double) {
     4188                    value = lowJSValue(element, ManualOperandSpeculation);
     4189                    if (m_node->arrayMode().type() == Array::Int32)
     4190                        RELEASE_ASSERT(!m_interpreter.needsTypeCheck(element, SpecInt32Only));
     4191                    storeType = Output::Store64;
     4192                } else {
     4193                    value = lowDouble(element);
     4194                    RELEASE_ASSERT(!m_interpreter.needsTypeCheck(element, SpecDoubleReal));
     4195                    storeType = Output::StoreDouble;
     4196                }
     4197
     4198                m_out.store(value, m_out.baseIndex(heap, buffer, m_out.constInt32(elementIndex), jsNumber(elementIndex)), storeType);
     4199            }
    41354200            ValueFromBlock fastResult = m_out.anchor(boxInt32(newLength));
    4136             m_out.jump(continuation);
    4137            
    4138             m_out.appendTo(slowPath, continuation);
     4201
     4202            m_out.branch(beyondVectorLength, unsure(slowCallPath), unsure(continuation));
     4203
     4204            m_out.appendTo(slowCallPath, continuation);
    41394205            LValue operation;
    41404206            if (m_node->arrayMode().type() != Array::Double)
    4141                 operation = m_out.operation(operationArrayPush);
     4207                operation = m_out.operation(operationArrayPushMultiple);
    41424208            else
    4143                 operation = m_out.operation(operationArrayPushDouble);
    4144             ValueFromBlock slowResult = m_out.anchor(
    4145                 vmCall(Int64, operation, m_callFrame, value, base));
     4209                operation = m_out.operation(operationArrayPushDoubleMultiple);
     4210            ValueFromBlock slowResult = m_out.anchor(vmCall(Int64, operation, m_callFrame, base, buffer, m_out.constInt32(elementCount)));
     4211            m_out.storePtr(m_out.constIntPtr(0), m_out.absolute(scratchBuffer->addressOfActiveLength()));
    41464212            m_out.jump(continuation);
    4147            
     4213
    41484214            m_out.appendTo(continuation, lastNext);
    41494215            setJSValue(m_out.phi(Int64, fastResult, slowResult));
    41504216            return;
    41514217        }
    4152            
     4218
    41534219        default:
    41544220            DFG_CRASH(m_graph, m_node, "Bad array type");

  • trunk/Source/JavaScriptCore/jit/JITOperations.h

    r222658 r222675  
    142142typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJC)(ExecState*, EncodedJSValue, JSCell*);
    143143typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJA)(ExecState*, EncodedJSValue, JSArray*);
     144typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EAPZ)(ExecState*, JSArray*, void*, int32_t);
    144145typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJArp)(ExecState*, EncodedJSValue, ArithProfile*);
    145146typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJI)(ExecState*, EncodedJSValue, UniquedStringImpl*);

  • trunk/Source/JavaScriptCore/runtime/ArrayPrototype.cpp

    r222658 r222675  
    765765    JSValue thisValue = exec->thisValue().toThis(exec, StrictMode);
    766766
    767     if (isJSArray(thisValue) && exec->argumentCount() == 1) {
     767    if (LIKELY(isJSArray(thisValue) && exec->argumentCount() == 1)) {
    768768        JSArray* array = asArray(thisValue);
    769769        scope.release();
    770         array->push(exec, exec->uncheckedArgument(0));
     770        array->pushInline(exec, exec->uncheckedArgument(0));
    771771        return JSValue::encode(jsNumber(array->length()));
    772772    }

  • trunk/Source/JavaScriptCore/runtime/JSArray.cpp

    r222658 r222675  
    4141namespace JSC {
    4242
    43 static const char* const LengthExceededTheMaximumArrayLengthError = "Length exceeded the maximum array length";
     43const char* const LengthExceededTheMaximumArrayLengthError = "Length exceeded the maximum array length";
    4444
    4545STATIC_ASSERT_IS_TRIVIALLY_DESTRUCTIBLE(JSArray);
    … …  
    726726//  - we always are writing beyond the current array bounds, so it is always necessary to update m_length & m_numValuesInVector.
    727727//  - pushing to an array of length 2^32-1 stores the property, but throws a range error.
    728 void JSArray::push(ExecState* exec, JSValue value)
    729 {
    730     VM& vm = exec->vm();
    731     auto scope = DECLARE_THROW_SCOPE(vm);
    732 
    733     Butterfly* butterfly = m_butterfly.getMayBeNull();
    734    
    735     switch (indexingType()) {
    736     case ArrayClass: {
    737         createInitialUndecided(vm, 0);
    738         FALLTHROUGH;
    739     }
    740        
    741     case ArrayWithUndecided: {
    742         convertUndecidedForValue(vm, value);
    743         scope.release();
    744         push(exec, value);
    745         return;
    746     }
    747        
    748     case ArrayWithInt32: {
    749         if (!value.isInt32()) {
    750             convertInt32ForValue(vm, value);
    751             scope.release();
    752             push(exec, value);
    753             return;
    754         }
    755 
    756         unsigned length = butterfly->publicLength();
    757         ASSERT(length <= butterfly->vectorLength());
    758         if (length < butterfly->vectorLength()) {
    759             butterfly->contiguousInt32()[length].setWithoutWriteBarrier(value);
    760             butterfly->setPublicLength(length + 1);
    761             return;
    762         }
    763        
    764         if (UNLIKELY(length > MAX_ARRAY_INDEX)) {
    765             methodTable(vm)->putByIndex(this, exec, length, value, true);
    766             if (!scope.exception())
    767                 throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
    768             return;
    769         }
    770 
    771         scope.release();
    772         putByIndexBeyondVectorLengthWithoutAttributes<Int32Shape>(exec, length, value);
    773         return;
    774     }
    775 
    776     case ArrayWithContiguous: {
    777         unsigned length = butterfly->publicLength();
    778         ASSERT(length <= butterfly->vectorLength());
    779         if (length < butterfly->vectorLength()) {
    780             butterfly->contiguous()[length].set(vm, this, value);
    781             butterfly->setPublicLength(length + 1);
    782             return;
    783         }
    784        
    785         if (UNLIKELY(length > MAX_ARRAY_INDEX)) {
    786             methodTable(vm)->putByIndex(this, exec, length, value, true);
    787             if (!scope.exception())
    788                 throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
    789             return;
    790         }
    791 
    792         scope.release();
    793         putByIndexBeyondVectorLengthWithoutAttributes<ContiguousShape>(exec, length, value);
    794         return;
    795     }
    796        
    797     case ArrayWithDouble: {
    798         if (!value.isNumber()) {
    799             convertDoubleToContiguous(vm);
    800             scope.release();
    801             push(exec, value);
    802             return;
    803         }
    804         double valueAsDouble = value.asNumber();
    805         if (valueAsDouble != valueAsDouble) {
    806             convertDoubleToContiguous(vm);
    807             scope.release();
    808             push(exec, value);
    809             return;
    810         }
    811 
    812         unsigned length = butterfly->publicLength();
    813         ASSERT(length <= butterfly->vectorLength());
    814         if (length < butterfly->vectorLength()) {
    815             butterfly->contiguousDouble()[length] = valueAsDouble;
    816             butterfly->setPublicLength(length + 1);
    817             return;
    818         }
    819        
    820         if (UNLIKELY(length > MAX_ARRAY_INDEX)) {
    821             methodTable(vm)->putByIndex(this, exec, length, value, true);
    822             if (!scope.exception())
    823                 throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
    824             return;
    825         }
    826 
    827         scope.release();
    828         putByIndexBeyondVectorLengthWithoutAttributes<DoubleShape>(exec, length, value);
    829         return;
    830     }
    831        
    832     case ArrayWithSlowPutArrayStorage: {
    833         unsigned oldLength = length();
    834         bool putResult = false;
    835         if (attemptToInterceptPutByIndexOnHole(exec, oldLength, value, true, putResult)) {
    836             if (!scope.exception() && oldLength < 0xFFFFFFFFu) {
    837                 scope.release();
    838                 setLength(exec, oldLength + 1, true);
    839             }
    840             return;
    841         }
    842         FALLTHROUGH;
    843     }
    844        
    845     case ArrayWithArrayStorage: {
    846         ArrayStorage* storage = butterfly->arrayStorage();
    847 
    848         // Fast case - push within vector, always update m_length & m_numValuesInVector.
    849         unsigned length = storage->length();
    850         if (length < storage->vectorLength()) {
    851             storage->m_vector[length].set(vm, this, value);
    852             storage->setLength(length + 1);
    853             ++storage->m_numValuesInVector;
    854             return;
    855         }
    856 
    857         // Pushing to an array of invalid length (2^31-1) stores the property, but throws a range error.
    858         if (UNLIKELY(storage->length() > MAX_ARRAY_INDEX)) {
    859             methodTable(vm)->putByIndex(this, exec, storage->length(), value, true);
    860             // Per ES5.1 15.4.4.7 step 6 & 15.4.5.1 step 3.d.
    861             if (!scope.exception())
    862                 throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
    863             return;
    864         }
    865 
    866         // Handled the same as putIndex.
    867         scope.release();
    868         putByIndexBeyondVectorLengthWithArrayStorage(exec, storage->length(), value, true, storage);
    869         return;
    870     }
    871        
    872     default:
    873         RELEASE_ASSERT_NOT_REACHED();
    874     }
     728NEVER_INLINE void JSArray::push(ExecState* exec, JSValue value)
     729{
     730    pushInline(exec, value);
    875731}
    876732

  • trunk/Source/JavaScriptCore/runtime/JSArray.h

    r222658 r222675  
    3232class LLIntOffsetsExtractor;
    3333
     34extern const char* const LengthExceededTheMaximumArrayLengthError;
     35
    3436class JSArray : public JSNonFinalObject {
    3537    friend class LLIntOffsetsExtractor;
    … …  
    9193    JS_EXPORT_PRIVATE bool setLength(ExecState*, unsigned, bool throwException = false);
    9294
     95    void pushInline(ExecState*, JSValue);
    9396    JS_EXPORT_PRIVATE void push(ExecState*, JSValue);
    9497    JS_EXPORT_PRIVATE JSValue pop(ExecState*);

  • trunk/Source/JavaScriptCore/runtime/JSArrayInlines.h

    r222658 r222675  
    2020#pragma once
    2121
     22#include "Error.h"
    2223#include "JSArray.h"
    2324#include "JSCellInlines.h"
    … …  
    8384}
    8485
     86ALWAYS_INLINE void JSArray::pushInline(ExecState* exec, JSValue value)
     87{
     88    VM& vm = exec->vm();
     89    auto scope = DECLARE_THROW_SCOPE(vm);
     90
     91    Butterfly* butterfly = m_butterfly.getMayBeNull();
     92
     93    switch (indexingType()) {
     94    case ArrayClass: {
     95        createInitialUndecided(vm, 0);
     96        FALLTHROUGH;
     97    }
     98
     99    case ArrayWithUndecided: {
     100        convertUndecidedForValue(vm, value);
     101        scope.release();
     102        push(exec, value);
     103        return;
     104    }
     105
     106    case ArrayWithInt32: {
     107        if (!value.isInt32()) {
     108            convertInt32ForValue(vm, value);
     109            scope.release();
     110            push(exec, value);
     111            return;
     112        }
     113
     114        unsigned length = butterfly->publicLength();
     115        ASSERT(length <= butterfly->vectorLength());
     116        if (length < butterfly->vectorLength()) {
     117            butterfly->contiguousInt32()[length].setWithoutWriteBarrier(value);
     118            butterfly->setPublicLength(length + 1);
     119            return;
     120        }
     121
     122        if (UNLIKELY(length > MAX_ARRAY_INDEX)) {
     123            methodTable(vm)->putByIndex(this, exec, length, value, true);
     124            if (!scope.exception())
     125                throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
     126            return;
     127        }
     128
     129        scope.release();
     130        putByIndexBeyondVectorLengthWithoutAttributes<Int32Shape>(exec, length, value);
     131        return;
     132    }
     133
     134    case ArrayWithContiguous: {
     135        unsigned length = butterfly->publicLength();
     136        ASSERT(length <= butterfly->vectorLength());
     137        if (length < butterfly->vectorLength()) {
     138            butterfly->contiguous()[length].set(vm, this, value);
     139            butterfly->setPublicLength(length + 1);
     140            return;
     141        }
     142
     143        if (UNLIKELY(length > MAX_ARRAY_INDEX)) {
     144            methodTable(vm)->putByIndex(this, exec, length, value, true);
     145            if (!scope.exception())
     146                throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
     147            return;
     148        }
     149
     150        scope.release();
     151        putByIndexBeyondVectorLengthWithoutAttributes<ContiguousShape>(exec, length, value);
     152        return;
     153    }
     154
     155    case ArrayWithDouble: {
     156        if (!value.isNumber()) {
     157            convertDoubleToContiguous(vm);
     158            scope.release();
     159            push(exec, value);
     160            return;
     161        }
     162        double valueAsDouble = value.asNumber();
     163        if (valueAsDouble != valueAsDouble) {
     164            convertDoubleToContiguous(vm);
     165            scope.release();
     166            push(exec, value);
     167            return;
     168        }
     169
     170        unsigned length = butterfly->publicLength();
     171        ASSERT(length <= butterfly->vectorLength());
     172        if (length < butterfly->vectorLength()) {
     173            butterfly->contiguousDouble()[length] = valueAsDouble;
     174            butterfly->setPublicLength(length + 1);
     175            return;
     176        }
     177
     178        if (UNLIKELY(length > MAX_ARRAY_INDEX)) {
     179            methodTable(vm)->putByIndex(this, exec, length, value, true);
     180            if (!scope.exception())
     181                throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
     182            return;
     183        }
     184
     185        scope.release();
     186        putByIndexBeyondVectorLengthWithoutAttributes<DoubleShape>(exec, length, value);
     187        return;
     188    }
     189
     190    case ArrayWithSlowPutArrayStorage: {
     191        unsigned oldLength = length();
     192        bool putResult = false;
     193        if (attemptToInterceptPutByIndexOnHole(exec, oldLength, value, true, putResult)) {
     194            if (!scope.exception() && oldLength < 0xFFFFFFFFu) {
     195                scope.release();
     196                setLength(exec, oldLength + 1, true);
     197            }
     198            return;
     199        }
     200        FALLTHROUGH;
     201    }
     202
     203    case ArrayWithArrayStorage: {
     204        ArrayStorage* storage = butterfly->arrayStorage();
     205
     206        // Fast case - push within vector, always update m_length & m_numValuesInVector.
     207        unsigned length = storage->length();
     208        if (length < storage->vectorLength()) {
     209            storage->m_vector[length].set(vm, this, value);
     210            storage->setLength(length + 1);
     211            ++storage->m_numValuesInVector;
     212            return;
     213        }
     214
     215        // Pushing to an array of invalid length (2^31-1) stores the property, but throws a range error.
     216        if (UNLIKELY(storage->length() > MAX_ARRAY_INDEX)) {
     217            methodTable(vm)->putByIndex(this, exec, storage->length(), value, true);
     218            // Per ES5.1 15.4.4.7 step 6 & 15.4.5.1 step 3.d.
     219            if (!scope.exception())
     220                throwException(exec, scope, createRangeError(exec, ASCIILiteral(LengthExceededTheMaximumArrayLengthError)));
     221            return;
     222        }
     223
     224        // Handled the same as putIndex.
     225        scope.release();
     226        putByIndexBeyondVectorLengthWithArrayStorage(exec, storage->length(), value, true, storage);
     227        return;
     228    }
     229
     230    default:
     231        RELEASE_ASSERT_NOT_REACHED();
     232    }
     233}
     234
    85235} // namespace JSC
Note: See TracChangeset for help on using the changeset viewer.