Changeset 187791 in webkit


Ignore:
Timestamp:
Aug 3, 2015 5:39:19 PM (9 years ago)
Author:
basile_clement@apple.com
Message:

jsc-tailcall: Implement the tail call opcodes in the DFG
​https://bugs.webkit.org/show_bug.cgi?id=146850

Reviewed by Michael Saboff.

This patch adds support for tail calls in the DFG. This requires a slightly high number of nodes:

  • TailCall and TailCallVarargs are straightforward. They are terminal nodes and have the semantics of an actual tail call.
  • TailCallInlinedCaller and TailCallVarargsInlinedCaller are here to perform a tail call inside an inlined function. They are non terminal nodes, and are performing the call as a regular call after popping an appropriate number of inlined tail call frames.
  • TailCallForwardVarargs and TailCallForwardVarargsInlinedCaller are the extension of TailCallVarargs and TailCallVarargsInlinedCaller to enable the varargs forwarding optimization so that we don't lose performance with a tail call instead of a regular call.

This also required two broad kind of changes:

  • Changes in the JIT itself (DFGSpeculativeJIT) are pretty straightforward since they are just an extension of the baseline JIT changes introduced previously.
  • Changes in the runtime are mostly related with handling inline call frames. The idea here is that we have a special TailCall type for call frames that indicates to the various pieces of code walking the inline call frame that they should (recursively) skip the caller in their analysis.
  • bytecode/CallMode.h:

(JSC::specializationKindFor):

  • bytecode/CodeOrigin.cpp:

(JSC::CodeOrigin::inlineDepthForCallFrame):
(JSC::CodeOrigin::isApproximatelyEqualTo):
(JSC::CodeOrigin::approximateHash):
(JSC::CodeOrigin::inlineStack):
(JSC::InlineCallFrame::dumpInContext):
(WTF::printInternal):

  • bytecode/CodeOrigin.h:

(JSC::InlineCallFrame::callModeFor):
(JSC::InlineCallFrame::kindFor):
(JSC::InlineCallFrame::varargsKindFor):
(JSC::InlineCallFrame::specializationKindFor):
(JSC::InlineCallFrame::isVarargs):
(JSC::InlineCallFrame::isTail):
(JSC::InlineCallFrame::computeCallerSkippingDeadFrames):
(JSC::InlineCallFrame::getCallerSkippingDeadFrames):
(JSC::InlineCallFrame::getCallerInlineFrameSkippingDeadFrames):

  • dfg/DFGAbstractInterpreterInlines.h:

(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):

  • dfg/DFGArgumentsEliminationPhase.cpp:
  • dfg/DFGBasicBlock.h:

(JSC::DFG::BasicBlock::findTerminal):

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::allInlineFramesAreTailCalls):
(JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
(JSC::DFG::ByteCodeParser::addCall):
(JSC::DFG::ByteCodeParser::getPredictionWithoutOSRExit):
(JSC::DFG::ByteCodeParser::getPrediction):
(JSC::DFG::ByteCodeParser::handleCall):
(JSC::DFG::ByteCodeParser::handleVarargsCall):
(JSC::DFG::ByteCodeParser::inliningCost):
(JSC::DFG::ByteCodeParser::inlineCall):
(JSC::DFG::ByteCodeParser::attemptToInlineCall):
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::InlineStackEntry::InlineStackEntry):
(JSC::DFG::ByteCodeParser::parseCodeBlock):

  • dfg/DFGCapabilities.cpp:

(JSC::DFG::capabilityLevel):

  • dfg/DFGClobberize.h:

(JSC::DFG::clobberize):

  • dfg/DFGDoesGC.cpp:

(JSC::DFG::doesGC):

  • dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::fixupNode):

  • dfg/DFGGraph.cpp:

(JSC::DFG::Graph::isLiveInBytecode):

  • dfg/DFGGraph.h:

(JSC::DFG::Graph::forAllLocalsLiveInBytecode):

  • dfg/DFGInPlaceAbstractState.cpp:

(JSC::DFG::InPlaceAbstractState::mergeToSuccessors):

  • dfg/DFGNode.h:

(JSC::DFG::Node::hasCallVarargsData):
(JSC::DFG::Node::isTerminal):
(JSC::DFG::Node::hasHeapPrediction):

  • dfg/DFGNodeType.h:
  • dfg/DFGOSRExitCompilerCommon.cpp:

(JSC::DFG::handleExitCounts):
(JSC::DFG::reifyInlinedCallFrames):

  • dfg/DFGOSRExitPreparation.cpp:

(JSC::DFG::prepareCodeOriginForOSRExit):

  • dfg/DFGOperations.cpp:
  • dfg/DFGPreciseLocalClobberize.h:

(JSC::DFG::PreciseLocalClobberizeAdaptor::readTop):

  • dfg/DFGPredictionPropagationPhase.cpp:

(JSC::DFG::PredictionPropagationPhase::propagate):

  • dfg/DFGSafeToExecute.h:

(JSC::DFG::safeToExecute):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGVarargsForwardingPhase.cpp:
  • interpreter/CallFrame.cpp:

(JSC::CallFrame::bytecodeOffsetFromCodeOriginIndex):

  • interpreter/StackVisitor.cpp:

(JSC::StackVisitor::gotoNextFrame):

  • jit/CCallHelpers.h:
  • tests/stress/dfg-tail-calls.js: Added.
Location:
branches/jsc-tailcall/Source/JavaScriptCore
Files:
1 added
29 edited

Legend:

Unmodified
Added
Removed

  • branches/jsc-tailcall/Source/JavaScriptCore/ChangeLog

    r187767 r187791  
     12015-07-31  Basile Clement  <basile_clement@apple.com>
     2
     3        jsc-tailcall: Implement the tail call opcodes in the DFG
     4        https://bugs.webkit.org/show_bug.cgi?id=146850
     5
     6        Reviewed by Michael Saboff.
     7
     8        This patch adds support for tail calls in the DFG. This requires a slightly high number of nodes:
     9
     10         - TailCall and TailCallVarargs are straightforward. They are terminal
     11           nodes and have the semantics of an actual tail call.
     12
     13         - TailCallInlinedCaller and TailCallVarargsInlinedCaller are here to perform a
     14           tail call inside an inlined function. They are non terminal nodes,
     15           and are performing the call as a regular call after popping an
     16           appropriate number of inlined tail call frames.
     17
     18         - TailCallForwardVarargs and TailCallForwardVarargsInlinedCaller are the
     19           extension of TailCallVarargs and TailCallVarargsInlinedCaller to enable
     20           the varargs forwarding optimization so that we don't lose
     21           performance with a tail call instead of a regular call.
     22
     23        This also required two broad kind of changes:
     24
     25         - Changes in the JIT itself (DFGSpeculativeJIT) are pretty
     26           straightforward since they are just an extension of the baseline JIT
     27           changes introduced previously.
     28
     29         - Changes in the runtime are mostly related with handling inline call
     30           frames. The idea here is that we have a special TailCall type for
     31           call frames that indicates to the various pieces of code walking the
     32           inline call frame that they should (recursively) skip the caller in
     33           their analysis.
     34
     35        * bytecode/CallMode.h:
     36        (JSC::specializationKindFor):
     37        * bytecode/CodeOrigin.cpp:
     38        (JSC::CodeOrigin::inlineDepthForCallFrame):
     39        (JSC::CodeOrigin::isApproximatelyEqualTo):
     40        (JSC::CodeOrigin::approximateHash):
     41        (JSC::CodeOrigin::inlineStack):
     42        (JSC::InlineCallFrame::dumpInContext):
     43        (WTF::printInternal):
     44        * bytecode/CodeOrigin.h:
     45        (JSC::InlineCallFrame::callModeFor):
     46        (JSC::InlineCallFrame::kindFor):
     47        (JSC::InlineCallFrame::varargsKindFor):
     48        (JSC::InlineCallFrame::specializationKindFor):
     49        (JSC::InlineCallFrame::isVarargs):
     50        (JSC::InlineCallFrame::isTail):
     51        (JSC::InlineCallFrame::computeCallerSkippingDeadFrames):
     52        (JSC::InlineCallFrame::getCallerSkippingDeadFrames):
     53        (JSC::InlineCallFrame::getCallerInlineFrameSkippingDeadFrames):
     54        * dfg/DFGAbstractInterpreterInlines.h:
     55        (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
     56        * dfg/DFGArgumentsEliminationPhase.cpp:
     57        * dfg/DFGBasicBlock.h:
     58        (JSC::DFG::BasicBlock::findTerminal):
     59        * dfg/DFGByteCodeParser.cpp:
     60        (JSC::DFG::ByteCodeParser::allInlineFramesAreTailCalls):
     61        (JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
     62        (JSC::DFG::ByteCodeParser::addCall):
     63        (JSC::DFG::ByteCodeParser::getPredictionWithoutOSRExit):
     64        (JSC::DFG::ByteCodeParser::getPrediction):
     65        (JSC::DFG::ByteCodeParser::handleCall):
     66        (JSC::DFG::ByteCodeParser::handleVarargsCall):
     67        (JSC::DFG::ByteCodeParser::inliningCost):
     68        (JSC::DFG::ByteCodeParser::inlineCall):
     69        (JSC::DFG::ByteCodeParser::attemptToInlineCall):
     70        (JSC::DFG::ByteCodeParser::parseBlock):
     71        (JSC::DFG::ByteCodeParser::InlineStackEntry::InlineStackEntry):
     72        (JSC::DFG::ByteCodeParser::parseCodeBlock):
     73        * dfg/DFGCapabilities.cpp:
     74        (JSC::DFG::capabilityLevel):
     75        * dfg/DFGClobberize.h:
     76        (JSC::DFG::clobberize):
     77        * dfg/DFGDoesGC.cpp:
     78        (JSC::DFG::doesGC):
     79        * dfg/DFGFixupPhase.cpp:
     80        (JSC::DFG::FixupPhase::fixupNode):
     81        * dfg/DFGGraph.cpp:
     82        (JSC::DFG::Graph::isLiveInBytecode):
     83        * dfg/DFGGraph.h:
     84        (JSC::DFG::Graph::forAllLocalsLiveInBytecode):
     85        * dfg/DFGInPlaceAbstractState.cpp:
     86        (JSC::DFG::InPlaceAbstractState::mergeToSuccessors):
     87        * dfg/DFGNode.h:
     88        (JSC::DFG::Node::hasCallVarargsData):
     89        (JSC::DFG::Node::isTerminal):
     90        (JSC::DFG::Node::hasHeapPrediction):
     91        * dfg/DFGNodeType.h:
     92        * dfg/DFGOSRExitCompilerCommon.cpp:
     93        (JSC::DFG::handleExitCounts):
     94        (JSC::DFG::reifyInlinedCallFrames):
     95        * dfg/DFGOSRExitPreparation.cpp:
     96        (JSC::DFG::prepareCodeOriginForOSRExit):
     97        * dfg/DFGOperations.cpp:
     98        * dfg/DFGPreciseLocalClobberize.h:
     99        (JSC::DFG::PreciseLocalClobberizeAdaptor::readTop):
     100        * dfg/DFGPredictionPropagationPhase.cpp:
     101        (JSC::DFG::PredictionPropagationPhase::propagate):
     102        * dfg/DFGSafeToExecute.h:
     103        (JSC::DFG::safeToExecute):
     104        * dfg/DFGSpeculativeJIT32_64.cpp:
     105        (JSC::DFG::SpeculativeJIT::emitCall):
     106        * dfg/DFGSpeculativeJIT64.cpp:
     107        (JSC::DFG::SpeculativeJIT::emitCall):
     108        (JSC::DFG::SpeculativeJIT::compile):
     109        * dfg/DFGVarargsForwardingPhase.cpp:
     110        * interpreter/CallFrame.cpp:
     111        (JSC::CallFrame::bytecodeOffsetFromCodeOriginIndex):
     112        * interpreter/StackVisitor.cpp:
     113        (JSC::StackVisitor::gotoNextFrame):
     114        * jit/CCallHelpers.h:
     115        * tests/stress/dfg-tail-calls.js: Added.
     116
    11172015-08-03  Basile Clement  <basile_clement@apple.com>
    2118

  • branches/jsc-tailcall/Source/JavaScriptCore/bytecode/CallMode.h

    r187629 r187791  
    2727#define CallMode_h
    2828
     29#include "CodeSpecializationKind.h"
     30
    2931namespace JSC {
    3032
    … …  
    3234
    3335enum FrameAction { KeepTheFrame = 0, ReuseTheFrame };
     36
     37inline CodeSpecializationKind specializationKindFor(CallMode callMode)
     38{
     39    switch (callMode) {
     40    case CallMode::Tail:
     41    case CallMode::Regular:
     42        return CodeForCall;
     43
     44    case CallMode::Construct:
     45        return CodeForConstruct;
     46    }
     47}
    3448
    3549} // namespace JSC

  • branches/jsc-tailcall/Source/JavaScriptCore/bytecode/CodeOrigin.cpp

    r185768 r187791  
    3737{
    3838    unsigned result = 1;
    39     for (InlineCallFrame* current = inlineCallFrame; current; current = current->caller.inlineCallFrame)
     39    for (InlineCallFrame* current = inlineCallFrame; current; current = current->directCaller.inlineCallFrame)
    4040        result++;
    4141    return result;
    … …  
    7878            return false;
    7979       
    80         a = a.inlineCallFrame->caller;
    81         b = b.inlineCallFrame->caller;
     80        a = a.inlineCallFrame->directCaller;
     81        b = b.inlineCallFrame->directCaller;
    8282    }
    8383}
    … …  
    100100        result += WTF::PtrHash<JSCell*>::hash(codeOrigin.inlineCallFrame->executable.get());
    101101       
    102         codeOrigin = codeOrigin.inlineCallFrame->caller;
     102        codeOrigin = codeOrigin.inlineCallFrame->directCaller;
    103103    }
    104104}
    … …  
    109109    result.last() = *this;
    110110    unsigned index = result.size() - 2;
    111     for (InlineCallFrame* current = inlineCallFrame; current; current = current->caller.inlineCallFrame)
    112         result[index--] = current->caller;
     111    for (InlineCallFrame* current = inlineCallFrame; current; current = current->directCaller.inlineCallFrame)
     112        result[index--] = current->directCaller;
    113113    RELEASE_ASSERT(!result[0].inlineCallFrame);
    114114    return result;
    … …  
    191191    if (executable->isStrictMode())
    192192        out.print(" (StrictMode)");
    193     out.print(", bc#", caller.bytecodeIndex, ", ", kind);
     193    out.print(", bc#", directCaller.bytecodeIndex, ", ", static_cast<Kind>(kind));
    194194    if (isClosureCall)
    195195        out.print(", closure call");
    … …  
    219219        out.print("Construct");
    220220        return;
     221    case JSC::InlineCallFrame::TailCall:
     222        out.print("TailCall");
     223        return;
    221224    case JSC::InlineCallFrame::CallVarargs:
    222225        out.print("CallVarargs");
    … …  
    225228        out.print("ConstructVarargs");
    226229        return;
     230    case JSC::InlineCallFrame::TailCallVarargs:
     231        out.print("TailCallVarargs");
     232        return;
    227233    case JSC::InlineCallFrame::GetterCall:
    228234        out.print("GetterCall");

  • branches/jsc-tailcall/Source/JavaScriptCore/bytecode/CodeOrigin.h

    r182759 r187791  
    2727#define CodeOrigin_h
    2828
     29#include "CallMode.h"
    2930#include "CodeBlockHash.h"
    3031#include "CodeSpecializationKind.h"
    … …  
    121122        Call,
    122123        Construct,
     124        TailCall,
    123125        CallVarargs,
    124126        ConstructVarargs,
     127        TailCallVarargs,
    125128       
    126129        // For these, the stackOffset incorporates the argument count plus the true return PC
    … …  
    129132        SetterCall
    130133    };
    131    
    132     static Kind kindFor(CodeSpecializationKind kind)
    133     {
    134         switch (kind) {
    135         case CodeForCall:
    136             return Call;
    137         case CodeForConstruct:
    138             return Construct;
    139         }
    140         RELEASE_ASSERT_NOT_REACHED();
    141         return Call;
    142     }
    143    
    144     static Kind varargsKindFor(CodeSpecializationKind kind)
    145     {
    146         switch (kind) {
    147         case CodeForCall:
    148             return CallVarargs;
    149         case CodeForConstruct:
    150             return ConstructVarargs;
    151         }
    152         RELEASE_ASSERT_NOT_REACHED();
    153         return Call;
    154     }
    155    
    156     static CodeSpecializationKind specializationKindFor(Kind kind)
     134
     135    static CallMode callModeFor(Kind kind)
    157136    {
    158137        switch (kind) {
    159138        case Call:
    160139        case CallVarargs:
     140        case GetterCall:
     141        case SetterCall:
     142            return CallMode::Regular;
     143        case TailCall:
     144        case TailCallVarargs:
     145            return CallMode::Tail;
     146        case Construct:
     147        case ConstructVarargs:
     148            return CallMode::Construct;
     149        }
     150    }
     151
     152    static Kind kindFor(CallMode callMode)
     153    {
     154        switch (callMode) {
     155        case CallMode::Regular:
     156            return Call;
     157        case CallMode::Construct:
     158            return Construct;
     159        case CallMode::Tail:
     160            return TailCall;
     161        }
     162        RELEASE_ASSERT_NOT_REACHED();
     163    }
     164   
     165    static Kind varargsKindFor(CallMode callMode)
     166    {
     167        switch (callMode) {
     168        case CallMode::Regular:
     169            return CallVarargs;
     170        case CallMode::Construct:
     171            return ConstructVarargs;
     172        case CallMode::Tail:
     173            return TailCallVarargs;
     174        }
     175        RELEASE_ASSERT_NOT_REACHED();
     176    }
     177   
     178    static CodeSpecializationKind specializationKindFor(Kind kind)
     179    {
     180        switch (kind) {
     181        case Call:
     182        case CallVarargs:
     183        case TailCall:
     184        case TailCallVarargs:
    161185        case GetterCall:
    162186        case SetterCall:
    … …  
    167191        }
    168192        RELEASE_ASSERT_NOT_REACHED();
    169         return CodeForCall;
    170193    }
    171194   
    … …  
    175198        case CallVarargs:
    176199        case ConstructVarargs:
     200        case TailCallVarargs:
    177201            return true;
    178202        default:
    … …  
    184208        return isVarargs(static_cast<Kind>(kind));
    185209    }
     210
     211    static bool isTail(Kind kind)
     212    {
     213        switch (kind) {
     214        case TailCall:
     215        case TailCallVarargs:
     216            return true;
     217        default:
     218            return false;
     219        }
     220    }
     221    bool isTail() const
     222    {
     223        return isTail(static_cast<Kind>(kind));
     224    }
     225
     226    static CodeOrigin* computeCallerSkippingDeadFrames(InlineCallFrame* inlineCallFrame)
     227    {
     228        CodeOrigin* codeOrigin;
     229        bool tailCallee;
     230        do {
     231            tailCallee = inlineCallFrame->isTail();
     232            codeOrigin = &inlineCallFrame->directCaller;
     233            inlineCallFrame = codeOrigin->inlineCallFrame;
     234        } while (inlineCallFrame && tailCallee);
     235        if (tailCallee)
     236            return nullptr;
     237        return codeOrigin;
     238    }
     239    CodeOrigin* getCallerSkippingDeadFrames()
     240    {
     241        return computeCallerSkippingDeadFrames(this);
     242    }
     243    InlineCallFrame* getCallerInlineFrameSkippingDeadFrames()
     244    {
     245        CodeOrigin* caller = getCallerSkippingDeadFrames();
     246        return caller ? caller->inlineCallFrame : nullptr;
     247    }
    186248   
    187249    Vector<ValueRecovery> arguments; // Includes 'this'.
    188250    WriteBarrier<ScriptExecutable> executable;
    189251    ValueRecovery calleeRecovery;
    190     CodeOrigin caller;
     252    CodeOrigin directCaller;
    191253
    192254    signed stackOffset : 28;

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h

    r185239 r187791  
    14431443        m_state.setIsValid(false);
    14441444        break;
     1445
     1446    case TailCall:
     1447    case TailCallVarargs:
     1448    case TailCallForwardVarargs:
     1449        clobberWorld(node->origin.semantic, clobberLimit);
     1450        m_state.setIsValid(false);
     1451        break;
    14451452       
    14461453    case Throw:
    … …  
    22542261           
    22552262    case Call:
     2263    case TailCallInlinedCaller:
    22562264    case Construct:
    22572265    case NativeCall:
    … …  
    22592267    case CallVarargs:
    22602268    case CallForwardVarargs:
     2269    case TailCallVarargsInlinedCaller:
    22612270    case ConstructVarargs:
    22622271    case ConstructForwardVarargs:
     2272    case TailCallForwardVarargsInlinedCaller:
    22632273        clobberWorld(node->origin.semantic, clobberLimit);
    22642274        forNode(node).makeHeapTop();

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGArgumentsEliminationPhase.cpp

    r184781 r187791  
    168168                case CallVarargs:
    169169                case ConstructVarargs:
     170                case TailCallVarargs:
     171                case TailCallVarargsInlinedCaller:
    170172                    escape(node->child1());
    171173                    escape(node->child3());
    … …  
    554556                   
    555557                case CallVarargs:
    556                 case ConstructVarargs: {
     558                case ConstructVarargs:
     559                case TailCallVarargs:
     560                case TailCallVarargsInlinedCaller: {
    557561                    Node* candidate = node->child2().node();
    558562                    if (!m_candidates.contains(candidate))
    … …  
    579583                        for (Node* argument : arguments)
    580584                            m_graph.m_varArgChildren.append(Edge(argument));
    581                         node->setOpAndDefaultFlags(
    582                             node->op() == CallVarargs ? Call : Construct);
     585                        switch (node->op()) {
     586                        case CallVarargs:
     587                            node->setOpAndDefaultFlags(Call);
     588                            break;
     589                        case ConstructVarargs:
     590                            node->setOpAndDefaultFlags(Construct);
     591                            break;
     592                        case TailCallVarargs:
     593                            node->setOpAndDefaultFlags(TailCall);
     594                            break;
     595                        case TailCallVarargsInlinedCaller:
     596                            node->setOpAndDefaultFlags(TailCallInlinedCaller);
     597                            break;
     598                        default:
     599                            RELEASE_ASSERT_NOT_REACHED();
     600                        }
    583601                        node->children = AdjacencyList(
    584602                            AdjacencyList::Variable,
    … …  
    587605                    }
    588606                   
    589                     node->setOpAndDefaultFlags(
    590                         node->op() == CallVarargs ? CallForwardVarargs : ConstructForwardVarargs);
     607                    switch (node->op()) {
     608                    case CallVarargs:
     609                        node->setOpAndDefaultFlags(CallForwardVarargs);
     610                        break;
     611                    case ConstructVarargs:
     612                        node->setOpAndDefaultFlags(ConstructForwardVarargs);
     613                        break;
     614                    case TailCallVarargs:
     615                        node->setOpAndDefaultFlags(TailCallForwardVarargs);
     616                        break;
     617                    case TailCallVarargsInlinedCaller:
     618                        node->setOpAndDefaultFlags(TailCallForwardVarargsInlinedCaller);
     619                        break;
     620                    default:
     621                        RELEASE_ASSERT_NOT_REACHED();
     622                    }
    591623                    break;
    592624                }

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGBasicBlock.h

    r183497 r187791  
    9393            case Switch:
    9494            case Return:
     95            case TailCall:
     96            case TailCallVarargs:
    9597            case Unreachable:
    9698                return NodeAndIndex(node, i);

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

    r187767 r187791  
    181181        SpeculatedType prediction);
    182182    void handleCall(
    183         int result, NodeType op, InlineCallFrame::Kind, unsigned instructionSize,
     183        int result, NodeType op, CallMode, unsigned instructionSize,
    184184        Node* callTarget, int argCount, int registerOffset, CallLinkStatus);
    185     void handleCall(int result, NodeType op, CodeSpecializationKind, unsigned instructionSize, int callee, int argCount, int registerOffset);
    186     void handleCall(Instruction* pc, NodeType op, CodeSpecializationKind);
    187     void handleVarargsCall(Instruction* pc, NodeType op, CodeSpecializationKind);
     185    void handleCall(int result, NodeType op, CallMode, unsigned instructionSize, int callee, int argCount, int registerOffset);
     186    void handleCall(Instruction* pc, NodeType op, CallMode);
     187    void handleVarargsCall(Instruction* pc, NodeType op, CallMode);
    188188    void emitFunctionChecks(CallVariant, Node* callTarget, VirtualRegister thisArgumnt);
    189189    void emitArgumentPhantoms(int registerOffset, int argumentCountIncludingThis);
    190     unsigned inliningCost(CallVariant, int argumentCountIncludingThis, CodeSpecializationKind); // Return UINT_MAX if it's not an inlining candidate. By convention, intrinsics have a cost of 1.
     190    unsigned inliningCost(CallVariant, int argumentCountIncludingThis, CallMode); // Return UINT_MAX if it's not an inlining candidate. By convention, intrinsics have a cost of 1.
    191191    // Handle inlining. Return true if it succeeded, false if we need to plant a call.
    192192    bool handleInlining(Node* callTargetNode, int resultOperand, const CallLinkStatus&, int registerOffset, VirtualRegister thisArgument, VirtualRegister argumentsArgument, unsigned argumentsOffset, int argumentCountIncludingThis, unsigned nextOffset, NodeType callOp, InlineCallFrame::Kind, SpeculatedType prediction);
    … …  
    601601    }
    602602
     603    bool allInlineFramesAreTailCalls()
     604    {
     605        return !inlineCallFrame() || !inlineCallFrame()->getCallerSkippingDeadFrames();
     606    }
     607
    603608    CodeOrigin currentCodeOrigin()
    604609    {
    … …  
    681686    Node* addCallWithoutSettingResult(
    682687        NodeType op, OpInfo opInfo, Node* callee, int argCount, int registerOffset,
    683         SpeculatedType prediction)
     688        OpInfo prediction)
    684689    {
    685690        addVarArgChild(callee);
    … …  
    694699            addVarArgChild(get(virtualRegisterForArgument(i, registerOffset)));
    695700
    696         return addToGraph(Node::VarArg, op, opInfo, OpInfo(prediction));
     701        return addToGraph(Node::VarArg, op, opInfo, prediction);
    697702    }
    698703   
    … …  
    701706        SpeculatedType prediction)
    702707    {
     708        if (op == TailCall) {
     709            if (allInlineFramesAreTailCalls())
     710                return addCallWithoutSettingResult(op, OpInfo(), callee, argCount, registerOffset, OpInfo());
     711            op = TailCallInlinedCaller;
     712        }
     713
     714
    703715        Node* call = addCallWithoutSettingResult(
    704             op, opInfo, callee, argCount, registerOffset, prediction);
     716            op, opInfo, callee, argCount, registerOffset, OpInfo(prediction));
    705717        VirtualRegister resultReg(result);
    706718        if (resultReg.isValid())
    … …  
    718730    SpeculatedType getPredictionWithoutOSRExit(unsigned bytecodeIndex)
    719731    {
    720         ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
    721         return m_inlineStackTop->m_profiledBlock->valueProfilePredictionForBytecodeOffset(locker, bytecodeIndex);
     732        SpeculatedType prediction;
     733        CodeBlock* profiledBlock = nullptr;
     734
     735        {
     736            ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock);
     737            prediction = m_inlineStackTop->m_profiledBlock->valueProfilePredictionForBytecodeOffset(locker, bytecodeIndex);
     738
     739            if (prediction == SpecNone) {
     740                // If we have no information about the values this
     741                // node generates, we check if by any chance it is
     742                // a tail call opcode. In that case, we walk up the
     743                // inline frames to find a call higher in the call
     744                // chain and use its prediction. If we only have
     745                // inlined tail call frames, we use SpecFullTop
     746                // to avoid a spurious OSR exit.
     747                Instruction* instruction = m_inlineStackTop->m_profiledBlock->instructions().begin() + bytecodeIndex;
     748                OpcodeID opcodeID = m_vm->interpreter->getOpcodeID(instruction->u.opcode);
     749
     750                switch (opcodeID) {
     751                case op_tail_call:
     752                case op_tail_call_varargs: {
     753                    if (!inlineCallFrame()) {
     754                        prediction = SpecFullTop;
     755                        break;
     756                    }
     757                    CodeOrigin* codeOrigin = inlineCallFrame()->getCallerSkippingDeadFrames();
     758                    if (!codeOrigin) {
     759                        prediction = SpecFullTop;
     760                        break;
     761                    }
     762                    InlineStackEntry* stack = m_inlineStackTop;
     763                    while (stack->m_inlineCallFrame != codeOrigin->inlineCallFrame)
     764                        stack = stack->m_caller;
     765                    bytecodeIndex = codeOrigin->bytecodeIndex;
     766                    profiledBlock = stack->m_profiledBlock;
     767                    break;
     768                }
     769
     770                default:
     771                    break;
     772                }
     773            }
     774        }
     775
     776        if (profiledBlock) {
     777            ConcurrentJITLocker locker(profiledBlock->m_lock);
     778            prediction = profiledBlock->valueProfilePredictionForBytecodeOffset(locker, bytecodeIndex);
     779        }
     780
     781        return prediction;
    722782    }
    723783
    … …  
    725785    {
    726786        SpeculatedType prediction = getPredictionWithoutOSRExit(bytecodeIndex);
    727        
     787
    728788        if (prediction == SpecNone) {
    729789            // We have no information about what values this node generates. Give up
    … …  
    10151075    return shouldContinueParsing
    10161076
    1017 void ByteCodeParser::handleCall(Instruction* pc, NodeType op, CodeSpecializationKind kind)
     1077void ByteCodeParser::handleCall(Instruction* pc, NodeType op, CallMode callMode)
    10181078{
    10191079    ASSERT(OPCODE_LENGTH(op_call) == OPCODE_LENGTH(op_construct));
     1080    ASSERT(OPCODE_LENGTH(op_call) == OPCODE_LENGTH(op_tail_call));
    10201081    handleCall(
    1021         pc[1].u.operand, op, kind, OPCODE_LENGTH(op_call),
     1082        pc[1].u.operand, op, callMode, OPCODE_LENGTH(op_call),
    10221083        pc[2].u.operand, pc[3].u.operand, -pc[4].u.operand);
    10231084}
    10241085
    10251086void ByteCodeParser::handleCall(
    1026     int result, NodeType op, CodeSpecializationKind kind, unsigned instructionSize,
     1087    int result, NodeType op, CallMode callMode, unsigned instructionSize,
    10271088    int callee, int argumentCountIncludingThis, int registerOffset)
    10281089{
    … …  
    10341095   
    10351096    handleCall(
    1036         result, op, InlineCallFrame::kindFor(kind), instructionSize, callTarget,
     1097        result, op, callMode, instructionSize, callTarget,
    10371098        argumentCountIncludingThis, registerOffset, callLinkStatus);
    10381099}
    10391100   
    10401101void ByteCodeParser::handleCall(
    1041     int result, NodeType op, InlineCallFrame::Kind kind, unsigned instructionSize,
     1102    int result, NodeType op, CallMode callMode, unsigned instructionSize,
    10421103    Node* callTarget, int argumentCountIncludingThis, int registerOffset,
    10431104    CallLinkStatus callLinkStatus)
    10441105{
    10451106    handleCall(
    1046         result, op, kind, instructionSize, callTarget, argumentCountIncludingThis,
     1107        result, op, InlineCallFrame::kindFor(callMode), instructionSize, callTarget, argumentCountIncludingThis,
    10471108        registerOffset, callLinkStatus, getPrediction());
    10481109}
    … …  
    10641125        // Oddly, this conflates calls that haven't executed with calls that behaved sufficiently polymorphically
    10651126        // that we cannot optimize them.
    1066        
     1127
    10671128        addCall(result, op, OpInfo(), callTarget, argumentCountIncludingThis, registerOffset, prediction);
    10681129        return;
    … …  
    10971158    }
    10981159#endif
    1099    
     1160
    11001161    addCall(result, op, callOpInfo, callTarget, argumentCountIncludingThis, registerOffset, prediction);
    11011162}
    11021163
    1103 void ByteCodeParser::handleVarargsCall(Instruction* pc, NodeType op, CodeSpecializationKind kind)
     1164void ByteCodeParser::handleVarargsCall(Instruction* pc, NodeType op, CallMode callMode)
    11041165{
    11051166    ASSERT(OPCODE_LENGTH(op_call_varargs) == OPCODE_LENGTH(op_construct_varargs));
     1167    ASSERT(OPCODE_LENGTH(op_call_varargs) == OPCODE_LENGTH(op_tail_call_varargs));
    11061168   
    11071169    int result = pc[1].u.operand;
    … …  
    11261188   
    11271189    if (callLinkStatus.canOptimize()
    1128         && handleInlining(callTarget, result, callLinkStatus, firstFreeReg, VirtualRegister(thisReg), VirtualRegister(arguments), firstVarArgOffset, 0, m_currentIndex + OPCODE_LENGTH(op_call_varargs), op, InlineCallFrame::varargsKindFor(kind), prediction)) {
     1190        && handleInlining(callTarget, result, callLinkStatus, firstFreeReg, VirtualRegister(thisReg), VirtualRegister(arguments), firstVarArgOffset, 0, m_currentIndex + OPCODE_LENGTH(op_call_varargs), op, InlineCallFrame::varargsKindFor(callMode), prediction)) {
    11291191        if (m_graph.compilation())
    11301192            m_graph.compilation()->noticeInlinedCall();
    … …  
    11361198   
    11371199    Node* thisChild = get(VirtualRegister(thisReg));
    1138    
     1200
     1201    if (op == TailCallVarargs) {
     1202        if (allInlineFramesAreTailCalls()) {
     1203            addToGraph(op, OpInfo(data), OpInfo(), callTarget, get(VirtualRegister(arguments)), thisChild);
     1204            return;
     1205        }
     1206        op = TailCallVarargsInlinedCaller;
     1207    }
     1208
    11391209    Node* call = addToGraph(op, OpInfo(data), OpInfo(prediction), callTarget, get(VirtualRegister(arguments)), thisChild);
    11401210    VirtualRegister resultReg(result);
    … …  
    11711241}
    11721242
    1173 unsigned ByteCodeParser::inliningCost(CallVariant callee, int argumentCountIncludingThis, CodeSpecializationKind kind)
     1243unsigned ByteCodeParser::inliningCost(CallVariant callee, int argumentCountIncludingThis, CallMode callMode)
    11741244{
     1245    CodeSpecializationKind kind = specializationKindFor(callMode);
    11751246    if (verbose)
    11761247        dataLog("Considering inlining ", callee, " into ", currentCodeOrigin(), "\n");
    … …  
    12141285        codeBlock, kind, callee.isClosureCall());
    12151286    if (verbose) {
    1216         dataLog("    Kind: ", kind, "\n");
     1287        dataLog("    Call mode: ", callMode, "\n");
    12171288        dataLog("    Is closure call: ", callee.isClosureCall(), "\n");
    12181289        dataLog("    Capability level: ", capabilityLevel, "\n");
    … …  
    12851356    CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
    12861357   
    1287     ASSERT(inliningCost(callee, argumentCountIncludingThis, specializationKind) != UINT_MAX);
     1358    ASSERT(inliningCost(callee, argumentCountIncludingThis, InlineCallFrame::callModeFor(kind)) != UINT_MAX);
    12881359   
    12891360    CodeBlock* codeBlock = callee.functionExecutable()->baselineCodeBlockFor(specializationKind);
    … …  
    13801451        return;
    13811452    }
    1382    
     1453
    13831454    if (Options::verboseDFGByteCodeParsing())
    13841455        dataLog("    Creating new block after inlining.\n");
    … …  
    14871558    }
    14881559   
    1489     unsigned myInliningCost = inliningCost(callee, argumentCountIncludingThis, specializationKind);
     1560    unsigned myInliningCost = inliningCost(callee, argumentCountIncludingThis, InlineCallFrame::callModeFor(kind));
    14901561    if (myInliningCost > inliningBalance)
    14911562        return false;
    … …  
    33553426
    33563427        case op_ret:
     3428            if (m_currentBlock->terminal()) {
     3429                // We could be the dummy return after a non-inlined, non-emulated tail call
     3430                Node* terminal = m_currentBlock->terminal();
     3431                ASSERT_UNUSED(terminal, terminal->op() == Return || terminal->op() == TailCall || terminal->op() == TailCallVarargs);
     3432                LAST_OPCODE(op_ret);
     3433            }
    33573434            if (inlineCallFrame()) {
    33583435                flushForReturn();
    … …  
    34023479           
    34033480        case op_call:
    3404             handleCall(currentInstruction, Call, CodeForCall);
     3481            handleCall(currentInstruction, Call, CallMode::Regular);
    34053482            // Verify that handleCall(), which could have inlined the callee, didn't trash m_currentInstruction
    34063483            ASSERT(m_currentInstruction == currentInstruction);
    34073484            NEXT_OPCODE(op_call);
    3408            
     3485
     3486        case op_tail_call:
     3487            flushForReturn();
     3488            handleCall(currentInstruction, TailCall, CallMode::Tail);
     3489            // Verify that handleCall(), which could have inlined the callee, didn't trash m_currentInstruction
     3490            ASSERT(m_currentInstruction == currentInstruction);
     3491            // We let the following op_ret handle cases related to
     3492            // inlining to keep things simple.
     3493            NEXT_OPCODE(op_tail_call);
     3494
    34093495        case op_construct:
    3410             handleCall(currentInstruction, Construct, CodeForConstruct);
     3496            handleCall(currentInstruction, Construct, CallMode::Construct);
    34113497            NEXT_OPCODE(op_construct);
    34123498           
    34133499        case op_call_varargs: {
    3414             handleVarargsCall(currentInstruction, CallVarargs, CodeForCall);
     3500            handleVarargsCall(currentInstruction, CallVarargs, CallMode::Regular);
    34153501            NEXT_OPCODE(op_call_varargs);
    34163502        }
     3503
     3504        case op_tail_call_varargs: {
     3505            flushForReturn();
     3506            handleVarargsCall(currentInstruction, TailCallVarargs, CallMode::Tail);
     3507            NEXT_OPCODE(op_tail_call_varargs);
     3508        }
    34173509           
    34183510        case op_construct_varargs: {
    3419             handleVarargsCall(currentInstruction, ConstructVarargs, CodeForConstruct);
     3511            handleVarargsCall(currentInstruction, ConstructVarargs, CallMode::Construct);
    34203512            NEXT_OPCODE(op_construct_varargs);
    34213513        }
    … …  
    40394131        } else
    40404132            m_inlineCallFrame->isClosureCall = true;
    4041         m_inlineCallFrame->caller = byteCodeParser->currentCodeOrigin();
     4133        m_inlineCallFrame->directCaller = byteCodeParser->currentCodeOrigin();
    40424134        m_inlineCallFrame->arguments.resizeToFit(argumentCountIncludingThis); // Set the number of arguments including this, but don't configure the value recoveries, yet.
    40434135        m_inlineCallFrame->kind = kind;
    … …  
    41144206        Vector<DeferredSourceDump>& deferredSourceDump = m_graph.m_plan.callback->ensureDeferredSourceDump();
    41154207        if (inlineCallFrame()) {
    4116             DeferredSourceDump dump(codeBlock->baselineVersion(), m_codeBlock, JITCode::DFGJIT, inlineCallFrame()->caller);
     4208            DeferredSourceDump dump(codeBlock->baselineVersion(), m_codeBlock, JITCode::DFGJIT, inlineCallFrame()->directCaller);
    41174209            deferredSourceDump.append(dump);
    41184210        } else
    … …  
    41254217            dataLog(
    41264218                " for inlining at ", CodeBlockWithJITType(m_codeBlock, JITCode::DFGJIT),
    4127                 " ", inlineCallFrame()->caller);
     4219                " ", inlineCallFrame()->directCaller);
    41284220        }
    41294221        dataLog(
    … …  
    42094301            }
    42104302           
    4211             m_currentBlock = 0;
     4303            m_currentBlock = nullptr;
    42124304        } while (m_currentIndex < limit);
    42134305    }

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGCapabilities.cpp

    r183373 r187791  
    180180    case op_throw_static_error:
    181181    case op_call:
     182    case op_tail_call:
    182183    case op_construct:
    183184    case op_call_varargs:
     185    case op_tail_call_varargs:
    184186    case op_construct_varargs:
    185187    case op_create_direct_arguments:

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGClobberize.h

    r185566 r187791  
    377377    case ArrayPop:
    378378    case Call:
     379    case TailCallInlinedCaller:
    379380    case Construct:
    380381    case NativeCall:
    … …  
    382383    case CallVarargs:
    383384    case CallForwardVarargs:
     385    case TailCallVarargsInlinedCaller:
     386    case TailCallForwardVarargsInlinedCaller:
    384387    case ConstructVarargs:
    385388    case ConstructForwardVarargs:
    … …  
    389392        read(World);
    390393        write(Heap);
     394        return;
     395
     396    case TailCall:
     397    case TailCallVarargs:
     398    case TailCallForwardVarargs:
     399        read(World);
     400        write(SideState);
    391401        return;
    392402       

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGDoesGC.cpp

    r184511 r187791  
    118118    case CompareStrictEq:
    119119    case Call:
     120    case TailCallInlinedCaller:
    120121    case Construct:
    121122    case CallVarargs:
     123    case TailCallVarargsInlinedCaller:
    122124    case ConstructVarargs:
    123125    case LoadVarargs:
    124126    case CallForwardVarargs:
    125127    case ConstructForwardVarargs:
     128    case TailCallForwardVarargs:
     129    case TailCallForwardVarargsInlinedCaller:
    126130    case NativeCall:
    127131    case NativeConstruct:
    … …  
    150154    case Switch:
    151155    case Return:
     156    case TailCall:
     157    case TailCallVarargs:
    152158    case Throw:
    153159    case CountExecution:

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp

    r185920 r187791  
    12791279        case VarInjectionWatchpoint:
    12801280        case Call:
     1281        case TailCallInlinedCaller:
    12811282        case Construct:
    12821283        case CallVarargs:
     1284        case TailCallVarargsInlinedCaller:
    12831285        case ConstructVarargs:
    12841286        case CallForwardVarargs:
    12851287        case ConstructForwardVarargs:
     1288        case TailCallForwardVarargs:
     1289        case TailCallForwardVarargsInlinedCaller:
    12861290        case LoadVarargs:
    12871291        case ProfileControlFlow:
    … …  
    13031307        case Jump:
    13041308        case Return:
     1309        case TailCall:
     1310        case TailCallVarargs:
    13051311        case Throw:
    13061312        case ThrowReferenceError:

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGGraph.cpp

    r186215 r187791  
    885885bool Graph::isLiveInBytecode(VirtualRegister operand, CodeOrigin codeOrigin)
    886886{
     887    CodeOrigin* codeOriginPtr = &codeOrigin;
    887888    for (;;) {
    888889        VirtualRegister reg = VirtualRegister(
    889             operand.offset() - codeOrigin.stackOffset());
    890        
    891         if (operand.offset() < codeOrigin.stackOffset() + JSStack::CallFrameHeaderSize) {
     890            operand.offset() - codeOriginPtr->stackOffset());
     891       
     892        if (operand.offset() < codeOriginPtr->stackOffset() + JSStack::CallFrameHeaderSize) {
    892893            if (reg.isArgument()) {
    893894                RELEASE_ASSERT(reg.offset() < JSStack::CallFrameHeaderSize);
    894895               
    895                 if (codeOrigin.inlineCallFrame->isClosureCall
     896                if (codeOriginPtr->inlineCallFrame->isClosureCall
    896897                    && reg.offset() == JSStack::Callee)
    897898                    return true;
    898899               
    899                 if (codeOrigin.inlineCallFrame->isVarargs()
     900                if (codeOriginPtr->inlineCallFrame->isVarargs()
    900901                    && reg.offset() == JSStack::ArgumentCount)
    901902                    return true;
    … …  
    904905            }
    905906           
    906             return livenessFor(codeOrigin.inlineCallFrame).operandIsLive(
    907                 reg.offset(), codeOrigin.bytecodeIndex);
    908         }
    909        
    910         InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame;
     907            return livenessFor(codeOriginPtr->inlineCallFrame).operandIsLive(
     908                reg.offset(), codeOriginPtr->bytecodeIndex);
     909        }
     910       
     911        InlineCallFrame* inlineCallFrame = codeOriginPtr->inlineCallFrame;
    911912        if (!inlineCallFrame)
    912913            break;
    … …  
    918919            return true;
    919920       
    920         codeOrigin = inlineCallFrame->caller;
     921        codeOriginPtr = inlineCallFrame->getCallerSkippingDeadFrames();
     922
     923        // The first inline call frame could be an inline tail call
     924        if (!codeOriginPtr)
     925            break;
    921926    }
    922927   

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGGraph.h

    r186279 r187791  
    697697        VirtualRegister exclusionStart;
    698698        VirtualRegister exclusionEnd;
     699
     700        CodeOrigin* codeOriginPtr = &codeOrigin;
    699701       
    700702        for (;;) {
    701             InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame;
     703            InlineCallFrame* inlineCallFrame = codeOriginPtr->inlineCallFrame;
    702704            VirtualRegister stackOffset(inlineCallFrame ? inlineCallFrame->stackOffset : 0);
    703705           
    … …  
    711713            CodeBlock* codeBlock = baselineCodeBlockFor(inlineCallFrame);
    712714            FullBytecodeLiveness& fullLiveness = livenessFor(codeBlock);
    713             const FastBitVector& liveness = fullLiveness.getLiveness(codeOrigin.bytecodeIndex);
     715            const FastBitVector& liveness = fullLiveness.getLiveness(codeOriginPtr->bytecodeIndex);
    714716            for (unsigned relativeLocal = codeBlock->m_numCalleeRegisters; relativeLocal--;) {
    715717                VirtualRegister reg = stackOffset + virtualRegisterForLocal(relativeLocal);
    … …  
    738740                functor(reg);
    739741           
    740             codeOrigin = inlineCallFrame->caller;
     742            codeOriginPtr = inlineCallFrame->getCallerSkippingDeadFrames();
     743
     744            // The first inline call frame could be an inline tail call
     745            if (!codeOriginPtr)
     746                break;
    741747        }
    742748    }

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGInPlaceAbstractState.cpp

    r184318 r187791  
    394394       
    395395    case Return:
     396    case TailCall:
     397    case TailCallVarargs:
    396398    case Unreachable:
    397399        ASSERT(basicBlock->cfaBranchDirection == InvalidBranchDirection);

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGNode.h

    r186136 r187791  
    996996        case CallVarargs:
    997997        case CallForwardVarargs:
     998        case TailCallVarargs:
     999        case TailCallForwardVarargs:
     1000        case TailCallVarargsInlinedCaller:
     1001        case TailCallForwardVarargsInlinedCaller:
    9981002        case ConstructVarargs:
    9991003        case ConstructForwardVarargs:
    … …  
    10931097        case Switch:
    10941098        case Return:
     1099        case TailCall:
     1100        case TailCallVarargs:
    10951101        case Unreachable:
    10961102            return true;
    … …  
    12431249        case GetByVal:
    12441250        case Call:
     1251        case TailCallInlinedCaller:
    12451252        case Construct:
    12461253        case CallVarargs:
     1254        case TailCallVarargsInlinedCaller:
    12471255        case ConstructVarargs:
    12481256        case CallForwardVarargs:
     1257        case TailCallForwardVarargsInlinedCaller:
    12491258        case NativeCall:
    12501259        case NativeConstruct:

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGNodeType.h

    r184511 r187791  
    240240    macro(ConstructVarargs, NodeResultJS | NodeMustGenerate) \
    241241    macro(ConstructForwardVarargs, NodeResultJS | NodeMustGenerate) \
     242    macro(TailCallInlinedCaller, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
     243    macro(TailCallVarargsInlinedCaller, NodeResultJS | NodeMustGenerate) \
     244    macro(TailCallForwardVarargsInlinedCaller, NodeResultJS | NodeMustGenerate) \
    242245    macro(NativeCall, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
    243246    macro(NativeConstruct, NodeResultJS | NodeMustGenerate | NodeHasVarArgs) \
    … …  
    305308    macro(Switch, NodeMustGenerate) \
    306309    macro(Return, NodeMustGenerate) \
     310    macro(TailCall, NodeMustGenerate | NodeHasVarArgs) \
     311    macro(TailCallVarargs, NodeMustGenerate) \
     312    macro(TailCallForwardVarargs, NodeMustGenerate) \
    307313    macro(Unreachable, NodeMustGenerate) \
    308314    \

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp

    r187639 r187791  
    6565    AssemblyHelpers::JumpList loopThreshold;
    6666   
    67     for (InlineCallFrame* inlineCallFrame = exit.m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->caller.inlineCallFrame) {
     67    for (InlineCallFrame* inlineCallFrame = exit.m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->directCaller.inlineCallFrame) {
    6868        loopThreshold.append(
    6969            jit.branchTest8(
    … …  
    137137void reifyInlinedCallFrames(CCallHelpers& jit, const OSRExitBase& exit)
    138138{
     139    // FIXME: We shouldn't leave holes on the stack when performing an OSR exit
     140    // in presence of inlined tail calls.
     141    // https://bugs.webkit.org/show_bug.cgi?id=147511
    139142    ASSERT(jit.baselineCodeBlock()->jitType() == JITCode::BaselineJIT);
    140143    jit.storePtr(AssemblyHelpers::TrustedImmPtr(jit.baselineCodeBlock()), AssemblyHelpers::addressFor((VirtualRegister)JSStack::CodeBlock));
    141144
    142     CodeOrigin codeOrigin;
    143     for (codeOrigin = exit.m_codeOrigin; codeOrigin.inlineCallFrame; codeOrigin = codeOrigin.inlineCallFrame->caller) {
    144         InlineCallFrame* inlineCallFrame = codeOrigin.inlineCallFrame;
    145         CodeBlock* baselineCodeBlock = jit.baselineCodeBlockFor(codeOrigin);
    146         CodeBlock* baselineCodeBlockForCaller = jit.baselineCodeBlockFor(inlineCallFrame->caller);
    147         void* jumpTarget = nullptr;
     145    const CodeOrigin* codeOrigin;
     146    for (codeOrigin = &exit.m_codeOrigin; codeOrigin && codeOrigin->inlineCallFrame; codeOrigin = codeOrigin->inlineCallFrame->getCallerSkippingDeadFrames()) {
     147        InlineCallFrame* inlineCallFrame = codeOrigin->inlineCallFrame;
     148        CodeBlock* baselineCodeBlock = jit.baselineCodeBlockFor(*codeOrigin);
     149        CodeOrigin* trueCaller = inlineCallFrame->getCallerSkippingDeadFrames();
    148150        void* trueReturnPC = nullptr;
    149        
    150         unsigned callBytecodeIndex = inlineCallFrame->caller.bytecodeIndex;
    151        
    152         switch (inlineCallFrame->kind) {
    153         case InlineCallFrame::Call:
    154         case InlineCallFrame::Construct:
    155         case InlineCallFrame::CallVarargs:
    156         case InlineCallFrame::ConstructVarargs: {
    157             CallLinkInfo* callLinkInfo =
    158                 baselineCodeBlockForCaller->getCallLinkInfoForBytecodeIndex(callBytecodeIndex);
    159             RELEASE_ASSERT(callLinkInfo);
    160            
    161             jumpTarget = callLinkInfo->callReturnLocation().executableAddress();
    162             break;
    163         }
    164            
    165         case InlineCallFrame::GetterCall:
    166         case InlineCallFrame::SetterCall: {
    167             StructureStubInfo* stubInfo =
    168                 baselineCodeBlockForCaller->findStubInfo(CodeOrigin(callBytecodeIndex));
    169             RELEASE_ASSERT(stubInfo);
    170            
     151        GPRReg callerFrameGPR = GPRInfo::callFrameRegister;
     152
     153        if (!trueCaller) {
     154            ASSERT(inlineCallFrame->isTail());
     155            jit.loadPtr(AssemblyHelpers::Address(GPRInfo::callFrameRegister, CallFrame::returnPCOffset()), GPRInfo::regT3);
     156            jit.storePtr(GPRInfo::regT3, AssemblyHelpers::addressForByteOffset(inlineCallFrame->returnPCOffset()));
     157            jit.loadPtr(AssemblyHelpers::Address(GPRInfo::callFrameRegister, CallFrame::callerFrameOffset()), GPRInfo::regT3);
     158            callerFrameGPR = GPRInfo::regT3;
     159        } else {
     160            CodeBlock* baselineCodeBlockForCaller = jit.baselineCodeBlockFor(*trueCaller);
     161            unsigned callBytecodeIndex = trueCaller->bytecodeIndex;
     162            void* jumpTarget = nullptr;
     163
    171164            switch (inlineCallFrame->kind) {
    172             case InlineCallFrame::GetterCall:
    173                 jumpTarget = jit.vm()->getCTIStub(baselineGetterReturnThunkGenerator).code().executableAddress();
    174                 break;
    175             case InlineCallFrame::SetterCall:
    176                 jumpTarget = jit.vm()->getCTIStub(baselineSetterReturnThunkGenerator).code().executableAddress();
    177                 break;
    178             default:
    179                 RELEASE_ASSERT_NOT_REACHED();
     165            case InlineCallFrame::Call:
     166            case InlineCallFrame::Construct:
     167            case InlineCallFrame::CallVarargs:
     168            case InlineCallFrame::ConstructVarargs:
     169            case InlineCallFrame::TailCall:
     170            case InlineCallFrame::TailCallVarargs: {
     171                CallLinkInfo* callLinkInfo =
     172                    baselineCodeBlockForCaller->getCallLinkInfoForBytecodeIndex(callBytecodeIndex);
     173                RELEASE_ASSERT(callLinkInfo);
     174
     175                jumpTarget = callLinkInfo->callReturnLocation().executableAddress();
    180176                break;
    181177            }
    182            
    183             trueReturnPC = stubInfo->callReturnLocation.labelAtOffset(
    184                 stubInfo->patch.deltaCallToDone).executableAddress();
    185             break;
    186         } }
    187 
    188         GPRReg callerFrameGPR;
    189         if (inlineCallFrame->caller.inlineCallFrame) {
    190             jit.addPtr(AssemblyHelpers::TrustedImm32(inlineCallFrame->caller.inlineCallFrame->stackOffset * sizeof(EncodedJSValue)), GPRInfo::callFrameRegister, GPRInfo::regT3);
    191             callerFrameGPR = GPRInfo::regT3;
    192         } else
    193             callerFrameGPR = GPRInfo::callFrameRegister;
    194        
    195         jit.storePtr(AssemblyHelpers::TrustedImmPtr(jumpTarget), AssemblyHelpers::addressForByteOffset(inlineCallFrame->returnPCOffset()));
     178
     179            case InlineCallFrame::GetterCall:
     180            case InlineCallFrame::SetterCall: {
     181                StructureStubInfo* stubInfo =
     182                    baselineCodeBlockForCaller->findStubInfo(CodeOrigin(callBytecodeIndex));
     183                RELEASE_ASSERT(stubInfo);
     184
     185                switch (inlineCallFrame->kind) {
     186                case InlineCallFrame::GetterCall:
     187                    jumpTarget = jit.vm()->getCTIStub(baselineGetterReturnThunkGenerator).code().executableAddress();
     188                    break;
     189                case InlineCallFrame::SetterCall:
     190                    jumpTarget = jit.vm()->getCTIStub(baselineSetterReturnThunkGenerator).code().executableAddress();
     191                    break;
     192                default:
     193                    RELEASE_ASSERT_NOT_REACHED();
     194                    break;
     195                }
     196
     197                trueReturnPC = stubInfo->callReturnLocation.labelAtOffset(
     198                    stubInfo->patch.deltaCallToDone).executableAddress();
     199                break;
     200            } }
     201
     202            if (trueCaller->inlineCallFrame) {
     203                jit.addPtr(
     204                    AssemblyHelpers::TrustedImm32(trueCaller->inlineCallFrame->stackOffset * sizeof(EncodedJSValue)),
     205                    GPRInfo::callFrameRegister,
     206                    GPRInfo::regT3);
     207                callerFrameGPR = GPRInfo::regT3;
     208            }
     209
     210            jit.storePtr(AssemblyHelpers::TrustedImmPtr(jumpTarget), AssemblyHelpers::addressForByteOffset(inlineCallFrame->returnPCOffset()));
     211        }
     212
    196213        if (trueReturnPC)
    197214            jit.storePtr(AssemblyHelpers::TrustedImmPtr(trueReturnPC), AssemblyHelpers::addressFor(inlineCallFrame->stackOffset + virtualRegisterForArgument(inlineCallFrame->arguments.size()).offset()));
    … …  
    203220#if USE(JSVALUE64)
    204221        jit.store64(callerFrameGPR, AssemblyHelpers::addressForByteOffset(inlineCallFrame->callerFrameOffset()));
    205         uint32_t locationBits = CallFrame::Location::encodeAsBytecodeOffset(codeOrigin.bytecodeIndex);
     222        uint32_t locationBits = CallFrame::Location::encodeAsBytecodeOffset(codeOrigin->bytecodeIndex);
    206223        jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(inlineCallFrame->stackOffset + JSStack::ArgumentCount)));
    207224        if (!inlineCallFrame->isClosureCall)
    … …  
    209226#else // USE(JSVALUE64) // so this is the 32-bit part
    210227        jit.storePtr(callerFrameGPR, AssemblyHelpers::addressForByteOffset(inlineCallFrame->callerFrameOffset()));
    211         Instruction* instruction = baselineCodeBlock->instructions().begin() + codeOrigin.bytecodeIndex;
     228        Instruction* instruction = baselineCodeBlock->instructions().begin() + codeOrigin->bytecodeIndex;
    212229        uint32_t locationBits = CallFrame::Location::encodeAsBytecodeInstruction(instruction);
    213230        jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(inlineCallFrame->stackOffset + JSStack::ArgumentCount)));
    … …  
    218235    }
    219236
     237    // Don't need to set the toplevel code origin if we only did inline tail calls
     238    if (codeOrigin) {
    220239#if USE(JSVALUE64)
    221     uint32_t locationBits = CallFrame::Location::encodeAsBytecodeOffset(codeOrigin.bytecodeIndex);
     240        uint32_t locationBits = CallFrame::Location::encodeAsBytecodeOffset(codeOrigin->bytecodeIndex);
    222241#else
    223     Instruction* instruction = jit.baselineCodeBlock()->instructions().begin() + codeOrigin.bytecodeIndex;
    224     uint32_t locationBits = CallFrame::Location::encodeAsBytecodeInstruction(instruction);
    225 #endif
    226     jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(JSStack::ArgumentCount)));
     242        Instruction* instruction = jit.baselineCodeBlock()->instructions().begin() + codeOrigin->bytecodeIndex;
     243        uint32_t locationBits = CallFrame::Location::encodeAsBytecodeInstruction(instruction);
     244#endif
     245        jit.store32(AssemblyHelpers::TrustedImm32(locationBits), AssemblyHelpers::tagFor((VirtualRegister)(JSStack::ArgumentCount)));
     246    }
    227247}
    228248

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGOSRExitPreparation.cpp

    r171362 r187791  
    4242    DeferGC deferGC(vm.heap);
    4343   
    44     for (; codeOrigin.inlineCallFrame; codeOrigin = codeOrigin.inlineCallFrame->caller) {
     44    for (; codeOrigin.inlineCallFrame; codeOrigin = codeOrigin.inlineCallFrame->directCaller) {
    4545        FunctionExecutable* executable =
    4646            static_cast<FunctionExecutable*>(codeOrigin.inlineCallFrame->executable.get());

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGOperations.cpp

    r186279 r187791  
    12771277   
    12781278    bool didTryToEnterIntoInlinedLoops = false;
    1279     for (InlineCallFrame* inlineCallFrame = exit->m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->caller.inlineCallFrame) {
     1279    for (InlineCallFrame* inlineCallFrame = exit->m_codeOrigin.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->directCaller.inlineCallFrame) {
    12801280        if (inlineCallFrame->executable->didTryToEnterInLoop()) {
    12811281            didTryToEnterIntoInlinedLoops = true;

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGPreciseLocalClobberize.h

    r184776 r187791  
    112112        case ForwardVarargs:
    113113        case CallForwardVarargs:
    114         case ConstructForwardVarargs: {
     114        case ConstructForwardVarargs:
     115        case TailCallForwardVarargs:
     116        case TailCallForwardVarargsInlinedCaller: {
    115117            InlineCallFrame* inlineCallFrame = m_node->child1()->origin.semantic.inlineCallFrame;
    116118            if (!inlineCallFrame) {
    … …  
    139141       
    140142            // Read all of the inline arguments and call frame headers that we didn't already capture.
    141             for (InlineCallFrame* inlineCallFrame = m_node->origin.semantic.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->caller.inlineCallFrame) {
     143            for (InlineCallFrame* inlineCallFrame = m_node->origin.semantic.inlineCallFrame; inlineCallFrame; inlineCallFrame = inlineCallFrame->getCallerInlineFrameSkippingDeadFrames()) {
    142144                for (unsigned i = inlineCallFrame->arguments.size(); i-- > 1;)
    143145                    m_read(VirtualRegister(inlineCallFrame->stackOffset + virtualRegisterForArgument(i).offset()));

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp

    r184933 r187791  
    195195        case GetDirectPname:
    196196        case Call:
     197        case TailCallInlinedCaller:
    197198        case Construct:
    198199        case CallVarargs:
     200        case TailCallVarargsInlinedCaller:
    199201        case ConstructVarargs:
    200202        case CallForwardVarargs:
    201203        case ConstructForwardVarargs:
     204        case TailCallForwardVarargsInlinedCaller:
    202205        case NativeCall:
    203206        case NativeConstruct:
    … …  
    631634        case PutToArguments:
    632635        case Return:
     636        case TailCall:
     637        case TailCallVarargs:
     638        case TailCallForwardVarargs:
    633639        case Throw:
    634640        case PutById:

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGSafeToExecute.h

    r185239 r187791  
    192192    case CompareStrictEq:
    193193    case Call:
     194    case TailCallInlinedCaller:
    194195    case Construct:
    195196    case CallVarargs:
     197    case TailCallVarargsInlinedCaller:
     198    case TailCallForwardVarargsInlinedCaller:
    196199    case ConstructVarargs:
    197200    case LoadVarargs:
    … …  
    236239    case Switch:
    237240    case Return:
     241    case TailCall:
     242    case TailCallVarargs:
     243    case TailCallForwardVarargs:
    238244    case Throw:
    239245    case ThrowReferenceError:

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp

    r187639 r187791  
    645645    bool isVarargs = false;
    646646    bool isForwardVarargs = false;
     647    bool isTail = false;
     648    bool isEmulatedTail = false;
    647649    switch (node->op()) {
    648650    case Call:
    649651        callType = CallLinkInfo::Call;
    650652        break;
     653    case TailCall:
     654        callType = CallLinkInfo::TailCall;
     655        isTail = true;
     656        break;
     657    case TailCallInlinedCaller:
     658        callType = CallLinkInfo::Call;
     659        isEmulatedTail = true;
     660        break;
    651661    case Construct:
    652662        callType = CallLinkInfo::Construct;
    … …  
    656666        isVarargs = true;
    657667        break;
     668    case TailCallVarargs:
     669        callType = CallLinkInfo::TailCallVarargs;
     670        isVarargs = true;
     671        isTail = true;
     672        break;
     673    case TailCallVarargsInlinedCaller:
     674        callType = CallLinkInfo::CallVarargs;
     675        isVarargs = true;
     676        isEmulatedTail = true;
     677        break;
    658678    case ConstructVarargs:
    659679        callType = CallLinkInfo::ConstructVarargs;
    … …  
    662682    case CallForwardVarargs:
    663683        callType = CallLinkInfo::CallVarargs;
     684        isForwardVarargs = true;
     685        break;
     686    case TailCallForwardVarargs:
     687        callType = CallLinkInfo::TailCallVarargs;
     688        isTail = true;
     689        isForwardVarargs = true;
     690        break;
     691    case TailCallForwardVarargsInlinedCaller:
     692        callType = CallLinkInfo::CallVarargs;
     693        isEmulatedTail = true;
    664694        isForwardVarargs = true;
    665695        break;
    … …  
    786816    m_jit.store32(calleeTagGPR, m_jit.calleeFrameTagSlot(JSStack::Callee));
    787817
    788     flushRegisters();
     818    // FIXME: We should do an efficient move of the arguments into
     819    // their target stack position instead of building then memmoving
     820    // the callee frame.
     821    // https://bugs.webkit.org/show_bug.cgi?id=147508
     822    if (isTail)
     823        ASSERT(isFlushed());
     824    else
     825        flushRegisters();
    789826
    790827    GPRFlushedCallResult resultPayload(this);
    … …  
    796833    JITCompiler::JumpList slowPath;
    797834
    798     m_jit.emitStoreCodeOrigin(node->origin.semantic);
     835    CodeOrigin staticOrigin = node->origin.semantic;
     836    ASSERT(!isTail || !staticOrigin.inlineCallFrame || !staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames());
     837    ASSERT(!isEmulatedTail || (staticOrigin.inlineCallFrame && staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames()));
     838    CodeOrigin dynamicOrigin =
     839        isEmulatedTail ? *staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames() : staticOrigin;
     840
     841    m_jit.emitStoreCodeOrigin(dynamicOrigin);
    799842   
    800843    CallLinkInfo* info = m_jit.codeBlock()->addCallLinkInfo();
    … …  
    803846    slowPath.append(m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleePayloadGPR, targetToCheck));
    804847
    805     JITCompiler::Call fastCall = m_jit.nearCall();
     848    if (isTail) {
     849        m_jit.emitRestoreCalleeSaves();
     850        // FIXME: We should do an efficient move of the arguments into
     851        // their target stack position instead of building then memmoving
     852        // the callee frame.
     853        // https://bugs.webkit.org/show_bug.cgi?id=147508
     854        m_jit.prepareForTailCallSlow();
     855    }
     856
     857    JITCompiler::Call fastCall = isTail ? m_jit.nearTailCall() : m_jit.nearCall();
    806858
    807859    JITCompiler::Jump done = m_jit.jump();
    … …  
    826878    done.link(&m_jit);
    827879
    828     m_jit.setupResults(resultPayloadGPR, resultTagGPR);
    829 
    830     jsValueResult(resultTagGPR, resultPayloadGPR, node, DataFormatJS, UseChildrenCalledExplicitly);
     880    if (isTail)
     881        m_jit.breakpoint();
     882    else {
     883        m_jit.setupResults(resultPayloadGPR, resultTagGPR);
     884
     885        jsValueResult(resultTagGPR, resultPayloadGPR, node, DataFormatJS, UseChildrenCalledExplicitly);
     886
     887        // After the calls are done, we need to reestablish our stack
     888        // pointer. We rely on this for varargs calls, calls with arity
     889        // mismatch (the callframe is slided) and tail calls.
     890        m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
     891    }
    831892
    832893    info->setUpCall(callType, node->origin.semantic, calleePayloadGPR);
    833894    m_jit.addJSCall(fastCall, slowCall, targetToCheck, info);
    834    
    835     // After the calls are done, we need to reestablish our stack
    836     // pointer. We rely on this for varargs calls, calls with arity
    837     // mismatch (the callframe is slided) and tail calls.
    838     m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
     895
    839896}
    840897
    … …  
    42204277
    42214278    case Call:
     4279    case TailCall:
     4280    case TailCallInlinedCaller:
    42224281    case Construct:
    42234282    case CallVarargs:
     4283    case TailCallVarargs:
     4284    case TailCallVarargsInlinedCaller:
     4285    case ConstructVarargs:
    42244286    case CallForwardVarargs:
    4225     case ConstructVarargs:
     4287    case TailCallForwardVarargs:
     4288    case TailCallForwardVarargsInlinedCaller:
    42264289    case ConstructForwardVarargs:
    42274290        emitCall(node);

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp

    r187639 r187791  
    631631    bool isVarargs = false;
    632632    bool isForwardVarargs = false;
     633    bool isTail = false;
     634    bool isEmulatedTail = false;
    633635    switch (node->op()) {
    634636    case Call:
    635637        callType = CallLinkInfo::Call;
    636638        break;
     639    case TailCall:
     640        callType = CallLinkInfo::TailCall;
     641        isTail = true;
     642        break;
     643    case TailCallInlinedCaller:
     644        callType = CallLinkInfo::Call;
     645        isEmulatedTail = true;
     646        break;
    637647    case Construct:
    638648        callType = CallLinkInfo::Construct;
    … …  
    642652        isVarargs = true;
    643653        break;
     654    case TailCallVarargs:
     655        callType = CallLinkInfo::TailCallVarargs;
     656        isVarargs = true;
     657        isTail = true;
     658        break;
     659    case TailCallVarargsInlinedCaller:
     660        callType = CallLinkInfo::CallVarargs;
     661        isVarargs = true;
     662        isEmulatedTail = true;
     663        break;
    644664    case ConstructVarargs:
    645665        callType = CallLinkInfo::ConstructVarargs;
    … …  
    652672    case ConstructForwardVarargs:
    653673        callType = CallLinkInfo::ConstructVarargs;
     674        isForwardVarargs = true;
     675        break;
     676    case TailCallForwardVarargs:
     677        callType = CallLinkInfo::TailCallVarargs;
     678        isTail = true;
     679        isForwardVarargs = true;
     680        break;
     681    case TailCallForwardVarargsInlinedCaller:
     682        callType = CallLinkInfo::CallVarargs;
     683        isEmulatedTail = true;
    654684        isForwardVarargs = true;
    655685        break;
    … …  
    762792    callee.use();
    763793    m_jit.store64(calleeGPR, JITCompiler::calleeFrameSlot(JSStack::Callee));
    764    
    765     flushRegisters();
     794
     795    // FIXME: We should do an efficient move of the arguments into
     796    // their target stack position instead of building then memmoving
     797    // the callee frame.
     798    // https://bugs.webkit.org/show_bug.cgi?id=147508
     799    if (isTail)
     800        ASSERT(isFlushed());
     801    else
     802        flushRegisters();
    766803
    767804    GPRFlushedCallResult result(this);
    768805    GPRReg resultGPR = result.gpr();
    769806
     807    CodeOrigin staticOrigin = node->origin.semantic;
     808    ASSERT(!isTail || !staticOrigin.inlineCallFrame || !staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames());
     809    ASSERT(!isEmulatedTail || (staticOrigin.inlineCallFrame && staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames()));
     810    CodeOrigin dynamicOrigin =
     811        isEmulatedTail ? *staticOrigin.inlineCallFrame->getCallerSkippingDeadFrames() : staticOrigin;
     812
     813    m_jit.emitStoreCodeOrigin(dynamicOrigin);
     814   
     815    CallLinkInfo* callLinkInfo = m_jit.codeBlock()->addCallLinkInfo();
     816
    770817    JITCompiler::DataLabelPtr targetToCheck;
    771     JITCompiler::Jump slowPath;
    772 
    773     m_jit.emitStoreCodeOrigin(node->origin.semantic);
    774    
    775     CallLinkInfo* callLinkInfo = m_jit.codeBlock()->addCallLinkInfo();
    776    
    777     slowPath = m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleeGPR, targetToCheck, MacroAssembler::TrustedImmPtr(0));
    778 
    779     JITCompiler::Call fastCall = m_jit.nearCall();
     818    JITCompiler::Jump slowPath = m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleeGPR, targetToCheck, MacroAssembler::TrustedImmPtr(0));
     819
     820    if (isTail) {
     821        m_jit.emitRestoreCalleeSaves();
     822        // FIXME: We should do an efficient move of the arguments into
     823        // their target stack position instead of building then memmoving
     824        // the callee frame.
     825        // https://bugs.webkit.org/show_bug.cgi?id=147508
     826        m_jit.prepareForTailCallSlow();
     827    }
     828
     829    JITCompiler::Call fastCall = isTail ? m_jit.nearTailCall() : m_jit.nearCall();
    780830
    781831    JITCompiler::Jump done = m_jit.jump();
    … …  
    786836    m_jit.move(MacroAssembler::TrustedImmPtr(callLinkInfo), GPRInfo::regT2); // Link info needs to be in regT2
    787837    JITCompiler::Call slowCall = m_jit.nearCall();
    788    
     838
    789839    done.link(&m_jit);
    790    
    791     m_jit.move(GPRInfo::returnValueGPR, resultGPR);
    792    
    793     jsValueResult(resultGPR, m_currentNode, DataFormatJS, UseChildrenCalledExplicitly);
    794    
     840
     841    if (isTail)
     842        m_jit.breakpoint();
     843    else {
     844        m_jit.move(GPRInfo::returnValueGPR, resultGPR);
     845
     846        jsValueResult(resultGPR, m_currentNode, DataFormatJS, UseChildrenCalledExplicitly);
     847
     848        // After the calls are done, we need to reestablish our stack
     849        // pointer. We rely on this for varargs calls, calls with arity
     850        // mismatch (the callframe is slided) and tail calls.
     851        m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
     852    }
     853
    795854    callLinkInfo->setUpCall(callType, m_currentNode->origin.semantic,  calleeGPR);   
    796855    m_jit.addJSCall(fastCall, slowCall, targetToCheck, callLinkInfo);
    797856
    798     // After the calls are done, we need to reestablish our stack
    799     // pointer. We rely on this for varargs calls, calls with arity
    800     // mismatch (the callframe is slided) and tail calls.
    801     m_jit.addPtr(TrustedImm32(m_jit.graph().stackPointerOffset() * sizeof(Register)), GPRInfo::callFrameRegister, JITCompiler::stackPointerRegister);
    802857}
    803858
    … …  
    42324287
    42334288    case Call:
     4289    case TailCall:
     4290    case TailCallInlinedCaller:
    42344291    case Construct:
    42354292    case CallVarargs:
     4293    case TailCallVarargs:
     4294    case TailCallVarargsInlinedCaller:
    42364295    case CallForwardVarargs:
    42374296    case ConstructVarargs:
    42384297    case ConstructForwardVarargs:
     4298    case TailCallForwardVarargs:
     4299    case TailCallForwardVarargsInlinedCaller:
    42394300        emitCall(node);
    42404301        break;
    4241        
     4302
    42424303    case LoadVarargs: {
    42434304        LoadVarargsData* data = node->loadVarargsData();

  • branches/jsc-tailcall/Source/JavaScriptCore/dfg/DFGVarargsForwardingPhase.cpp

    r184288 r187791  
    136136            case CallVarargs:
    137137            case ConstructVarargs:
     138            case TailCallVarargs:
     139            case TailCallVarargsInlinedCaller:
    138140                if (node->child1() == candidate || node->child3() == candidate) {
    139141                    if (verbose)
    … …  
    283285                node->setOpAndDefaultFlags(ConstructForwardVarargs);
    284286                break;
    285                
     287
     288            case TailCallVarargs:
     289                if (node->child2() != candidate)
     290                    break;
     291                node->setOpAndDefaultFlags(TailCallForwardVarargs);
     292                break;
     293
     294            case TailCallVarargsInlinedCaller:
     295                if (node->child2() != candidate)
     296                    break;
     297                node->setOpAndDefaultFlags(TailCallForwardVarargsInlinedCaller);
     298                break;
     299
    286300            case SetLocal:
    287301                // This is super odd. We don't have to do anything here, since in DFG IR, the phantom

  • branches/jsc-tailcall/Source/JavaScriptCore/interpreter/CallFrame.cpp

    r178143 r187791  
    8686            return codeOrigin.bytecodeIndex;
    8787
    88         codeOrigin = inlineCallFrame->caller;
     88        codeOrigin = *inlineCallFrame->getCallerSkippingDeadFrames();
    8989        inlineCallFrame = codeOrigin.inlineCallFrame;
    9090    }

  • branches/jsc-tailcall/Source/JavaScriptCore/interpreter/StackVisitor.cpp

    r182934 r187791  
    6060    if (m_frame.isInlinedFrame()) {
    6161        InlineCallFrame* inlineCallFrame = m_frame.inlineCallFrame();
    62         CodeOrigin* callerCodeOrigin = &inlineCallFrame->caller;
    63         readInlinedFrame(m_frame.callFrame(), callerCodeOrigin);
     62        CodeOrigin* callerCodeOrigin = inlineCallFrame->getCallerSkippingDeadFrames();
     63        if (!callerCodeOrigin) {
     64            while (inlineCallFrame) {
     65                readInlinedFrame(m_frame.callFrame(), &inlineCallFrame->directCaller);
     66                inlineCallFrame = m_frame.inlineCallFrame();
     67            }
     68            m_frame.m_VMEntryFrame = m_frame.m_CallerVMEntryFrame;
     69            readFrame(m_frame.callerFrame());
     70        } else
     71            readInlinedFrame(m_frame.callFrame(), callerCodeOrigin);
    6472        return;
    6573    }

  • branches/jsc-tailcall/Source/JavaScriptCore/jit/CCallHelpers.h

    r187676 r187791  
    20682068        COMPILE_ASSERT(sizeof(void*) * 2 == sizeof(Register), Register_is_two_pointers_sized);
    20692069        lshift32(TrustedImm32(1), temp2);
     2070#else
     2071        COMPILE_ASSERT(sizeof(void*) == sizeof(Register), Register_is_one_pointer_sized);
    20702072#endif
    20712073
Note: See TracChangeset for help on using the changeset viewer.