Changeset 96375 in webkit

Timestamp:

Sep 29, 2011 4:17:19 PM (13 years ago)

Author:

fpizlo@apple.com

Message:

DFG JIT should infer which uses of a variable are not aliased
​https://bugs.webkit.org/show_bug.cgi?id=68593

Reviewed by Oliver Hunt.

This separates how a variable is stored (i.e. its virtual register)
from how it's predicted. Each variable now takes a
VariableAccessData as its operand, instead of the virtual register.
The VariableAccessData stores the operand and the prediction. If
multiple uses of a variable are aliased, their VariableAccessDatas
are unified.

This also adds tracking of which argument values are used. It
correctly observes that an argument value is not used, if the
argument is assigned to inside the function before being used.

This also adds tracking of which variables are live at the head of
a basic block, and separates that from a variable being live at the
tail.

Finally, this communicates to both OSR entry and OSR exit code how
a variable is predicted at a particular point in the code, rather
than just communicating how it was predicted in the entire code
block (since with this patch there is no longer the notion of a
variable having just one prediction for a code block).

• JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj:
• JavaScriptCore.vcproj/WTF/WTF.vcproj:
• JavaScriptCore.xcodeproj/project.pbxproj:
• bytecode/ActionablePrediction.h: Added.

(JSC::actionablePredictionFromPredictedType):
(JSC::valueObeysPrediction):
(JSC::actionablePredictionToString):
(JSC::ActionablePredictions::ActionablePredictions):
(JSC::ActionablePredictions::setArgument):
(JSC::ActionablePredictions::argument):
(JSC::ActionablePredictions::setVariable):
(JSC::ActionablePredictions::variable):
(JSC::ActionablePredictions::argumentUpperBound):
(JSC::ActionablePredictions::variableUpperBound):
(JSC::ActionablePredictions::pack):
(JSC::ActionablePredictions::packVector):

• bytecode/CodeBlock.h:
• bytecode/PredictionTracker.h:
• dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::newVariableAccessData):
(JSC::DFG::ByteCodeParser::getLocal):
(JSC::DFG::ByteCodeParser::setLocal):
(JSC::DFG::ByteCodeParser::getArgument):
(JSC::DFG::ByteCodeParser::setArgument):
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::processPhiStack):
(JSC::DFG::ByteCodeParser::parse):

• dfg/DFGDriver.cpp:

(JSC::DFG::compile):

• dfg/DFGGraph.cpp:

(JSC::DFG::Graph::nameOfVariableAccessData):
(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::predictArgumentTypes):

• dfg/DFGGraph.h:

(JSC::DFG::operandIsArgument):
(JSC::DFG::VariableRecord::setFirstTime):
(JSC::DFG::BasicBlock::BasicBlock):
(JSC::DFG::Graph::predict):
(JSC::DFG::Graph::getPrediction):

• dfg/DFGJITCompiler.h:

(JSC::DFG::JITCompiler::noticeOSREntry):

• dfg/DFGNode.h:

(JSC::DFG::Node::hasVariableAccessData):
(JSC::DFG::Node::hasLocal):
(JSC::DFG::Node::variableAccessData):
(JSC::DFG::Node::local):

• dfg/DFGOSREntry.cpp:

(JSC::DFG::prepareOSREntry):

• dfg/DFGOSREntry.h:
• dfg/DFGPropagator.cpp:

(JSC::DFG::Propagator::propagateNodePredictions):

• dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::ValueSource::dump):
(JSC::DFG::OSRExit::OSRExit):
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::checkArgumentTypes):
(JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):

• dfg/DFGSpeculativeJIT.h:

(JSC::DFG::ValueSource::ValueSource):
(JSC::DFG::ValueSource::forPrediction):
(JSC::DFG::ValueSource::isSet):
(JSC::DFG::ValueSource::kind):
(JSC::DFG::ValueSource::nodeIndex):
(JSC::DFG::ValueSource::nodeIndexFromKind):
(JSC::DFG::ValueSource::kindFromNodeIndex):
(JSC::DFG::SpeculativeJIT::isKnownArray):
(JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):
(JSC::DFG::SpeculativeJIT::SpeculativeJIT):

• dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::OSRExit::OSRExit):
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::checkArgumentTypes):
(JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):

• wtf/PackedIntVector.h: Added.

(WTF::PackedIntVector::PackedIntVector):
(WTF::PackedIntVector::operator=):
(WTF::PackedIntVector::size):
(WTF::PackedIntVector::ensureSize):
(WTF::PackedIntVector::resize):
(WTF::PackedIntVector::clearAll):
(WTF::PackedIntVector::get):
(WTF::PackedIntVector::set):
(WTF::PackedIntVector::mask):

• wtf/Platform.h:
• wtf/UnionFind.h: Added.

(WTF::UnionFind::UnionFind):
(WTF::UnionFind::find):
(WTF::UnionFind::unify):

Location:

trunk/Source/JavaScriptCore

Files:

• ChangeLog (modified) (1 diff)
• JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj (modified) (1 diff)
• JavaScriptCore.vcproj/WTF/WTF.vcproj (modified) (1 diff)
• JavaScriptCore.xcodeproj/project.pbxproj (modified) (9 diffs)
• bytecode/ActionablePrediction.h (added)
• bytecode/CodeBlock.h (modified) (2 diffs)
• bytecode/PredictionTracker.h (modified) (4 diffs)
• dfg/DFGByteCodeParser.cpp (modified) (9 diffs)
• dfg/DFGDriver.cpp (modified) (2 diffs)
• dfg/DFGGraph.cpp (modified) (5 diffs)
• dfg/DFGGraph.h (modified) (11 diffs)
• dfg/DFGJITCompiler.cpp (modified) (1 diff)
• dfg/DFGJITCompiler.h (modified) (1 diff)
• dfg/DFGNode.h (modified) (4 diffs)
• dfg/DFGOSREntry.cpp (modified) (5 diffs)
• dfg/DFGOSREntry.h (modified) (2 diffs)
• dfg/DFGPropagator.cpp (modified) (4 diffs)
• dfg/DFGSpeculativeJIT.cpp (modified) (8 diffs)
• dfg/DFGSpeculativeJIT.h (modified) (7 diffs)
• dfg/DFGSpeculativeJIT32_64.cpp (modified) (8 diffs)
• wtf/PackedIntVector.h (added)
• wtf/UnionFind.h (added)

trunk/Source/JavaScriptCore/ChangeLog

@@ +1 @@
+2011-09-28  Filip Pizlo  <fpizlo@apple.com>
+
+        DFG JIT should infer which uses of a variable are not aliased
+        https://bugs.webkit.org/show_bug.cgi?id=68593
+
+        Reviewed by Oliver Hunt.
+        
+        This separates how a variable is stored (i.e. its virtual register)
+        from how it's predicted. Each variable now takes a
+        VariableAccessData as its operand, instead of the virtual register.
+        The VariableAccessData stores the operand and the prediction. If
+        multiple uses of a variable are aliased, their VariableAccessDatas
+        are unified.
+        
+        This also adds tracking of which argument values are used. It
+        correctly observes that an argument value is not used, if the
+        argument is assigned to inside the function before being used.
+        
+        This also adds tracking of which variables are live at the head of
+        a basic block, and separates that from a variable being live at the
+        tail.
+        
+        Finally, this communicates to both OSR entry and OSR exit code how
+        a variable is predicted at a particular point in the code, rather
+        than just communicating how it was predicted in the entire code
+        block (since with this patch there is no longer the notion of a
+        variable having just one prediction for a code block).
+
+        * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj:
+        * JavaScriptCore.vcproj/WTF/WTF.vcproj:
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * bytecode/ActionablePrediction.h: Added.
+        (JSC::actionablePredictionFromPredictedType):
+        (JSC::valueObeysPrediction):
+        (JSC::actionablePredictionToString):
+        (JSC::ActionablePredictions::ActionablePredictions):
+        (JSC::ActionablePredictions::setArgument):
+        (JSC::ActionablePredictions::argument):
+        (JSC::ActionablePredictions::setVariable):
+        (JSC::ActionablePredictions::variable):
+        (JSC::ActionablePredictions::argumentUpperBound):
+        (JSC::ActionablePredictions::variableUpperBound):
+        (JSC::ActionablePredictions::pack):
+        (JSC::ActionablePredictions::packVector):
+        * bytecode/CodeBlock.h:
+        * bytecode/PredictionTracker.h:
+        * dfg/DFGByteCodeParser.cpp:
+        (JSC::DFG::ByteCodeParser::newVariableAccessData):
+        (JSC::DFG::ByteCodeParser::getLocal):
+        (JSC::DFG::ByteCodeParser::setLocal):
+        (JSC::DFG::ByteCodeParser::getArgument):
+        (JSC::DFG::ByteCodeParser::setArgument):
+        (JSC::DFG::ByteCodeParser::parseBlock):
+        (JSC::DFG::ByteCodeParser::processPhiStack):
+        (JSC::DFG::ByteCodeParser::parse):
+        * dfg/DFGDriver.cpp:
+        (JSC::DFG::compile):
+        * dfg/DFGGraph.cpp:
+        (JSC::DFG::Graph::nameOfVariableAccessData):
+        (JSC::DFG::Graph::dump):
+        (JSC::DFG::Graph::predictArgumentTypes):
+        * dfg/DFGGraph.h:
+        (JSC::DFG::operandIsArgument):
+        (JSC::DFG::VariableRecord::setFirstTime):
+        (JSC::DFG::BasicBlock::BasicBlock):
+        (JSC::DFG::Graph::predict):
+        (JSC::DFG::Graph::getPrediction):
+        * dfg/DFGJITCompiler.h:
+        (JSC::DFG::JITCompiler::noticeOSREntry):
+        * dfg/DFGNode.h:
+        (JSC::DFG::Node::hasVariableAccessData):
+        (JSC::DFG::Node::hasLocal):
+        (JSC::DFG::Node::variableAccessData):
+        (JSC::DFG::Node::local):
+        * dfg/DFGOSREntry.cpp:
+        (JSC::DFG::prepareOSREntry):
+        * dfg/DFGOSREntry.h:
+        * dfg/DFGPropagator.cpp:
+        (JSC::DFG::Propagator::propagateNodePredictions):
+        * dfg/DFGSpeculativeJIT.cpp:
+        (JSC::DFG::ValueSource::dump):
+        (JSC::DFG::OSRExit::OSRExit):
+        (JSC::DFG::SpeculativeJIT::compile):
+        (JSC::DFG::SpeculativeJIT::checkArgumentTypes):
+        (JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):
+        * dfg/DFGSpeculativeJIT.h:
+        (JSC::DFG::ValueSource::ValueSource):
+        (JSC::DFG::ValueSource::forPrediction):
+        (JSC::DFG::ValueSource::isSet):
+        (JSC::DFG::ValueSource::kind):
+        (JSC::DFG::ValueSource::nodeIndex):
+        (JSC::DFG::ValueSource::nodeIndexFromKind):
+        (JSC::DFG::ValueSource::kindFromNodeIndex):
+        (JSC::DFG::SpeculativeJIT::isKnownArray):
+        (JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):
+        (JSC::DFG::SpeculativeJIT::SpeculativeJIT):
+        * dfg/DFGSpeculativeJIT32_64.cpp:
+        (JSC::DFG::OSRExit::OSRExit):
+        (JSC::DFG::SpeculativeJIT::compile):
+        (JSC::DFG::SpeculativeJIT::checkArgumentTypes):
+        (JSC::DFG::SpeculativeJIT::computeValueRecoveryFor):
+        * wtf/PackedIntVector.h: Added.
+        (WTF::PackedIntVector::PackedIntVector):
+        (WTF::PackedIntVector::operator=):
+        (WTF::PackedIntVector::size):
+        (WTF::PackedIntVector::ensureSize):
+        (WTF::PackedIntVector::resize):
+        (WTF::PackedIntVector::clearAll):
+        (WTF::PackedIntVector::get):
+        (WTF::PackedIntVector::set):
+        (WTF::PackedIntVector::mask):
+        * wtf/Platform.h:
+        * wtf/UnionFind.h: Added.
+        (WTF::UnionFind::UnionFind):
+        (WTF::UnionFind::find):
+        (WTF::UnionFind::unify):
+
 2011-09-29  Oliver Hunt  <oliver@apple.com>
 

trunk/Source/JavaScriptCore/JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj

@@ -1431 +1431 @@
                         >
                         <File
+                                RelativePath="..\..\bytecode\ActionablePrediction.h"
+                                >
+                        </File>
+                        <File
                                 RelativePath="..\..\bytecode\CodeBlock.cpp"
                                 >

trunk/Source/JavaScriptCore/JavaScriptCore.vcproj/WTF/WTF.vcproj

@@ -942 +942 @@
                 </File>
                 <File
+                        RelativePath="..\..\wtf\PackedIntVector.h"
+                        >
+                </File>
+                <File
                         RelativePath="..\..\wtf\PageAllocation.h"
                         >

trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

@@ -50 +50 @@
                 0BDFFAE10FC6193100D69EF4 /* OwnFastMallocPtr.h in Headers */ = {isa = PBXBuildFile; fileRef = 0BDFFAD10FC616EC00D69EF4 /* OwnFastMallocPtr.h */; settings = {ATTRIBUTES = (Private, ); }; };
                 0BF28A2911A33DC300638F84 /* SizeLimits.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0BF28A2811A33DC300638F84 /* SizeLimits.cpp */; };
+                0F16D726142C39C000CF784A /* BitVector.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F16D724142C39A200CF784A /* BitVector.cpp */; };
                 0F242DA713F3B1E8007ADD4C /* WeakReferenceHarvester.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F242DA513F3B1BB007ADD4C /* WeakReferenceHarvester.h */; settings = {ATTRIBUTES = (Private, ); }; };
-                0F604E8B142D696D009CEB92 /* BitVector.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F604E8A142D6969009CEB92 /* BitVector.cpp */; };
+                0F636DA0142D27D700B2E66A /* PackedIntVector.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F636D9F142D27D200B2E66A /* PackedIntVector.h */; };
+                0F636DA2142D27F000B2E66A /* ActionablePrediction.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F636DA1142D27ED00B2E66A /* ActionablePrediction.h */; };
                 0F7700921402FF3C0078EB39 /* SamplingCounter.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F7700911402FF280078EB39 /* SamplingCounter.cpp */; };
                 0F963B2713F753BB0002D9B2 /* RedBlackTree.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F963B2613F753990002D9B2 /* RedBlackTree.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -66 +68 @@
                 0FD3C82714115D4F00FD81CB /* DFGPropagator.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD3C82414115D2200FD81CB /* DFGPropagator.h */; };
                 0FD3C82814115D4F00FD81CB /* DFGDriver.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD3C82214115D0E00FD81CB /* DFGDriver.h */; };
+                0FD52AAE143035A00026DC9F /* UnionFind.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD52AAC1430359D0026DC9F /* UnionFind.h */; settings = {ATTRIBUTES = (Private, ); }; };
                 0FD82E2114172CE300179C94 /* DFGCapabilities.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD82E1E14172C2F00179C94 /* DFGCapabilities.cpp */; };
                 0FD82E39141AB14D00179C94 /* CompactJITCodeMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FD82E37141AB14200179C94 /* CompactJITCodeMap.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -798 +801 @@
                 0BDFFAD40FC6171000D69EF4 /* CrossThreadRefCounted.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CrossThreadRefCounted.h; sourceTree = "<group>"; };
                 0BF28A2811A33DC300638F84 /* SizeLimits.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SizeLimits.cpp; sourceTree = "<group>"; };
+                0F16D724142C39A200CF784A /* BitVector.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = BitVector.cpp; sourceTree = "<group>"; };
                 0F242DA513F3B1BB007ADD4C /* WeakReferenceHarvester.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WeakReferenceHarvester.h; sourceTree = "<group>"; };
-                0F604E8A142D6969009CEB92 /* BitVector.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = BitVector.cpp; sourceTree = "<group>"; };
+                0F636D9F142D27D200B2E66A /* PackedIntVector.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PackedIntVector.h; sourceTree = "<group>"; };
+                0F636DA1142D27ED00B2E66A /* ActionablePrediction.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ActionablePrediction.h; sourceTree = "<group>"; };
                 0F77008E1402FDD60078EB39 /* SamplingCounter.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SamplingCounter.h; sourceTree = "<group>"; };
                 0F7700911402FF280078EB39 /* SamplingCounter.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SamplingCounter.cpp; sourceTree = "<group>"; };
@@ -815 +820 @@
                 0FD3C82314115D1A00FD81CB /* DFGPropagator.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGPropagator.cpp; path = dfg/DFGPropagator.cpp; sourceTree = "<group>"; };
                 0FD3C82414115D2200FD81CB /* DFGPropagator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGPropagator.h; path = dfg/DFGPropagator.h; sourceTree = "<group>"; };
+                0FD52AAC1430359D0026DC9F /* UnionFind.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = UnionFind.h; sourceTree = "<group>"; };
                 0FD82E1E14172C2F00179C94 /* DFGCapabilities.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGCapabilities.cpp; path = dfg/DFGCapabilities.cpp; sourceTree = "<group>"; };
                 0FD82E1F14172C2F00179C94 /* DFGCapabilities.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGCapabilities.h; path = dfg/DFGCapabilities.h; sourceTree = "<group>"; };
@@ -1825 +1831 @@
                         isa = PBXGroup;
                         children = (
-                                0F604E8A142D6969009CEB92 /* BitVector.cpp */,
+                                0FD52AAC1430359D0026DC9F /* UnionFind.h */,
+                                0F636D9F142D27D200B2E66A /* PackedIntVector.h */,
+                                0F16D724142C39A200CF784A /* BitVector.cpp */,
                                 0FD82F491428069200179C94 /* BitVector.h */,
                                 C22C524813FAF6EF00B7DC0D /* dtoa */,
@@ -2356 +2364 @@
                         isa = PBXGroup;
                         children = (
+                                0F636DA1142D27ED00B2E66A /* ActionablePrediction.h */,
                                 0FD82E8E14207A5100179C94 /* ValueProfile.cpp */,
                                 0FD82E84141F3FDA00179C94 /* PredictedType.cpp */,
@@ -2839 +2848 @@
                                 A70456B01427FB910037DA68 /* AllocationSpace.h in Headers */,
                                 86FA9E92142BBB2E001773B7 /* JSBoundFunction.h in Headers */,
+                                0F636DA0142D27D700B2E66A /* PackedIntVector.h in Headers */,
+                                0F636DA2142D27F000B2E66A /* ActionablePrediction.h in Headers */,
                                 A7521E131429169A003C8D0C /* CardSet.h in Headers */,
+                                0FD52AAE143035A00026DC9F /* UnionFind.h in Headers */,
                                 86880F1E14328BB900B08D42 /* DFGJITCompilerInlineMethods.h in Headers */,
                         );
@@ -3154 +3166 @@
                         buildActionMask = 2147483647;
                         files = (
+                                0F16D726142C39C000CF784A /* BitVector.cpp in Sources */,
                                 0FD82F2C1426CA7400179C94 /* JettisonedCodeBlocks.cpp in Sources */,
                                 0FD82E9014207A5F00179C94 /* ValueProfile.cpp in Sources */,
@@ -3365 +3378 @@
                                 A70456B11427FB950037DA68 /* AllocationSpace.cpp in Sources */,
                                 86FA9E91142BBB2E001773B7 /* JSBoundFunction.cpp in Sources */,
-                                0F604E8B142D696D009CEB92 /* BitVector.cpp in Sources */,
                                 86880F1C14328BB900B08D42 /* DFGJITCodeGenerator32_64.cpp in Sources */,
                                 86880F1D14328BB900B08D42 /* DFGJITCompiler32_64.cpp in Sources */,

trunk/Source/JavaScriptCore/bytecode/CodeBlock.h

@@ -241 +241 @@
         PassOwnPtr<CodeBlock> releaseAlternative() { return m_alternative.release(); }
         
-        void setPredictions(PassOwnPtr<PredictionTracker> predictions) { m_predictions = predictions; }
-        PredictionTracker* predictions() const { return m_predictions.get(); }
-
         void visitAggregate(SlotVisitor&);
         void visitWeakReferences(SlotVisitor&);
@@ -976 +973 @@
         OwnPtr<CodeBlock> m_alternative;
         
-        OwnPtr<PredictionTracker> m_predictions;
-
         int32_t m_executeCounter;
         uint32_t m_speculativeSuccessCounter;

trunk/Source/JavaScriptCore/bytecode/PredictionTracker.h

@@ -29 +29 @@
 #include "PredictedType.h"
 #include <wtf/HashMap.h>
-#include <wtf/Vector.h>
 
 namespace JSC {
-
-// helper function to distinguish vars & temporaries from arguments.
-inline bool operandIsArgument(int operand) { return operand < 0; }
 
 struct PredictionSlot {
@@ -51 +47 @@
     }
     
-    PredictionTracker(unsigned numArguments, unsigned numVariables)
-        : m_arguments(numArguments)
-        , m_variables(numVariables)
-    {
-    }
-    
-    void initializeSimilarTo(const PredictionTracker& other)
-    {
-        m_arguments.resize(other.numberOfArguments());
-        m_variables.resize(other.numberOfVariables());
-    }
-    
-    void copyLocalsFrom(const PredictionTracker& other)
-    {
-        m_arguments = other.m_arguments;
-        m_variables = other.m_variables;
-    }
-    
-    size_t numberOfArguments() const { return m_arguments.size(); }
-    size_t numberOfVariables() const { return m_variables.size(); }
-    
-    unsigned argumentIndexForOperand(int operand) const { return operand + m_arguments.size() + RegisterFile::CallFrameHeaderSize; }
-
-    bool predictArgument(unsigned argument, PredictedType prediction)
-    {
-        return mergePrediction(m_arguments[argument].m_value, prediction);
-    }
-    
-    bool predict(int operand, PredictedType prediction)
-    {
-        if (operandIsArgument(operand))
-            return predictArgument(argumentIndexForOperand(operand), prediction);
-        if ((unsigned)operand >= m_variables.size()) {
-            ASSERT(operand >= 0);
-            m_variables.resize(operand + 1);
-        }
-        
-        return mergePrediction(m_variables[operand].m_value, prediction);
-    }
-    
     bool predictGlobalVar(unsigned varNumber, PredictedType prediction)
     {
@@ -103 +59 @@
     }
     
-    PredictedType getArgumentPrediction(unsigned argument)
-    {
-        return m_arguments[argument].m_value;
-    }
-
-    PredictedType getPrediction(int operand)
-    {
-        if (operandIsArgument(operand))
-            return getArgumentPrediction(argumentIndexForOperand(operand));
-        if ((unsigned)operand < m_variables.size())
-            return m_variables[operand].m_value;
-        return PredictNone;
-    }
-    
     PredictedType getGlobalVarPrediction(unsigned varNumber)
     {
@@ -126 +68 @@
     
 private:
-    Vector<PredictionSlot, 16> m_arguments;
-    Vector<PredictionSlot, 16> m_variables;
     HashMap<unsigned, PredictionSlot> m_globalVars;
 };

trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

@@ -84 +84 @@
     // Add spill locations to nodes.
     void allocateVirtualRegisters();
-
+    
+    VariableAccessData* newVariableAccessData(int operand)
+    {
+        ASSERT(operand < FirstConstantRegisterIndex);
+        
+        m_graph.m_variableAccessData.append(VariableAccessData(static_cast<VirtualRegister>(operand)));
+        return &m_graph.m_variableAccessData.last();
+    }
+    
     // Get/Set the operands/result of a bytecode instruction.
     NodeIndex get(int operand)
@@ -117 +125 @@
     NodeIndex getLocal(unsigned operand)
     {
-        NodeIndex nodeIndex = m_currentBlock->m_locals[operand].value;
+        NodeIndex nodeIndex = m_currentBlock->m_localsAtTail[operand].value;
 
         if (nodeIndex != NoNode) {
@@ -130 +138 @@
         // expand m_preservedVars to cover these.
         m_preservedVars = std::max(m_preservedVars, operand + 1);
-
-        NodeIndex phi = addToGraph(Phi);
+        
+        VariableAccessData* variableAccessData = newVariableAccessData(operand);
+        
+        NodeIndex phi = addToGraph(Phi, OpInfo(variableAccessData));
         m_localPhiStack.append(PhiStackEntry(m_currentBlock, phi, operand));
-        nodeIndex = addToGraph(GetLocal, OpInfo(operand), phi);
-        m_currentBlock->m_locals[operand].value = nodeIndex;
+        nodeIndex = addToGraph(GetLocal, OpInfo(variableAccessData), phi);
+        m_currentBlock->m_localsAtTail[operand].value = nodeIndex;
+        
+        m_currentBlock->m_localsAtHead[operand].setFirstTime(nodeIndex);
+        
         return nodeIndex;
     }
     void setLocal(unsigned operand, NodeIndex value)
     {
-        m_currentBlock->m_locals[operand].value = addToGraph(SetLocal, OpInfo(operand), value);
+        m_currentBlock->m_localsAtTail[operand].value = addToGraph(SetLocal, OpInfo(newVariableAccessData(operand)), value);
     }
 
@@ -148 +161 @@
         ASSERT(argument < m_numArguments);
 
-        NodeIndex nodeIndex = m_currentBlock->m_arguments[argument].value;
+        NodeIndex nodeIndex = m_currentBlock->m_argumentsAtTail[argument].value;
 
         if (nodeIndex != NoNode) {
             Node& node = m_graph[nodeIndex];
+            if (node.op == SetArgument) {
+                // We're getting an argument in the first basic block; link
+                // the GetLocal to the SetArgument.
+                ASSERT(node.local() == static_cast<VirtualRegister>(operand));
+                nodeIndex = addToGraph(GetLocal, OpInfo(node.variableAccessData()), nodeIndex);
+                m_currentBlock->m_argumentsAtTail[argument].value = nodeIndex;
+                return nodeIndex;
+            }
+            
             if (node.op == GetLocal)
                 return nodeIndex;
+            
             ASSERT(node.op == SetLocal);
             return node.child1();
         }
-
-        NodeIndex phi = addToGraph(Phi);
+        
+        VariableAccessData* variableAccessData = newVariableAccessData(operand);
+
+        NodeIndex phi = addToGraph(Phi, OpInfo(variableAccessData));
         m_argumentPhiStack.append(PhiStackEntry(m_currentBlock, phi, argument));
-        nodeIndex = addToGraph(GetLocal, OpInfo(operand), phi);
-        m_currentBlock->m_arguments[argument].value = nodeIndex;
+        nodeIndex = addToGraph(GetLocal, OpInfo(variableAccessData), phi);
+        m_currentBlock->m_argumentsAtTail[argument].value = nodeIndex;
+        
+        m_currentBlock->m_argumentsAtHead[argument].setFirstTime(nodeIndex);
+        
         return nodeIndex;
     }
@@ -169 +197 @@
         ASSERT(argument < m_numArguments);
 
-        m_currentBlock->m_arguments[argument].value = addToGraph(SetLocal, OpInfo(operand), value);
+        m_currentBlock->m_argumentsAtTail[argument].value = addToGraph(SetLocal, OpInfo(newVariableAccessData(operand)), value);
     }
 
@@ -688 +716 @@
         for (unsigned i = 0; i < m_constants.size(); ++i)
             m_constants[i] = ConstantRecord();
+    }
+    
+    // If we are the first basic block, introduce markers for arguments. This allows
+    // us to track if a use of an argument may use the actual argument passed, as
+    // opposed to using a value we set explicitly.
+    if (!m_currentIndex) {
+        m_graph.m_arguments.resize(m_numArguments);
+        for (unsigned argument = 0; argument < m_numArguments; ++argument) {
+            NodeIndex setArgument = addToGraph(SetArgument, OpInfo(newVariableAccessData(argument - m_codeBlock->m_numParameters - RegisterFile::CallFrameHeaderSize)));
+            m_graph.m_arguments[argument] = setArgument;
+            m_currentBlock->m_argumentsAtHead[argument].setFirstTime(setArgument);
+            m_currentBlock->m_argumentsAtTail[argument].setFirstTime(setArgument);
+        }
     }
 
@@ -1449 +1490 @@
         PredecessorList& predecessors = entry.m_block->m_predecessors;
         unsigned varNo = entry.m_varNo;
+        VariableAccessData* dataForPhi = m_graph[entry.m_phi].variableAccessData();
 
         for (size_t i = 0; i < predecessors.size(); ++i) {
             BasicBlock* predecessorBlock = m_graph.m_blocks[predecessors[i]].get();
 
-            VariableRecord& var = (stackType == ArgumentPhiStack) ? predecessorBlock->m_arguments[varNo] : predecessorBlock->m_locals[varNo];
+            VariableRecord& var = (stackType == ArgumentPhiStack) ? predecessorBlock->m_argumentsAtTail[varNo] : predecessorBlock->m_localsAtTail[varNo];
 
             NodeIndex valueInPredecessor = var.value;
             if (valueInPredecessor == NoNode) {
-                valueInPredecessor = addToGraph(Phi);
-                var.value = valueInPredecessor;
+                valueInPredecessor = addToGraph(Phi, OpInfo(newVariableAccessData(stackType == ArgumentPhiStack ? varNo - m_codeBlock->m_numParameters - RegisterFile::CallFrameHeaderSize : varNo)));
+                var.setFirstTime(valueInPredecessor);
+                if (stackType == ArgumentPhiStack)
+                    predecessorBlock->m_argumentsAtHead[varNo].setFirstTime(valueInPredecessor);
+                else
+                    predecessorBlock->m_localsAtHead[varNo].setFirstTime(valueInPredecessor);
                 phiStack.append(PhiStackEntry(predecessorBlock, valueInPredecessor, varNo));
-            } else if (m_graph[valueInPredecessor].op == GetLocal)
+            } else if (m_graph[valueInPredecessor].op == GetLocal) {
+                // We want to ensure that the VariableAccessDatas are identical between the
+                // GetLocal and its block-local Phi. Strictly speaking we only need the two
+                // to be unified. But for efficiency, we want the code that creates GetLocals
+                // and Phis to try to reuse VariableAccessDatas as much as possible.
+                ASSERT(m_graph[valueInPredecessor].variableAccessData() == m_graph[m_graph[valueInPredecessor].child1()].variableAccessData());
+                
                 valueInPredecessor = m_graph[valueInPredecessor].child1();
-            ASSERT(m_graph[valueInPredecessor].op == SetLocal || m_graph[valueInPredecessor].op == Phi);
+            }
+            ASSERT(m_graph[valueInPredecessor].op == SetLocal || m_graph[valueInPredecessor].op == Phi || (m_graph[valueInPredecessor].op == SetArgument && stackType == ArgumentPhiStack));
+            
+            VariableAccessData* dataForPredecessor = m_graph[valueInPredecessor].variableAccessData();
+            
+            dataForPredecessor->unify(dataForPhi);
 
             Node* phiNode = &m_graph[entry.m_phi];
@@ -1480 +1537 @@
                 continue;
             }
-
-            NodeIndex newPhi = addToGraph(Phi);
+            
+            NodeIndex newPhi = addToGraph(Phi, OpInfo(dataForPhi));
             
             phiNode = &m_graph[entry.m_phi]; // reload after vector resize
@@ -1549 +1606 @@
     
     m_graph.m_preservedVars = m_preservedVars;
+    m_graph.m_localVars = m_numLocals;
     m_graph.m_parameterSlots = m_parameterSlots;
 

trunk/Source/JavaScriptCore/dfg/DFGDriver.cpp

@@ -39 +39 @@
 {
     JSGlobalData* globalData = &exec->globalData();
-    Graph dfg(codeBlock->m_numParameters, codeBlock->m_numVars);
+    Graph dfg;
     if (!parse(dfg, globalData, codeBlock))
         return false;
@@ -47 +47 @@
     
     propagate(dfg, globalData, codeBlock);
-    
-    // Save the predictions we've made, so that OSR entry can verify them. Predictions
-    // are saved in the CodeBlock from which we will be doing OSR entry, since the
-    // CodeBlock to which we are OSRing may be replaced at any time.
-    OwnPtr<PredictionTracker> predictions = adoptPtr(new PredictionTracker());
-    *predictions = dfg.predictions();
-    ASSERT(codeBlock->alternative());
-    ASSERT(codeBlock->alternative()->getJITType() == JITCode::BaselineJIT);
-    codeBlock->alternative()->setPredictions(predictions.release());
     
     JITCompiler dataFlowJIT(globalData, dfg, codeBlock);

trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp

@@ -45 +45 @@
 {
     return dfgOpNames[op & NodeIdMask];
+}
+
+const char* Graph::nameOfVariableAccessData(VariableAccessData* variableAccessData)
+{
+    // Variables are already numbered. For readability of IR dumps, this returns
+    // an alphabetic name for the variable access data, so that you don't have to
+    // reason about two numbers (variable number and live range number), but instead
+    // a number and a letter.
+    
+    unsigned index = std::numeric_limits<unsigned>::max();
+    for (unsigned i = 0; i < m_variableAccessData.size(); ++i) {
+        if (&m_variableAccessData[i] == variableAccessData) {
+            index = i;
+            break;
+        }
+    }
+    
+    ASSERT(index != std::numeric_limits<unsigned>::max());
+    
+    if (!index)
+        return "A";
+
+    static char buf[10];
+    BoundsCheckedPointer<char> ptr(buf, sizeof(buf));
+    
+    while (index) {
+        *ptr++ = 'A' + (index % 26);
+        index /= 26;
+    }
+    
+    *ptr++ = 0;
+    
+    return buf;
 }
 
@@ -117 +150 @@
         hasPrinted = true;
     }
-    if (node.hasLocal()) {
-        int local = node.local();
-        if (operandIsArgument(local))
-            printf("%sarg%u", hasPrinted ? ", " : "", local - codeBlock->thisRegister());
+    ASSERT(node.hasVariableAccessData() == node.hasLocal());
+    if (node.hasVariableAccessData()) {
+        VariableAccessData* variableAccessData = node.variableAccessData();
+        int operand = variableAccessData->operand();
+        if (operandIsArgument(operand))
+            printf("%sarg%u(%s)", hasPrinted ? ", " : "", operand - codeBlock->thisRegister(), nameOfVariableAccessData(variableAccessData));
         else
-            printf("%sr%u", hasPrinted ? ", " : "", local);
+            printf("%sr%u(%s)", hasPrinted ? ", " : "", operand, nameOfVariableAccessData(variableAccessData));
         hasPrinted = true;
     }
@@ -146 +181 @@
 
     if (!skipped) {
-        if (node.hasLocal())
-            printf("  predicting %s", predictionToString(getPrediction(node.local())));
+        if (node.hasVariableAccessData())
+            printf("  predicting %s", predictionToString(node.variableAccessData()->prediction()));
         else if (node.hasVarNumber())
             printf("  predicting %s", predictionToString(getGlobalVarPrediction(node.varNumber())));
@@ -205 +240 @@
 {
     if (exec) {
-        size_t numberOfArguments = std::min(exec->argumentCountIncludingThis(), m_predictions.numberOfArguments());
+        size_t numberOfArguments = std::min(exec->argumentCountIncludingThis(), m_arguments.size());
         
         for (size_t arg = 1; arg < numberOfArguments; ++arg)
-            m_predictions.predictArgument(arg, predictionFromValue(exec->argument(arg - 1)));
+            at(m_arguments[arg]).variableAccessData()->predict(predictionFromValue(exec->argument(arg - 1)));
     }
     
@@ -221 +256 @@
             continue;
         
-        m_predictions.predictArgument(arg, profile->computeUpdatedPrediction());
+        at(m_arguments[arg]).variableAccessData()->predict(profile->computeUpdatedPrediction());
         
 #if ENABLE(DFG_DEBUG_VERBOSE)
-        printf("Argument [%lu] prediction: %s\n", arg, predictionToString(m_predictions.getArgumentPrediction(arg)));
+        printf("Argument [%lu] prediction: %s\n", arg, predictionToString(at(m_arguments[arg]).variableAccessData()->prediction()));
 #endif
     }

trunk/Source/JavaScriptCore/dfg/DFGGraph.h

@@ -44 +44 @@
 namespace DFG {
 
+// helper function to distinguish vars & temporaries from arguments.
+inline bool operandIsArgument(int operand) { return operand < 0; }
+
 typedef uint32_t BlockIndex;
 
@@ -53 +56 @@
         : value(NoNode)
     {
+    }
+    
+    void setFirstTime(NodeIndex nodeIndex)
+    {
+        ASSERT(value == NoNode);
+        value = nodeIndex;
     }
 
@@ -111 +120 @@
         , end(NoNode)
         , isOSRTarget(false)
-        , m_arguments(numArguments)
-        , m_locals(numLocals)
+        , m_argumentsAtHead(numArguments)
+        , m_localsAtHead(numLocals)
+        , m_argumentsAtTail(numArguments)
+        , m_localsAtTail(numLocals)
     {
     }
@@ -127 +138 @@
 
     PredecessorList m_predecessors;
-    Vector <VariableRecord, 8> m_arguments;
-    Vector <VariableRecord, 16> m_locals;
+    
+    Vector<VariableRecord, 8> m_argumentsAtHead;
+    Vector<VariableRecord, 16> m_localsAtHead;
+    
+    Vector<VariableRecord, 8> m_argumentsAtTail;
+    Vector<VariableRecord, 16> m_localsAtTail;
 };
 
@@ -139 +154 @@
 class Graph : public Vector<Node, 64> {
 public:
-    Graph(unsigned numArguments, unsigned numVariables)
-        : m_predictions(numArguments, numVariables)
-    {
-    }
-
     // Mark a node as being referenced.
     void ref(NodeIndex nodeIndex)
@@ -172 +182 @@
     }
     
-    PredictionTracker& predictions()
-    {
-        return m_predictions;
-    }
-    
-    bool predict(int operand, PredictedType prediction)
-    {
-        return m_predictions.predict(operand, prediction);
-    }
-    
     bool predictGlobalVar(unsigned varNumber, PredictedType prediction)
     {
@@ -191 +191 @@
         switch (node.op) {
         case GetLocal:
-            return predict(node.local(), prediction);
-            break;
+        case SetLocal:
+        case Phi:
+        case SetArgument:
+            return node.variableAccessData()->predict(prediction);
         case GetGlobalVar:
             return predictGlobalVar(node.varNumber(), prediction);
-            break;
         case GetById:
         case GetMethod:
@@ -213 +214 @@
     }
 
-    PredictedType getPrediction(int operand)
-    {
-        return m_predictions.getPrediction(operand);
-    }
-    
     PredictedType getGlobalVarPrediction(unsigned varNumber)
     {
@@ -245 +241 @@
         switch (nodePtr->op) {
         case GetLocal:
-            return getPrediction(nodePtr->local());
+        case SetLocal:
+        case SetArgument:
+        case Phi:
+            return nodePtr->variableAccessData()->prediction();
         case GetGlobalVar:
             return getGlobalVarPrediction(nodePtr->varNumber());
@@ -325 +324 @@
 #ifndef NDEBUG
     static const char *opName(NodeType);
+    
+    // This is O(n), and should only be used for verbose dumps.
+    const char* nameOfVariableAccessData(VariableAccessData*);
 #endif
 
@@ -334 +336 @@
     Vector<StorageAccessData> m_storageAccessData;
     Vector<ResolveGlobalData> m_resolveGlobalData;
+    Vector<NodeIndex, 8> m_arguments;
+    SegmentedVector<VariableAccessData, 16> m_variableAccessData;
     unsigned m_preservedVars;
+    unsigned m_localVars;
     unsigned m_parameterSlots;
 private:

trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp

@@ -577 +577 @@
     // Link the code, populate data in CodeBlock data structures.
 #if ENABLE(DFG_DEBUG_VERBOSE)
-    fprintf(stderr, "JIT code for %p start at [%p, %p)\n", m_codeBlock, linkBuffer.debugAddress(), static_cast<char*>(linkBuffer.debugAddress()) + linkBuffer.debugSize());
+    fprintf(stderr, "JIT code for %p start at [%p, %p). Size = %lu.\n", m_codeBlock, linkBuffer.debugAddress(), static_cast<char*>(linkBuffer.debugAddress()) + linkBuffer.debugSize(), linkBuffer.debugSize());
 #endif
 

trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.h

@@ -394 +394 @@
         OSREntryData* entry = codeBlock()->appendDFGOSREntryData(basicBlock.bytecodeBegin, differenceBetween(m_startOfCode, label()));
         
-        unsigned lastLiveArgument = 0;
-        unsigned lastLiveLocal = 0;
+        ActionablePredictions actionablePredictions(basicBlock.m_argumentsAtHead.size(), basicBlock.m_localsAtHead.size());
+
+        for (unsigned i = 0; i < basicBlock.m_argumentsAtHead.size(); ++i) {
+            NodeIndex nodeIndex = basicBlock.m_argumentsAtHead[i].value;
+            if (nodeIndex != NoNode)
+                actionablePredictions.setArgument(i, actionablePredictionFromPredictedType(m_graph[nodeIndex].variableAccessData()->prediction()));
+        }
         
-        for (unsigned i = 0; i < basicBlock.m_arguments.size(); ++i) {
-            if (basicBlock.m_arguments[i].value != NoNode)
-                lastLiveArgument = i;
+        for (unsigned i = 0; i < basicBlock.m_localsAtHead.size(); ++i) {
+            NodeIndex nodeIndex = basicBlock.m_localsAtHead[i].value;
+            if (nodeIndex != NoNode)
+                actionablePredictions.setVariable(i, actionablePredictionFromPredictedType(m_graph[nodeIndex].variableAccessData()->prediction()));
         }
         
-        for (unsigned i = 0; i < basicBlock.m_locals.size(); ++i) {
-            if (basicBlock.m_locals[i].value != NoNode)
-                lastLiveLocal = i;
-        }
+        actionablePredictions.pack();
         
-        if (lastLiveArgument) {
-            entry->m_liveArguments.resize(lastLiveArgument + 1);
-            entry->m_liveArguments.clearAll();
-            
-            for (unsigned i = 0; i <= lastLiveArgument; ++i) {
-                if (basicBlock.m_arguments[i].value != NoNode)
-                    entry->m_liveArguments.set(i);
-            }
-        } else
-            entry->m_liveArguments.clearAll();
-        
-        if (lastLiveLocal) {
-            entry->m_liveVariables.resize(lastLiveLocal + 1);
-            entry->m_liveVariables.clearAll();
-            
-            for (unsigned i = 0; i <= lastLiveLocal; ++i) {
-                if (basicBlock.m_locals[i].value != NoNode)
-                    entry->m_liveVariables.set(i);
-            }
-        } else
-            entry->m_liveVariables.clearAll();
+        entry->m_predictions = actionablePredictions;
 #else
         UNUSED_PARAM(basicBlock);

trunk/Source/JavaScriptCore/dfg/DFGNode.h

@@ -29 +29 @@
 #include <wtf/BoundsCheckedPointer.h>
 #include <wtf/Platform.h>
+#include <wtf/UnionFind.h>
 
 // Emit various logging information for debugging, including dumping the dataflow graphs.
@@ -109 +110 @@
 };
 
+class VariableAccessData: public UnionFind<VariableAccessData> {
+public:
+    VariableAccessData()
+        : m_local(static_cast<VirtualRegister>(std::numeric_limits<int>::min()))
+        , m_prediction(PredictNone)
+    {
+    }
+    
+    VariableAccessData(VirtualRegister local)
+        : m_local(local)
+        , m_prediction(PredictNone)
+    {
+    }
+    
+    VirtualRegister local()
+    {
+        ASSERT(m_local == find()->m_local);
+        return m_local;
+    }
+    
+    int operand()
+    {
+        return static_cast<int>(local());
+    }
+    
+    bool predict(PredictedType prediction)
+    {
+        return mergePrediction(find()->m_prediction, prediction);
+    }
+    
+    PredictedType prediction()
+    {
+        return find()->m_prediction;
+    }
+    
+private:
+    // This is slightly space-inefficient, since anything we're unified with
+    // will have the same operand and should have the same prediction. But
+    // putting them here simplifies the code, and we don't expect DFG space
+    // usage for variable access nodes do be significant.
+
+    VirtualRegister m_local;
+    PredictedType m_prediction;
+};
+
 typedef unsigned ArithNodeFlags;
 #define NodeUseBottom      0x00
@@ -223 +269 @@
     macro(Phantom, NodeMustGenerate) \
     macro(Phi, 0) \
+    \
+    /* Marker for arguments being set. */\
+    macro(SetArgument, 0) \
     \
     /* Nodes for bitwise operations. */\
@@ -465 +514 @@
     }
     
+    bool hasVariableAccessData()
+    {
+        switch (op) {
+        case GetLocal:
+        case SetLocal:
+        case Phi:
+        case SetArgument:
+            return true;
+        default:
+            return false;
+        }
+    }
+    
     bool hasLocal()
     {
-        return op == GetLocal || op == SetLocal;
-    }
-
+        return hasVariableAccessData();
+    }
+    
+    VariableAccessData* variableAccessData()
+    {
+        return reinterpret_cast<VariableAccessData*>(m_opInfo)->find();
+    }
+    
     VirtualRegister local()
     {
-        ASSERT(hasLocal());
-        return (VirtualRegister)m_opInfo;
+        return variableAccessData()->local();
     }
 

trunk/Source/JavaScriptCore/dfg/DFGOSREntry.cpp

@@ -36 +36 @@
 namespace JSC { namespace DFG {
 
-inline bool predictionIsValid(JSGlobalData* globalData, JSValue value, PredictedType type)
-{
-    // this takes into account only local variable predictions that get enforced
-    // on SetLocal.
-    
-    if (isInt32Prediction(type))
-        return value.isInt32();
-    
-    if (isArrayPrediction(type))
-        return isJSArray(globalData, value);
-    
-    if (isBooleanPrediction(type))
-        return value.isBoolean();
-    
-    return true;
-}
-
 void* prepareOSREntry(ExecState* exec, CodeBlock* codeBlock, unsigned bytecodeIndex)
 {
@@ -67 +50 @@
     
     JSGlobalData* globalData = &exec->globalData();
-    CodeBlock* baselineCodeBlock = codeBlock->alternative();
     OSREntryData* entry = codeBlock->dfgOSREntryDataForBytecodeIndex(bytecodeIndex);
     
@@ -96 +78 @@
     //    OSR at this time.
     
-    PredictionTracker* predictions = baselineCodeBlock->predictions();
-    
-    if (predictions->numberOfArguments() > exec->argumentCountIncludingThis())
-        return 0;
-    
-    for (unsigned i = 1; i < predictions->numberOfArguments(); ++i) {
-        if (i < entry->m_liveArguments.size() && entry->m_liveArguments.get(i)
-            && !predictionIsValid(globalData, exec->argument(i - 1), predictions->getArgumentPrediction(i))) {
+    for (unsigned i = 1; i < entry->m_predictions.argumentUpperBound(); ++i) {
+        ActionablePrediction prediction = entry->m_predictions.argument(i);
+        if (prediction == NoActionablePrediction)
+            continue;
+
+        if (i >= exec->argumentCountIncludingThis()) {
 #if ENABLE(JIT_VERBOSE_OSR)
-            printf("    OSR failed because argument %u is %s, expected %s.\n", i, exec->argument(i - 1).description(), predictionToString(predictions->getArgumentPrediction(i)));
+            printf("    OSR failed because argument %u was not passed, expected %s.\n", i, actionablePredictionToString(prediction));
+#endif
+            return 0;
+        }
+        
+        if (!valueObeysPrediction(globalData, exec->argument(i - 1), prediction)) {
+#if ENABLE(JIT_VERBOSE_OSR)
+            printf("    OSR failed because argument %u is %s, expected %s.\n", i, exec->argument(i - 1).description(), actionablePredictionToString(prediction));
 #endif
             return 0;
@@ -111 +98 @@
     }
     
-    for (unsigned i = 0; i < predictions->numberOfVariables(); ++i) {
-        if (i < entry->m_liveVariables.size() && entry->m_liveVariables.get(i)
-            && !predictionIsValid(globalData, exec->registers()[i].jsValue(), predictions->getPrediction(i))) {
+    for (unsigned i = 0; i < entry->m_predictions.variableUpperBound(); ++i) {
+        ActionablePrediction prediction = entry->m_predictions.variable(i);
+        if (prediction == NoActionablePrediction)
+            continue;
+        
+        if (!valueObeysPrediction(globalData, exec->registers()[i].jsValue(), prediction)) {
 #if ENABLE(JIT_VERBOSE_OSR)
-            printf("    OSR failed because variable %u is %s, expected %s.\n", i, exec->registers()[i].jsValue().description(), predictionToString(predictions->getPrediction(i)));
+            printf("    OSR failed because variable %u is %s, expected %s.\n", i, exec->registers()[i].jsValue().description(), actionablePredictionToString(prediction));
 #endif
             return 0;
@@ -130 +120 @@
     if (!globalData->interpreter->registerFile().grow(&exec->registers()[codeBlock->m_numCalleeRegisters])) {
 #if ENABLE(JIT_VERBOSE_OSR)
-        printf("    OSR failed because stack growth failed..\n");
+        printf("    OSR failed because stack growth failed.\n");
 #endif
         return 0;

trunk/Source/JavaScriptCore/dfg/DFGOSREntry.h

@@ -27 +27 @@
 #define DFGOSREntry_h
 
-#include <wtf/BitVector.h>
+#include "ActionablePrediction.h"
 
 namespace JSC {
@@ -40 +40 @@
     unsigned m_bytecodeIndex;
     unsigned m_machineCodeOffset;
-    BitVector m_liveArguments;
-    BitVector m_liveVariables;
+    ActionablePredictions m_predictions;
 };
 

trunk/Source/JavaScriptCore/dfg/DFGPropagator.cpp

@@ -320 +320 @@
             
         case GetLocal: {
-            PredictedType prediction = m_graph.getPrediction(node.local());
+            PredictedType prediction = node.variableAccessData()->prediction();
             if (prediction)
                 changed |= mergePrediction(prediction);
@@ -327 +327 @@
             
         case SetLocal: {
-            changed |= m_graph.predict(node.local(), m_predictions[node.child1()]);
+            changed |= node.variableAccessData()->predict(m_predictions[node.child1()]);
             break;
         }
@@ -473 +473 @@
             
         case ConvertThis: {
-            changed |= setPrediction(PredictObjectUnknown);
+            PredictedType prediction = m_predictions[node.child1()];
+            if (prediction) {
+                if (prediction & ~PredictObjectMask) {
+                    prediction &= ~PredictObjectMask;
+                    prediction |= PredictObjectUnknown;
+                }
+                changed |= mergePrediction(prediction);
+            }
             break;
         }
@@ -572 +579 @@
         case ThrowReferenceError:
         case ForceOSRExit:
-            break;
-            
-        // This gets ignored because it doesn't do anything.
-        case Phantom:
+        case SetArgument:
         case PutByVal:
         case PutByValAlias:
         case PutById:
         case PutByIdDirect:
+            break;
+            
+        // This gets ignored because it doesn't do anything.
+        case Phantom:
             break;
 #else

trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

@@ -156 +156 @@
 void ValueSource::dump(FILE* out) const
 {
-    fprintf(out, "Node(%d)", m_nodeIndex);
+    switch (kind()) {
+    case SourceNotSet:
+        fprintf(out, "NotSet");
+        break;
+    case ValueInRegisterFile:
+        fprintf(out, "InRegFile");
+        break;
+    case Int32InRegisterFile:
+        fprintf(out, "Int32");
+        break;
+    case HaveNode:
+        fprintf(out, "Node(%d)", m_nodeIndex);
+        break;
+    }
 }
 
@@ -204 +217 @@
     ASSERT(m_bytecodeIndex != std::numeric_limits<uint32_t>::max());
     for (unsigned argument = 0; argument < m_arguments.size(); ++argument)
-        m_arguments[argument] = jit->computeValueRecoveryFor(operandForArgument(argument), jit->m_arguments[argument]);
+        m_arguments[argument] = jit->computeValueRecoveryFor(jit->m_arguments[argument]);
     for (unsigned variable = 0; variable < m_variables.size(); ++variable)
-        m_variables[variable] = jit->computeValueRecoveryFor(variable, jit->m_variables[variable]);
+        m_variables[variable] = jit->computeValueRecoveryFor(jit->m_variables[variable]);
 }
 
@@ -723 +736 @@
 
     case GetLocal: {
-        PredictedType prediction = m_jit.graph().getPrediction(node.local());
+        PredictedType prediction = node.variableAccessData()->prediction();
 
         // If we have no prediction for this local, then don't attempt to compile.
@@ -798 +811 @@
         m_bytecodeIndexForOSR = nextNode.codeOrigin.bytecodeIndex();
         
-        PredictedType predictedType = m_jit.graph().getPrediction(node.local());
+        PredictedType predictedType = node.variableAccessData()->prediction();
         if (isInt32Prediction(predictedType)) {
             SpeculateIntegerOperand value(this, node.child1());
@@ -821 +834 @@
         // Indicate that it's no longer necessary to retrieve the value of
         // this bytecode variable from registers or other locations in the register file.
-        valueSourceReferenceForOperand(node.local()) = ValueSource();
-        break;
-    }
+        valueSourceReferenceForOperand(node.local()) = ValueSource::forPrediction(predictedType);
+        break;
+    }
+        
+    case SetArgument:
+        // This is a no-op; it just marks the fact that the argument is being used.
+        // But it may be profitable to use this as a hook to run speculation checks
+        // on arguments, thereby allowing us to trivially eliminate such checks if
+        // the argument is not used.
+        break;
 
     case BitAnd:
@@ -2202 +2222 @@
 #endif
     
-    for (size_t i = 0; i < m_arguments.size(); ++i)
-        m_arguments[i] = ValueSource();
-    for (size_t i = 0; i < m_variables.size(); ++i)
-        m_variables[i] = ValueSource();
+    ASSERT(m_arguments.size() == block.m_argumentsAtHead.size());
+    for (size_t i = 0; i < m_arguments.size(); ++i) {
+        NodeIndex nodeIndex = block.m_argumentsAtHead[i].value;
+        if (nodeIndex == NoNode)
+            m_arguments[i] = ValueSource(ValueInRegisterFile);
+        else
+            m_arguments[i] = ValueSource::forPrediction(m_jit.graph()[nodeIndex].variableAccessData()->prediction());
+    }
+    
+    ASSERT(m_variables.size() == block.m_localsAtHead.size());
+    for (size_t i = 0; i < m_variables.size(); ++i) {
+        NodeIndex nodeIndex = block.m_localsAtHead[i].value;
+        if (nodeIndex == NoNode)
+            m_variables[i] = ValueSource(ValueInRegisterFile);
+        else
+            m_variables[i] = ValueSource::forPrediction(m_jit.graph()[nodeIndex].variableAccessData()->prediction());
+    }
+    
     m_lastSetOperand = std::numeric_limits<int>::max();
     m_bytecodeIndexForOSR = std::numeric_limits<uint32_t>::max();
@@ -2276 +2310 @@
     ASSERT(!m_compileIndex);
     m_bytecodeIndexForOSR = 0;
+
+    for (size_t i = 0; i < m_arguments.size(); ++i)
+        m_arguments[i] = ValueSource(ValueInRegisterFile);
+    for (size_t i = 0; i < m_variables.size(); ++i)
+        m_variables[i] = ValueSource(ValueInRegisterFile);
+    
     for (int i = 0; i < m_jit.codeBlock()->m_numParameters; ++i) {
         VirtualRegister virtualRegister = (VirtualRegister)(m_jit.codeBlock()->thisRegister() + i);
-        PredictedType predictedType = m_jit.graph().getPrediction(virtualRegister);
+        PredictedType predictedType = m_jit.graph()[m_jit.graph().m_arguments[i]].variableAccessData()->prediction();
         if (isInt32Prediction(predictedType))
             speculationCheck(m_jit.branchPtr(MacroAssembler::Below, JITCompiler::addressFor(virtualRegister), GPRInfo::tagTypeNumberRegister));
@@ -2306 +2346 @@
 }
 
-ValueRecovery SpeculativeJIT::computeValueRecoveryFor(int operand, const ValueSource& valueSource)
+ValueRecovery SpeculativeJIT::computeValueRecoveryFor(const ValueSource& valueSource)
 {
-    if (!valueSource.isSet()) {
-        if (m_bytecodeIndexForOSR && isInt32Prediction(m_jit.graph().getPrediction(operand)))
-            return ValueRecovery::alreadyInRegisterFileAsUnboxedInt32();
+    switch (valueSource.kind()) {
+    case ValueInRegisterFile:
         return ValueRecovery::alreadyInRegisterFile();
-    }
-
-    if (m_jit.isConstant(valueSource.nodeIndex()))
-        return ValueRecovery::constant(m_jit.valueOfJSConstant(valueSource.nodeIndex()));
-    
-    Node* nodePtr = &m_jit.graph()[valueSource.nodeIndex()];
-    if (!nodePtr->shouldGenerate()) {
-        // It's legitimately dead. As in, nobody will ever use this node, or operand,
-        // ever. Set it to Undefined to make the GC happy after the OSR.
-        return ValueRecovery::constant(jsUndefined());
-    }
-    
-    GenerationInfo* infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
-    if (!infoPtr->alive() || infoPtr->nodeIndex() != valueSource.nodeIndex()) {
-        // Try to see if there is an alternate node that would contain the value we want.
-        // There are four possibilities:
-        //
-        // ValueToNumber: If the only live version of the value is a ValueToNumber node
-        //    then it means that all remaining uses of the value would have performed a
-        //    ValueToNumber conversion anyway. Thus, we can substitute ValueToNumber.
-        //
-        // ValueToInt32: Likewise, if the only remaining live version of the value is
-        //    ValueToInt32, then we can use it. But if there is both a ValueToInt32
-        //    and a ValueToNumber, then we better go with ValueToNumber because it
-        //    means that some remaining uses would have converted to number while
-        //    others would have converted to Int32.
-        //
-        // UInt32ToNumber: If the only live version of the value is a UInt32ToNumber
-        //    then the only remaining uses are ones that want a properly formed number
-        //    rather than a UInt32 intermediate.
-        //
-        // The reverse of the above: This node could be a UInt32ToNumber, but its
-        //    alternative is still alive. This means that the only remaining uses of
-        //    the number would be fine with a UInt32 intermediate.
-        
-        bool found = false;
-        
-        if (nodePtr->op == UInt32ToNumber) {
-            NodeIndex nodeIndex = nodePtr->child1();
-            nodePtr = &m_jit.graph()[nodeIndex];
-            infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
-            if (infoPtr->alive() && infoPtr->nodeIndex() == nodeIndex)
-                found = true;
-        }
-        
-        if (!found) {
-            NodeIndex valueToNumberIndex = NoNode;
-            NodeIndex valueToInt32Index = NoNode;
-            NodeIndex uint32ToNumberIndex = NoNode;
-            
-            for (unsigned virtualRegister = 0; virtualRegister < m_generationInfo.size(); ++virtualRegister) {
-                GenerationInfo& info = m_generationInfo[virtualRegister];
-                if (!info.alive())
-                    continue;
-                if (info.nodeIndex() == NoNode)
-                    continue;
-                Node& node = m_jit.graph()[info.nodeIndex()];
-                if (node.child1Unchecked() != valueSource.nodeIndex())
-                    continue;
-                switch (node.op) {
-                case ValueToNumber:
-                case ValueToDouble:
-                    valueToNumberIndex = info.nodeIndex();
-                    break;
-                case ValueToInt32:
-                    valueToInt32Index = info.nodeIndex();
-                    break;
-                case UInt32ToNumber:
-                    uint32ToNumberIndex = info.nodeIndex();
-                    break;
-                default:
-                    break;
+        
+    case Int32InRegisterFile:
+        return ValueRecovery::alreadyInRegisterFileAsUnboxedInt32();
+        
+    case HaveNode: {
+        if (m_jit.isConstant(valueSource.nodeIndex()))
+            return ValueRecovery::constant(m_jit.valueOfJSConstant(valueSource.nodeIndex()));
+    
+        Node* nodePtr = &m_jit.graph()[valueSource.nodeIndex()];
+        if (!nodePtr->shouldGenerate()) {
+            // It's legitimately dead. As in, nobody will ever use this node, or operand,
+            // ever. Set it to Undefined to make the GC happy after the OSR.
+            return ValueRecovery::constant(jsUndefined());
+        }
+    
+        GenerationInfo* infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
+        if (!infoPtr->alive() || infoPtr->nodeIndex() != valueSource.nodeIndex()) {
+            // Try to see if there is an alternate node that would contain the value we want.
+            // There are four possibilities:
+            //
+            // ValueToNumber: If the only live version of the value is a ValueToNumber node
+            //    then it means that all remaining uses of the value would have performed a
+            //    ValueToNumber conversion anyway. Thus, we can substitute ValueToNumber.
+            //
+            // ValueToInt32: Likewise, if the only remaining live version of the value is
+            //    ValueToInt32, then we can use it. But if there is both a ValueToInt32
+            //    and a ValueToNumber, then we better go with ValueToNumber because it
+            //    means that some remaining uses would have converted to number while
+            //    others would have converted to Int32.
+            //
+            // UInt32ToNumber: If the only live version of the value is a UInt32ToNumber
+            //    then the only remaining uses are ones that want a properly formed number
+            //    rather than a UInt32 intermediate.
+            //
+            // The reverse of the above: This node could be a UInt32ToNumber, but its
+            //    alternative is still alive. This means that the only remaining uses of
+            //    the number would be fine with a UInt32 intermediate.
+        
+            bool found = false;
+        
+            if (nodePtr->op == UInt32ToNumber) {
+                NodeIndex nodeIndex = nodePtr->child1();
+                nodePtr = &m_jit.graph()[nodeIndex];
+                infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
+                if (infoPtr->alive() && infoPtr->nodeIndex() == nodeIndex)
+                    found = true;
+            }
+        
+            if (!found) {
+                NodeIndex valueToNumberIndex = NoNode;
+                NodeIndex valueToInt32Index = NoNode;
+                NodeIndex uint32ToNumberIndex = NoNode;
+            
+                for (unsigned virtualRegister = 0; virtualRegister < m_generationInfo.size(); ++virtualRegister) {
+                    GenerationInfo& info = m_generationInfo[virtualRegister];
+                    if (!info.alive())
+                        continue;
+                    if (info.nodeIndex() == NoNode)
+                        continue;
+                    Node& node = m_jit.graph()[info.nodeIndex()];
+                    if (node.child1Unchecked() != valueSource.nodeIndex())
+                        continue;
+                    switch (node.op) {
+                    case ValueToNumber:
+                    case ValueToDouble:
+                        valueToNumberIndex = info.nodeIndex();
+                        break;
+                    case ValueToInt32:
+                        valueToInt32Index = info.nodeIndex();
+                        break;
+                    case UInt32ToNumber:
+                        uint32ToNumberIndex = info.nodeIndex();
+                        break;
+                    default:
+                        break;
+                    }
+                }
+            
+                NodeIndex nodeIndexToUse;
+                if (valueToNumberIndex != NoNode)
+                    nodeIndexToUse = valueToNumberIndex;
+                else if (valueToInt32Index != NoNode)
+                    nodeIndexToUse = valueToInt32Index;
+                else if (uint32ToNumberIndex != NoNode)
+                    nodeIndexToUse = uint32ToNumberIndex;
+                else
+                    nodeIndexToUse = NoNode;
+            
+                if (nodeIndexToUse != NoNode) {
+                    nodePtr = &m_jit.graph()[nodeIndexToUse];
+                    infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
+                    ASSERT(infoPtr->alive() && infoPtr->nodeIndex() == nodeIndexToUse);
+                    found = true;
                 }
             }
-            
-            NodeIndex nodeIndexToUse;
-            if (valueToNumberIndex != NoNode)
-                nodeIndexToUse = valueToNumberIndex;
-            else if (valueToInt32Index != NoNode)
-                nodeIndexToUse = valueToInt32Index;
-            else if (uint32ToNumberIndex != NoNode)
-                nodeIndexToUse = uint32ToNumberIndex;
-            else
-                nodeIndexToUse = NoNode;
-            
-            if (nodeIndexToUse != NoNode) {
-                nodePtr = &m_jit.graph()[nodeIndexToUse];
-                infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
-                ASSERT(infoPtr->alive() && infoPtr->nodeIndex() == nodeIndexToUse);
-                found = true;
-            }
-        }
-        
-        if (!found)
-            return ValueRecovery::constant(jsUndefined());
-    }
-    
-    ASSERT(infoPtr->alive());
-
-    if (infoPtr->registerFormat() != DataFormatNone) {
-        if (infoPtr->registerFormat() == DataFormatDouble)
-            return ValueRecovery::inFPR(infoPtr->fpr());
-        return ValueRecovery::inGPR(infoPtr->gpr(), infoPtr->registerFormat());
-    }
-    if (infoPtr->spillFormat() != DataFormatNone)
-        return ValueRecovery::displacedInRegisterFile(static_cast<VirtualRegister>(nodePtr->virtualRegister()));
-    
-    ASSERT_NOT_REACHED();
-    return ValueRecovery();
+        
+            if (!found)
+                return ValueRecovery::constant(jsUndefined());
+        }
+    
+        ASSERT(infoPtr->alive());
+
+        if (infoPtr->registerFormat() != DataFormatNone) {
+            if (infoPtr->registerFormat() == DataFormatDouble)
+                return ValueRecovery::inFPR(infoPtr->fpr());
+            return ValueRecovery::inGPR(infoPtr->gpr(), infoPtr->registerFormat());
+        }
+        if (infoPtr->spillFormat() != DataFormatNone)
+            return ValueRecovery::displacedInRegisterFile(static_cast<VirtualRegister>(nodePtr->virtualRegister()));
+    
+        ASSERT_NOT_REACHED();
+        return ValueRecovery();
+    }
+        
+    default:
+        ASSERT_NOT_REACHED();
+        return ValueRecovery();
+    }
 }
 

trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h

@@ -67 +67 @@
 };
 
+enum ValueSourceKind {
+    SourceNotSet,
+    ValueInRegisterFile,
+    Int32InRegisterFile,
+    HaveNode
+};
+
 class ValueSource {
 public:
     ValueSource()
-        : m_nodeIndex(NoNode)
-    {
+        : m_nodeIndex(nodeIndexFromKind(SourceNotSet))
+    {
+    }
+    
+    explicit ValueSource(ValueSourceKind valueSourceKind)
+        : m_nodeIndex(nodeIndexFromKind(valueSourceKind))
+    {
+        ASSERT(kind() != SourceNotSet);
+        ASSERT(kind() != HaveNode);
     }
     
@@ -77 +91 @@
         : m_nodeIndex(nodeIndex)
     {
+        ASSERT(kind() == HaveNode);
+    }
+    
+    static ValueSource forPrediction(PredictedType prediction)
+    {
+        if (isInt32Prediction(prediction))
+            return ValueSource(Int32InRegisterFile);
+        return ValueSource(ValueInRegisterFile);
     }
     
     bool isSet() const
     {
-        return m_nodeIndex != NoNode;
+        return kindFromNodeIndex(m_nodeIndex) != SourceNotSet;
+    }
+    
+    ValueSourceKind kind() const
+    {
+        return kindFromNodeIndex(m_nodeIndex);
     }
     
     NodeIndex nodeIndex() const
     {
-        ASSERT(isSet());
+        ASSERT(kind() == HaveNode);
         return m_nodeIndex;
     }
@@ -95 +122 @@
     
 private:
+    static NodeIndex nodeIndexFromKind(ValueSourceKind kind)
+    {
+        ASSERT(kind >= SourceNotSet && kind < HaveNode);
+        return NoNode - kind;
+    }
+    
+    static ValueSourceKind kindFromNodeIndex(NodeIndex nodeIndex)
+    {
+        unsigned kind = static_cast<unsigned>(NoNode - nodeIndex);
+        if (kind >= static_cast<unsigned>(HaveNode))
+            return HaveNode;
+        return static_cast<ValueSourceKind>(kind);
+    }
+    
     NodeIndex m_nodeIndex;
 };
@@ -518 +559 @@
         switch (node.op) {
         case GetLocal:
-            return isArrayPrediction(m_jit.graph().getPrediction(node.local()));
+            return isArrayPrediction(node.variableAccessData()->prediction());
             
         case NewArray:
@@ -591 +632 @@
     // Tracking for which nodes are currently holding the values of arguments and bytecode
     // operand-indexed variables.
-
+    
     ValueSource valueSourceForOperand(int operand)
     {
@@ -624 +665 @@
     uint32_t m_bytecodeIndexForOSR;
     
-    ValueRecovery computeValueRecoveryFor(int operand, const ValueSource&);
+    ValueRecovery computeValueRecoveryFor(const ValueSource&);
 
     ValueRecovery computeValueRecoveryFor(int operand)
     {
-        return computeValueRecoveryFor(operand, valueSourceForOperand(operand));
+        return computeValueRecoveryFor(valueSourceForOperand(operand));
     }
 };
@@ -853 +894 @@
     , m_compileOkay(true)
     , m_arguments(jit.codeBlock()->m_numParameters)
-    , m_variables(jit.codeBlock()->m_numVars)
+    , m_variables(jit.graph().m_localVars)
     , m_lastSetOperand(std::numeric_limits<int>::max())
     , m_bytecodeIndexForOSR(std::numeric_limits<uint32_t>::max())

trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp

@@ -118 +118 @@
 void ValueSource::dump(FILE* out) const
 {
-    fprintf(out, "Node(%d)", m_nodeIndex);
+    switch (kind()) {
+    case SourceNotSet:
+        fprintf(out, "NotSet");
+        break;
+    case ValueInRegisterFile:
+        fprintf(out, "InRegFile");
+        break;
+    case Int32InRegisterFile:
+        fprintf(out, "Int32");
+        break;
+    case HaveNode:
+        fprintf(out, "Node(%d)", m_nodeIndex);
+        break;
+    }
 }
 
@@ -171 +184 @@
     ASSERT(m_bytecodeIndex != std::numeric_limits<uint32_t>::max());
     for (unsigned argument = 0; argument < m_arguments.size(); ++argument)
-        m_arguments[argument] = jit->computeValueRecoveryFor(operandForArgument(argument), jit->m_arguments[argument]);
+        m_arguments[argument] = jit->computeValueRecoveryFor(jit->m_arguments[argument]);
     for (unsigned variable = 0; variable < m_variables.size(); ++variable)
-        m_variables[variable] = jit->computeValueRecoveryFor(variable, jit->m_variables[variable]);
+        m_variables[variable] = jit->computeValueRecoveryFor(jit->m_variables[variable]);
 }
 
@@ -632 +645 @@
 
     case GetLocal: {
-        PredictedType prediction = m_jit.graph().getPrediction(node.local());
+        PredictedType prediction = node.variableAccessData()->prediction();
 
         // If we have no prediction for this local, then don't attempt to compile.
@@ -706 +719 @@
         m_bytecodeIndexForOSR = nextNode.codeOrigin.bytecodeIndex();
         
-        PredictedType predictedType = m_jit.graph().getPrediction(node.local());
+        PredictedType predictedType = node.variableAccessData()->prediction();
         if (isInt32Prediction(predictedType)) {
             SpeculateIntegerOperand value(this, node.child1());
@@ -726 +739 @@
         // Indicate that it's no longer necessary to retrieve the value of
         // this bytecode variable from registers or other locations in the register file.
-        valueSourceReferenceForOperand(node.local()) = ValueSource();
-        break;
-    }
+        valueSourceReferenceForOperand(node.local()) = ValueSource::forPrediction(predictedType);
+        break;
+    }
+
+    case SetArgument:
+        // This is a no-op; it just marks the fact that the argument is being used.
+        // But it may be profitable to use this as a hook to run speculation checks
+        // on arguments, thereby allowing us to trivially eliminate such checks if
+        // the argument is not used.
+        break;
 
     case BitAnd:
@@ -2135 +2155 @@
 #endif
 
-    for (size_t i = 0; i < m_arguments.size(); ++i)
-        m_arguments[i] = ValueSource();
-    for (size_t i = 0; i < m_variables.size(); ++i)
-        m_variables[i] = ValueSource();
+    ASSERT(m_arguments.size() == block.m_argumentsAtHead.size());
+    for (size_t i = 0; i < m_arguments.size(); ++i) {
+        NodeIndex nodeIndex = block.m_argumentsAtHead[i].value;
+        if (nodeIndex == NoNode)
+            m_arguments[i] = ValueSource(ValueInRegisterFile);
+        else
+            m_arguments[i] = ValueSource::forPrediction(m_jit.graph()[nodeIndex].variableAccessData()->prediction());
+    }
+    
+    ASSERT(m_variables.size() == block.m_localsAtHead.size());
+    for (size_t i = 0; i < m_variables.size(); ++i) {
+        NodeIndex nodeIndex = block.m_localsAtHead[i].value;
+        if (nodeIndex == NoNode)
+            m_variables[i] = ValueSource(ValueInRegisterFile);
+        else
+            m_variables[i] = ValueSource::forPrediction(m_jit.graph()[nodeIndex].variableAccessData()->prediction());
+    }
+    
     m_lastSetOperand = std::numeric_limits<int>::max();
     m_bytecodeIndexForOSR = std::numeric_limits<uint32_t>::max();
@@ -2211 +2245 @@
     ASSERT(!m_compileIndex);
     m_bytecodeIndexForOSR = 0;
+
+    for (size_t i = 0; i < m_arguments.size(); ++i)
+        m_arguments[i] = ValueSource(ValueInRegisterFile);
+    for (size_t i = 0; i < m_variables.size(); ++i)
+        m_variables[i] = ValueSource(ValueInRegisterFile);
+    
     for (int i = 0; i < m_jit.codeBlock()->m_numParameters; ++i) {
         VirtualRegister virtualRegister = (VirtualRegister)(m_jit.codeBlock()->thisRegister() + i);
-        PredictedType predictedType = m_jit.graph().getPrediction(virtualRegister);
+        PredictedType predictedType = m_jit.graph()[m_jit.graph().m_arguments[i]].variableAccessData()->prediction();
         if (isInt32Prediction(predictedType))
             speculationCheck(m_jit.branch32(MacroAssembler::NotEqual, JITCompiler::tagFor(virtualRegister), TrustedImm32(JSValue::Int32Tag)));
@@ -2237 +2277 @@
 }
 
-ValueRecovery SpeculativeJIT::computeValueRecoveryFor(int operand, const ValueSource& valueSource)
+ValueRecovery SpeculativeJIT::computeValueRecoveryFor(const ValueSource& valueSource)
 {
-    if (!valueSource.isSet()) {
-        if (m_bytecodeIndexForOSR && isInt32Prediction(m_jit.graph().getPrediction(operand)))
-            return ValueRecovery::alreadyInRegisterFileAsUnboxedInt32();
+    switch (valueSource.kind()) {
+    case ValueInRegisterFile:
         return ValueRecovery::alreadyInRegisterFile();
-    }
-
-    if (m_jit.isConstant(valueSource.nodeIndex()))
-        return ValueRecovery::constant(m_jit.valueOfJSConstant(valueSource.nodeIndex()));
-    
-    Node* nodePtr = &m_jit.graph()[valueSource.nodeIndex()];
-    if (!nodePtr->shouldGenerate()) {
-        // It's legitimately dead. As in, nobody will ever use this node, or operand,
-        // ever. Set it to Undefined to make the GC happy after the OSR.
-        return ValueRecovery::constant(jsUndefined());
-    }
-    
-    GenerationInfo* infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
-    if (!infoPtr->alive() || infoPtr->nodeIndex() != valueSource.nodeIndex()) {
-        // Try to see if there is an alternate node that would contain the value we want.
-        // There are four possibilities:
-        //
-        // ValueToNumber: If the only live version of the value is a ValueToNumber node
-        //    then it means that all remaining uses of the value would have performed a
-        //    ValueToNumber conversion anyway. Thus, we can substitute ValueToNumber.
-        //
-        // ValueToInt32: Likewise, if the only remaining live version of the value is
-        //    ValueToInt32, then we can use it. But if there is both a ValueToInt32
-        //    and a ValueToNumber, then we better go with ValueToNumber because it
-        //    means that some remaining uses would have converted to number while
-        //    others would have converted to Int32.
-        //
-        // UInt32ToNumber: If the only live version of the value is a UInt32ToNumber
-        //    then the only remaining uses are ones that want a properly formed number
-        //    rather than a UInt32 intermediate.
-        //
-        // The reverse of the above: This node could be a UInt32ToNumber, but its
-        //    alternative is still alive. This means that the only remaining uses of
-        //    the number would be fine with a UInt32 intermediate.
-        
-        bool found = false;
-        
-        if (nodePtr->op == UInt32ToNumber) {
-            NodeIndex nodeIndex = nodePtr->child1();
-            nodePtr = &m_jit.graph()[nodeIndex];
-            infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
-            if (infoPtr->alive() && infoPtr->nodeIndex() == nodeIndex)
-                found = true;
-        }
-        
-        if (!found) {
-            NodeIndex valueToNumberIndex = NoNode;
-            NodeIndex valueToInt32Index = NoNode;
-            NodeIndex uint32ToNumberIndex = NoNode;
-            
-            for (unsigned virtualRegister = 0; virtualRegister < m_generationInfo.size(); ++virtualRegister) {
-                GenerationInfo& info = m_generationInfo[virtualRegister];
-                if (!info.alive())
-                    continue;
-                if (info.nodeIndex() == NoNode)
-                    continue;
-                Node& node = m_jit.graph()[info.nodeIndex()];
-                if (node.child1Unchecked() != valueSource.nodeIndex())
-                    continue;
-                switch (node.op) {
-                case ValueToNumber:
-                case ValueToDouble:
-                    valueToNumberIndex = info.nodeIndex();
-                    break;
-                case ValueToInt32:
-                    valueToInt32Index = info.nodeIndex();
-                    break;
-                case UInt32ToNumber:
-                    uint32ToNumberIndex = info.nodeIndex();
-                    break;
-                default:
-                    break;
+        
+    case Int32InRegisterFile:
+        return ValueRecovery::alreadyInRegisterFileAsUnboxedInt32();
+
+    case HaveNode: {
+        if (m_jit.isConstant(valueSource.nodeIndex()))
+            return ValueRecovery::constant(m_jit.valueOfJSConstant(valueSource.nodeIndex()));
+    
+        Node* nodePtr = &m_jit.graph()[valueSource.nodeIndex()];
+        if (!nodePtr->shouldGenerate()) {
+            // It's legitimately dead. As in, nobody will ever use this node, or operand,
+            // ever. Set it to Undefined to make the GC happy after the OSR.
+            return ValueRecovery::constant(jsUndefined());
+        }
+    
+        GenerationInfo* infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
+        if (!infoPtr->alive() || infoPtr->nodeIndex() != valueSource.nodeIndex()) {
+            // Try to see if there is an alternate node that would contain the value we want.
+            // There are four possibilities:
+            //
+            // ValueToNumber: If the only live version of the value is a ValueToNumber node
+            //    then it means that all remaining uses of the value would have performed a
+            //    ValueToNumber conversion anyway. Thus, we can substitute ValueToNumber.
+            //
+            // ValueToInt32: Likewise, if the only remaining live version of the value is
+            //    ValueToInt32, then we can use it. But if there is both a ValueToInt32
+            //    and a ValueToNumber, then we better go with ValueToNumber because it
+            //    means that some remaining uses would have converted to number while
+            //    others would have converted to Int32.
+            //
+            // UInt32ToNumber: If the only live version of the value is a UInt32ToNumber
+            //    then the only remaining uses are ones that want a properly formed number
+            //    rather than a UInt32 intermediate.
+            //
+            // The reverse of the above: This node could be a UInt32ToNumber, but its
+            //    alternative is still alive. This means that the only remaining uses of
+            //    the number would be fine with a UInt32 intermediate.
+        
+            bool found = false;
+        
+            if (nodePtr->op == UInt32ToNumber) {
+                NodeIndex nodeIndex = nodePtr->child1();
+                nodePtr = &m_jit.graph()[nodeIndex];
+                infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
+                if (infoPtr->alive() && infoPtr->nodeIndex() == nodeIndex)
+                    found = true;
+            }
+        
+            if (!found) {
+                NodeIndex valueToNumberIndex = NoNode;
+                NodeIndex valueToInt32Index = NoNode;
+                NodeIndex uint32ToNumberIndex = NoNode;
+            
+                for (unsigned virtualRegister = 0; virtualRegister < m_generationInfo.size(); ++virtualRegister) {
+                    GenerationInfo& info = m_generationInfo[virtualRegister];
+                    if (!info.alive())
+                        continue;
+                    if (info.nodeIndex() == NoNode)
+                        continue;
+                    Node& node = m_jit.graph()[info.nodeIndex()];
+                    if (node.child1Unchecked() != valueSource.nodeIndex())
+                        continue;
+                    switch (node.op) {
+                    case ValueToNumber:
+                    case ValueToDouble:
+                        valueToNumberIndex = info.nodeIndex();
+                        break;
+                    case ValueToInt32:
+                        valueToInt32Index = info.nodeIndex();
+                        break;
+                    case UInt32ToNumber:
+                        uint32ToNumberIndex = info.nodeIndex();
+                        break;
+                    default:
+                        break;
+                    }
+                }
+            
+                NodeIndex nodeIndexToUse;
+                if (valueToNumberIndex != NoNode)
+                    nodeIndexToUse = valueToNumberIndex;
+                else if (valueToInt32Index != NoNode)
+                    nodeIndexToUse = valueToInt32Index;
+                else if (uint32ToNumberIndex != NoNode)
+                    nodeIndexToUse = uint32ToNumberIndex;
+                else
+                    nodeIndexToUse = NoNode;
+            
+                if (nodeIndexToUse != NoNode) {
+                    nodePtr = &m_jit.graph()[nodeIndexToUse];
+                    infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
+                    ASSERT(infoPtr->alive() && infoPtr->nodeIndex() == nodeIndexToUse);
+                    found = true;
                 }
             }
-            
-            NodeIndex nodeIndexToUse;
-            if (valueToNumberIndex != NoNode)
-                nodeIndexToUse = valueToNumberIndex;
-            else if (valueToInt32Index != NoNode)
-                nodeIndexToUse = valueToInt32Index;
-            else if (uint32ToNumberIndex != NoNode)
-                nodeIndexToUse = uint32ToNumberIndex;
-            else
-                nodeIndexToUse = NoNode;
-            
-            if (nodeIndexToUse != NoNode) {
-                nodePtr = &m_jit.graph()[nodeIndexToUse];
-                infoPtr = &m_generationInfo[nodePtr->virtualRegister()];
-                ASSERT(infoPtr->alive() && infoPtr->nodeIndex() == nodeIndexToUse);
-                found = true;
-            }
-        }
-        
-        if (!found)
-            return ValueRecovery::constant(jsUndefined());
-    }
-    
-    ASSERT(infoPtr->alive());
-
-    if (infoPtr->registerFormat() != DataFormatNone) {
-        if (infoPtr->registerFormat() == DataFormatDouble)
-            return ValueRecovery::inFPR(infoPtr->fpr());
-        if (infoPtr->registerFormat() & DataFormatJS)
-            return ValueRecovery::inPair(infoPtr->tagGPR(), infoPtr->payloadGPR());
-        return ValueRecovery::inGPR(infoPtr->gpr(), infoPtr->registerFormat());
-    }
-    if (infoPtr->spillFormat() != DataFormatNone)
-        return ValueRecovery::displacedInRegisterFile(static_cast<VirtualRegister>(nodePtr->virtualRegister()));
-    
-    ASSERT_NOT_REACHED();
-    return ValueRecovery();
+        
+            if (!found)
+                return ValueRecovery::constant(jsUndefined());
+        }
+    
+        ASSERT(infoPtr->alive());
+
+        if (infoPtr->registerFormat() != DataFormatNone) {
+            if (infoPtr->registerFormat() == DataFormatDouble)
+                return ValueRecovery::inFPR(infoPtr->fpr());
+            if (infoPtr->registerFormat() & DataFormatJS)
+                return ValueRecovery::inPair(infoPtr->tagGPR(), infoPtr->payloadGPR());
+            return ValueRecovery::inGPR(infoPtr->gpr(), infoPtr->registerFormat());
+        }
+        if (infoPtr->spillFormat() != DataFormatNone)
+            return ValueRecovery::displacedInRegisterFile(static_cast<VirtualRegister>(nodePtr->virtualRegister()));
+    
+        ASSERT_NOT_REACHED();
+        return ValueRecovery();
+    }
+        
+    default:
+        ASSERT_NOT_REACHED();
+        return ValueRecovery();
+    }
 }