Changeset 192600 in webkit

Timestamp:

Nov 18, 2015 4:54:37 PM (8 years ago)

Author:

mark.lam@apple.com

Message:

Snippefy op_mul for the baseline JIT.
​https://bugs.webkit.org/show_bug.cgi?id=151393

Reviewed by Geoffrey Garen.

Benchmarks shows that perf is neutral on x86 and x86_64 with the DFG enabled.

With the DFG disabled (relying on the baseline JIT for perf), LongSpider
3d-morph shows a 7.6% regression. However, there are other benchmarks that shows
a progression e.g. on Kraken, audio-beat-detection and audio-fft.

Upon inspection of the generated code for 3d-morph, the only differences is the
added use of a scratch register for the result as well as a jump around the
code that handles double types. It does not look like we're generating bad code.
I'll consider the perf acceptable in aggregate.

• CMakeLists.txt:
• JavaScriptCore.vcxproj/JavaScriptCore.vcxproj:
• JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters:
• JavaScriptCore.xcodeproj/project.pbxproj:

• bytecode/PolymorphicAccess.cpp:

(JSC::AccessCase::generate):

• jit/AssemblyHelpers.h:

(JSC::AssemblyHelpers::boxInt32):

• jit/IntrinsicEmitter.cpp:

(JSC::AccessCase::emitIntrinsicGetter):

• Changed AssemblyHelpers::boxInt32() to take a TagRegistersMode. The pre-existing boxInt32() always assume that the tag registers are not available. Since we should assume we have tag registers by default, I also changed all the other clients to explicitly specify a more of DoNotHaveTagRegisters. That is except for the snippet generators that do have the tag registers.

• jit/JIT.h:
• jit/JITArithmetic.cpp:

(JSC::JIT::compileBinaryArithOpSlowCase):
(JSC::JIT::emit_op_div):
(JSC::JIT::emitSlow_op_add):
(JSC::JIT::emit_op_mul):
(JSC::JIT::emitSlow_op_mul):
(JSC::JIT::emit_op_sub):
(JSC::JIT::compileBinaryArithOp): Deleted.

• jit/JITArithmetic32_64.cpp:

(JSC::JIT::emitBinaryDoubleOp):
(JSC::JIT::emit_op_div):
(JSC::JIT::emit_op_mul): Deleted.
(JSC::JIT::emitSlow_op_mul): Deleted.

• jit/JITMulGenerator.cpp: Added.

(JSC::JITMulGenerator::generateFastPath):

• jit/JITMulGenerator.h: Added.

(JSC::JITMulGenerator::JITMulGenerator):
(JSC::JITMulGenerator::didEmitFastPath):
(JSC::JITMulGenerator::endJumpList):
(JSC::JITMulGenerator::slowPathJumpList):

• tests/stress/op_mul.js: Added.

(o1.valueOf):
(generateScenarios):
(printScenarios):
(testCases.func):
(func):
(initializeTestCases):
(stringifyIfNeeded):
(isIdentical):
(runTest):

• Tests that JIT op_mul results are equivalent to the expected values as defined by the LLINT.

Location:

trunk/Source/JavaScriptCore

Files:

• CMakeLists.txt (modified) (1 diff)
• ChangeLog (modified) (1 diff)
• JavaScriptCore.vcxproj/JavaScriptCore.vcxproj (modified) (2 diffs)
• JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters (modified) (2 diffs)
• JavaScriptCore.xcodeproj/project.pbxproj (modified) (5 diffs)
• bytecode/PolymorphicAccess.cpp (modified) (2 diffs)
• jit/AssemblyHelpers.h (modified) (1 diff)
• jit/IntrinsicEmitter.cpp (modified) (3 diffs)
• jit/JIT.h (modified) (1 diff)
• jit/JITArithmetic.cpp (modified) (7 diffs)
• jit/JITArithmetic32_64.cpp (modified) (3 diffs)
• jit/JITMulGenerator.cpp (added)
• jit/JITMulGenerator.h (added)
• tests/stress/op_mul.js (added)

trunk/Source/JavaScriptCore/CMakeLists.txt

@@ -446 +446 @@
     jit/JITExceptions.cpp
     jit/JITInlineCacheGenerator.cpp
+    jit/JITMulGenerator.cpp
     jit/JITOpcodes.cpp
     jit/JITOpcodes32_64.cpp

trunk/Source/JavaScriptCore/ChangeLog

@@ +1 @@
+2015-11-18  Mark Lam  <mark.lam@apple.com>
+
+        Snippefy op_mul for the baseline JIT.
+        https://bugs.webkit.org/show_bug.cgi?id=151393
+
+        Reviewed by Geoffrey Garen.
+
+        Benchmarks shows that perf is neutral on x86 and x86_64 with the DFG enabled.
+
+        With the DFG disabled (relying on the baseline JIT for perf), LongSpider
+        3d-morph shows a 7.6% regression.  However, there are other benchmarks that shows
+        a progression e.g. on Kraken, audio-beat-detection and audio-fft.
+
+        Upon inspection of the generated code for 3d-morph, the only differences is the
+        added use of a scratch register for the result as well as a jump around the
+        code that handles double types.  It does not look like we're generating bad code.
+        I'll consider the perf acceptable in aggregate.
+
+        * CMakeLists.txt:
+        * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj:
+        * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters:
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+
+        * bytecode/PolymorphicAccess.cpp:
+        (JSC::AccessCase::generate):
+        * jit/AssemblyHelpers.h:
+        (JSC::AssemblyHelpers::boxInt32):
+        * jit/IntrinsicEmitter.cpp:
+        (JSC::AccessCase::emitIntrinsicGetter):
+        - Changed AssemblyHelpers::boxInt32() to take a TagRegistersMode.
+          The pre-existing boxInt32() always assume that the tag registers are not
+          available.  Since we should assume we have tag registers by default, I also
+          changed all the other clients to explicitly specify a more of
+          DoNotHaveTagRegisters.  That is except for the snippet generators that do have
+          the tag registers.
+
+        * jit/JIT.h:
+        * jit/JITArithmetic.cpp:
+        (JSC::JIT::compileBinaryArithOpSlowCase):
+        (JSC::JIT::emit_op_div):
+        (JSC::JIT::emitSlow_op_add):
+        (JSC::JIT::emit_op_mul):
+        (JSC::JIT::emitSlow_op_mul):
+        (JSC::JIT::emit_op_sub):
+        (JSC::JIT::compileBinaryArithOp): Deleted.
+        * jit/JITArithmetic32_64.cpp:
+        (JSC::JIT::emitBinaryDoubleOp):
+        (JSC::JIT::emit_op_div):
+        (JSC::JIT::emit_op_mul): Deleted.
+        (JSC::JIT::emitSlow_op_mul): Deleted.
+        * jit/JITMulGenerator.cpp: Added.
+        (JSC::JITMulGenerator::generateFastPath):
+        * jit/JITMulGenerator.h: Added.
+        (JSC::JITMulGenerator::JITMulGenerator):
+        (JSC::JITMulGenerator::didEmitFastPath):
+        (JSC::JITMulGenerator::endJumpList):
+        (JSC::JITMulGenerator::slowPathJumpList):
+
+        * tests/stress/op_mul.js: Added.
+        (o1.valueOf):
+        (generateScenarios):
+        (printScenarios):
+        (testCases.func):
+        (func):
+        (initializeTestCases):
+        (stringifyIfNeeded):
+        (isIdentical):
+        (runTest):
+        - Tests that JIT op_mul results are equivalent to the expected values as
+          defined by the LLINT.
+
 2015-11-18  Mark Lam  <mark.lam@apple.com>
 

trunk/Source/JavaScriptCore/JavaScriptCore.vcxproj/JavaScriptCore.vcxproj

@@ -650 +650 @@
     <ClCompile Include="..\jit\JITExceptions.cpp" />
     <ClCompile Include="..\jit\JITInlineCacheGenerator.cpp" />
+    <ClCompile Include="..\jit\JITMulGenerator.cpp" />
     <ClCompile Include="..\jit\JITOpcodes.cpp" />
     <ClCompile Include="..\jit\JITOpcodes32_64.cpp" />
@@ -1468 +1469 @@
     <ClInclude Include="..\jit\JITInlineCacheGenerator.h" />
     <ClInclude Include="..\jit\JITInlines.h" />
+    <ClInclude Include="..\jit\JITMulGenerator.h" />
     <ClInclude Include="..\jit\JITOperations.h" />
     <ClInclude Include="..\jit\JITStubRoutine.h" />

trunk/Source/JavaScriptCore/JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters

@@ -445 +445 @@
       <Filter>jit</Filter>
     </ClCompile>
+    <ClCompile Include="..\jit\JITMulGenerator.cpp">
+      <Filter>jit</Filter>
+    </ClCompile>
     <ClCompile Include="..\jit\JITOpcodes.cpp">
       <Filter>jit</Filter>
@@ -2518 +2521 @@
     </ClInclude>
     <ClInclude Include="..\jit\JITInlines.h">
+      <Filter>jit</Filter>
+    </ClInclude>
+    <ClInclude Include="..\jit\JITMulGenerator.h">
       <Filter>jit</Filter>
     </ClInclude>

trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

@@ -1963 +1963 @@
                 FE1220271BE7F58C0039E6F2 /* JITAddGenerator.h in Headers */ = {isa = PBXBuildFile; fileRef = FE1220261BE7F5640039E6F2 /* JITAddGenerator.h */; };
                 FE1220281BE7F5910039E6F2 /* JITAddGenerator.cpp in Sources */ = {isa = PBXBuildFile; fileRef = FE1220251BE7F5640039E6F2 /* JITAddGenerator.cpp */; };
+                FE187A011BFBE55E0038BBCA /* JITMulGenerator.cpp in Sources */ = {isa = PBXBuildFile; fileRef = FE1879FF1BFBC73C0038BBCA /* JITMulGenerator.cpp */; settings = {ASSET_TAGS = (); }; };
+                FE187A021BFBE5610038BBCA /* JITMulGenerator.h in Headers */ = {isa = PBXBuildFile; fileRef = FE187A001BFBC73C0038BBCA /* JITMulGenerator.h */; settings = {ASSET_TAGS = (); }; };
                 FE1C0FFD1B193E9800B53FCA /* Exception.h in Headers */ = {isa = PBXBuildFile; fileRef = FE1C0FFC1B193E9800B53FCA /* Exception.h */; settings = {ATTRIBUTES = (Private, ); }; };
                 FE1C0FFF1B194FD100B53FCA /* Exception.cpp in Sources */ = {isa = PBXBuildFile; fileRef = FE1C0FFE1B194FD100B53FCA /* Exception.cpp */; };
@@ -4093 +4095 @@
                 FE1220251BE7F5640039E6F2 /* JITAddGenerator.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JITAddGenerator.cpp; sourceTree = "<group>"; };
                 FE1220261BE7F5640039E6F2 /* JITAddGenerator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITAddGenerator.h; sourceTree = "<group>"; };
+                FE1879FF1BFBC73C0038BBCA /* JITMulGenerator.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JITMulGenerator.cpp; sourceTree = "<group>"; };
+                FE187A001BFBC73C0038BBCA /* JITMulGenerator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMulGenerator.h; sourceTree = "<group>"; };
                 FE1C0FFC1B193E9800B53FCA /* Exception.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Exception.h; sourceTree = "<group>"; };
                 FE1C0FFE1B194FD100B53FCA /* Exception.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = Exception.cpp; sourceTree = "<group>"; };
@@ -4775 +4779 @@
                                 0FB14E1D18124ACE009B6B4D /* JITInlineCacheGenerator.h */,
                                 86CC85A00EE79A4700288682 /* JITInlines.h */,
+                                FE1879FF1BFBC73C0038BBCA /* JITMulGenerator.cpp */,
+                                FE187A001BFBC73C0038BBCA /* JITMulGenerator.h */,
                                 BCDD51E90FB8DF74004A8BDC /* JITOpcodes.cpp */,
                                 A71236E41195F33C00BD2174 /* JITOpcodes32_64.cpp */,
@@ -7418 +7424 @@
                                 BC18C4310E16F5CD00B34460 /* Lexer.h in Headers */,
                                 BC18C52E0E16FCE100B34460 /* Lexer.lut.h in Headers */,
+                                FE187A021BFBE5610038BBCA /* JITMulGenerator.h in Headers */,
                                 86D3B3C310159D7F002865E7 /* LinkBuffer.h in Headers */,
                                 0F431738146BAC69007E3890 /* ListableHandler.h in Headers */,
@@ -8962 +8969 @@
                                 FE5932A7183C5A2600A1ECCC /* VMEntryScope.cpp in Sources */,
                                 7B0247561B8682E100542440 /* WASMFunctionParser.cpp in Sources */,
+                                FE187A011BFBE55E0038BBCA /* JITMulGenerator.cpp in Sources */,
                                 7B39F76D1B62DE2E00360FB4 /* WASMModuleParser.cpp in Sources */,
                                 7B39F7721B63574D00360FB4 /* WASMReader.cpp in Sources */,

trunk/Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp

@@ -1059 +1059 @@
         state.failAndIgnore.append(
             jit.branch32(CCallHelpers::LessThan, scratchGPR, CCallHelpers::TrustedImm32(0)));
-        jit.boxInt32(scratchGPR, valueRegs);
+        jit.boxInt32(scratchGPR, valueRegs, CCallHelpers::DoNotHaveTagRegisters);
         state.succeed();
         return;
@@ -1066 +1066 @@
     case StringLength: {
         jit.load32(CCallHelpers::Address(baseGPR, JSString::offsetOfLength()), valueRegs.payloadGPR());
-        jit.boxInt32(valueRegs.payloadGPR(), valueRegs);
+        jit.boxInt32(valueRegs.payloadGPR(), valueRegs, CCallHelpers::DoNotHaveTagRegisters);
         state.succeed();
         return;

trunk/Source/JavaScriptCore/jit/AssemblyHelpers.h

@@ -1113 +1113 @@
     }
 
-    void boxInt32(GPRReg intGPR, JSValueRegs boxedRegs)
-    {
-#if USE(JSVALUE64)
-        move(intGPR, boxedRegs.gpr());
-        or64(TrustedImm64(TagTypeNumber), boxedRegs.gpr());
-#else
+    void boxInt32(GPRReg intGPR, JSValueRegs boxedRegs, TagRegistersMode mode = HaveTagRegisters)
+    {
+#if USE(JSVALUE64)
+        if (mode == DoNotHaveTagRegisters) {
+            move(intGPR, boxedRegs.gpr());
+            or64(TrustedImm64(TagTypeNumber), boxedRegs.gpr());
+        } else
+            or64(GPRInfo::tagTypeNumberRegister, intGPR, boxedRegs.gpr());
+#else
+        UNUSED_PARAM(mode);
         move(intGPR, boxedRegs.payloadGPR());
         move(TrustedImm32(JSValue::Int32Tag), boxedRegs.tagGPR());

trunk/Source/JavaScriptCore/jit/IntrinsicEmitter.cpp

@@ -78 +78 @@
     case TypedArrayLengthIntrinsic: {
         jit.load32(MacroAssembler::Address(state.baseGPR, JSArrayBufferView::offsetOfLength()), valueGPR);
-        jit.boxInt32(valueGPR, valueRegs);
+        jit.boxInt32(valueGPR, valueRegs, CCallHelpers::DoNotHaveTagRegisters);
         state.succeed();
         return;
@@ -93 +93 @@
         }
 
-        jit.boxInt32(valueGPR, valueRegs);
+        jit.boxInt32(valueGPR, valueRegs, CCallHelpers::DoNotHaveTagRegisters);
         state.succeed();
         return;
@@ -119 +119 @@
         done.link(&jit);
         
-        jit.boxInt32(valueGPR, valueRegs);
+        jit.boxInt32(valueGPR, valueRegs, CCallHelpers::DoNotHaveTagRegisters);
         state.succeed();
         return;

trunk/Source/JavaScriptCore/jit/JIT.h

@@ -463 +463 @@
 
         void emitTagBool(RegisterID);
-        void compileBinaryArithOp(OpcodeID, int dst, int src1, int src2, OperandTypes opi);
         void compileBinaryArithOpSlowCase(Instruction*, OpcodeID, Vector<SlowCaseEntry>::iterator&, int dst, int src1, int src2, OperandTypes, bool op1HasImmediateIntFastCase, bool op2HasImmediateIntFastCase);
 

trunk/Source/JavaScriptCore/jit/JITArithmetic.cpp

@@ -32 +32 @@
 #include "JITAddGenerator.h"
 #include "JITInlines.h"
+#include "JITMulGenerator.h"
 #include "JITOperations.h"
 #include "JITSubGenerator.h"
@@ -663 +664 @@
 /* ------------------------------ BEGIN: USE(JSVALUE64) (OP_ADD, OP_SUB, OP_MUL) ------------------------------ */
 
-void JIT::compileBinaryArithOp(OpcodeID opcodeID, int, int op1, int op2, OperandTypes)
-{
-    emitGetVirtualRegisters(op1, regT0, op2, regT1);
-    emitJumpSlowCaseIfNotInt(regT0);
-    emitJumpSlowCaseIfNotInt(regT1);
-    ASSERT_UNUSED(opcodeID, opcodeID == op_mul);
-    if (shouldEmitProfiling()) {
-        // We want to be able to measure if this is taking the slow case just
-        // because of negative zero. If this produces positive zero, then we
-        // don't want the slow case to be taken because that will throw off
-        // speculative compilation.
-        move(regT0, regT2);
-        addSlowCase(branchMul32(Overflow, regT1, regT2));
-        JumpList done;
-        done.append(branchTest32(NonZero, regT2));
-        Jump negativeZero = branch32(LessThan, regT0, TrustedImm32(0));
-        done.append(branch32(GreaterThanOrEqual, regT1, TrustedImm32(0)));
-        negativeZero.link(this);
-        // We only get here if we have a genuine negative zero. Record this,
-        // so that the speculative JIT knows that we failed speculation
-        // because of a negative zero.
-        add32(TrustedImm32(1), AbsoluteAddress(&m_codeBlock->addSpecialFastCaseProfile(m_bytecodeOffset)->m_counter));
-        addSlowCase(jump());
-        done.link(this);
-        move(regT2, regT0);
-    } else {
-        addSlowCase(branchMul32(Overflow, regT1, regT0));
-        addSlowCase(branchTest32(Zero, regT0));
-    }
-    emitTagInt(regT0, regT0);
-}
-
 void JIT::compileBinaryArithOpSlowCase(Instruction* currentInstruction, OpcodeID opcodeID, Vector<SlowCaseEntry>::iterator& iter, int result, int op1, int op2, OperandTypes types, bool op1HasImmediateIntFastCase, bool op2HasImmediateIntFastCase)
 {
@@ -712 +681 @@
 
     linkSlowCase(iter); // Integer overflow case - we could handle this in JIT code, but this is likely rare.
-    if (opcodeID == op_mul && !op1HasImmediateIntFastCase && !op2HasImmediateIntFastCase) // op_mul has an extra slow case to handle 0 * negative number.
-        linkSlowCase(iter);
 
     Label stubFunctionCall(this);
@@ -761 +728 @@
     }
 
-    if (opcodeID == op_mul)
-        mulDouble(fpRegT2, fpRegT1);
-    else {
-        ASSERT(opcodeID == op_div);
-        divDouble(fpRegT2, fpRegT1);
-    }
+    ASSERT_UNUSED(opcodeID, opcodeID == op_div);
+    divDouble(fpRegT2, fpRegT1);
+
     moveDoubleTo64(fpRegT1, regT0);
     sub64(tagTypeNumberRegister, regT0);
@@ -772 +736 @@
 
     end.link(this);
-}
-
-void JIT::emit_op_mul(Instruction* currentInstruction)
-{
-    int result = currentInstruction[1].u.operand;
-    int op1 = currentInstruction[2].u.operand;
-    int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
-
-    // For now, only plant a fast int case if the constant operand is greater than zero.
-    int32_t value;
-    if (isOperandConstantInt(op1) && ((value = getOperandConstantInt(op1)) > 0)) {
-        emitGetVirtualRegister(op2, regT0);
-        emitJumpSlowCaseIfNotInt(regT0);
-        addSlowCase(branchMul32(Overflow, regT0, Imm32(value), regT1));
-        emitTagInt(regT1, regT0);
-    } else if (isOperandConstantInt(op2) && ((value = getOperandConstantInt(op2)) > 0)) {
-        emitGetVirtualRegister(op1, regT0);
-        emitJumpSlowCaseIfNotInt(regT0);
-        addSlowCase(branchMul32(Overflow, regT0, Imm32(value), regT1));
-        emitTagInt(regT1, regT0);
-    } else
-        compileBinaryArithOp(op_mul, result, op1, op2, types);
-
-    emitPutVirtualRegister(result);
-}
-
-void JIT::emitSlow_op_mul(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
-{
-    int result = currentInstruction[1].u.operand;
-    int op1 = currentInstruction[2].u.operand;
-    int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
-
-    bool op1HasImmediateIntFastCase = isOperandConstantInt(op1) && getOperandConstantInt(op1) > 0;
-    bool op2HasImmediateIntFastCase = !op1HasImmediateIntFastCase && isOperandConstantInt(op2) && getOperandConstantInt(op2) > 0;
-    compileBinaryArithOpSlowCase(currentInstruction, op_mul, iter, result, op1, op2, types, op1HasImmediateIntFastCase, op2HasImmediateIntFastCase);
 }
 
@@ -980 +907 @@
 }
 
-void JIT::emit_op_sub(Instruction* currentInstruction)
+void JIT::emit_op_mul(Instruction* currentInstruction)
 {
     int result = currentInstruction[1].u.operand;
@@ -1001 +928 @@
 #endif
 
+    bool leftIsConstInt32 = isOperandConstantInt(op1);
+    bool rightIsConstInt32 = isOperandConstantInt(op2);
+    ResultType leftType = types.first();
+    ResultType rightType = types.second();
+    int32_t leftConstInt32 = 0;
+    int32_t rightConstInt32 = 0;
+
+    uint32_t* profilingCounter = nullptr;
+    if (shouldEmitProfiling())
+        profilingCounter = &m_codeBlock->addSpecialFastCaseProfile(m_bytecodeOffset)->m_counter;
+
+    ASSERT(!leftIsConstInt32 || !rightIsConstInt32);
+
+    if (leftIsConstInt32)
+        leftConstInt32 = getOperandConstantInt(op1);
+    if (rightIsConstInt32)
+        rightConstInt32 = getOperandConstantInt(op2);
+
+    bool leftIsPositiveConstInt32 = leftIsConstInt32 && (leftConstInt32 > 0);
+    bool rightIsPositiveConstInt32 = rightIsConstInt32 && (rightConstInt32 > 0);
+
+    if (leftIsPositiveConstInt32)
+        emitGetVirtualRegister(op2, rightRegs);
+    else if (rightIsPositiveConstInt32)
+        emitGetVirtualRegister(op1, leftRegs);
+    else {
+        emitGetVirtualRegister(op1, leftRegs);
+        emitGetVirtualRegister(op2, rightRegs);
+    }
+
+    JITMulGenerator gen(resultRegs, leftRegs, rightRegs, leftType, rightType,
+        leftIsPositiveConstInt32, rightIsPositiveConstInt32, leftConstInt32, rightConstInt32,
+        fpRegT0, fpRegT1, scratchGPR, scratchFPR, profilingCounter);
+
+    gen.generateFastPath(*this);
+
+    if (gen.didEmitFastPath()) {
+        gen.endJumpList().link(this);
+        emitPutVirtualRegister(result, resultRegs);
+
+        addSlowCase(gen.slowPathJumpList());
+    } else {
+        ASSERT(gen.endJumpList().empty());
+        ASSERT(gen.slowPathJumpList().empty());
+        JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);
+        slowPathCall.call();
+    }
+}
+
+void JIT::emitSlow_op_mul(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
+{
+    linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset);
+    
+    JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);
+    slowPathCall.call();
+}
+
+void JIT::emit_op_sub(Instruction* currentInstruction)
+{
+    int result = currentInstruction[1].u.operand;
+    int op1 = currentInstruction[2].u.operand;
+    int op2 = currentInstruction[3].u.operand;
+    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
+
+#if USE(JSVALUE64)
+    JSValueRegs leftRegs = JSValueRegs(regT0);
+    JSValueRegs rightRegs = JSValueRegs(regT1);
+    JSValueRegs resultRegs = leftRegs;
+    GPRReg scratchGPR = regT2;
+    FPRReg scratchFPR = InvalidFPRReg;
+#else
+    JSValueRegs leftRegs = JSValueRegs(regT1, regT0);
+    JSValueRegs rightRegs = JSValueRegs(regT3, regT2);
+    JSValueRegs resultRegs = leftRegs;
+    GPRReg scratchGPR = regT4;
+    FPRReg scratchFPR = fpRegT2;
+#endif
+
     emitGetVirtualRegister(op1, leftRegs);
     emitGetVirtualRegister(op2, rightRegs);

trunk/Source/JavaScriptCore/jit/JITArithmetic32_64.cpp

@@ -513 +513 @@
         doTheMath.link(this);
         switch (opcodeID) {
-            case op_mul:
-                emitLoadDouble(op1, fpRegT2);
-                mulDouble(fpRegT2, fpRegT0);
-                emitStoreDouble(dst, fpRegT0);
-                break;
             case op_div: {
                 emitLoadDouble(op1, fpRegT1);
@@ -604 +599 @@
         // Do the math.
         switch (opcodeID) {
-            case op_mul:
-                emitLoadDouble(op2, fpRegT2);
-                mulDouble(fpRegT2, fpRegT0);
-                emitStoreDouble(dst, fpRegT0);
-                break;
             case op_div: {
                 emitLoadDouble(op2, fpRegT2);
@@ -677 +667 @@
 }
 
-// Multiplication (*)
-
-void JIT::emit_op_mul(Instruction* currentInstruction)
-{
-    int dst = currentInstruction[1].u.operand;
-    int op1 = currentInstruction[2].u.operand;
-    int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
-
-    m_codeBlock->addSpecialFastCaseProfile(m_bytecodeOffset);
-
-    JumpList notInt32Op1;
-    JumpList notInt32Op2;
-
-    emitLoad2(op1, regT1, regT0, op2, regT3, regT2);
-    notInt32Op1.append(branch32(NotEqual, regT1, TrustedImm32(JSValue::Int32Tag)));
-    notInt32Op2.append(branch32(NotEqual, regT3, TrustedImm32(JSValue::Int32Tag)));
-
-    // Int32 case.
-    move(regT0, regT3);
-    addSlowCase(branchMul32(Overflow, regT2, regT0));
-    addSlowCase(branchTest32(Zero, regT0));
-    emitStoreInt32(dst, regT0, (op1 == dst || op2 == dst));
-
-    if (!supportsFloatingPoint()) {
-        addSlowCase(notInt32Op1);
-        addSlowCase(notInt32Op2);
-        return;
-    }
-    Jump end = jump();
-
-    // Double case.
-    emitBinaryDoubleOp(op_mul, dst, op1, op2, types, notInt32Op1, notInt32Op2);
-    end.link(this);
-}
-
-void JIT::emitSlow_op_mul(Instruction* currentInstruction, Vector<SlowCaseEntry>::iterator& iter)
-{
-    int dst = currentInstruction[1].u.operand;
-    int op1 = currentInstruction[2].u.operand;
-    int op2 = currentInstruction[3].u.operand;
-    OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand);
-
-    Jump overflow = getSlowCase(iter); // overflow check
-    linkSlowCase(iter); // zero result check
-
-    Jump negZero = branchOr32(Signed, regT2, regT3);
-    emitStoreInt32(dst, TrustedImm32(0), (op1 == dst || op2 == dst));
-
-    emitJumpSlowToHot(jump(), OPCODE_LENGTH(op_mul));
-
-    negZero.link(this);
-    // We only get here if we have a genuine negative zero. Record this,
-    // so that the speculative JIT knows that we failed speculation
-    // because of a negative zero.
-    add32(TrustedImm32(1), AbsoluteAddress(&m_codeBlock->specialFastCaseProfileForBytecodeOffset(m_bytecodeOffset)->m_counter));
-    overflow.link(this);
-
-    if (!supportsFloatingPoint()) {
-        linkSlowCase(iter); // int32 check
-        linkSlowCase(iter); // int32 check
-    }
-
-    if (supportsFloatingPoint()) {
-        if (!types.first().definitelyIsNumber())
-            linkSlowCase(iter); // double check
-
-        if (!types.second().definitelyIsNumber()) {
-            linkSlowCase(iter); // int32 check
-            linkSlowCase(iter); // double check
-        }
-    }
-
-    JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_mul);
-    slowPathCall.call();
-}
-
 // Division (/)