Changeset 203537 in webkit
- Timestamp:
- Jul 21, 2016 4:41:44 PM (8 years ago)
- Location:
- trunk/Source
- Files:
-
- 7 added
- 40 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/Source/JavaScriptCore/CMakeLists.txt
r203491 r203537 177 177 178 178 bytecode/AdaptiveInferredPropertyValueWatchpointBase.cpp 179 bytecode/ArithProfile.cpp 179 180 bytecode/ArrayAllocationProfile.cpp 180 181 bytecode/ArrayProfile.cpp -
trunk/Source/JavaScriptCore/ChangeLog
r203512 r203537 1 2016-07-21 Saam Barati <sbarati@apple.com> 2 3 op_add/ValueAdd should be an IC in all JIT tiers 4 https://bugs.webkit.org/show_bug.cgi?id=159649 5 6 Reviewed by Benjamin Poulain. 7 8 This patch makes Add an IC inside all JIT tiers. It does so in a 9 simple, but effective, way. We will try to generate an int+int add 10 that will repatch itself if its type checks fail. Sometimes though, 11 we have runtime type data saying that the add won't be int+int. 12 In those cases, we will just generate a full snippet that doesn't patch itself. 13 Other times, we may generate no inline code and defer to making a C call. A lot 14 of this patch is just refactoring ResultProfile into what we're now calling ArithProfile. 15 ArithProfile does everything ResultProfile used to do, and more. It records simple type 16 data about the LHS/RHS operands it sees. This allows us to determine if an op_add 17 has only seen int+int operands, etc. ArithProfile will also contain the ResultType 18 for the LHS/RHS that the parser feeds into op_add. ArithProfile now fits into 32-bits. 19 This means instead of having a side table like we did for ResultProfile, we just 20 inject the ArithProfile into the bytecode instruction stream. This makes asking 21 for ArithProfile faster; we no longer need to lock around this operation. 22 23 The size of an Add has gone down on average, but we can still do better. 24 We still generate a lot of code because we generate calls to the slow path. 25 I think we can make this better by moving the slow path to a shared thunk 26 system. This patch mostly lays the foundation for future improvements to Add, 27 and a framework to move all other arithmetic operations to be typed-based ICs. 28 29 Here is some data I took on the average op_add/ValueAdd size on various benchmarks: 30 | JetStream | Speedometer | Unity 3D | 31 ------| -------------|----------------------------- 32 Old | 189 bytes | 169 bytes | 192 bytes | 33 ------| -------------|----------------------------- 34 New | 148 bytes | 124 bytes | 143 bytes | 35 --------------------------------------------------- 36 37 Making an arithmetic IC is now easy. The JITMathIC class will hold a snippet 38 generator as a member variable. To make a snippet an IC, you need to implement 39 a generateInline(.) method, which generates the inline IC. Then, you need to 40 generate the IC where you used to generate the snippet. When generating the 41 IC, we need to inform JITMathIC of various data like we do with StructureStubInfo. 42 We need to tell it about where the slow path starts, where the slow path call is, etc. 43 When generating a JITMathIC, it may tell you that it didn't generate any code inline. 44 This is a request to the user of JITMathIC to just generate a C call along the 45 fast path. JITMathIC may also have the snippet tell it to just generate the full 46 snippet instead of the int+int path along the fast path. 47 48 In subsequent patches, we can improve upon how we decide to generate int+int or 49 the full snippet. I tried to get clever by having double+double, double+int, int+double, 50 fast paths, but they didn't work out nearly as well as the int+int fast path. I ended up 51 generating a lot of code when I did this and ended up using more memory than just generating 52 the full snippet. There is probably some way we can be clever and generate specialized fast 53 paths that are more successful than what I tried implementing, but I think that's worth deferring 54 this to follow up patches once the JITMathIC foundation has landed. 55 56 This patch also fixes a bug inside the slow path lambdas in the DFG. 57 Before, it was not legal to emit an exception check inside them. Now, 58 it is. So it's now easy to define arbitrary late paths using the DFG 59 slow path lambda API. 60 61 * CMakeLists.txt: 62 * JavaScriptCore.xcodeproj/project.pbxproj: 63 * bytecode/ArithProfile.cpp: Added. 64 (JSC::ArithProfile::emitObserveResult): 65 (JSC::ArithProfile::shouldEmitSetDouble): 66 (JSC::ArithProfile::emitSetDouble): 67 (JSC::ArithProfile::shouldEmitSetNonNumber): 68 (JSC::ArithProfile::emitSetNonNumber): 69 (WTF::printInternal): 70 * bytecode/ArithProfile.h: Added. 71 (JSC::ObservedType::ObservedType): 72 (JSC::ObservedType::sawInt32): 73 (JSC::ObservedType::isOnlyInt32): 74 (JSC::ObservedType::sawNumber): 75 (JSC::ObservedType::isOnlyNumber): 76 (JSC::ObservedType::sawNonNumber): 77 (JSC::ObservedType::isOnlyNonNumber): 78 (JSC::ObservedType::isEmpty): 79 (JSC::ObservedType::bits): 80 (JSC::ObservedType::withInt32): 81 (JSC::ObservedType::withNumber): 82 (JSC::ObservedType::withNonNumber): 83 (JSC::ObservedType::withoutNonNumber): 84 (JSC::ObservedType::operator==): 85 (JSC::ArithProfile::ArithProfile): 86 (JSC::ArithProfile::fromInt): 87 (JSC::ArithProfile::lhsResultType): 88 (JSC::ArithProfile::rhsResultType): 89 (JSC::ArithProfile::lhsObservedType): 90 (JSC::ArithProfile::rhsObservedType): 91 (JSC::ArithProfile::setLhsObservedType): 92 (JSC::ArithProfile::setRhsObservedType): 93 (JSC::ArithProfile::tookSpecialFastPath): 94 (JSC::ArithProfile::didObserveNonInt32): 95 (JSC::ArithProfile::didObserveDouble): 96 (JSC::ArithProfile::didObserveNonNegZeroDouble): 97 (JSC::ArithProfile::didObserveNegZeroDouble): 98 (JSC::ArithProfile::didObserveNonNumber): 99 (JSC::ArithProfile::didObserveInt32Overflow): 100 (JSC::ArithProfile::didObserveInt52Overflow): 101 (JSC::ArithProfile::setObservedNonNegZeroDouble): 102 (JSC::ArithProfile::setObservedNegZeroDouble): 103 (JSC::ArithProfile::setObservedNonNumber): 104 (JSC::ArithProfile::setObservedInt32Overflow): 105 (JSC::ArithProfile::setObservedInt52Overflow): 106 (JSC::ArithProfile::addressOfBits): 107 (JSC::ArithProfile::observeResult): 108 (JSC::ArithProfile::lhsSawInt32): 109 (JSC::ArithProfile::lhsSawNumber): 110 (JSC::ArithProfile::lhsSawNonNumber): 111 (JSC::ArithProfile::rhsSawInt32): 112 (JSC::ArithProfile::rhsSawNumber): 113 (JSC::ArithProfile::rhsSawNonNumber): 114 (JSC::ArithProfile::observeLHSAndRHS): 115 (JSC::ArithProfile::bits): 116 (JSC::ArithProfile::hasBits): 117 (JSC::ArithProfile::setBit): 118 * bytecode/CodeBlock.cpp: 119 (JSC::CodeBlock::dumpRareCaseProfile): 120 (JSC::CodeBlock::dumpArithProfile): 121 (JSC::CodeBlock::dumpBytecode): 122 (JSC::CodeBlock::addStubInfo): 123 (JSC::CodeBlock::addJITAddIC): 124 (JSC::CodeBlock::findStubInfo): 125 (JSC::CodeBlock::resetJITData): 126 (JSC::CodeBlock::shrinkToFit): 127 (JSC::CodeBlock::dumpValueProfiles): 128 (JSC::CodeBlock::rareCaseProfileCountForBytecodeOffset): 129 (JSC::CodeBlock::arithProfileForBytecodeOffset): 130 (JSC::CodeBlock::arithProfileForPC): 131 (JSC::CodeBlock::couldTakeSpecialFastCase): 132 (JSC::CodeBlock::dumpResultProfile): Deleted. 133 (JSC::CodeBlock::resultProfileForBytecodeOffset): Deleted. 134 (JSC::CodeBlock::specialFastCaseProfileCountForBytecodeOffset): Deleted. 135 (JSC::CodeBlock::ensureResultProfile): Deleted. 136 * bytecode/CodeBlock.h: 137 (JSC::CodeBlock::stubInfoBegin): 138 (JSC::CodeBlock::stubInfoEnd): 139 (JSC::CodeBlock::couldTakeSlowCase): 140 (JSC::CodeBlock::numberOfResultProfiles): Deleted. 141 * bytecode/MethodOfGettingAValueProfile.cpp: 142 (JSC::MethodOfGettingAValueProfile::emitReportValue): 143 * bytecode/MethodOfGettingAValueProfile.h: 144 (JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile): 145 * bytecode/ValueProfile.cpp: 146 (JSC::ResultProfile::emitDetectNumericness): Deleted. 147 (JSC::ResultProfile::emitSetDouble): Deleted. 148 (JSC::ResultProfile::emitSetNonNumber): Deleted. 149 (WTF::printInternal): Deleted. 150 * bytecode/ValueProfile.h: 151 (JSC::getRareCaseProfileBytecodeOffset): 152 (JSC::ResultProfile::ResultProfile): Deleted. 153 (JSC::ResultProfile::bytecodeOffset): Deleted. 154 (JSC::ResultProfile::specialFastPathCount): Deleted. 155 (JSC::ResultProfile::didObserveNonInt32): Deleted. 156 (JSC::ResultProfile::didObserveDouble): Deleted. 157 (JSC::ResultProfile::didObserveNonNegZeroDouble): Deleted. 158 (JSC::ResultProfile::didObserveNegZeroDouble): Deleted. 159 (JSC::ResultProfile::didObserveNonNumber): Deleted. 160 (JSC::ResultProfile::didObserveInt32Overflow): Deleted. 161 (JSC::ResultProfile::didObserveInt52Overflow): Deleted. 162 (JSC::ResultProfile::setObservedNonNegZeroDouble): Deleted. 163 (JSC::ResultProfile::setObservedNegZeroDouble): Deleted. 164 (JSC::ResultProfile::setObservedNonNumber): Deleted. 165 (JSC::ResultProfile::setObservedInt32Overflow): Deleted. 166 (JSC::ResultProfile::setObservedInt52Overflow): Deleted. 167 (JSC::ResultProfile::addressOfFlags): Deleted. 168 (JSC::ResultProfile::addressOfSpecialFastPathCount): Deleted. 169 (JSC::ResultProfile::detectNumericness): Deleted. 170 (JSC::ResultProfile::hasBits): Deleted. 171 (JSC::ResultProfile::setBit): Deleted. 172 (JSC::getResultProfileBytecodeOffset): Deleted. 173 * bytecompiler/BytecodeGenerator.cpp: 174 (JSC::BytecodeGenerator::emitBinaryOp): 175 * dfg/DFGByteCodeParser.cpp: 176 (JSC::DFG::ByteCodeParser::makeSafe): 177 * dfg/DFGGraph.cpp: 178 (JSC::DFG::Graph::methodOfGettingAValueProfileFor): 179 * dfg/DFGJITCompiler.cpp: 180 (JSC::DFG::JITCompiler::exceptionCheck): 181 * dfg/DFGSlowPathGenerator.h: 182 (JSC::DFG::SlowPathGenerator::generate): 183 * dfg/DFGSpeculativeJIT.cpp: 184 (JSC::DFG::SpeculativeJIT::addSlowPathGenerator): 185 (JSC::DFG::SpeculativeJIT::runSlowPathGenerators): 186 (JSC::DFG::SpeculativeJIT::compileValueAdd): 187 * dfg/DFGSpeculativeJIT.h: 188 (JSC::DFG::SpeculativeJIT::silentSpillAllRegistersImpl): 189 (JSC::DFG::SpeculativeJIT::silentSpillAllRegisters): 190 (JSC::DFG::SpeculativeJIT::callOperation): 191 * ftl/FTLLowerDFGToB3.cpp: 192 (JSC::FTL::DFG::LowerDFGToB3::compileValueAdd): 193 (JSC::FTL::DFG::LowerDFGToB3::compileStrCat): 194 * jit/CCallHelpers.h: 195 (JSC::CCallHelpers::setupArgumentsWithExecState): 196 (JSC::CCallHelpers::setupArguments): 197 * jit/JIT.h: 198 * jit/JITAddGenerator.cpp: 199 (JSC::JITAddGenerator::generateInline): 200 (JSC::JITAddGenerator::generateFastPath): 201 * jit/JITAddGenerator.h: 202 (JSC::JITAddGenerator::JITAddGenerator): 203 (JSC::JITAddGenerator::didEmitFastPath): Deleted. 204 (JSC::JITAddGenerator::endJumpList): Deleted. 205 (JSC::JITAddGenerator::slowPathJumpList): Deleted. 206 * jit/JITArithmetic.cpp: 207 (JSC::JIT::emit_op_jless): 208 (JSC::JIT::emitSlow_op_urshift): 209 (JSC::getOperandTypes): 210 (JSC::JIT::emit_op_add): 211 (JSC::JIT::emitSlow_op_add): 212 (JSC::JIT::emit_op_div): 213 (JSC::JIT::emit_op_mul): 214 (JSC::JIT::emitSlow_op_mul): 215 (JSC::JIT::emit_op_sub): 216 (JSC::JIT::emitSlow_op_sub): 217 * jit/JITDivGenerator.cpp: 218 (JSC::JITDivGenerator::generateFastPath): 219 * jit/JITDivGenerator.h: 220 (JSC::JITDivGenerator::JITDivGenerator): 221 * jit/JITInlines.h: 222 (JSC::JIT::callOperation): 223 * jit/JITMathIC.h: Added. 224 (JSC::JITMathIC::doneLocation): 225 (JSC::JITMathIC::slowPathStartLocation): 226 (JSC::JITMathIC::slowPathCallLocation): 227 (JSC::JITMathIC::generateInline): 228 (JSC::JITMathIC::generateOutOfLine): 229 (JSC::JITMathIC::finalizeInlineCode): 230 * jit/JITMathICForwards.h: Added. 231 * jit/JITMathICInlineResult.h: Added. 232 * jit/JITMulGenerator.cpp: 233 (JSC::JITMulGenerator::generateFastPath): 234 * jit/JITMulGenerator.h: 235 (JSC::JITMulGenerator::JITMulGenerator): 236 * jit/JITOperations.cpp: 237 * jit/JITOperations.h: 238 * jit/JITSubGenerator.cpp: 239 (JSC::JITSubGenerator::generateFastPath): 240 * jit/JITSubGenerator.h: 241 (JSC::JITSubGenerator::JITSubGenerator): 242 * jit/Repatch.cpp: 243 (JSC::readCallTarget): 244 (JSC::ftlThunkAwareRepatchCall): 245 (JSC::tryCacheGetByID): 246 (JSC::repatchGetByID): 247 (JSC::appropriateGenericPutByIdFunction): 248 (JSC::tryCachePutByID): 249 (JSC::repatchPutByID): 250 (JSC::tryRepatchIn): 251 (JSC::repatchIn): 252 (JSC::linkSlowFor): 253 (JSC::resetGetByID): 254 (JSC::resetPutByID): 255 (JSC::repatchCall): Deleted. 256 * jit/Repatch.h: 257 * llint/LLIntData.cpp: 258 (JSC::LLInt::Data::performAssertions): 259 * llint/LowLevelInterpreter.asm: 260 * llint/LowLevelInterpreter32_64.asm: 261 * llint/LowLevelInterpreter64.asm: 262 * parser/ResultType.h: 263 (JSC::ResultType::ResultType): 264 (JSC::ResultType::isInt32): 265 (JSC::ResultType::definitelyIsNumber): 266 (JSC::ResultType::definitelyIsString): 267 (JSC::ResultType::definitelyIsBoolean): 268 (JSC::ResultType::mightBeNumber): 269 (JSC::ResultType::isNotNumber): 270 (JSC::ResultType::forBitOp): 271 (JSC::ResultType::bits): 272 (JSC::OperandTypes::OperandTypes): 273 * runtime/CommonSlowPaths.cpp: 274 (JSC::SLOW_PATH_DECL): 275 (JSC::updateArithProfileForBinaryArithOp): 276 (JSC::updateResultProfileForBinaryArithOp): Deleted. 277 * tests/stress/op-add-exceptions.js: Added. 278 (assert): 279 (f1): 280 (f2): 281 (f3): 282 (let.oException.valueOf): 283 (foo): 284 (ident): 285 (bar): 286 1 287 2016-07-21 Csaba Osztrogonác <ossy@webkit.org> 2 288 -
trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
r203491 r203537 1280 1280 79160DBD1C8E3EC8008C085A /* ProxyRevoke.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */; }; 1281 1281 79160DBE1C8E3EC8008C085A /* ProxyRevoke.h in Headers */ = {isa = PBXBuildFile; fileRef = 79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1282 79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */ = {isa = PBXBuildFile; fileRef = 79233C291D34715700C5A834 /* JITMathIC.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1282 1283 792CB3491C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */; }; 1283 1284 792CB34A1C4EED5C00D13AF3 /* PCToCodeOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; }; … … 1286 1287 797E07AA1B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h in Headers */ = {isa = PBXBuildFile; fileRef = 797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1287 1288 799EF7C41C56ED96002B0534 /* B3PCToOriginMap.h in Headers */ = {isa = PBXBuildFile; fileRef = 799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1289 79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79A228331D35D71E00D8E067 /* ArithProfile.cpp */; }; 1290 79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A228341D35D71E00D8E067 /* ArithProfile.h */; }; 1291 79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */ = {isa = PBXBuildFile; fileRef = 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1288 1292 79B00CBC1C6AB07E0088C65D /* ProxyConstructor.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */; }; 1289 1293 79B00CBD1C6AB07E0088C65D /* ProxyConstructor.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1290 1294 79B00CBE1C6AB07E0088C65D /* ProxyObject.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79B00CBA1C6AB07E0088C65D /* ProxyObject.cpp */; settings = {COMPILER_FLAGS = "-fno-optimize-sibling-calls"; }; }; 1291 1295 79B00CBF1C6AB07E0088C65D /* ProxyObject.h in Headers */ = {isa = PBXBuildFile; fileRef = 79B00CBB1C6AB07E0088C65D /* ProxyObject.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1296 79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */ = {isa = PBXBuildFile; fileRef = 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */; settings = {ATTRIBUTES = (Private, ); }; }; 1292 1297 79C4B15D1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 79C4B15B1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.cpp */; }; 1293 1298 79C4B15E1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 79C4B15C1BA2158F00FD592E /* DFGLiveCatchVariablePreservationPhase.h */; settings = {ATTRIBUTES = (Private, ); }; }; … … 3480 3485 79160DBB1C8E3EC8008C085A /* ProxyRevoke.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyRevoke.cpp; sourceTree = "<group>"; }; 3481 3486 79160DBC1C8E3EC8008C085A /* ProxyRevoke.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyRevoke.h; sourceTree = "<group>"; }; 3487 79233C291D34715700C5A834 /* JITMathIC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathIC.h; sourceTree = "<group>"; }; 3482 3488 792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PCToCodeOriginMap.cpp; sourceTree = "<group>"; }; 3483 3489 792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PCToCodeOriginMap.h; sourceTree = "<group>"; }; … … 3486 3492 797E07A81B8FCFB9008400BA /* JSGlobalLexicalEnvironment.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSGlobalLexicalEnvironment.h; sourceTree = "<group>"; }; 3487 3493 799EF7C31C56ED96002B0534 /* B3PCToOriginMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = B3PCToOriginMap.h; path = b3/B3PCToOriginMap.h; sourceTree = "<group>"; }; 3494 79A228331D35D71E00D8E067 /* ArithProfile.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ArithProfile.cpp; sourceTree = "<group>"; }; 3495 79A228341D35D71E00D8E067 /* ArithProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ArithProfile.h; sourceTree = "<group>"; }; 3496 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICForwards.h; sourceTree = "<group>"; }; 3497 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITMathICInlineResult.h; sourceTree = "<group>"; }; 3488 3498 79B00CB81C6AB07E0088C65D /* ProxyConstructor.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ProxyConstructor.cpp; sourceTree = "<group>"; }; 3489 3499 79B00CB91C6AB07E0088C65D /* ProxyConstructor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProxyConstructor.h; sourceTree = "<group>"; }; … … 5071 5081 1429D92D0ED22D7000B89619 /* JIT.cpp */, 5072 5082 1429D92E0ED22D7000B89619 /* JIT.h */, 5083 79233C291D34715700C5A834 /* JITMathIC.h */, 5084 79A899FE1D38612E00D18C73 /* JITMathICForwards.h */, 5085 79AF0BE31D3EFD4C00E95FA5 /* JITMathICInlineResult.h */, 5073 5086 FE1220251BE7F5640039E6F2 /* JITAddGenerator.cpp */, 5074 5087 FE1220261BE7F5640039E6F2 /* JITAddGenerator.h */, … … 6572 6585 0F63945115D07051006A597C /* ArrayProfile.cpp */, 6573 6586 0F63945215D07051006A597C /* ArrayProfile.h */, 6587 79A228331D35D71E00D8E067 /* ArithProfile.cpp */, 6588 79A228341D35D71E00D8E067 /* ArithProfile.h */, 6574 6589 C2FCAE0C17A9C24E0034C735 /* BytecodeBasicBlock.cpp */, 6575 6590 C2FCAE0D17A9C24E0034C735 /* BytecodeBasicBlock.h */, … … 7084 7099 0FEC84FF1BDACDAC0080FF74 /* B3ArgumentRegValue.h in Headers */, 7085 7100 0FEC85011BDACDAC0080FF74 /* B3BasicBlock.h in Headers */, 7101 79233C2B1D34715700C5A834 /* JITMathIC.h in Headers */, 7086 7102 0FEC85021BDACDAC0080FF74 /* B3BasicBlockInlines.h in Headers */, 7087 7103 0FEC85031BDACDAC0080FF74 /* B3BasicBlockUtils.h in Headers */, … … 7664 7680 BC11667B0E199C05008066DD /* InternalFunction.h in Headers */, 7665 7681 1429D77C0ED20D7300B89619 /* Interpreter.h in Headers */, 7682 79AF0BE41D3EFD4C00E95FA5 /* JITMathICInlineResult.h in Headers */, 7666 7683 A1B9E23A1B4E0D6700BC7FED /* IntlCollator.h in Headers */, 7667 7684 A1B9E23C1B4E0D6700BC7FED /* IntlCollatorConstructor.h in Headers */, … … 7773 7790 A50E4B6418809DD50068A46D /* JSGlobalObjectRuntimeAgent.h in Headers */, 7774 7791 A503FA2A188F105900110F14 /* JSGlobalObjectScriptDebugServer.h in Headers */, 7792 79A228361D35D71F00D8E067 /* ArithProfile.h in Headers */, 7775 7793 A513E5C0185BFACC007E95AD /* JSInjectedScriptHost.h in Headers */, 7776 7794 A513E5C2185BFACC007E95AD /* JSInjectedScriptHostPrototype.h in Headers */, … … 8198 8216 9688CB160ED12B4E001D649F /* X86Assembler.h in Headers */, 8199 8217 9959E92E1BD17FA4001AA413 /* xxd.pl in Headers */, 8218 79B1788E1D399B8000B1A567 /* JITMathICForwards.h in Headers */, 8200 8219 451539B912DC994500EF7AC4 /* Yarr.h in Headers */, 8201 8220 86704B8512DBA33700A9FE7B /* YarrInterpreter.h in Headers */, … … 9283 9302 86C568E011A213EE0007F7F0 /* MacroAssemblerARM.cpp in Sources */, 9284 9303 FEB137571BB11EF900CD5100 /* MacroAssemblerARM64.cpp in Sources */, 9304 79A228351D35D71E00D8E067 /* ArithProfile.cpp in Sources */, 9285 9305 A729009C17976C6000317298 /* MacroAssemblerARMv7.cpp in Sources */, 9286 9306 FE68C6381B90DE0B0042BCB3 /* MacroAssemblerPrinter.cpp in Sources */, -
trunk/Source/JavaScriptCore/bytecode/CodeBlock.cpp
r203499 r203537 31 31 #include "CodeBlock.h" 32 32 33 #include "ArithProfile.h" 33 34 #include "BasicBlockLocation.h" 34 35 #include "BytecodeGenerator.h" … … 46 47 #include "Interpreter.h" 47 48 #include "JIT.h" 49 #include "JITMathIC.h" 48 50 #include "JSCJSValue.h" 49 51 #include "JSFunction.h" … … 770 772 } 771 773 772 void CodeBlock::dump ResultProfile(PrintStream& out, ResultProfile* profile, bool& hasPrintedProfiling)774 void CodeBlock::dumpArithProfile(PrintStream& out, ArithProfile* profile, bool& hasPrintedProfiling) 773 775 { 774 776 if (!profile) … … 1774 1776 dumpRareCaseProfile(out, "rare case: ", rareCaseProfileForBytecodeOffset(location), hasPrintedProfiling); 1775 1777 { 1776 ConcurrentJITLocker locker(m_lock); 1777 dumpResultProfile(out, resultProfileForBytecodeOffset(locker, location), hasPrintedProfiling); 1778 dumpArithProfile(out, arithProfileForBytecodeOffset(location), hasPrintedProfiling); 1778 1779 } 1779 1780 … … 3008 3009 } 3009 3010 3011 JITAddIC* CodeBlock::addJITAddIC() 3012 { 3013 return m_addICs.add(); 3014 } 3015 3010 3016 StructureStubInfo* CodeBlock::findStubInfo(CodeOrigin codeOrigin) 3011 3017 { … … 3054 3060 // there is JIT code. 3055 3061 m_rareCaseProfiles.clear(); 3056 3057 // We can clear these because the DFG only accesses members of this data structure when3058 // holding the lock or after querying whether we have JIT code.3059 m_resultProfiles.clear();3060 m_bytecodeOffsetToResultProfileIndexMap = nullptr;3061 3062 } 3062 3063 #endif … … 3288 3289 3289 3290 m_rareCaseProfiles.shrinkToFit(); 3290 m_resultProfiles.shrinkToFit();3291 3291 3292 3292 if (shrinkMode == EarlyShrink) { … … 4164 4164 dataLogF(" bc = %d: %u\n", profile->m_bytecodeOffset, profile->m_counter); 4165 4165 } 4166 dataLog("ResultProfile for ", *this, ":\n");4167 for (unsigned i = 0; i < numberOfResultProfiles(); ++i) {4168 const ResultProfile& profile = *resultProfile(i);4169 dataLog(" bc = ", profile.bytecodeOffset(), ": ", profile, "\n");4170 }4171 4166 } 4172 4167 #endif // ENABLE(VERBOSE_VALUE_PROFILE) … … 4371 4366 } 4372 4367 4373 ResultProfile* CodeBlock::resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset) 4374 { 4375 if (!m_bytecodeOffsetToResultProfileIndexMap) 4368 ArithProfile* CodeBlock::arithProfileForBytecodeOffset(int bytecodeOffset) 4369 { 4370 auto opcodeID = vm()->interpreter->getOpcodeID(instructions()[bytecodeOffset].u.opcode); 4371 switch (opcodeID) { 4372 case op_bitor: 4373 case op_bitand: 4374 case op_bitxor: 4375 case op_add: 4376 case op_mul: 4377 case op_sub: 4378 case op_div: 4379 break; 4380 default: 4376 4381 return nullptr; 4377 auto iterator = m_bytecodeOffsetToResultProfileIndexMap->find(bytecodeOffset); 4378 if (iterator == m_bytecodeOffsetToResultProfileIndexMap->end()) 4379 return nullptr; 4380 return &m_resultProfiles[iterator->value]; 4381 } 4382 4383 unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset) 4384 { 4385 ConcurrentJITLocker locker(m_lock); 4386 return specialFastCaseProfileCountForBytecodeOffset(locker, bytecodeOffset); 4387 } 4388 4389 unsigned CodeBlock::specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker& locker, int bytecodeOffset) 4390 { 4391 ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset); 4392 if (!profile) 4393 return 0; 4394 return profile->specialFastPathCount(); 4382 } 4383 4384 return &arithProfileForPC(instructions().begin() + bytecodeOffset); 4385 } 4386 4387 ArithProfile& CodeBlock::arithProfileForPC(Instruction* pc) 4388 { 4389 if (!ASSERT_DISABLED) { 4390 ASSERT(pc >= instructions().begin() && pc < instructions().end()); 4391 auto opcodeID = vm()->interpreter->getOpcodeID(pc[0].u.opcode); 4392 switch (opcodeID) { 4393 case op_bitor: 4394 case op_bitand: 4395 case op_bitxor: 4396 case op_add: 4397 case op_mul: 4398 case op_sub: 4399 case op_div: 4400 break; 4401 default: 4402 ASSERT_NOT_REACHED(); 4403 } 4404 } 4405 4406 return *bitwise_cast<ArithProfile*>(&pc[4].u.operand); 4395 4407 } 4396 4408 … … 4399 4411 if (!hasBaselineJITProfiling()) 4400 4412 return false; 4401 unsigned specialFastCaseCount = specialFastCaseProfileCountForBytecodeOffset(bytecodeOffset); 4402 return specialFastCaseCount >= Options::couldTakeSlowCaseMinimumCount(); 4403 } 4404 4405 ResultProfile* CodeBlock::ensureResultProfile(int bytecodeOffset) 4406 { 4407 ConcurrentJITLocker locker(m_lock); 4408 return ensureResultProfile(locker, bytecodeOffset); 4409 } 4410 4411 ResultProfile* CodeBlock::ensureResultProfile(const ConcurrentJITLocker& locker, int bytecodeOffset) 4412 { 4413 ResultProfile* profile = resultProfileForBytecodeOffset(locker, bytecodeOffset); 4414 if (!profile) { 4415 m_resultProfiles.append(ResultProfile(bytecodeOffset)); 4416 profile = &m_resultProfiles.last(); 4417 ASSERT(&m_resultProfiles.last() == &m_resultProfiles[m_resultProfiles.size() - 1]); 4418 if (!m_bytecodeOffsetToResultProfileIndexMap) 4419 m_bytecodeOffsetToResultProfileIndexMap = std::make_unique<BytecodeOffsetToResultProfileIndexMap>(); 4420 m_bytecodeOffsetToResultProfileIndexMap->add(bytecodeOffset, m_resultProfiles.size() - 1); 4421 } 4422 return profile; 4413 ArithProfile* profile = arithProfileForBytecodeOffset(bytecodeOffset); 4414 if (!profile) 4415 return false; 4416 return profile->tookSpecialFastPath(); 4423 4417 } 4424 4418 -
trunk/Source/JavaScriptCore/bytecode/CodeBlock.h
r202778 r203537 52 52 #include "Instruction.h" 53 53 #include "JITCode.h" 54 #include "JITMathICForwards.h" 54 55 #include "JITWriteBarrier.h" 55 56 #include "JSCell.h" … … 80 81 81 82 class ExecState; 83 class JITAddGenerator; 82 84 class JSModuleEnvironment; 83 85 class LLIntOffsetsExtractor; … … 88 90 89 91 enum class AccessType : int8_t; 92 93 struct ArithProfile; 90 94 91 95 typedef HashMap<CodeOrigin, StructureStubInfo*, CodeOriginApproximateHash> StubInfoMap; … … 246 250 #if ENABLE(JIT) 247 251 StructureStubInfo* addStubInfo(AccessType); 252 JITAddIC* addJITAddIC(); 248 253 Bag<StructureStubInfo>::iterator stubInfoBegin() { return m_stubInfos.begin(); } 249 254 Bag<StructureStubInfo>::iterator stubInfoEnd() { return m_stubInfos.end(); } … … 441 446 } 442 447 443 ResultProfile* ensureResultProfile(int bytecodeOffset); 444 ResultProfile* ensureResultProfile(const ConcurrentJITLocker&, int bytecodeOffset); 445 unsigned numberOfResultProfiles() { return m_resultProfiles.size(); } 446 ResultProfile* resultProfileForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset); 447 448 unsigned specialFastCaseProfileCountForBytecodeOffset(const ConcurrentJITLocker&, int bytecodeOffset); 449 unsigned specialFastCaseProfileCountForBytecodeOffset(int bytecodeOffset); 448 ArithProfile* arithProfileForBytecodeOffset(int bytecodeOffset); 449 ArithProfile& arithProfileForPC(Instruction*); 450 450 451 451 bool couldTakeSpecialFastCase(int bytecodeOffset); … … 958 958 void dumpArrayProfiling(PrintStream&, const Instruction*&, bool& hasPrintedProfiling); 959 959 void dumpRareCaseProfile(PrintStream&, const char* name, RareCaseProfile*, bool& hasPrintedProfiling); 960 void dump ResultProfile(PrintStream&, ResultProfile*, bool& hasPrintedProfiling);960 void dumpArithProfile(PrintStream&, ArithProfile*, bool& hasPrintedProfiling); 961 961 962 962 bool shouldVisitStrongly(); … … 1014 1014 std::unique_ptr<RegisterAtOffsetList> m_calleeSaveRegisters; 1015 1015 Bag<StructureStubInfo> m_stubInfos; 1016 Bag<JITAddIC> m_addICs; 1016 1017 Bag<ByValInfo> m_byValInfos; 1017 1018 Bag<CallLinkInfo> m_callLinkInfos; … … 1030 1031 RefCountedArray<ValueProfile> m_valueProfiles; 1031 1032 SegmentedVector<RareCaseProfile, 8> m_rareCaseProfiles; 1032 SegmentedVector<ResultProfile, 8> m_resultProfiles;1033 typedef HashMap<unsigned, unsigned, IntHash<unsigned>, WTF::UnsignedWithZeroKeyHashTraits<unsigned>> BytecodeOffsetToResultProfileIndexMap;1034 std::unique_ptr<BytecodeOffsetToResultProfileIndexMap> m_bytecodeOffsetToResultProfileIndexMap;1035 1033 RefCountedArray<ArrayAllocationProfile> m_arrayAllocationProfiles; 1036 1034 ArrayProfileVector m_arrayProfiles; -
trunk/Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.cpp
r200606 r203537 29 29 #if ENABLE(DFG_JIT) 30 30 31 #include "ArithProfile.h" 31 32 #include "CCallHelpers.h" 32 33 #include "CodeBlock.h" … … 66 67 } 67 68 68 case ResultProfileReady: {69 u. resultProfile->emitDetectNumericness(jit, regs, DoNotHaveTagRegisters);69 case ArithProfileReady: { 70 u.arithProfile->emitObserveResult(jit, regs, DoNotHaveTagRegisters); 70 71 return; 71 72 } } -
trunk/Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.h
r200606 r203537 41 41 class CodeBlock; 42 42 class LazyOperandValueProfileKey; 43 struct ResultProfile;43 struct ArithProfile; 44 44 struct ValueProfile; 45 45 … … 60 60 } 61 61 62 MethodOfGettingAValueProfile( ResultProfile* profile)62 MethodOfGettingAValueProfile(ArithProfile* profile) 63 63 { 64 64 if (profile) { 65 m_kind = ResultProfileReady;66 u. resultProfile = profile;65 m_kind = ArithProfileReady; 66 u.arithProfile = profile; 67 67 } else 68 68 m_kind = None; … … 80 80 None, 81 81 Ready, 82 ResultProfileReady,82 ArithProfileReady, 83 83 LazyOperand 84 84 }; … … 87 87 union { 88 88 ValueProfile* profile; 89 ResultProfile* resultProfile;89 ArithProfile* arithProfile; 90 90 struct { 91 91 CodeBlock* codeBlock; -
trunk/Source/JavaScriptCore/bytecode/ValueProfile.cpp
r200613 r203537 32 32 namespace JSC { 33 33 34 #if ENABLE(JIT)35 void ResultProfile::emitDetectNumericness(CCallHelpers& jit, JSValueRegs regs, TagRegistersMode mode)36 {37 CCallHelpers::Jump isInt32 = jit.branchIfInt32(regs, mode);38 CCallHelpers::Jump notDouble = jit.branchIfNotDoubleKnownNotInt32(regs, mode);39 // FIXME: We could be more precise here.40 emitSetDouble(jit);41 CCallHelpers::Jump done = jit.jump();42 notDouble.link(&jit);43 emitSetNonNumber(jit);44 done.link(&jit);45 isInt32.link(&jit);46 }47 48 void ResultProfile::emitSetDouble(CCallHelpers& jit)49 {50 jit.or32(CCallHelpers::TrustedImm32(ResultProfile::Int32Overflow | ResultProfile::Int52Overflow | ResultProfile::NegZeroDouble | ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(addressOfFlags()));51 }52 53 void ResultProfile::emitSetNonNumber(CCallHelpers& jit)54 {55 jit.or32(CCallHelpers::TrustedImm32(ResultProfile::NonNumber), CCallHelpers::AbsoluteAddress(addressOfFlags()));56 }57 #endif // ENABLE(JIT)58 59 34 } // namespace JSC 60 61 namespace WTF {62 63 using namespace JSC;64 65 void printInternal(PrintStream& out, const ResultProfile& profile)66 {67 const char* separator = "";68 69 if (!profile.didObserveNonInt32()) {70 out.print("Int32");71 separator = "|";72 } else {73 if (profile.didObserveNegZeroDouble()) {74 out.print(separator, "NegZeroDouble");75 separator = "|";76 }77 if (profile.didObserveNonNegZeroDouble()) {78 out.print("NonNegZeroDouble");79 separator = "|";80 }81 if (profile.didObserveNonNumber()) {82 out.print("NonNumber");83 separator = "|";84 }85 if (profile.didObserveInt32Overflow()) {86 out.print("Int32Overflow");87 separator = "|";88 }89 if (profile.didObserveInt52Overflow()) {90 out.print("Int52Overflow");91 separator = "|";92 }93 }94 if (profile.specialFastPathCount()) {95 out.print(" special fast path: ");96 out.print(profile.specialFastPathCount());97 }98 }99 100 } // namespace WTF -
trunk/Source/JavaScriptCore/bytecode/ValueProfile.h
r200613 r203537 210 210 } 211 211 212 struct ResultProfile {213 private:214 static const int numberOfFlagBits = 5;215 216 public:217 ResultProfile(int bytecodeOffset)218 : m_bytecodeOffsetAndFlags(bytecodeOffset << numberOfFlagBits)219 {220 ASSERT(((bytecodeOffset << numberOfFlagBits) >> numberOfFlagBits) == bytecodeOffset);221 }222 223 enum ObservedResults {224 NonNegZeroDouble = 1 << 0,225 NegZeroDouble = 1 << 1,226 NonNumber = 1 << 2,227 Int32Overflow = 1 << 3,228 Int52Overflow = 1 << 4,229 };230 231 int bytecodeOffset() const { return m_bytecodeOffsetAndFlags >> numberOfFlagBits; }232 unsigned specialFastPathCount() const { return m_specialFastPathCount; }233 234 bool didObserveNonInt32() const { return hasBits(NonNegZeroDouble | NegZeroDouble | NonNumber); }235 bool didObserveDouble() const { return hasBits(NonNegZeroDouble | NegZeroDouble); }236 bool didObserveNonNegZeroDouble() const { return hasBits(NonNegZeroDouble); }237 bool didObserveNegZeroDouble() const { return hasBits(NegZeroDouble); }238 bool didObserveNonNumber() const { return hasBits(NonNumber); }239 bool didObserveInt32Overflow() const { return hasBits(Int32Overflow); }240 bool didObserveInt52Overflow() const { return hasBits(Int52Overflow); }241 242 void setObservedNonNegZeroDouble() { setBit(NonNegZeroDouble); }243 void setObservedNegZeroDouble() { setBit(NegZeroDouble); }244 void setObservedNonNumber() { setBit(NonNumber); }245 void setObservedInt32Overflow() { setBit(Int32Overflow); }246 void setObservedInt52Overflow() { setBit(Int52Overflow); }247 248 void* addressOfFlags() { return &m_bytecodeOffsetAndFlags; }249 void* addressOfSpecialFastPathCount() { return &m_specialFastPathCount; }250 251 void detectNumericness(JSValue value)252 {253 if (value.isInt32())254 return;255 if (value.isNumber()) {256 m_bytecodeOffsetAndFlags |= Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble;257 return;258 }259 m_bytecodeOffsetAndFlags |= NonNumber;260 }261 262 #if ENABLE(JIT)263 // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble) if it sees a264 // double. Sets NonNumber if it sees a non-number.265 void emitDetectNumericness(CCallHelpers&, JSValueRegs, TagRegistersMode = HaveTagRegisters);266 267 // Sets (Int32Overflow | Int52Overflow | NonNegZeroDouble | NegZeroDouble).268 void emitSetDouble(CCallHelpers&);269 270 // Sets NonNumber.271 void emitSetNonNumber(CCallHelpers&);272 #endif // ENABLE(JIT)273 274 private:275 bool hasBits(int mask) const { return m_bytecodeOffsetAndFlags & mask; }276 void setBit(int mask) { m_bytecodeOffsetAndFlags |= mask; }277 278 int m_bytecodeOffsetAndFlags;279 unsigned m_specialFastPathCount { 0 };280 };281 282 inline int getResultProfileBytecodeOffset(ResultProfile* profile)283 {284 return profile->bytecodeOffset();285 }286 287 212 } // namespace JSC 288 213 289 namespace WTF {290 291 void printInternal(PrintStream&, const JSC::ResultProfile&);292 293 } // namespace WTF294 295 214 #endif // ValueProfile_h 296 -
trunk/Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp
r203361 r203537 32 32 #include "BytecodeGenerator.h" 33 33 34 #include "ArithProfile.h" 34 35 #include "BuiltinExecutables.h" 35 36 #include "BytecodeLivenessAnalysis.h" … … 1601 1602 if (opcodeID == op_bitor || opcodeID == op_bitand || opcodeID == op_bitxor || 1602 1603 opcodeID == op_add || opcodeID == op_mul || opcodeID == op_sub || opcodeID == op_div) 1603 instructions().append( types.toInt());1604 instructions().append(ArithProfile(types.first(), types.second()).bits()); 1604 1605 1605 1606 return dst; -
trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
r203499 r203537 29 29 #if ENABLE(DFG_JIT) 30 30 31 #include "ArithProfile.h" 31 32 #include "ArrayConstructor.h" 32 33 #include "BasicBlockLocation.h" … … 915 916 916 917 { 917 ConcurrentJITLocker locker(m_inlineStackTop->m_profiledBlock->m_lock); 918 ResultProfile* resultProfile = m_inlineStackTop->m_profiledBlock->resultProfileForBytecodeOffset(locker, m_currentIndex); 919 if (resultProfile) { 918 ArithProfile* arithProfile = m_inlineStackTop->m_profiledBlock->arithProfileForBytecodeOffset(m_currentIndex); 919 if (arithProfile) { 920 920 switch (node->op()) { 921 921 case ArithAdd: 922 922 case ArithSub: 923 923 case ValueAdd: 924 if ( resultProfile->didObserveDouble())924 if (arithProfile->didObserveDouble()) 925 925 node->mergeFlags(NodeMayHaveDoubleResult); 926 if ( resultProfile->didObserveNonNumber())926 if (arithProfile->didObserveNonNumber()) 927 927 node->mergeFlags(NodeMayHaveNonNumberResult); 928 928 break; 929 929 930 930 case ArithMul: { 931 if ( resultProfile->didObserveInt52Overflow())931 if (arithProfile->didObserveInt52Overflow()) 932 932 node->mergeFlags(NodeMayOverflowInt52); 933 if ( resultProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow))933 if (arithProfile->didObserveInt32Overflow() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, Overflow)) 934 934 node->mergeFlags(NodeMayOverflowInt32InBaseline); 935 if ( resultProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero))935 if (arithProfile->didObserveNegZeroDouble() || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, NegativeZero)) 936 936 node->mergeFlags(NodeMayNegZeroInBaseline); 937 if ( resultProfile->didObserveDouble())937 if (arithProfile->didObserveDouble()) 938 938 node->mergeFlags(NodeMayHaveDoubleResult); 939 if ( resultProfile->didObserveNonNumber())939 if (arithProfile->didObserveNonNumber()) 940 940 node->mergeFlags(NodeMayHaveNonNumberResult); 941 941 break; -
trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp
r203006 r203537 1549 1549 1550 1550 { 1551 ConcurrentJITLocker locker(profiledBlock->m_lock);1552 1551 if (profiledBlock->hasBaselineJITProfiling()) { 1553 if ( ResultProfile* result = profiledBlock->resultProfileForBytecodeOffset(locker,node->origin.semantic.bytecodeIndex))1552 if (ArithProfile* result = profiledBlock->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex)) 1554 1553 return result; 1555 1554 } -
trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp
r203130 r203537 605 605 bool willCatchException = m_graph.willCatchExceptionInMachineFrame(m_speculative->m_currentNode->origin.forExit, opCatchOrigin, exceptionHandler); 606 606 if (willCatchException) { 607 unsigned streamIndex = m_speculative->m_outOfLineStreamIndex != UINT_MAX ?m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size();607 unsigned streamIndex = m_speculative->m_outOfLineStreamIndex ? *m_speculative->m_outOfLineStreamIndex : m_speculative->m_stream->size(); 608 608 MacroAssembler::Jump hadException = emitNonPatchableExceptionCheck(); 609 609 // We assume here that this is called after callOpeartion()/appendCall() is called. -
trunk/Source/JavaScriptCore/dfg/DFGSlowPathGenerator.h
r197563 r203537 53 53 jit->m_origin = m_origin; 54 54 generateInternal(jit); 55 jit->m_outOfLineStreamIndex = UINT_MAX;55 jit->m_outOfLineStreamIndex = Nullopt; 56 56 if (!ASSERT_DISABLED) 57 57 jit->m_jit.abortWithReason(DFGSlowPathGeneratorFellThrough); -
trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
r203491 r203537 45 45 #include "JITDivGenerator.h" 46 46 #include "JITLeftShiftGenerator.h" 47 #include "JITMathIC.h" 47 48 #include "JITMulGenerator.h" 48 49 #include "JITRightShiftGenerator.h" … … 57 58 #include "ScratchRegisterAllocator.h" 58 59 #include "WriteBarrierBuffer.h" 60 #include <wtf/Box.h> 59 61 #include <wtf/MathExtras.h> 60 62 … … 363 365 void SpeculativeJIT::addSlowPathGenerator(std::function<void()> lambda) 364 366 { 365 m_slowPathLambdas.append( std::make_pair(lambda, m_currentNode));367 m_slowPathLambdas.append(SlowPathLambda{ lambda, m_currentNode, static_cast<unsigned>(m_stream->size()) }); 366 368 } 367 369 … … 372 374 slowPathGenerator->generate(this); 373 375 } 374 for (auto& generatorPair: m_slowPathLambdas) {375 Node* currentNode = generatorPair.second;376 for (auto& slowPathLambda : m_slowPathLambdas) { 377 Node* currentNode = slowPathLambda.currentNode; 376 378 m_currentNode = currentNode; 379 m_outOfLineStreamIndex = slowPathLambda.streamIndex; 377 380 pcToCodeOriginMapBuilder.appendItem(m_jit.label(), currentNode->origin.semantic); 378 generatorPair.first(); 381 slowPathLambda.generator(); 382 m_outOfLineStreamIndex = Nullopt; 379 383 } 380 384 } … … 3380 3384 } 3381 3385 3382 JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, 3383 leftFPR, rightFPR, scratchGPR, scratchFPR); 3384 gen.generateFastPath(m_jit); 3385 3386 ASSERT(gen.didEmitFastPath()); 3387 gen.endJumpList().append(m_jit.jump()); 3388 3389 gen.slowPathJumpList().link(&m_jit); 3390 3391 silentSpillAllRegisters(resultRegs); 3392 3393 if (leftOperand.isConst()) { 3394 leftRegs = resultRegs; 3395 m_jit.moveValue(leftChild->asJSValue(), leftRegs); 3396 } else if (rightOperand.isConst()) { 3397 rightRegs = resultRegs; 3398 m_jit.moveValue(rightChild->asJSValue(), rightRegs); 3399 } 3400 3401 callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs); 3402 3403 silentFillAllRegisters(resultRegs); 3404 m_jit.exceptionCheck(); 3405 3406 gen.endJumpList().link(&m_jit); 3386 JITAddIC* addIC = m_jit.codeBlock()->addJITAddIC(); 3387 Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create(); 3388 ArithProfile* arithProfile = m_jit.graph().baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex); 3389 addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, leftFPR, rightFPR, scratchGPR, scratchFPR, arithProfile); 3390 3391 bool generatedInline = addIC->generateInline(m_jit, *addICGenerationState); 3392 3393 if (generatedInline) { 3394 ASSERT(!addICGenerationState->slowPathJumps.empty()); 3395 3396 Vector<SilentRegisterSavePlan> savePlans; 3397 silentSpillAllRegistersImpl(false, savePlans, resultRegs); 3398 3399 auto done = m_jit.label(); 3400 3401 addSlowPathGenerator([=, savePlans = WTFMove(savePlans)] () { 3402 addICGenerationState->slowPathJumps.link(&m_jit); 3403 addICGenerationState->slowPathStart = m_jit.label(); 3404 3405 silentSpill(savePlans); 3406 3407 auto innerLeftRegs = leftRegs; 3408 auto innerRightRegs = rightRegs; 3409 if (leftOperand.isConst()) { 3410 innerLeftRegs = resultRegs; 3411 m_jit.moveValue(leftChild->asJSValue(), innerLeftRegs); 3412 } else if (rightOperand.isConst()) { 3413 innerRightRegs = resultRegs; 3414 m_jit.moveValue(rightChild->asJSValue(), innerRightRegs); 3415 } 3416 3417 if (addICGenerationState->shouldSlowPathRepatch) 3418 addICGenerationState->slowPathCall = callOperation(operationValueAddOptimize, resultRegs, innerLeftRegs, innerRightRegs, addIC); 3419 else 3420 addICGenerationState->slowPathCall = callOperation(operationValueAdd, resultRegs, innerLeftRegs, innerRightRegs); 3421 3422 silentFill(savePlans); 3423 m_jit.exceptionCheck(); 3424 m_jit.jump().linkTo(done, &m_jit); 3425 3426 m_jit.addLinkTask([=] (LinkBuffer& linkBuffer) { 3427 addIC->finalizeInlineCode(*addICGenerationState, linkBuffer); 3428 }); 3429 }); 3430 } else { 3431 if (leftOperand.isConst()) { 3432 left = JSValueOperand(this, leftChild); 3433 leftRegs = left->jsValueRegs(); 3434 } else if (rightOperand.isConst()) { 3435 right = JSValueOperand(this, rightChild); 3436 rightRegs = right->jsValueRegs(); 3437 } 3438 3439 flushRegisters(); 3440 callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs); 3441 m_jit.exceptionCheck(); 3442 } 3443 3407 3444 jsValueResult(resultRegs, node); 3408 3445 return; -
trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
r203491 r203537 389 389 silentSpillAllRegistersImpl(doSpill, plans, InvalidGPRReg, InvalidGPRReg, exclude); 390 390 } 391 #if USE(JSVALUE32_64)392 391 template<typename CollectionType> 393 392 void silentSpillAllRegistersImpl(bool doSpill, CollectionType& plans, JSValueRegs exclude) 394 393 { 394 #if USE(JSVALUE32_64) 395 395 silentSpillAllRegistersImpl(doSpill, plans, exclude.tagGPR(), exclude.payloadGPR()); 396 } 397 #endif 396 #else 397 silentSpillAllRegistersImpl(doSpill, plans, exclude.gpr()); 398 #endif 399 } 398 400 399 401 void silentSpillAllRegisters(GPRReg exclude, GPRReg exclude2 = InvalidGPRReg, FPRReg fprExclude = InvalidFPRReg) … … 1286 1288 1287 1289 #if USE(JSVALUE64) 1290 1291 JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC) 1292 { 1293 m_jit.setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), TrustedImmPtr(addIC)); 1294 return appendCallSetResult(operation, result.gpr()); 1295 } 1296 1288 1297 JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg result, GPRReg arg1, GPRReg arg2, UniquedStringImpl* uid) 1289 1298 { … … 1702 1711 } 1703 1712 #else // USE(JSVALUE32_64) 1713 1714 JITCompiler::Call callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC) 1715 { 1716 m_jit.setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), TrustedImmPtr(addIC)); 1717 return appendCallSetResult(operation, result.payloadGPR(), result.tagGPR()); 1718 } 1704 1719 1705 1720 JITCompiler::Call callOperation(J_JITOperation_EJJI operation, GPRReg resultTag, GPRReg resultPayload, GPRReg arg1Tag, GPRReg arg1Payload, GPRReg arg2Tag, GPRReg arg2Payload, UniquedStringImpl* uid) … … 2833 2848 2834 2849 Vector<std::unique_ptr<SlowPathGenerator>, 8> m_slowPathGenerators; 2835 Vector<std::pair<std::function<void()>, Node*>, 8> m_slowPathLambdas; 2850 struct SlowPathLambda { 2851 std::function<void()> generator; 2852 Node* currentNode; 2853 unsigned streamIndex; 2854 }; 2855 Vector<SlowPathLambda> m_slowPathLambdas; 2836 2856 Vector<SilentRegisterSavePlan> m_plans; 2837 unsigned m_outOfLineStreamIndex { UINT_MAX };2857 Optional<unsigned> m_outOfLineStreamIndex; 2838 2858 }; 2839 2859 -
trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
r203491 r203537 63 63 #include "JITInlineCacheGenerator.h" 64 64 #include "JITLeftShiftGenerator.h" 65 #include "JITMathIC.h" 65 66 #include "JITMulGenerator.h" 66 67 #include "JITRightShiftGenerator.h" … … 1539 1540 void compileValueAdd() 1540 1541 { 1541 emitBinarySnippet<JITAddGenerator>(operationValueAdd); 1542 Node* node = m_node; 1543 1544 LValue left = lowJSValue(node->child1()); 1545 LValue right = lowJSValue(node->child2()); 1546 1547 SnippetOperand leftOperand(m_state.forNode(node->child1()).resultType()); 1548 SnippetOperand rightOperand(m_state.forNode(node->child2()).resultType()); 1549 1550 PatchpointValue* patchpoint = m_out.patchpoint(Int64); 1551 patchpoint->appendSomeRegister(left); 1552 patchpoint->appendSomeRegister(right); 1553 patchpoint->append(m_tagMask, ValueRep::lateReg(GPRInfo::tagMaskRegister)); 1554 patchpoint->append(m_tagTypeNumber, ValueRep::lateReg(GPRInfo::tagTypeNumberRegister)); 1555 RefPtr<PatchpointExceptionHandle> exceptionHandle = 1556 preparePatchpointForExceptions(patchpoint); 1557 patchpoint->numGPScratchRegisters = 1; 1558 patchpoint->numFPScratchRegisters = 2; 1559 patchpoint->clobber(RegisterSet::macroScratchRegisters()); 1560 State* state = &m_ftlState; 1561 patchpoint->setGenerator( 1562 [=] (CCallHelpers& jit, const StackmapGenerationParams& params) { 1563 AllowMacroScratchRegisterUsage allowScratch(jit); 1564 1565 Box<CCallHelpers::JumpList> exceptions = 1566 exceptionHandle->scheduleExitCreation(params)->jumps(jit); 1567 1568 JITAddIC* addIC = jit.codeBlock()->addJITAddIC(); 1569 Box<MathICGenerationState> addICGenerationState = Box<MathICGenerationState>::create(); 1570 ArithProfile* arithProfile = state->graph.baselineCodeBlockFor(node->origin.semantic)->arithProfileForBytecodeOffset(node->origin.semantic.bytecodeIndex); 1571 addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, JSValueRegs(params[0].gpr()), 1572 JSValueRegs(params[1].gpr()), JSValueRegs(params[2].gpr()), params.fpScratch(0), 1573 params.fpScratch(1), params.gpScratch(0), InvalidFPRReg, arithProfile); 1574 1575 bool generatedInline = addIC->generateInline(jit, *addICGenerationState); 1576 1577 if (generatedInline) { 1578 ASSERT(!addICGenerationState->slowPathJumps.empty()); 1579 auto done = jit.label(); 1580 params.addLatePath([=] (CCallHelpers& jit) { 1581 AllowMacroScratchRegisterUsage allowScratch(jit); 1582 addICGenerationState->slowPathJumps.link(&jit); 1583 addICGenerationState->slowPathStart = jit.label(); 1584 1585 if (addICGenerationState->shouldSlowPathRepatch) { 1586 SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(), 1587 operationValueAddOptimize, params[0].gpr(), params[1].gpr(), params[2].gpr(), CCallHelpers::TrustedImmPtr(addIC)); 1588 addICGenerationState->slowPathCall = call.call(); 1589 } else { 1590 SlowPathCall call = callOperation(*state, params.unavailableRegisters(), jit, node->origin.semantic, 1591 exceptions.get(), operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr()); 1592 addICGenerationState->slowPathCall = call.call(); 1593 } 1594 jit.jump().linkTo(done, &jit); 1595 1596 jit.addLinkTask([=] (LinkBuffer& linkBuffer) { 1597 addIC->finalizeInlineCode(*addICGenerationState, linkBuffer); 1598 }); 1599 }); 1600 } else { 1601 callOperation( 1602 *state, params.unavailableRegisters(), jit, node->origin.semantic, exceptions.get(), 1603 operationValueAdd, params[0].gpr(), params[1].gpr(), params[2].gpr()); 1604 } 1605 }); 1606 1607 setJSValue(patchpoint); 1542 1608 } 1543 1609 -
trunk/Source/JavaScriptCore/jit/CCallHelpers.h
r203006 r203537 816 816 817 817 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, GPRReg arg5, TrustedImm32 arg6) 818 { 819 resetCallArguments(); 820 addCallArgument(GPRInfo::callFrameRegister); 821 addCallArgument(arg1); 822 addCallArgument(arg2); 823 addCallArgument(arg3); 824 addCallArgument(arg4); 825 addCallArgument(arg5); 826 addCallArgument(arg6); 827 } 828 829 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6) 818 830 { 819 831 resetCallArguments(); … … 1595 1607 } 1596 1608 1609 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5, TrustedImmPtr arg6) 1610 { 1611 poke(arg6, POKE_ARGUMENT_OFFSET + 2); 1612 poke(arg5, POKE_ARGUMENT_OFFSET + 1); 1613 poke(arg4, POKE_ARGUMENT_OFFSET); 1614 setupArgumentsWithExecState(arg1, arg2, arg3); 1615 } 1616 1597 1617 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, GPRReg arg3, GPRReg arg4, TrustedImmPtr arg5) 1598 1618 { … … 2100 2120 } 2101 2121 2122 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, GPRReg arg2, TrustedImmPtr arg3, TrustedImmPtr arg4) 2123 { 2124 setupTwoStubArgsGPR<GPRInfo::argumentGPR1, GPRInfo::argumentGPR2>(arg1, arg2); 2125 move(arg3, GPRInfo::argumentGPR3); 2126 move(arg4, GPRInfo::argumentGPR4); 2127 move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0); 2128 } 2129 2102 2130 ALWAYS_INLINE void setupArgumentsWithExecState(GPRReg arg1, TrustedImmPtr arg2, TrustedImm32 arg3, GPRReg arg4, GPRReg arg5) 2103 2131 { … … 2208 2236 #endif 2209 2237 2238 void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2) 2239 { 2240 #if USE(JSVALUE64) 2241 setupArgumentsWithExecState(arg1.gpr(), arg2.gpr()); 2242 #else 2243 setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR()); 2244 #endif 2245 } 2246 2210 2247 void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3) 2211 2248 { … … 2214 2251 #else 2215 2252 setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3); 2253 #endif 2254 } 2255 2256 void setupArgumentsWithExecState(JSValueRegs arg1, JSValueRegs arg2, TrustedImmPtr arg3, TrustedImmPtr arg4) 2257 { 2258 #if USE(JSVALUE64) 2259 setupArgumentsWithExecState(arg1.gpr(), arg2.gpr(), arg3, arg4); 2260 #else 2261 setupArgumentsWithExecState(EABI_32BIT_DUMMY_ARG arg1.payloadGPR(), arg1.tagGPR(), arg2.payloadGPR(), arg2.tagGPR(), arg3, arg4); 2216 2262 #endif 2217 2263 } -
trunk/Source/JavaScriptCore/jit/JIT.h
r203499 r203537 44 44 #include "JITDisassembler.h" 45 45 #include "JITInlineCacheGenerator.h" 46 #include "JITMathIC.h" 46 47 #include "JSInterfaceJIT.h" 47 48 #include "Opcode.h" 48 49 #include "PCToCodeOriginMap.h" 49 #include "ResultType.h"50 50 #include "UnusedPointer.h" 51 51 … … 436 436 437 437 // Arithmetic opcode helpers 438 void emitSub32Constant(int dst, int op, int32_t constant, ResultType opType);439 438 void emitBinaryDoubleOp(OpcodeID, int dst, int op1, int op2, OperandTypes, JumpList& notInt32Op1, JumpList& notInt32Op2, bool op1IsInRegisters = true, bool op2IsInRegisters = true); 440 439 … … 765 764 MacroAssembler::Call callOperation(J_JITOperation_EJI, int, GPRReg, UniquedStringImpl*); 766 765 MacroAssembler::Call callOperation(J_JITOperation_EJJ, int, GPRReg, GPRReg); 767 MacroAssembler::Call callOperation(J_JITOperation_EJJRp, JSValueRegs, JSValueRegs, JSValueRegs, ResultProfile*); 766 MacroAssembler::Call callOperation(J_JITOperation_EJJArp, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*); 767 MacroAssembler::Call callOperation(J_JITOperation_EJJ, JSValueRegs, JSValueRegs, JSValueRegs); 768 MacroAssembler::Call callOperation(J_JITOperation_EJJArpJaic, JSValueRegs, JSValueRegs, JSValueRegs, ArithProfile*, JITAddIC*); 769 MacroAssembler::Call callOperation(J_JITOperation_EJJJaic, JSValueRegs, JSValueRegs, JSValueRegs, JITAddIC*); 768 770 MacroAssembler::Call callOperation(J_JITOperation_EJJAp, int, GPRReg, GPRReg, ArrayProfile*); 769 771 MacroAssembler::Call callOperation(J_JITOperation_EJJBy, int, GPRReg, GPRReg, ByValInfo*); … … 955 957 PCToCodeOriginMapBuilder m_pcToCodeOriginMapBuilder; 956 958 959 HashMap<Instruction*, JITAddIC*> m_instructionToJITAddIC; 960 HashMap<Instruction*, MathICGenerationState> m_instructionToJITAddICGenerationState; 961 957 962 bool m_canBeOptimized; 958 963 bool m_canBeOptimizedOrInlined; -
trunk/Source/JavaScriptCore/jit/JITAddGenerator.cpp
r200606 r203537 27 27 #include "JITAddGenerator.h" 28 28 29 #include "ArithProfile.h" 30 #include "JITMathIC.h" 31 #include "LinkBuffer.h" 32 29 33 #if ENABLE(JIT) 30 34 31 35 namespace JSC { 32 36 33 void JITAddGenerator::generateFastPath(CCallHelpers& jit) 37 JITMathICInlineResult JITAddGenerator::generateInline(CCallHelpers& jit, MathICGenerationState& state) 38 { 39 // We default to speculating int32. 40 ObservedType lhs = ObservedType().withInt32(); 41 ObservedType rhs = ObservedType().withInt32(); 42 if (m_arithProfile) { 43 lhs = m_arithProfile->lhsObservedType(); 44 rhs = m_arithProfile->rhsObservedType(); 45 if (lhs.isEmpty() || rhs.isEmpty()) { 46 lhs = ObservedType().withInt32(); 47 rhs = ObservedType().withInt32(); 48 } 49 } 50 51 if (lhs.isOnlyNonNumber() && rhs.isOnlyNonNumber()) 52 return JITMathICInlineResult::DontGenerate; 53 54 if ((lhs.isOnlyInt32() || m_leftOperand.isConstInt32()) && (rhs.isOnlyInt32() || m_rightOperand.isConstInt32())) { 55 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32()); 56 if (!m_leftOperand.isConstInt32()) 57 state.slowPathJumps.append(jit.branchIfNotInt32(m_left)); 58 if (!m_rightOperand.isConstInt32()) 59 state.slowPathJumps.append(jit.branchIfNotInt32(m_right)); 60 61 if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) { 62 JSValueRegs var = m_leftOperand.isConstInt32() ? m_right : m_left; 63 int32_t constValue = m_leftOperand.isConstInt32() ? m_leftOperand.asConstInt32() : m_rightOperand.asConstInt32(); 64 state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constValue), m_scratchGPR)); 65 } else 66 state.slowPathJumps.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR)); 67 jit.boxInt32(m_scratchGPR, m_result); 68 return JITMathICInlineResult::GeneratedFastPath; 69 } 70 71 return JITMathICInlineResult::GenerateFullSnippet; 72 } 73 74 bool JITAddGenerator::generateFastPath(CCallHelpers& jit, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList) 34 75 { 35 76 ASSERT(m_scratchGPR != InvalidGPRReg); … … 44 85 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32()); 45 86 46 if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber()) { 47 ASSERT(!m_didEmitFastPath); 48 return; 49 } 50 51 m_didEmitFastPath = true; 87 if (!m_leftOperand.mightBeNumber() || !m_rightOperand.mightBeNumber()) 88 return false; 52 89 53 90 if (m_leftOperand.isConstInt32() || m_rightOperand.isConstInt32()) { … … 59 96 CCallHelpers::Jump notInt32 = jit.branchIfNotInt32(var); 60 97 61 m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR));98 slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, var.payloadGPR(), CCallHelpers::Imm32(constOpr.asConstInt32()), m_scratchGPR)); 62 99 63 100 jit.boxInt32(m_scratchGPR, m_result); 64 m_endJumpList.append(jit.jump());101 endJumpList.append(jit.jump()); 65 102 66 103 if (!jit.supportsFloatingPoint()) { 67 m_slowPathJumpList.append(notInt32);68 return ;104 slowPathJumpList.append(notInt32); 105 return true; 69 106 } 70 107 … … 72 109 notInt32.link(&jit); 73 110 if (!varOpr.definitelyIsNumber()) 74 m_slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR));111 slowPathJumpList.append(jit.branchIfNotNumber(var, m_scratchGPR)); 75 112 76 113 jit.unboxDoubleNonDestructive(var, m_leftFPR, m_scratchGPR, m_scratchFPR); … … 90 127 rightNotInt = jit.branchIfNotInt32(m_right); 91 128 92 m_slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR));129 slowPathJumpList.append(jit.branchAdd32(CCallHelpers::Overflow, m_right.payloadGPR(), m_left.payloadGPR(), m_scratchGPR)); 93 130 94 131 jit.boxInt32(m_scratchGPR, m_result); 95 m_endJumpList.append(jit.jump()); 132 endJumpList.append(jit.jump()); 133 96 134 97 135 if (!jit.supportsFloatingPoint()) { 98 m_slowPathJumpList.append(leftNotInt);99 m_slowPathJumpList.append(rightNotInt);100 return ;136 slowPathJumpList.append(leftNotInt); 137 slowPathJumpList.append(rightNotInt); 138 return true; 101 139 } 102 140 103 141 leftNotInt.link(&jit); 104 142 if (!m_leftOperand.definitelyIsNumber()) 105 m_slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR));143 slowPathJumpList.append(jit.branchIfNotNumber(m_left, m_scratchGPR)); 106 144 if (!m_rightOperand.definitelyIsNumber()) 107 m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));145 slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR)); 108 146 109 147 jit.unboxDoubleNonDestructive(m_left, m_leftFPR, m_scratchGPR, m_scratchFPR); … … 115 153 rightNotInt.link(&jit); 116 154 if (!m_rightOperand.definitelyIsNumber()) 117 m_slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR));155 slowPathJumpList.append(jit.branchIfNotNumber(m_right, m_scratchGPR)); 118 156 119 157 jit.convertInt32ToDouble(m_left.payloadGPR(), m_leftFPR); … … 129 167 // Do doubleVar + doubleVar. 130 168 jit.addDouble(m_rightFPR, m_leftFPR); 131 if (m_ resultProfile)132 m_ resultProfile->emitSetDouble(jit);169 if (m_arithProfile) 170 m_arithProfile->emitSetDouble(jit); 133 171 134 172 jit.boxDouble(m_leftFPR, m_result); 173 174 return true; 135 175 } 136 176 -
trunk/Source/JavaScriptCore/jit/JITAddGenerator.h
r200606 r203537 30 30 31 31 #include "CCallHelpers.h" 32 #include "JITMathICInlineResult.h" 33 #include "JITOperations.h" 32 34 #include "SnippetOperand.h" 33 35 34 36 namespace JSC { 35 37 38 struct MathICGenerationState; 39 36 40 class JITAddGenerator { 37 41 public: 42 JITAddGenerator() 43 { } 44 38 45 JITAddGenerator(SnippetOperand leftOperand, SnippetOperand rightOperand, 39 46 JSValueRegs result, JSValueRegs left, JSValueRegs right, 40 47 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR, 41 ResultProfile* resultProfile = nullptr)48 ArithProfile* arithProfile = nullptr) 42 49 : m_leftOperand(leftOperand) 43 50 , m_rightOperand(rightOperand) … … 49 56 , m_scratchGPR(scratchGPR) 50 57 , m_scratchFPR(scratchFPR) 51 , m_ resultProfile(resultProfile)58 , m_arithProfile(arithProfile) 52 59 { 53 60 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32()); 54 61 } 55 62 56 void generateFastPath(CCallHelpers&); 57 58 bool didEmitFastPath() const { return m_didEmitFastPath; } 59 CCallHelpers::JumpList& endJumpList() { return m_endJumpList; } 60 CCallHelpers::JumpList& slowPathJumpList() { return m_slowPathJumpList; } 63 JITMathICInlineResult generateInline(CCallHelpers&, MathICGenerationState&); 64 bool generateFastPath(CCallHelpers&, CCallHelpers::JumpList& endJumpList, CCallHelpers::JumpList& slowPathJumpList); 61 65 62 66 private: … … 70 74 GPRReg m_scratchGPR; 71 75 FPRReg m_scratchFPR; 72 ResultProfile* m_resultProfile; 73 bool m_didEmitFastPath { false }; 74 75 CCallHelpers::JumpList m_endJumpList; 76 CCallHelpers::JumpList m_slowPathJumpList; 76 ArithProfile* m_arithProfile; 77 77 }; 78 78 -
trunk/Source/JavaScriptCore/jit/JITArithmetic.cpp
r203499 r203537 29 29 #include "JIT.h" 30 30 31 #include "ArithProfile.h" 31 32 #include "CodeBlock.h" 32 33 #include "JITAddGenerator.h" … … 37 38 #include "JITInlines.h" 38 39 #include "JITLeftShiftGenerator.h" 40 #include "JITMathIC.h" 39 41 #include "JITMulGenerator.h" 40 42 #include "JITNegGenerator.h" … … 46 48 #include "Interpreter.h" 47 49 #include "JSCInlines.h" 50 #include "LinkBuffer.h" 48 51 #include "ResultType.h" 49 52 #include "SlowPathCall.h" 50 51 53 52 54 namespace JSC { … … 676 678 } 677 679 680 ALWAYS_INLINE static OperandTypes getOperandTypes(Instruction* instruction) 681 { 682 return OperandTypes(ArithProfile::fromInt(instruction[4].u.operand).lhsResultType(), ArithProfile::fromInt(instruction[4].u.operand).rhsResultType()); 683 } 684 678 685 void JIT::emit_op_add(Instruction* currentInstruction) 679 686 { … … 681 688 int op1 = currentInstruction[2].u.operand; 682 689 int op2 = currentInstruction[3].u.operand; 683 OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand); 684 685 #if USE(JSVALUE64) 690 691 #if USE(JSVALUE64) 692 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction)); 686 693 JSValueRegs leftRegs = JSValueRegs(regT0); 687 694 JSValueRegs rightRegs = JSValueRegs(regT1); … … 690 697 FPRReg scratchFPR = InvalidFPRReg; 691 698 #else 699 OperandTypes types = getOperandTypes(currentInstruction); 692 700 JSValueRegs leftRegs = JSValueRegs(regT1, regT0); 693 701 JSValueRegs rightRegs = JSValueRegs(regT3, regT2); … … 697 705 #endif 698 706 699 ResultProfile* resultProfile = nullptr;707 ArithProfile* arithProfile = nullptr; 700 708 if (shouldEmitProfiling()) 701 resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);709 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction); 702 710 703 711 SnippetOperand leftOperand(types.first()); … … 716 724 emitGetVirtualRegister(op2, rightRegs); 717 725 718 JITAddGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, 719 fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile); 720 721 gen.generateFastPath(*this); 722 723 if (gen.didEmitFastPath()) { 724 gen.endJumpList().link(this); 725 emitPutVirtualRegister(result, resultRegs); 726 727 addSlowCase(gen.slowPathJumpList()); 728 } else { 729 ASSERT(gen.endJumpList().empty()); 730 ASSERT(gen.slowPathJumpList().empty()); 731 if (resultProfile) { 732 if (leftOperand.isConst()) 733 emitGetVirtualRegister(op1, leftRegs); 734 if (rightOperand.isConst()) 735 emitGetVirtualRegister(op2, rightRegs); 736 callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile); 737 emitPutVirtualRegister(result, resultRegs); 738 } else { 739 JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add); 740 slowPathCall.call(); 741 } 742 } 726 JITAddIC* addIC = m_codeBlock->addJITAddIC(); 727 728 m_instructionToJITAddIC.add(currentInstruction, addIC); 729 MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.add(currentInstruction, MathICGenerationState()).iterator->value; 730 731 addIC->m_generator = JITAddGenerator(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile); 732 733 bool generatedInlineCode = addIC->generateInline(*this, addICGenerationState); 734 if (!generatedInlineCode) { 735 if (leftOperand.isConst()) 736 emitGetVirtualRegister(op1, leftRegs); 737 else if (rightOperand.isConst()) 738 emitGetVirtualRegister(op2, rightRegs); 739 740 if (arithProfile) 741 callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, arithProfile); 742 else 743 callOperation(operationValueAdd, resultRegs, leftRegs, rightRegs); 744 } else 745 addSlowCase(addICGenerationState.slowPathJumps); 746 emitPutVirtualRegister(result, resultRegs); 743 747 } 744 748 … … 746 750 { 747 751 linkAllSlowCasesForBytecodeOffset(m_slowCases, iter, m_bytecodeOffset); 752 753 MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value; 754 addICGenerationState.slowPathStart = label(); 748 755 749 756 int result = currentInstruction[1].u.operand; 750 757 int op1 = currentInstruction[2].u.operand; 751 758 int op2 = currentInstruction[3].u.operand; 752 OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand); 753 754 #if USE(JSVALUE64) 759 760 #if USE(JSVALUE64) 761 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction)); 755 762 JSValueRegs leftRegs = JSValueRegs(regT0); 756 763 JSValueRegs rightRegs = JSValueRegs(regT1); 757 764 JSValueRegs resultRegs = leftRegs; 758 765 #else 766 OperandTypes types = getOperandTypes(currentInstruction); 759 767 JSValueRegs leftRegs = JSValueRegs(regT1, regT0); 760 768 JSValueRegs rightRegs = JSValueRegs(regT3, regT2); … … 770 778 rightOperand.setConstInt32(getOperandConstantInt(op2)); 771 779 780 if (leftOperand.isConst()) 781 emitGetVirtualRegister(op1, leftRegs); 782 if (rightOperand.isConst()) 783 emitGetVirtualRegister(op2, rightRegs); 784 785 JITAddIC* addIC = m_instructionToJITAddIC.get(currentInstruction); 772 786 if (shouldEmitProfiling()) { 773 if (leftOperand.isConst()) 774 emitGetVirtualRegister(op1, leftRegs); 775 if (rightOperand.isConst()) 776 emitGetVirtualRegister(op2, rightRegs); 777 ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset); 778 callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, resultProfile); 779 emitPutVirtualRegister(result, resultRegs); 780 } else { 781 JITSlowPathCall slowPathCall(this, currentInstruction, slow_path_add); 782 slowPathCall.call(); 783 } 787 ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction); 788 if (addICGenerationState.shouldSlowPathRepatch) 789 addICGenerationState.slowPathCall = callOperation(operationValueAddProfiledOptimize, resultRegs, leftRegs, rightRegs, &arithProfile, addIC); 790 else 791 addICGenerationState.slowPathCall = callOperation(operationValueAddProfiled, resultRegs, leftRegs, rightRegs, &arithProfile); 792 } else 793 addICGenerationState.slowPathCall = callOperation(operationValueAddOptimize, resultRegs, leftRegs, rightRegs, addIC); 794 emitPutVirtualRegister(result, resultRegs); 795 796 addLinkTask([=] (LinkBuffer& linkBuffer) { 797 MathICGenerationState& addICGenerationState = m_instructionToJITAddICGenerationState.find(currentInstruction)->value; 798 addIC->finalizeInlineCode(addICGenerationState, linkBuffer); 799 }); 784 800 } 785 801 … … 789 805 int op1 = currentInstruction[2].u.operand; 790 806 int op2 = currentInstruction[3].u.operand; 791 OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand); 792 793 #if USE(JSVALUE64) 807 808 #if USE(JSVALUE64) 809 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction)); 794 810 JSValueRegs leftRegs = JSValueRegs(regT0); 795 811 JSValueRegs rightRegs = JSValueRegs(regT1); … … 797 813 GPRReg scratchGPR = regT2; 798 814 #else 815 OperandTypes types = getOperandTypes(currentInstruction); 799 816 JSValueRegs leftRegs = JSValueRegs(regT1, regT0); 800 817 JSValueRegs rightRegs = JSValueRegs(regT3, regT2); … … 804 821 FPRReg scratchFPR = fpRegT2; 805 822 806 ResultProfile* resultProfile = nullptr;823 ArithProfile* arithProfile = nullptr; 807 824 if (shouldEmitProfiling()) 808 resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);825 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction); 809 826 810 827 SnippetOperand leftOperand(types.first()); … … 832 849 833 850 JITDivGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, 834 fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);851 fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile); 835 852 836 853 gen.generateFastPath(*this); … … 862 879 int op1 = currentInstruction[2].u.operand; 863 880 int op2 = currentInstruction[3].u.operand; 864 OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand); 865 866 #if USE(JSVALUE64) 881 882 #if USE(JSVALUE64) 883 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction)); 867 884 JSValueRegs leftRegs = JSValueRegs(regT0); 868 885 JSValueRegs rightRegs = JSValueRegs(regT1); … … 871 888 FPRReg scratchFPR = InvalidFPRReg; 872 889 #else 890 OperandTypes types = getOperandTypes(currentInstruction); 873 891 JSValueRegs leftRegs = JSValueRegs(regT1, regT0); 874 892 JSValueRegs rightRegs = JSValueRegs(regT3, regT2); … … 878 896 #endif 879 897 880 ResultProfile* resultProfile = nullptr;898 ArithProfile* arithProfile = nullptr; 881 899 if (shouldEmitProfiling()) 882 resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);900 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction); 883 901 884 902 SnippetOperand leftOperand(types.first()); … … 898 916 899 917 JITMulGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, 900 fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);918 fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile); 901 919 902 920 gen.generateFastPath(*this); … … 910 928 ASSERT(gen.endJumpList().empty()); 911 929 ASSERT(gen.slowPathJumpList().empty()); 912 if ( resultProfile) {930 if (arithProfile) { 913 931 if (leftOperand.isPositiveConstInt32()) 914 932 emitGetVirtualRegister(op1, leftRegs); 915 933 if (rightOperand.isPositiveConstInt32()) 916 934 emitGetVirtualRegister(op2, rightRegs); 917 callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);935 callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, arithProfile); 918 936 emitPutVirtualRegister(result, resultRegs); 919 937 } else { … … 931 949 int op1 = currentInstruction[2].u.operand; 932 950 int op2 = currentInstruction[3].u.operand; 933 OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand); 934 935 #if USE(JSVALUE64) 951 952 #if USE(JSVALUE64) 953 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction)); 936 954 JSValueRegs leftRegs = JSValueRegs(regT0); 937 955 JSValueRegs rightRegs = JSValueRegs(regT1); 938 956 JSValueRegs resultRegs = leftRegs; 939 957 #else 958 OperandTypes types = getOperandTypes(currentInstruction); 940 959 JSValueRegs leftRegs = JSValueRegs(regT1, regT0); 941 960 JSValueRegs rightRegs = JSValueRegs(regT3, regT2); … … 956 975 if (rightOperand.isPositiveConstInt32()) 957 976 emitGetVirtualRegister(op2, rightRegs); 958 ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);959 callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, resultProfile);977 ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction); 978 callOperation(operationValueMulProfiled, resultRegs, leftRegs, rightRegs, &arithProfile); 960 979 emitPutVirtualRegister(result, resultRegs); 961 980 } else { … … 970 989 int op1 = currentInstruction[2].u.operand; 971 990 int op2 = currentInstruction[3].u.operand; 972 OperandTypes types = OperandTypes::fromInt(currentInstruction[4].u.operand); 973 974 #if USE(JSVALUE64) 991 992 #if USE(JSVALUE64) 993 OperandTypes types = getOperandTypes(copiedInstruction(currentInstruction)); 975 994 JSValueRegs leftRegs = JSValueRegs(regT0); 976 995 JSValueRegs rightRegs = JSValueRegs(regT1); … … 979 998 FPRReg scratchFPR = InvalidFPRReg; 980 999 #else 1000 OperandTypes types = getOperandTypes(currentInstruction); 981 1001 JSValueRegs leftRegs = JSValueRegs(regT1, regT0); 982 1002 JSValueRegs rightRegs = JSValueRegs(regT3, regT2); … … 986 1006 #endif 987 1007 988 ResultProfile* resultProfile = nullptr;1008 ArithProfile* arithProfile = nullptr; 989 1009 if (shouldEmitProfiling()) 990 resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);1010 arithProfile = &m_codeBlock->arithProfileForPC(currentInstruction); 991 1011 992 1012 SnippetOperand leftOperand(types.first()); … … 997 1017 998 1018 JITSubGenerator gen(leftOperand, rightOperand, resultRegs, leftRegs, rightRegs, 999 fpRegT0, fpRegT1, scratchGPR, scratchFPR, resultProfile);1019 fpRegT0, fpRegT1, scratchGPR, scratchFPR, arithProfile); 1000 1020 1001 1021 gen.generateFastPath(*this); … … 1024 1044 1025 1045 if (shouldEmitProfiling()) { 1026 ResultProfile* resultProfile = m_codeBlock->ensureResultProfile(m_bytecodeOffset);1027 callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, resultProfile);1046 ArithProfile& arithProfile = m_codeBlock->arithProfileForPC(currentInstruction); 1047 callOperation(operationValueSubProfiled, resultRegs, leftRegs, rightRegs, &arithProfile); 1028 1048 emitPutVirtualRegister(result, resultRegs); 1029 1049 } else { -
trunk/Source/JavaScriptCore/jit/JITDivGenerator.cpp
r199866 r203537 29 29 #if ENABLE(JIT) 30 30 31 #include "ArithProfile.h" 31 32 #include "JSCJSValueInlines.h" 32 33 #include "MathCommon.h" … … 102 103 103 104 // Is the result actually an integer? The DFG JIT would really like to know. If it's 104 // not an integer, we increment a count. If this together with the slow case counter105 // are below threshold then the DFG JIT will compile this division with a speculation106 // that theremainder is zero.105 // not an integer, we set a bit. If this together with the slow case counter are below 106 // threshold then the DFG JIT will compile this division with a speculation that the 107 // remainder is zero. 107 108 108 109 // As well, there are cases where a double result here would cause an important field … … 129 130 notDoubleZero.link(&jit); 130 131 #endif 131 if (m_ resultProfile)132 jit. add32(CCallHelpers::TrustedImm32(1), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfSpecialFastPathCount()));132 if (m_arithProfile) 133 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::specialFastPathBit), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits())); 133 134 jit.boxDouble(m_leftFPR, m_result); 134 135 } -
trunk/Source/JavaScriptCore/jit/JITDivGenerator.h
r194294 r203537 39 39 JSValueRegs result, JSValueRegs left, JSValueRegs right, 40 40 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR, 41 ResultProfile* resultProfile = nullptr)41 ArithProfile* arithProfile = nullptr) 42 42 : m_leftOperand(leftOperand) 43 43 , m_rightOperand(rightOperand) … … 49 49 , m_scratchGPR(scratchGPR) 50 50 , m_scratchFPR(scratchFPR) 51 , m_ resultProfile(resultProfile)51 , m_arithProfile(arithProfile) 52 52 { 53 53 ASSERT(!m_leftOperand.isConstInt32() || !m_rightOperand.isConstInt32()); … … 72 72 GPRReg m_scratchGPR; 73 73 FPRReg m_scratchFPR; 74 ResultProfile* m_resultProfile;74 ArithProfile* m_arithProfile; 75 75 bool m_didEmitFastPath { false }; 76 76 -
trunk/Source/JavaScriptCore/jit/JITInlines.h
r203006 r203537 417 417 } 418 418 419 inline MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJRp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ResultProfile* resultProfile) 420 { 421 setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(resultProfile)); 419 ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJ operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2) 420 { 421 setupArgumentsWithExecState(arg1, arg2); 422 Call call = appendCallWithExceptionCheck(operation); 423 setupResults(result); 424 return call; 425 } 426 427 ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArp operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile) 428 { 429 setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile)); 430 Call call = appendCallWithExceptionCheck(operation); 431 setupResults(result); 432 return call; 433 } 434 435 ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJArpJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, ArithProfile* arithProfile, JITAddIC* addIC) 436 { 437 setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(arithProfile), TrustedImmPtr(addIC)); 438 Call call = appendCallWithExceptionCheck(operation); 439 setupResults(result); 440 return call; 441 } 442 443 ALWAYS_INLINE MacroAssembler::Call JIT::callOperation(J_JITOperation_EJJJaic operation, JSValueRegs result, JSValueRegs arg1, JSValueRegs arg2, JITAddIC* addIC) 444 { 445 setupArgumentsWithExecState(arg1, arg2, TrustedImmPtr(addIC)); 422 446 Call call = appendCallWithExceptionCheck(operation); 423 447 setupResults(result); -
trunk/Source/JavaScriptCore/jit/JITMulGenerator.cpp
r200606 r203537 29 29 #if ENABLE(JIT) 30 30 31 #include "ArithProfile.h" 32 31 33 namespace JSC { 32 34 … … 138 140 jit.mulDouble(m_rightFPR, m_leftFPR); 139 141 140 if (!m_ resultProfile)142 if (!m_arithProfile) 141 143 jit.boxDouble(m_leftFPR, m_result); 142 144 else { … … 150 152 CCallHelpers::Jump notNegativeZero = jit.branch64(CCallHelpers::NotEqual, m_result.payloadGPR(), CCallHelpers::TrustedImm64(negativeZeroBits)); 151 153 152 jit.or32(CCallHelpers::TrustedImm32( ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));154 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits())); 153 155 CCallHelpers::Jump done = jit.jump(); 154 156 155 157 notNegativeZero.link(&jit); 156 jit.or32(CCallHelpers::TrustedImm32( ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));158 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits())); 157 159 158 160 jit.move(m_result.payloadGPR(), m_scratchGPR); … … 161 163 CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431)); 162 164 163 jit.or32(CCallHelpers::TrustedImm32( ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));165 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits())); 164 166 noInt52Overflow.link(&jit); 165 167 … … 172 174 notNegativeZero.append(jit.branch32(CCallHelpers::NotEqual, m_result.tagGPR(), CCallHelpers::TrustedImm32(negativeZeroBits >> 32))); 173 175 174 jit.or32(CCallHelpers::TrustedImm32( ResultProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));176 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits())); 175 177 CCallHelpers::Jump done = jit.jump(); 176 178 177 179 notNegativeZero.link(&jit); 178 jit.or32(CCallHelpers::TrustedImm32( ResultProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));180 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::NonNegZeroDouble), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits())); 179 181 180 182 jit.move(m_result.tagGPR(), m_scratchGPR); … … 183 185 CCallHelpers::Jump noInt52Overflow = jit.branch32(CCallHelpers::LessThanOrEqual, m_scratchGPR, CCallHelpers::TrustedImm32(0x431)); 184 186 185 jit.or32(CCallHelpers::TrustedImm32( ResultProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_resultProfile->addressOfFlags()));187 jit.or32(CCallHelpers::TrustedImm32(ArithProfile::Int52Overflow), CCallHelpers::AbsoluteAddress(m_arithProfile->addressOfBits())); 186 188 187 189 m_endJumpList.append(noInt52Overflow); -
trunk/Source/JavaScriptCore/jit/JITMulGenerator.h
r194294 r203537 39 39 JSValueRegs result, JSValueRegs left, JSValueRegs right, 40 40 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR, 41 ResultProfile* resultProfile = nullptr)41 ArithProfile* arithProfile = nullptr) 42 42 : m_leftOperand(leftOperand) 43 43 , m_rightOperand(rightOperand) … … 49 49 , m_scratchGPR(scratchGPR) 50 50 , m_scratchFPR(scratchFPR) 51 , m_ resultProfile(resultProfile)51 , m_arithProfile(arithProfile) 52 52 { 53 53 ASSERT(!m_leftOperand.isPositiveConstInt32() || !m_rightOperand.isPositiveConstInt32()); … … 70 70 GPRReg m_scratchGPR; 71 71 FPRReg m_scratchFPR; 72 ResultProfile* m_resultProfile;72 ArithProfile* m_arithProfile; 73 73 bool m_didEmitFastPath { false }; 74 74 -
trunk/Source/JavaScriptCore/jit/JITOperations.cpp
r203452 r203537 29 29 #if ENABLE(JIT) 30 30 31 #include "ArithProfile.h" 31 32 #include "ArrayConstructor.h" 32 33 #include "CommonSlowPaths.h" … … 2248 2249 } 2249 2250 2250 EncodedJSValue JIT_OPERATION operationValueAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2)2251 ALWAYS_INLINE static EncodedJSValue unprofiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) 2251 2252 { 2252 2253 VM* vm = &exec->vm(); … … 2259 2260 } 2260 2261 2261 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)2262 ALWAYS_INLINE static EncodedJSValue profiledAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile) 2262 2263 { 2263 2264 VM* vm = &exec->vm(); … … 2266 2267 JSValue op1 = JSValue::decode(encodedOp1); 2267 2268 JSValue op2 = JSValue::decode(encodedOp2); 2268 2269 2270 ASSERT(arithProfile); 2271 arithProfile->observeLHSAndRHS(op1, op2); 2272 2269 2273 JSValue result = jsAdd(exec, op1, op2); 2270 resultProfile->detectNumericness(result); 2274 arithProfile->observeResult(result); 2275 2276 return JSValue::encode(result); 2277 } 2278 2279 EncodedJSValue JIT_OPERATION operationValueAdd(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) 2280 { 2281 return unprofiledAdd(exec, encodedOp1, encodedOp2); 2282 } 2283 2284 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile) 2285 { 2286 return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile); 2287 } 2288 2289 EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC* addIC) 2290 { 2291 VM* vm = &exec->vm(); 2292 NativeCallFrameTracer tracer(vm, exec); 2293 2294 JSValue op1 = JSValue::decode(encodedOp1); 2295 JSValue op2 = JSValue::decode(encodedOp2); 2296 2297 ASSERT(arithProfile); 2298 arithProfile->observeLHSAndRHS(op1, op2); 2299 auto nonOptimizeVariant = operationValueAddProfiledNoOptimize; 2300 addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant); 2301 2302 JSValue result = jsAdd(exec, op1, op2); 2303 arithProfile->observeResult(result); 2304 2305 return JSValue::encode(result); 2306 } 2307 2308 EncodedJSValue JIT_OPERATION operationValueAddProfiledNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile, JITAddIC*) 2309 { 2310 return profiledAdd(exec, encodedOp1, encodedOp2, arithProfile); 2311 } 2312 2313 EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC* addIC) 2314 { 2315 VM* vm = &exec->vm(); 2316 NativeCallFrameTracer tracer(vm, exec); 2317 2318 auto nonOptimizeVariant = operationValueAddNoOptimize; 2319 addIC->generateOutOfLine(*vm, exec->codeBlock(), nonOptimizeVariant); 2320 2321 JSValue op1 = JSValue::decode(encodedOp1); 2322 JSValue op2 = JSValue::decode(encodedOp2); 2323 return JSValue::encode(jsAdd(exec, op1, op2)); 2324 } 2325 2326 EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) 2327 { 2328 VM* vm = &exec->vm(); 2329 NativeCallFrameTracer tracer(vm, exec); 2330 2331 JSValue op1 = JSValue::decode(encodedOp1); 2332 JSValue op2 = JSValue::decode(encodedOp2); 2333 2334 JSValue result = jsAdd(exec, op1, op2); 2335 2271 2336 return JSValue::encode(result); 2272 2337 } … … 2285 2350 } 2286 2351 2287 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)2352 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile) 2288 2353 { 2289 2354 VM* vm = &exec->vm(); … … 2297 2362 2298 2363 JSValue result = jsNumber(a * b); 2299 resultProfile->detectNumericness(result);2364 arithProfile->observeResult(result); 2300 2365 return JSValue::encode(result); 2301 2366 } … … 2314 2379 } 2315 2380 2316 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile* resultProfile)2381 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState* exec, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile* arithProfile) 2317 2382 { 2318 2383 VM* vm = &exec->vm(); … … 2326 2391 2327 2392 JSValue result = jsNumber(a - b); 2328 resultProfile->detectNumericness(result);2393 arithProfile->observeResult(result); 2329 2394 return JSValue::encode(result); 2330 2395 } -
trunk/Source/JavaScriptCore/jit/JITOperations.h
r202125 r203537 29 29 #if ENABLE(JIT) 30 30 31 #include "JITMathICForwards.h" 31 32 #include "MacroAssemblerCodeRef.h" 32 33 #include "PropertyOffset.h" … … 43 44 class CodeBlock; 44 45 class ExecState; 46 class JITAddGenerator; 45 47 class JSArray; 46 48 class JSFunction; … … 55 57 struct ByValInfo; 56 58 struct InlineCallFrame; 57 struct ResultProfile;59 struct ArithProfile; 58 60 59 61 typedef ExecState CallFrame; … … 70 72 Aap: ArrayAllocationProfile* 71 73 Ap: ArrayProfile* 74 Arp: ArithProfile* 72 75 By: ByValInfo* 73 76 C: JSCell* … … 82 85 Idc: const Identifier* 83 86 J: EncodedJSValue 87 Jaic: JITAddIC* 84 88 Jcp: const JSValue* 85 89 Jsc: JSScope* … … 93 97 R: Register 94 98 Reo: RegExpObject* 95 Rp: ResultProfile*96 99 S: size_t 97 100 Sprt: SlowPathReturnType … … 134 137 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJBy)(ExecState*, EncodedJSValue, EncodedJSValue, ByValInfo*); 135 138 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJ)(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue); 136 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJRp)(ExecState*, EncodedJSValue, EncodedJSValue, ResultProfile*); 139 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArp)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*); 140 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJArpJaic)(ExecState*, EncodedJSValue, EncodedJSValue, ArithProfile*, JITAddIC*); 141 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJJJaic)(ExecState*, EncodedJSValue, EncodedJSValue, JITAddIC*); 137 142 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssZ)(ExecState*, JSString*, int32_t); 138 143 typedef EncodedJSValue (JIT_OPERATION *J_JITOperation_EJssReo)(ExecState*, JSString*, RegExpObject*); … … 409 414 410 415 EncodedJSValue JIT_OPERATION operationValueAdd(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL; 411 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL; 416 EncodedJSValue JIT_OPERATION operationValueAddProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL; 417 EncodedJSValue JIT_OPERATION operationValueAddProfiledOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL; 418 EncodedJSValue JIT_OPERATION operationValueAddProfiledNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*, JITAddIC*) WTF_INTERNAL; 419 EncodedJSValue JIT_OPERATION operationValueAddOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL; 420 EncodedJSValue JIT_OPERATION operationValueAddNoOptimize(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, JITAddIC*) WTF_INTERNAL; 412 421 EncodedJSValue JIT_OPERATION operationValueMul(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL; 413 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;422 EncodedJSValue JIT_OPERATION operationValueMulProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL; 414 423 EncodedJSValue JIT_OPERATION operationValueSub(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2) WTF_INTERNAL; 415 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ResultProfile*) WTF_INTERNAL;424 EncodedJSValue JIT_OPERATION operationValueSubProfiled(ExecState*, EncodedJSValue encodedOp1, EncodedJSValue encodedOp2, ArithProfile*) WTF_INTERNAL; 416 425 417 426 void JIT_OPERATION operationProcessTypeProfilerLog(ExecState*) WTF_INTERNAL; -
trunk/Source/JavaScriptCore/jit/JITSubGenerator.cpp
r200606 r203537 26 26 #include "config.h" 27 27 #include "JITSubGenerator.h" 28 29 #include "ArithProfile.h" 28 30 29 31 #if ENABLE(JIT) … … 84 86 85 87 jit.subDouble(m_rightFPR, m_leftFPR); 86 if (m_ resultProfile)87 m_ resultProfile->emitSetDouble(jit);88 if (m_arithProfile) 89 m_arithProfile->emitSetDouble(jit); 88 90 89 91 jit.boxDouble(m_leftFPR, m_result); -
trunk/Source/JavaScriptCore/jit/JITSubGenerator.h
r200606 r203537 39 39 JSValueRegs result, JSValueRegs left, JSValueRegs right, 40 40 FPRReg leftFPR, FPRReg rightFPR, GPRReg scratchGPR, FPRReg scratchFPR, 41 ResultProfile* resultProfile = nullptr)41 ArithProfile* arithProfile = nullptr) 42 42 : m_leftOperand(leftOperand) 43 43 , m_rightOperand(rightOperand) … … 49 49 , m_scratchGPR(scratchGPR) 50 50 , m_scratchFPR(scratchFPR) 51 , m_ resultProfile(resultProfile)51 , m_arithProfile(arithProfile) 52 52 { } 53 53 … … 68 68 GPRReg m_scratchGPR; 69 69 FPRReg m_scratchFPR; 70 ResultProfile* m_resultProfile;70 ArithProfile* m_arithProfile; 71 71 bool m_didEmitFastPath { false }; 72 72 -
trunk/Source/JavaScriptCore/jit/Repatch.cpp
r203272 r203537 73 73 } 74 74 75 static void repatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction)75 void ftlThunkAwareRepatchCall(CodeBlock* codeBlock, CodeLocationCall call, FunctionPtr newCalleeFunction) 76 76 { 77 77 #if ENABLE(FTL_JIT) … … 168 168 bool generatedCodeInline = InlineAccess::generateArrayLength(*codeBlock->vm(), stubInfo, jsCast<JSArray*>(baseValue)); 169 169 if (generatedCodeInline) { 170 repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));170 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind)); 171 171 stubInfo.initArrayLength(); 172 172 return RetryCacheLater; … … 221 221 LOG_IC((ICEvent::GetByIdSelfPatch, structure->classInfo(), propertyName)); 222 222 structure->startWatchingPropertyForReplacements(vm, slot.cachedOffset()); 223 repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));223 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind)); 224 224 stubInfo.initGetByIdSelf(codeBlock, structure, slot.cachedOffset()); 225 225 return RetryCacheLater; … … 315 315 316 316 if (tryCacheGetByID(exec, baseValue, propertyName, slot, stubInfo, kind) == GiveUpOnCache) 317 repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind));317 ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericGetByIdFunction(kind)); 318 318 } 319 319 … … 373 373 if (generatedCodeInline) { 374 374 LOG_IC((ICEvent::PutByIdSelfPatch, structure->classInfo(), ident)); 375 repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind));375 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingPutByIdFunction(slot, putKind)); 376 376 stubInfo.initPutByIdReplace(codeBlock, structure, slot.cachedOffset()); 377 377 return RetryCacheLater; … … 469 469 470 470 if (tryCachePutByID(exec, baseValue, structure, propertyName, slot, stubInfo, putKind) == GiveUpOnCache) 471 repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind));471 ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), appropriateGenericPutByIdFunction(slot, putKind)); 472 472 } 473 473 … … 530 530 SuperSamplerScope superSamplerScope(false); 531 531 if (tryRepatchIn(exec, base, ident, wasFound, slot, stubInfo) == GiveUpOnCache) 532 repatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn);532 ftlThunkAwareRepatchCall(exec->codeBlock(), stubInfo.slowPathCallLocation(), operationIn); 533 533 } 534 534 … … 902 902 void resetGetByID(CodeBlock* codeBlock, StructureStubInfo& stubInfo, GetByIDKind kind) 903 903 { 904 repatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind));904 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), appropriateOptimizingGetByIdFunction(kind)); 905 905 InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation()); 906 906 } … … 921 921 } 922 922 923 repatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction);923 ftlThunkAwareRepatchCall(codeBlock, stubInfo.slowPathCallLocation(), optimizedFunction); 924 924 InlineAccess::rewireStubAsJump(*codeBlock->vm(), stubInfo, stubInfo.slowPathStartLocation()); 925 925 } -
trunk/Source/JavaScriptCore/jit/Repatch.h
r199170 r203537 55 55 void resetPutByID(CodeBlock*, StructureStubInfo&); 56 56 void resetIn(CodeBlock*, StructureStubInfo&); 57 void ftlThunkAwareRepatchCall(CodeBlock*, CodeLocationCall, FunctionPtr newCalleeFunction); 57 58 58 59 } // namespace JSC -
trunk/Source/JavaScriptCore/llint/LLIntData.cpp
r203006 r203537 27 27 #include "LLIntData.h" 28 28 29 #include "ArithProfile.h" 29 30 #include "BytecodeConventions.h" 30 31 #include "CodeBlock.h" … … 224 225 225 226 ASSERT(StringImpl::s_hashFlag8BitBuffer == 8); 227 228 { 229 uint32_t bits = 0x120000; 230 UNUSED_PARAM(bits); 231 ArithProfile arithProfile; 232 arithProfile.lhsSawInt32(); 233 arithProfile.rhsSawInt32(); 234 ASSERT(arithProfile.bits() == bits); 235 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32()); 236 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32()); 237 } 238 { 239 uint32_t bits = 0x220000; 240 UNUSED_PARAM(bits); 241 ArithProfile arithProfile; 242 arithProfile.lhsSawNumber(); 243 arithProfile.rhsSawInt32(); 244 ASSERT(arithProfile.bits() == bits); 245 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber()); 246 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyInt32()); 247 } 248 { 249 uint32_t bits = 0x240000; 250 UNUSED_PARAM(bits); 251 ArithProfile arithProfile; 252 arithProfile.lhsSawNumber(); 253 arithProfile.rhsSawNumber(); 254 ASSERT(arithProfile.bits() == bits); 255 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyNumber()); 256 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber()); 257 } 258 { 259 uint32_t bits = 0x140000; 260 UNUSED_PARAM(bits); 261 ArithProfile arithProfile; 262 arithProfile.lhsSawInt32(); 263 arithProfile.rhsSawNumber(); 264 ASSERT(arithProfile.bits() == bits); 265 ASSERT(ArithProfile::fromInt(bits).lhsObservedType().isOnlyInt32()); 266 ASSERT(ArithProfile::fromInt(bits).rhsObservedType().isOnlyNumber()); 267 } 226 268 } 227 269 #if COMPILER(CLANG) -
trunk/Source/JavaScriptCore/llint/LowLevelInterpreter.asm
r203499 r203537 252 252 # ShadowChicken data 253 253 const ShadowChickenTailMarker = 0x7a11 254 255 # ArithProfile data 256 const ArithProfileIntInt = 0x120000 257 const ArithProfileNumberInt = 0x220000 258 const ArithProfileNumberNumber = 0x240000 259 const ArithProfileIntNumber = 0x140000 254 260 255 261 # Some register conventions. … … 287 293 end 288 294 295 macro storeisToInstruction(value, offset) 296 storei value, offset * 8[PB, PC, 8] 297 end 298 289 299 macro storepToInstruction(value, offset) 290 300 storep value, offset * 8[PB, PC, 8] … … 299 309 macro loadpFromInstruction(offset, dest) 300 310 loadp offset * 4[PC], dest 311 end 312 313 macro storeisToInstruction(value, offset) 314 storei value, offset * 4[PC] 301 315 end 302 316 end -
trunk/Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm
r203361 r203537 1015 1015 bineq t2, Int32Tag, .op1NotInt 1016 1016 bineq t3, Int32Tag, .op2NotInt 1017 loadisFromInstruction(4, t5) 1018 ori ArithProfileIntInt, t5 1019 storeisToInstruction(t5, 4) 1017 1020 loadi 4[PC], t2 1018 1021 integerOperationAndStore(t3, t1, t0, .slow, t2) … … 1024 1027 bib t3, LowestTag, .op1NotIntOp2Double 1025 1028 bineq t3, Int32Tag, .slow 1029 loadisFromInstruction(4, t5) 1030 ori ArithProfileNumberInt, t5 1031 storeisToInstruction(t5, 4) 1026 1032 ci2d t1, ft1 1027 1033 jmp .op1NotIntReady 1028 1034 .op1NotIntOp2Double: 1029 1035 fii2d t1, t3, ft1 1036 loadisFromInstruction(4, t5) 1037 ori ArithProfileNumberNumber, t5 1038 storeisToInstruction(t5, 4) 1030 1039 .op1NotIntReady: 1031 1040 loadi 4[PC], t1 … … 1039 1048 loadi 4[PC], t2 1040 1049 bia t3, LowestTag, .slow 1050 loadisFromInstruction(4, t5) 1051 ori ArithProfileIntNumber, t5 1052 storeisToInstruction(t5, 4) 1041 1053 ci2d t0, ft0 1042 1054 fii2d t1, t3, ft1 -
trunk/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
r203361 r203537 894 894 loadisFromInstruction(1, t2) 895 895 integerOperationAndStore(t1, t0, .slow, t2) 896 loadisFromInstruction(4, t1) 897 ori ArithProfileIntInt, t1 898 storeisToInstruction(t1, 4) 896 899 dispatch(5) 897 900 … … 903 906 addq tagTypeNumber, t1 904 907 fq2d t1, ft1 908 loadisFromInstruction(4, t2) 909 ori ArithProfileNumberNumber, t2 910 storeisToInstruction(t2, 4) 905 911 jmp .op1NotIntReady 906 912 .op1NotIntOp2Int: 913 loadisFromInstruction(4, t2) 914 ori ArithProfileNumberInt, t2 915 storeisToInstruction(t2, 4) 907 916 ci2d t1, ft1 908 917 .op1NotIntReady: … … 920 929 loadisFromInstruction(1, t2) 921 930 btqz t1, tagTypeNumber, .slow 931 loadisFromInstruction(4, t3) 932 ori ArithProfileIntNumber, t3 933 storeisToInstruction(t3, 4) 922 934 ci2d t0, ft0 923 935 addq tagTypeNumber, t1 -
trunk/Source/JavaScriptCore/parser/ResultType.h
r192836 r203537 33 33 friend struct OperandTypes; 34 34 35 typedef charType;35 typedef uint8_t Type; 36 36 static const Type TypeInt32 = 1; 37 38 static const Type TypeMaybeNumber = 0x04; 39 static const Type TypeMaybeString = 0x08; 40 static const Type TypeMaybeNull = 0x10; 41 static const Type TypeMaybeBool = 0x20; 42 static const Type TypeMaybeOther = 0x40; 37 static const Type TypeMaybeNumber = 0x02; 38 static const Type TypeMaybeString = 0x04; 39 static const Type TypeMaybeNull = 0x08; 40 static const Type TypeMaybeBool = 0x10; 41 static const Type TypeMaybeOther = 0x20; 43 42 44 43 static const Type TypeBits = TypeMaybeNumber | TypeMaybeString | TypeMaybeNull | TypeMaybeBool | TypeMaybeOther; 45 44 45 public: 46 static const int numBitsNeeded = 6; 47 static_assert((TypeBits & ((1 << numBitsNeeded) - 1)) == TypeBits, "This is necessary for correctness."); 48 46 49 explicit ResultType(Type type) 47 : m_ type(type)50 : m_bits(type) 48 51 { 49 52 } 50 53 51 public:52 54 bool isInt32() const 53 55 { 54 return m_ type& TypeInt32;56 return m_bits & TypeInt32; 55 57 } 56 58 57 59 bool definitelyIsNumber() const 58 60 { 59 return (m_ type& TypeBits) == TypeMaybeNumber;61 return (m_bits & TypeBits) == TypeMaybeNumber; 60 62 } 61 63 62 64 bool definitelyIsString() const 63 65 { 64 return (m_ type& TypeBits) == TypeMaybeString;66 return (m_bits & TypeBits) == TypeMaybeString; 65 67 } 66 68 67 69 bool definitelyIsBoolean() const 68 70 { 69 return (m_ type& TypeBits) == TypeMaybeBool;71 return (m_bits & TypeBits) == TypeMaybeBool; 70 72 } 71 73 72 74 bool mightBeNumber() const 73 75 { 74 return m_ type& TypeMaybeNumber;76 return m_bits & TypeMaybeNumber; 75 77 } 76 78 … … 142 144 } 143 145 146 Type bits() const { return m_bits; } 147 144 148 private: 145 Type m_ type;149 Type m_bits; 146 150 }; 147 151 … … 153 157 // the entire struct is initialized. 154 158 m_u.i = 0; 155 m_u.rds.first = first.m_ type;156 m_u.rds.second = second.m_ type;159 m_u.rds.first = first.m_bits; 160 m_u.rds.second = second.m_bits; 157 161 } 158 162 -
trunk/Source/JavaScriptCore/runtime/CommonSlowPaths.cpp
r203499 r203537 27 27 #include "CommonSlowPaths.h" 28 28 29 #include "ArithProfile.h" 29 30 #include "ArrayConstructor.h" 30 31 #include "BuiltinNames.h" … … 360 361 361 362 #if ENABLE(DFG_JIT) 362 static void update ResultProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right)363 static void updateArithProfileForBinaryArithOp(ExecState* exec, Instruction* pc, JSValue result, JSValue left, JSValue right) 363 364 { 364 365 CodeBlock* codeBlock = exec->codeBlock(); 365 unsigned bytecodeOffset = codeBlock->bytecodeOffset(pc); 366 ResultProfile* profile = codeBlock->ensureResultProfile(bytecodeOffset); 366 ArithProfile& profile = codeBlock->arithProfileForPC(pc); 367 367 368 368 if (result.isNumber()) { 369 369 if (!result.isInt32()) { 370 370 if (left.isInt32() && right.isInt32()) 371 profile ->setObservedInt32Overflow();371 profile.setObservedInt32Overflow(); 372 372 373 373 double doubleVal = result.asNumber(); 374 374 if (!doubleVal && std::signbit(doubleVal)) 375 profile ->setObservedNegZeroDouble();375 profile.setObservedNegZeroDouble(); 376 376 else { 377 profile ->setObservedNonNegZeroDouble();377 profile.setObservedNonNegZeroDouble(); 378 378 379 379 // The Int52 overflow check here intentionally omits 1ll << 51 as a valid negative Int52 value. … … 383 383 int64_t int64Val = static_cast<int64_t>(std::abs(doubleVal)); 384 384 if (int64Val >= int52OverflowPoint) 385 profile ->setObservedInt52Overflow();385 profile.setObservedInt52Overflow(); 386 386 } 387 387 } 388 388 } else 389 profile ->setObservedNonNumber();389 profile.setObservedNonNumber(); 390 390 } 391 391 #else 392 static void update ResultProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { }392 static void updateArithProfileForBinaryArithOp(ExecState*, Instruction*, JSValue, JSValue, JSValue) { } 393 393 #endif 394 394 … … 407 407 JSValue v2 = OP_C(3).jsValue(); 408 408 JSValue result; 409 410 ArithProfile& arithProfile = exec->codeBlock()->arithProfileForPC(pc); 411 arithProfile.observeLHSAndRHS(v1, v2); 409 412 410 413 if (v1.isString() && !v2.isObject()) … … 416 419 417 420 RETURN_WITH_PROFILING(result, { 418 update ResultProfileForBinaryArithOp(exec, pc, result, v1, v2);421 updateArithProfileForBinaryArithOp(exec, pc, result, v1, v2); 419 422 }); 420 423 } … … 433 436 JSValue result = jsNumber(a * b); 434 437 RETURN_WITH_PROFILING(result, { 435 update ResultProfileForBinaryArithOp(exec, pc, result, left, right);438 updateArithProfileForBinaryArithOp(exec, pc, result, left, right); 436 439 }); 437 440 } … … 446 449 JSValue result = jsNumber(a - b); 447 450 RETURN_WITH_PROFILING(result, { 448 update ResultProfileForBinaryArithOp(exec, pc, result, left, right);451 updateArithProfileForBinaryArithOp(exec, pc, result, left, right); 449 452 }); 450 453 } … … 459 462 JSValue result = jsNumber(a / b); 460 463 RETURN_WITH_PROFILING(result, { 461 update ResultProfileForBinaryArithOp(exec, pc, result, left, right);464 updateArithProfileForBinaryArithOp(exec, pc, result, left, right); 462 465 }); 463 466 } -
trunk/Source/WebCore/ChangeLog
r203535 r203537 1 2016-07-21 Saam Barati <sbarati@apple.com> 2 3 op_add/ValueAdd should be an IC in all JIT tiers 4 https://bugs.webkit.org/show_bug.cgi?id=159649 5 6 Reviewed by Benjamin Poulain. 7 8 * ForwardingHeaders/jit/JITMathICForwards.h: Added. 9 1 10 2016-07-21 Chris Dumez <cdumez@apple.com> 2 11
Note: See TracChangeset
for help on using the changeset viewer.