Changeset 160796 in webkit
- Timestamp:
- Dec 18, 2013 2:50:40 PM (10 years ago)
- Location:
- trunk/Source
- Files:
-
- 4 added
- 31 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/Source/JavaScriptCore/CMakeLists.txt
r160588 r160796 173 173 dfg/DFGSpeculativeJIT64.cpp 174 174 dfg/DFGStackLayoutPhase.cpp 175 dfg/DFGStoreBarrierElisionPhase.cpp 175 176 dfg/DFGStrengthReductionPhase.cpp 176 177 dfg/DFGThunks.cpp … … 220 221 heap/WeakHandleOwner.cpp 221 222 heap/WeakSet.cpp 223 heap/WriteBarrierBuffer.cpp 222 224 heap/WriteBarrierSupport.cpp 223 225 -
trunk/Source/JavaScriptCore/ChangeLog
r160758 r160796 1 2013-12-18 Mark Hahnenberg <mhahnenberg@apple.com> 2 3 DFG should have a separate StoreBarrier node 4 https://bugs.webkit.org/show_bug.cgi?id=125530 5 6 Reviewed by Filip Pizlo. 7 8 This is in preparation for GenGC. We use a separate StoreBarrier node instead of making them implicitly 9 part of other nodes so that it's easier to run analyses on them, e.g. for the StoreBarrierElisionPhase. 10 They are inserted during the fixup phase. Initially they do not generate any code. 11 12 * CMakeLists.txt: 13 * GNUmakefile.list.am: 14 * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj: 15 * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters: 16 * JavaScriptCore.xcodeproj/project.pbxproj: 17 * dfg/DFGAbstractHeap.h: 18 * dfg/DFGAbstractInterpreter.h: 19 (JSC::DFG::AbstractInterpreter::isKnownNotCell): 20 * dfg/DFGAbstractInterpreterInlines.h: 21 (JSC::DFG::::executeEffects): 22 * dfg/DFGClobberize.h: 23 (JSC::DFG::clobberizeForAllocation): 24 (JSC::DFG::clobberize): 25 * dfg/DFGConstantFoldingPhase.cpp: 26 (JSC::DFG::ConstantFoldingPhase::foldConstants): Whenever we insert new nodes that require StoreBarriers, 27 we have to add those new StoreBarriers too. It's important to note that AllocatePropertyStorage and 28 ReallocatePropertyStorage nodes require their StoreBarriers to come after them since they allocate first, 29 which could cause a GC, and then store the resulting buffer into their JSCell, which requires the barrier. 30 If we ever require that write barriers occur before stores, we'll have to split these nodes into 31 AllocatePropertyStorage + StoreBarrier + PutPropertyStorage. 32 * dfg/DFGFixupPhase.cpp: 33 (JSC::DFG::FixupPhase::fixupNode): 34 (JSC::DFG::FixupPhase::insertStoreBarrier): 35 * dfg/DFGNode.h: 36 (JSC::DFG::Node::isStoreBarrier): 37 * dfg/DFGNodeType.h: 38 * dfg/DFGOSRExitCompiler32_64.cpp: 39 (JSC::DFG::OSRExitCompiler::compileExit): 40 * dfg/DFGOSRExitCompiler64.cpp: 41 (JSC::DFG::OSRExitCompiler::compileExit): 42 * dfg/DFGPlan.cpp: 43 (JSC::DFG::Plan::compileInThreadImpl): 44 * dfg/DFGPredictionPropagationPhase.cpp: 45 (JSC::DFG::PredictionPropagationPhase::propagate): 46 * dfg/DFGSafeToExecute.h: 47 (JSC::DFG::safeToExecute): 48 * dfg/DFGSpeculativeJIT.cpp: 49 (JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage): 50 (JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage): 51 (JSC::DFG::SpeculativeJIT::compileStoreBarrier): 52 (JSC::DFG::SpeculativeJIT::genericWriteBarrier): The fast path write barrier check. It loads the 53 byte that contains the mark bit of the object. 54 (JSC::DFG::SpeculativeJIT::storeToWriteBarrierBuffer): If the fast path check fails we try to store the 55 cell in the WriteBarrierBuffer so as to avoid frequently flushing all registers in order to make a C call. 56 (JSC::DFG::SpeculativeJIT::writeBarrier): 57 (JSC::DFG::SpeculativeJIT::osrWriteBarrier): More barebones version of the write barrier to be executed 58 during an OSR exit into baseline code. We must do this so that the baseline JIT object and array profiles 59 are properly cleared during GC. 60 * dfg/DFGSpeculativeJIT.h: 61 (JSC::DFG::SpeculativeJIT::callOperation): 62 * dfg/DFGSpeculativeJIT32_64.cpp: 63 (JSC::DFG::SpeculativeJIT::cachedPutById): 64 (JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): 65 (JSC::DFG::SpeculativeJIT::compile): 66 (JSC::DFG::SpeculativeJIT::writeBarrier): 67 * dfg/DFGSpeculativeJIT64.cpp: 68 (JSC::DFG::SpeculativeJIT::cachedPutById): 69 (JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): 70 (JSC::DFG::SpeculativeJIT::compile): 71 (JSC::DFG::SpeculativeJIT::writeBarrier): 72 * dfg/DFGStoreBarrierElisionPhase.cpp: Added. New DFG phase that does block-local elision of redundant 73 StoreBarriers. Every time a StoreBarrier on a particular object is executed, a bit is set indicating that 74 that object doesn't need any more StoreBarriers. 75 (JSC::DFG::StoreBarrierElisionPhase::StoreBarrierElisionPhase): 76 (JSC::DFG::StoreBarrierElisionPhase::couldCauseGC): Nodes that could cause a GC reset the bits for all of the 77 objects known in the current block. 78 (JSC::DFG::StoreBarrierElisionPhase::allocatesFreshObject): A node that creates a new object automatically 79 sets the bit for that object since if a GC occurred as the result of that object's allocation then that 80 object would not need a barrier since it would be guaranteed to be a young generation object until the 81 next GC point. 82 (JSC::DFG::StoreBarrierElisionPhase::noticeFreshObject): 83 (JSC::DFG::StoreBarrierElisionPhase::getBaseOfStore): 84 (JSC::DFG::StoreBarrierElisionPhase::shouldBeElided): 85 (JSC::DFG::StoreBarrierElisionPhase::elideBarrier): 86 (JSC::DFG::StoreBarrierElisionPhase::handleNode): 87 (JSC::DFG::StoreBarrierElisionPhase::handleBlock): 88 (JSC::DFG::StoreBarrierElisionPhase::run): 89 (JSC::DFG::performStoreBarrierElision): 90 * dfg/DFGStoreBarrierElisionPhase.h: Added. 91 * heap/Heap.cpp: 92 (JSC::Heap::Heap): 93 (JSC::Heap::flushWriteBarrierBuffer): 94 * heap/Heap.h: 95 (JSC::Heap::writeBarrier): 96 * heap/MarkedBlock.h: 97 (JSC::MarkedBlock::offsetOfMarks): 98 * heap/WriteBarrierBuffer.cpp: Added. The WriteBarrierBuffer buffers a set of JSCells that are awaiting 99 a pending WriteBarrier. This buffer is used by the DFG to avoid the overhead of calling out to C repeatedly 100 to invoke a write barrier on a single JSCell. Instead the DFG has inline code to fill the WriteBarrier buffer 101 until its full, and then to call out to C to flush it. The WriteBarrierBuffer will also be flushed prior to 102 each EdenCollection. 103 (JSC::WriteBarrierBuffer::WriteBarrierBuffer): 104 (JSC::WriteBarrierBuffer::~WriteBarrierBuffer): 105 (JSC::WriteBarrierBuffer::flush): 106 (JSC::WriteBarrierBuffer::reset): 107 (JSC::WriteBarrierBuffer::add): 108 * heap/WriteBarrierBuffer.h: Added. 109 (JSC::WriteBarrierBuffer::currentIndexOffset): 110 (JSC::WriteBarrierBuffer::capacityOffset): 111 (JSC::WriteBarrierBuffer::bufferOffset): 112 * jit/JITOperations.cpp: 113 * jit/JITOperations.h: 114 * runtime/VM.h: 115 1 116 2013-12-18 Carlos Garcia Campos <cgarcia@igalia.com> 2 117 -
trunk/Source/JavaScriptCore/GNUmakefile.list.am
r160557 r160796 388 388 Source/JavaScriptCore/dfg/DFGStackLayoutPhase.cpp \ 389 389 Source/JavaScriptCore/dfg/DFGStackLayoutPhase.h \ 390 Source/JavaScriptCore/dfg/DFGStoreBarrierElisionPhase.cpp \ 391 Source/JavaScriptCore/dfg/DFGStoreBarrierElisionPhase.h \ 390 392 Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.cpp \ 391 393 Source/JavaScriptCore/dfg/DFGStrengthReductionPhase.h \ … … 577 579 Source/JavaScriptCore/heap/WeakSetInlines.h \ 578 580 Source/JavaScriptCore/heap/WeakReferenceHarvester.h \ 581 Source/JavaScriptCore/heap/WriteBarrierBuffer.cpp \ 582 Source/JavaScriptCore/heap/WriteBarrierBuffer.h \ 579 583 Source/JavaScriptCore/heap/WriteBarrierSupport.cpp \ 580 584 Source/JavaScriptCore/heap/WriteBarrierSupport.h \ -
trunk/Source/JavaScriptCore/JavaScriptCore.vcxproj/JavaScriptCore.vcxproj
r160655 r160796 429 429 <ClCompile Include="..\dfg\DFGSSALoweringPhase.cpp" /> 430 430 <ClCompile Include="..\dfg\DFGStackLayoutPhase.cpp" /> 431 <ClCompile Include="..\dfg\DFGStoreBarrierElisionPhase.cpp" /> 431 432 <ClCompile Include="..\dfg\DFGStrengthReductionPhase.cpp" /> 432 433 <ClCompile Include="..\dfg\DFGThunks.cpp" /> … … 472 473 <ClCompile Include="..\heap\WeakHandleOwner.cpp" /> 473 474 <ClCompile Include="..\heap\WeakSet.cpp" /> 475 <ClCompile Include="..\heap\WriteBarrierBuffer.cpp" /> 474 476 <ClCompile Include="..\heap\WriteBarrierSupport.cpp" /> 475 477 <ClCompile Include="..\inspector\InspectorAgentRegistry.cpp" /> … … 949 951 <ClInclude Include="..\dfg\DFGSSALoweringPhase.h" /> 950 952 <ClInclude Include="..\dfg\DFGStackLayoutPhase.h" /> 953 <ClInclude Include="..\dfg\DFGStoreBarrierElisionPhase.h" /> 951 954 <ClInclude Include="..\dfg\DFGStrengthReductionPhase.h" /> 952 955 <ClInclude Include="..\dfg\DFGStructureAbstractValue.h" /> … … 1027 1030 <ClInclude Include="..\heap\WeakSet.h" /> 1028 1031 <ClInclude Include="..\heap\WeakSetInlines.h" /> 1032 <ClInclude Include="..\heap\WriteBarrierBuffer.h" /> 1029 1033 <ClInclude Include="..\heap\WriteBarrierSupport.h" /> 1030 1034 <ClInclude Include="..\inspector\InspectorAgentBase.h" /> -
trunk/Source/JavaScriptCore/JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters
r160557 r160796 280 280 <Filter>heap</Filter> 281 281 </ClCompile> 282 <ClCompile Include="..\heap\WriteBarrierBuffer.cpp"> 283 <Filter>heap</Filter> 284 </ClCompile> 282 285 <ClCompile Include="..\heap\WriteBarrierSupport.cpp"> 283 286 <Filter>heap</Filter> … … 1148 1151 </ClCompile> 1149 1152 <ClCompile Include="..\dfg\DFGStackLayoutPhase.cpp"> 1153 <Filter>dfg</Filter> 1154 </ClCompile> 1155 <ClCompile Include="..\dfg\DFGStoreBarrierElisionPhase.cpp"> 1150 1156 <Filter>dfg</Filter> 1151 1157 </ClCompile> … … 1755 1761 <Filter>heap</Filter> 1756 1762 </ClInclude> 1763 <ClInclude Include="..\heap\WriteBarrierBuffer.h"> 1764 <Filter>heap</Filter> 1765 </ClInclude> 1757 1766 <ClInclude Include="..\heap\WriteBarrierSupport.h"> 1758 1767 <Filter>heap</Filter> … … 3056 3065 </ClInclude> 3057 3066 <ClInclude Include="..\dfg\DFGStackLayoutPhase.h"> 3067 <Filter>dfg</Filter> 3068 </ClInclude> 3069 <ClInclude Include="..\dfg\DFGStoreBarrierElisionPhase.h"> 3058 3070 <Filter>dfg</Filter> 3059 3071 </ClInclude> -
trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
r160594 r160796 719 719 2A2825D018341F2D0087FBA9 /* DelayedReleaseScope.h in Headers */ = {isa = PBXBuildFile; fileRef = 2A2825CF18341F2D0087FBA9 /* DelayedReleaseScope.h */; }; 720 720 2A48D1911772365B00C65A5F /* APICallbackFunction.h in Headers */ = {isa = PBXBuildFile; fileRef = C211B574176A224D000E2A23 /* APICallbackFunction.h */; }; 721 2A4EC90B1860D6C20094F782 /* WriteBarrierBuffer.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2A4EC9091860D6C20094F782 /* WriteBarrierBuffer.cpp */; }; 722 2A4EC90C1860D6C20094F782 /* WriteBarrierBuffer.h in Headers */ = {isa = PBXBuildFile; fileRef = 2A4EC90A1860D6C20094F782 /* WriteBarrierBuffer.h */; settings = {ATTRIBUTES = (Private, ); }; }; 721 723 2A6F462617E959CE00C45C98 /* HeapOperation.h in Headers */ = {isa = PBXBuildFile; fileRef = 2A6F462517E959CE00C45C98 /* HeapOperation.h */; settings = {ATTRIBUTES = (Private, ); }; }; 722 724 2A7A58EF1808A4C40020BDF7 /* DeferGC.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2A7A58EE1808A4C40020BDF7 /* DeferGC.cpp */; }; 723 725 2AAD964A18569417001F93BE /* RecursiveAllocationScope.h in Headers */ = {isa = PBXBuildFile; fileRef = 2AAD964918569417001F93BE /* RecursiveAllocationScope.h */; }; 726 2ACCF3DE185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2ACCF3DC185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp */; }; 727 2ACCF3DF185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 2ACCF3DD185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h */; }; 724 728 2AD8932B17E3868F00668276 /* HeapIterationScope.h in Headers */ = {isa = PBXBuildFile; fileRef = 2AD8932917E3868F00668276 /* HeapIterationScope.h */; }; 725 729 371D842D17C98B6E00ECF994 /* libz.dylib in Frameworks */ = {isa = PBXBuildFile; fileRef = 371D842C17C98B6E00ECF994 /* libz.dylib */; }; … … 2022 2026 2600B5A5152BAAA70091EE5F /* JSStringJoiner.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSStringJoiner.h; sourceTree = "<group>"; }; 2023 2027 2A2825CF18341F2D0087FBA9 /* DelayedReleaseScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DelayedReleaseScope.h; sourceTree = "<group>"; }; 2028 2A4EC9091860D6C20094F782 /* WriteBarrierBuffer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WriteBarrierBuffer.cpp; sourceTree = "<group>"; }; 2029 2A4EC90A1860D6C20094F782 /* WriteBarrierBuffer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WriteBarrierBuffer.h; sourceTree = "<group>"; }; 2024 2030 2A6F462517E959CE00C45C98 /* HeapOperation.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HeapOperation.h; sourceTree = "<group>"; }; 2025 2031 2A7A58EE1808A4C40020BDF7 /* DeferGC.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = DeferGC.cpp; sourceTree = "<group>"; }; 2026 2032 2AAD964918569417001F93BE /* RecursiveAllocationScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RecursiveAllocationScope.h; sourceTree = "<group>"; }; 2033 2ACCF3DC185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStoreBarrierElisionPhase.cpp; path = dfg/DFGStoreBarrierElisionPhase.cpp; sourceTree = "<group>"; }; 2034 2ACCF3DD185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStoreBarrierElisionPhase.h; path = dfg/DFGStoreBarrierElisionPhase.h; sourceTree = "<group>"; }; 2027 2035 2AD8932917E3868F00668276 /* HeapIterationScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HeapIterationScope.h; sourceTree = "<group>"; }; 2028 2036 371D842C17C98B6E00ECF994 /* libz.dylib */ = {isa = PBXFileReference; lastKnownFileType = "compiled.mach-o.dylib"; name = libz.dylib; path = usr/lib/libz.dylib; sourceTree = SDKROOT; }; … … 3112 3120 isa = PBXGroup; 3113 3121 children = ( 3122 2A4EC9091860D6C20094F782 /* WriteBarrierBuffer.cpp */, 3123 2A4EC90A1860D6C20094F782 /* WriteBarrierBuffer.h */, 3114 3124 14816E19154CC56C00B8054C /* BlockAllocator.cpp */, 3115 3125 14816E1A154CC56C00B8054C /* BlockAllocator.h */, … … 3848 3858 isa = PBXGroup; 3849 3859 children = ( 3860 2ACCF3DC185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp */, 3861 2ACCF3DD185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h */, 3850 3862 A77A423617A0BBFD00A8DB81 /* DFGAbstractHeap.cpp */, 3851 3863 A77A423717A0BBFD00A8DB81 /* DFGAbstractHeap.h */, … … 4362 4374 86ADD1450FDDEA980006EEC2 /* ARMv7Assembler.h in Headers */, 4363 4375 65C0285D1717966800351E35 /* ARMv7DOpcode.h in Headers */, 4376 2A4EC90C1860D6C20094F782 /* WriteBarrierBuffer.h in Headers */, 4364 4377 A532439218569709002ED692 /* CodeGeneratorInspector.py in Headers */, 4365 4378 A532439318569709002ED692 /* CodeGeneratorInspectorStrings.py in Headers */, … … 5051 5064 A7CA3AE417DA41AE006538AF /* WeakMapConstructor.h in Headers */, 5052 5065 A7CA3AEC17DA5168006538AF /* WeakMapData.h in Headers */, 5066 2ACCF3DF185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h in Headers */, 5053 5067 A55D93AC18514F7900400DED /* InspectorTypeBuilder.h in Headers */, 5054 5068 A7CA3AE617DA41AE006538AF /* WeakMapPrototype.h in Headers */, … … 5706 5720 A78853F917972629001440E4 /* IntendedStructureChain.cpp in Sources */, 5707 5721 147F39CF107EC37600427A48 /* InternalFunction.cpp in Sources */, 5722 2ACCF3DE185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp in Sources */, 5708 5723 1429D7D40ED2128200B89619 /* Interpreter.cpp in Sources */, 5709 5724 1429D92F0ED22D7000B89619 /* JIT.cpp in Sources */, … … 5718 5733 BCDD51EB0FB8DF74004A8BDC /* JITOpcodes.cpp in Sources */, 5719 5734 A71236E51195F33C00BD2174 /* JITOpcodes32_64.cpp in Sources */, 5735 2A4EC90B1860D6C20094F782 /* WriteBarrierBuffer.cpp in Sources */, 5720 5736 0F24E54C17EE274900ABB217 /* JITOperations.cpp in Sources */, 5721 5737 86CC85C40EE7A89400288682 /* JITPropertyAccess.cpp in Sources */, -
trunk/Source/JavaScriptCore/dfg/DFGAbstractHeap.h
r158304 r160796 69 69 macro(TypedArrayProperties) \ 70 70 macro(GCState) \ 71 macro(BarrierState) \ 71 72 macro(RegExpState) \ 72 73 macro(InternalState) \ -
trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h
r160347 r160796 1589 1589 break; 1590 1590 1591 case ConditionalStoreBarrier: { 1592 if (!needsTypeCheck(node->child2().node(), ~SpecCell)) 1593 m_state.setFoundConstants(true); 1594 filter(node->child1(), SpecCell); 1595 break; 1596 } 1597 1598 case StoreBarrier: { 1599 filter(node->child1(), SpecCell); 1600 break; 1601 } 1602 1603 case StoreBarrierWithNullCheck: { 1604 break; 1605 } 1606 1591 1607 case CheckTierUpAndOSREnter: 1592 1608 case LoopHint: -
trunk/Source/JavaScriptCore/dfg/DFGClobberize.h
r160587 r160796 36 36 37 37 namespace JSC { namespace DFG { 38 39 template<typename ReadFunctor, typename WriteFunctor> 40 void clobberizeForAllocation(ReadFunctor& read, WriteFunctor& write) 41 { 42 read(GCState); 43 read(BarrierState); 44 write(GCState); 45 write(BarrierState); 46 } 38 47 39 48 template<typename ReadFunctor, typename WriteFunctor> … … 161 170 case CreateActivation: 162 171 case CreateArguments: 172 clobberizeForAllocation(read, write); 163 173 write(SideState); 164 174 write(Watchpoint_fire); 165 read(GCState);166 write(GCState);167 175 return; 168 176 … … 174 182 case CreateThis: 175 183 read(MiscFields); 176 read(GCState); 177 write(GCState); 184 clobberizeForAllocation(read, write); 178 185 return; 179 186 … … 436 443 case AllocatePropertyStorage: 437 444 write(JSObject_butterfly); 438 read(GCState); 439 write(GCState); 445 clobberizeForAllocation(read, write); 440 446 return; 441 447 … … 443 449 read(JSObject_butterfly); 444 450 write(JSObject_butterfly); 445 read(GCState); 446 write(GCState); 451 clobberizeForAllocation(read, write); 447 452 return; 448 453 … … 457 462 write(JSCell_structure); 458 463 write(JSObject_butterfly); 459 read(GCState); 460 write(GCState); 464 clobberizeForAllocation(read, write); 461 465 return; 462 466 … … 545 549 case NewFunction: 546 550 case NewFunctionExpression: 547 read(GCState); 548 write(GCState); 551 clobberizeForAllocation(read, write); 549 552 return; 550 553 551 554 case NewTypedArray: 555 clobberizeForAllocation(read, write); 552 556 switch (node->child1().useKind()) { 553 557 case Int32Use: 554 read(GCState);555 write(GCState);556 558 return; 557 559 case UntypedUse: … … 642 644 case ThrowReferenceError: 643 645 write(SideState); 644 read(GCState); 645 write(GCState); 646 clobberizeForAllocation(read, write); 646 647 return; 647 648 … … 650 651 read(InternalState); 651 652 write(InternalState); 653 return; 654 655 case StoreBarrier: 656 case ConditionalStoreBarrier: 657 case StoreBarrierWithNullCheck: 658 read(BarrierState); 659 write(BarrierState); 652 660 return; 653 661 -
trunk/Source/JavaScriptCore/dfg/DFGConstantFoldingPhase.cpp
r160347 r160796 301 301 ASSERT(status.newStructure()->outOfLineCapacity()); 302 302 ASSERT(!isInlineOffset(status.offset())); 303 propertyStorage = Edge(m_insertionSet.insertNode(303 Node* allocatePropertyStorage = m_insertionSet.insertNode( 304 304 indexInBlock, SpecNone, AllocatePropertyStorage, 305 codeOrigin, OpInfo(transitionData), childEdge)); 305 codeOrigin, OpInfo(transitionData), childEdge); 306 m_insertionSet.insertNode(indexInBlock, SpecNone, StoreBarrier, codeOrigin, Edge(node->child1().node(), KnownCellUse)); 307 propertyStorage = Edge(allocatePropertyStorage); 306 308 } else { 307 309 ASSERT(structure->outOfLineCapacity()); … … 309 311 ASSERT(!isInlineOffset(status.offset())); 310 312 311 propertyStorage = Edge(m_insertionSet.insertNode(313 Node* reallocatePropertyStorage = m_insertionSet.insertNode( 312 314 indexInBlock, SpecNone, ReallocatePropertyStorage, codeOrigin, 313 315 OpInfo(transitionData), childEdge, 314 316 Edge(m_insertionSet.insertNode( 315 indexInBlock, SpecNone, GetButterfly, codeOrigin, childEdge)))); 317 indexInBlock, SpecNone, GetButterfly, codeOrigin, childEdge))); 318 m_insertionSet.insertNode(indexInBlock, SpecNone, StoreBarrier, codeOrigin, Edge(node->child1().node(), KnownCellUse)); 319 propertyStorage = Edge(reallocatePropertyStorage); 316 320 } 317 321 318 322 if (status.isSimpleTransition()) { 319 m_insertionSet.insertNode(320 indexInBlock, SpecNone, PutStructure, codeOrigin,321 OpInfo(transitionData), childEdge);322 } 323 323 Node* putStructure = m_graph.addNode(SpecNone, PutStructure, codeOrigin, OpInfo(transitionData), childEdge); 324 m_insertionSet.insertNode(indexInBlock, SpecNone, StoreBarrier, codeOrigin, Edge(node->child1().node(), KnownCellUse)); 325 m_insertionSet.insert(indexInBlock, putStructure); 326 } 327 324 328 node->convertToPutByOffset(m_graph.m_storageAccessData.size(), propertyStorage); 329 m_insertionSet.insertNode(indexInBlock, SpecNone, ConditionalStoreBarrier, codeOrigin, 330 Edge(node->child2().node(), KnownCellUse), Edge(node->child3().node(), UntypedUse)); 325 331 326 332 StorageAccessData storageAccessData; … … 330 336 break; 331 337 } 332 338 339 case ConditionalStoreBarrier: { 340 if (!m_interpreter.needsTypeCheck(node->child2().node(), ~SpecCell)) { 341 node->convertToPhantom(); 342 eliminated = true; 343 } 344 break; 345 } 346 347 case StoreBarrier: 348 case StoreBarrierWithNullCheck: { 349 break; 350 } 351 333 352 default: 334 353 break; -
trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp
r160587 r160796 550 550 fixEdge<NumberUse>(child3); 551 551 break; 552 case Array::Contiguous: 553 case Array::ArrayStorage: 554 case Array::SlowPutArrayStorage: 555 case Array::Arguments: 556 fixEdge<KnownCellUse>(child1); 557 fixEdge<Int32Use>(child2); 558 insertStoreBarrier(m_indexInBlock, child1, child3); 559 break; 552 560 default: 553 561 fixEdge<KnownCellUse>(child1); … … 582 590 case Array::Double: 583 591 fixEdge<RealNumberUse>(node->child2()); 592 break; 593 case Array::Contiguous: 594 case Array::ArrayStorage: 595 insertStoreBarrier(m_indexInBlock, node->child1(), node->child2()); 584 596 break; 585 597 default: … … 768 780 } 769 781 782 case PutStructure: { 783 fixEdge<KnownCellUse>(node->child1()); 784 insertStoreBarrier(m_indexInBlock, node->child1()); 785 break; 786 } 787 788 case PutClosureVar: { 789 fixEdge<KnownCellUse>(node->child1()); 790 insertStoreBarrier(m_indexInBlock, node->child1(), node->child3()); 791 break; 792 } 793 770 794 case GetClosureRegisters: 771 case PutClosureVar:772 795 case SkipTopScope: 773 796 case SkipScope: 774 case PutStructure:775 case AllocatePropertyStorage:776 case ReallocatePropertyStorage:777 797 case GetScope: { 778 798 fixEdge<KnownCellUse>(node->child1()); … … 780 800 } 781 801 802 case AllocatePropertyStorage: 803 case ReallocatePropertyStorage: { 804 fixEdge<KnownCellUse>(node->child1()); 805 insertStoreBarrier(m_indexInBlock + 1, node->child1()); 806 break; 807 } 808 782 809 case GetById: 783 810 case GetByIdFlush: { … … 801 828 } 802 829 830 case PutById: 831 case PutByIdDirect: { 832 fixEdge<CellUse>(node->child1()); 833 insertStoreBarrier(m_indexInBlock, node->child1(), node->child2()); 834 break; 835 } 836 803 837 case CheckExecutable: 804 838 case CheckStructure: 805 839 case StructureTransitionWatchpoint: 806 840 case CheckFunction: 807 case PutById:808 case PutByIdDirect:809 841 case CheckHasInstance: 810 842 case CreateThis: … … 833 865 fixEdge<KnownCellUse>(node->child1()); 834 866 fixEdge<KnownCellUse>(node->child2()); 867 insertStoreBarrier(m_indexInBlock, node->child2(), node->child3()); 835 868 break; 836 869 } … … 892 925 RELEASE_ASSERT_NOT_REACHED(); 893 926 break; 927 928 case PutGlobalVar: { 929 Node* globalObjectNode = m_insertionSet.insertNode(m_indexInBlock, SpecNone, WeakJSConstant, node->codeOrigin, 930 OpInfo(m_graph.globalObjectFor(node->codeOrigin))); 931 Node* barrierNode = m_graph.addNode(SpecNone, ConditionalStoreBarrier, m_currentNode->codeOrigin, 932 Edge(globalObjectNode, KnownCellUse), Edge(node->child1().node(), UntypedUse)); 933 fixupNode(barrierNode); 934 m_insertionSet.insert(m_indexInBlock, barrierNode); 935 break; 936 } 937 938 case TearOffActivation: { 939 Node* barrierNode = m_graph.addNode(SpecNone, StoreBarrierWithNullCheck, m_currentNode->codeOrigin, 940 Edge(node->child1().node(), UntypedUse)); 941 fixupNode(barrierNode); 942 m_insertionSet.insert(m_indexInBlock, barrierNode); 943 break; 944 } 894 945 895 946 case IsString: … … 915 966 case GetClosureVar: 916 967 case GetGlobalVar: 917 case PutGlobalVar:918 968 case NotifyWrite: 919 969 case VariableWatchpoint: … … 932 982 case IsFunction: 933 983 case CreateActivation: 934 case TearOffActivation:935 984 case CreateArguments: 936 985 case PhantomArguments: … … 952 1001 case ExtractOSREntryLocal: 953 1002 case LoopHint: 1003 case StoreBarrier: 1004 case ConditionalStoreBarrier: 1005 case StoreBarrierWithNullCheck: 954 1006 case FunctionReentryWatchpoint: 955 1007 case TypedArrayWatchpoint: … … 1504 1556 } 1505 1557 1558 void insertStoreBarrier(unsigned indexInBlock, Edge child1, Edge child2 = Edge()) 1559 { 1560 Node* barrierNode; 1561 if (!child2) 1562 barrierNode = m_graph.addNode(SpecNone, StoreBarrier, m_currentNode->codeOrigin, Edge(child1.node(), child1.useKind())); 1563 else { 1564 barrierNode = m_graph.addNode(SpecNone, ConditionalStoreBarrier, m_currentNode->codeOrigin, 1565 Edge(child1.node(), child1.useKind()), Edge(child2.node(), child2.useKind())); 1566 } 1567 fixupNode(barrierNode); 1568 m_insertionSet.insert(indexInBlock, barrierNode); 1569 } 1570 1506 1571 void fixIntEdge(Edge& edge) 1507 1572 { -
trunk/Source/JavaScriptCore/dfg/DFGNode.h
r160295 r160796 607 607 return bitwise_cast<Node*>(m_opInfo); 608 608 } 609 609 610 bool isStoreBarrier() 611 { 612 switch (op()) { 613 case StoreBarrier: 614 case ConditionalStoreBarrier: 615 case StoreBarrierWithNullCheck: 616 return true; 617 default: 618 return false; 619 } 620 } 621 610 622 bool hasIdentifier() 611 623 { … … 1556 1568 Node* replacement; 1557 1569 BasicBlock* owner; 1570 bool needsBarrier; 1558 1571 } misc; 1559 1572 }; -
trunk/Source/JavaScriptCore/dfg/DFGNodeType.h
r160587 r160796 292 292 /* baseline JIT to redo the watchdog timer check, and service the timer. */ \ 293 293 macro(CheckWatchdogTimer, NodeMustGenerate) \ 294 /* Write barriers ! */\ 295 macro(StoreBarrier, NodeMustGenerate) \ 296 macro(ConditionalStoreBarrier, NodeMustGenerate) \ 297 macro(StoreBarrierWithNullCheck, NodeMustGenerate) \ 294 298 295 299 // This enum generates a monotonically increasing id for all Node types, -
trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompiler32_64.cpp
r160587 r160796 31 31 #include "DFGOperations.h" 32 32 #include "DFGOSRExitCompilerCommon.h" 33 #include "DFGSpeculativeJIT.h" 33 34 #include "Operations.h" 34 35 #include <wtf/DataLog.h> … … 449 450 } 450 451 } 451 452 // 11) And finish. 453 452 453 #if ENABLE(GGC) 454 // 11) Write barrier the owner executable because we're jumping into a different block. 455 for (CodeOrigin codeOrigin = exit.m_codeOrigin; ; codeOrigin = codeOrigin.inlineCallFrame->caller) { 456 CodeBlock* baselineCodeBlock = m_jit.baselineCodeBlockFor(codeOrigin); 457 m_jit.move(AssemblyHelpers::TrustedImmPtr(baselineCodeBlock->ownerExecutable()), GPRInfo::nonArgGPR0); 458 SpeculativeJIT::osrWriteBarrier(m_jit, GPRInfo::nonArgGPR0, GPRInfo::nonArgGPR1, GPRInfo::nonArgGPR2); 459 if (!codeOrigin.inlineCallFrame) 460 break; 461 } 462 #endif 463 464 // 12) And finish. 454 465 adjustAndJumpToTarget(m_jit, exit); 455 466 } -
trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompiler64.cpp
r160587 r160796 31 31 #include "DFGOperations.h" 32 32 #include "DFGOSRExitCompilerCommon.h" 33 #include "DFGSpeculativeJIT.h" 33 34 #include "Operations.h" 34 35 #include "VirtualRegister.h" … … 403 404 } 404 405 } 405 406 // 11) And finish. 407 406 407 #if ENABLE(GGC) 408 // 11) Write barrier the owner executable because we're jumping into a different block. 409 for (CodeOrigin codeOrigin = exit.m_codeOrigin; ; codeOrigin = codeOrigin.inlineCallFrame->caller) { 410 CodeBlock* baselineCodeBlock = m_jit.baselineCodeBlockFor(codeOrigin); 411 m_jit.move(AssemblyHelpers::TrustedImmPtr(baselineCodeBlock->ownerExecutable()), GPRInfo::nonArgGPR0); 412 SpeculativeJIT::osrWriteBarrier(m_jit, GPRInfo::nonArgGPR0, GPRInfo::nonArgGPR1, GPRInfo::nonArgGPR2); 413 if (!codeOrigin.inlineCallFrame) 414 break; 415 } 416 #endif 417 418 // 12) And finish. 408 419 adjustAndJumpToTarget(m_jit, exit); 409 420 } -
trunk/Source/JavaScriptCore/dfg/DFGPlan.cpp
r160347 r160796 55 55 #include "DFGSSALoweringPhase.h" 56 56 #include "DFGStackLayoutPhase.h" 57 #include "DFGStoreBarrierElisionPhase.h" 57 58 #include "DFGStrengthReductionPhase.h" 58 59 #include "DFGTierUpCheckInjectionPhase.h" … … 224 225 dfg.m_fixpointState = FixpointConverged; 225 226 227 performStoreBarrierElision(dfg); 226 228 performStoreElimination(dfg); 227 229 -
trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp
r160587 r160796 547 547 #ifndef NDEBUG 548 548 // These get ignored because they don't return anything. 549 case StoreBarrier: 550 case ConditionalStoreBarrier: 551 case StoreBarrierWithNullCheck: 549 552 case PutByValDirect: 550 553 case PutByVal: -
trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h
r160347 r160796 242 242 case Int52ToDouble: 243 243 case Int52ToValue: 244 case StoreBarrier: 245 case ConditionalStoreBarrier: 246 case StoreBarrierWithNullCheck: 244 247 case InvalidationPoint: 245 248 case NotifyWrite: -
trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
r160587 r160796 38 38 #include "JSCJSValueInlines.h" 39 39 #include "LinkBuffer.h" 40 #include "WriteBarrierBuffer.h" 40 41 #include <wtf/MathExtras.h> 41 42 … … 908 909 use(child3); 909 910 } 910 }911 912 void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, GPRReg valueGPR, Edge valueUse, WriteBarrierUseKind useKind, GPRReg scratch1, GPRReg scratch2)913 {914 UNUSED_PARAM(ownerGPR);915 UNUSED_PARAM(valueGPR);916 UNUSED_PARAM(scratch1);917 UNUSED_PARAM(scratch2);918 UNUSED_PARAM(useKind);919 920 if (isKnownNotCell(valueUse.node()))921 return;922 923 #if ENABLE(WRITE_BARRIER_PROFILING)924 JITCompiler::emitCount(m_jit, WriteBarrierCounters::jitCounterFor(useKind));925 #endif926 }927 928 void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, JSCell* value, WriteBarrierUseKind useKind, GPRReg scratch1, GPRReg scratch2)929 {930 UNUSED_PARAM(ownerGPR);931 UNUSED_PARAM(value);932 UNUSED_PARAM(scratch1);933 UNUSED_PARAM(scratch2);934 UNUSED_PARAM(useKind);935 936 if (Heap::isMarked(value))937 return;938 939 #if ENABLE(WRITE_BARRIER_PROFILING)940 JITCompiler::emitCount(m_jit, WriteBarrierCounters::jitCounterFor(useKind));941 #endif942 }943 944 void SpeculativeJIT::writeBarrier(JSCell* owner, GPRReg valueGPR, Edge valueUse, WriteBarrierUseKind useKind, GPRReg scratch)945 {946 UNUSED_PARAM(owner);947 UNUSED_PARAM(valueGPR);948 UNUSED_PARAM(scratch);949 UNUSED_PARAM(useKind);950 951 if (isKnownNotCell(valueUse.node()))952 return;953 954 #if ENABLE(WRITE_BARRIER_PROFILING)955 JITCompiler::emitCount(m_jit, WriteBarrierCounters::jitCounterFor(useKind));956 #endif957 911 } 958 912 … … 4330 4284 4331 4285 SpeculateCellOperand base(this, node->child1()); 4332 GPRTemporary scratch (this);4286 GPRTemporary scratch1(this); 4333 4287 4334 4288 GPRReg baseGPR = base.gpr(); 4335 GPRReg scratchGPR = scratch.gpr();4289 GPRReg scratchGPR1 = scratch1.gpr(); 4336 4290 4337 4291 ASSERT(!node->structureTransitionData().previousStructure->outOfLineCapacity()); … … 4340 4294 JITCompiler::Jump slowPath = 4341 4295 emitAllocateBasicStorage( 4342 TrustedImm32(initialOutOfLineCapacity * sizeof(JSValue)), scratchGPR );4343 4344 m_jit.addPtr(JITCompiler::TrustedImm32(sizeof(IndexingHeader)), scratchGPR );4296 TrustedImm32(initialOutOfLineCapacity * sizeof(JSValue)), scratchGPR1); 4297 4298 m_jit.addPtr(JITCompiler::TrustedImm32(sizeof(IndexingHeader)), scratchGPR1); 4345 4299 4346 4300 addSlowPathGenerator( 4347 slowPathCall(slowPath, this, operationAllocatePropertyStorageWithInitialCapacity, scratchGPR ));4348 4349 m_jit.storePtr(scratchGPR , JITCompiler::Address(baseGPR, JSObject::butterflyOffset()));4350 4351 storageResult(scratchGPR , node);4301 slowPathCall(slowPath, this, operationAllocatePropertyStorageWithInitialCapacity, scratchGPR1)); 4302 4303 m_jit.storePtr(scratchGPR1, JITCompiler::Address(baseGPR, JSObject::butterflyOffset())); 4304 4305 storageResult(scratchGPR1, node); 4352 4306 } 4353 4307 … … 4368 4322 callOperation(operationReallocateButterflyToGrowPropertyStorage, result.gpr(), baseGPR, newSize / sizeof(JSValue)); 4369 4323 4324 MacroAssembler::Jump notNull = m_jit.branchTestPtr(MacroAssembler::NonZero, result.gpr()); 4325 m_jit.breakpoint(); 4326 notNull.link(&m_jit); 4327 4370 4328 storageResult(result.gpr(), node); 4371 4329 return; … … 4383 4341 4384 4342 JITCompiler::Jump slowPath = 4385 emitAllocateBasicStorage(TrustedImm32(newSize), scratchGPR 2);4386 4387 m_jit.addPtr(JITCompiler::TrustedImm32(sizeof(IndexingHeader)), scratchGPR 2);4343 emitAllocateBasicStorage(TrustedImm32(newSize), scratchGPR1); 4344 4345 m_jit.addPtr(JITCompiler::TrustedImm32(sizeof(IndexingHeader)), scratchGPR1); 4388 4346 4389 4347 addSlowPathGenerator( 4390 slowPathCall(slowPath, this, operationAllocatePropertyStorage, scratchGPR2, newSize / sizeof(JSValue))); 4391 // We have scratchGPR2 = new storage, scratchGPR1 = scratch 4348 slowPathCall(slowPath, this, operationAllocatePropertyStorage, scratchGPR1, newSize / sizeof(JSValue))); 4349 4350 // We have scratchGPR1 = new storage, scratchGPR2 = scratch 4392 4351 for (ptrdiff_t offset = 0; offset < static_cast<ptrdiff_t>(oldSize); offset += sizeof(void*)) { 4393 m_jit.loadPtr(JITCompiler::Address(oldStorageGPR, -(offset + sizeof(JSValue) + sizeof(void*))), scratchGPR 1);4394 m_jit.storePtr(scratchGPR 1, JITCompiler::Address(scratchGPR2, -(offset + sizeof(JSValue) + sizeof(void*))));4395 } 4396 m_jit.storePtr(scratchGPR 2, JITCompiler::Address(baseGPR, JSObject::butterflyOffset()));4397 4398 storageResult(scratchGPR 2, node);4352 m_jit.loadPtr(JITCompiler::Address(oldStorageGPR, -(offset + sizeof(JSValue) + sizeof(void*))), scratchGPR2); 4353 m_jit.storePtr(scratchGPR2, JITCompiler::Address(scratchGPR1, -(offset + sizeof(JSValue) + sizeof(void*)))); 4354 } 4355 m_jit.storePtr(scratchGPR1, JITCompiler::Address(baseGPR, JSObject::butterflyOffset())); 4356 4357 storageResult(scratchGPR1, node); 4399 4358 } 4400 4359 … … 5452 5411 } 5453 5412 5413 #if ENABLE(GGC) 5414 void SpeculativeJIT::compileStoreBarrier(Node* node) 5415 { 5416 switch (node->op()) { 5417 case ConditionalStoreBarrier: { 5418 compileBaseValueStoreBarrier(node->child1(), node->child2()); 5419 break; 5420 } 5421 5422 case StoreBarrier: { 5423 SpeculateCellOperand base(this, node->child1()); 5424 GPRTemporary scratch1(this); 5425 GPRTemporary scratch2(this); 5426 5427 writeBarrier(base.gpr(), scratch1.gpr(), scratch2.gpr()); 5428 break; 5429 } 5430 5431 case StoreBarrierWithNullCheck: { 5432 JSValueOperand base(this, node->child1()); 5433 GPRTemporary scratch1(this); 5434 GPRTemporary scratch2(this); 5435 5436 #if USE(JSVALUE64) 5437 JITCompiler::Jump isNull = m_jit.branchTest64(JITCompiler::Zero, base.gpr()); 5438 writeBarrier(base.gpr(), scratch1.gpr(), scratch2.gpr()); 5439 #else 5440 JITCompiler::Jump isNull = m_jit.branch32(JITCompiler::Equal, base.tagGPR(), TrustedImm32(JSValue::EmptyValueTag)); 5441 writeBarrier(base.payloadGPR(), scratch1.gpr(), scratch2.gpr()); 5442 #endif 5443 isNull.link(&m_jit); 5444 break; 5445 } 5446 5447 default: 5448 RELEASE_ASSERT_NOT_REACHED(); 5449 break; 5450 } 5451 5452 noResult(node); 5453 } 5454 5455 JITCompiler::Jump SpeculativeJIT::genericWriteBarrier(CCallHelpers& jit, GPRReg owner, GPRReg scratch1, GPRReg scratch2) 5456 { 5457 jit.move(owner, scratch1); 5458 jit.move(owner, scratch2); 5459 5460 jit.andPtr(MacroAssembler::TrustedImmPtr(MarkedBlock::blockMask), scratch1); 5461 jit.andPtr(MacroAssembler::TrustedImmPtr(~MarkedBlock::blockMask), scratch2); 5462 5463 // Shift index 5464 #if USE(JSVALUE64) 5465 jit.rshift64(MacroAssembler::TrustedImm32(MarkedBlock::atomShiftAmount + MarkedBlock::markByteShiftAmount), scratch2); 5466 #else 5467 jit.rshift32(MacroAssembler::TrustedImm32(MarkedBlock::atomShiftAmount + MarkedBlock::markByteShiftAmount), scratch2); 5468 #endif 5469 5470 // Emit load and branch 5471 return jit.branchTest8(MacroAssembler::Zero, MacroAssembler::BaseIndex(scratch1, scratch2, MacroAssembler::TimesOne, MarkedBlock::offsetOfMarks())); 5472 } 5473 5474 JITCompiler::Jump SpeculativeJIT::genericWriteBarrier(CCallHelpers& jit, JSCell* owner) 5475 { 5476 MarkedBlock* block = MarkedBlock::blockFor(owner); 5477 size_t markIndex = (reinterpret_cast<size_t>(owner) & ~MarkedBlock::blockMask) >> (MarkedBlock::atomShiftAmount + MarkedBlock::markByteShiftAmount); 5478 uint8_t* address = reinterpret_cast<uint8_t*>(reinterpret_cast<char*>(block) + MarkedBlock::offsetOfMarks()) + markIndex; 5479 return jit.branchTest8(MacroAssembler::Zero, MacroAssembler::AbsoluteAddress(address)); 5480 } 5481 5482 MacroAssembler::Call SpeculativeJIT::storeToWriteBarrierBuffer(CCallHelpers& jit, GPRReg cell, GPRReg scratch1, GPRReg scratch2) 5483 { 5484 ASSERT(scratch1 != scratch2); 5485 // Load WriteBarrierBuffer from Heap 5486 WriteBarrierBuffer* writeBarrierBuffer = &jit.vm()->heap.m_writeBarrierBuffer; 5487 jit.move(TrustedImmPtr(writeBarrierBuffer), scratch1); 5488 // Load currentIndex 5489 jit.load32(MacroAssembler::Address(scratch1, WriteBarrierBuffer::currentIndexOffset()), scratch2); 5490 // Branch if currentIndex >= capacity 5491 JITCompiler::Jump needToFlush = jit.branch32(MacroAssembler::AboveOrEqual, scratch2, MacroAssembler::Address(scratch1, WriteBarrierBuffer::capacityOffset())); 5492 5493 // Store new currentIndex 5494 jit.add32(TrustedImm32(1), scratch2); 5495 jit.store32(scratch2, MacroAssembler::Address(scratch1, WriteBarrierBuffer::currentIndexOffset())); 5496 5497 // Load buffer 5498 jit.loadPtr(MacroAssembler::Address(scratch1, WriteBarrierBuffer::bufferOffset()), scratch1); 5499 // Store cell into buffer. We use an offset of -sizeof(void*) because we already added 1 to scratch2. 5500 jit.storePtr(cell, MacroAssembler::BaseIndex(scratch1, scratch2, MacroAssembler::ScalePtr, static_cast<int32_t>(-sizeof(void*)))); 5501 5502 // Jump to done 5503 JITCompiler::Jump done = jit.jump(); 5504 // Link from branch 5505 needToFlush.link(&jit); 5506 5507 // Call C slow path. 5508 for (unsigned i = 0; i < GPRInfo::numberOfRegisters; ++i) 5509 jit.storePtr(GPRInfo::toRegister(i), &jit.vm()->writeBarrierRegisterBuffer[i]); 5510 5511 #if CPU(X86) 5512 jit.push(scratch1); 5513 jit.push(scratch1); 5514 #endif 5515 5516 jit.setupArgumentsWithExecState(cell); 5517 MacroAssembler::Call call = jit.call(); 5518 5519 #if CPU(X86) 5520 jit.pop(scratch1); 5521 jit.pop(scratch1); 5522 #endif 5523 5524 for (unsigned i = GPRInfo::numberOfRegisters; i--;) 5525 jit.loadPtr(&jit.vm()->writeBarrierRegisterBuffer[i], GPRInfo::toRegister(i)); 5526 5527 // Link Done 5528 done.link(&jit); 5529 5530 return call; 5531 } 5532 5533 void SpeculativeJIT::storeToWriteBarrierBuffer(GPRReg cell, GPRReg scratch1, GPRReg scratch2) 5534 { 5535 ASSERT(scratch1 != scratch2); 5536 WriteBarrierBuffer* writeBarrierBuffer = &m_jit.vm()->heap.m_writeBarrierBuffer; 5537 m_jit.move(TrustedImmPtr(writeBarrierBuffer), scratch1); 5538 m_jit.load32(MacroAssembler::Address(scratch1, WriteBarrierBuffer::currentIndexOffset()), scratch2); 5539 JITCompiler::Jump needToFlush = m_jit.branch32(MacroAssembler::AboveOrEqual, scratch2, MacroAssembler::Address(scratch1, WriteBarrierBuffer::capacityOffset())); 5540 5541 m_jit.add32(TrustedImm32(1), scratch2); 5542 m_jit.store32(scratch2, MacroAssembler::Address(scratch1, WriteBarrierBuffer::currentIndexOffset())); 5543 5544 m_jit.loadPtr(MacroAssembler::Address(scratch1, WriteBarrierBuffer::bufferOffset()), scratch1); 5545 // We use an offset of -sizeof(void*) because we already added 1 to scratch2. 5546 m_jit.storePtr(cell, MacroAssembler::BaseIndex(scratch1, scratch2, MacroAssembler::ScalePtr, static_cast<int32_t>(-sizeof(void*)))); 5547 5548 JITCompiler::Jump done = m_jit.jump(); 5549 needToFlush.link(&m_jit); 5550 5551 silentSpillAllRegisters(InvalidGPRReg); 5552 callOperation(operationFlushWriteBarrierBuffer, cell); 5553 silentFillAllRegisters(InvalidGPRReg); 5554 5555 done.link(&m_jit); 5556 } 5557 5558 void SpeculativeJIT::storeToWriteBarrierBuffer(JSCell* cell, GPRReg scratch1, GPRReg scratch2) 5559 { 5560 ASSERT(scratch1 != scratch2); 5561 WriteBarrierBuffer* writeBarrierBuffer = &m_jit.vm()->heap.m_writeBarrierBuffer; 5562 m_jit.move(TrustedImmPtr(writeBarrierBuffer), scratch1); 5563 m_jit.load32(MacroAssembler::Address(scratch1, WriteBarrierBuffer::currentIndexOffset()), scratch2); 5564 JITCompiler::Jump needToFlush = m_jit.branch32(MacroAssembler::AboveOrEqual, scratch2, MacroAssembler::Address(scratch1, WriteBarrierBuffer::capacityOffset())); 5565 5566 m_jit.add32(TrustedImm32(1), scratch2); 5567 m_jit.store32(scratch2, MacroAssembler::Address(scratch1, WriteBarrierBuffer::currentIndexOffset())); 5568 5569 m_jit.loadPtr(MacroAssembler::Address(scratch1, WriteBarrierBuffer::bufferOffset()), scratch1); 5570 // We use an offset of -sizeof(void*) because we already added 1 to scratch2. 5571 m_jit.storePtr(TrustedImmPtr(cell), MacroAssembler::BaseIndex(scratch1, scratch2, MacroAssembler::ScalePtr, static_cast<int32_t>(-sizeof(void*)))); 5572 5573 JITCompiler::Jump done = m_jit.jump(); 5574 needToFlush.link(&m_jit); 5575 5576 // Call C slow path 5577 silentSpillAllRegisters(InvalidGPRReg); 5578 callOperation(operationFlushWriteBarrierBuffer, cell); 5579 silentFillAllRegisters(InvalidGPRReg); 5580 5581 done.link(&m_jit); 5582 } 5583 5584 void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, JSCell* value, GPRReg scratch1, GPRReg scratch2) 5585 { 5586 if (Heap::isMarked(value)) 5587 return; 5588 5589 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(m_jit, ownerGPR, scratch1, scratch2); 5590 storeToWriteBarrierBuffer(ownerGPR, scratch1, scratch2); 5591 definitelyNotMarked.link(&m_jit); 5592 } 5593 5594 void SpeculativeJIT::osrWriteBarrier(CCallHelpers& jit, GPRReg owner, GPRReg scratch1, GPRReg scratch2) 5595 { 5596 UNUSED_PARAM(jit); 5597 UNUSED_PARAM(owner); 5598 UNUSED_PARAM(scratch1); 5599 UNUSED_PARAM(scratch2); 5600 ASSERT(owner != scratch1); 5601 ASSERT(owner != scratch2); 5602 5603 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(jit, owner, scratch1, scratch2); 5604 5605 for (unsigned i = 0; i < GPRInfo::numberOfRegisters; ++i) 5606 jit.storePtr(GPRInfo::toRegister(i), &jit.vm()->writeBarrierRegisterBuffer[i]); 5607 5608 // We need these extra slots because setupArgumentsWithExecState will use poke on x86. 5609 #if CPU(X86) 5610 jit.push(scratch1); 5611 jit.push(scratch1); 5612 jit.push(scratch1); 5613 #endif 5614 5615 jit.setupArgumentsWithExecState(owner); 5616 jit.move(TrustedImmPtr(reinterpret_cast<void*>(operationOSRWriteBarrier)), scratch1); 5617 jit.call(scratch1); 5618 5619 #if CPU(X86) 5620 jit.pop(scratch1); 5621 jit.pop(scratch1); 5622 jit.pop(scratch1); 5623 #endif 5624 for (unsigned i = GPRInfo::numberOfRegisters; i--;) 5625 jit.loadPtr(&jit.vm()->writeBarrierRegisterBuffer[i], GPRInfo::toRegister(i)); 5626 5627 definitelyNotMarked.link(&jit); 5628 } 5629 5630 MacroAssembler::Call SpeculativeJIT::writeBarrier(CCallHelpers& jit, GPRReg owner, GPRReg scratch1, GPRReg scratch2) 5631 { 5632 ASSERT(owner != scratch1); 5633 ASSERT(owner != scratch2); 5634 5635 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(jit, owner, scratch1, scratch2); 5636 MacroAssembler::Call call = storeToWriteBarrierBuffer(jit, owner, scratch1, scratch2); 5637 definitelyNotMarked.link(&jit); 5638 return call; 5639 } 5640 5641 void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, GPRReg scratch1, GPRReg scratch2) 5642 { 5643 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(m_jit, ownerGPR, scratch1, scratch2); 5644 storeToWriteBarrierBuffer(ownerGPR, scratch1, scratch2); 5645 definitelyNotMarked.link(&m_jit); 5646 } 5647 #else 5648 void SpeculativeJIT::compileStoreBarrier(Node* node) 5649 { 5650 DFG_NODE_DO_TO_CHILDREN(m_jit.graph(), node, speculate); 5651 noResult(node); 5652 } 5653 #endif // ENABLE(GGC) 5654 5454 5655 } } // namespace JSC::DFG 5455 5656 -
trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h
r160587 r160796 290 290 } 291 291 292 void writeBarrier(GPRReg ownerGPR, GPRReg valueGPR, Edge valueUse, WriteBarrierUseKind, GPRReg scratchGPR1 = InvalidGPRReg, GPRReg scratchGPR2 = InvalidGPRReg); 293 void writeBarrier(GPRReg ownerGPR, JSCell* value, WriteBarrierUseKind, GPRReg scratchGPR1 = InvalidGPRReg, GPRReg scratchGPR2 = InvalidGPRReg); 294 void writeBarrier(JSCell* owner, GPRReg valueGPR, Edge valueUse, WriteBarrierUseKind, GPRReg scratchGPR1 = InvalidGPRReg); 292 #if ENABLE(GGC) 293 static MacroAssembler::Call storeToWriteBarrierBuffer(CCallHelpers& jit, GPRReg cell, GPRReg scratch1, GPRReg scratch2); 294 void storeToWriteBarrierBuffer(GPRReg cell, GPRReg scratch1, GPRReg scratch2); 295 void storeToWriteBarrierBuffer(JSCell*, GPRReg scratch1, GPRReg scratch2); 296 297 static JITCompiler::Jump genericWriteBarrier(CCallHelpers& jit, GPRReg owner, GPRReg scratch1, GPRReg scratch2); 298 static JITCompiler::Jump genericWriteBarrier(CCallHelpers& jit, JSCell* owner); 299 static void osrWriteBarrier(CCallHelpers&, GPRReg owner, GPRReg scratch1, GPRReg scratch2); 300 static MacroAssembler::Call writeBarrier(CCallHelpers&, GPRReg owner, GPRReg scratch1, GPRReg scratch2); 301 void writeBarrier(GPRReg owner, GPRReg scratch1, GPRReg scratch2); 302 void writeBarrier(GPRReg owner, JSCell* value, GPRReg scratch1, GPRReg scratch2); 303 304 void writeBarrier(GPRReg owner, GPRReg value, Edge valueUse, GPRReg scratch1, GPRReg scratch2); 305 void writeBarrier(JSCell* owner, GPRReg value, Edge valueUse, GPRReg scratch1, GPRReg scratch2); 306 #endif 307 void compileStoreBarrier(Node*); 295 308 296 309 static GPRReg selectScratchGPR(GPRReg preserve1 = InvalidGPRReg, GPRReg preserve2 = InvalidGPRReg, GPRReg preserve3 = InvalidGPRReg, GPRReg preserve4 = InvalidGPRReg) … … 701 714 #if USE(JSVALUE64) 702 715 void cachedGetById(CodeOrigin, GPRReg baseGPR, GPRReg resultGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill); 703 void cachedPutById(CodeOrigin, GPRReg base, GPRReg value, Edge valueUse,GPRReg scratchGPR, unsigned identifierNumber, PutKind, JITCompiler::Jump slowPathTarget = JITCompiler::Jump());716 void cachedPutById(CodeOrigin, GPRReg base, GPRReg value, GPRReg scratchGPR, unsigned identifierNumber, PutKind, JITCompiler::Jump slowPathTarget = JITCompiler::Jump()); 704 717 #elif USE(JSVALUE32_64) 705 718 void cachedGetById(CodeOrigin, GPRReg baseTagGPROrNone, GPRReg basePayloadGPR, GPRReg resultTagGPR, GPRReg resultPayloadGPR, unsigned identifierNumber, JITCompiler::Jump slowPathTarget = JITCompiler::Jump(), SpillRegistersMode = NeedToSpill); 706 void cachedPutById(CodeOrigin, GPRReg basePayloadGPR, GPRReg valueTagGPR, GPRReg valuePayloadGPR, Edge valueUse,GPRReg scratchGPR, unsigned identifierNumber, PutKind, JITCompiler::Jump slowPathTarget = JITCompiler::Jump());719 void cachedPutById(CodeOrigin, GPRReg basePayloadGPR, GPRReg valueTagGPR, GPRReg valuePayloadGPR, GPRReg scratchGPR, unsigned identifierNumber, PutKind, JITCompiler::Jump slowPathTarget = JITCompiler::Jump()); 707 720 #endif 708 721 709 722 void compileIn(Node*); 710 723 724 void compileBaseValueStoreBarrier(Edge& baseEdge, Edge& valueEdge); 725 711 726 void nonSpeculativeNonPeepholeCompareNull(Edge operand, bool invert = false); 712 727 void nonSpeculativePeepholeBranchNull(Edge operand, Node* branchNode, bool invert = false); … … 1060 1075 } 1061 1076 1077 JITCompiler::Call callOperation(V_JITOperation_EC operation, JSCell* arg1) 1078 { 1079 m_jit.setupArgumentsWithExecState(TrustedImmPtr(arg1)); 1080 return appendCallWithExceptionCheck(operation); 1081 } 1082 1062 1083 JITCompiler::Call callOperation(V_JITOperation_ECIcf operation, GPRReg arg1, InlineCallFrame* inlineCallFrame) 1063 1084 { … … 1076 1097 return appendCallWithExceptionCheck(operation); 1077 1098 } 1078 1079 1099 JITCompiler::Call callOperation(V_JITOperation_ECC operation, GPRReg arg1, GPRReg arg2) 1080 1100 { 1081 1101 m_jit.setupArgumentsWithExecState(arg1, arg2); 1102 return appendCallWithExceptionCheck(operation); 1103 } 1104 JITCompiler::Call callOperation(V_JITOperation_ECC operation, GPRReg arg1, JSCell* arg2) 1105 { 1106 m_jit.setupArgumentsWithExecState(arg1, TrustedImmPtr(arg2)); 1107 return appendCallWithExceptionCheck(operation); 1108 } 1109 JITCompiler::Call callOperation(V_JITOperation_ECC operation, JSCell* arg1, GPRReg arg2) 1110 { 1111 m_jit.setupArgumentsWithExecState(TrustedImmPtr(arg1), arg2); 1082 1112 return appendCallWithExceptionCheck(operation); 1083 1113 } … … 1341 1371 return appendCallWithExceptionCheck(operation); 1342 1372 } 1343 1344 1373 JITCompiler::Call callOperation(V_JITOperation_EJPP operation, GPRReg arg1, GPRReg arg2, void* pointer) 1345 1374 { -
trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
r160587 r160796 201 201 } 202 202 203 void SpeculativeJIT::cachedPutById(CodeOrigin codeOrigin, GPRReg basePayloadGPR, GPRReg valueTagGPR, GPRReg valuePayloadGPR, Edge valueUse,GPRReg scratchGPR, unsigned identifierNumber, PutKind putKind, JITCompiler::Jump slowPathTarget)203 void SpeculativeJIT::cachedPutById(CodeOrigin codeOrigin, GPRReg basePayloadGPR, GPRReg valueTagGPR, GPRReg valuePayloadGPR, GPRReg scratchGPR, unsigned identifierNumber, PutKind putKind, JITCompiler::Jump slowPathTarget) 204 204 { 205 writeBarrier(basePayloadGPR, valueTagGPR, valueUse, WriteBarrierForPropertyAccess, scratchGPR);206 207 205 JITPutByIdGenerator gen( 208 206 m_jit.codeBlock(), codeOrigin, usedRegisters(), GPRInfo::callFrameRegister, … … 1113 1111 } 1114 1112 1113 void SpeculativeJIT::compileBaseValueStoreBarrier(Edge& baseEdge, Edge& valueEdge) 1114 { 1115 #if ENABLE(GGC) 1116 ASSERT(!isKnownNotCell(valueEdge.node())); 1117 1118 SpeculateCellOperand base(this, baseEdge); 1119 JSValueOperand value(this, valueEdge); 1120 GPRTemporary scratch1(this); 1121 GPRTemporary scratch2(this); 1122 1123 writeBarrier(base.gpr(), value.tagGPR(), valueEdge, scratch1.gpr(), scratch2.gpr()); 1124 #else 1125 UNUSED_PARAM(baseEdge); 1126 UNUSED_PARAM(valueEdge); 1127 #endif 1128 } 1129 1115 1130 void SpeculativeJIT::compileObjectEquality(Node* node) 1116 1131 { … … 2612 2627 if (!m_compileOkay) 2613 2628 return; 2614 2615 if (Heap::isWriteBarrierEnabled()) { 2616 GPRTemporary scratch(this); 2617 writeBarrier(baseReg, valueTagReg, child3, WriteBarrierForPropertyAccess, scratch.gpr()); 2618 } 2619 2629 2620 2630 compileContiguousPutByVal(node, base, property, value, valuePayloadReg, valueTagReg); 2621 2631 break; … … 2634 2644 if (!m_compileOkay) 2635 2645 return; 2636 2637 { 2638 GPRTemporary scratch(this); 2639 GPRReg scratchReg = scratch.gpr(); 2640 writeBarrier(baseReg, valueTagReg, child3, WriteBarrierForPropertyAccess, scratchReg); 2641 } 2642 2646 2643 2647 StorageOperand storage(this, child4); 2644 2648 GPRReg storageReg = storage.gpr(); … … 2820 2824 GPRReg valuePayloadGPR = value.payloadGPR(); 2821 2825 2822 if (Heap::isWriteBarrierEnabled()) {2823 GPRTemporary scratch(this);2824 writeBarrier(baseGPR, valueTagGPR, node->child2(), WriteBarrierForPropertyAccess, scratch.gpr(), storageLengthGPR);2825 }2826 2827 2826 m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR); 2828 2827 MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength())); … … 2872 2871 GPRReg valueTagGPR = value.tagGPR(); 2873 2872 GPRReg valuePayloadGPR = value.payloadGPR(); 2874 2875 if (Heap::isWriteBarrierEnabled()) {2876 GPRTemporary scratch(this);2877 writeBarrier(baseGPR, valueTagGPR, node->child2(), WriteBarrierForPropertyAccess, scratch.gpr(), storageLengthGPR);2878 }2879 2873 2880 2874 m_jit.load32(MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()), storageLengthGPR); … … 3674 3668 } 3675 3669 case PutClosureVar: { 3676 SpeculateCellOperand scope(this, node->child1());3677 3670 StorageOperand registers(this, node->child2()); 3678 3671 JSValueOperand value(this, node->child3()); 3679 3672 GPRTemporary scratchRegister(this); 3680 GPRReg scopeGPR = scope.gpr(); 3673 3681 3674 GPRReg registersGPR = registers.gpr(); 3682 3675 GPRReg valueTagGPR = value.tagGPR(); 3683 3676 GPRReg valuePayloadGPR = value.payloadGPR(); 3684 GPRReg scratchGPR = scratchRegister.gpr(); 3677 3678 speculate(node, node->child1()); 3685 3679 3686 3680 m_jit.store32(valueTagGPR, JITCompiler::Address(registersGPR, node->varNumber() * sizeof(Register) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.tag))); 3687 3681 m_jit.store32(valuePayloadGPR, JITCompiler::Address(registersGPR, node->varNumber() * sizeof(Register) + OBJECT_OFFSETOF(EncodedValueDescriptor, asBits.payload))); 3688 writeBarrier(scopeGPR, valueTagGPR, node->child3(), WriteBarrierForVariableAccess, scratchGPR);3689 3682 noResult(node); 3690 3683 break; … … 3883 3876 GPRReg baseGPR = base.gpr(); 3884 3877 3885 #if ENABLE(WRITE_BARRIER_PROFILING)3886 // Must always emit this write barrier as the structure transition itself requires it3887 writeBarrier(baseGPR, node->structureTransitionData().newStructure, WriteBarrierForGenericAccess);3888 #endif3889 3890 3878 m_jit.storePtr(MacroAssembler::TrustedImmPtr(node->structureTransitionData().newStructure), MacroAssembler::Address(baseGPR, JSCell::structureOffset())); 3891 3879 … … 3949 3937 3950 3938 case PutByOffset: { 3951 #if ENABLE(WRITE_BARRIER_PROFILING)3952 SpeculateCellOperand base(this, node->child2());3953 #endif3954 3939 StorageOperand storage(this, node->child1()); 3955 3940 JSValueOperand value(this, node->child3()); … … 3958 3943 GPRReg valueTagGPR = value.tagGPR(); 3959 3944 GPRReg valuePayloadGPR = value.payloadGPR(); 3960 3961 #if ENABLE(WRITE_BARRIER_PROFILING) 3962 writeBarrier(base.gpr(), valueTagGPR, node->child3(), WriteBarrierForPropertyAccess); 3963 #endif 3945 3946 speculate(node, node->child2()); 3964 3947 3965 3948 StorageAccessData& storageAccessData = m_jit.graph().m_storageAccessData[node->storageAccessDataIndex()]; … … 3982 3965 GPRReg scratchGPR = scratch.gpr(); 3983 3966 3984 base.use(); 3985 value.use(); 3986 3987 cachedPutById(node->codeOrigin, baseGPR, valueTagGPR, valuePayloadGPR, node->child2(), scratchGPR, node->identifierNumber(), NotDirect); 3988 3989 noResult(node, UseChildrenCalledExplicitly); 3967 cachedPutById(node->codeOrigin, baseGPR, valueTagGPR, valuePayloadGPR, scratchGPR, node->identifierNumber(), NotDirect); 3968 3969 noResult(node); 3990 3970 break; 3991 3971 } … … 4001 3981 GPRReg scratchGPR = scratch.gpr(); 4002 3982 4003 base.use(); 4004 value.use(); 4005 4006 cachedPutById(node->codeOrigin, baseGPR, valueTagGPR, valuePayloadGPR, node->child2(), scratchGPR, node->identifierNumber(), Direct); 4007 4008 noResult(node, UseChildrenCalledExplicitly); 3983 cachedPutById(node->codeOrigin, baseGPR, valueTagGPR, valuePayloadGPR, scratchGPR, node->identifierNumber(), Direct); 3984 3985 noResult(node); 4009 3986 break; 4010 3987 } … … 4024 4001 case PutGlobalVar: { 4025 4002 JSValueOperand value(this, node->child1()); 4026 if (Heap::isWriteBarrierEnabled()) {4027 GPRTemporary scratch(this);4028 GPRReg scratchReg = scratch.gpr();4029 4030 writeBarrier(m_jit.globalObjectFor(node->codeOrigin), value.tagGPR(), node->child1(), WriteBarrierForVariableAccess, scratchReg);4031 }4032 4003 4033 4004 // FIXME: if we happen to have a spare register - and _ONLY_ if we happen to have … … 4695 4666 break; 4696 4667 4668 case StoreBarrier: 4669 case ConditionalStoreBarrier: 4670 case StoreBarrierWithNullCheck: { 4671 compileStoreBarrier(node); 4672 break; 4673 } 4674 4697 4675 case ForceOSRExit: { 4698 4676 terminateSpeculativeExecution(InadequateCoverage, JSValueRegs(), 0); … … 4753 4731 } 4754 4732 4733 #if ENABLE(GGC) 4734 void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, GPRReg valueTagGPR, Edge valueUse, GPRReg scratch1, GPRReg scratch2) 4735 { 4736 JITCompiler::Jump isNotCell; 4737 if (!isKnownCell(valueUse.node())) 4738 isNotCell = m_jit.branch32(JITCompiler::NotEqual, valueTagGPR, JITCompiler::TrustedImm32(JSValue::CellTag)); 4739 4740 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(m_jit, ownerGPR, scratch1, scratch2); 4741 storeToWriteBarrierBuffer(ownerGPR, scratch1, scratch2); 4742 definitelyNotMarked.link(&m_jit); 4743 4744 if (!isKnownCell(valueUse.node())) 4745 isNotCell.link(&m_jit); 4746 } 4747 4748 void SpeculativeJIT::writeBarrier(JSCell* owner, GPRReg valueTagGPR, Edge valueUse, GPRReg scratch1, GPRReg scratch2) 4749 { 4750 JITCompiler::Jump isNotCell; 4751 if (!isKnownCell(valueUse.node())) 4752 isNotCell = m_jit.branch32(JITCompiler::NotEqual, valueTagGPR, JITCompiler::TrustedImm32(JSValue::CellTag)); 4753 4754 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(m_jit, owner); 4755 storeToWriteBarrierBuffer(owner, scratch1, scratch2); 4756 definitelyNotMarked.link(&m_jit); 4757 4758 if (!isKnownCell(valueUse.node())) 4759 isNotCell.link(&m_jit); 4760 } 4761 #endif // ENABLE(GGC) 4762 4755 4763 #endif 4756 4764 -
trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
r160587 r160796 207 207 } 208 208 209 void SpeculativeJIT::cachedPutById(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg valueGPR, Edge valueUse,GPRReg scratchGPR, unsigned identifierNumber, PutKind putKind, JITCompiler::Jump slowPathTarget)209 void SpeculativeJIT::cachedPutById(CodeOrigin codeOrigin, GPRReg baseGPR, GPRReg valueGPR, GPRReg scratchGPR, unsigned identifierNumber, PutKind putKind, JITCompiler::Jump slowPathTarget) 210 210 { 211 writeBarrier(baseGPR, valueGPR, valueUse, WriteBarrierForPropertyAccess, scratchGPR);212 213 211 JITPutByIdGenerator gen( 214 212 m_jit.codeBlock(), codeOrigin, usedRegisters(), GPRInfo::callFrameRegister, … … 1474 1472 } 1475 1473 1474 void SpeculativeJIT::compileBaseValueStoreBarrier(Edge& baseEdge, Edge& valueEdge) 1475 { 1476 #if ENABLE(GGC) 1477 ASSERT(!isKnownNotCell(valueEdge.node())); 1478 1479 SpeculateCellOperand base(this, baseEdge); 1480 JSValueOperand value(this, valueEdge); 1481 GPRTemporary scratch1(this); 1482 GPRTemporary scratch2(this); 1483 1484 writeBarrier(base.gpr(), value.gpr(), valueEdge, scratch1.gpr(), scratch2.gpr()); 1485 #else 1486 UNUSED_PARAM(baseEdge); 1487 UNUSED_PARAM(valueEdge); 1488 #endif 1489 } 1490 1476 1491 void SpeculativeJIT::compileObjectEquality(Node* node) 1477 1492 { … … 2908 2923 MacroAssembler::Below, valueReg, GPRInfo::tagTypeNumberRegister)); 2909 2924 } 2910 2911 if (arrayMode.type() == Array::Contiguous && Heap::isWriteBarrierEnabled()) {2912 GPRTemporary scratch(this);2913 writeBarrier(baseReg, value.gpr(), child3, WriteBarrierForPropertyAccess, scratch.gpr());2914 }2915 2925 2916 2926 StorageOperand storage(this, child4); … … 2988 2998 if (!m_compileOkay) 2989 2999 return; 2990 2991 if (Heap::isWriteBarrierEnabled()) {2992 GPRTemporary scratch(this);2993 writeBarrier(baseReg, value.gpr(), child3, WriteBarrierForPropertyAccess, scratch.gpr());2994 }2995 3000 2996 3001 StorageOperand storage(this, child4); … … 3093 3098 MacroAssembler::Address( 3094 3099 baseReg, Arguments::offsetOfSlowArgumentData()))); 3095 3100 3096 3101 m_jit.move(propertyReg, scratch2Reg); 3097 3102 m_jit.signExtend32ToPtr(scratch2Reg, scratch2Reg); … … 3194 3199 } 3195 3200 3196 if (node->arrayMode().type() != Array::Int32 && Heap::isWriteBarrierEnabled()) {3197 GPRTemporary scratch(this);3198 writeBarrier(baseGPR, valueGPR, node->child2(), WriteBarrierForPropertyAccess, scratch.gpr(), storageLengthGPR);3199 }3200 3201 3201 m_jit.load32(MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()), storageLengthGPR); 3202 3202 MacroAssembler::Jump slowPath = m_jit.branch32(MacroAssembler::AboveOrEqual, storageLengthGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength())); … … 3242 3242 JSValueOperand value(this, node->child2()); 3243 3243 GPRReg valueGPR = value.gpr(); 3244 3245 if (Heap::isWriteBarrierEnabled()) {3246 GPRTemporary scratch(this);3247 writeBarrier(baseGPR, valueGPR, node->child2(), WriteBarrierForPropertyAccess, scratch.gpr(), storageLengthGPR);3248 }3249 3244 3250 3245 m_jit.load32(MacroAssembler::Address(storageGPR, ArrayStorage::lengthOffset()), storageLengthGPR); … … 3980 3975 } 3981 3976 case PutClosureVar: { 3982 SpeculateCellOperand scope(this, node->child1());3983 3977 StorageOperand registers(this, node->child2()); 3984 3978 JSValueOperand value(this, node->child3()); 3985 GPRTemporary scratchRegister(this); 3986 3987 GPRReg scopeGPR = scope.gpr(); 3979 3988 3980 GPRReg registersGPR = registers.gpr(); 3989 3981 GPRReg valueGPR = value.gpr(); 3990 GPRReg scratchGPR = scratchRegister.gpr(); 3982 3983 speculate(node, node->child1()); 3991 3984 3992 3985 m_jit.store64(valueGPR, JITCompiler::Address(registersGPR, node->varNumber() * sizeof(Register))); 3993 writeBarrier(scopeGPR, valueGPR, node->child3(), WriteBarrierForVariableAccess, scratchGPR);3994 3986 noResult(node); 3995 3987 break; … … 4180 4172 4181 4173 SpeculateCellOperand base(this, node->child1()); 4174 GPRTemporary scratch1(this); 4175 GPRTemporary scratch2(this); 4182 4176 GPRReg baseGPR = base.gpr(); 4183 4184 4185 #if ENABLE(WRITE_BARRIER_PROFILING)4186 // Must always emit this write barrier as the structure transition itself requires it4187 writeBarrier(baseGPR, node->structureTransitionData().newStructure, WriteBarrierForGenericAccess);4188 #endif4189 4177 4190 4178 m_jit.storePtr(MacroAssembler::TrustedImmPtr(node->structureTransitionData().newStructure), MacroAssembler::Address(baseGPR, JSCell::structureOffset())); … … 4246 4234 4247 4235 case PutByOffset: { 4248 #if ENABLE(WRITE_BARRIER_PROFILING)4249 SpeculateCellOperand base(this, node->child2());4250 #endif4251 4236 StorageOperand storage(this, node->child1()); 4252 4237 JSValueOperand value(this, node->child3()); 4238 GPRTemporary scratch1(this); 4239 GPRTemporary scratch2(this); 4253 4240 4254 4241 GPRReg storageGPR = storage.gpr(); 4255 4242 GPRReg valueGPR = value.gpr(); 4256 4257 #if ENABLE(WRITE_BARRIER_PROFILING) 4258 writeBarrier(base.gpr(), value.gpr(), node->child3(), WriteBarrierForPropertyAccess); 4259 #endif 4243 4244 speculate(node, node->child2()); 4260 4245 4261 4246 StorageAccessData& storageAccessData = m_jit.graph().m_storageAccessData[node->storageAccessDataIndex()]; 4262 4247 4263 4248 m_jit.store64(valueGPR, JITCompiler::Address(storageGPR, offsetRelativeToBase(storageAccessData.offset))); 4264 4249 4265 4250 noResult(node); 4266 4251 break; … … 4276 4261 GPRReg scratchGPR = scratch.gpr(); 4277 4262 4278 base.use(); 4279 value.use(); 4280 4281 cachedPutById(node->codeOrigin, baseGPR, valueGPR, node->child2(), scratchGPR, node->identifierNumber(), NotDirect); 4282 4283 noResult(node, UseChildrenCalledExplicitly); 4263 cachedPutById(node->codeOrigin, baseGPR, valueGPR, scratchGPR, node->identifierNumber(), NotDirect); 4264 4265 noResult(node); 4284 4266 break; 4285 4267 } … … 4294 4276 GPRReg scratchGPR = scratch.gpr(); 4295 4277 4296 base.use(); 4297 value.use(); 4298 4299 cachedPutById(node->codeOrigin, baseGPR, valueGPR, node->child2(), scratchGPR, node->identifierNumber(), Direct); 4300 4301 noResult(node, UseChildrenCalledExplicitly); 4278 cachedPutById(node->codeOrigin, baseGPR, valueGPR, scratchGPR, node->identifierNumber(), Direct); 4279 4280 noResult(node); 4302 4281 break; 4303 4282 } … … 4314 4293 case PutGlobalVar: { 4315 4294 JSValueOperand value(this, node->child1()); 4316 4317 if (Heap::isWriteBarrierEnabled()) { 4318 GPRTemporary scratch(this); 4319 GPRReg scratchReg = scratch.gpr(); 4320 4321 writeBarrier(m_jit.globalObjectFor(node->codeOrigin), value.gpr(), node->child1(), WriteBarrierForVariableAccess, scratchReg); 4322 } 4323 4295 4324 4296 m_jit.store64(value.gpr(), node->registerPointer()); 4325 4297 … … 4974 4946 break; 4975 4947 4948 case StoreBarrier: 4949 case ConditionalStoreBarrier: 4950 case StoreBarrierWithNullCheck: { 4951 compileStoreBarrier(node); 4952 break; 4953 } 4954 4976 4955 #if ENABLE(FTL_JIT) 4977 4956 case CheckTierUpInLoop: { … … 5054 5033 } 5055 5034 5035 #if ENABLE(GGC) 5036 void SpeculativeJIT::writeBarrier(GPRReg ownerGPR, GPRReg valueGPR, Edge valueUse, GPRReg scratch1, GPRReg scratch2) 5037 { 5038 JITCompiler::Jump isNotCell; 5039 if (!isKnownCell(valueUse.node())) 5040 isNotCell = m_jit.branchTest64(JITCompiler::NonZero, valueGPR, GPRInfo::tagMaskRegister); 5041 5042 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(m_jit, ownerGPR, scratch1, scratch2); 5043 storeToWriteBarrierBuffer(ownerGPR, scratch1, scratch2); 5044 definitelyNotMarked.link(&m_jit); 5045 5046 if (!isKnownCell(valueUse.node())) 5047 isNotCell.link(&m_jit); 5048 } 5049 5050 void SpeculativeJIT::writeBarrier(JSCell* owner, GPRReg valueGPR, Edge valueUse, GPRReg scratch1, GPRReg scratch2) 5051 { 5052 JITCompiler::Jump isNotCell; 5053 if (!isKnownCell(valueUse.node())) 5054 isNotCell = m_jit.branchTest64(JITCompiler::NonZero, valueGPR, GPRInfo::tagMaskRegister); 5055 5056 JITCompiler::Jump definitelyNotMarked = genericWriteBarrier(m_jit, owner); 5057 storeToWriteBarrierBuffer(owner, scratch1, scratch2); 5058 definitelyNotMarked.link(&m_jit); 5059 5060 if (!isKnownCell(valueUse.node())) 5061 isNotCell.link(&m_jit); 5062 } 5063 #endif // ENABLE(GGC) 5064 5056 5065 #endif 5057 5066 -
trunk/Source/JavaScriptCore/dfg/DFGVariableAccessData.h
r159394 r160796 355 355 return FlushedAt(flushFormat(), machineLocal()); 356 356 } 357 357 358 358 private: 359 359 // This is slightly space-inefficient, since anything we're unified with -
trunk/Source/JavaScriptCore/heap/Heap.cpp
r160377 r160796 270 270 , m_handleSet(vm) 271 271 , m_isSafeToCollect(false) 272 #if ENABLE(GGC) 273 , m_writeBarrierBuffer(128) 274 #endif 272 275 , m_vm(vm) 273 276 , m_lastGCLength(0) … … 997 1000 } 998 1001 1002 void Heap::flushWriteBarrierBuffer(JSCell* cell) 1003 { 1004 #if ENABLE(GGC) 1005 m_writeBarrierBuffer.flush(*this); 1006 m_writeBarrierBuffer.add(cell); 1007 #else 1008 UNUSED_PARAM(cell); 1009 #endif 1010 } 1011 999 1012 } // namespace JSC -
trunk/Source/JavaScriptCore/heap/Heap.h
r160377 r160796 40 40 #include "SlotVisitor.h" 41 41 #include "WeakHandleOwner.h" 42 #include "WriteBarrierBuffer.h" 42 43 #include "WriteBarrierSupport.h" 43 44 #include <wtf/HashCountedSet.h> … … 78 79 public: 79 80 friend class JIT; 81 friend class DFG::SpeculativeJIT; 80 82 friend class GCThreadSharedData; 81 83 static Heap* heap(const JSValue); // 0 for immediate values … … 94 96 95 97 static bool isWriteBarrierEnabled(); 98 static void writeBarrier(const JSCell*); 96 99 static void writeBarrier(const JSCell*, JSValue); 97 100 static void writeBarrier(const JSCell*, JSCell*); 98 101 static uint8_t* addressOfCardFor(JSCell*); 102 103 void flushWriteBarrierBuffer(JSCell*); 99 104 100 105 Heap(VM*, HeapType); … … 285 290 bool m_isSafeToCollect; 286 291 292 #if ENABLE(GGC) 293 WriteBarrierBuffer m_writeBarrierBuffer; 294 #endif 295 287 296 VM* m_vm; 288 297 double m_lastGCLength; … … 371 380 } 372 381 382 inline void Heap::writeBarrier(const JSCell*) 383 { 384 WriteBarrierCounters::countWriteBarrier(); 385 } 386 373 387 inline void Heap::writeBarrier(const JSCell*, JSCell*) 374 388 { -
trunk/Source/JavaScriptCore/heap/MarkedBlock.h
r157653 r160796 74 74 public: 75 75 static const size_t atomSize = 8; // bytes 76 static const size_t atomShiftAmount = 4; // log_2(atomSize) FIXME: Change atomSize to 16. 76 77 static const size_t blockSize = 64 * KB; 77 78 static const size_t blockMask = ~(blockSize - 1); // blockSize must be a power of two. … … 79 80 static const size_t atomsPerBlock = blockSize / atomSize; 80 81 static const size_t atomMask = atomsPerBlock - 1; 82 83 static const size_t markByteShiftAmount = 3; // log_2(word size for m_marks) FIXME: Change word size for m_marks to uint8_t. 81 84 82 85 struct FreeCell { … … 168 171 template <typename Functor> void forEachLiveCell(Functor&); 169 172 template <typename Functor> void forEachDeadCell(Functor&); 173 174 static ptrdiff_t offsetOfMarks() { return OBJECT_OFFSETOF(MarkedBlock, m_marks); } 170 175 171 176 private: -
trunk/Source/JavaScriptCore/jit/JITOperations.cpp
r160254 r160796 1697 1697 } 1698 1698 1699 void JIT_OPERATION operationFlushWriteBarrierBuffer(ExecState* exec, JSCell* cell) 1700 { 1701 VM* vm = &exec->vm(); 1702 NativeCallFrameTracer tracer(vm, exec); 1703 vm->heap.flushWriteBarrierBuffer(cell); 1704 } 1705 1706 void JIT_OPERATION operationOSRWriteBarrier(ExecState* exec, JSCell* cell) 1707 { 1708 VM* vm = &exec->vm(); 1709 NativeCallFrameTracer tracer(vm, exec); 1710 exec->heap()->writeBarrier(cell); 1711 } 1712 1713 // NB: We don't include the value as part of the barrier because the write barrier elision 1714 // phase in the DFG only tracks whether the object being stored to has been barriered. It 1715 // would be much more complicated to try to model the value being stored as well. 1716 void JIT_OPERATION operationUnconditionalWriteBarrier(ExecState* exec, JSCell* cell) 1717 { 1718 VM* vm = &exec->vm(); 1719 NativeCallFrameTracer tracer(vm, exec); 1720 Heap::writeBarrier(cell); 1721 } 1722 1723 void JIT_OPERATION operationInitGlobalConst(ExecState* exec, Instruction* pc) 1724 { 1725 VM* vm = &exec->vm(); 1726 NativeCallFrameTracer tracer(vm, exec); 1727 1728 JSValue value = exec->r(pc[2].u.operand).jsValue(); 1729 pc[1].u.registerPointer->set(*vm, exec->codeBlock()->globalObject(), value); 1730 } 1731 1699 1732 void JIT_OPERATION lookupExceptionHandler(ExecState* exec) 1700 1733 { -
trunk/Source/JavaScriptCore/jit/JITOperations.h
r160244 r160796 142 142 typedef void JIT_OPERATION (*V_JITOperation_ECCIcf)(ExecState*, JSCell*, JSCell*, InlineCallFrame*); 143 143 typedef void JIT_OPERATION (*V_JITOperation_ECJJ)(ExecState*, JSCell*, EncodedJSValue, EncodedJSValue); 144 typedef void JIT_OPERATION (*V_JITOperation_ECPSPS)(ExecState*, JSCell*, void*, size_t, void*, size_t); 144 145 typedef void JIT_OPERATION (*V_JITOperation_ECZ)(ExecState*, JSCell*, int32_t); 145 146 typedef void JIT_OPERATION (*V_JITOperation_ECC)(ExecState*, JSCell*, JSCell*); … … 278 279 void JIT_OPERATION operationPutToScope(ExecState*, Instruction* bytecodePC) WTF_INTERNAL; 279 280 281 void JIT_OPERATION operationFlushWriteBarrierBuffer(ExecState*, JSCell*); 282 void JIT_OPERATION operationWriteBarrier(ExecState*, JSCell*, JSCell*); 283 void JIT_OPERATION operationUnconditionalWriteBarrier(ExecState*, JSCell*); 284 void JIT_OPERATION operationOSRWriteBarrier(ExecState*, JSCell*); 285 286 void JIT_OPERATION operationInitGlobalConst(ExecState*, Instruction*); 287 280 288 } // extern "C" 281 289 -
trunk/Source/JavaScriptCore/runtime/VM.h
r160628 r160796 32 32 #include "DateInstanceCache.h" 33 33 #include "ExecutableAllocator.h" 34 #include "GPRInfo.h" 34 35 #include "Heap.h" 35 36 #include "Intrinsic.h" … … 388 389 const ClassInfo* const jsFinalObjectClassInfo; 389 390 391 void* writeBarrierRegisterBuffer[GPRInfo::numberOfRegisters]; 392 390 393 ReturnAddressPtr exceptionLocation; 391 394 JSValue hostCallReturnValue; -
trunk/Source/WTF/ChangeLog
r160733 r160796 1 2013-12-18 Mark Hahnenberg <mhahnenberg@apple.com> 2 3 DFG should have a separate StoreBarrier node 4 https://bugs.webkit.org/show_bug.cgi?id=125530 5 6 Reviewed by Filip Pizlo. 7 8 * wtf/Platform.h: Added an #define for ENABLE(GGC) which will be used for landing things related to GenGC. 9 1 10 2013-12-16 Daniel Bates <dabates@apple.com> 2 11 -
trunk/Source/WTF/wtf/Platform.h
r160725 r160796 785 785 #endif 786 786 787 /* Generational collector for JSC */ 788 #if !defined(ENABLE_GGC) 789 #define ENABLE_GGC 0 790 #endif 791 787 792 /* Counts uses of write barriers using sampling counters. Be sure to also 788 793 set ENABLE_SAMPLING_COUNTERS to 1. */
Note: See TracChangeset
for help on using the changeset viewer.