Changeset 223113 in webkit


Ignore:
Timestamp:
Oct 9, 2017 6:40:53 PM (7 years ago)
Author:
fpizlo@apple.com
Message:

Enable gigacage on iOS
​https://bugs.webkit.org/show_bug.cgi?id=177586

Reviewed by JF Bastien.
JSTests:


Add tests for when Gigacage gets runtime disabled.

  • stress/disable-gigacage-arrays.js: Added.

(foo):

  • stress/disable-gigacage-strings.js: Added.

(foo):

  • stress/disable-gigacage-typed-arrays.js: Added.

(foo):

Source/bmalloc:


Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
allocation fails, we free all of the gigacages and turn off gigacage support.

Reland this after confirming that the 20% Kraken regression was a one-bot fluke. Local testing on the
same kind of system did not show the regression. Saam and I both tried independently.

  • CMakeLists.txt:
  • bmalloc.xcodeproj/project.pbxproj:
  • bmalloc/Cache.cpp:

(bmalloc::Cache::scavenge):

  • bmalloc/Cache.h:

(bmalloc::Cache::tryAllocate):
(bmalloc::Cache::allocate):
(bmalloc::Cache::deallocate):
(bmalloc::Cache::reallocate):

  • bmalloc/Gigacage.cpp:

(Gigacage::ensureGigacage):
(Gigacage::runway):
(Gigacage::totalSize):
(Gigacage::shouldBeEnabled):
(): Deleted.
(Gigacage::Callback::Callback): Deleted.
(Gigacage::Callback::function): Deleted.
(Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.

  • bmalloc/Gigacage.h:

(Gigacage::wasEnabled):
(Gigacage::isEnabled):
(Gigacage::runway): Deleted.
(Gigacage::totalSize): Deleted.

  • bmalloc/HeapKind.cpp: Added.

(bmalloc::isActiveHeapKind):
(bmalloc::mapToActiveHeapKind):

  • bmalloc/HeapKind.h:

(bmalloc::isActiveHeapKindAfterEnsuringGigacage):
(bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):

  • bmalloc/Scavenger.cpp:

(bmalloc::Scavenger::scavenge):

  • bmalloc/bmalloc.h:

(bmalloc::api::tryLargeMemalignVirtual):
(bmalloc::api::freeLargeVirtual):
(bmalloc::api::isEnabled):

Source/JavaScriptCore:

The hardest part of enabling Gigacage on iOS is that it requires loading global variables while
executing JS, so the LLInt needs to know how to load from global variables on all platforms that
have Gigacage. So, this teaches ARM64 how to load from global variables.

Also, this makes the code handle disabling the gigacage a bit better.

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::caged):

  • jit/AssemblyHelpers.h:

(JSC::AssemblyHelpers::cage):
(JSC::AssemblyHelpers::cageConditionally):

  • offlineasm/arm64.rb:
  • offlineasm/asm.rb:
  • offlineasm/instructions.rb:

Tools:


Add a mode to test disabling Gigacage.

  • Scripts/run-jsc-stress-tests:
  • Scripts/webkitruby/jsc-stress-test-writer-default.rb:
Location:
trunk
Files:
4 added
21 edited

Legend:

Unmodified
Added
Removed

  • trunk/JSTests/ChangeLog

    r223081 r223113  
     12017-09-29  Filip Pizlo  <fpizlo@apple.com>
     2
     3        Enable gigacage on iOS
     4        https://bugs.webkit.org/show_bug.cgi?id=177586
     5
     6        Reviewed by JF Bastien.
     7       
     8        Add tests for when Gigacage gets runtime disabled.
     9
     10        * stress/disable-gigacage-arrays.js: Added.
     11        (foo):
     12        * stress/disable-gigacage-strings.js: Added.
     13        (foo):
     14        * stress/disable-gigacage-typed-arrays.js: Added.
     15        (foo):
     16
    1172017-10-09  Michael Saboff  <msaboff@apple.com>
    218

  • trunk/Source/JavaScriptCore/ChangeLog

    r223112 r223113  
     12017-09-29  Filip Pizlo  <fpizlo@apple.com>
     2
     3        Enable gigacage on iOS
     4        https://bugs.webkit.org/show_bug.cgi?id=177586
     5
     6        Reviewed by JF Bastien.
     7
     8        The hardest part of enabling Gigacage on iOS is that it requires loading global variables while
     9        executing JS, so the LLInt needs to know how to load from global variables on all platforms that
     10        have Gigacage. So, this teaches ARM64 how to load from global variables.
     11       
     12        Also, this makes the code handle disabling the gigacage a bit better.
     13
     14        * ftl/FTLLowerDFGToB3.cpp:
     15        (JSC::FTL::DFG::LowerDFGToB3::caged):
     16        * jit/AssemblyHelpers.h:
     17        (JSC::AssemblyHelpers::cage):
     18        (JSC::AssemblyHelpers::cageConditionally):
     19        * offlineasm/arm64.rb:
     20        * offlineasm/asm.rb:
     21        * offlineasm/instructions.rb:
     22
    1232017-10-09  Robin Morisset  <rmorisset@apple.com>
    224

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

    r223069 r223113  
    1199511995    LValue caged(Gigacage::Kind kind, LValue ptr)
    1199611996    {
    11997         if (!Gigacage::shouldBeEnabled())
     11997        if (!Gigacage::isEnabled(kind))
    1199811998            return ptr;
    1199911999       

  • trunk/Source/JavaScriptCore/jit/AssemblyHelpers.h

    r223069 r223113  
    13151315    {
    13161316#if GIGACAGE_ENABLED
    1317         if (!Gigacage::shouldBeEnabled())
     1317        if (!Gigacage::isEnabled(kind))
    13181318            return;
    13191319       
    … …  
    13291329    {
    13301330#if GIGACAGE_ENABLED
    1331         if (!Gigacage::shouldBeEnabled())
     1331        if (!Gigacage::isEnabled(kind))
    13321332            return;
    13331333       

  • trunk/Source/JavaScriptCore/offlineasm/arm64.rb

    r223069 r223113  
    261261end
    262262
     263def arm64LowerLabelReferences(list)
     264    newList = []
     265    list.each {
     266        | node |
     267        if node.is_a? Instruction
     268            case node.opcode
     269            when "loadi", "loadis", "loadp", "loadq", "loadb", "loadbs", "loadh", "loadhs"
     270                labelRef = node.operands[0]
     271                if labelRef.is_a? LabelReference
     272                    tmp = Tmp.new(node.codeOrigin, :gpr)
     273                    newList << Instruction.new(codeOrigin, "globaladdr", [LabelReference.new(node.codeOrigin, labelRef.label), tmp])
     274                    newList << Instruction.new(codeOrigin, node.opcode, [Address.new(node.codeOrigin, tmp, Immediate.new(node.codeOrigin, labelRef.offset)), node.operands[1]])
     275                else
     276                    newList << node
     277                end
     278            else
     279                newList << node
     280            end
     281        else
     282            newList << node
     283        end
     284    }
     285    newList
     286end
     287
    263288# Workaround for Cortex-A53 erratum (835769)
    264289def arm64CortexA53Fix835769(list)
    … …  
    297322        result = riscLowerShiftOps(result)
    298323        result = arm64LowerMalformedLoadStoreAddresses(result)
     324        result = arm64LowerLabelReferences(result)
    299325        result = riscLowerMalformedAddresses(result) {
    300326            | node, address |
    … …  
    905931            $asm.puts "nop"
    906932            $asm.putStr("#endif")
     933        when "globaladdr"
     934            uid = $asm.newUID
     935            $asm.puts "L_offlineasm_loh_adrp_#{uid}:"
     936            $asm.puts "adrp #{operands[1].arm64Operand(:ptr)}, #{operands[0].asmLabel}@GOTPAGE"
     937            $asm.puts "L_offlineasm_loh_ldr_#{uid}:"
     938            $asm.puts "ldr #{operands[1].arm64Operand(:ptr)}, [#{operands[1].arm64Operand(:ptr)}, #{operands[0].asmLabel}@GOTPAGEOFF]"
     939            $asm.deferAction {
     940                $asm.puts ".loh AdrpLdrGot L_offlineasm_loh_adrp_#{uid}, L_offlineasm_loh_ldr_#{uid}"
     941            }
    907942        else
    908943            lowerDefault

  • trunk/Source/JavaScriptCore/offlineasm/asm.rb

    r223069 r223113  
    4747        @numLocalLabels = 0
    4848        @numGlobalLabels = 0
     49        @deferredActions = []
     50        @count = 0
    4951
    5052        @newlineSpacerState = :none
    … …  
    7476        end
    7577        putsLastComment
     78        @deferredActions.each {
     79            | action |
     80            action.call()
     81        }
    7682        @outp.puts "OFFLINE_ASM_END" if !$emitWinAsm
    7783        @state = :cpp
     84    end
     85   
     86    def deferAction(&proc)
     87        @deferredActions << proc
     88    end
     89   
     90    def newUID
     91        @count += 1
     92        @count
    7893    end
    7994   

  • trunk/Source/JavaScriptCore/offlineasm/instructions.rb

    r223069 r223113  
    268268    [
    269269     "pcrtoaddr",   # Address from PC relative offset - adr instruction
    270      "nopFixCortexA53Err835769" # nop on Cortex-A53 (nothing otherwise)
     270     "nopFixCortexA53Err835769", # nop on Cortex-A53 (nothing otherwise)
     271     "globaladdr"
    271272    ]
    272273

  • trunk/Source/bmalloc/CMakeLists.txt

    r223069 r223113  
    1414    bmalloc/Gigacage.cpp
    1515    bmalloc/Heap.cpp
     16    bmalloc/HeapKind.cpp
    1617    bmalloc/LargeMap.cpp
    1718    bmalloc/Logging.cpp

  • trunk/Source/bmalloc/ChangeLog

    r223069 r223113  
     12017-09-29  Filip Pizlo  <fpizlo@apple.com>
     2
     3        Enable gigacage on iOS
     4        https://bugs.webkit.org/show_bug.cgi?id=177586
     5
     6        Reviewed by JF Bastien.
     7       
     8        Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
     9        allocation fails, we free all of the gigacages and turn off gigacage support.
     10       
     11        Reland this after confirming that the 20% Kraken regression was a one-bot fluke. Local testing on the
     12        same kind of system did not show the regression. Saam and I both tried independently.
     13
     14        * CMakeLists.txt:
     15        * bmalloc.xcodeproj/project.pbxproj:
     16        * bmalloc/Cache.cpp:
     17        (bmalloc::Cache::scavenge):
     18        * bmalloc/Cache.h:
     19        (bmalloc::Cache::tryAllocate):
     20        (bmalloc::Cache::allocate):
     21        (bmalloc::Cache::deallocate):
     22        (bmalloc::Cache::reallocate):
     23        * bmalloc/Gigacage.cpp:
     24        (Gigacage::ensureGigacage):
     25        (Gigacage::runway):
     26        (Gigacage::totalSize):
     27        (Gigacage::shouldBeEnabled):
     28        (): Deleted.
     29        (Gigacage::Callback::Callback): Deleted.
     30        (Gigacage::Callback::function): Deleted.
     31        (Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
     32        * bmalloc/Gigacage.h:
     33        (Gigacage::wasEnabled):
     34        (Gigacage::isEnabled):
     35        (Gigacage::runway): Deleted.
     36        (Gigacage::totalSize): Deleted.
     37        * bmalloc/HeapKind.cpp: Added.
     38        (bmalloc::isActiveHeapKind):
     39        (bmalloc::mapToActiveHeapKind):
     40        * bmalloc/HeapKind.h:
     41        (bmalloc::isActiveHeapKindAfterEnsuringGigacage):
     42        (bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
     43        * bmalloc/Scavenger.cpp:
     44        (bmalloc::Scavenger::scavenge):
     45        * bmalloc/bmalloc.h:
     46        (bmalloc::api::tryLargeMemalignVirtual):
     47        (bmalloc::api::freeLargeVirtual):
     48        (bmalloc::api::isEnabled):
     49
    1502017-10-09  Commit Queue  <commit-queue@webkit.org>
    251

  • trunk/Source/bmalloc/bmalloc.xcodeproj/project.pbxproj

    r223069 r223113  
    1616                0F5BF1531F22E1570029D91D /* Scavenger.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F5BF1511F22E1570029D91D /* Scavenger.h */; settings = {ATTRIBUTES = (Private, ); }; };
    1717                0F5BF1731F23C5710029D91D /* BExport.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F5BF1721F23C5710029D91D /* BExport.h */; settings = {ATTRIBUTES = (Private, ); }; };
     18                0FD557331F7EDB7B00B1F0A3 /* HeapKind.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FD557321F7EDB7B00B1F0A3 /* HeapKind.cpp */; };
    1819                1400274918F89C1300115C97 /* Heap.h in Headers */ = {isa = PBXBuildFile; fileRef = 14DA320C18875B09007269E0 /* Heap.h */; settings = {ATTRIBUTES = (Private, ); }; };
    1920                1400274A18F89C2300115C97 /* VMHeap.h in Headers */ = {isa = PBXBuildFile; fileRef = 144F7BFC18BFC517003537F3 /* VMHeap.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    9192                0F5BF1511F22E1570029D91D /* Scavenger.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = Scavenger.h; path = bmalloc/Scavenger.h; sourceTree = "<group>"; };
    9293                0F5BF1721F23C5710029D91D /* BExport.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = BExport.h; path = bmalloc/BExport.h; sourceTree = "<group>"; };
     94                0FD557321F7EDB7B00B1F0A3 /* HeapKind.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = HeapKind.cpp; path = bmalloc/HeapKind.cpp; sourceTree = "<group>"; };
    9395                140FA00219CE429C00FFD3C8 /* BumpRange.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = BumpRange.h; path = bmalloc/BumpRange.h; sourceTree = "<group>"; };
    9496                140FA00419CE4B6800FFD3C8 /* LineMetadata.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = LineMetadata.h; path = bmalloc/LineMetadata.h; sourceTree = "<group>"; };
    … …  
    288290                                14C919C818FCC59F0028DB43 /* BPlatform.h */,
    289291                                14D9DB4517F2447100EAAB79 /* FixedVector.h */,
     292                                0FD557321F7EDB7B00B1F0A3 /* HeapKind.cpp */,
    290293                                0F5BF1461F22A8B10029D91D /* HeapKind.h */,
    291294                                141D9AFF1C8E51C0000ABBA0 /* List.h */,
    … …  
    457460                        files = (
    458461                                0F5BF1521F22E1570029D91D /* Scavenger.cpp in Sources */,
     462                                0FD557331F7EDB7B00B1F0A3 /* HeapKind.cpp in Sources */,
    459463                                14F271C318EA3978008C152F /* Allocator.cpp in Sources */,
    460464                                6599C5CC1EC3F15900A2F7BB /* AvailableMemory.cpp in Sources */,

  • trunk/Source/bmalloc/bmalloc/Cache.cpp

    r223069 r223113  
    3636    if (!caches)
    3737        return;
     38    if (!isActiveHeapKind(heapKind))
     39        return;
    3840
    3941    caches->at(heapKind).allocator().scavenge();
    … …  
    4951BNO_INLINE void* Cache::tryAllocateSlowCaseNullCache(HeapKind heapKind, size_t size)
    5052{
    51     return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().tryAllocate(size);
     53    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().tryAllocate(size);
    5254}
    5355
    5456BNO_INLINE void* Cache::allocateSlowCaseNullCache(HeapKind heapKind, size_t size)
    5557{
    56     return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().allocate(size);
     58    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().allocate(size);
    5759}
    5860
    5961BNO_INLINE void* Cache::allocateSlowCaseNullCache(HeapKind heapKind, size_t alignment, size_t size)
    6062{
    61     return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().allocate(alignment, size);
     63    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().allocate(alignment, size);
    6264}
    6365
    6466BNO_INLINE void Cache::deallocateSlowCaseNullCache(HeapKind heapKind, void* object)
    6567{
    66     PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).deallocator().deallocate(object);
     68    PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).deallocator().deallocate(object);
    6769}
    6870
    6971BNO_INLINE void* Cache::reallocateSlowCaseNullCache(HeapKind heapKind, void* object, size_t newSize)
    7072{
    71     return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(heapKind).allocator().reallocate(object, newSize);
     73    return PerThread<PerHeapKind<Cache>>::getSlowCase()->at(mapToActiveHeapKind(heapKind)).allocator().reallocate(object, newSize);
    7274}
    7375

  • trunk/Source/bmalloc/bmalloc/Cache.h

    r223069 r223113  
    6969    if (!caches)
    7070        return tryAllocateSlowCaseNullCache(heapKind, size);
    71     return caches->at(heapKind).allocator().tryAllocate(size);
     71    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().tryAllocate(size);
    7272}
    7373
    … …  
    7777    if (!caches)
    7878        return allocateSlowCaseNullCache(heapKind, size);
    79     return caches->at(heapKind).allocator().allocate(size);
     79    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().allocate(size);
    8080}
    8181
    … …  
    8585    if (!caches)
    8686        return allocateSlowCaseNullCache(heapKind, alignment, size);
    87     return caches->at(heapKind).allocator().tryAllocate(alignment, size);
     87    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().tryAllocate(alignment, size);
    8888}
    8989
    … …  
    9393    if (!caches)
    9494        return allocateSlowCaseNullCache(heapKind, alignment, size);
    95     return caches->at(heapKind).allocator().allocate(alignment, size);
     95    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().allocate(alignment, size);
    9696}
    9797
    … …  
    101101    if (!caches)
    102102        return deallocateSlowCaseNullCache(heapKind, object);
    103     return caches->at(heapKind).deallocator().deallocate(object);
     103    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).deallocator().deallocate(object);
    104104}
    105105
    … …  
    109109    if (!caches)
    110110        return reallocateSlowCaseNullCache(heapKind, object, newSize);
    111     return caches->at(heapKind).allocator().reallocate(object, newSize);
     111    return caches->at(mapToActiveHeapKindAfterEnsuringGigacage(heapKind)).allocator().reallocate(object, newSize);
    112112}
    113113

  • trunk/Source/bmalloc/bmalloc/Gigacage.cpp

    r223069 r223113  
    3434#include <mutex>
    3535
     36#if BCPU(ARM64)
     37// FIXME: There is no good reason for ARM64 to be special.
     38// https://bugs.webkit.org/show_bug.cgi?id=177605
     39#define PRIMITIVE_GIGACAGE_RUNWAY 0
     40#else
     41// FIXME: Consider making this 32GB, in case unsigned 32-bit indices find their way into indexed accesses.
     42// https://bugs.webkit.org/show_bug.cgi?id=175062
     43#define PRIMITIVE_GIGACAGE_RUNWAY (16llu * 1024 * 1024 * 1024)
     44#endif
     45
     46// FIXME: Reconsider this.
     47// https://bugs.webkit.org/show_bug.cgi?id=175921
     48#define JSVALUE_GIGACAGE_RUNWAY 0
     49#define STRING_GIGACAGE_RUNWAY 0
     50
    3651char g_gigacageBasePtrs[GIGACAGE_BASE_PTRS_SIZE] __attribute__((aligned(GIGACAGE_BASE_PTRS_SIZE)));
    3752
    … …  
    3954
    4055namespace Gigacage {
     56
     57bool g_wasEnabled;
    4158
    4259namespace {
    … …  
    7087};
    7188
    72 } // anonymous namespce
    73 
    7489struct Callback {
    7590    Callback() { }
    … …  
    90105    Vector<Callback> callbacks;
    91106};
     107
     108} // anonymous namespace
    92109
    93110void ensureGigacage()
    … …  
    101118                return;
    102119           
     120            bool ok = true;
     121           
    103122            forEachKind(
    104123                [&] (Kind kind) {
     124                    if (!ok)
     125                        return;
    105126                    // FIXME: Randomize where this goes.
    106127                    // https://bugs.webkit.org/show_bug.cgi?id=175245
    107128                    basePtr(kind) = tryVMAllocate(alignment(kind), totalSize(kind));
    108129                    if (!basePtr(kind)) {
     130                        if (GIGACAGE_ALLOCATION_CAN_FAIL) {
     131                            ok = false;
     132                            return;
     133                        }
    109134                        fprintf(stderr, "FATAL: Could not allocate %s gigacage.\n", name(kind));
    110135                        BCRASH();
    … …  
    114139                });
    115140           
     141            if (!ok) {
     142                forEachKind(
     143                    [&] (Kind kind) {
     144                        if (!basePtr(kind))
     145                            return;
     146                       
     147                        vmDeallocate(basePtr(kind), totalSize(kind));
     148                       
     149                        basePtr(kind) = nullptr;
     150                    });
     151                return;
     152            }
     153           
    116154            protectGigacageBasePtrs();
     155            g_wasEnabled = true;
    117156        });
    118157#endif // GIGACAGE_ENABLED
     158}
     159
     160size_t runway(Kind kind)
     161{
     162    switch (kind) {
     163    case Primitive:
     164        return static_cast<size_t>(PRIMITIVE_GIGACAGE_RUNWAY);
     165    case JSValue:
     166        return static_cast<size_t>(JSVALUE_GIGACAGE_RUNWAY);
     167    case String:
     168        return static_cast<size_t>(STRING_GIGACAGE_RUNWAY);
     169    }
     170    BCRASH();
     171    return 0;
     172}
     173
     174size_t totalSize(Kind kind)
     175{
     176    return size(kind) + runway(kind);
    119177}
    120178
    … …  
    188246bool shouldBeEnabled()
    189247{
    190     return GIGACAGE_ENABLED && !PerProcess<Environment>::get()->isDebugHeapEnabled();
     248    static std::once_flag onceFlag;
     249    static bool cached;
     250    std::call_once(
     251        onceFlag,
     252        [] {
     253            bool result = GIGACAGE_ENABLED && !PerProcess<Environment>::get()->isDebugHeapEnabled();
     254            if (!result)
     255                return;
     256           
     257            if (char* gigacageEnabled = getenv("GIGACAGE_ENABLED")) {
     258                if (!strcasecmp(gigacageEnabled, "no") || !strcasecmp(gigacageEnabled, "false") || !strcasecmp(gigacageEnabled, "0")) {
     259                    fprintf(stderr, "Warning: disabling gigacage because GIGACAGE_ENABLED=%s!\n", gigacageEnabled);
     260                    return;
     261                } else if (strcasecmp(gigacageEnabled, "yes") && strcasecmp(gigacageEnabled, "true") && strcasecmp(gigacageEnabled, "1"))
     262                    fprintf(stderr, "Warning: invalid argument to GIGACAGE_ENABLED: %s\n", gigacageEnabled);
     263            }
     264           
     265            cached = true;
     266        });
     267   
     268    return cached;
    191269}
    192270

  • trunk/Source/bmalloc/bmalloc/Gigacage.h

    r223069 r223113  
    3333#include <inttypes.h>
    3434
     35#if BCPU(ARM64)
     36// FIXME: This can probably be a lot bigger on iOS. I just haven't tried to make it bigger yet.
     37// https://bugs.webkit.org/show_bug.cgi?id=177605
     38#define PRIMITIVE_GIGACAGE_SIZE 0x40000000llu
     39#define JSVALUE_GIGACAGE_SIZE 0x40000000llu
     40#define STRING_GIGACAGE_SIZE 0x40000000llu
     41#define GIGACAGE_ALLOCATION_CAN_FAIL 1
     42#else
    3543#define PRIMITIVE_GIGACAGE_SIZE 0x800000000llu
    3644#define JSVALUE_GIGACAGE_SIZE 0x400000000llu
    3745#define STRING_GIGACAGE_SIZE 0x400000000llu
     46#define GIGACAGE_ALLOCATION_CAN_FAIL 0
     47#endif
    3848
    3949#define GIGACAGE_SIZE_TO_MASK(size) ((size) - 1)
    … …  
    4353#define STRING_GIGACAGE_MASK GIGACAGE_SIZE_TO_MASK(STRING_GIGACAGE_SIZE)
    4454
    45 // FIXME: Consider making this 32GB, in case unsigned 32-bit indices find their way into indexed accesses.
    46 // https://bugs.webkit.org/show_bug.cgi?id=175062
    47 #define PRIMITIVE_GIGACAGE_RUNWAY (16llu * 1024 * 1024 * 1024)
    48 
    49 // FIXME: Reconsider this.
    50 // https://bugs.webkit.org/show_bug.cgi?id=175921
    51 #define JSVALUE_GIGACAGE_RUNWAY 0
    52 #define STRING_GIGACAGE_RUNWAY 0
    53 
    54 #if (BOS(DARWIN) || BOS(LINUX)) && BCPU(X86_64)
     55#if (BOS(DARWIN) && (BCPU(ARM64) || BCPU(X86_64))) || (BOS(LINUX) && BCPU(X86_64))
    5556#define GIGACAGE_ENABLED 1
    5657#else
    … …  
    5859#endif
    5960
    60 #define GIGACAGE_BASE_PTRS_SIZE 8192
     61#if BCPU(ARM64)
     62#define GIGACAGE_BASE_PTRS_SIZE 16384
     63#else
     64#define GIGACAGE_BASE_PTRS_SIZE 4096
     65#endif
    6166
    6267extern "C" BEXPORT char g_gigacageBasePtrs[GIGACAGE_BASE_PTRS_SIZE] __attribute__((aligned(GIGACAGE_BASE_PTRS_SIZE)));
    6368
    6469namespace Gigacage {
     70
     71extern BEXPORT bool g_wasEnabled;
     72BINLINE bool wasEnabled() { return g_wasEnabled; }
    6573
    6674struct BasePtrs {
    … …  
    128136}
    129137
     138BINLINE bool isEnabled(Kind kind)
     139{
     140    return !!basePtr(kind);
     141}
     142
    130143BINLINE size_t size(Kind kind)
    131144{
    … …  
    152165}
    153166
    154 BINLINE size_t runway(Kind kind)
    155 {
    156     switch (kind) {
    157     case Primitive:
    158         return static_cast<size_t>(PRIMITIVE_GIGACAGE_RUNWAY);
    159     case JSValue:
    160         return static_cast<size_t>(JSVALUE_GIGACAGE_RUNWAY);
    161     case String:
    162         return static_cast<size_t>(STRING_GIGACAGE_RUNWAY);
    163     }
    164     BCRASH();
    165     return 0;
    166 }
    167 
    168 BINLINE size_t totalSize(Kind kind)
    169 {
    170     return size(kind) + runway(kind);
    171 }
     167size_t runway(Kind kind);
     168size_t totalSize(Kind kind);
    172169
    173170template<typename Func>

  • trunk/Source/bmalloc/bmalloc/Heap.cpp

    r223069 r223113  
    178178void Heap::allocateSmallChunk(std::lock_guard<StaticMutex>& lock, size_t pageClass)
    179179{
     180    RELEASE_BASSERT(isActiveHeapKind(m_kind));
     181   
    180182    size_t pageSize = bmalloc::pageSize(pageClass);
    181183
    … …  
    222224SmallPage* Heap::allocateSmallPage(std::lock_guard<StaticMutex>& lock, size_t sizeClass, LineCache& lineCache)
    223225{
     226    RELEASE_BASSERT(isActiveHeapKind(m_kind));
     227
    224228    if (!lineCache[sizeClass].isEmpty())
    225229        return lineCache[sizeClass].popFront();
    … …  
    301305    LineCache& lineCache)
    302306{
     307    RELEASE_BASSERT(isActiveHeapKind(m_kind));
     308
    303309    SmallPage* page = allocateSmallPage(lock, sizeClass, lineCache);
    304310    SmallLine* lines = page->begin();
    … …  
    363369    LineCache& lineCache)
    364370{
     371    RELEASE_BASSERT(isActiveHeapKind(m_kind));
     372
    365373    size_t size = allocator.size();
    366374    SmallPage* page = allocateSmallPage(lock, sizeClass, lineCache);
    … …  
    415423LargeRange Heap::splitAndAllocate(LargeRange& range, size_t alignment, size_t size, AllocationKind allocationKind)
    416424{
     425    RELEASE_BASSERT(isActiveHeapKind(m_kind));
     426
    417427    LargeRange prev;
    418428    LargeRange next;
    … …  
    462472void* Heap::tryAllocateLarge(std::lock_guard<StaticMutex>&, size_t alignment, size_t size, AllocationKind allocationKind)
    463473{
     474    RELEASE_BASSERT(isActiveHeapKind(m_kind));
     475
    464476    BASSERT(isPowerOfTwo(alignment));
    465477   

  • trunk/Source/bmalloc/bmalloc/HeapKind.h

    r223069 r223113  
    8686}
    8787
     88BINLINE bool isActiveHeapKindAfterEnsuringGigacage(HeapKind kind)
     89{
     90    switch (kind) {
     91    case HeapKind::PrimitiveGigacage:
     92    case HeapKind::JSValueGigacage:
     93    case HeapKind::StringGigacage:
     94        if (Gigacage::wasEnabled())
     95            return true;
     96        return false;
     97    default:
     98        return true;
     99    }
     100}
     101
     102BEXPORT bool isActiveHeapKind(HeapKind);
     103
     104BINLINE HeapKind mapToActiveHeapKindAfterEnsuringGigacage(HeapKind kind)
     105{
     106    switch (kind) {
     107    case HeapKind::PrimitiveGigacage:
     108    case HeapKind::JSValueGigacage:
     109    case HeapKind::StringGigacage:
     110        if (Gigacage::wasEnabled())
     111            return kind;
     112        return HeapKind::Primary;
     113    default:
     114        return kind;
     115    }
     116}
     117
     118BEXPORT HeapKind mapToActiveHeapKind(HeapKind);
     119
    88120} // namespace bmalloc
    89121

  • trunk/Source/bmalloc/bmalloc/Scavenger.cpp

    r223069 r223113  
    118118{
    119119    std::lock_guard<StaticMutex> lock(Heap::mutex());
    120     for (unsigned i = numHeaps; i--;)
     120    for (unsigned i = numHeaps; i--;) {
     121        if (!isActiveHeapKind(static_cast<HeapKind>(i)))
     122            continue;
    121123        PerProcess<PerHeapKind<Heap>>::get()->at(i).scavenge(lock);
     124    }
    122125}
    123126

  • trunk/Source/bmalloc/bmalloc/bmalloc.h

    r223069 r223113  
    6969inline void* tryLargeMemalignVirtual(size_t alignment, size_t size, HeapKind kind = HeapKind::Primary)
    7070{
     71    kind = mapToActiveHeapKind(kind);
    7172    Heap& heap = PerProcess<PerHeapKind<Heap>>::get()->at(kind);
    7273    std::lock_guard<StaticMutex> lock(Heap::mutex());
    … …  
    8182inline void freeLargeVirtual(void* object, HeapKind kind = HeapKind::Primary)
    8283{
     84    kind = mapToActiveHeapKind(kind);
    8385    Heap& heap = PerProcess<PerHeapKind<Heap>>::get()->at(kind);
    8486    std::lock_guard<StaticMutex> lock(Heap::mutex());
    … …  
    101103inline bool isEnabled(HeapKind kind = HeapKind::Primary)
    102104{
     105    kind = mapToActiveHeapKind(kind);
    103106    std::unique_lock<StaticMutex> lock(Heap::mutex());
    104107    return !PerProcess<PerHeapKind<Heap>>::getFastCase()->at(kind).debugHeap();

  • trunk/Tools/ChangeLog

    r223069 r223113  
     12017-09-29  Filip Pizlo  <fpizlo@apple.com>
     2
     3        Enable gigacage on iOS
     4        https://bugs.webkit.org/show_bug.cgi?id=177586
     5
     6        Reviewed by JF Bastien.
     7       
     8        Add a mode to test disabling Gigacage.
     9
     10        * Scripts/run-jsc-stress-tests:
     11        * Scripts/webkitruby/jsc-stress-test-writer-default.rb:
     12
    1132017-10-09  Commit Queue  <commit-queue@webkit.org>
    214

  • trunk/Tools/Scripts/run-jsc-stress-tests

    r223069 r223113  
    514514end
    515515
    516 def addRunCommand(kind, command, outputHandler, errorHandler)
     516def addRunCommand(kind, command, outputHandler, errorHandler, *additionalEnv)
    517517    $didAddRunCommand = true
    518518    name = baseOutputName(kind)
    … …  
    523523        $benchmarkDirectory, command, "#{$collectionName}/#{$benchmark}", name, outputHandler,
    524524        errorHandler)
     525    plan.additionalEnv.push(*additionalEnv)
    525526    if $numChildProcesses > 1 and $runCommandOptions[:isSlow]
    526527        $runlist.unshift plan
    … …  
    13081309end
    13091310
     1311def runNoisyTestImpl(kind, options, additionalEnv)
     1312    addRunCommand(kind, [pathToVM.to_s] + BASE_OPTIONS + options + [$benchmark.to_s], noisyOutputHandler, noisyErrorHandler, *additionalEnv)
     1313end
     1314
    13101315def runNoisyTest(kind, *options)
    1311     addRunCommand(kind, [pathToVM.to_s] + BASE_OPTIONS + options + [$benchmark.to_s], noisyOutputHandler, noisyErrorHandler)
     1316    runNoisyTestImpl(kind, options, [])
     1317end
     1318
     1319def runNoisyTestWithEnv(kind, *additionalEnv)
     1320    runNoisyTestImpl(kind, [], additionalEnv)
    13121321end
    13131322

  • trunk/Tools/Scripts/webkitruby/jsc-stress-test-writer-default.rb

    r223069 r223113  
    216216
    217217class Plan
    218     attr_reader :directory, :arguments, :family, :name, :outputHandler, :errorHandler
     218    attr_reader :directory, :arguments, :family, :name, :outputHandler, :errorHandler, :additionalEnv
    219219    attr_accessor :index
    220220   
    … …  
    227227        @errorHandler = errorHandler
    228228        @isSlow = !!$runCommandOptions[:isSlow]
     229        @additionalEnv = []
    229230    end
    230231   
    … …  
    234235        # have to bend over backwards to do things relative to the root.
    235236        script = "(cd ../#{Shellwords.shellescape(@directory.to_s)} && ("
    236         $envVars.each { |var| script += "export " << var << "; " }
     237        ($envVars + additionalEnv).each { |var| script += "export " << var << "; " }
    237238        script += "\"$@\" " + escapeAll(@arguments) + "))"
    238239        return script
Note: See TracChangeset for help on using the changeset viewer.