Changeset 147324 in webkit


Ignore:
Timestamp:
Apr 1, 2013 8:23:49 AM (11 years ago)
Author:
mhahnenberg@apple.com
Message:

Regions should be allocated from the same contiguous segment of virtual memory
​https://bugs.webkit.org/show_bug.cgi?id=113662

Reviewed by Filip Pizlo.

Instead of letting the OS spread our Regions all over the place, we should allocate them all within
some range of each other. This change will open the door to some other optimizations, e.g. doing simple
range checks for our write barriers and compressing JSCell pointers to 32-bits.

Source/JavaScriptCore:

Added new SuperRegion class that encapsulates allocating Regions from a contiguous reserved chunk of
virtual address space. It functions very similarly to the FixedVMPoolExecutableAllocator class used by the JIT.

Also added two new subclasses of Region, NormalRegion and ExcessRegion.

NormalRegion is the type of Region that is normally allocated when there is available space remaining
in the SuperRegion. If we ever run out of space in the SuperRegion, we fall back to allocating
ExcessRegions, which are identical to how Regions have behaved up until now, i.e. they contain a
PageAllocationAligned.

We only use the SuperRegion (and NormalRegions) on 64-bit systems, since it doesn't make sense to reserve the
entire 4 GB address space on 32-bit systems just for the JS heap.

(JSC::BlockAllocator::BlockAllocator):

  • heap/BlockAllocator.h:

(JSC):
(BlockAllocator):
(JSC::BlockAllocator::allocate):
(JSC::BlockAllocator::allocateCustomSize):
(JSC::BlockAllocator::deallocateCustomSize):

  • heap/Heap.cpp:

(JSC::Heap::Heap):
(JSC):
(JSC::Heap::didExceedFixedHeapSizeLimit):

  • heap/Heap.h:

(Heap):

  • heap/MarkedBlock.cpp:

(JSC::MarkedBlock::create):

  • heap/Region.h:

(Region):
(JSC):
(NormalRegion):
(JSC::NormalRegion::base):
(JSC::NormalRegion::size):
(ExcessRegion):
(JSC::ExcessRegion::base):
(JSC::ExcessRegion::size):
(JSC::NormalRegion::NormalRegion):
(JSC::NormalRegion::tryCreate):
(JSC::NormalRegion::tryCreateCustomSize):
(JSC::NormalRegion::reset):
(JSC::ExcessRegion::ExcessRegion):
(JSC::ExcessRegion::~ExcessRegion):
(JSC::ExcessRegion::create):
(JSC::ExcessRegion::createCustomSize):
(JSC::ExcessRegion::reset):
(JSC::Region::Region):
(JSC::Region::initializeBlockList):
(JSC::Region::create):
(JSC::Region::createCustomSize):
(JSC::Region::~Region):
(JSC::Region::destroy):
(JSC::Region::reset):
(JSC::Region::deallocate):
(JSC::Region::base):
(JSC::Region::size):

  • heap/SuperRegion.cpp: Added.

(JSC):
(JSC::SuperRegion::SuperRegion):
(JSC::SuperRegion::getAlignedBase):
(JSC::SuperRegion::allocateNewSpace):
(JSC::SuperRegion::notifyNeedPage):
(JSC::SuperRegion::notifyPageIsFree):

  • heap/SuperRegion.h: Added.

(JSC):
(SuperRegion):

Source/WTF:

  • wtf/MetaAllocator.cpp: Changed the MetaAllocator to allow custom page sizes if the derived class wants to

use something other than the system page size.
(WTF::MetaAllocator::MetaAllocator):

  • wtf/MetaAllocator.h:

(MetaAllocator):

Location:
trunk/Source
Files:
2 added
18 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/CMakeLists.txt

    r146494 r147324  
    140140    heap/IncrementalSweeper.cpp
    141141    heap/JITStubRoutineSet.cpp
    142     heap/MachineStackMarker.cpp
     142    heap/MachineStackMarker.cppp/MarkedAllocator.cpp
     143
    143144    heap/MarkedAllocator.cpp
    144145    heap/MarkedBlock.cpp
    … …  
    146147    heap/MarkStack.cpp
    147148    heap/SlotVisitor.cpp
     149    heap/SuperRegion.cpp
    148150    heap/WeakSet.cpp
    149151    heap/WeakHandleOwner.cpp

  • trunk/Source/JavaScriptCore/ChangeLog

    r147316 r147324  
     12013-03-31  Mark Hahnenberg  <mhahnenberg@apple.com>
     2
     3        Regions should be allocated from the same contiguous segment of virtual memory
     4        https://bugs.webkit.org/show_bug.cgi?id=113662
     5
     6        Reviewed by Filip Pizlo.
     7
     8        Instead of letting the OS spread our Regions all over the place, we should allocate them all within
     9        some range of each other. This change will open the door to some other optimizations, e.g. doing simple
     10        range checks for our write barriers and compressing JSCell pointers to 32-bits.
     11
     12        Added new SuperRegion class that encapsulates allocating Regions from a contiguous reserved chunk of
     13        virtual address space. It functions very similarly to the FixedVMPoolExecutableAllocator class used by the JIT.
     14
     15        Also added two new subclasses of Region, NormalRegion and ExcessRegion.
     16       
     17        NormalRegion is the type of Region that is normally allocated when there is available space remaining
     18        in the SuperRegion. If we ever run out of space in the SuperRegion, we fall back to allocating
     19        ExcessRegions, which are identical to how Regions have behaved up until now, i.e. they contain a
     20        PageAllocationAligned.
     21
     22        We only use the SuperRegion (and NormalRegions) on 64-bit systems, since it doesn't make sense to reserve the
     23        entire 4 GB address space on 32-bit systems just for the JS heap.
     24
     25        * GNUmakefile.list.am:
     26        * JavaScriptCore.gypi:
     27        * JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj:
     28        * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj:
     29        * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters:
     30        * JavaScriptCore.xcodeproj/project.pbxproj:
     31        * Target.pri:
     32        * heap/BlockAllocator.cpp:
     33        (JSC::BlockAllocator::BlockAllocator):
     34        * heap/BlockAllocator.h:
     35        (JSC):
     36        (BlockAllocator):
     37        (JSC::BlockAllocator::allocate):
     38        (JSC::BlockAllocator::allocateCustomSize):
     39        (JSC::BlockAllocator::deallocateCustomSize):
     40        * heap/Heap.cpp:
     41        (JSC::Heap::Heap):
     42        (JSC):
     43        (JSC::Heap::didExceedFixedHeapSizeLimit):
     44        * heap/Heap.h:
     45        (Heap):
     46        * heap/MarkedBlock.cpp:
     47        (JSC::MarkedBlock::create):
     48        * heap/Region.h:
     49        (Region):
     50        (JSC):
     51        (NormalRegion):
     52        (JSC::NormalRegion::base):
     53        (JSC::NormalRegion::size):
     54        (ExcessRegion):
     55        (JSC::ExcessRegion::base):
     56        (JSC::ExcessRegion::size):
     57        (JSC::NormalRegion::NormalRegion):
     58        (JSC::NormalRegion::tryCreate):
     59        (JSC::NormalRegion::tryCreateCustomSize):
     60        (JSC::NormalRegion::reset):
     61        (JSC::ExcessRegion::ExcessRegion):
     62        (JSC::ExcessRegion::~ExcessRegion):
     63        (JSC::ExcessRegion::create):
     64        (JSC::ExcessRegion::createCustomSize):
     65        (JSC::ExcessRegion::reset):
     66        (JSC::Region::Region):
     67        (JSC::Region::initializeBlockList):
     68        (JSC::Region::create):
     69        (JSC::Region::createCustomSize):
     70        (JSC::Region::~Region):
     71        (JSC::Region::destroy):
     72        (JSC::Region::reset):
     73        (JSC::Region::deallocate):
     74        (JSC::Region::base):
     75        (JSC::Region::size):
     76        * heap/SuperRegion.cpp: Added.
     77        (JSC):
     78        (JSC::SuperRegion::SuperRegion):
     79        (JSC::SuperRegion::getAlignedBase):
     80        (JSC::SuperRegion::allocateNewSpace):
     81        (JSC::SuperRegion::notifyNeedPage):
     82        (JSC::SuperRegion::notifyPageIsFree):
     83        * heap/SuperRegion.h: Added.
     84        (JSC):
     85        (SuperRegion):
     86
    1872013-04-01  Benjamin Poulain  <benjamin@webkit.org>
    288

  • trunk/Source/JavaScriptCore/GNUmakefile.list.am

    r147282 r147324  
    360360        Source/JavaScriptCore/heap/Strong.h \
    361361        Source/JavaScriptCore/heap/StrongInlines.h \
     362    Source/JavaScriptCore/heap/SuperRegion.cpp \
     363    Source/JavaScriptCore/heap/SuperRegion.h \
    362364        Source/JavaScriptCore/heap/UnconditionalFinalizer.h \
    363365        Source/JavaScriptCore/heap/VTableSpectrum.cpp \

  • trunk/Source/JavaScriptCore/JavaScriptCore.gypi

    r147294 r147324  
    394394            'heap/Strong.h',
    395395            'heap/StrongInlines.h',
     396            'heap/SuperRegion.cpp',
     397            'heap/SuperRegion.h',
    396398            'heap/TinyBloomFilter.h',
    397399            'heap/UnconditionalFinalizer.h',

  • trunk/Source/JavaScriptCore/JavaScriptCore.vcproj/JavaScriptCore/JavaScriptCore.vcproj

    r147282 r147324  
    27952795                        </File>
    27962796                        <File
     2797                                RelativePath="..\..\heap\SuperRegion.cpp"
     2798                                >
     2799                        </File>
     2800                        <File
     2801                                RelativePath="..\..\heap\SuperRegion.h"
     2802                                >
     2803                        </File>
     2804                        <File
    27972805                                RelativePath="..\..\heap\VTableSpectrum.cpp"
    27982806                                >

  • trunk/Source/JavaScriptCore/JavaScriptCore.vcxproj/JavaScriptCore.vcxproj

    r147282 r147324  
    212212    <ClCompile Include="..\heap\MarkStack.cpp" />
    213213    <ClCompile Include="..\heap\SlotVisitor.cpp" />
     214    <ClInclude Include="..\heap\SuperRegion.cpp" />
    214215    <ClCompile Include="..\heap\VTableSpectrum.cpp" />
    215216    <ClCompile Include="..\heap\WeakBlock.cpp" />
    … …  
    536537    <ClInclude Include="..\heap\Strong.h" />
    537538    <ClInclude Include="..\heap\StrongInlines.h" />
     539    <ClInclude Include="..\heap\SuperRegion.h" />
    538540    <ClInclude Include="..\heap\TinyBloomFilter.h" />
    539541    <ClInclude Include="..\heap\UnconditionalFinalizer.h" />

  • trunk/Source/JavaScriptCore/JavaScriptCore.vcxproj/JavaScriptCore.vcxproj.filters

    r147282 r147324  
    256256      <Filter>heap</Filter>
    257257    </ClCompile>
     258    <ClCompile Include="..\heap\SuperRegion.cpp">
     259      <Filter>heap</Filter>
     260    </ClCompile>
    258261    <ClCompile Include="..\heap\VTableSpectrum.cpp">
    259262      <Filter>heap</Filter>
    … …  
    11551158      <Filter>heap</Filter>
    11561159    </ClInclude>
     1160    <ClCompile Include="..\heap\SuperRegion.h">
     1161      <Filter>heap</Filter>
     1162    </ClCompile>
    11571163    <ClInclude Include="..\heap\TinyBloomFilter.h">
    11581164      <Filter>heap</Filter>

  • trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

    r147282 r147324  
    850850                C2CF39C216E15A8100DD69BE /* JSAPIWrapperObject.h in Headers */ = {isa = PBXBuildFile; fileRef = C2CF39C016E15A8100DD69BE /* JSAPIWrapperObject.h */; };
    851851                C2D58C3415912FEE0021A844 /* GCActivityCallback.cpp in Sources */ = {isa = PBXBuildFile; fileRef = C2D58C3315912FEE0021A844 /* GCActivityCallback.cpp */; };
     852                C2DF442F1707AC0100A5CA96 /* SuperRegion.cpp in Sources */ = {isa = PBXBuildFile; fileRef = C2DF442D1707AC0100A5CA96 /* SuperRegion.cpp */; };
     853                C2DF44301707AC0100A5CA96 /* SuperRegion.h in Headers */ = {isa = PBXBuildFile; fileRef = C2DF442E1707AC0100A5CA96 /* SuperRegion.h */; settings = {ATTRIBUTES = (Private, ); }; };
    852854                C2E526BD1590EF000054E48D /* HeapTimer.cpp in Sources */ = {isa = PBXBuildFile; fileRef = C2E526BB1590EF000054E48D /* HeapTimer.cpp */; };
    853855                C2E526BE1590EF000054E48D /* HeapTimer.h in Headers */ = {isa = PBXBuildFile; fileRef = C2E526BC1590EF000054E48D /* HeapTimer.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    17551757                C2CF39C016E15A8100DD69BE /* JSAPIWrapperObject.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSAPIWrapperObject.h; sourceTree = "<group>"; };
    17561758                C2D58C3315912FEE0021A844 /* GCActivityCallback.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = GCActivityCallback.cpp; sourceTree = "<group>"; };
     1759                C2DF442D1707AC0100A5CA96 /* SuperRegion.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SuperRegion.cpp; sourceTree = "<group>"; };
     1760                C2DF442E1707AC0100A5CA96 /* SuperRegion.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SuperRegion.h; sourceTree = "<group>"; };
    17571761                C2E526BB1590EF000054E48D /* HeapTimer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HeapTimer.cpp; sourceTree = "<group>"; };
    17581762                C2E526BC1590EF000054E48D /* HeapTimer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HeapTimer.h; sourceTree = "<group>"; };
    … …  
    21672171                                C283190116FE533E00157BFD /* HandleBlockInlines.h */,
    21682172                                C20B25981706536200C21F4E /* Region.h */,
     2173                                C2DF442D1707AC0100A5CA96 /* SuperRegion.cpp */,
     2174                                C2DF442E1707AC0100A5CA96 /* SuperRegion.h */,
    21692175                        );
    21702176                        path = heap;
    … …  
    29662972                                0F63945515D07057006A597C /* ArrayProfile.h in Headers */,
    29672973                                BC18C3E70E16F5CD00B34460 /* ArrayPrototype.h in Headers */,
     2974                                C2DF44301707AC0100A5CA96 /* SuperRegion.h in Headers */,
    29682975                                C20B25991706536200C21F4E /* Region.h in Headers */,
    29692976                                C283190016FE4B7D00157BFD /* HandleBlock.h in Headers */,
    … …  
    38523859                                0FDDBFB51666EED800C55FEF /* DFGVariableAccessDataDump.cpp in Sources */,
    38533860                                0F2BDC5115228FFD00CD8910 /* DFGVariableEvent.cpp in Sources */,
     3861                                C2DF442F1707AC0100A5CA96 /* SuperRegion.cpp in Sources */,
    38543862                                0F2BDC4A1522809A00CD8910 /* DFGVariableEventStream.cpp in Sources */,
    38553863                                0FFFC95F14EF90BB00C72532 /* DFGVirtualRegisterAllocationPhase.cpp in Sources */,

  • trunk/Source/JavaScriptCore/Target.pri

    r146494 r147324  
    106106    heap/MarkedSpace.cpp \
    107107    heap/SlotVisitor.cpp \
     108    heap/SuperRegion.cpp \
    108109    heap/VTableSpectrum.cpp \
    109110    heap/WriteBarrierSupport.cpp \

  • trunk/Source/JavaScriptCore/heap/BlockAllocator.cpp

    r146734 r147324  
    3535namespace JSC {
    3636
    37 BlockAllocator::BlockAllocator()
    38     : m_copiedRegionSet(CopiedBlock::blockSize)
     37BlockAllocator::BlockAllocator(JSGlobalData* globalData)
     38    : m_superRegion(globalData)
     39    , m_copiedRegionSet(CopiedBlock::blockSize)
    3940    , m_markedRegionSet(MarkedBlock::blockSize)
    4041    , m_fourKBBlockRegionSet(WeakBlock::blockSize)

  • trunk/Source/JavaScriptCore/heap/BlockAllocator.h

    r147282 r147324  
    4141class CopyWorkListSegment;
    4242class HandleBlock;
     43class JSGlobalData;
    4344class MarkStackSegment;
    4445class MarkedBlock;
    … …  
    5051class BlockAllocator {
    5152public:
    52     BlockAllocator();
     53    BlockAllocator(JSGlobalData*);
    5354    ~BlockAllocator();
    5455
    … …  
    8384    template <typename T> RegionSet& regionSetFor();
    8485
     86    SuperRegion m_superRegion;
    8587    RegionSet m_copiedRegionSet;
    8688    RegionSet m_markedRegionSet;
    … …  
    141143    }
    142144
    143     Region* newRegion = Region::create(T::blockSize);
     145    Region* newRegion = Region::create(&m_superRegion, T::blockSize);
    144146
    145147    SpinLockHolder locker(&m_regionLock);
    … …  
    154156{
    155157    size_t realSize = WTF::roundUpToMultipleOf(blockAlignment, blockSize);
    156     Region* newRegion = Region::createCustomSize(realSize, blockAlignment);
     158    Region* newRegion = Region::createCustomSize(&m_superRegion, realSize, blockAlignment);
    157159    DeadBlock* block = newRegion->allocate();
    158160    ASSERT(block);
    … …  
    200202    ASSERT(region->isCustomSize());
    201203    region->deallocate(block);
    202     delete region;
     204    region->destroy();
    203205}
    204206

  • trunk/Source/JavaScriptCore/heap/Heap.cpp

    r143279 r147324  
    251251    , m_bytesAbandoned(0)
    252252    , m_operationInProgress(NoOperation)
     253    , m_blockAllocator(globalData)
    253254    , m_objectSpace(this)
    254255    , m_storageSpace(this)

  • trunk/Source/JavaScriptCore/heap/Heap.h

    r146734 r147324  
    190190        friend class CopyVisitor;
    191191        friend class SlotVisitor;
     192        friend class SuperRegion;
    192193        friend class IncrementalSweeper;
    193194        friend class HeapStatistics;

  • trunk/Source/JavaScriptCore/heap/MarkedBlock.cpp

    r141190 r147324  
    3636MarkedBlock* MarkedBlock::create(DeadBlock* block, MarkedAllocator* allocator, size_t cellSize, DestructorType destructorType)
    3737{
     38    ASSERT(reinterpret_cast<size_t>(block) == (reinterpret_cast<size_t>(block) & blockMask));
    3839    Region* region = block->region();
    3940    return new (NotNull, block) MarkedBlock(region, allocator, cellSize, destructorType);

  • trunk/Source/JavaScriptCore/heap/Region.h

    r147282 r147324  
    2828
    2929#include "HeapBlock.h"
     30#include "SuperRegion.h"
    3031#include <wtf/DoublyLinkedList.h>
     32#include <wtf/MetaAllocatorHandle.h>
    3133#include <wtf/PageAllocationAligned.h>
     34
     35#define HEAP_MEMORY_ID reinterpret_cast<void*>(static_cast<intptr_t>(-3))
     36
     37#ifndef ENABLE_SUPER_REGION
     38#if USE(JSVALUE64)
     39#define ENABLE_SUPER_REGION 1
     40#else
     41#define ENABLE_SUPER_REGION 0
     42#endif
     43#endif
    3244
    3345namespace JSC {
    … …  
    4860public:
    4961    ~Region();
    50     static Region* create(size_t blockSize);
    51     static Region* createCustomSize(size_t blockSize, size_t blockAlignment);
     62    static Region* create(SuperRegion*, size_t blockSize);
     63    static Region* createCustomSize(SuperRegion*, size_t blockSize, size_t blockAlignment);
    5264    Region* reset(size_t blockSize);
     65    void destroy();
    5366
    5467    size_t blockSize() const { return m_blockSize; }
    … …  
    6174
    6275    static const size_t s_regionSize = 64 * KB;
     76    static const size_t s_regionMask = ~(s_regionSize - 1);
     77
     78protected:
     79    Region(size_t blockSize, size_t totalBlocks, bool isExcess);
     80    void initializeBlockList();
     81
     82    bool m_isExcess;
    6383
    6484private:
    65     Region(PageAllocationAligned&, size_t blockSize, size_t totalBlocks);
    66 
    67     PageAllocationAligned m_allocation;
     85    void* base();
     86    size_t size();
     87
    6888    size_t m_totalBlocks;
    6989    size_t m_blocksInUse;
    … …  
    7595};
    7696
    77 inline Region* Region::create(size_t blockSize)
    78 {
    79     ASSERT(blockSize <= s_regionSize);
    80     ASSERT(!(s_regionSize % blockSize));
     97
     98class NormalRegion : public Region {
     99    friend class Region;
     100private:
     101    NormalRegion(PassRefPtr<WTF::MetaAllocatorHandle>, size_t blockSize, size_t totalBlocks);
     102
     103    static NormalRegion* tryCreate(SuperRegion*, size_t blockSize);
     104    static NormalRegion* tryCreateCustomSize(SuperRegion*, size_t blockSize, size_t blockAlignment);
     105
     106    void* base() { return m_allocation->start(); }
     107    size_t size() { return m_allocation->sizeInBytes(); }
     108
     109    NormalRegion* reset(size_t blockSize);
     110
     111    RefPtr<WTF::MetaAllocatorHandle> m_allocation;
     112};
     113
     114class ExcessRegion : public Region {
     115    friend class Region;
     116private:
     117    ExcessRegion(PageAllocationAligned&, size_t blockSize, size_t totalBlocks);
     118
     119    ~ExcessRegion();
     120
     121    static ExcessRegion* create(size_t blockSize);
     122    static ExcessRegion* createCustomSize(size_t blockSize, size_t blockAlignment);
     123
     124    void* base() { return m_allocation.base(); }
     125    size_t size() { return m_allocation.size(); }
     126
     127    ExcessRegion* reset(size_t blockSize);
     128
     129    PageAllocationAligned m_allocation;
     130};
     131
     132inline NormalRegion::NormalRegion(PassRefPtr<WTF::MetaAllocatorHandle> allocation, size_t blockSize, size_t totalBlocks)
     133    : Region(blockSize, totalBlocks, false)
     134    , m_allocation(allocation)
     135{
     136    initializeBlockList();
     137}
     138
     139inline NormalRegion* NormalRegion::tryCreate(SuperRegion* superRegion, size_t blockSize)
     140{
     141    RefPtr<WTF::MetaAllocatorHandle> allocation = superRegion->allocate(s_regionSize, HEAP_MEMORY_ID);
     142    if (!allocation)
     143        return 0;
     144    return new NormalRegion(allocation, blockSize, s_regionSize / blockSize);
     145}
     146
     147inline NormalRegion* NormalRegion::tryCreateCustomSize(SuperRegion* superRegion, size_t blockSize, size_t blockAlignment)
     148{
     149    ASSERT_UNUSED(blockAlignment, blockAlignment <= s_regionSize);
     150    RefPtr<WTF::MetaAllocatorHandle> allocation = superRegion->allocate(blockSize, HEAP_MEMORY_ID);
     151    if (!allocation)
     152        return 0;
     153    return new NormalRegion(allocation, blockSize, 1);
     154}
     155
     156inline NormalRegion* NormalRegion::reset(size_t blockSize)
     157{
     158    ASSERT(!m_isExcess);
     159    RefPtr<WTF::MetaAllocatorHandle> allocation = m_allocation.release();
     160    return new (NotNull, this) NormalRegion(allocation.release(), blockSize, s_regionSize / blockSize);
     161}
     162
     163inline ExcessRegion::ExcessRegion(PageAllocationAligned& allocation, size_t blockSize, size_t totalBlocks)
     164    : Region(blockSize, totalBlocks, true)
     165    , m_allocation(allocation)
     166{
     167    initializeBlockList();
     168}
     169
     170inline ExcessRegion::~ExcessRegion()
     171{
     172    m_allocation.deallocate();
     173}
     174
     175inline ExcessRegion* ExcessRegion::create(size_t blockSize)
     176{
    81177    PageAllocationAligned allocation = PageAllocationAligned::allocate(s_regionSize, s_regionSize, OSAllocator::JSGCHeapPages);
    82     RELEASE_ASSERT(static_cast<bool>(allocation));
    83     return new Region(allocation, blockSize, s_regionSize / blockSize);
    84 }
    85 
    86 inline Region* Region::createCustomSize(size_t blockSize, size_t blockAlignment)
     178    ASSERT(static_cast<bool>(allocation));
     179    return new ExcessRegion(allocation, blockSize, s_regionSize / blockSize);
     180}
     181
     182inline ExcessRegion* ExcessRegion::createCustomSize(size_t blockSize, size_t blockAlignment)
    87183{
    88184    PageAllocationAligned allocation = PageAllocationAligned::allocate(blockSize, blockAlignment, OSAllocator::JSGCHeapPages);
    89     RELEASE_ASSERT(static_cast<bool>(allocation));
    90     Region* region = new Region(allocation, blockSize, 1);
    91     region->m_isCustomSize = true;
    92     return region;
    93 }
    94 
    95 inline Region::Region(PageAllocationAligned& allocation, size_t blockSize, size_t totalBlocks)
     185    ASSERT(static_cast<bool>(allocation));
     186    return new ExcessRegion(allocation, blockSize, 1);
     187}
     188
     189inline ExcessRegion* ExcessRegion::reset(size_t blockSize)
     190{
     191    ASSERT(m_isExcess);
     192    PageAllocationAligned allocation = m_allocation;
     193    return new (NotNull, this) ExcessRegion(allocation, blockSize, s_regionSize / blockSize);
     194}
     195
     196inline Region::Region(size_t blockSize, size_t totalBlocks, bool isExcess)
    96197    : DoublyLinkedListNode<Region>()
    97     , m_allocation(allocation)
     198    , m_isExcess(isExcess)
    98199    , m_totalBlocks(totalBlocks)
    99200    , m_blocksInUse(0)
    … …  
    103204    , m_next(0)
    104205{
    105     ASSERT(allocation);
    106     char* start = static_cast<char*>(m_allocation.base());
    107     char* end = start + m_allocation.size();
    108     for (char* current = start; current < end; current += blockSize)
     206}
     207
     208inline void Region::initializeBlockList()
     209{
     210    char* start = static_cast<char*>(base());
     211    char* current = start;
     212    for (size_t i = 0; i < m_totalBlocks; i++) {
     213        ASSERT(current < start + size());
    109214        m_deadBlocks.append(new (NotNull, current) DeadBlock(this));
     215        current += m_blockSize;
     216    }
     217}
     218
     219inline Region* Region::create(SuperRegion* superRegion, size_t blockSize)
     220{
     221#if ENABLE(SUPER_REGION)
     222    ASSERT(blockSize <= s_regionSize);
     223    ASSERT(!(s_regionSize % blockSize));
     224    Region* region = NormalRegion::tryCreate(superRegion, blockSize);
     225    if (LIKELY(!!region))
     226        return region;
     227#else
     228    UNUSED_PARAM(superRegion);
     229#endif
     230    return ExcessRegion::create(blockSize);
     231}
     232
     233inline Region* Region::createCustomSize(SuperRegion* superRegion, size_t blockSize, size_t blockAlignment)
     234{
     235#if ENABLE(SUPER_REGION)
     236    Region* region = NormalRegion::tryCreateCustomSize(superRegion, blockSize, blockAlignment);
     237    if (UNLIKELY(!region))
     238        region = ExcessRegion::createCustomSize(blockSize, blockAlignment);
     239#else
     240    UNUSED_PARAM(superRegion);
     241    Region* region = ExcessRegion::createCustomSize(blockSize, blockAlignment);
     242#endif
     243    region->m_isCustomSize = true;
     244    return region;
    110245}
    111246
    … …  
    113248{
    114249    ASSERT(isEmpty());
    115     m_allocation.deallocate();
     250}
     251
     252inline void Region::destroy()
     253{
     254#if ENABLE(SUPER_REGION)
     255    if (UNLIKELY(m_isExcess))
     256        delete static_cast<ExcessRegion*>(this);
     257    else
     258        delete static_cast<NormalRegion*>(this);
     259#else
     260    delete static_cast<ExcessRegion*>(this);
     261#endif
    116262}
    117263
    118264inline Region* Region::reset(size_t blockSize)
    119265{
     266#if ENABLE(SUPER_REGION)
    120267    ASSERT(isEmpty());
    121     PageAllocationAligned allocation = m_allocation;
    122     return new (NotNull, this) Region(allocation, blockSize, s_regionSize / blockSize);
     268    if (UNLIKELY(m_isExcess))
     269        return static_cast<ExcessRegion*>(this)->reset(blockSize);
     270    return static_cast<NormalRegion*>(this)->reset(blockSize);
     271#else
     272    return static_cast<ExcessRegion*>(this)->reset(blockSize);
     273#endif
    123274}
    124275
    … …  
    134285    ASSERT(base);
    135286    ASSERT(m_blocksInUse);
    136     ASSERT(base >= m_allocation.base() && base < static_cast<char*>(m_allocation.base()) + m_allocation.size());
     287    ASSERT(base >= this->base() && base < static_cast<char*>(this->base()) + size());
    137288    DeadBlock* block = new (NotNull, base) DeadBlock(this);
    138289    m_deadBlocks.push(block);
    … …  
    140291}
    141292
     293inline void* Region::base()
     294{
     295#if ENABLE(SUPER_REGION)
     296    if (UNLIKELY(m_isExcess))
     297        return static_cast<ExcessRegion*>(this)->ExcessRegion::base();
     298    return static_cast<NormalRegion*>(this)->NormalRegion::base();
     299#else
     300    return static_cast<ExcessRegion*>(this)->ExcessRegion::base();
     301#endif
     302}
     303
     304inline size_t Region::size()
     305{
     306#if ENABLE(SUPER_REGION)
     307    if (UNLIKELY(m_isExcess))
     308        return static_cast<ExcessRegion*>(this)->ExcessRegion::size();
     309    return static_cast<NormalRegion*>(this)->NormalRegion::size();
     310#else
     311    return static_cast<ExcessRegion*>(this)->ExcessRegion::size();
     312#endif
     313}
     314
    142315} // namespace JSC
    143316

  • trunk/Source/WTF/ChangeLog

    r147313 r147324  
     12013-03-31  Mark Hahnenberg  <mhahnenberg@apple.com>
     2
     3        Regions should be allocated from the same contiguous segment of virtual memory
     4        https://bugs.webkit.org/show_bug.cgi?id=113662
     5
     6        Reviewed by Filip Pizlo.
     7
     8        Instead of letting the OS spread our Regions all over the place, we should allocate them all within
     9        some range of each other. This change will open the door to some other optimizations, e.g. doing simple
     10        range checks for our write barriers and compressing JSCell pointers to 32-bits.
     11
     12        * wtf/MetaAllocator.cpp: Changed the MetaAllocator to allow custom page sizes if the derived class wants to
     13        use something other than the system page size.
     14        (WTF::MetaAllocator::MetaAllocator):
     15        * wtf/MetaAllocator.h:
     16        (MetaAllocator):
     17
    1182013-04-01  Yury Semikhatsky  <yurys@chromium.org>
    219

  • trunk/Source/WTF/wtf/MetaAllocator.cpp

    r146932 r147324  
    115115}
    116116
    117 MetaAllocator::MetaAllocator(size_t allocationGranule)
     117MetaAllocator::MetaAllocator(size_t allocationGranule, size_t pageSize)
    118118    : m_allocationGranule(allocationGranule)
    119     , m_pageSize(pageSize())
     119    , m_pageSize(pageSize)
    120120    , m_bytesAllocated(0)
    121121    , m_bytesReserved(0)

  • trunk/Source/WTF/wtf/MetaAllocator.h

    r127484 r147324  
    6565
    6666public:
    67     WTF_EXPORT_PRIVATE MetaAllocator(size_t allocationGranule);
     67    WTF_EXPORT_PRIVATE MetaAllocator(size_t allocationGranule, size_t pageSize = WTF::pageSize());
    6868   
    6969    WTF_EXPORT_PRIVATE virtual ~MetaAllocator();
Note: See TracChangeset for help on using the changeset viewer.