Changeset 41443 in webkit


Ignore:
Timestamp:
Mar 4, 2009 10:04:18 PM (15 years ago)
Author:
mrowe@apple.com
Message:

<rdar://problem/6354858> FastMallocZone's enumeration code reports fragmented administration space

Reviewed by Oliver Hunt.

The handling of MALLOC_ADMIN_REGION_RANGE_TYPE in FastMalloc's zone was incorrect. It was attempting
to record the memory containing and individual span as an administrative region, when all memory
allocated via MetaDataAlloc should in fact be recorded. This was causing memory regions allocated
via MetaDataAlloc to appear as "VM_ALLOCATE ?" in vmmap output. They are now correctly reported as
"MALLOC_OTHER" regions associated with the JavaScriptCore FastMalloc zone.

Memory is allocated via MetaDataAlloc from two locations: PageHeapAllocator, and TCMalloc_PageMap{2,3}.
These two cases are handled differently.

PageHeapAllocator is extended to keep a linked list of memory regions that it has allocated. The
first object in an allocated region contains the link to the previously allocated region. To record
the administrative regions of a PageHeapAllocator we can simply walk the linked list and record
each allocated region we encounter.

TCMalloc_PageMaps allocate memory via MetaDataAlloc to store each level of the radix tree. To record
the administrative regions of a TCMalloc_PageMap we walk the tree and record the storage used for nodes
at each position rather than the nodes themselves.

A small performance improvement is achieved by coalescing adjacent memory regions inside the PageMapMemoryUsageRecorder
so that fewer calls in to the range recorder are necessary. We further reduce the number of calls to the
range recorder by aggregating the in-use ranges of a given memory region into a local buffer before recording
them with a single call. A similar approach is also used by AdminRegionRecorder.

  • wtf/FastMalloc.cpp:

(WTF::PageHeapAllocator::Init):
(WTF::PageHeapAllocator::New):
(WTF::PageHeapAllocator::recordAdministrativeRegions):
(WTF::TCMallocStats::FreeObjectFinder::isFreeObject):
(WTF::TCMallocStats::PageMapMemoryUsageRecorder::~PageMapMemoryUsageRecorder):
(WTF::TCMallocStats::PageMapMemoryUsageRecorder::recordPendingRegions):
(WTF::TCMallocStats::PageMapMemoryUsageRecorder::visit):
(WTF::TCMallocStats::AdminRegionRecorder::AdminRegionRecorder):
(WTF::TCMallocStats::AdminRegionRecorder::recordRegion):
(WTF::TCMallocStats::AdminRegionRecorder::visit):
(WTF::TCMallocStats::AdminRegionRecorder::recordPendingRegions):
(WTF::TCMallocStats::AdminRegionRecorder::~AdminRegionRecorder):
(WTF::TCMallocStats::FastMallocZone::enumerate):
(WTF::TCMallocStats::FastMallocZone::FastMallocZone):
(WTF::TCMallocStats::FastMallocZone::init):

  • wtf/TCPageMap.h:

(TCMalloc_PageMap2::visitValues):
(TCMalloc_PageMap2::visitAllocations):
(TCMalloc_PageMap3::visitValues):
(TCMalloc_PageMap3::visitAllocations):

Location:
trunk/JavaScriptCore
Files:
3 edited

Legend:

Unmodified
Added
Removed
  • trunk/JavaScriptCore/ChangeLog

    r41431 r41443  
     12009-03-04  Mark Rowe  <mrowe@apple.com>
     2
     3        Reviewed by Oliver Hunt.
     4
     5        <rdar://problem/6354858> FastMallocZone's enumeration code reports fragmented administration space
     6
     7        The handling of MALLOC_ADMIN_REGION_RANGE_TYPE in FastMalloc's zone was incorrect.  It was attempting
     8        to record the memory containing and individual span as an administrative region, when all memory
     9        allocated via MetaDataAlloc should in fact be recorded.  This was causing memory regions allocated
     10        via MetaDataAlloc to appear as "VM_ALLOCATE ?" in vmmap output.  They are now correctly reported as
     11        "MALLOC_OTHER" regions associated with the JavaScriptCore FastMalloc zone.
     12
     13        Memory is allocated via MetaDataAlloc from two locations: PageHeapAllocator, and TCMalloc_PageMap{2,3}.
     14        These two cases are handled differently.
     15
     16        PageHeapAllocator is extended to keep a linked list of memory regions that it has allocated.  The
     17        first object in an allocated region contains the link to the previously allocated region. To record
     18        the administrative regions of a PageHeapAllocator we can simply walk the linked list and record
     19        each allocated region we encounter.
     20
     21        TCMalloc_PageMaps allocate memory via MetaDataAlloc to store each level of the radix tree.  To record
     22        the administrative regions of a TCMalloc_PageMap we walk the tree and record the storage used for nodes
     23        at each position rather than the nodes themselves.
     24
     25        A small performance improvement is achieved by coalescing adjacent memory regions inside the PageMapMemoryUsageRecorder
     26        so that fewer calls in to the range recorder are necessary.  We further reduce the number of calls to the
     27        range recorder by aggregating the in-use ranges of a given memory region into a local buffer before recording
     28        them with a single call.  A similar approach is also used by AdminRegionRecorder.
     29
     30        * wtf/FastMalloc.cpp:
     31        (WTF::PageHeapAllocator::Init):
     32        (WTF::PageHeapAllocator::New):
     33        (WTF::PageHeapAllocator::recordAdministrativeRegions):
     34        (WTF::TCMallocStats::FreeObjectFinder::isFreeObject):
     35        (WTF::TCMallocStats::PageMapMemoryUsageRecorder::~PageMapMemoryUsageRecorder):
     36        (WTF::TCMallocStats::PageMapMemoryUsageRecorder::recordPendingRegions):
     37        (WTF::TCMallocStats::PageMapMemoryUsageRecorder::visit):
     38        (WTF::TCMallocStats::AdminRegionRecorder::AdminRegionRecorder):
     39        (WTF::TCMallocStats::AdminRegionRecorder::recordRegion):
     40        (WTF::TCMallocStats::AdminRegionRecorder::visit):
     41        (WTF::TCMallocStats::AdminRegionRecorder::recordPendingRegions):
     42        (WTF::TCMallocStats::AdminRegionRecorder::~AdminRegionRecorder):
     43        (WTF::TCMallocStats::FastMallocZone::enumerate):
     44        (WTF::TCMallocStats::FastMallocZone::FastMallocZone):
     45        (WTF::TCMallocStats::FastMallocZone::init):
     46        * wtf/TCPageMap.h:
     47        (TCMalloc_PageMap2::visitValues):
     48        (TCMalloc_PageMap2::visitAllocations):
     49        (TCMalloc_PageMap3::visitValues):
     50        (TCMalloc_PageMap3::visitAllocations):
     51
    1522009-03-04  Antti Koivisto  <antti@apple.com>
    253
  • trunk/JavaScriptCore/wtf/FastMalloc.cpp

    r41023 r41443  
    322322
    323323#if PLATFORM(DARWIN)
     324class Span;
     325class TCMalloc_Central_FreeListPadded;
    324326class TCMalloc_PageHeap;
    325327class TCMalloc_ThreadCache;
    326 class TCMalloc_Central_FreeListPadded;
     328template <typename T> class PageHeapAllocator;
    327329
    328330class FastMallocZone {
     
    340342
    341343private:
    342     FastMallocZone(TCMalloc_PageHeap*, TCMalloc_ThreadCache**, TCMalloc_Central_FreeListPadded*);
     344    FastMallocZone(TCMalloc_PageHeap*, TCMalloc_ThreadCache**, TCMalloc_Central_FreeListPadded*, PageHeapAllocator<Span>*, PageHeapAllocator<TCMalloc_ThreadCache>*);
    343345    static size_t size(malloc_zone_t*, const void*);
    344346    static void* zoneMalloc(malloc_zone_t*, size_t);
     
    353355    TCMalloc_ThreadCache** m_threadHeaps;
    354356    TCMalloc_Central_FreeListPadded* m_centralCaches;
     357    PageHeapAllocator<Span>* m_spanAllocator;
     358    PageHeapAllocator<TCMalloc_ThreadCache>* m_pageHeapAllocator;
    355359};
    356360
     
    821825  size_t free_avail_;
    822826
     827  // Linked list of all regions allocated by this allocator
     828  void* allocated_regions_;
     829
    823830  // Free list of already carved objects
    824831  void* free_list_;
     
    831838    ASSERT(kAlignedSize <= kAllocIncrement);
    832839    inuse_ = 0;
     840    allocated_regions_ = 0;
    833841    free_area_ = NULL;
    834842    free_avail_ = 0;
     
    845853      if (free_avail_ < kAlignedSize) {
    846854        // Need more room
    847         free_area_ = reinterpret_cast<char*>(MetaDataAlloc(kAllocIncrement));
    848         if (free_area_ == NULL) CRASH();
    849         free_avail_ = kAllocIncrement;
     855        char* new_allocation = reinterpret_cast<char*>(MetaDataAlloc(kAllocIncrement));
     856        if (!new_allocation)
     857          CRASH();
     858
     859        *(void**)new_allocation = allocated_regions_;
     860        allocated_regions_ = new_allocation;
     861        free_area_ = new_allocation + kAlignedSize;
     862        free_avail_ = kAllocIncrement - kAlignedSize;
    850863      }
    851864      result = free_area_;
     
    864877
    865878  int inuse() const { return inuse_; }
     879
     880#if defined(WTF_CHANGES) && PLATFORM(DARWIN)
     881  template <class Recorder>
     882  void recordAdministrativeRegions(Recorder& recorder, const RemoteMemoryReader& reader)
     883  {
     884      vm_address_t adminAllocation = reinterpret_cast<vm_address_t>(allocated_regions_);
     885      while (adminAllocation) {
     886          recorder.recordRegion(adminAllocation, kAllocIncrement);
     887          adminAllocation = *reader(reinterpret_cast<vm_address_t*>(adminAllocation));
     888      }
     889  }
     890#endif
    866891};
    867892
     
    36313656    void visit(void* ptr) { m_freeObjects.add(ptr); }
    36323657    bool isFreeObject(void* ptr) const { return m_freeObjects.contains(ptr); }
     3658    bool isFreeObject(vm_address_t ptr) const { return isFreeObject(reinterpret_cast<void*>(ptr)); }
    36333659    size_t freeObjectCount() const { return m_freeObjects.size(); }
    36343660
     
    36813707    const RemoteMemoryReader& m_reader;
    36823708    const FreeObjectFinder& m_freeObjectFinder;
    3683     mutable HashSet<void*> m_seenPointers;
     3709
     3710    HashSet<void*> m_seenPointers;
     3711    Vector<Span*> m_coalescedSpans;
    36843712
    36853713public:
     
    36933721    { }
    36943722
    3695     int visit(void* ptr) const
     3723    ~PageMapMemoryUsageRecorder()
     3724    {
     3725        ASSERT(!m_coalescedSpans.size());
     3726    }
     3727
     3728    void recordPendingRegions()
     3729    {
     3730        Span* lastSpan = m_coalescedSpans[m_coalescedSpans.size() - 1];
     3731        vm_range_t ptrRange = { m_coalescedSpans[0]->start << kPageShift, 0 };
     3732        ptrRange.size = (lastSpan->start << kPageShift) - ptrRange.address + (lastSpan->length * kPageSize);
     3733
     3734        // Mark the memory region the spans represent as a candidate for containing pointers
     3735        if (m_typeMask & MALLOC_PTR_REGION_RANGE_TYPE)
     3736            (*m_recorder)(m_task, m_context, MALLOC_PTR_REGION_RANGE_TYPE, &ptrRange, 1);
     3737
     3738        if (!(m_typeMask & MALLOC_PTR_IN_USE_RANGE_TYPE)) {
     3739            m_coalescedSpans.clear();
     3740            return;
     3741        }
     3742
     3743        Vector<vm_range_t, 1024> allocatedPointers;
     3744        for (size_t i = 0; i < m_coalescedSpans.size(); ++i) {
     3745            Span *theSpan = m_coalescedSpans[i];
     3746            if (theSpan->free)
     3747                continue;
     3748
     3749            vm_address_t spanStartAddress = theSpan->start << kPageShift;
     3750            vm_size_t spanSizeInBytes = theSpan->length * kPageSize;
     3751
     3752            if (!theSpan->sizeclass) {
     3753                // If it's an allocated large object span, mark it as in use
     3754                if (!m_freeObjectFinder.isFreeObject(spanStartAddress))
     3755                    allocatedPointers.append((vm_range_t){spanStartAddress, spanSizeInBytes});
     3756            } else {
     3757                const size_t objectSize = ByteSizeForClass(theSpan->sizeclass);
     3758
     3759                // Mark each allocated small object within the span as in use
     3760                const vm_address_t endOfSpan = spanStartAddress + spanSizeInBytes;
     3761                for (vm_address_t object = spanStartAddress; object + objectSize <= endOfSpan; object += objectSize) {
     3762                    if (!m_freeObjectFinder.isFreeObject(object))
     3763                        allocatedPointers.append((vm_range_t){object, objectSize});
     3764                }
     3765            }
     3766        }
     3767
     3768        (*m_recorder)(m_task, m_context, MALLOC_PTR_IN_USE_RANGE_TYPE, allocatedPointers.data(), allocatedPointers.size());
     3769
     3770        m_coalescedSpans.clear();
     3771    }
     3772
     3773    int visit(void* ptr)
    36963774    {
    36973775        if (!ptr)
     
    36993777
    37003778        Span* span = m_reader(reinterpret_cast<Span*>(ptr));
     3779        if (!span->start)
     3780            return 1;
     3781
    37013782        if (m_seenPointers.contains(ptr))
    37023783            return span->length;
    37033784        m_seenPointers.add(ptr);
    37043785
    3705         // Mark the memory used for the Span itself as an administrative region
    3706         vm_range_t ptrRange = { reinterpret_cast<vm_address_t>(ptr), sizeof(Span) };
    3707         if (m_typeMask & (MALLOC_PTR_REGION_RANGE_TYPE | MALLOC_ADMIN_REGION_RANGE_TYPE))
    3708             (*m_recorder)(m_task, m_context, MALLOC_ADMIN_REGION_RANGE_TYPE, &ptrRange, 1);
    3709 
    3710         ptrRange.address = span->start << kPageShift;
    3711         ptrRange.size = span->length * kPageSize;
    3712 
    3713         // Mark the memory region the span represents as candidates for containing pointers
    3714         if (m_typeMask & (MALLOC_PTR_REGION_RANGE_TYPE | MALLOC_ADMIN_REGION_RANGE_TYPE))
    3715             (*m_recorder)(m_task, m_context, MALLOC_PTR_REGION_RANGE_TYPE, &ptrRange, 1);
    3716 
    3717         if (!span->free && (m_typeMask & MALLOC_PTR_IN_USE_RANGE_TYPE)) {
    3718             // If it's an allocated large object span, mark it as in use
    3719             if (span->sizeclass == 0 && !m_freeObjectFinder.isFreeObject(reinterpret_cast<void*>(ptrRange.address)))
    3720                 (*m_recorder)(m_task, m_context, MALLOC_PTR_IN_USE_RANGE_TYPE, &ptrRange, 1);
    3721             else if (span->sizeclass) {
    3722                 const size_t byteSize = ByteSizeForClass(span->sizeclass);
    3723                 unsigned totalObjects = (span->length << kPageShift) / byteSize;
    3724                 ASSERT(span->refcount <= totalObjects);
    3725                 char* ptr = reinterpret_cast<char*>(span->start << kPageShift);
    3726 
    3727                 // Mark each allocated small object within the span as in use
    3728                 for (unsigned i = 0; i < totalObjects; i++) {
    3729                     char* thisObject = ptr + (i * byteSize);
    3730                     if (m_freeObjectFinder.isFreeObject(thisObject))
    3731                         continue;
    3732 
    3733                     vm_range_t objectRange = { reinterpret_cast<vm_address_t>(thisObject), byteSize };
    3734                     (*m_recorder)(m_task, m_context, MALLOC_PTR_IN_USE_RANGE_TYPE, &objectRange, 1);
    3735                 }
    3736             }
     3786        if (!m_coalescedSpans.size()) {
     3787            m_coalescedSpans.append(span);
     3788            return span->length;
    37373789        }
    37383790
     3791        Span* previousSpan = m_coalescedSpans[m_coalescedSpans.size() - 1];
     3792        vm_address_t previousSpanStartAddress = previousSpan->start << kPageShift;
     3793        vm_size_t previousSpanSizeInBytes = previousSpan->length * kPageSize;
     3794
     3795        // If the new span is adjacent to the previous span, do nothing for now.
     3796        vm_address_t spanStartAddress = span->start << kPageShift;
     3797        if (spanStartAddress == previousSpanStartAddress + previousSpanSizeInBytes) {
     3798            m_coalescedSpans.append(span);
     3799            return span->length;
     3800        }
     3801
     3802        // New span is not adjacent to previous span, so record the spans coalesced so far.
     3803        recordPendingRegions();
     3804        m_coalescedSpans.append(span);
     3805
    37393806        return span->length;
     3807    }
     3808};
     3809
     3810class AdminRegionRecorder {
     3811    task_t m_task;
     3812    void* m_context;
     3813    unsigned m_typeMask;
     3814    vm_range_recorder_t* m_recorder;
     3815    const RemoteMemoryReader& m_reader;
     3816
     3817    Vector<vm_range_t, 1024> m_pendingRegions;
     3818
     3819public:
     3820    AdminRegionRecorder(task_t task, void* context, unsigned typeMask, vm_range_recorder_t* recorder, const RemoteMemoryReader& reader)
     3821        : m_task(task)
     3822        , m_context(context)
     3823        , m_typeMask(typeMask)
     3824        , m_recorder(recorder)
     3825        , m_reader(reader)
     3826    { }
     3827
     3828    void recordRegion(vm_address_t ptr, size_t size)
     3829    {
     3830        if (m_typeMask & MALLOC_ADMIN_REGION_RANGE_TYPE)
     3831            m_pendingRegions.append((vm_range_t){ ptr, size });
     3832    }
     3833
     3834    void visit(void *ptr, size_t size)
     3835    {
     3836        recordRegion(reinterpret_cast<vm_address_t>(ptr), size);
     3837    }
     3838
     3839    void recordPendingRegions()
     3840    {
     3841        if (m_pendingRegions.size()) {
     3842            (*m_recorder)(m_task, m_context, MALLOC_ADMIN_REGION_RANGE_TYPE, m_pendingRegions.data(), m_pendingRegions.size());
     3843            m_pendingRegions.clear();
     3844        }
     3845    }
     3846
     3847    ~AdminRegionRecorder()
     3848    {
     3849        ASSERT(!m_pendingRegions.size());
    37403850    }
    37413851};
     
    37603870    TCMalloc_PageHeap::PageMap* pageMap = &pageHeap->pagemap_;
    37613871    PageMapFreeObjectFinder pageMapFinder(memoryReader, finder);
    3762     pageMap->visit(pageMapFinder, memoryReader);
     3872    pageMap->visitValues(pageMapFinder, memoryReader);
    37633873
    37643874    PageMapMemoryUsageRecorder usageRecorder(task, context, typeMask, recorder, memoryReader, finder);
    3765     pageMap->visit(usageRecorder, memoryReader);
     3875    pageMap->visitValues(usageRecorder, memoryReader);
     3876    usageRecorder.recordPendingRegions();
     3877
     3878    AdminRegionRecorder adminRegionRecorder(task, context, typeMask, recorder, memoryReader);
     3879    pageMap->visitAllocations(adminRegionRecorder, memoryReader);
     3880
     3881    PageHeapAllocator<Span>* spanAllocator = memoryReader(mzone->m_spanAllocator);
     3882    PageHeapAllocator<TCMalloc_ThreadCache>* pageHeapAllocator = memoryReader(mzone->m_pageHeapAllocator);
     3883
     3884    spanAllocator->recordAdministrativeRegions(adminRegionRecorder, memoryReader);
     3885    pageHeapAllocator->recordAdministrativeRegions(adminRegionRecorder, memoryReader);
     3886
     3887    adminRegionRecorder.recordPendingRegions();
    37663888
    37673889    return 0;
     
    38133935}
    38143936
    3815 FastMallocZone::FastMallocZone(TCMalloc_PageHeap* pageHeap, TCMalloc_ThreadCache** threadHeaps, TCMalloc_Central_FreeListPadded* centralCaches)
     3937FastMallocZone::FastMallocZone(TCMalloc_PageHeap* pageHeap, TCMalloc_ThreadCache** threadHeaps, TCMalloc_Central_FreeListPadded* centralCaches, PageHeapAllocator<Span>* spanAllocator, PageHeapAllocator<TCMalloc_ThreadCache>* pageHeapAllocator)
    38163938    : m_pageHeap(pageHeap)
    38173939    , m_threadHeaps(threadHeaps)
    38183940    , m_centralCaches(centralCaches)
     3941    , m_spanAllocator(spanAllocator)
     3942    , m_pageHeapAllocator(pageHeapAllocator)
    38193943{
    38203944    memset(&m_zone, 0, sizeof(m_zone));
     
    38353959void FastMallocZone::init()
    38363960{
    3837     static FastMallocZone zone(pageheap, &thread_heaps, static_cast<TCMalloc_Central_FreeListPadded*>(central_cache));
     3961    static FastMallocZone zone(pageheap, &thread_heaps, static_cast<TCMalloc_Central_FreeListPadded*>(central_cache), &span_allocator, &threadheap_allocator);
    38383962}
    38393963
  • trunk/JavaScriptCore/wtf/TCPageMap.h

    r28434 r41443  
    5555
    5656#include <string.h>
    57 
    5857#include "Assertions.h"
    5958
     
    165164#ifdef WTF_CHANGES
    166165  template<class Visitor, class MemoryReader>
    167   void visit(const Visitor& visitor, const MemoryReader& reader)
     166  void visitValues(Visitor& visitor, const MemoryReader& reader)
    168167  {
    169168    for (int i = 0; i < ROOT_LENGTH; i++) {
     
    174173      for (int j = 0; j < LEAF_LENGTH; j += visitor.visit(l->values[j]))
    175174        ;
     175    }
     176  }
     177
     178  template<class Visitor, class MemoryReader>
     179  void visitAllocations(Visitor& visitor, const MemoryReader&) {
     180    for (int i = 0; i < ROOT_LENGTH; i++) {
     181      if (root_[i])
     182        visitor.visit(root_[i], sizeof(Leaf));
    176183    }
    177184  }
     
    267274#ifdef WTF_CHANGES
    268275  template<class Visitor, class MemoryReader>
    269   void visit(const Visitor& visitor, const MemoryReader& reader) {
     276  void visitValues(Visitor& visitor, const MemoryReader& reader) {
    270277    Node* root = reader(root_);
    271278    for (int i = 0; i < INTERIOR_LENGTH; i++) {
     
    284291    }
    285292  }
     293
     294  template<class Visitor, class MemoryReader>
     295  void visitAllocations(Visitor& visitor, const MemoryReader& reader) {
     296    visitor.visit(root_, sizeof(Node));
     297
     298    Node* root = reader(root_);
     299    for (int i = 0; i < INTERIOR_LENGTH; i++) {
     300      if (!root->ptrs[i])
     301        continue;
     302
     303      visitor.visit(root->ptrs[i], sizeof(Node));
     304      Node* n = reader(root->ptrs[i]);
     305      for (int j = 0; j < INTERIOR_LENGTH; j++) {
     306        if (!n->ptrs[j])
     307          continue;
     308
     309        visitor.visit(n->ptrs[j], sizeof(Leaf));
     310      }
     311    }
     312  }
    286313#endif
    287314};
Note: See TracChangeset for help on using the changeset viewer.