Changeset 126208 in webkit


Ignore:
Timestamp:
Aug 21, 2012 4:16:24 PM (12 years ago)
Author:
mhahnenberg@apple.com
Message:

WTF Threading leaks kernel objects on platforms that use pthreads
​https://bugs.webkit.org/show_bug.cgi?id=94636

Reviewed by Geoffrey Garen.

Source/WTF:

Creating lots of Web workers on Mac platforms leaks lots of Mach ports. Eventually, the
process can exhaust its allocation of Mach ports from the kernel, which can then cause
all sorts of badness, including the inability to allocate new virtual memory from the
kernel. ThreadingPthreads.cpp and ThreadIdentifierDataPthreads.cpp need to be refactored
so that we do not leak these kernel resources. I would assume that we also leak kernel
resources on other pthreads platforms as well.

  • wtf/ThreadIdentifierDataPthreads.cpp:

(WTF):
(WTF::ThreadIdentifierData::~ThreadIdentifierData): Now calls the event threadDidExit, which
handles all relevant tear-down of the thread metadata in the thread map.

  • wtf/ThreadingPthreads.cpp: Added a new class called PthreadState that encapsulates the

state of a thread and the possible transitions between those states.
(PthreadState):
(WTF::PthreadState::PthreadState):
(WTF::PthreadState::joinableState): Returns the current state of the pthread.
(WTF::PthreadState::pthreadHandle): Returns the pthread_t for this particular thread. This needs to
remain valid even after the thread has exited because somebody could come along at any time in the
future and call join on the thread.
(WTF::PthreadState::didBecomeDetached): Signals that the thread was detached.
(WTF::PthreadState::didExit): Signals that the thread's exit destructor (~ThreadIdentifierData) has run.
(WTF::PthreadState::didJoin): Signals that the thread has been joined on successfully.
(WTF::PthreadState::hasExited): Returns whether or not the thread's exit destructor has run.
(WTF):
(WTF::identifierByPthreadHandle): Changed to also check hasExited() on the threads it finds in the map. We
should only have one valid pthread_t in the map, but there are other pthread_t's that need to remain in the
thread map for when somebody joins on that thread id later.
(WTF::establishIdentifierForPthreadHandle): Changed to put the allocate the new PthreadState data structure and
put it in the map.
(WTF::pthreadHandleForIdentifierWithLockAlreadyHeld):
(WTF::wtfThreadEntryPoint):
(WTF::waitForThreadCompletion): We now do the relevant cleanup after the specified thread has been
successfully joined on depending on if the joined thread has already exited.
(WTF::detachThread): Performs relevant cleanup if the thread has already exited. Otherwise signals to the
PthreadState that the thread has been detached.
(WTF::threadDidExit): Function called by ~ThreadIdentifierData to indicate that the thread has exited.
Only cleans up after itself if the thread isn't Joinable (i.e. Joined or Detached).

LayoutTests:

Added a test that creates a bunch of workers that immediately return. This should stress
the new WTF threading code on platforms that use pthreads, so any major regressions in correctness
will probably cause this test to fail since it creates both joinable and detached threads.

  • fast/js/create-lots-of-workers-expected.txt: Added.
  • fast/js/create-lots-of-workers.html: Added.
  • fast/js/resources/empty-worker.js: Added.
Location:
trunk
Files:
3 added
4 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r126205 r126208  
     12012-08-21  Mark Hahnenberg  <mhahnenberg@apple.com>
     2
     3        WTF Threading leaks kernel objects on platforms that use pthreads
     4        https://bugs.webkit.org/show_bug.cgi?id=94636
     5
     6        Reviewed by Geoffrey Garen.
     7
     8        Added a test that creates a bunch of workers that immediately return. This should stress
     9        the new WTF threading code on platforms that use pthreads, so any major regressions in correctness
     10        will probably cause this test to fail since it creates both joinable and detached threads.
     11
     12        * fast/js/create-lots-of-workers-expected.txt: Added.
     13        * fast/js/create-lots-of-workers.html: Added.
     14        * fast/js/resources/empty-worker.js: Added.
     15
    1162012-08-21  Florin Malita  <fmalita@chromium.org>
    217

  • trunk/Source/WTF/ChangeLog

    r126196 r126208  
     12012-08-21  Mark Hahnenberg  <mhahnenberg@apple.com>
     2
     3        WTF Threading leaks kernel objects on platforms that use pthreads
     4        https://bugs.webkit.org/show_bug.cgi?id=94636
     5
     6        Reviewed by Geoffrey Garen.
     7
     8        Creating lots of Web workers on Mac platforms leaks lots of Mach ports. Eventually, the
     9        process can exhaust its allocation of Mach ports from the kernel, which can then cause
     10        all sorts of badness, including the inability to allocate new virtual memory from the
     11        kernel. ThreadingPthreads.cpp and ThreadIdentifierDataPthreads.cpp need to be refactored
     12        so that we do not leak these kernel resources. I would assume that we also leak kernel
     13        resources on other pthreads platforms as well.
     14
     15        * wtf/ThreadIdentifierDataPthreads.cpp:
     16        (WTF):
     17        (WTF::ThreadIdentifierData::~ThreadIdentifierData): Now calls the event threadDidExit, which
     18        handles all relevant tear-down of the thread metadata in the thread map.
     19        * wtf/ThreadingPthreads.cpp: Added a new class called PthreadState that encapsulates the
     20        state of a thread and the possible transitions between those states.
     21        (PthreadState):
     22        (WTF::PthreadState::PthreadState):
     23        (WTF::PthreadState::joinableState): Returns the current state of the pthread.
     24        (WTF::PthreadState::pthreadHandle): Returns the pthread_t for this particular thread. This needs to
     25        remain valid even after the thread has exited because somebody could come along at any time in the
     26        future and call join on the thread.
     27        (WTF::PthreadState::didBecomeDetached): Signals that the thread was detached.
     28        (WTF::PthreadState::didExit): Signals that the thread's exit destructor (~ThreadIdentifierData) has run.
     29        (WTF::PthreadState::didJoin): Signals that the thread has been joined on successfully.
     30        (WTF::PthreadState::hasExited): Returns whether or not the thread's exit destructor has run.
     31        (WTF):
     32        (WTF::identifierByPthreadHandle): Changed to also check hasExited() on the threads it finds in the map. We
     33        should only have one valid pthread_t in the map, but there are other pthread_t's that need to remain in the
     34        thread map for when somebody joins on that thread id later.
     35        (WTF::establishIdentifierForPthreadHandle): Changed to put the allocate the new PthreadState data structure and
     36        put it in the map.
     37        (WTF::pthreadHandleForIdentifierWithLockAlreadyHeld):
     38        (WTF::wtfThreadEntryPoint):
     39        (WTF::waitForThreadCompletion): We now do the relevant cleanup after the specified thread has been
     40        successfully joined on depending on if the joined thread has already exited.
     41        (WTF::detachThread): Performs relevant cleanup if the thread has already exited. Otherwise signals to the
     42        PthreadState that the thread has been detached.
     43        (WTF::threadDidExit): Function called by ~ThreadIdentifierData to indicate that the thread has exited.
     44        Only cleans up after itself if the thread isn't Joinable (i.e. Joined or Detached).
     45
    1462012-08-21  Ulan Degenbaev  <ulan@chromium.org>
    247

  • trunk/Source/WTF/wtf/ThreadIdentifierDataPthreads.cpp

    r111778 r126208  
    4848pthread_key_t ThreadIdentifierData::m_key = PTHREAD_KEYS_MAX;
    4949
    50 void clearPthreadHandleForIdentifier(ThreadIdentifier);
     50void threadDidExit(ThreadIdentifier);
    5151
    5252ThreadIdentifierData::~ThreadIdentifierData()
    5353{
    54     clearPthreadHandleForIdentifier(m_identifier);
     54    threadDidExit(m_identifier);
    5555}
    5656

  • trunk/Source/WTF/wtf/ThreadingPthreads.cpp

    r123344 r126208  
    6262namespace WTF {
    6363
    64 typedef HashMap<ThreadIdentifier, pthread_t> ThreadMap;
     64class PthreadState {
     65public:
     66    enum JoinableState {
     67        Joinable, // The default thread state. The thread can be joined on.
     68
     69        Joined, // Somebody waited on this thread to exit and this thread finally exited. This state is here because there can be a
     70                // period of time between when the thread exits (which causes pthread_join to return and the remainder of waitOnThreadCompletion to run)
     71                // and when threadDidExit is called. We need threadDidExit to take charge and delete the thread data since there's
     72                // nobody else to pick up the slack in this case (since waitOnThreadCompletion has already returned).
     73
     74        Detached // The thread has been detached and can no longer be joined on. At this point, the thread must take care of cleaning up after itself.
     75    };
     76
     77    // Currently all threads created by WTF start out as joinable.
     78    PthreadState(pthread_t handle)
     79        : m_joinableState(Joinable)
     80        , m_didExit(false)
     81        , m_pthreadHandle(handle)
     82    {
     83    }
     84
     85    JoinableState joinableState() { return m_joinableState; }
     86    pthread_t pthreadHandle() { return m_pthreadHandle; }
     87    void didBecomeDetached() { m_joinableState = Detached; }
     88    void didExit() { m_didExit = true; }
     89    void didJoin() { m_joinableState = Joined; }
     90    bool hasExited() { return m_didExit; }
     91
     92private:
     93    JoinableState m_joinableState;
     94    bool m_didExit;
     95    pthread_t m_pthreadHandle;
     96};
     97
     98typedef HashMap<ThreadIdentifier, OwnPtr<PthreadState> > ThreadMap;
    6599
    66100static Mutex* atomicallyInitializedStaticMutex;
    67101
    68 void clearPthreadHandleForIdentifier(ThreadIdentifier);
     102void unsafeThreadWasDetached(ThreadIdentifier);
     103void threadDidExit(ThreadIdentifier);
     104void threadWasJoined(ThreadIdentifier);
    69105
    70106static Mutex& threadMapMutex()
    … …  
    115151    ThreadMap::iterator i = threadMap().begin();
    116152    for (; i != threadMap().end(); ++i) {
    117         if (pthread_equal(i->second, pthreadHandle))
     153        if (pthread_equal(i->second->pthreadHandle(), pthreadHandle) && !i->second->hasExited())
    118154            return i->first;
    119155    }
    … …  
    125161{
    126162    ASSERT(!identifierByPthreadHandle(pthreadHandle));
    127 
    128163    MutexLocker locker(threadMapMutex());
    129 
    130164    static ThreadIdentifier identifierCount = 1;
    131 
    132     threadMap().add(identifierCount, pthreadHandle);
    133 
     165    threadMap().add(identifierCount, adoptPtr(new PthreadState(pthreadHandle)));
    134166    return identifierCount++;
    135167}
    136168
    137 static pthread_t pthreadHandleForIdentifier(ThreadIdentifier id)
    138 {
    139     MutexLocker locker(threadMapMutex());
    140 
    141     return threadMap().get(id);
    142 }
    143 
    144 void clearPthreadHandleForIdentifier(ThreadIdentifier id)
    145 {
    146     MutexLocker locker(threadMapMutex());
    147 
    148     ASSERT(threadMap().contains(id));
    149 
    150     threadMap().remove(id);
     169static pthread_t pthreadHandleForIdentifierWithLockAlreadyHeld(ThreadIdentifier id)
     170{
     171    return threadMap().get(id)->pthreadHandle();
    151172}
    152173
    … …  
    156177    OwnPtr<ThreadFunctionInvocation> invocation = adoptPtr(static_cast<ThreadFunctionInvocation*>(param));
    157178    invocation->function(invocation->data);
    158 
    159179    return 0;
    160180}
    … …  
    199219int waitForThreadCompletion(ThreadIdentifier threadID)
    200220{
     221    pthread_t pthreadHandle;
    201222    ASSERT(threadID);
    202223
    203     pthread_t pthreadHandle = pthreadHandleForIdentifier(threadID);
    204     if (!pthreadHandle)
    205         return 0;
     224    {
     225        // We don't want to lock across the call to join, since that can block our thread and cause deadlock.
     226        MutexLocker locker(threadMapMutex());
     227        pthreadHandle = pthreadHandleForIdentifierWithLockAlreadyHeld(threadID);
     228        ASSERT(pthreadHandle);
     229    }
    206230
    207231    int joinResult = pthread_join(pthreadHandle, 0);
     232
    208233    if (joinResult == EDEADLK)
    209234        LOG_ERROR("ThreadIdentifier %u was found to be deadlocked trying to quit", threadID);
     235    else if (joinResult)
     236        LOG_ERROR("ThreadIdentifier %u was unable to be joined.\n", threadID);
     237
     238    MutexLocker locker(threadMapMutex());
     239    PthreadState* state = threadMap().get(threadID);
     240    ASSERT(state);
     241    ASSERT(state->joinableState() == PthreadState::Joinable);
     242
     243    // The thread has already exited, so clean up after it.
     244    if (state->hasExited())
     245        threadMap().remove(threadID);
     246    // The thread hasn't exited yet, so don't clean anything up. Just signal that we've already joined on it so that it will clean up after itself.
     247    else
     248        state->didJoin();
    210249
    211250    return joinResult;
    … …  
    216255    ASSERT(threadID);
    217256
    218     pthread_t pthreadHandle = pthreadHandleForIdentifier(threadID);
    219     if (!pthreadHandle)
    220         return;
    221 
    222     pthread_detach(pthreadHandle);
     257    MutexLocker locker(threadMapMutex());
     258    pthread_t pthreadHandle = pthreadHandleForIdentifierWithLockAlreadyHeld(threadID);
     259    ASSERT(pthreadHandle);
     260
     261    int detachResult = pthread_detach(pthreadHandle);
     262    if (detachResult)
     263        LOG_ERROR("ThreadIdentifier %u was unable to be detached\n", threadID);
     264
     265    PthreadState* state = threadMap().get(threadID);
     266    ASSERT(state);
     267    if (state->hasExited())
     268        threadMap().remove(threadID);
     269    else
     270        threadMap().get(threadID)->didBecomeDetached();
     271}
     272
     273void threadDidExit(ThreadIdentifier threadID)
     274{
     275    MutexLocker locker(threadMapMutex());
     276    PthreadState* state = threadMap().get(threadID);
     277    ASSERT(state);
     278   
     279    state->didExit();
     280
     281    if (state->joinableState() != PthreadState::Joinable)
     282        threadMap().remove(threadID);
    223283}
    224284
Note: See TracChangeset for help on using the changeset viewer.