Changeset 269077 in webkit


Ignore:
Timestamp:
Oct 27, 2020 3:21:28 PM (21 months ago)
Author:
jer.noble@apple.com
Message:

[Mac] Audio and Video element creation up to 300x slower than other browsers
​https://bugs.webkit.org/show_bug.cgi?id=218206
<rdar://problem/62451019>

Reviewed by Eric Carlson.

PerformanceTests:

  • Media/AudioElementCreation.html: Added.
  • Media/VideoElementCreation.html: Added.

Source/WebCore:

Tests: PerformanceTests/Media/AudioElementCreation.html

PerformanceTests/Media/VideoElementCreation.html

Currently, a large percent of the element creation code occurrs as a result of adding its
session to PlatformMediaSessionManager, which forces iterating over all extant sessions and
then to set various properties of the audio hardware in response. This patch addresses the
bulk of those expensive calls, but more performance optimizations are available to further
reduce media element creation costs.

When an <audio> element is created, we set the preferred audio output buffer size to a large
value for performance reasons. However, there's no need to repeatedly call into CoreAudio if
the buffer size is already set to that same high value. Store the result of setting the
preferred buffer size, and also add a property change listener to detect other callers
modifying that same value, so that all set operations with identical sizes become no-ops,
and all queries just return cached values.

When any media element is created, the entire list of extant sessions is iterated and
properties on each are queried. Rather than do these inside the same run-loop, use a
TaskQueue to enqueue a task to query the list of created elements during the next run-loop.

Between these two optimization, the runtime cost of creating 1000 audio elements is reduced
(on this engineer's machine) from 2s to 40ms.

  • platform/audio/PlatformMediaSessionManager.cpp:

(WebCore::PlatformMediaSessionManager::beginInterruption):
(WebCore::PlatformMediaSessionManager::addSession):
(WebCore::PlatformMediaSessionManager::removeSession):
(WebCore::PlatformMediaSessionManager::sessionStateChanged):
(WebCore::PlatformMediaSessionManager::forEachDocumentSession):
(WebCore::PlatformMediaSessionManager::forEachSession):
(WebCore::PlatformMediaSessionManager::anyOfSessions const):

  • platform/audio/PlatformMediaSessionManager.h:
  • platform/audio/mac/AudioSessionMac.mm:

(WebCore::AudioSessionPrivate::addSampleRateObserverIfNeeded):
(WebCore::AudioSessionPrivate::handleSampleRateChange):
(WebCore::AudioSessionPrivate::addBufferSizeObserverIfNeeded):
(WebCore::AudioSessionPrivate::handleBufferSizeChange):
(WebCore::AudioSession::sampleRate const):
(WebCore::AudioSession::bufferSize const):
(WebCore::AudioSession::preferredBufferSize const):
(WebCore::AudioSession::setPreferredBufferSize):

Location:
trunk
Files:
2 added
5 edited

Legend:

Unmodified
Added
Removed

  • trunk/PerformanceTests/ChangeLog

    r269028 r269077  
     12020-10-27  Jer Noble  <jer.noble@apple.com>
     2
     3        [Mac] Audio and Video element creation up to 300x slower than other browsers
     4        https://bugs.webkit.org/show_bug.cgi?id=218206
     5        <rdar://problem/62451019>
     6
     7        Reviewed by Eric Carlson.
     8
     9        * Media/AudioElementCreation.html: Added.
     10        * Media/VideoElementCreation.html: Added.
     11
    1122020-10-27  Caio Lima  <ticaiolima@gmail.com>
    213

  • trunk/Source/WebCore/ChangeLog

    r269073 r269077  
     12020-10-27  Jer Noble  <jer.noble@apple.com>
     2
     3        [Mac] Audio and Video element creation up to 300x slower than other browsers
     4        https://bugs.webkit.org/show_bug.cgi?id=218206
     5        <rdar://problem/62451019>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Tests: PerformanceTests/Media/AudioElementCreation.html
     10               PerformanceTests/Media/VideoElementCreation.html
     11
     12        Currently, a large percent of the element creation code occurrs as a result of adding its
     13        session to PlatformMediaSessionManager, which forces iterating over all extant sessions and
     14        then to set various properties of the audio hardware in response. This patch addresses the
     15        bulk of those expensive calls, but more performance optimizations are available to further
     16        reduce media element creation costs.
     17
     18        When an <audio> element is created, we set the preferred audio output buffer size to a large
     19        value for performance reasons. However, there's no need to repeatedly call into CoreAudio if
     20        the buffer size is already set to that same high value. Store the result of setting the
     21        preferred buffer size, and also add a property change listener to detect other callers
     22        modifying that same value, so that all set operations with identical sizes become no-ops,
     23        and all queries just return cached values.
     24
     25        When any media element is created, the entire list of extant sessions is iterated and
     26        properties on each are queried. Rather than do these inside the same run-loop, use a
     27        TaskQueue to enqueue a task to query the list of created elements during the next run-loop.
     28
     29        Between these two optimization, the runtime cost of creating 1000 audio elements is reduced
     30        (on this engineer's machine) from 2s to 40ms.
     31
     32        * platform/audio/PlatformMediaSessionManager.cpp:
     33        (WebCore::PlatformMediaSessionManager::beginInterruption):
     34        (WebCore::PlatformMediaSessionManager::addSession):
     35        (WebCore::PlatformMediaSessionManager::removeSession):
     36        (WebCore::PlatformMediaSessionManager::sessionStateChanged):
     37        (WebCore::PlatformMediaSessionManager::forEachDocumentSession):
     38        (WebCore::PlatformMediaSessionManager::forEachSession):
     39        (WebCore::PlatformMediaSessionManager::anyOfSessions const):
     40        * platform/audio/PlatformMediaSessionManager.h:
     41        * platform/audio/mac/AudioSessionMac.mm:
     42        (WebCore::AudioSessionPrivate::addSampleRateObserverIfNeeded):
     43        (WebCore::AudioSessionPrivate::handleSampleRateChange):
     44        (WebCore::AudioSessionPrivate::addBufferSizeObserverIfNeeded):
     45        (WebCore::AudioSessionPrivate::handleBufferSizeChange):
     46        (WebCore::AudioSession::sampleRate const):
     47        (WebCore::AudioSession::bufferSize const):
     48        (WebCore::AudioSession::preferredBufferSize const):
     49        (WebCore::AudioSession::setPreferredBufferSize):
     50
    1512020-10-27  Chris Dumez  <cdumez@apple.com>
    252

  • trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.cpp

    r268268 r269077  
    140140        session.beginInterruption(type);
    141141    });
    142     updateSessionState();
     142    scheduleUpdateSessionState();
    143143}
    144144
    … …  
    164164#endif
    165165
    166     updateSessionState();
     166    scheduleUpdateSessionState();
    167167}
    168168
    … …  
    191191#endif
    192192
    193     updateSessionState();
     193    scheduleUpdateSessionState();
    194194}
    195195
    … …  
    282282void PlatformMediaSessionManager::sessionStateChanged(PlatformMediaSession&)
    283283{
    284     updateSessionState();
     284    scheduleUpdateSessionState();
    285285}
    286286
    … …  
    562562    ASSERT(!m_audioCaptureSources.contains(source));
    563563    m_audioCaptureSources.add(source);
    564     updateSessionState();
     564    scheduleUpdateSessionState();
    565565}
    566566
    … …  
    570570    ASSERT(m_audioCaptureSources.contains(source));
    571571    m_audioCaptureSources.remove(source);
    572     updateSessionState();
     572    scheduleUpdateSessionState();
     573}
     574
     575void PlatformMediaSessionManager::scheduleUpdateSessionState()
     576{
     577    if (updateSessionStateQueue.hasPendingTasks())
     578        return;
     579
     580    updateSessionStateQueue.enqueueTask([this] {
     581        updateSessionState();
     582    });
    573583}
    574584

  • trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h

    r268268 r269077  
    2828
    2929#include "DocumentIdentifier.h"
     30#include "GenericTaskQueue.h"
    3031#include "MediaSessionIdentifier.h"
    3132#include "PlatformMediaSession.h"
    … …  
    180181    friend class Internals;
    181182
     183    void scheduleUpdateSessionState();
    182184    virtual void updateSessionState() { }
    183185
    … …  
    198200
    199201    WeakHashSet<PlatformMediaSession::AudioCaptureSource> m_audioCaptureSources;
     202    GenericTaskQueue<Timer> updateSessionStateQueue;
    200203
    201204#if !RELEASE_LOG_DISABLED

  • trunk/Source/WebCore/platform/audio/mac/AudioSessionMac.mm

    r266559 r269077  
    8282public:
    8383    explicit AudioSessionPrivate() = default;
     84
     85    void addSampleRateObserverIfNeeded();
     86    void addBufferSizeObserverIfNeeded();
     87
     88    static OSStatus handleSampleRateChange(AudioObjectID, UInt32, const AudioObjectPropertyAddress*, void* inClientData);
     89    static OSStatus handleBufferSizeChange(AudioObjectID, UInt32, const AudioObjectPropertyAddress*, void* inClientData);
     90
    8491    Optional<bool> lastMutedState;
    8592    AudioSession::CategoryType category { AudioSession::None };
    … …  
    9198    AudioSession::CategoryType m_categoryOverride;
    9299    bool inRoutingArbitration { false };
     100    bool hasSampleRateObserver { false };
     101    bool hasBufferSizeObserver { false };
     102    Optional<double> sampleRate;
     103    Optional<size_t> bufferSize;
    93104};
     105
     106void AudioSessionPrivate::addSampleRateObserverIfNeeded()
     107{
     108    if (hasSampleRateObserver)
     109        return;
     110    hasSampleRateObserver = true;
     111
     112    AudioObjectPropertyAddress nominalSampleRateAddress = { kAudioDevicePropertyNominalSampleRate, kAudioObjectPropertyScopeGlobal, kAudioObjectPropertyElementMaster };
     113    AudioObjectAddPropertyListener(defaultDevice(), &nominalSampleRateAddress, handleSampleRateChange, this);
     114}
     115
     116OSStatus AudioSessionPrivate::handleSampleRateChange(AudioObjectID device, UInt32, const AudioObjectPropertyAddress* sampleRateAddress, void* inClientData)
     117{
     118    ASSERT(inClientData);
     119    if (!inClientData)
     120        return noErr;
     121
     122    auto* sessionPrivate = static_cast<AudioSessionPrivate*>(inClientData);
     123
     124    Float64 nominalSampleRate;
     125    UInt32 nominalSampleRateSize = sizeof(Float64);
     126    OSStatus result = AudioObjectGetPropertyData(device, sampleRateAddress, 0, 0, &nominalSampleRateSize, (void*)&nominalSampleRate);
     127    if (result)
     128        return result;
     129
     130    sessionPrivate->sampleRate = narrowPrecisionToFloat(nominalSampleRate);
     131    return noErr;
     132}
     133
     134void AudioSessionPrivate::addBufferSizeObserverIfNeeded()
     135{
     136    if (hasBufferSizeObserver)
     137        return;
     138
     139    AudioObjectPropertyAddress bufferSizeAddress = { kAudioDevicePropertyBufferFrameSize, kAudioObjectPropertyScopeGlobal, kAudioObjectPropertyElementMaster };
     140    AudioObjectAddPropertyListener(defaultDevice(), &bufferSizeAddress, handleBufferSizeChange, this);
     141}
     142
     143OSStatus AudioSessionPrivate::handleBufferSizeChange(AudioObjectID device, UInt32, const AudioObjectPropertyAddress* bufferSizeAddress, void* inClientData)
     144{
     145    ASSERT(inClientData);
     146    if (!inClientData)
     147        return noErr;
     148
     149    auto* sessionPrivate = static_cast<AudioSessionPrivate*>(inClientData);
     150
     151    UInt32 bufferSize;
     152    UInt32 bufferSizeSize = sizeof(bufferSize);
     153    OSStatus result = AudioObjectGetPropertyData(device, bufferSizeAddress, 0, 0, &bufferSizeSize, &bufferSize);
     154    if (result)
     155        return result;
     156
     157    sessionPrivate->bufferSize = bufferSize;
     158    return noErr;
     159}
    94160
    95161AudioSession::AudioSession()
    … …  
    188254float AudioSession::sampleRate() const
    189255{
     256    if (m_private->sampleRate)
     257        return *m_private->sampleRate;
     258
     259    m_private->addSampleRateObserverIfNeeded();
     260
    190261    Float64 nominalSampleRate;
    191262    UInt32 nominalSampleRateSize = sizeof(Float64);
    … …  
    199270        return 0;
    200271
     272    m_private->sampleRate = narrowPrecisionToFloat(nominalSampleRate);
     273
    201274    return narrowPrecisionToFloat(nominalSampleRate);
    202275}
    … …  
    204277size_t AudioSession::bufferSize() const
    205278{
     279    if (m_private->bufferSize)
     280        return *m_private->bufferSize;
     281
     282    m_private->addBufferSizeObserverIfNeeded();
     283
    206284    UInt32 bufferSize;
    207285    UInt32 bufferSizeSize = sizeof(bufferSize);
    … …  
    215293    if (result)
    216294        return 0;
     295
     296    m_private->bufferSize = bufferSize;
     297
    217298    return bufferSize;
    218299}
    … …  
    268349size_t AudioSession::preferredBufferSize() const
    269350{
    270     UInt32 bufferSize;
    271     UInt32 bufferSizeSize = sizeof(bufferSize);
    272 
    273     AudioObjectPropertyAddress preferredBufferSizeAddress = {
    274         kAudioDevicePropertyBufferFrameSize,
    275         kAudioObjectPropertyScopeGlobal,
    276         kAudioObjectPropertyElementMaster };
    277     OSStatus result = AudioObjectGetPropertyData(defaultDevice(), &preferredBufferSizeAddress, 0, 0, &bufferSizeSize, &bufferSize);
    278 
    279     if (result)
    280         return 0;
    281     return bufferSize;
     351    return bufferSize();
    282352}
    283353
    284354void AudioSession::setPreferredBufferSize(size_t bufferSize)
    285355{
     356    if (m_private->bufferSize == bufferSize)
     357        return;
     358
    286359    AudioValueRange bufferSizeRange = {0, 0};
    287360    UInt32 bufferSizeRangeSize = sizeof(AudioValueRange);
    … …  
    306379    result = AudioObjectSetPropertyData(defaultDevice(), &preferredBufferSizeAddress, 0, 0, sizeof(bufferSizeOut), (void*)&bufferSizeOut);
    307380
    308 #if LOG_DISABLED
    309     UNUSED_PARAM(result);
    310 #else
     381    if (!result)
     382        m_private->bufferSize = bufferSizeOut;
     383
     384#if !LOG_DISABLED
    311385    if (result)
    312386        LOG(Media, "AudioSession::setPreferredBufferSize(%zu) - failed with error %d", bufferSize, static_cast<int>(result));
Note: See TracChangeset for help on using the changeset viewer.