⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 270961 in webkit


Ignore:
Timestamp:
Dec 17, 2020, 5:21:42 PM (6 years ago)
Author:
Chris Dumez
Message:

[GPUProcess] ​https://www.waveplayer.info/createmediaelementsource-test/ demo is flaky
​https://bugs.webkit.org/show_bug.cgi?id=219951

Reviewed by Geoff Garen.

The issue was with the following line in AudioSourceProviderAVFObjC::prepare:
m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();

In the case where m_ringBuffer was non-null before the assignment, we would have
2 RingBuffers that would coexist for a very small period of time. When the new
one was created, we would send an IPC to the remote process with the shared
memory handle of the new RingBuffer. However, very shortly after, the old
ring buffer would get destroyed, causing us to send another IPC to the remote
process with a null handle (since the shared memory associated with the old
ring buffer is getting destroyed). As a result, of this ordering issue, the
remote process would end up with a RingBuffer with a null shared memory handle
and no audio would be rendered.

We could have addressed the issue like so:
`
m_ringBuffer = nullptr;
m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();
`
However, this would be super fragile. Instead, I have made the following changes:

  1. If there is already a ringBuffer, reuse it instead of reconstructing it. Calling allocate() with the new parameters on the existing ring buffer is sufficient in this case.
  2. Because of 1, the ring buffer creation callback no longer needs to call CARingBuffer::allocate().

I also made the following changes to make the code simpler and to reduce code
duplication:

  • The storage change handler passed to SharedRingBufferStorage is now given as parameter the CAAudioStreamDescription & frameCount. What the handler always does is send an IPC to the remote process to tell it that the storage changed and in all cases, it needs to provide these 2 parameters as well. This is because the remote process will need to call CARingBuffer::allocate(), which requires those 2 parameters. This simplifies our code in some cases since we no longer need a mechanism to retrieve those 2 parameters from inside the storage change handler.
  • The logic of the StorageChange IPC recipient to update its ringbuffer with the new shared memory handle is complicated and was duplicated in a LOT of places. To address this, I introduced a new SharedRingBufferStorage::updateReadOnlyStorage() function which does exactly what we need.

Source/WebCore:

  • platform/audio/cocoa/CARingBuffer.cpp:

(WebCore::CARingBuffer::allocate):

  • platform/audio/cocoa/CARingBuffer.h:
  • platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
  • platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:

(WebCore::AudioSourceProviderAVFObjC::AudioSourceProviderAVFObjC):
(WebCore::AudioSourceProviderAVFObjC::prepare):
(WebCore::AudioSourceProviderAVFObjC::setRingBufferCreationCallback):

Source/WebKit:

  • GPUProcess/media/RemoteAudioDestinationManager.cpp:

(WebKit::RemoteAudioDestination::audioSamplesStorageChanged):

  • GPUProcess/media/RemoteAudioSourceProviderProxy.cpp:

(WebKit::RemoteAudioSourceProviderProxy::create):
(WebKit::RemoteAudioSourceProviderProxy::createRingBuffer):
(WebKit::RemoteAudioSourceProviderProxy::storageChanged):

  • GPUProcess/media/RemoteAudioSourceProviderProxy.h:
  • GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp:

(WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesStorageChanged):

  • GPUProcess/webrtc/RemoteMediaRecorder.cpp:

(WebKit::RemoteMediaRecorder::audioSamplesStorageChanged):

  • Shared/Cocoa/SharedRingBufferStorage.cpp:

(WebKit::SharedRingBufferStorage::setStorage):
(WebKit::SharedRingBufferStorage::updateReadOnlyStorage):
(WebKit::SharedRingBufferStorage::allocate):
(WebKit::SharedRingBufferStorage::deallocate):

  • Shared/Cocoa/SharedRingBufferStorage.h:

(WebKit::SharedRingBufferStorage::SharedRingBufferStorage):
(WebKit::SharedRingBufferStorage::storage const):

  • UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:

(WebKit::UserMediaCaptureManagerProxy::SourceProxy::SourceProxy):
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::storageChanged):

  • UIProcess/SpeechRecognitionRemoteRealtimeMediaSource.cpp:

(WebKit::SpeechRecognitionRemoteRealtimeMediaSource::setStorage):

  • WebProcess/GPU/media/RemoteAudioDestinationProxy.cpp:

(WebKit::RemoteAudioDestinationProxy::RemoteAudioDestinationProxy):
(WebKit::RemoteAudioDestinationProxy::storageChanged):

  • WebProcess/GPU/media/RemoteAudioDestinationProxy.h:
  • WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp:

(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::setStorage):

  • WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp:

(WebKit::AudioMediaStreamTrackRenderer::AudioMediaStreamTrackRenderer):
(WebKit::AudioMediaStreamTrackRenderer::storageChanged):

  • WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h:
  • WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp:

(WebKit::MediaRecorderPrivate::startRecording):
(WebKit::MediaRecorderPrivate::storageChanged):

  • WebProcess/GPU/webrtc/MediaRecorderPrivate.h:
  • WebProcess/cocoa/RemoteCaptureSampleManager.cpp:

(WebKit::RemoteCaptureSampleManager::RemoteAudio::setStorage):

Location:
trunk/Source
Files:
23 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r270958 r270961  
     12020-12-17  Chris Dumez  <cdumez@apple.com>
     2
     3        [GPUProcess] https://www.waveplayer.info/createmediaelementsource-test/ demo is flaky
     4        https://bugs.webkit.org/show_bug.cgi?id=219951
     5
     6        Reviewed by Geoff Garen.
     7
     8        The issue was with the following line in AudioSourceProviderAVFObjC::prepare:
     9        `m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();`
     10
     11        In the case where m_ringBuffer was non-null before the assignment, we would have
     12        2 RingBuffers that would coexist for a very small period of time. When the new
     13        one was created, we would send an IPC to the remote process with the shared
     14        memory handle of the new RingBuffer. However, very shortly after, the old
     15        ring buffer would get destroyed, causing us to send another IPC to the remote
     16        process with a null handle (since the shared memory associated with the old
     17        ring buffer is getting destroyed). As a result, of this ordering issue, the
     18        remote process would end up with a RingBuffer with a null shared memory handle
     19        and no audio would be rendered.
     20
     21        We could have addressed the issue like so:
     22        ```
     23        m_ringBuffer = nullptr;
     24        m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();
     25        ```
     26        However, this would be super fragile. Instead, I have made the following changes:
     27        1. If there is already a ringBuffer, reuse it instead of reconstructing it.
     28           Calling allocate() with the new parameters on the existing ring buffer is
     29           sufficient in this case.
     30        2. Because of 1, the ring buffer creation callback no longer needs to call
     31           CARingBuffer::allocate().
     32
     33        I also made the following changes to make the code simpler and to reduce code
     34        duplication:
     35        - The storage change handler passed to SharedRingBufferStorage is now given
     36          as parameter the CAAudioStreamDescription & frameCount. What the handler
     37          always does is send an IPC to the remote process to tell it that the
     38          storage changed and in all cases, it needs to provide these 2 parameters
     39          as well. This is because the remote process will need to call
     40          CARingBuffer::allocate(), which requires those 2 parameters. This
     41          simplifies our code in some cases since we no longer need a mechanism
     42          to retrieve those 2 parameters from inside the storage change handler.
     43        - The logic of the StorageChange IPC recipient to update its ringbuffer
     44          with the new shared memory handle is complicated and was duplicated
     45          in a LOT of places. To address this, I introduced a new
     46          SharedRingBufferStorage::updateReadOnlyStorage() function which does
     47          exactly what we need.
     48
     49        * platform/audio/cocoa/CARingBuffer.cpp:
     50        (WebCore::CARingBuffer::allocate):
     51        * platform/audio/cocoa/CARingBuffer.h:
     52        * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
     53        * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:
     54        (WebCore::AudioSourceProviderAVFObjC::AudioSourceProviderAVFObjC):
     55        (WebCore::AudioSourceProviderAVFObjC::prepare):
     56        (WebCore::AudioSourceProviderAVFObjC::setRingBufferCreationCallback):
     57
    1582020-12-17  Zalan Bujtas  <zalan@apple.com>
    259

  • trunk/Source/WebCore/platform/audio/cocoa/CARingBuffer.cpp

    r270938 r270961  
    7272    m_capacityBytes = m_bytesPerFrame * frameCount;
    7373
    74     m_buffers->allocate(m_capacityBytes * m_channelCount);
     74    m_buffers->allocate(m_capacityBytes * m_channelCount, format, frameCount);
    7575
    7676    m_pointers.resize(m_channelCount);

  • trunk/Source/WebCore/platform/audio/cocoa/CARingBuffer.h

    r270938 r270961  
    4343public:
    4444    virtual ~CARingBufferStorage() = default;
    45     virtual void allocate(size_t) = 0;
     45    virtual void allocate(size_t, const CAAudioStreamDescription& format, size_t frameCount) = 0;
    4646    virtual void deallocate() = 0;
    4747    virtual void* data() = 0;
    … …  
    6060
    6161private:
    62     void allocate(size_t byteCount) final { m_buffer.grow(byteCount); }
     62    void allocate(size_t byteCount, const CAAudioStreamDescription&, size_t) final { m_buffer.grow(byteCount); }
    6363    void deallocate() final { m_buffer.clear(); }
    6464    void* data() final { return m_buffer.data(); }
    … …  
    9999    };
    100100
    101     WEBCORE_EXPORT void allocate(const CAAudioStreamDescription&, size_t);
     101    WEBCORE_EXPORT void allocate(const CAAudioStreamDescription&, size_t frameCount);
    102102    WEBCORE_EXPORT void deallocate();
    103103

  • trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h

    r268577 r270961  
    6464    using AudioCallback = Function<void(uint64_t startFrame, uint64_t numberOfFrames)>;
    6565    WEBCORE_EXPORT void setAudioCallback(AudioCallback&&);
    66     using RingBufferCreationCallback = Function<UniqueRef<CARingBuffer>(const CAAudioStreamDescription&, size_t)>;
     66    using RingBufferCreationCallback = Function<UniqueRef<CARingBuffer>()>;
    6767    WEBCORE_EXPORT void setRingBufferCreationCallback(RingBufferCreationCallback&&);
    6868
    … …  
    112112    RefPtr<TapStorage> m_tapStorage;
    113113    AudioCallback m_audioCallback;
    114     RingBufferCreationCallback m_ringBufferCallback;
     114    RingBufferCreationCallback m_ringBufferCreationCallback;
    115115};
    116116

  • trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm

    r268577 r270961  
    8181AudioSourceProviderAVFObjC::AudioSourceProviderAVFObjC(AVPlayerItem *item)
    8282    : m_avPlayerItem(item)
     83    , m_ringBufferCreationCallback([] { return makeUniqueRef<CARingBuffer>(); })
    8384{
    8485}
    … …  
    339340
    340341    CAAudioStreamDescription description { *processingFormat };
    341     if (m_ringBufferCallback)
    342         m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();
    343     else {
    344         m_ringBuffer = makeUnique<CARingBuffer>();
    345         m_ringBuffer->allocate(description, capacity);
    346     }
     342    if (!m_ringBuffer)
     343        m_ringBuffer = m_ringBufferCreationCallback().moveToUniquePtr();
     344    m_ringBuffer->allocate(description, capacity);
    347345
    348346    // AudioBufferList is a variable-length struct, so create on the heap with a generic new() operator
    … …  
    439437{
    440438    ASSERT(!m_avAudioMix);
    441     m_ringBufferCallback = WTFMove(callback);
     439    m_ringBufferCreationCallback = WTFMove(callback);
    442440}
    443441

  • trunk/Source/WebKit/ChangeLog

    r270951 r270961  
     12020-12-17  Chris Dumez  <cdumez@apple.com>
     2
     3        [GPUProcess] https://www.waveplayer.info/createmediaelementsource-test/ demo is flaky
     4        https://bugs.webkit.org/show_bug.cgi?id=219951
     5
     6        Reviewed by Geoff Garen.
     7
     8        The issue was with the following line in AudioSourceProviderAVFObjC::prepare:
     9        `m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();`
     10
     11        In the case where m_ringBuffer was non-null before the assignment, we would have
     12        2 RingBuffers that would coexist for a very small period of time. When the new
     13        one was created, we would send an IPC to the remote process with the shared
     14        memory handle of the new RingBuffer. However, very shortly after, the old
     15        ring buffer would get destroyed, causing us to send another IPC to the remote
     16        process with a null handle (since the shared memory associated with the old
     17        ring buffer is getting destroyed). As a result, of this ordering issue, the
     18        remote process would end up with a RingBuffer with a null shared memory handle
     19        and no audio would be rendered.
     20
     21        We could have addressed the issue like so:
     22        ```
     23        m_ringBuffer = nullptr;
     24        m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();
     25        ```
     26        However, this would be super fragile. Instead, I have made the following changes:
     27        1. If there is already a ringBuffer, reuse it instead of reconstructing it.
     28           Calling allocate() with the new parameters on the existing ring buffer is
     29           sufficient in this case.
     30        2. Because of 1, the ring buffer creation callback no longer needs to call
     31           CARingBuffer::allocate().
     32
     33        I also made the following changes to make the code simpler and to reduce code
     34        duplication:
     35        - The storage change handler passed to SharedRingBufferStorage is now given
     36          as parameter the CAAudioStreamDescription & frameCount. What the handler
     37          always does is send an IPC to the remote process to tell it that the
     38          storage changed and in all cases, it needs to provide these 2 parameters
     39          as well. This is because the remote process will need to call
     40          CARingBuffer::allocate(), which requires those 2 parameters. This
     41          simplifies our code in some cases since we no longer need a mechanism
     42          to retrieve those 2 parameters from inside the storage change handler.
     43        - The logic of the StorageChange IPC recipient to update its ringbuffer
     44          with the new shared memory handle is complicated and was duplicated
     45          in a LOT of places. To address this, I introduced a new
     46          SharedRingBufferStorage::updateReadOnlyStorage() function which does
     47          exactly what we need.
     48
     49        * GPUProcess/media/RemoteAudioDestinationManager.cpp:
     50        (WebKit::RemoteAudioDestination::audioSamplesStorageChanged):
     51        * GPUProcess/media/RemoteAudioSourceProviderProxy.cpp:
     52        (WebKit::RemoteAudioSourceProviderProxy::create):
     53        (WebKit::RemoteAudioSourceProviderProxy::createRingBuffer):
     54        (WebKit::RemoteAudioSourceProviderProxy::storageChanged):
     55        * GPUProcess/media/RemoteAudioSourceProviderProxy.h:
     56        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp:
     57        (WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesStorageChanged):
     58        * GPUProcess/webrtc/RemoteMediaRecorder.cpp:
     59        (WebKit::RemoteMediaRecorder::audioSamplesStorageChanged):
     60        * Shared/Cocoa/SharedRingBufferStorage.cpp:
     61        (WebKit::SharedRingBufferStorage::setStorage):
     62        (WebKit::SharedRingBufferStorage::updateReadOnlyStorage):
     63        (WebKit::SharedRingBufferStorage::allocate):
     64        (WebKit::SharedRingBufferStorage::deallocate):
     65        * Shared/Cocoa/SharedRingBufferStorage.h:
     66        (WebKit::SharedRingBufferStorage::SharedRingBufferStorage):
     67        (WebKit::SharedRingBufferStorage::storage const):
     68        * UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
     69        (WebKit::UserMediaCaptureManagerProxy::SourceProxy::SourceProxy):
     70        (WebKit::UserMediaCaptureManagerProxy::SourceProxy::storageChanged):
     71        * UIProcess/SpeechRecognitionRemoteRealtimeMediaSource.cpp:
     72        (WebKit::SpeechRecognitionRemoteRealtimeMediaSource::setStorage):
     73        * WebProcess/GPU/media/RemoteAudioDestinationProxy.cpp:
     74        (WebKit::RemoteAudioDestinationProxy::RemoteAudioDestinationProxy):
     75        (WebKit::RemoteAudioDestinationProxy::storageChanged):
     76        * WebProcess/GPU/media/RemoteAudioDestinationProxy.h:
     77        * WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp:
     78        (WebKit::RemoteAudioSourceProviderManager::RemoteAudio::setStorage):
     79        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp:
     80        (WebKit::AudioMediaStreamTrackRenderer::AudioMediaStreamTrackRenderer):
     81        (WebKit::AudioMediaStreamTrackRenderer::storageChanged):
     82        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h:
     83        * WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp:
     84        (WebKit::MediaRecorderPrivate::startRecording):
     85        (WebKit::MediaRecorderPrivate::storageChanged):
     86        * WebProcess/GPU/webrtc/MediaRecorderPrivate.h:
     87        * WebProcess/cocoa/RemoteCaptureSampleManager.cpp:
     88        (WebKit::RemoteCaptureSampleManager::RemoteAudio::setStorage):
     89
    1902020-12-17  Chris Dumez  <cdumez@apple.com>
    291

  • trunk/Source/WebKit/GPUProcess/media/RemoteAudioDestinationManager.cpp

    r270938 r270961  
    7373    void audioSamplesStorageChanged(const SharedMemory::IPCHandle& ipcHandle, const WebCore::CAAudioStreamDescription& description, uint64_t numberOfFrames)
    7474    {
    75         m_description = description;
    76 
    77         if (ipcHandle.handle.isNull()) {
    78             m_ringBuffer->deallocate();
    79             storage().setReadOnly(false);
    80             storage().setStorage(nullptr);
    81             return;
    82         }
    83 
    84         auto memory = SharedMemory::map(ipcHandle.handle, SharedMemory::Protection::ReadOnly);
    85         storage().setStorage(WTFMove(memory));
    86         storage().setReadOnly(true);
    87         m_ringBuffer->allocate(description, numberOfFrames);
     75        storage().updateReadOnlyStorage(m_ringBuffer.get(), ipcHandle.handle, description, numberOfFrames);
    8876    }
    8977#endif
    … …  
    165153    WebCore::AudioOutputUnitAdaptor m_audioOutputUnitAdaptor;
    166154
    167     WebCore::CAAudioStreamDescription m_description;
    168155    UniqueRef<WebCore::CARingBuffer> m_ringBuffer;
    169156    MachSemaphore m_renderSemaphore;

  • trunk/Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.cpp

    r270804 r270961  
    3838    auto remoteProvider = adoptRef(*new RemoteAudioSourceProviderProxy(identifier, WTFMove(connection)));
    3939
    40     localProvider.setRingBufferCreationCallback([remoteProvider](auto description, auto capacity) {
    41         return remoteProvider->createRingBuffer(description, capacity);
     40    localProvider.setRingBufferCreationCallback([remoteProvider]() {
     41        return remoteProvider->createRingBuffer();
    4242    });
    4343    localProvider.setAudioCallback([remoteProvider](auto startFrame, auto numberOfFrames) {
    … …  
    5656RemoteAudioSourceProviderProxy::~RemoteAudioSourceProviderProxy() = default;
    5757
    58 UniqueRef<CARingBuffer> RemoteAudioSourceProviderProxy::createRingBuffer(const CAAudioStreamDescription& description, size_t capacity)
     58UniqueRef<CARingBuffer> RemoteAudioSourceProviderProxy::createRingBuffer()
    5959{
    60     m_ringBufferDescription = description;
    61     m_ringBufferCapacity = capacity;
    62     auto ringBuffer = makeUniqueRef<CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>([protectedThis = makeRef(*this)](SharedMemory* memory) mutable {
    63         protectedThis->storageChanged(memory);
     60    return makeUniqueRef<CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>([protectedThis = makeRef(*this)](SharedMemory* memory, const CAAudioStreamDescription& format, size_t frameCount) mutable {
     61        protectedThis->storageChanged(memory, format, frameCount);
    6462    }));
    65     ringBuffer->allocate(description, capacity);
    66     return ringBuffer;
    6763}
    6864
    … …  
    7268}
    7369
    74 void RemoteAudioSourceProviderProxy::storageChanged(SharedMemory* memory)
     70void RemoteAudioSourceProviderProxy::storageChanged(SharedMemory* memory, const CAAudioStreamDescription& format, size_t frameCount)
    7571{
    7672    SharedMemory::Handle handle;
    … …  
    8480    uint64_t dataSize = 0;
    8581#endif
    86     m_connection->send(Messages::RemoteAudioSourceProviderManager::AudioStorageChanged { m_identifier, SharedMemory::IPCHandle { WTFMove(handle),  dataSize }, m_ringBufferDescription, m_ringBufferCapacity }, 0);
     82    m_connection->send(Messages::RemoteAudioSourceProviderManager::AudioStorageChanged { m_identifier, SharedMemory::IPCHandle { WTFMove(handle),  dataSize }, format, frameCount }, 0);
    8783}
    8884

  • trunk/Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.h

    r270804 r270961  
    4848    ~RemoteAudioSourceProviderProxy();
    4949
    50     UniqueRef<WebCore::CARingBuffer> createRingBuffer(const WebCore::CAAudioStreamDescription&, size_t);
     50    UniqueRef<WebCore::CARingBuffer> createRingBuffer();
    5151    void newAudioSamples(uint64_t startFrame, uint64_t endFrame);
    5252
    … …  
    5454    RemoteAudioSourceProviderProxy(WebCore::MediaPlayerIdentifier, Ref<IPC::Connection>&&);
    5555
    56     void storageChanged(SharedMemory*);
     56    void storageChanged(SharedMemory*, const WebCore::CAAudioStreamDescription& format, size_t frameCount);
    5757
    5858    // AudioSourceProviderClient
    … …  
    6161    WebCore::MediaPlayerIdentifier m_identifier;
    6262    Ref<IPC::Connection> m_connection;
    63 
    64     WebCore::CAAudioStreamDescription m_ringBufferDescription;
    65     size_t m_ringBufferCapacity { 0 };
    6663};
    6764

  • trunk/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp

    r270804 r270961  
    106106    m_description = description;
    107107
    108     if (ipcHandle.handle.isNull()) {
    109         m_ringBuffer->deallocate();
    110         storage().setReadOnly(false);
    111         storage().setStorage(nullptr);
    112         return;
    113     }
    114 
    115     auto memory = SharedMemory::map(ipcHandle.handle, SharedMemory::Protection::ReadOnly);
    116     storage().setStorage(WTFMove(memory));
    117     storage().setReadOnly(true);
    118 
    119     m_ringBuffer->allocate(description, numberOfFrames);
     108    storage().updateReadOnlyStorage(m_ringBuffer.get(), ipcHandle.handle, description, numberOfFrames);
    120109
    121110    m_audioBufferList = makeUnique<WebAudioBufferList>(m_description);

  • trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp

    r270804 r270961  
    7575    m_description = description;
    7676
    77     if (ipcHandle.handle.isNull()) {
    78         m_ringBuffer->deallocate();
    79         storage().setReadOnly(false);
    80         storage().setStorage(nullptr);
    81         return;
    82     }
    83 
    84     auto memory = SharedMemory::map(ipcHandle.handle, SharedMemory::Protection::ReadOnly);
    85     storage().setStorage(WTFMove(memory));
    86     storage().setReadOnly(true);
    87 
    88     m_ringBuffer->allocate(m_description, numberOfFrames);
     77    storage().updateReadOnlyStorage(*m_ringBuffer, ipcHandle.handle, description, numberOfFrames);
    8978    m_audioBufferList = makeUnique<WebAudioBufferList>(m_description);
    9079}

  • trunk/Source/WebKit/Shared/Cocoa/SharedRingBufferStorage.cpp

    r270938 r270961  
    2929#if USE(MEDIATOOLBOX)
    3030
     31#include <WebCore/CARingBuffer.h>
     32
    3133namespace WebKit {
    3234
    33 void SharedRingBufferStorage::setStorage(RefPtr<SharedMemory>&& storage)
     35void SharedRingBufferStorage::setStorage(RefPtr<SharedMemory>&& storage, const CAAudioStreamDescription& format, size_t frameCount)
    3436{
    3537    ASSERT(storage || !m_readOnly);
    3638    m_storage = WTFMove(storage);
    3739    if (m_storageChangedHandler)
    38         m_storageChangedHandler(m_storage.get());
     40        m_storageChangedHandler(m_storage.get(), format, frameCount);
    3941}
    4042
    41 void SharedRingBufferStorage::setReadOnly(bool readOnly)
     43void SharedRingBufferStorage::updateReadOnlyStorage(WebCore::CARingBuffer& ringBuffer, const SharedMemory::Handle& handle, const CAAudioStreamDescription& format, size_t frameCount)
    4244{
    43     ASSERT(m_storage || !readOnly);
    44     m_readOnly = readOnly;
     45    if (handle.isNull()) {
     46        ringBuffer.deallocate();
     47        m_readOnly = false;
     48        m_storage = nullptr;
     49        return;
     50    }
     51
     52    auto memory = SharedMemory::map(handle, SharedMemory::Protection::ReadOnly);
     53    m_storage = WTFMove(memory);
     54    m_readOnly = true;
     55    ringBuffer.allocate(format, frameCount);
    4556}
    4657
    47 void SharedRingBufferStorage::allocate(size_t byteCount)
     58void SharedRingBufferStorage::allocate(size_t byteCount, const CAAudioStreamDescription& format, size_t frameCount)
    4859{
    4960    if (!m_readOnly) {
    5061        auto sharedMemory = SharedMemory::allocate(byteCount + sizeof(FrameBounds));
    5162        new (NotNull, sharedMemory->data()) FrameBounds;
    52         setStorage(WTFMove(sharedMemory));
     63        setStorage(WTFMove(sharedMemory), format, frameCount);
    5364    }
    5465}
    … …  
    5768{
    5869    if (!m_readOnly)
    59         setStorage(nullptr);
     70        setStorage(nullptr, { }, 0);
    6071}
    6172

  • trunk/Source/WebKit/Shared/Cocoa/SharedRingBufferStorage.h

    r270938 r270961  
    3333#include <wtf/Function.h>
    3434
     35namespace WebCore {
     36class CARingBuffer;
     37}
     38
    3539namespace WebKit {
    3640
    3741class SharedRingBufferStorage : public WebCore::CARingBufferStorage {
    3842public:
    39     SharedRingBufferStorage(Function<void(SharedMemory*)>&& storageChangedHandler = nullptr)
     43    SharedRingBufferStorage(Function<void(SharedMemory*, const WebCore::CAAudioStreamDescription& format, size_t frameCount)>&& storageChangedHandler = nullptr)
    4044        : m_storageChangedHandler(WTFMove(storageChangedHandler))
    4145    {
    … …  
    4448    void invalidate() { m_storageChangedHandler = nullptr; }
    4549
    46     RefPtr<SharedMemory> storage() const { return m_storage; }
    47     void setStorage(RefPtr<SharedMemory>&&);
    48 
    49     bool readOnly() const { return m_readOnly; }
    50     void setReadOnly(bool);
     50    SharedMemory* storage() const { return m_storage.get(); }
     51    void updateReadOnlyStorage(WebCore::CARingBuffer&, const SharedMemory::Handle&, const WebCore::CAAudioStreamDescription& format, size_t frameCount);
    5152
    5253    // WebCore::CARingBufferStorage
    53     void allocate(size_t) final;
     54    void allocate(size_t, const WebCore::CAAudioStreamDescription& format, size_t frameCount) final;
    5455    void deallocate() final;
    5556    void* data() final;
    … …  
    6869    };
    6970
     71    void setStorage(RefPtr<SharedMemory>&&, const WebCore::CAAudioStreamDescription& format, size_t frameCount);
    7072    FrameBounds* sharedFrameBounds() const;
    7173
    72     Function<void(SharedMemory*)> m_storageChangedHandler;
     74    Function<void(SharedMemory*, const WebCore::CAAudioStreamDescription& format, size_t frameCount)> m_storageChangedHandler;
    7375    RefPtr<SharedMemory> m_storage;
    7476    bool m_readOnly { false };

  • trunk/Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp

    r270804 r270961  
    6060        , m_connection(WTFMove(connection))
    6161        , m_source(WTFMove(source))
    62         , m_ringBuffer(makeUniqueRef<SharedRingBufferStorage>(std::bind(&SourceProxy::storageChanged, this, std::placeholders::_1)))
     62        , m_ringBuffer(makeUniqueRef<SharedRingBufferStorage>(std::bind(&SourceProxy::storageChanged, this, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3)))
    6363    {
    6464        m_source->addObserver(*this);
    … …  
    195195    }
    196196
    197     void storageChanged(SharedMemory* storage)
     197    void storageChanged(SharedMemory* storage, const WebCore::CAAudioStreamDescription& format, size_t frameCount)
    198198    {
    199199        SharedMemory::Handle handle;
    … …  
    207207        uint64_t dataSize = 0;
    208208#endif
    209         m_connection->send(Messages::RemoteCaptureSampleManager::AudioStorageChanged(m_id, SharedMemory::IPCHandle { WTFMove(handle),  dataSize }, m_description, m_numberOfFrames), 0);
     209        m_connection->send(Messages::RemoteCaptureSampleManager::AudioStorageChanged(m_id, SharedMemory::IPCHandle { WTFMove(handle),  dataSize }, format, frameCount), 0);
    210210    }
    211211

  • trunk/Source/WebKit/UIProcess/SpeechRecognitionRemoteRealtimeMediaSource.cpp

    r270804 r270961  
    7878    m_description = description;
    7979
    80     RefPtr<SharedMemory> memory;
    81     if (!handle.isNull()) {
    82         memory = SharedMemory::map(handle, SharedMemory::Protection::ReadOnly);
    83         LOG_ERROR("Unable to create shared memory for remote source");
    84     }
    85 
    8680    auto& storage = static_cast<SharedRingBufferStorage&>(m_ringBuffer->storage());
    87     if (!memory) {
    88         m_ringBuffer->deallocate();
    89         storage.setReadOnly(false);
    90         storage.setStorage(nullptr);
    91         return;
    92     }
    93 
    94     storage.setStorage(memory.releaseNonNull());
    95     storage.setReadOnly(true);
    96     m_ringBuffer->allocate(description, numberOfFrames);
     81    storage.updateReadOnlyStorage(*m_ringBuffer, handle, description, numberOfFrames);
    9782    m_buffer = makeUnique<WebAudioBufferList>(description, numberOfFrames);
    9883}

  • trunk/Source/WebKit/WebProcess/GPU/media/RemoteAudioDestinationProxy.cpp

    r270951 r270961  
    6262    : WebCore::AudioDestinationCocoa(callback, numberOfOutputChannels, sampleRate, false)
    6363    , m_numberOfFrames(hardwareSampleRate() * ringBufferSizeInSecond)
    64     , m_ringBuffer(makeUnique<WebCore::CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(std::bind(&RemoteAudioDestinationProxy::storageChanged, this, std::placeholders::_1))))
     64    , m_ringBuffer(makeUnique<WebCore::CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(std::bind(&RemoteAudioDestinationProxy::storageChanged, this, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3))))
    6565    , m_sampleRate(hardwareSampleRate())
    6666#else
    … …  
    183183
    184184#if PLATFORM(COCOA)
    185 void RemoteAudioDestinationProxy::storageChanged(SharedMemory* storage)
     185void RemoteAudioDestinationProxy::storageChanged(SharedMemory* storage, const WebCore::CAAudioStreamDescription& format, size_t frameCount)
    186186{
    187187    SharedMemory::Handle handle;
    … …  
    196196#endif
    197197
    198     AudioStreamBasicDescription streamFormat;
    199     getAudioStreamBasicDescription(streamFormat);
    200     WebCore::CAAudioStreamDescription description(streamFormat);
    201 
    202     WebProcess::singleton().ensureGPUProcessConnection().connection().send(Messages::RemoteAudioDestinationManager::AudioSamplesStorageChanged { m_destinationID, SharedMemory::IPCHandle { WTFMove(handle), dataSize }, streamFormat, m_numberOfFrames }, 0);
     198    WebProcess::singleton().ensureGPUProcessConnection().connection().send(Messages::RemoteAudioDestinationManager::AudioSamplesStorageChanged { m_destinationID, SharedMemory::IPCHandle { WTFMove(handle), dataSize }, format, frameCount }, 0);
    203199}
    204200#endif

  • trunk/Source/WebKit/WebProcess/GPU/media/RemoteAudioDestinationProxy.h

    r270947 r270961  
    9797
    9898#if PLATFORM(COCOA)
    99     void storageChanged(SharedMemory*);
     99    void storageChanged(SharedMemory*, const WebCore::CAAudioStreamDescription& format, size_t frameCount);
    100100#endif
    101101

  • trunk/Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp

    r270804 r270961  
    125125    m_description = description;
    126126
    127     RefPtr<SharedMemory> memory;
    128     if (!handle.isNull()) {
    129         memory = SharedMemory::map(handle, SharedMemory::Protection::ReadOnly);
    130         RELEASE_LOG_ERROR_IF(!memory, Media, "Unable to create shared memory for audio provider %llu", m_provider->identifier().toUInt64());
    131     }
    132 
    133127    auto& storage = static_cast<SharedRingBufferStorage&>(m_ringBuffer->storage());
    134     if (!memory) {
    135         m_ringBuffer->deallocate();
    136         storage.setReadOnly(false);
    137         storage.setStorage(nullptr);
    138         return;
    139     }
    140 
    141     storage.setStorage(memory.releaseNonNull());
    142     storage.setReadOnly(true);
    143     m_ringBuffer->allocate(description, numberOfFrames);
     128    storage.updateReadOnlyStorage(*m_ringBuffer, handle, description, numberOfFrames);
    144129    m_buffer = makeUnique<WebAudioBufferList>(description, numberOfFrames);
    145130}

  • trunk/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp

    r270804 r270961  
    4747    : m_connection(WTFMove(connection))
    4848    , m_identifier(AudioMediaStreamTrackRendererIdentifier::generate())
    49     , m_ringBuffer(makeUnique<WebCore::CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(std::bind(&AudioMediaStreamTrackRenderer::storageChanged, this, std::placeholders::_1))))
     49    , m_ringBuffer(makeUnique<WebCore::CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(std::bind(&AudioMediaStreamTrackRenderer::storageChanged, this, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3))))
    5050{
    5151    m_connection->send(Messages::RemoteAudioMediaStreamTrackRendererManager::CreateRenderer { m_identifier }, 0);
    … …  
    9999}
    100100
    101 void AudioMediaStreamTrackRenderer::storageChanged(SharedMemory* storage)
     101void AudioMediaStreamTrackRenderer::storageChanged(SharedMemory* storage, const WebCore::CAAudioStreamDescription& format, size_t frameCount)
    102102{
    103103    SharedMemory::Handle handle;
    … …  
    111111    uint64_t dataSize = 0;
    112112#endif
    113     m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::AudioSamplesStorageChanged { SharedMemory::IPCHandle { WTFMove(handle), dataSize }, m_description, static_cast<uint64_t>(m_numberOfFrames) }, m_identifier);
     113    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::AudioSamplesStorageChanged { SharedMemory::IPCHandle { WTFMove(handle), dataSize }, format, frameCount }, m_identifier);
    114114}
    115115

  • trunk/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h

    r270804 r270961  
    5050    explicit AudioMediaStreamTrackRenderer(Ref<IPC::Connection>&&);
    5151
    52     void storageChanged(SharedMemory*);
     52    void storageChanged(SharedMemory*, const WebCore::CAAudioStreamDescription& format, size_t frameCount);
    5353
    5454    // WebCore::AudioMediaStreamTrackRenderer

  • trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp

    r270804 r270961  
    6060    auto selectedTracks = MediaRecorderPrivate::selectTracks(m_stream);
    6161    if (selectedTracks.audioTrack)
    62         m_ringBuffer = makeUnique<CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(std::bind(&MediaRecorderPrivate::storageChanged, this, std::placeholders::_1)));
     62        m_ringBuffer = makeUnique<CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(std::bind(&MediaRecorderPrivate::storageChanged, this, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3)));
    6363
    6464    m_connection->sendWithAsyncReply(Messages::RemoteMediaRecorderManager::CreateRecorder { m_identifier, !!selectedTracks.audioTrack, !!selectedTracks.videoTrack, m_options }, [this, weakThis = makeWeakPtr(this), audioTrack = makeRefPtr(selectedTracks.audioTrack), videoTrack = makeRefPtr(selectedTracks.videoTrack), callback = WTFMove(callback)](auto&& exception, String&& mimeType, unsigned audioBitRate, unsigned videoBitRate) mutable {
    … …  
    108108}
    109109
    110 void MediaRecorderPrivate::storageChanged(SharedMemory* storage)
     110void MediaRecorderPrivate::storageChanged(SharedMemory* storage, const WebCore::CAAudioStreamDescription& format, size_t frameCount)
    111111{
    112112    SharedMemory::Handle handle;
    … …  
    120120    uint64_t dataSize = 0;
    121121#endif
    122     m_connection->send(Messages::RemoteMediaRecorder::AudioSamplesStorageChanged { SharedMemory::IPCHandle { WTFMove(handle), dataSize }, m_description, static_cast<uint64_t>(m_numberOfFrames) }, m_identifier);
     122    m_connection->send(Messages::RemoteMediaRecorder::AudioSamplesStorageChanged { SharedMemory::IPCHandle { WTFMove(handle), dataSize }, format, frameCount }, m_identifier);
    123123}
    124124

  • trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h

    r270804 r270961  
    6464    void resumeRecording(CompletionHandler<void()>&&) final;
    6565
    66     void storageChanged(SharedMemory*);
     66    void storageChanged(SharedMemory*, const WebCore::CAAudioStreamDescription& format, size_t frameCount);
    6767
    6868    MediaRecorderIdentifier m_identifier;

  • trunk/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp

    r270804 r270961  
    121121{
    122122    m_description = description;
    123 
    124     RefPtr<SharedMemory> memory;
    125     if (!handle.isNull()) {
    126         memory = SharedMemory::map(handle, SharedMemory::Protection::ReadOnly);
    127         RELEASE_LOG_ERROR_IF(!memory, WebRTC, "Unable to create shared memory for audio source %llu", m_source->identifier().toUInt64());
    128     }
    129 
    130123    auto& storage = static_cast<SharedRingBufferStorage&>(m_ringBuffer->storage());
    131     if (!memory) {
    132         m_ringBuffer->deallocate();
    133         storage.setReadOnly(false);
    134         storage.setStorage(nullptr);
    135         return;
    136     }
    137 
    138     storage.setStorage(memory.releaseNonNull());
    139     storage.setReadOnly(true);
    140     m_ringBuffer->allocate(description, numberOfFrames);
    141     m_buffer = makeUnique<WebAudioBufferList>(description, numberOfFrames);
     124    storage.updateReadOnlyStorage(*m_ringBuffer, handle, description, numberOfFrames);
    142125}
    143126
Note: See TracChangeset for help on using the changeset viewer.