Changeset 268577 in webkit


Ignore:
Timestamp:
Oct 16, 2020 2:48:57 AM (4 years ago)
Author:
youenn@apple.com
Message:

Add support for GPUProcess WebAudio media element providers
​https://bugs.webkit.org/show_bug.cgi?id=217704

Reviewed by Eric Carlson.

Source/WebCore:

Update AudioSourceProviderAVFObjC so that a specific ring buffer can be provided and to be notified of new samples being pushed.

Covered by existing web audio tests run with GPUProcess.

  • WebCore.xcodeproj/project.pbxproj:
  • platform/audio/AudioSourceProvider.h:

(WebCore::AudioSourceProvider::isHandlingAVPlayer const):

  • platform/audio/cocoa/AudioSampleDataSource.mm:

(WebCore::AudioSampleDataSource::pullSamplesInternal):
When we are too close to the end of available data, do as if we are paused and keep the same time entry for the next read operation.
This will basically make us wait for the next push of data before restarting to read succesfully.

  • platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:

(isType):

  • platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:

(WebCore::AudioSourceProviderAVFObjC::prepare):
(WebCore::AudioSourceProviderAVFObjC::process):
(WebCore::AudioSourceProviderAVFObjC::setAudioCallback):
(WebCore::AudioSourceProviderAVFObjC::setRingBufferCreationCallback):

  • platform/mediastream/mac/WebAudioSourceProviderCocoa.mm:

(WebCore::WebAudioSourceProviderCocoa::receivedNewAudioSamples):
We should use the input sample rate to compute the input media time.

Source/WebKit:

Introduce RemoteAudioSourceProviderManager which receives IPC messages from GPU process for web audio data.
RemoteAudioSourceProviderManager sends it to the identified RemoteAudioSourceProvider.

Introduce RemoteAudioSourceProviderProxy which is a client to the AV AudioSourceProviderAVFObjC.
RemoteAudioSourceProviderProxy will send IPC messages to RemoteAudioSourceProviderManager to send audio data and description.

  • CMakeLists.txt:
  • DerivedSources-input.xcfilelist:
  • DerivedSources-output.xcfilelist:
  • DerivedSources.make:
  • GPUProcess/media/RemoteAudioSourceProviderProxy.cpp: Added.

(WebKit::RemoteAudioSourceProviderProxy::create):
(WebKit::RemoteAudioSourceProviderProxy::RemoteAudioSourceProviderProxy):
(WebKit::RemoteAudioSourceProviderProxy::createRingBuffer):
(WebKit::RemoteAudioSourceProviderProxy::newAudioSamples):
(WebKit::RemoteAudioSourceProviderProxy::storageChanged):

  • GPUProcess/media/RemoteAudioSourceProviderProxy.h: Added.
  • GPUProcess/media/RemoteMediaPlayerProxy.cpp:

(WebKit::RemoteMediaPlayerProxy::~RemoteMediaPlayerProxy):
(WebKit::RemoteMediaPlayerProxy::createAudioSourceProvider):
(WebKit::RemoteMediaPlayerProxy::setShouldEnableAudioSourceProvider):

  • GPUProcess/media/RemoteMediaPlayerProxy.h:
  • GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
  • SourcesCocoa.txt:
  • WebKit.xcodeproj/project.pbxproj:
  • WebProcess/GPU/GPUProcessConnection.cpp:

(WebKit::GPUProcessConnection::audioSourceProviderManager):

  • WebProcess/GPU/GPUProcessConnection.h:
  • WebProcess/GPU/media/RemoteAudioSourceProvider.cpp:

(WebKit::RemoteAudioSourceProvider::create):
(WebKit::RemoteAudioSourceProvider::RemoteAudioSourceProvider):
(WebKit::RemoteAudioSourceProvider::close):
(WebKit::RemoteAudioSourceProvider::hasNewClient):

  • WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp: Added.

(WebKit::RemoteAudioSourceProviderManager::RemoteAudioSourceProviderManager):
(WebKit::RemoteAudioSourceProviderManager::~RemoteAudioSourceProviderManager):
(WebKit::RemoteAudioSourceProviderManager::setConnection):
(WebKit::RemoteAudioSourceProviderManager::addProvider):
(WebKit::RemoteAudioSourceProviderManager::removeProvider):
(WebKit::RemoteAudioSourceProviderManager::dispatchToThread):
(WebKit::RemoteAudioSourceProviderManager::audioStorageChanged):
(WebKit::RemoteAudioSourceProviderManager::audioSamplesAvailable):
(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::RemoteAudio):
(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::setStorage):
(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::audioSamplesAvailable):

  • WebProcess/GPU/media/RemoteAudioSourceProviderManager.h: Added.

(WebKit::RemoteAudioSourceProviderManager::create):

  • WebProcess/GPU/media/RemoteAudioSourceProviderManager.messages.in: Added.
  • WebProcess/cocoa/RemoteCaptureSampleManager.cpp:
Location:
trunk/Source
Files:
5 added
22 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r268576 r268577  
     12020-10-16  Youenn Fablet  <youenn@apple.com>
     2
     3        Add support for GPUProcess WebAudio media element providers
     4        https://bugs.webkit.org/show_bug.cgi?id=217704
     5
     6        Reviewed by Eric Carlson.
     7
     8        Update AudioSourceProviderAVFObjC so that a specific ring buffer can be provided and to be notified of new samples being pushed.
     9
     10        Covered by existing web audio tests run with GPUProcess.
     11
     12        * WebCore.xcodeproj/project.pbxproj:
     13        * platform/audio/AudioSourceProvider.h:
     14        (WebCore::AudioSourceProvider::isHandlingAVPlayer const):
     15        * platform/audio/cocoa/AudioSampleDataSource.mm:
     16        (WebCore::AudioSampleDataSource::pullSamplesInternal):
     17        When we are too close to the end of available data, do as if we are paused and keep the same time entry for the next read operation.
     18        This will basically make us wait for the next push of data before restarting to read succesfully.
     19        * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
     20        (isType):
     21        * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:
     22        (WebCore::AudioSourceProviderAVFObjC::prepare):
     23        (WebCore::AudioSourceProviderAVFObjC::process):
     24        (WebCore::AudioSourceProviderAVFObjC::setAudioCallback):
     25        (WebCore::AudioSourceProviderAVFObjC::setRingBufferCreationCallback):
     26        * platform/mediastream/mac/WebAudioSourceProviderCocoa.mm:
     27        (WebCore::WebAudioSourceProviderCocoa::receivedNewAudioSamples):
     28        We should use the input sample rate to compute the input media time.
     29
    1302020-10-16  Philippe Normand  <pnormand@igalia.com>
    231

  • trunk/Source/WebCore/Headers.cmake

    r268484 r268577  
    10681068    platform/audio/AudioSession.h
    10691069    platform/audio/AudioSourceProvider.h
     1070    platform/audio/AudioSourceProviderClient.h
    10701071    platform/audio/AudioStreamDescription.h
    10711072    platform/audio/AudioUtilities.h

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r268560 r268577  
    41984198                CD83D36221122A210076E11C /* DeferrableTask.h in Headers */ = {isa = PBXBuildFile; fileRef = CD83D35A211110820076E11C /* DeferrableTask.h */; settings = {ATTRIBUTES = (Private, ); }; };
    41994199                CD8A7BBB197735FE00CBD643 /* AudioSourceProviderAVFObjC.mm in Sources */ = {isa = PBXBuildFile; fileRef = CD8A7BB9197735FE00CBD643 /* AudioSourceProviderAVFObjC.mm */; };
    4200                 CD8A7BBC197735FE00CBD643 /* AudioSourceProviderAVFObjC.h in Headers */ = {isa = PBXBuildFile; fileRef = CD8A7BBA197735FE00CBD643 /* AudioSourceProviderAVFObjC.h */; };
     4200                CD8A7BBC197735FE00CBD643 /* AudioSourceProviderAVFObjC.h in Headers */ = {isa = PBXBuildFile; fileRef = CD8A7BBA197735FE00CBD643 /* AudioSourceProviderAVFObjC.h */; settings = {ATTRIBUTES = (Private, ); }; };
    42014201                CD8ACA891D237AA200ECC59E /* RemoteCommandListenerMac.h in Headers */ = {isa = PBXBuildFile; fileRef = CD8ACA871D237AA200ECC59E /* RemoteCommandListenerMac.h */; };
    42024202                CD8ACA8F1D23971900ECC59E /* MediaRemoteSoftLink.h in Headers */ = {isa = PBXBuildFile; fileRef = CD8ACA8D1D23971900ECC59E /* MediaRemoteSoftLink.h */; };

  • trunk/Source/WebCore/platform/audio/AudioSourceProvider.h

    r223728 r268577  
    4444    virtual void setClient(AudioSourceProviderClient*) { };
    4545
     46    virtual bool isHandlingAVPlayer() const { return false; }
     47
    4648    virtual ~AudioSourceProvider() = default;
    4749};

  • trunk/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm

    r268521 r268577  
    261261                m_endFrameWhenNotEnoughData = endFrame;
    262262        } else {
    263             // We are too close from endFrame, let's back up a little bit.
    264             uint64_t framesAvailable = endFrame - timeStamp;
    265             m_outputSampleOffset -= sampleCount - framesAvailable;
     263            // We are too close from endFrame, let's wait for more data to be pushed.
     264            m_outputSampleOffset -= sampleCount;
    266265            dispatch_async(dispatch_get_main_queue(), [logIdentifier = LOGIDENTIFIER, outputSampleOffset = m_outputSampleOffset, this, protectedThis = makeRefPtr(*this)] {
    267266                ALWAYS_LOG(logIdentifier, "updating offset to ", outputSampleOffset);

  • trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h

    r248432 r268577  
    2424 */
    2525
    26 #ifndef AudioSourceProviderAVFObjC_h
    27 #define AudioSourceProviderAVFObjC_h
     26#pragma once
    2827
    2928#if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
    … …  
    3433#include <wtf/RetainPtr.h>
    3534#include <wtf/ThreadSafeRefCounted.h>
     35#include <wtf/TypeCasts.h>
     36#include <wtf/UniqueRef.h>
    3637
    3738OBJC_CLASS AVAssetTrack;
    … …  
    4849namespace WebCore {
    4950
     51class AudioStreamDescription;
     52class CAAudioStreamDescription;
    5053class CARingBuffer;
     54class PlatformAudioData;
    5155
    5256class AudioSourceProviderAVFObjC : public ThreadSafeRefCounted<AudioSourceProviderAVFObjC>, public AudioSourceProvider {
    … …  
    5761    void setPlayerItem(AVPlayerItem *);
    5862    void setAudioTrack(AVAssetTrack *);
     63
     64    using AudioCallback = Function<void(uint64_t startFrame, uint64_t numberOfFrames)>;
     65    WEBCORE_EXPORT void setAudioCallback(AudioCallback&&);
     66    using RingBufferCreationCallback = Function<UniqueRef<CARingBuffer>(const CAAudioStreamDescription&, size_t)>;
     67    WEBCORE_EXPORT void setRingBufferCreationCallback(RingBufferCreationCallback&&);
    5968
    6069private:
    … …  
    6776    void provideInput(AudioBus*, size_t framesToProcess) override;
    6877    void setClient(AudioSourceProviderClient*) override;
     78    bool isHandlingAVPlayer() const final { return true; }
    6979
    7080    static void initCallback(MTAudioProcessingTapRef, void*, void**);
    … …  
    101111    class TapStorage;
    102112    RefPtr<TapStorage> m_tapStorage;
     113    AudioCallback m_audioCallback;
     114    RingBufferCreationCallback m_ringBufferCallback;
    103115};
    104    
     116
    105117}
    106118
    107 #endif
     119SPECIALIZE_TYPE_TRAITS_BEGIN(WebCore::AudioSourceProviderAVFObjC)
     120    static bool isType(const WebCore::AudioSourceProvider& provider) { return provider.isHandlingAVPlayer(); }
     121SPECIALIZE_TYPE_TRAITS_END()
    108122
    109 #endif
     123#endif // ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)

  • trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm

    r268001 r268577  
    338338    size_t capacity = std::max(static_cast<size_t>(2 * maxFrames), static_cast<size_t>(kRingBufferDuration * sampleRate));
    339339
    340     m_ringBuffer = makeUnique<CARingBuffer>();
    341     m_ringBuffer->allocate(CAAudioStreamDescription(*processingFormat), capacity);
     340    CAAudioStreamDescription description { *processingFormat };
     341    if (m_ringBufferCallback)
     342        m_ringBuffer = m_ringBufferCallback(description, capacity).moveToUniquePtr();
     343    else {
     344        m_ringBuffer = makeUnique<CARingBuffer>();
     345        m_ringBuffer->allocate(description, capacity);
     346    }
    342347
    343348    // AudioBufferList is a variable-length struct, so create on the heap with a generic new() operator
    … …  
    420425    }
    421426    *numberFramesOut = 0;
     427
     428    if (m_audioCallback)
     429        m_audioCallback(endFrame, itemCount);
     430}
     431
     432void AudioSourceProviderAVFObjC::setAudioCallback(AudioCallback&& callback)
     433{
     434    ASSERT(!m_avAudioMix);
     435    m_audioCallback = WTFMove(callback);
     436}
     437
     438void AudioSourceProviderAVFObjC::setRingBufferCreationCallback(RingBufferCreationCallback&& callback)
     439{
     440    ASSERT(!m_avAudioMix);
     441    m_ringBufferCallback = WTFMove(callback);
    422442}
    423443

  • trunk/Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderCocoa.mm

    r268521 r268577  
    142142        return;
    143143
    144     m_dataSource->pushSamples(MediaTime(m_writeCount, m_outputDescription->sampleRate()), data, frameCount);
     144    m_dataSource->pushSamples(MediaTime(m_writeCount, m_inputDescription->sampleRate()), data, frameCount);
    145145
    146146    m_writeCount += frameCount;

  • trunk/Source/WebKit/CMakeLists.txt

    r268503 r268577  
    220220    WebProcess/GPU/media/RemoteAudioDestinationProxy
    221221    WebProcess/GPU/media/RemoteAudioSession
     222    WebProcess/GPU/media/RemoteAudioSourceProviderManager
    222223    WebProcess/GPU/media/RemoteCDMInstanceSession
    223224    WebProcess/GPU/media/RemoteLegacyCDMSession

  • trunk/Source/WebKit/ChangeLog

    r268575 r268577  
     12020-10-16  Youenn Fablet  <youenn@apple.com>
     2
     3        Add support for GPUProcess WebAudio media element providers
     4        https://bugs.webkit.org/show_bug.cgi?id=217704
     5
     6        Reviewed by Eric Carlson.
     7
     8        Introduce RemoteAudioSourceProviderManager which receives IPC messages from GPU process for web audio data.
     9        RemoteAudioSourceProviderManager sends it to the identified RemoteAudioSourceProvider.
     10
     11        Introduce RemoteAudioSourceProviderProxy which is a client to the AV AudioSourceProviderAVFObjC.
     12        RemoteAudioSourceProviderProxy will send IPC messages to RemoteAudioSourceProviderManager to send audio data and description.
     13
     14        * CMakeLists.txt:
     15        * DerivedSources-input.xcfilelist:
     16        * DerivedSources-output.xcfilelist:
     17        * DerivedSources.make:
     18        * GPUProcess/media/RemoteAudioSourceProviderProxy.cpp: Added.
     19        (WebKit::RemoteAudioSourceProviderProxy::create):
     20        (WebKit::RemoteAudioSourceProviderProxy::RemoteAudioSourceProviderProxy):
     21        (WebKit::RemoteAudioSourceProviderProxy::createRingBuffer):
     22        (WebKit::RemoteAudioSourceProviderProxy::newAudioSamples):
     23        (WebKit::RemoteAudioSourceProviderProxy::storageChanged):
     24        * GPUProcess/media/RemoteAudioSourceProviderProxy.h: Added.
     25        * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     26        (WebKit::RemoteMediaPlayerProxy::~RemoteMediaPlayerProxy):
     27        (WebKit::RemoteMediaPlayerProxy::createAudioSourceProvider):
     28        (WebKit::RemoteMediaPlayerProxy::setShouldEnableAudioSourceProvider):
     29        * GPUProcess/media/RemoteMediaPlayerProxy.h:
     30        * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
     31        * SourcesCocoa.txt:
     32        * WebKit.xcodeproj/project.pbxproj:
     33        * WebProcess/GPU/GPUProcessConnection.cpp:
     34        (WebKit::GPUProcessConnection::audioSourceProviderManager):
     35        * WebProcess/GPU/GPUProcessConnection.h:
     36        * WebProcess/GPU/media/RemoteAudioSourceProvider.cpp:
     37        (WebKit::RemoteAudioSourceProvider::create):
     38        (WebKit::RemoteAudioSourceProvider::RemoteAudioSourceProvider):
     39        (WebKit::RemoteAudioSourceProvider::close):
     40        (WebKit::RemoteAudioSourceProvider::hasNewClient):
     41        * WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp: Added.
     42        (WebKit::RemoteAudioSourceProviderManager::RemoteAudioSourceProviderManager):
     43        (WebKit::RemoteAudioSourceProviderManager::~RemoteAudioSourceProviderManager):
     44        (WebKit::RemoteAudioSourceProviderManager::setConnection):
     45        (WebKit::RemoteAudioSourceProviderManager::addProvider):
     46        (WebKit::RemoteAudioSourceProviderManager::removeProvider):
     47        (WebKit::RemoteAudioSourceProviderManager::dispatchToThread):
     48        (WebKit::RemoteAudioSourceProviderManager::audioStorageChanged):
     49        (WebKit::RemoteAudioSourceProviderManager::audioSamplesAvailable):
     50        (WebKit::RemoteAudioSourceProviderManager::RemoteAudio::RemoteAudio):
     51        (WebKit::RemoteAudioSourceProviderManager::RemoteAudio::setStorage):
     52        (WebKit::RemoteAudioSourceProviderManager::RemoteAudio::audioSamplesAvailable):
     53        * WebProcess/GPU/media/RemoteAudioSourceProviderManager.h: Added.
     54        (WebKit::RemoteAudioSourceProviderManager::create):
     55        * WebProcess/GPU/media/RemoteAudioSourceProviderManager.messages.in: Added.
     56        * WebProcess/cocoa/RemoteCaptureSampleManager.cpp:
     57
    1582020-10-15  Jiewen Tan  <jiewen_tan@apple.com>
    259

  • trunk/Source/WebKit/DerivedSources-input.xcfilelist

    r268342 r268577  
    128128$(PROJECT_DIR)/WebProcess/GPU/media/RemoteAudioDestinationProxy.messages.in
    129129$(PROJECT_DIR)/WebProcess/GPU/media/RemoteAudioSession.messages.in
     130$(PROJECT_DIR)/WebProcess/GPU/media/RemoteAudioSourceProviderManager.messages.in
    130131$(PROJECT_DIR)/WebProcess/GPU/media/RemoteCDMInstanceSession.messages.in
    131132$(PROJECT_DIR)/WebProcess/GPU/media/RemoteLegacyCDMSession.messages.in

  • trunk/Source/WebKit/DerivedSources-output.xcfilelist

    r268503 r268577  
    155155$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioSessionProxyMessages.h
    156156$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioSessionProxyMessagesReplies.h
     157$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioSourceProviderManagerMessageReceiver.cpp
     158$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioSourceProviderManagerMessages.h
     159$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioSourceProviderManagerMessagesReplies.h
    157160$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteCDMFactoryProxyMessageReceiver.cpp
    158161$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteCDMFactoryProxyMessages.h

  • trunk/Source/WebKit/DerivedSources.make

    r268503 r268577  
    176176        WebProcess/GPU/media/RemoteAudioDestinationProxy \
    177177        WebProcess/GPU/media/RemoteAudioSession \
     178        WebProcess/GPU/media/RemoteAudioSourceProviderManager \
    178179        WebProcess/WebStorage/StorageAreaMap \
    179180        WebProcess/UserContent/WebUserContentController \

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp

    r268118 r268577  
    3333#include "LayerHostingContext.h"
    3434#include "MediaPlayerPrivateRemoteMessages.h"
     35#include "RemoteAudioSourceProviderProxy.h"
    3536#include "RemoteAudioTrackProxy.h"
    3637#include "RemoteLegacyCDMFactoryProxy.h"
    … …  
    6061#endif
    6162
     63#if PLATFORM(COCOA)
     64#include <WebCore/AudioSourceProviderAVFObjC.h>
     65#endif
     66
    6267namespace WebKit {
    6368
    … …  
    8388    if (m_performTaskAtMediaTimeCompletionHandler)
    8489        m_performTaskAtMediaTimeCompletionHandler(WTF::nullopt);
     90    setShouldEnableAudioSourceProvider(false);
    8591}
    8692
    … …  
    879885}
    880886
     887void RemoteMediaPlayerProxy::createAudioSourceProvider()
     888{
     889#if ENABLE(WEB_AUDIO) && PLATFORM(COCOA)
     890    if (!m_player)
     891        return;
     892
     893    auto* provider = m_player->audioSourceProvider();
     894    if (!provider || !is<AudioSourceProviderAVFObjC>(provider))
     895        return;
     896
     897    m_remoteAudioSourceProvider = RemoteAudioSourceProviderProxy::create(m_id, m_webProcessConnection.copyRef(), downcast<AudioSourceProviderAVFObjC>(*provider));
     898#endif
     899}
     900
     901void RemoteMediaPlayerProxy::setShouldEnableAudioSourceProvider(bool shouldEnable)
     902{
     903#if ENABLE(WEB_AUDIO) && PLATFORM(COCOA)
     904    if (auto* provider = m_player->audioSourceProvider())
     905        provider->setClient(shouldEnable ? m_remoteAudioSourceProvider.get() : nullptr);
     906#endif
     907}
     908
    881909} // namespace WebKit
    882910

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h

    r268070 r268577  
    5353#endif
    5454
     55#if PLATFORM(COCOA)
     56#include "SharedRingBufferStorage.h"
     57#endif
     58
    5559namespace WTF {
    5660class MachSendRight;
    … …  
    6670using LayerHostingContextID = uint32_t;
    6771class LayerHostingContext;
     72class RemoteAudioSourceProviderProxy;
    6873class RemoteAudioTrackProxy;
    6974class RemoteMediaPlayerManagerProxy;
    … …  
    268273    void timerFired();
    269274
     275    void createAudioSourceProvider();
     276    void setShouldEnableAudioSourceProvider(bool);
     277
    270278#if !RELEASE_LOG_DISABLED
    271279    const Logger& mediaPlayerLogger() final { return m_logger; }
    … …  
    304312#endif
    305313
     314#if ENABLE(WEB_AUDIO) && PLATFORM(COCOA)
     315    RefPtr<RemoteAudioSourceProviderProxy> m_remoteAudioSourceProvider;
     316#endif
     317
    306318#if !RELEASE_LOG_DISABLED
    307319    const Logger& m_logger;

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in

    r267725 r268577  
    117117    AccessLog() -> (String accessLog) Synchronous
    118118#endif
     119
     120#if ENABLE(WEB_AUDIO)
     121    CreateAudioSourceProvider()
     122    SetShouldEnableAudioSourceProvider(bool shouldEnable)
     123#endif
    119124}
    120125

  • trunk/Source/WebKit/SourcesCocoa.txt

    r268575 r268577  
    5959GPUProcess/ios/GPUProcessIOS.mm
    6060GPUProcess/mac/GPUProcessMac.mm
     61GPUProcess/media/RemoteAudioSourceProviderProxy.cpp
    6162GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm
    6263GPUProcess/media/ios/RemoteMediaSessionHelperProxy.cpp
    … …  
    567568WebProcess/GPU/graphics/cocoa/ImageBufferShareableIOSurfaceBackend.cpp
    568569WebProcess/GPU/media/RemoteAudioSourceProvider.cpp
     570WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp
    569571WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm
    570572WebProcess/GPU/media/cocoa/VideoLayerRemoteCocoa.mm
    … …  
    669671RemoteAudioMediaStreamTrackRendererManagerMessageReceiver.cpp
    670672RemoteAudioMediaStreamTrackRendererMessageReceiver.cpp
     673RemoteAudioSourceProviderManagerMessageReceiver.cpp
    671674RemoteMediaRecorderMessageReceiver.cpp
    672675RemoteMediaRecorderManagerMessageReceiver.cpp

  • trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj

    r268575 r268577  
    34283428                412FF91625373F9D001DF036 /* RemoteAudioSourceProvider.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteAudioSourceProvider.cpp; sourceTree = "<group>"; };
    34293429                412FF91725373F9D001DF036 /* RemoteAudioSourceProvider.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteAudioSourceProvider.h; sourceTree = "<group>"; };
     3430                412FF92325382BD3001DF036 /* RemoteAudioSourceProviderManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteAudioSourceProviderManager.cpp; sourceTree = "<group>"; };
     3431                412FF92425382BD5001DF036 /* RemoteAudioSourceProviderManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteAudioSourceProviderManager.h; sourceTree = "<group>"; };
     3432                412FF92525382BD5001DF036 /* RemoteAudioSourceProviderManager.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RemoteAudioSourceProviderManager.messages.in; sourceTree = "<group>"; };
     3433                412FF92625382D85001DF036 /* RemoteAudioSourceProviderProxy.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteAudioSourceProviderProxy.cpp; sourceTree = "<group>"; };
     3434                412FF92725382D85001DF036 /* RemoteAudioSourceProviderProxy.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteAudioSourceProviderProxy.h; sourceTree = "<group>"; };
    34303435                4130759A1DE84FB00039EC69 /* NetworkRTCMonitor.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkRTCMonitor.cpp; sourceTree = "<group>"; };
    34313436                4130759B1DE84FB00039EC69 /* NetworkRTCMonitor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkRTCMonitor.h; sourceTree = "<group>"; };
    … …  
    45394544                9B47908C25314D8300EC11AB /* MessageArgumentDescriptions.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = MessageArgumentDescriptions.h; sourceTree = "<group>"; };
    45404545                9B47908E253151CC00EC11AB /* JSIPCBinding.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = JSIPCBinding.h; sourceTree = "<group>"; };
    4541                 9B4790902531563200EC11AB /* MessageArgumentDescriptions.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = MessageArgumentDescriptions.cpp; path = MessageArgumentDescriptions.cpp; sourceTree = "<group>"; };
     4546                9B4790902531563200EC11AB /* MessageArgumentDescriptions.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MessageArgumentDescriptions.cpp; sourceTree = "<group>"; };
    45424547                9B5499AC2362A6F600DF8BA5 /* _WKTextManipulationConfiguration.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = _WKTextManipulationConfiguration.h; sourceTree = "<group>"; };
    45434548                9B5499AE2362A7A700DF8BA5 /* _WKTextManipulationConfiguration.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = _WKTextManipulationConfiguration.mm; sourceTree = "<group>"; };
    … …  
    55755580                                412FF91625373F9D001DF036 /* RemoteAudioSourceProvider.cpp */,
    55765581                                412FF91725373F9D001DF036 /* RemoteAudioSourceProvider.h */,
     5582                                412FF92325382BD3001DF036 /* RemoteAudioSourceProviderManager.cpp */,
     5583                                412FF92425382BD5001DF036 /* RemoteAudioSourceProviderManager.h */,
     5584                                412FF92525382BD5001DF036 /* RemoteAudioSourceProviderManager.messages.in */,
    55775585                                CDAC20BF23FC2F750021DEE3 /* RemoteCDM.cpp */,
    55785586                                CDAC20BA23FC2F740021DEE3 /* RemoteCDM.h */,
    … …  
    56305638                                CDBB49FA240D963D0017C292 /* RemoteAudioSessionProxyManager.cpp */,
    56315639                                CDBB49F9240D963C0017C292 /* RemoteAudioSessionProxyManager.h */,
     5640                                412FF92625382D85001DF036 /* RemoteAudioSourceProviderProxy.cpp */,
     5641                                412FF92725382D85001DF036 /* RemoteAudioSourceProviderProxy.h */,
    56325642                                071BC57E23CA5DB100680D7C /* RemoteAudioTrackProxy.cpp */,
    56335643                                071BC57F23CA5DB100680D7C /* RemoteAudioTrackProxy.h */,

  • trunk/Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp

    r267818 r268577  
    3434#include "LibWebRTCCodecsMessages.h"
    3535#include "MediaPlayerPrivateRemoteMessages.h"
     36#include "RemoteAudioSourceProviderManager.h"
    3637#include "RemoteCDMFactory.h"
    3738#include "RemoteCDMProxy.h"
    … …  
    101102    return *WebProcess::singleton().supplement<RemoteMediaPlayerManager>();
    102103}
     104
     105#if PLATFORM(COCOA) && ENABLE(WEB_AUDIO)
     106RemoteAudioSourceProviderManager& GPUProcessConnection::audioSourceProviderManager()
     107{
     108    if (!m_audioSourceProviderManager)
     109        m_audioSourceProviderManager = RemoteAudioSourceProviderManager::create();
     110    return *m_audioSourceProviderManager;
     111}
     112#endif
    103113
    104114#if ENABLE(ENCRYPTED_MEDIA)

  • trunk/Source/WebKit/WebProcess/GPU/GPUProcessConnection.h

    r267079 r268577  
    4141namespace WebKit {
    4242
     43class RemoteAudioSourceProviderManager;
    4344class RemoteCDMFactory;
    4445class RemoteMediaPlayerManager;
    … …  
    6667
    6768    RemoteMediaPlayerManager& mediaPlayerManager();
     69
     70#if PLATFORM(COCOA) && ENABLE(WEB_AUDIO)
     71    RemoteAudioSourceProviderManager& audioSourceProviderManager();
     72#endif
    6873
    6974#if ENABLE(ENCRYPTED_MEDIA)
    … …  
    101106    std::unique_ptr<SampleBufferDisplayLayerManager> m_sampleBufferDisplayLayerManager;
    102107#endif
     108#if PLATFORM(COCOA) && ENABLE(WEB_AUDIO)
     109    RefPtr<RemoteAudioSourceProviderManager> m_audioSourceProviderManager;
     110#endif
    103111#if ENABLE(VP9)
    104112    bool m_enableVP9Decoder { false };

  • trunk/Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProvider.cpp

    r268521 r268577  
    2929#if ENABLE(GPU_PROCESS) && ENABLE(WEB_AUDIO) && PLATFORM(COCOA)
    3030
     31#include "RemoteAudioSourceProviderManager.h"
     32#include "RemoteMediaPlayerProxyMessages.h"
     33
    3134namespace WebCore {
    3235#if !RELEASE_LOG_DISABLED
    … …  
    4043Ref<RemoteAudioSourceProvider> RemoteAudioSourceProvider::create(WebCore::MediaPlayerIdentifier identifier, WTF::LoggerHelper& helper)
    4144{
    42     return adoptRef(*new RemoteAudioSourceProvider(identifier, helper));
     45    auto provider = adoptRef(*new RemoteAudioSourceProvider(identifier, helper));
     46
     47    auto& gpuProcessConnection = WebProcess::singleton().ensureGPUProcessConnection();
     48    gpuProcessConnection.audioSourceProviderManager().addProvider(provider.copyRef());
     49
     50#if ENABLE(WEB_AUDIO)
     51    gpuProcessConnection.connection().send(Messages::RemoteMediaPlayerProxy::CreateAudioSourceProvider { }, provider->identifier());
     52#endif
     53
     54    return provider;
    4355}
    4456
    … …  
    5062#endif
    5163{
     64    ASSERT(isMainThread());
    5265    UNUSED_PARAM(helper);
    5366}
    … …  
    5972void RemoteAudioSourceProvider::close()
    6073{
     74    ASSERT(isMainThread());
     75    WebProcess::singleton().ensureGPUProcessConnection().audioSourceProviderManager().removeProvider(m_identifier);
    6176}
    6277
    63 void RemoteAudioSourceProvider::hasNewClient(AudioSourceProviderClient*)
     78void RemoteAudioSourceProvider::hasNewClient(AudioSourceProviderClient* client)
    6479{
    65     // FIXME: register/unregister as needed.
     80    WebProcess::singleton().ensureGPUProcessConnection().connection().send(Messages::RemoteMediaPlayerProxy::SetShouldEnableAudioSourceProvider { !!client }, m_identifier);
    6681}
    6782

  • trunk/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp

    r266745 r268577  
    2727#include "RemoteCaptureSampleManager.h"
    2828
     29#include "Logging.h"
    2930#include "RemoteCaptureSampleManagerMessages.h"
    3031#include "SharedRingBufferStorage.h"
    … …  
    3435
    3536namespace WebKit {
    36 using namespace PAL;
    3737using namespace WebCore;
    3838
Note: See TracChangeset for help on using the changeset viewer.