Changeset 254505 in webkit


Ignore:
Timestamp:
Jan 14, 2020 6:37:51 AM (4 years ago)
Author:
youenn@apple.com
Message:

Add support for MediaStream audio track rendering in GPUProcess
​https://bugs.webkit.org/show_bug.cgi?id=206175

Reviewed by Eric Carlson.

Source/WebCore:

Simplify model to use start/stop instead of setPaused.
Simplify and fix issue in computation of the muted state of the renderer.
Covered by existing tests run with GPU process enabled and manual testing

  • platform/mediastream/AudioMediaStreamTrackRenderer.h:
  • platform/mediastream/AudioTrackPrivateMediaStream.cpp:

(WebCore::AudioTrackPrivateMediaStream::playInternal):
(WebCore::AudioTrackPrivateMediaStream::pause):
(WebCore::AudioTrackPrivateMediaStream::audioSamplesAvailable):
(WebCore::AudioTrackPrivateMediaStream::updateRendererMutedState):

  • platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.cpp:

(WebCore::AudioMediaStreamTrackRendererCocoa::start):
(WebCore::AudioMediaStreamTrackRendererCocoa::stop):
(WebCore::AudioMediaStreamTrackRendererCocoa::clear):
(WebCore::AudioMediaStreamTrackRendererCocoa::pushSamples):
(WebCore::AudioMediaStreamTrackRendererCocoa::render):

  • platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.h:

Source/WebKit:

Implement an AudioMediaStreamTrackRenderer at WebKit level by creating a remote renderer in GPUProcess and sending IPC to pass
audio data as well as orders (start/stop/setMuted).

Implement the remote renderer using WebCore audio track renderer.

Enable WebKit remote renderer F GPU process for media is enabled.

  • DerivedSources-input.xcfilelist:
  • DerivedSources-output.xcfilelist:
  • DerivedSources.make:
  • GPUProcess/GPUConnectionToWebProcess.cpp:

(WebKit::GPUConnectionToWebProcess::audioTrackRendererManager):
(WebKit::GPUConnectionToWebProcess::didReceiveMessage):

  • GPUProcess/GPUConnectionToWebProcess.h:
  • GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp: Added.

(WebKit::nextLogIdentifier):
(WebKit::nullLogger):
(WebKit::RemoteAudioMediaStreamTrackRenderer::RemoteAudioMediaStreamTrackRenderer):
(WebKit::RemoteAudioMediaStreamTrackRenderer::~RemoteAudioMediaStreamTrackRenderer):
(WebKit::RemoteAudioMediaStreamTrackRenderer::storage):
(WebKit::RemoteAudioMediaStreamTrackRenderer::start):
(WebKit::RemoteAudioMediaStreamTrackRenderer::stop):
(WebKit::RemoteAudioMediaStreamTrackRenderer::clear):
(WebKit::RemoteAudioMediaStreamTrackRenderer::setMuted):
(WebKit::RemoteAudioMediaStreamTrackRenderer::setVolume):
(WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesStorageChanged):
(WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesAvailable):

  • GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h: Added.
  • GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in: Added.
  • GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp: Added.

(WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveRendererMessage):
(WebKit::RemoteAudioMediaStreamTrackRendererManager::createRenderer):
(WebKit::RemoteAudioMediaStreamTrackRendererManager::releaseRenderer):

  • GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.h: Added.

(WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveMessageFromWebProcess):

  • GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in: Added.
  • Scripts/webkit/messages.py:
  • Sources.txt:
  • SourcesCocoa.txt:
  • WebKit.xcodeproj/project.pbxproj:
  • WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:

(WebKit::RemoteMediaPlayerManager::updatePreferences):

  • WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp: Added.

(WebKit::AudioMediaStreamTrackRenderer::create):
(WebKit::AudioMediaStreamTrackRenderer::AudioMediaStreamTrackRenderer):
(WebKit::AudioMediaStreamTrackRenderer::~AudioMediaStreamTrackRenderer):
(WebKit::AudioMediaStreamTrackRenderer::start):
(WebKit::AudioMediaStreamTrackRenderer::stop):
(WebKit::AudioMediaStreamTrackRenderer::clear):
(WebKit::AudioMediaStreamTrackRenderer::setMuted):
(WebKit::AudioMediaStreamTrackRenderer::setVolume):
(WebKit::AudioMediaStreamTrackRenderer::pushSamples):
(WebKit::AudioMediaStreamTrackRenderer::storageChanged):

  • WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h: Added.

(WebKit::AudioMediaStreamTrackRenderer::identifier const):

  • WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererIdentifier.h: Added.

LayoutTests:

  • gpu-process/TestExpectations:
Location:
trunk
Files:
8 added
18 edited
1 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r254504 r254505  
     12020-01-14  youenn fablet  <youenn@apple.com>
     2
     3        Add support for MediaStream audio track rendering in GPUProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=206175
     5
     6        Reviewed by Eric Carlson.
     7
     8        * gpu-process/TestExpectations:
     9
    1102020-01-14  Pablo Saavedra  <psaavedra@igalia.com>
    211

  • trunk/LayoutTests/gpu-process/TestExpectations

    r254499 r254505  
    230230fast/css/relative-position-replaced-in-table-display-crash.html [ Skip ]
    231231fast/mediacapturefromelement [ Skip ]
    232 fast/mediastream [ Skip ]
    233232fullscreen/full-screen-iframe-legacy.html [ Skip ]
    234233fullscreen/video-controls-timeline.html [ Skip ]
    … …  
    241240http/tests/navigation/page-cache-mediastream.html [ Skip ]
    242241http/wpt/mediarecorder [ Skip ]
    243 imported/w3c/web-platform-tests/mediacapture-streams [ Skip ]
    244242imported/w3c/web-platform-tests/mediacapture-fromelement/idlharness.html [ Skip ]
    245243imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-stop.html [ Skip ]
    … …  
    258256
    259257webgpu [ Skip ]
    260 
    261 webrtc [ Skip ]
    262 platform/mac/webrtc [ Skip ]

  • trunk/Source/WebCore/ChangeLog

    r254503 r254505  
     12020-01-14  youenn fablet  <youenn@apple.com>
     2
     3        Add support for MediaStream audio track rendering in GPUProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=206175
     5
     6        Reviewed by Eric Carlson.
     7
     8        Simplify model to use start/stop instead of setPaused.
     9        Simplify and fix issue in computation of the muted state of the renderer.
     10        Covered by existing tests run with GPU process enabled and manual testing
     11
     12        * platform/mediastream/AudioMediaStreamTrackRenderer.h:
     13        * platform/mediastream/AudioTrackPrivateMediaStream.cpp:
     14        (WebCore::AudioTrackPrivateMediaStream::playInternal):
     15        (WebCore::AudioTrackPrivateMediaStream::pause):
     16        (WebCore::AudioTrackPrivateMediaStream::audioSamplesAvailable):
     17        (WebCore::AudioTrackPrivateMediaStream::updateRendererMutedState):
     18        * platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.cpp:
     19        (WebCore::AudioMediaStreamTrackRendererCocoa::start):
     20        (WebCore::AudioMediaStreamTrackRendererCocoa::stop):
     21        (WebCore::AudioMediaStreamTrackRendererCocoa::clear):
     22        (WebCore::AudioMediaStreamTrackRendererCocoa::pushSamples):
     23        (WebCore::AudioMediaStreamTrackRendererCocoa::render):
     24        * platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.h:
     25
    1262020-01-14  Xabier Rodriguez Calvar  <calvaris@igalia.com>
    227

  • trunk/Source/WebCore/platform/mediastream/AudioMediaStreamTrackRenderer.h

    r254446 r254505  
    4545    virtual ~AudioMediaStreamTrackRenderer() = default;
    4646
    47     virtual void setPaused(bool) = 0;
     47    virtual void start() = 0;
    4848    virtual void stop() = 0;
    4949    virtual void clear() = 0;

  • trunk/Source/WebCore/platform/mediastream/AudioTrackPrivateMediaStream.cpp

    r254446 r254505  
    7878    m_autoPlay = false;
    7979
    80     m_renderer->setPaused(false);
     80    m_renderer->start();
    8181}
    8282
    … …  
    9696    m_autoPlay = false;
    9797
    98     m_renderer->setPaused(true);
     98    m_renderer->stop();
    9999}
    100100
    … …  
    112112void AudioTrackPrivateMediaStream::audioSamplesAvailable(MediaStreamTrackPrivate&, const MediaTime& sampleTime, const PlatformAudioData& audioData, const AudioStreamDescription& description, size_t sampleCount)
    113113{
    114     if (!m_isPlaying) {
    115         m_renderer->stop();
    116         return;
    117     }
    118 
    119114    m_renderer->pushSamples(sampleTime, audioData, description, sampleCount);
    120115
    … …  
    140135void AudioTrackPrivateMediaStream::updateRendererMutedState()
    141136{
    142     m_renderer->setMuted(m_isPlaying && !streamTrack().muted() && !streamTrack().ended() && streamTrack().enabled());
     137    m_renderer->setMuted(streamTrack().muted() || streamTrack().ended() || !streamTrack().enabled());
    143138}
    144139

  • trunk/Source/WebCore/platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.cpp

    r254446 r254505  
    4444AudioMediaStreamTrackRendererCocoa::~AudioMediaStreamTrackRendererCocoa() = default;
    4545
     46void AudioMediaStreamTrackRendererCocoa::start()
     47{
     48    if (m_dataSource)
     49        m_dataSource->setPaused(false);
     50
     51    if (m_isAudioUnitStarted)
     52        return;
     53
     54    if (auto error = AudioOutputUnitStart(m_remoteIOUnit)) {
     55        ERROR_LOG(LOGIDENTIFIER, "AudioOutputUnitStart failed, error = ", error, " (", (const char*)&error, ")");
     56        return;
     57    }
     58    m_isAudioUnitStarted = true;
     59}
     60
    4661void AudioMediaStreamTrackRendererCocoa::stop()
    4762{
     63    if (m_dataSource)
     64        m_dataSource->setPaused(true);
     65
    4866    if (!m_isAudioUnitStarted)
    4967        return;
    5068
     69    m_isAudioUnitStarted = false;
    5170    if (m_remoteIOUnit)
    5271        AudioOutputUnitStop(m_remoteIOUnit);
    … …  
    6786    m_inputDescription = nullptr;
    6887    m_outputDescription = nullptr;
    69 }
    70 
    71 void AudioMediaStreamTrackRendererCocoa::setPaused(bool value)
    72 {
    73     m_paused = value;
    74     if (m_dataSource)
    75         m_dataSource->setPaused(value);
     88    m_isAudioUnitStarted = false;
    7689}
    7790
    … …  
    188201
    189202    m_dataSource->pushSamples(sampleTime, audioData, sampleCount);
    190 
    191     if (!m_isAudioUnitStarted) {
    192         if (auto error = AudioOutputUnitStart(m_remoteIOUnit)) {
    193             ERROR_LOG(LOGIDENTIFIER, "AudioOutputUnitStart failed, error = ", error, " (", (const char*)&error, ")");
    194             return;
    195         }
    196         m_isAudioUnitStarted = true;
    197     }
    198203}
    199204
    200205OSStatus AudioMediaStreamTrackRendererCocoa::render(UInt32 sampleCount, AudioBufferList& ioData, UInt32 /*inBusNumber*/, const AudioTimeStamp& timeStamp, AudioUnitRenderActionFlags& actionFlags)
    201206{
    202     if (isMuted() || m_paused || !m_dataSource) {
     207    if (isMuted() || !m_dataSource) {
    203208        AudioSampleBufferList::zeroABL(ioData, static_cast<size_t>(sampleCount * m_outputDescription->bytesPerFrame()));
    204209        actionFlags = kAudioUnitRenderAction_OutputIsSilence;

  • trunk/Source/WebCore/platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.h

    r254446 r254505  
    4949    // AudioMediaStreamTrackRenderer
    5050    void pushSamples(const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final;
     51    void start() final;
    5152    void stop() final;
    5253    void clear() final;
    53     void setPaused(bool) final;
    5454
    5555    static OSStatus inputProc(void*, AudioUnitRenderActionFlags*, const AudioTimeStamp*, UInt32 inBusNumber, UInt32 numberOfFrames, AudioBufferList*);
    … …  
    6666    RefPtr<AudioSampleDataSource> m_dataSource;
    6767    bool m_isAudioUnitStarted { false };
    68     bool m_paused { false };
    6968};
    7069

  • trunk/Source/WebKit/ChangeLog

    r254501 r254505  
     12020-01-14  youenn fablet  <youenn@apple.com>
     2
     3        Add support for MediaStream audio track rendering in GPUProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=206175
     5
     6        Reviewed by Eric Carlson.
     7
     8        Implement an AudioMediaStreamTrackRenderer at WebKit level by creating a remote renderer in GPUProcess and sending IPC to pass
     9        audio data as well as orders (start/stop/setMuted).
     10
     11        Implement the remote renderer using WebCore audio track renderer.
     12
     13        Enable WebKit remote renderer F GPU process for media is enabled.
     14
     15        * DerivedSources-input.xcfilelist:
     16        * DerivedSources-output.xcfilelist:
     17        * DerivedSources.make:
     18        * GPUProcess/GPUConnectionToWebProcess.cpp:
     19        (WebKit::GPUConnectionToWebProcess::audioTrackRendererManager):
     20        (WebKit::GPUConnectionToWebProcess::didReceiveMessage):
     21        * GPUProcess/GPUConnectionToWebProcess.h:
     22        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp: Added.
     23        (WebKit::nextLogIdentifier):
     24        (WebKit::nullLogger):
     25        (WebKit::RemoteAudioMediaStreamTrackRenderer::RemoteAudioMediaStreamTrackRenderer):
     26        (WebKit::RemoteAudioMediaStreamTrackRenderer::~RemoteAudioMediaStreamTrackRenderer):
     27        (WebKit::RemoteAudioMediaStreamTrackRenderer::storage):
     28        (WebKit::RemoteAudioMediaStreamTrackRenderer::start):
     29        (WebKit::RemoteAudioMediaStreamTrackRenderer::stop):
     30        (WebKit::RemoteAudioMediaStreamTrackRenderer::clear):
     31        (WebKit::RemoteAudioMediaStreamTrackRenderer::setMuted):
     32        (WebKit::RemoteAudioMediaStreamTrackRenderer::setVolume):
     33        (WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesStorageChanged):
     34        (WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesAvailable):
     35        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h: Added.
     36        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in: Added.
     37        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp: Added.
     38        (WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveRendererMessage):
     39        (WebKit::RemoteAudioMediaStreamTrackRendererManager::createRenderer):
     40        (WebKit::RemoteAudioMediaStreamTrackRendererManager::releaseRenderer):
     41        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.h: Added.
     42        (WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveMessageFromWebProcess):
     43        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in: Added.
     44        * Scripts/webkit/messages.py:
     45        * Sources.txt:
     46        * SourcesCocoa.txt:
     47        * WebKit.xcodeproj/project.pbxproj:
     48        * WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
     49        (WebKit::RemoteMediaPlayerManager::updatePreferences):
     50        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp: Added.
     51        (WebKit::AudioMediaStreamTrackRenderer::create):
     52        (WebKit::AudioMediaStreamTrackRenderer::AudioMediaStreamTrackRenderer):
     53        (WebKit::AudioMediaStreamTrackRenderer::~AudioMediaStreamTrackRenderer):
     54        (WebKit::AudioMediaStreamTrackRenderer::start):
     55        (WebKit::AudioMediaStreamTrackRenderer::stop):
     56        (WebKit::AudioMediaStreamTrackRenderer::clear):
     57        (WebKit::AudioMediaStreamTrackRenderer::setMuted):
     58        (WebKit::AudioMediaStreamTrackRenderer::setVolume):
     59        (WebKit::AudioMediaStreamTrackRenderer::pushSamples):
     60        (WebKit::AudioMediaStreamTrackRenderer::storageChanged):
     61        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h: Added.
     62        (WebKit::AudioMediaStreamTrackRenderer::identifier const):
     63        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererIdentifier.h: Added.
     64
    1652020-01-14  youenn fablet  <youenn@apple.com>
    266

  • trunk/Source/WebKit/DerivedSources-input.xcfilelist

    r254194 r254505  
    2020$(PROJECT_DIR)/GPUProcess/media/RemoteMediaResourceManager.messages.in
    2121$(PROJECT_DIR)/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
     22$(PROJECT_DIR)/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in
     23$(PROJECT_DIR)/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in
    2224$(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
    2325$(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in

  • trunk/Source/WebKit/DerivedSources-output.xcfilelist

    r254194 r254505  
    121121$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/PluginProxyMessages.h
    122122$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/PluginProxyMessagesReplies.h
     123$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererManagerMessageReceiver.cpp
     124$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererManagerMessages.h
     125$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererManagerMessagesReplies.h
     126$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererMessageReceiver.cpp
     127$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererMessages.h
     128$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererMessagesReplies.h
    123129$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteLayerTreeDrawingAreaProxyMessageReceiver.cpp
    124130$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteLayerTreeDrawingAreaProxyMessages.h

  • trunk/Source/WebKit/DerivedSources.make

    r254194 r254505  
    141141    PluginProcessProxy \
    142142    PluginProxy \
     143    RemoteAudioMediaStreamTrackRendererManager \
     144    RemoteAudioMediaStreamTrackRenderer \
    143145    RemoteLayerTreeDrawingAreaProxy \
    144146    RemoteMediaPlayerManager \

  • trunk/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp

    r254194 r254505  
    3737#include "LibWebRTCCodecsProxyMessages.h"
    3838#include "Logging.h"
     39#include "RemoteAudioMediaStreamTrackRendererManager.h"
     40#include "RemoteAudioMediaStreamTrackRendererManagerMessages.h"
     41#include "RemoteAudioMediaStreamTrackRendererMessages.h"
    3942#include "RemoteLayerTreeDrawingAreaProxyMessages.h"
    4043#include "RemoteMediaPlayerManagerProxy.h"
    … …  
    149152    return *m_remoteMediaRecorderManager;
    150153}
     154
     155#if ENABLE(VIDEO_TRACK)
     156RemoteAudioMediaStreamTrackRendererManager& GPUConnectionToWebProcess::audioTrackRendererManager()
     157{
     158    if (!m_audioTrackRendererManager)
     159        m_audioTrackRendererManager = makeUnique<RemoteAudioMediaStreamTrackRendererManager>();
     160
     161    return *m_audioTrackRendererManager;
     162}
     163#endif
    151164#endif
    152165
    … …  
    183196        return;
    184197    }
     198#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK)
     199    if (decoder.messageReceiverName() == Messages::RemoteAudioMediaStreamTrackRendererManager::messageReceiverName()) {
     200        audioTrackRendererManager().didReceiveMessageFromWebProcess(connection, decoder);
     201        return;
     202    }
     203    if (decoder.messageReceiverName() == Messages::RemoteAudioMediaStreamTrackRenderer::messageReceiverName()) {
     204        audioTrackRendererManager().didReceiveRendererMessage(connection, decoder);
     205        return;
     206    }
     207#endif
    185208#endif
    186209#if PLATFORM(COCOA) && USE(LIBWEBRTC)

  • trunk/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h

    r254194 r254505  
    4141class GPUProcess;
    4242class LibWebRTCCodecsProxy;
     43class RemoteAudioMediaStreamTrackRendererManager;
    4344class RemoteMediaPlayerManagerProxy;
    4445class RemoteMediaRecorderManager;
    … …  
    7475    UserMediaCaptureManagerProxy& userMediaCaptureManagerProxy();
    7576    RemoteMediaRecorderManager& mediaRecorderManager();
     77#if ENABLE(VIDEO_TRACK)
     78    RemoteAudioMediaStreamTrackRendererManager& audioTrackRendererManager();
     79#endif
    7680#endif
    7781
    … …  
    9397    std::unique_ptr<UserMediaCaptureManagerProxy> m_userMediaCaptureManagerProxy;
    9498    std::unique_ptr<RemoteMediaRecorderManager> m_remoteMediaRecorderManager;
     99#if ENABLE(VIDEO_TRACK)
     100    std::unique_ptr<RemoteAudioMediaStreamTrackRendererManager> m_audioTrackRendererManager;
     101#endif
    95102#endif
    96103#if PLATFORM(COCOA) && USE(LIBWEBRTC)

  • trunk/Source/WebKit/Scripts/webkit/messages.py

    r254454 r254505  
    219219        'WebCore::SWServerConnectionIdentifier',
    220220        'WebKit::ActivityStateChangeID',
     221        'WebKit::AudioMediaStreamTrackRendererIdentifier',
    221222        'WebKit::LayerHostingContextID',
    222223        'WebKit::MediaPlayerPrivateRemoteIdentifier',

  • trunk/Source/WebKit/Sources.txt

    r254499 r254505  
    2525GPUProcess/GPUConnectionToWebProcess.cpp
    2626GPUProcess/GPUProcessCreationParameters.cpp
     27GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp
     28GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp
    2729GPUProcess/webrtc/RemoteMediaRecorder.cpp
    2830GPUProcess/webrtc/RemoteMediaRecorderManager.cpp
    … …  
    522524WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
    523525WebProcess/GPU/webrtc/MediaRecorderProvider.cpp
     526WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp
    524527
    525528WebProcess/Network/NetworkProcessConnection.cpp

  • trunk/Source/WebKit/SourcesCocoa.txt

    r254489 r254505  
    640640LibWebRTCCodecsProxyMessageReceiver.cpp
    641641LibWebRTCCodecsMessageReceiver.cpp
     642RemoteAudioMediaStreamTrackRendererManagerMessageReceiver.cpp
     643RemoteAudioMediaStreamTrackRendererMessageReceiver.cpp
    642644RemoteMediaRecorderMessageReceiver.cpp
    643645RemoteMediaRecorderManagerMessageReceiver.cpp

  • trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj

    r254499 r254505  
    33813381                41D5C6D8238EB20D00B9B3CB /* ServiceWorkerSoftUpdateLoader.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ServiceWorkerSoftUpdateLoader.h; sourceTree = "<group>"; };
    33823382                41D5C6D9238EB20E00B9B3CB /* ServiceWorkerSoftUpdateLoader.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ServiceWorkerSoftUpdateLoader.cpp; sourceTree = "<group>"; };
     3383                41D6455223CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.messages.in */ = {isa = PBXFileReference; lastKnownFileType = text; name = RemoteAudioMediaStreamTrackRenderer.messages.in; path = GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in; sourceTree = SOURCE_ROOT; };
     3384                41D6455323CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = RemoteAudioMediaStreamTrackRenderer.h; path = GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h; sourceTree = SOURCE_ROOT; };
     3385                41D6455423CCB92200486E0E /* RemoteAudioMediaStreamTrackRenderer.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = RemoteAudioMediaStreamTrackRenderer.cpp; path = GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp; sourceTree = SOURCE_ROOT; };
     3386                41D6455523CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RemoteAudioMediaStreamTrackRendererManager.messages.in; sourceTree = "<group>"; };
     3387                41D6455623CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteAudioMediaStreamTrackRendererManager.cpp; sourceTree = "<group>"; };
     3388                41D6455723CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteAudioMediaStreamTrackRendererManager.h; sourceTree = "<group>"; };
    33833389                41DC45941E3D6E1E00B11F51 /* NetworkRTCProvider.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkRTCProvider.h; sourceTree = "<group>"; };
    33843390                41DC45951E3D6E1E00B11F51 /* NetworkRTCProvider.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkRTCProvider.cpp; sourceTree = "<group>"; };
    … …  
    34013407                41FBE823206DA79C000F0741 /* NetworkContentRuleListManager.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = NetworkContentRuleListManager.messages.in; sourceTree = "<group>"; };
    34023408                41FBE824206DA79C000F0741 /* NetworkContentRuleListManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkContentRuleListManager.cpp; sourceTree = "<group>"; };
     3409                41FCD6AD23CCC26000C62567 /* AudioMediaStreamTrackRenderer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioMediaStreamTrackRenderer.h; sourceTree = "<group>"; };
     3410                41FCD6AE23CCC26100C62567 /* AudioMediaStreamTrackRenderer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AudioMediaStreamTrackRenderer.cpp; sourceTree = "<group>"; };
     3411                41FCD6AF23CCC26100C62567 /* AudioMediaStreamTrackRendererIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioMediaStreamTrackRendererIdentifier.h; sourceTree = "<group>"; };
    34033412                4450AEBF1DC3FAE5009943F2 /* SharedMemoryCocoa.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SharedMemoryCocoa.cpp; sourceTree = "<group>"; };
    34043413                4459984122833E6000E61373 /* SyntheticEditingCommandType.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = SyntheticEditingCommandType.h; sourceTree = "<group>"; };
    … …  
    71777186                        isa = PBXGroup;
    71787187                        children = (
     7188                                41FCD6AE23CCC26100C62567 /* AudioMediaStreamTrackRenderer.cpp */,
     7189                                41FCD6AD23CCC26000C62567 /* AudioMediaStreamTrackRenderer.h */,
     7190                                41FCD6AF23CCC26100C62567 /* AudioMediaStreamTrackRendererIdentifier.h */,
    71797191                                4172198A23B6128200AE5686 /* LibWebRTCCodecs.cpp */,
    71807192                                4172198923B6128200AE5686 /* LibWebRTCCodecs.h */,
    … …  
    71977209                                41E0A7C723B6397900561060 /* LibWebRTCCodecsProxy.messages.in */,
    71987210                                41E0A7C823B6397900561060 /* LibWebRTCCodecsProxy.mm */,
     7211                                41D6455423CCB92200486E0E /* RemoteAudioMediaStreamTrackRenderer.cpp */,
     7212                                41D6455323CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.h */,
     7213                                41D6455223CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.messages.in */,
     7214                                41D6455623CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.cpp */,
     7215                                41D6455723CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.h */,
     7216                                41D6455523CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.messages.in */,
    71997217                                4176E89D23C376D5003E83FE /* RemoteMediaRecorder.cpp */,
    72007218                                4176E89823C36677003E83FE /* RemoteMediaRecorder.h */,

  • trunk/Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.cpp

    r254499 r254505  
    2929#if ENABLE(GPU_PROCESS)
    3030
     31#include "AudioMediaStreamTrackRenderer.h"
    3132#include "MediaPlayerPrivateRemote.h"
     33#include "RemoteAudioMediaStreamTrackRenderer.h"
    3234#include "RemoteMediaPlayerConfiguration.h"
    3335#include "RemoteMediaPlayerMIMETypeCache.h"
    … …  
    361363
    362364    RemoteMediaPlayerSupport::setRegisterRemotePlayerCallback(settings.useGPUProcessForMedia() ? WTFMove(registerEngine) : RemoteMediaPlayerSupport::RegisterRemotePlayerCallback());
     365
     366    if (settings.useGPUProcessForMedia())
     367        WebCore::AudioMediaStreamTrackRenderer::setCreator(WebKit::AudioMediaStreamTrackRenderer::create);
    363368}
    364369

  • trunk/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h

    r254504 r254505  
    11/*
    2  * Copyright (C) 2017 Apple Inc. All rights reserved.
     2 * Copyright (C) 2020 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2626#pragma once
    2727
     28#if ENABLE(GPU_PROCESS)
    2829#if ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
    2930
    30 #include "AudioMediaStreamTrackRenderer.h"
    31 #include "Logging.h"
     31#include "AudioMediaStreamTrackRendererIdentifier.h"
     32#include "SharedRingBufferStorage.h"
     33#include <WebCore/AudioMediaStreamTrackRenderer.h>
     34#include <wtf/MediaTime.h>
    3235
    33 #include <AudioToolbox/AudioToolbox.h>
    34 #include <CoreAudio/CoreAudioTypes.h>
     36namespace IPC {
     37class Connection;
     38}
    3539
    36 namespace WebCore {
     40namespace WebKit {
    3741
    38 class AudioSampleDataSource;
    39 class AudioSampleBufferList;
    40 class CAAudioStreamDescription;
    41 
    42 class AudioMediaStreamTrackRendererCocoa : public AudioMediaStreamTrackRenderer {
     42class AudioMediaStreamTrackRenderer final : public WebCore::AudioMediaStreamTrackRenderer, public SharedRingBufferStorage::Client {
    4343    WTF_MAKE_FAST_ALLOCATED;
    4444public:
    45     AudioMediaStreamTrackRendererCocoa();
    46     ~AudioMediaStreamTrackRendererCocoa();
     45    static std::unique_ptr<WebCore::AudioMediaStreamTrackRenderer> create();
     46    ~AudioMediaStreamTrackRenderer();
     47
     48    AudioMediaStreamTrackRendererIdentifier identifier() const { return m_identifier; }
    4749
    4850private:
    49     // AudioMediaStreamTrackRenderer
    50     void pushSamples(const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final;
     51    explicit AudioMediaStreamTrackRenderer(Ref<IPC::Connection>&&);
     52
     53    // SharedRingBufferStorage::Client
     54    void storageChanged(SharedMemory*);
     55
     56    // WebCore::AudioMediaStreamTrackRenderer
     57    void start() final;
    5158    void stop() final;
    5259    void clear() final;
    53     void setPaused(bool) final;
     60    void setMuted(bool) final;
     61    void setVolume(float) final;
     62    void pushSamples(const MediaTime&, const WebCore::PlatformAudioData&, const WebCore::AudioStreamDescription&, size_t) final;
    5463
    55     static OSStatus inputProc(void*, AudioUnitRenderActionFlags*, const AudioTimeStamp*, UInt32 inBusNumber, UInt32 numberOfFrames, AudioBufferList*);
    56     OSStatus render(UInt32 sampleCount, AudioBufferList&, UInt32 inBusNumber, const AudioTimeStamp&, AudioUnitRenderActionFlags&);
     64    Ref<IPC::Connection> m_connection;
     65    AudioMediaStreamTrackRendererIdentifier m_identifier;
    5766
    58     AudioComponentInstance createAudioUnit(CAAudioStreamDescription&);
    59 
    60     // Audio thread members
    61     AudioComponentInstance m_remoteIOUnit { nullptr };
    62     std::unique_ptr<CAAudioStreamDescription> m_inputDescription;
    63     std::unique_ptr<CAAudioStreamDescription> m_outputDescription;
    64 
    65     // Cross thread members
    66     RefPtr<AudioSampleDataSource> m_dataSource;
    67     bool m_isAudioUnitStarted { false };
    68     bool m_paused { false };
     67    std::unique_ptr<WebCore::CARingBuffer> m_ringBuffer;
     68    WebCore::CAAudioStreamDescription m_description { };
     69    int64_t m_numberOfFrames { 0 };
     70    bool m_isPlaying { false };
    6971};
    7072
    … …  
    7274
    7375#endif // ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
     76#endif // ENABLE(GPU_PROCESS)
Note: See TracChangeset for help on using the changeset viewer.