Changeset 207052 in webkit


Ignore:
Timestamp:
Oct 10, 2016 11:39:15 PM (8 years ago)
Author:
commit-queue@webkit.org
Message:

WebRTC: Make MediaEndpointPeerConnection handle remotely assigned mids correctly
​https://bugs.webkit.org/show_bug.cgi?id=163202

Patch by Adam Bergkvist <​adam.bergkvist@ericsson.com> and Alejandro G. Castro <​alex@igalia.com> on 2016-10-10
Reviewed by Eric Carlson.

Source/WebCore:

An RTCRtpTransceiver has a null mid until it's been associated with a
media description (with a mid) [1]. During that time, it's identified by
a provisional mid that might become its real mid, but the transceiver
can also get its mid assigned by a remote media description. In the
second case, the mid value is initially unknown. A transceiver's
RTCRtpSender must directly (synchronously in the script) provide a muted
remote source that is playable by, for example, a media element. This
source is initially registered in the MediaEndpoint (WebRTC backend)
with the transceiver's provisional mid. So, if the real mid is set by a
remote description, the registered mid must be updated to preserve the
association between the registered source and the transceiver.

[1] ​https://w3c.github.io/webrtc-pc/archives/20160913/webrtc.html#dom-rtcrtptransceiver-mid

Test: fast/mediastream/RTCPeerConnection-remotely-assigned-transceiver-mid.html

  • Modules/mediastream/MediaEndpointPeerConnection.cpp:

Don't break after finding the first transceiver in the loop that builds the send source map.
Update the mid used to register the muted remote source if the a transceiver's mid gets
assigned by a remote media description.
(WebCore::createSourceMap):
(WebCore::MediaEndpointPeerConnection::setRemoteDescriptionTask):

  • platform/mediastream/MediaEndpoint.cpp:
  • platform/mediastream/MediaEndpoint.h:
  • platform/mock/MockMediaEndpoint.cpp:

(WebCore::MockMediaEndpoint::MockMediaEndpoint):
(WebCore::MockMediaEndpoint::updateReceiveConfiguration):
(WebCore::MockMediaEndpoint::updateSendConfiguration):
(WebCore::MockMediaEndpoint::createMutedRemoteSource):
(WebCore::MockMediaEndpoint::replaceMutedRemoteSourceMid):
(WebCore::MockMediaEndpoint::emulatePlatformEvent):
Add "unmute-remote-sources-by-mid" action that emulates data arriving on media descriptions
which unmutes the remote sources.
(WebCore::MockMediaEndpoint::updateConfigurationMids):
(WebCore::MockMediaEndpoint::unmuteRemoteSourcesByMid):
(WebCore::MockMediaEndpoint::unmuteTimerFired):

  • platform/mock/MockMediaEndpoint.h:

LayoutTests:

Test the case when an RTCRtpTransceiver gets its mid assigned from a remote session
description.

  • fast/mediastream/RTCPeerConnection-remotely-assigned-transceiver-mid-expected.txt: Added.
  • fast/mediastream/RTCPeerConnection-remotely-assigned-transceiver-mid.html: Added.
  • platform/mac/TestExpectations:

Skip above test until the Mac port builds with WEB_RTC.

Location:
trunk
Files:
2 added
8 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r207045 r207052  
     12016-10-10  Adam Bergkvist  <adam.bergkvist@ericsson.com> and Alejandro G. Castro <alex@igalia.com>
     2
     3        WebRTC: Make MediaEndpointPeerConnection handle remotely assigned mids correctly
     4        https://bugs.webkit.org/show_bug.cgi?id=163202
     5
     6        Reviewed by Eric Carlson.
     7
     8        Test the case when an RTCRtpTransceiver gets its mid assigned from a remote session
     9        description.
     10
     11        * fast/mediastream/RTCPeerConnection-remotely-assigned-transceiver-mid-expected.txt: Added.
     12        * fast/mediastream/RTCPeerConnection-remotely-assigned-transceiver-mid.html: Added.
     13        * platform/mac/TestExpectations:
     14        Skip above test until the Mac port builds with WEB_RTC.
     15
    1162016-10-10  Gyuyoung Kim  <gyuyoung.kim@navercorp.com>
    217

  • trunk/LayoutTests/platform/mac/TestExpectations

    r207031 r207052  
    209209fast/mediastream/RTCPeerConnection-icecandidate-event.html
    210210fast/mediastream/RTCPeerConnection-iceconnectionstatechange-event.html
     211fast/mediastream/RTCPeerConnection-remotely-assigned-transceiver-mid.html
    211212
    212213# Asserts in debug.

  • trunk/Source/WebCore/ChangeLog

    r207050 r207052  
     12016-10-10  Adam Bergkvist  <adam.bergkvist@ericsson.com> and Alejandro G. Castro <alex@igalia.com>
     2
     3        WebRTC: Make MediaEndpointPeerConnection handle remotely assigned mids correctly
     4        https://bugs.webkit.org/show_bug.cgi?id=163202
     5
     6        Reviewed by Eric Carlson.
     7
     8        An RTCRtpTransceiver has a null mid until it's been associated with a
     9        media description (with a mid) [1]. During that time, it's identified by
     10        a provisional mid that might become its real mid, but the transceiver
     11        can also get its mid assigned by a remote media description. In the
     12        second case, the mid value is initially unknown. A transceiver's
     13        RTCRtpSender must directly (synchronously in the script) provide a muted
     14        remote source that is playable by, for example, a media element. This
     15        source is initially registered in the MediaEndpoint (WebRTC backend)
     16        with the transceiver's provisional mid. So, if the real mid is set by a
     17        remote description, the registered mid must be updated to preserve the
     18        association between the registered source and the transceiver.
     19
     20        [1] https://w3c.github.io/webrtc-pc/archives/20160913/webrtc.html#dom-rtcrtptransceiver-mid
     21
     22        Test: fast/mediastream/RTCPeerConnection-remotely-assigned-transceiver-mid.html
     23
     24        * Modules/mediastream/MediaEndpointPeerConnection.cpp:
     25        Don't break after finding the first transceiver in the loop that builds the send source map.
     26        Update the mid used to register the muted remote source if the a transceiver's mid gets
     27        assigned by a remote media description.
     28        (WebCore::createSourceMap):
     29        (WebCore::MediaEndpointPeerConnection::setRemoteDescriptionTask):
     30        * platform/mediastream/MediaEndpoint.cpp:
     31        * platform/mediastream/MediaEndpoint.h:
     32        * platform/mock/MockMediaEndpoint.cpp:
     33        (WebCore::MockMediaEndpoint::MockMediaEndpoint):
     34        (WebCore::MockMediaEndpoint::updateReceiveConfiguration):
     35        (WebCore::MockMediaEndpoint::updateSendConfiguration):
     36        (WebCore::MockMediaEndpoint::createMutedRemoteSource):
     37        (WebCore::MockMediaEndpoint::replaceMutedRemoteSourceMid):
     38        (WebCore::MockMediaEndpoint::emulatePlatformEvent):
     39        Add "unmute-remote-sources-by-mid" action that emulates data arriving on media descriptions
     40        which unmutes the remote sources.
     41        (WebCore::MockMediaEndpoint::updateConfigurationMids):
     42        (WebCore::MockMediaEndpoint::unmuteRemoteSourcesByMid):
     43        (WebCore::MockMediaEndpoint::unmuteTimerFired):
     44        * platform/mock/MockMediaEndpoint.h:
     45
    1462016-10-10  Darin Adler  <darin@apple.com>
    247

  • trunk/Source/WebCore/Modules/mediastream/MediaEndpointPeerConnection.cpp

    r206976 r207052  
    304304            if (transceiver->hasSendingDirection() && transceiver->sender()->track())
    305305                sourceMap.set(transceiver->mid(), &transceiver->sender()->track()->source());
    306             break;
    307306        }
    308307    }
    … …  
    501500                });
    502501
    503                 if (transceiver)
     502                if (transceiver) {
     503                    // This transceiver was created locally with a provisional mid. Its real mid will now be set by the remote
     504                    // description so we need to update the mid of the transceiver's muted source to preserve the association.
    504505                    transceiver->setMid(mediaDescription->mid());
    505                 else
     506                    m_mediaEndpoint->replaceMutedRemoteSourceMid(transceiver->provisionalMid(), mediaDescription->mid());
     507                } else
    506508                    receiveOnlyFlag = true;
    507509            }

  • trunk/Source/WebCore/platform/mediastream/MediaEndpoint.cpp

    r202624 r207052  
    7272    Ref<RealtimeMediaSource> createMutedRemoteSource(const String&, RealtimeMediaSource::Type) override { return EmptyRealtimeMediaSource::create(); }
    7373    void replaceSendSource(RealtimeMediaSource&, const String&) override { }
     74    void replaceMutedRemoteSourceMid(const String&, const String&) override { };
    7475
    7576    void stop() override { }

  • trunk/Source/WebCore/platform/mediastream/MediaEndpoint.h

    r206908 r207052  
    7878    virtual Ref<RealtimeMediaSource> createMutedRemoteSource(const String& mid, RealtimeMediaSource::Type) = 0;
    7979    virtual void replaceSendSource(RealtimeMediaSource&, const String& mid) = 0;
     80    virtual void replaceMutedRemoteSourceMid(const String& oldMid, const String& newMid) = 0;
    8081
    8182    virtual void stop() = 0;

  • trunk/Source/WebCore/platform/mock/MockMediaEndpoint.cpp

    r206908 r207052  
    3838#include "MockRealtimeAudioSource.h"
    3939#include "MockRealtimeVideoSource.h"
     40#include "RealtimeMediaSource.h"
    4041#include <wtf/MainThread.h>
    4142
    … …  
    5455    , m_iceCandidateTimer(*this, &MockMediaEndpoint::iceCandidateTimerFired)
    5556    , m_iceTransportTimer(*this, &MockMediaEndpoint::iceTransportTimerFired)
     57    , m_unmuteTimer(*this, &MockMediaEndpoint::unmuteTimerFired)
    5658{
    5759}
    … …  
    164166    UNUSED_PARAM(isInitiator);
    165167
    166     Vector<String> mids;
    167     for (const RefPtr<PeerMediaDescription>& mediaDescription : configuration->mediaDescriptions())
    168         mids.append(mediaDescription->mid());
    169     m_mids.swap(mids);
    170 
     168    updateConfigurationMids(*configuration);
    171169    return UpdateResult::Success;
    172170}
    … …  
    174172MediaEndpoint::UpdateResult MockMediaEndpoint::updateSendConfiguration(MediaEndpointSessionConfiguration* configuration, const RealtimeMediaSourceMap& sendSourceMap, bool isInitiator)
    175173{
    176     UNUSED_PARAM(configuration);
    177174    UNUSED_PARAM(sendSourceMap);
    178175    UNUSED_PARAM(isInitiator);
    179176
     177    updateConfigurationMids(*configuration);
    180178    return UpdateResult::Success;
    181179}
    … …  
    189187}
    190188
    191 Ref<RealtimeMediaSource> MockMediaEndpoint::createMutedRemoteSource(const String&, RealtimeMediaSource::Type type)
    192 {
    193     if (type == RealtimeMediaSource::Audio)
    194         return MockRealtimeAudioSource::createMuted("remote audio");
    195 
    196     ASSERT(type == RealtimeMediaSource::Video);
    197     return MockRealtimeVideoSource::createMuted("remote video");
     189Ref<RealtimeMediaSource> MockMediaEndpoint::createMutedRemoteSource(const String& mid, RealtimeMediaSource::Type type)
     190{
     191    RefPtr<RealtimeMediaSource> source;
     192
     193    switch (type) {
     194    case RealtimeMediaSource::Audio: source = MockRealtimeAudioSource::createMuted("remote audio"); break;
     195    case RealtimeMediaSource::Video: source = MockRealtimeVideoSource::createMuted("remote video"); break;
     196    case RealtimeMediaSource::None:
     197        ASSERT_NOT_REACHED();
     198    }
     199
     200    m_mutedRemoteSources.set(mid, source);
     201    return *source;
    198202}
    199203
    … …  
    202206    UNUSED_PARAM(newSource);
    203207    UNUSED_PARAM(mid);
     208}
     209
     210void MockMediaEndpoint::replaceMutedRemoteSourceMid(const String& oldMid, const String& newMid)
     211{
     212    RefPtr<RealtimeMediaSource> remoteSource = m_mutedRemoteSources.take(oldMid);
     213    m_mutedRemoteSources.set(newMid, WTFMove(remoteSource));
    204214}
    205215
    … …  
    214224    else if (action == "step-ice-transport-states")
    215225        stepIceTransportStates();
     226    else if (action == "unmute-remote-sources-by-mid")
     227        unmuteRemoteSourcesByMid();
     228}
     229
     230void MockMediaEndpoint::updateConfigurationMids(const MediaEndpointSessionConfiguration& configuration)
     231{
     232    Vector<String> mids;
     233    for (const RefPtr<PeerMediaDescription>& mediaDescription : configuration.mediaDescriptions())
     234        mids.append(mediaDescription->mid());
     235    m_mids.swap(mids);
    216236}
    217237
    … …  
    317337}
    318338
     339void MockMediaEndpoint::unmuteRemoteSourcesByMid()
     340{
     341    if (m_mids.isEmpty())
     342        return;
     343
     344    // Looking up each source by its mid, instead of simply iterating over the list of muted sources,
     345    // emulates remote media arriving on a media description with a specific mid (RTCRtpTransceiver).
     346
     347    // Copy values in reverse order to maintain the original order while using takeLast()
     348    for (int i = m_mids.size() - 1; i >= 0; --i)
     349        m_midsOfSourcesToUnmute.append(m_mids[i]);
     350
     351    m_unmuteTimer.startOneShot(0);
     352}
     353
     354void MockMediaEndpoint::unmuteTimerFired()
     355{
     356    RefPtr<RealtimeMediaSource> source = m_mutedRemoteSources.get(m_midsOfSourcesToUnmute.takeLast());
     357    if (source)
     358        source->setMuted(false);
     359
     360    if (!m_midsOfSourcesToUnmute.isEmpty())
     361        m_unmuteTimer.startOneShot(0);
     362}
     363
    319364} // namespace WebCore
    320365

  • trunk/Source/WebCore/platform/mock/MockMediaEndpoint.h

    r206908 r207052  
    6060    Ref<RealtimeMediaSource> createMutedRemoteSource(const String& mid, RealtimeMediaSource::Type) override;
    6161    void replaceSendSource(RealtimeMediaSource&, const String& mid) override;
     62    void replaceMutedRemoteSourceMid(const String& oldMid, const String& newMid) override;
    6263
    6364    void stop() override;
    … …  
    6667
    6768private:
     69    void updateConfigurationMids(const MediaEndpointSessionConfiguration&);
     70
    6871    void dispatchFakeIceCandidates();
    6972    void iceCandidateTimerFired();
    … …  
    7275    void iceTransportTimerFired();
    7376
     77    void unmuteRemoteSourcesByMid();
     78    void unmuteTimerFired();
     79
    7480    MediaEndpointClient& m_client;
    7581    Vector<String> m_mids;
     82    HashMap<String, RefPtr<RealtimeMediaSource>> m_mutedRemoteSources;
    7683
    7784    Vector<RefPtr<IceCandidate>> m_fakeIceCandidates;
    … …  
    8087    Vector<std::pair<String, MediaEndpoint::IceTransportState>> m_iceTransportStateChanges;
    8188    Timer m_iceTransportTimer;
     89
     90    Vector<String> m_midsOfSourcesToUnmute;
     91    Timer m_unmuteTimer;
    8292};
    8393
Note: See TracChangeset for help on using the changeset viewer.