Changeset 270486 in webkit


Ignore:
Timestamp:
Dec 6, 2020 11:21:19 AM (20 months ago)
Author:
youenn@apple.com
Message:

Add support for RTCRtpSender::setStreams
​https://bugs.webkit.org/show_bug.cgi?id=219542

Reviewed by Eric Carlson.

LayoutTests/imported/w3c:

  • web-platform-tests/webrtc/RTCPeerConnection-setRemoteDescription-rollback-expected.txt:
  • web-platform-tests/webrtc/RTCRtpSender-setStreams.https-expected.txt:

Source/WebCore:

Pipe stream IDs from WebCore sender to libqebrtc backend sender.
A follow-up patch should handle creation of remote streams in case of renegotiation.
Covered by updated and rebased tests.

  • Modules/mediastream/RTCRtpSender.cpp:

(WebCore::RTCRtpSender::create):
(WebCore::RTCRtpSender::RTCRtpSender):
(WebCore::RTCRtpSender::setStreams):
(WebCore::RTCRtpSender::setMediaStreamIds):

  • Modules/mediastream/RTCRtpSender.h:
  • Modules/mediastream/RTCRtpSender.idl:
  • Modules/mediastream/RTCRtpSenderBackend.h:
  • Modules/mediastream/libwebrtc/LibWebRTCPeerConnectionBackend.cpp:

(WebCore::LibWebRTCPeerConnectionBackend::addTrack):
(WebCore::LibWebRTCPeerConnectionBackend::addTransceiverFromTrackOrKind):
(WebCore::LibWebRTCPeerConnectionBackend::newRemoteTransceiver):

  • Modules/mediastream/libwebrtc/LibWebRTCRtpSenderBackend.cpp:

(WebCore::LibWebRTCRtpSenderBackend::setMediaStreamIds):

  • Modules/mediastream/libwebrtc/LibWebRTCRtpSenderBackend.h:

LayoutTests:

  • webrtc/video.html:
Location:
trunk
Files:
14 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r270471 r270486  
     12020-12-06  Youenn Fablet  <youenn@apple.com>
     2
     3        Add support for RTCRtpSender::setStreams
     4        https://bugs.webkit.org/show_bug.cgi?id=219542
     5
     6        Reviewed by Eric Carlson.
     7
     8        * webrtc/video.html:
     9
    1102020-12-05  Diego Pino Garcia  <dpino@igalia.com>
    211

  • trunk/LayoutTests/imported/w3c/ChangeLog

    r270459 r270486  
     12020-12-06  Youenn Fablet  <youenn@apple.com>
     2
     3        Add support for RTCRtpSender::setStreams
     4        https://bugs.webkit.org/show_bug.cgi?id=219542
     5
     6        Reviewed by Eric Carlson.
     7
     8        * web-platform-tests/webrtc/RTCPeerConnection-setRemoteDescription-rollback-expected.txt:
     9        * web-platform-tests/webrtc/RTCRtpSender-setStreams.https-expected.txt:
     10
    1112020-12-04  Chris Dumez  <cdumez@apple.com>
    212

  • trunk/LayoutTests/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-setRemoteDescription-rollback-expected.txt

    r267649 r270486  
    1818FAIL two transceivers with same mids promise_test: Unhandled rejection with value: object "InvalidStateError: Description type incompatible with current signaling state"
    1919FAIL onremovetrack fires during remote rollback promise_test: Unhandled rejection with value: object "InvalidStateError: Description type incompatible with current signaling state"
    20 FAIL rollback of a remote offer with stream changes promise_test: Unhandled rejection with value: object "TypeError: pc1.getTransceivers()[0].sender.setStreams is not a function. (In 'pc1.getTransceivers()[0].sender.setStreams(stream2)', 'pc1.getTransceivers()[0].sender.setStreams' is undefined)"
     20FAIL rollback of a remote offer with stream changes assert_equals: expected 2 but got 1
    2121PASS removeTrack() with a sender being rolled back does not crash or throw
    2222FAIL Implicit rollback with only a datachannel works promise_test: Unhandled rejection with value: object "InvalidStateError: Description type incompatible with current signaling state"

  • trunk/LayoutTests/imported/w3c/web-platform-tests/webrtc/RTCRtpSender-setStreams.https-expected.txt

    r264202 r270486  
    11
    2 FAIL setStreams causes streams to be reported via ontrack on callee promise_test: Unhandled rejection with value: object "TypeError: sender.setStreams is not a function. (In 'sender.setStreams(stream1, stream2)', 'sender.setStreams' is undefined)"
    3 FAIL setStreams can be used to reconstruct a stream with a track on the remote side promise_test: Unhandled rejection with value: object "TypeError: sender.setStreams is not a function. (In 'sender.setStreams(stream)', 'sender.setStreams' is undefined)"
    4 FAIL Adding streams and changing direction causes new streams to be reported via ontrack on callee promise_test: Unhandled rejection with value: object "TypeError: transceiver.sender.setStreams is not a function. (In 'transceiver.sender.setStreams(stream1, stream2)', 'transceiver.sender.setStreams' is undefined)"
    5 FAIL Adding streams to an active transceiver causes new streams to be reported via ontrack on callee promise_test: Unhandled rejection with value: object "TypeError: transceiver.sender.setStreams is not a function. (In 'transceiver.sender.setStreams(stream1, stream2)', 'transceiver.sender.setStreams' is undefined)"
    6 FAIL setStreams() fires InvalidStateError on a closed peer connection. assert_throws_dom: function "() => transceiver.sender.setStreams(stream1, stream2)" threw object "TypeError: transceiver.sender.setStreams is not a function. (In 'transceiver.sender.setStreams(stream1, stream2)', 'transceiver.sender.setStreams' is undefined)" that is not a DOMException InvalidStateError: property "code" is equal to undefined, expected 11
     2Harness Error (TIMEOUT), message = null
    73
     4PASS setStreams causes streams to be reported via ontrack on callee
     5PASS setStreams can be used to reconstruct a stream with a track on the remote side
     6PASS Adding streams and changing direction causes new streams to be reported via ontrack on callee
     7TIMEOUT Adding streams to an active transceiver causes new streams to be reported via ontrack on callee Test timed out
     8PASS setStreams() fires InvalidStateError on a closed peer connection.
     9

  • trunk/LayoutTests/imported/w3c/web-platform-tests/webrtc/idlharness.https.window-expected.txt

    r270101 r270486  
    238238PASS RTCRtpSender interface: operation getParameters()
    239239PASS RTCRtpSender interface: operation replaceTrack(MediaStreamTrack?)
    240 FAIL RTCRtpSender interface: operation setStreams(MediaStream...) assert_own_property: interface prototype object missing non-static operation expected property "setStreams" missing
     240PASS RTCRtpSender interface: operation setStreams(MediaStream...)
    241241PASS RTCRtpSender interface: operation getStats()
    242242PASS RTCRtpSender interface: attribute dtmf
    … …  
    252252PASS RTCRtpSender interface: new RTCPeerConnection().addTransceiver('audio').sender must inherit property "replaceTrack(MediaStreamTrack?)" with the proper type
    253253PASS RTCRtpSender interface: calling replaceTrack(MediaStreamTrack?) on new RTCPeerConnection().addTransceiver('audio').sender with too few arguments must throw TypeError
    254 FAIL RTCRtpSender interface: new RTCPeerConnection().addTransceiver('audio').sender must inherit property "setStreams(MediaStream...)" with the proper type assert_inherits: property "setStreams" not found in prototype chain
    255 FAIL RTCRtpSender interface: calling setStreams(MediaStream...) on new RTCPeerConnection().addTransceiver('audio').sender with too few arguments must throw TypeError assert_inherits: property "setStreams" not found in prototype chain
     254PASS RTCRtpSender interface: new RTCPeerConnection().addTransceiver('audio').sender must inherit property "setStreams(MediaStream...)" with the proper type
     255PASS RTCRtpSender interface: calling setStreams(MediaStream...) on new RTCPeerConnection().addTransceiver('audio').sender with too few arguments must throw TypeError
    256256PASS RTCRtpSender interface: new RTCPeerConnection().addTransceiver('audio').sender must inherit property "getStats()" with the proper type
    257257PASS RTCRtpSender interface: new RTCPeerConnection().addTransceiver('audio').sender must inherit property "dtmf" with the proper type

  • trunk/LayoutTests/webrtc/video.html

    r265047 r270486  
    5050
    5151    const localStream = await navigator.mediaDevices.getUserMedia({video: {advanced: [{width:{min:640}}, {height:{min:480} } ]}});
     52    const localStream2 = new MediaStream([localStream.getVideoTracks()[0]]);
    5253    if (window.internals)
    5354        assert_true(internals.pageMediaState().includes('HasActiveVideoCaptureDevice'), "Unexpected HasActiveVideoCaptureDevice");
    … …  
    5556        createConnections((firstConnection) => {
    5657            pc1 = firstConnection;
    57             firstConnection.addTrack(localStream.getVideoTracks()[0], localStream);
     58            const sender = firstConnection.addTrack(localStream.getVideoTracks()[0]);
     59            sender.setStreams(localStream, localStream2);
    5860        }, (secondConnection) => {
    5961            pc2 = secondConnection;
    … …  
    6365                assert_true(Array.isArray(trackEvent.streams), "Array.isArray() should return true");
    6466                assert_true(Object.isFrozen(trackEvent.streams), "Object.isFrozen() should return true");
     67                assert_equals(trackEvent.streams.length, 2);
     68                assert_equals(trackEvent.streams[0].id, localStream.id, "first stream id");
     69                assert_equals(trackEvent.streams[1].id, localStream2.id, "second stream id");
    6570                assert_equals(trackEvent.track.id, localStream.getVideoTracks()[0].id);
    6671                assert_equals(trackEvent.track, trackEvent.streams[0].getVideoTracks()[0]);

  • trunk/Source/WebCore/ChangeLog

    r270485 r270486  
     12020-12-06  Youenn Fablet  <youenn@apple.com>
     2
     3        Add support for RTCRtpSender::setStreams
     4        https://bugs.webkit.org/show_bug.cgi?id=219542
     5
     6        Reviewed by Eric Carlson.
     7
     8        Pipe stream IDs from WebCore sender to libqebrtc backend sender.
     9        A follow-up patch should handle creation of remote streams in case of renegotiation.
     10        Covered by updated and rebased tests.
     11
     12        * Modules/mediastream/RTCRtpSender.cpp:
     13        (WebCore::RTCRtpSender::create):
     14        (WebCore::RTCRtpSender::RTCRtpSender):
     15        (WebCore::RTCRtpSender::setStreams):
     16        (WebCore::RTCRtpSender::setMediaStreamIds):
     17        * Modules/mediastream/RTCRtpSender.h:
     18        * Modules/mediastream/RTCRtpSender.idl:
     19        * Modules/mediastream/RTCRtpSenderBackend.h:
     20        * Modules/mediastream/libwebrtc/LibWebRTCPeerConnectionBackend.cpp:
     21        (WebCore::LibWebRTCPeerConnectionBackend::addTrack):
     22        (WebCore::LibWebRTCPeerConnectionBackend::addTransceiverFromTrackOrKind):
     23        (WebCore::LibWebRTCPeerConnectionBackend::newRemoteTransceiver):
     24        * Modules/mediastream/libwebrtc/LibWebRTCRtpSenderBackend.cpp:
     25        (WebCore::LibWebRTCRtpSenderBackend::setMediaStreamIds):
     26        * Modules/mediastream/libwebrtc/LibWebRTCRtpSenderBackend.h:
     27
    1282020-12-06  Zalan Bujtas  <zalan@apple.com>
    229

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpSender.cpp

    r270107 r270486  
    4646WTF_MAKE_ISO_ALLOCATED_IMPL(RTCRtpSender);
    4747
    48 Ref<RTCRtpSender> RTCRtpSender::create(RTCPeerConnection& connection, Ref<MediaStreamTrack>&& track, Vector<String>&& mediaStreamIds, std::unique_ptr<RTCRtpSenderBackend>&& backend)
    49 {
    50     auto sender = adoptRef(*new RTCRtpSender(connection, String(track->kind()), WTFMove(mediaStreamIds), WTFMove(backend)));
     48Ref<RTCRtpSender> RTCRtpSender::create(RTCPeerConnection& connection, Ref<MediaStreamTrack>&& track, std::unique_ptr<RTCRtpSenderBackend>&& backend)
     49{
     50    auto sender = adoptRef(*new RTCRtpSender(connection, String(track->kind()), WTFMove(backend)));
    5151    sender->setTrack(WTFMove(track));
    5252    return sender;
    5353}
    5454
    55 Ref<RTCRtpSender> RTCRtpSender::create(RTCPeerConnection& connection, String&& trackKind, Vector<String>&& mediaStreamIds, std::unique_ptr<RTCRtpSenderBackend>&& backend)
    56 {
    57     return adoptRef(*new RTCRtpSender(connection, WTFMove(trackKind), WTFMove(mediaStreamIds), WTFMove(backend)));
    58 }
    59 
    60 RTCRtpSender::RTCRtpSender(RTCPeerConnection& connection, String&& trackKind, Vector<String>&& mediaStreamIds, std::unique_ptr<RTCRtpSenderBackend>&& backend)
     55Ref<RTCRtpSender> RTCRtpSender::create(RTCPeerConnection& connection, String&& trackKind, std::unique_ptr<RTCRtpSenderBackend>&& backend)
     56{
     57    return adoptRef(*new RTCRtpSender(connection, WTFMove(trackKind), WTFMove(backend)));
     58}
     59
     60RTCRtpSender::RTCRtpSender(RTCPeerConnection& connection, String&& trackKind, std::unique_ptr<RTCRtpSenderBackend>&& backend)
    6161    : m_trackKind(WTFMove(trackKind))
    62     , m_mediaStreamIds(WTFMove(mediaStreamIds))
    6362    , m_backend(WTFMove(backend))
    6463    , m_connection(makeWeakPtr(connection))
    … …  
    146145    }
    147146    return m_backend->setParameters(parameters, WTFMove(promise));
     147}
     148
     149ExceptionOr<void> RTCRtpSender::setStreams(const Vector<std::reference_wrapper<MediaStream>>& streams)
     150{
     151    return setMediaStreamIds(WTF::map(streams, [](auto& stream) -> String {
     152        return stream.get().id();
     153    }));
     154}
     155
     156ExceptionOr<void> RTCRtpSender::setMediaStreamIds(const Vector<String>& streamIds)
     157{
     158    if (!m_connection || m_connection->isClosed() || !m_backend)
     159        return Exception { InvalidStateError, "connection is closed"_s };
     160    m_backend->setMediaStreamIds(streamIds);
     161    return { };
    148162}
    149163

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpSender.h

    r270107 r270486  
    4343namespace WebCore {
    4444
     45class MediaStream;
    4546class RTCDTMFSender;
    4647class RTCPeerConnection;
    … …  
    5051    WTF_MAKE_ISO_ALLOCATED(RTCRtpSender);
    5152public:
    52     static Ref<RTCRtpSender> create(RTCPeerConnection&, Ref<MediaStreamTrack>&&, Vector<String>&& mediaStreamIds, std::unique_ptr<RTCRtpSenderBackend>&&);
    53     static Ref<RTCRtpSender> create(RTCPeerConnection&, String&& trackKind, Vector<String>&& mediaStreamIds, std::unique_ptr<RTCRtpSenderBackend>&&);
     53    static Ref<RTCRtpSender> create(RTCPeerConnection&, Ref<MediaStreamTrack>&&, std::unique_ptr<RTCRtpSenderBackend>&&);
     54    static Ref<RTCRtpSender> create(RTCPeerConnection&, String&& trackKind, std::unique_ptr<RTCRtpSenderBackend>&&);
    5455    ~RTCRtpSender();
    5556
    … …  
    6162    const String& trackKind() const { return m_trackKind; }
    6263
    63     const Vector<String>& mediaStreamIds() const { return m_mediaStreamIds; }
    64     void setMediaStreamIds(Vector<String>&& mediaStreamIds) { m_mediaStreamIds = WTFMove(mediaStreamIds); }
     64    ExceptionOr<void> setMediaStreamIds(const Vector<String>&);
     65    ExceptionOr<void> setStreams(const Vector<std::reference_wrapper<MediaStream>>&);
    6566
    6667    bool isStopped() const { return !m_backend; }
    … …  
    8788
    8889private:
    89     RTCRtpSender(RTCPeerConnection&, String&& trackKind, Vector<String>&& mediaStreamIds, std::unique_ptr<RTCRtpSenderBackend>&&);
     90    RTCRtpSender(RTCPeerConnection&, String&& trackKind, std::unique_ptr<RTCRtpSenderBackend>&&);
    9091
    9192    RefPtr<MediaStreamTrack> m_track;
    9293    String m_trackId;
    9394    String m_trackKind;
    94     Vector<String> m_mediaStreamIds;
    9595    std::unique_ptr<RTCRtpSenderBackend> m_backend;
    9696    WeakPtr<RTCPeerConnection> m_connection;

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpSender.idl

    r267813 r270486  
    3737] interface RTCRtpSender {
    3838    readonly attribute MediaStreamTrack? track;
    39     // FIXME 169662: missing transport
    40     // FIXME 169662: missing rtcpTransport
    41     // FIXME 169662: missing setStreams
     39    // FIXME 169662: readonly attribute RTCDtlsTransport? transport;
    4240    [CallWith=ScriptExecutionContext] static RTCRtpCapabilities? getCapabilities(DOMString kind);
    4341    RTCRtpSendParameters getParameters();
    4442    Promise<undefined> setParameters(RTCRtpSendParameters parameters);
    4543    Promise<undefined> replaceTrack(MediaStreamTrack? withTrack);
     44    [MayThrowException] undefined setStreams(MediaStream... streams);
    4645    Promise<RTCStatsReport> getStats();
    4746

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpSenderBackend.h

    r269764 r270486  
    2727#if ENABLE(WEB_RTC)
    2828
     29#include <wtf/Forward.h>
     30#include <wtf/Vector.h>
     31
    2932namespace WebCore {
    3033
    … …  
    4851    virtual std::unique_ptr<RTCDTMFSenderBackend> createDTMFBackend() = 0;
    4952    virtual Ref<RTCRtpTransformBackend> createRTCRtpTransformBackend() = 0;
     53    virtual void setMediaStreamIds(const Vector<String>&) = 0;
    5054};
    5155

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCPeerConnectionBackend.cpp

    r266511 r270486  
    371371        backendFromRTPSender(*sender).takeSource(*senderBackend);
    372372        sender->setTrack(makeRef(track));
    373         sender->setMediaStreamIds(WTFMove(mediaStreamIds));
     373        sender->setMediaStreamIds(mediaStreamIds);
    374374        return sender.releaseNonNull();
    375375    }
    … …  
    377377    auto transceiverBackend = m_endpoint->transceiverBackendFromSender(*senderBackend);
    378378
    379     auto sender = RTCRtpSender::create(m_peerConnection, makeRef(track), WTFMove(mediaStreamIds), WTFMove(senderBackend));
     379    auto sender = RTCRtpSender::create(m_peerConnection, makeRef(track), WTFMove(senderBackend));
     380    sender->setMediaStreamIds(mediaStreamIds);
    380381    auto receiver = createReceiver(transceiverBackend->createReceiverBackend());
    381382    auto transceiver = RTCRtpTransceiver::create(sender.copyRef(), WTFMove(receiver), WTFMove(transceiverBackend));
    … …  
    391392        return Exception { InvalidAccessError, "Unable to add transceiver"_s };
    392393
    393     auto sender = RTCRtpSender::create(m_peerConnection, WTFMove(trackOrKind), Vector<String> { }, WTFMove(backends->senderBackend));
     394    auto sender = RTCRtpSender::create(m_peerConnection, WTFMove(trackOrKind), WTFMove(backends->senderBackend));
    394395    auto receiver = createReceiver(WTFMove(backends->receiverBackend));
    395396    auto transceiver = RTCRtpTransceiver::create(WTFMove(sender), WTFMove(receiver), WTFMove(backends->transceiverBackend));
    … …  
    429430RTCRtpTransceiver& LibWebRTCPeerConnectionBackend::newRemoteTransceiver(std::unique_ptr<LibWebRTCRtpTransceiverBackend>&& transceiverBackend, RealtimeMediaSource::Type type)
    430431{
    431     auto sender = RTCRtpSender::create(m_peerConnection, type == RealtimeMediaSource::Type::Audio ? "audio"_s : "video"_s, Vector<String> { }, transceiverBackend->createSenderBackend(*this, nullptr));
     432    auto sender = RTCRtpSender::create(m_peerConnection, type == RealtimeMediaSource::Type::Audio ? "audio"_s : "video"_s, transceiverBackend->createSenderBackend(*this, nullptr));
    432433    auto receiver = createReceiver(transceiverBackend->createReceiverBackend());
    433434    auto transceiver = RTCRtpTransceiver::create(WTFMove(sender), WTFMove(receiver), WTFMove(transceiverBackend));

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderBackend.cpp

    r269764 r270486  
    144144}
    145145
     146void LibWebRTCRtpSenderBackend::setMediaStreamIds(const Vector<String>& streamIds)
     147{
     148    std::vector<std::string> ids;
     149    for (auto& id : streamIds)
     150        ids.push_back(id.utf8().data());
     151    m_rtcSender->SetStreams(ids);
     152}
     153
    146154RealtimeOutgoingVideoSource* LibWebRTCRtpSenderBackend::videoSource()
    147155{

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderBackend.h

    r269764 r270486  
    7272    std::unique_ptr<RTCDTMFSenderBackend> createDTMFBackend() final;
    7373    Ref<RTCRtpTransformBackend> createRTCRtpTransformBackend() final;
     74    void setMediaStreamIds(const Vector<String>&) final;
    7475
    7576    void startSource();
Note: See TracChangeset for help on using the changeset viewer.