Changeset 270294 in webkit


Ignore:
Timestamp:
Dec 1, 2020 4:34:23 AM (20 months ago)
Author:
youenn@apple.com
Message:

Allow RTCRtpScriptTransformer to request key frames
​https://bugs.webkit.org/show_bug.cgi?id=219199
Source/ThirdParty/libwebrtc:

Reviewed by Eric Carlson.

Add GenerateKeyFrame to sender and receiver interface.
Pipe the request to the necessary underlying objects implementing this as follows:

  • Ask encoder to generate a key frame for sender transforms
  • Send a FIR network request from receiver to sender for receiver transforms
  • Source/webrtc/api/rtp_receiver_interface.h:
  • Source/webrtc/api/rtp_sender_interface.h:
  • Source/webrtc/call/video_send_stream.h:
  • Source/webrtc/media/base/media_channel.h:

(cricket::MediaChannel::GenerateKeyFrame):

  • Source/webrtc/media/engine/webrtc_video_engine.cc:
  • Source/webrtc/media/engine/webrtc_video_engine.h:
  • Source/webrtc/pc/rtp_sender.cc:
  • Source/webrtc/pc/rtp_sender.h:
  • Source/webrtc/pc/video_rtp_receiver.cc:
  • Source/webrtc/pc/video_rtp_receiver.h:
  • Source/webrtc/video/video_send_stream.cc:
  • Source/webrtc/video/video_send_stream.h:

Source/WebCore:

<rdar://problem/71763010>

Reviewed by Eric Carlson.

Expose RTCRtpScriptTransformerContext to RTCRtpScriptTransformer as part of third parameter of the 'start' callback.
This context exposes some states (sender or receiver transform, audio or video transform).
It also exposes the ability to request key frames for video transforms.

Make RTCRtpTransformableFrame ref counted so that we can keep it even though the frame was piped to the writable stream.
Make LibWebRTCRtpTransformableFrame able to cope with a nullptr internal rtc frame.

Add support for RTCEncodedVideoFrame.type.

Tests: http/wpt/webrtc/audio-script-transform.html

http/wpt/webrtc/video-script-transform.html

  • CMakeLists.txt:
  • DerivedSources-input.xcfilelist:
  • DerivedSources-output.xcfilelist:
  • DerivedSources.make:
  • Modules/mediastream/RTCEncodedVideoFrame.cpp:

(WebCore::RTCEncodedVideoFrame::RTCEncodedVideoFrame):

  • Modules/mediastream/RTCEncodedVideoFrame.h:

(WebCore::RTCEncodedVideoFrame::type const):

  • Modules/mediastream/RTCEncodedVideoFrame.idl:
  • Modules/mediastream/RTCRtpScriptTransformerContext.h: Added.

(WebCore::RTCRtpScriptTransformerContext::side const):
(WebCore::RTCRtpScriptTransformerContext::mediaType const):
(WebCore::RTCRtpScriptTransformerContext::create):
(WebCore::RTCRtpScriptTransformerContext::RTCRtpScriptTransformerContext):
(WebCore::RTCRtpScriptTransformerContext::requestKeyFrame):

  • Modules/mediastream/RTCRtpScriptTransformerContext.idl: Added.
  • Modules/mediastream/RTCRtpTransformBackend.h:
  • Modules/mediastream/RTCRtpTransformableFrame.h:
  • Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp:

(WebCore::LibWebRTCRtpReceiverTransformBackend::requestKeyFrame):

  • Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h:
  • Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp:

(WebCore::LibWebRTCRtpSenderTransformBackend::requestKeyFrame):

  • Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h:
  • Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp:

(WebCore::LibWebRTCRtpTransformableFrame::isKeyFrame const):

  • Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h:
  • Sources.txt:
  • WebCore.xcodeproj/project.pbxproj:

LayoutTests:

Reviewed by Eric Carlson.

  • http/wpt/webrtc/audio-script-transform-expected.txt: Added.
  • http/wpt/webrtc/audio-script-transform.html: Added.
  • http/wpt/webrtc/video-script-transform-expected.txt: Added.
  • http/wpt/webrtc/video-script-transform.html: Added.
  • http/wpt/webrtc/context-transform.js: Added.
Location:
trunk
Files:
5 added
32 edited
2 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r270290 r270294  
     12020-12-01  Youenn Fablet  <youenn@apple.com>
     2
     3        Allow RTCRtpScriptTransformer to request key frames
     4        https://bugs.webkit.org/show_bug.cgi?id=219199
     5
     6        Reviewed by Eric Carlson.
     7
     8        * http/wpt/webrtc/audio-script-transform-expected.txt: Added.
     9        * http/wpt/webrtc/audio-script-transform.html: Added.
     10        * http/wpt/webrtc/video-script-transform-expected.txt: Added.
     11        * http/wpt/webrtc/video-script-transform.html: Added.
     12        * http/wpt/webrtc/context-transform.js: Added.
     13
    1142020-12-01  Youenn Fablet  <youenn@apple.com>
    215

  • trunk/Source/ThirdParty/libwebrtc/ChangeLog

    r270256 r270294  
     12020-12-01  Youenn Fablet  <youenn@apple.com>
     2
     3        Allow RTCRtpScriptTransformer to request key frames
     4        https://bugs.webkit.org/show_bug.cgi?id=219199
     5
     6        Reviewed by Eric Carlson.
     7
     8        Add GenerateKeyFrame to sender and receiver interface.
     9        Pipe the request to the necessary underlying objects implementing this as follows:
     10        - Ask encoder to generate a key frame for sender transforms
     11        - Send a FIR network request from receiver to sender for receiver transforms
     12
     13        * Source/webrtc/api/rtp_receiver_interface.h:
     14        * Source/webrtc/api/rtp_sender_interface.h:
     15        * Source/webrtc/call/video_send_stream.h:
     16        * Source/webrtc/media/base/media_channel.h:
     17        (cricket::MediaChannel::GenerateKeyFrame):
     18        * Source/webrtc/media/engine/webrtc_video_engine.cc:
     19        * Source/webrtc/media/engine/webrtc_video_engine.h:
     20        * Source/webrtc/pc/rtp_sender.cc:
     21        * Source/webrtc/pc/rtp_sender.h:
     22        * Source/webrtc/pc/video_rtp_receiver.cc:
     23        * Source/webrtc/pc/video_rtp_receiver.h:
     24        * Source/webrtc/video/video_send_stream.cc:
     25        * Source/webrtc/video/video_send_stream.h:
     26
    1272020-11-30  Youenn Fablet  <youenn@apple.com>
    228

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_receiver_interface.h

    r269642 r270294  
    114114  virtual void SetDepacketizerToDecoderFrameTransformer(
    115115      rtc::scoped_refptr<FrameTransformerInterface> frame_transformer);
     116#if defined(WEBRTC_WEBKIT_BUILD)
     117  virtual void GenerateKeyFrame() { }
     118#endif
    116119
    117120 protected:
    … …  
    143146              SetDepacketizerToDecoderFrameTransformer,
    144147              rtc::scoped_refptr<FrameTransformerInterface>)
     148#if defined(WEBRTC_WEBKIT_BUILD)
     149PROXY_METHOD0(void, GenerateKeyFrame)
     150#endif
    145151END_PROXY_MAP()
    146152

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_sender_interface.h

    r269642 r270294  
    9797  virtual void SetEncoderToPacketizerFrameTransformer(
    9898      rtc::scoped_refptr<FrameTransformerInterface> frame_transformer);
     99#if defined(WEBRTC_WEBKIT_BUILD)
     100  virtual void GenerateKeyFrame() { }
     101#endif
    99102
    100103 protected:
    … …  
    127130              SetEncoderToPacketizerFrameTransformer,
    128131              rtc::scoped_refptr<FrameTransformerInterface>)
     132#if defined(WEBRTC_WEBKIT_BUILD)
     133PROXY_METHOD0(void, GenerateKeyFrame)
     134#endif
    129135END_PROXY_MAP()
    130136

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/call/video_send_stream.h

    r269642 r270294  
    239239  virtual Stats GetStats() = 0;
    240240
     241#if defined(WEBRTC_WEBKIT_BUILD)
     242  virtual void GenerateKeyFrame() = 0;
     243#endif
     244
    241245 protected:
    242246  virtual ~VideoSendStream() {}

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.cc

    r269642 r270294  
    26442644    RecreateWebRtcStream();
    26452645}
     2646
     2647#if defined(WEBRTC_WEBKIT_BUILD)
     2648void WebRtcVideoChannel::WebRtcVideoSendStream::GenerateKeyFrame() {
     2649  RTC_DCHECK_RUN_ON(&thread_checker_);
     2650  if (stream_ != NULL)
     2651    stream_->GenerateKeyFrame();
     2652}
     2653#endif
    26462654
    26472655void WebRtcVideoChannel::WebRtcVideoSendStream::RecreateWebRtcStream() {
    … …  
    33173325  if (stream) {
    33183326    stream->GenerateKeyFrame();
    3319   } else {
    3320     RTC_LOG(LS_ERROR)
    3321         << "Absent receive stream; ignoring key frame generation for ssrc "
    3322         << ssrc;
    3323   }
     3327    return;
     3328  }
     3329#if defined(WEBRTC_WEBKIT_BUILD)
     3330  if (ssrc != 0) {
     3331    auto matching_stream = send_streams_.find(ssrc);
     3332    if (matching_stream != send_streams_.end()) {
     3333      matching_stream->second->GenerateKeyFrame();
     3334      return;
     3335    }
     3336  }
     3337#endif
     3338  RTC_LOG(LS_ERROR)
     3339      << "Absent receive stream; ignoring key frame generation for ssrc "
     3340      << ssrc;
    33243341}
    33253342

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.h

    r269642 r270294  
    360360        rtc::scoped_refptr<webrtc::FrameTransformerInterface>
    361361            frame_transformer);
     362#if defined(WEBRTC_WEBKIT_BUILD)
     363    void GenerateKeyFrame();
     364#endif
    362365
    363366   private:

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.cc

    r269642 r270294  
    648648}
    649649
     650#if defined(WEBRTC_WEBKIT_BUILD)
     651void VideoRtpSender::GenerateKeyFrame()
     652{
     653  if (video_media_channel() && ssrc_ && !stopped_) {
     654    worker_thread_->Invoke<void>(RTC_FROM_HERE, [&] {
     655        video_media_channel()->GenerateKeyFrame(ssrc_);
     656    });
     657  }
     658}
     659#endif
     660
    650661}  // namespace webrtc

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.h

    r269642 r270294  
    349349  void AttachTrack() override;
    350350
     351#if defined(WEBRTC_WEBKIT_BUILD)
     352  void GenerateKeyFrame() override;
     353#endif
     354
    351355 private:
    352356  cricket::VideoMediaChannel* video_media_channel() {

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.cc

    r269642 r270294  
    116116}
    117117
     118#if defined(WEBRTC_WEBKIT_BUILD)
     119void VideoRtpReceiver::GenerateKeyFrame()
     120{
     121  worker_thread_->Invoke<void>(RTC_FROM_HERE, [&] {
     122    RTC_DCHECK_RUN_ON(worker_thread_);
     123    if (media_channel_ && !stopped_) {
     124      media_channel_->GenerateKeyFrame(ssrc_.value_or(0));
     125    }
     126  });
     127}
     128#endif
     129
    118130void VideoRtpReceiver::Stop() {
    119131  // TODO(deadbeef): Need to do more here to fully stop receiving packets.

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.h

    r269642 r270294  
    8989  void SetDepacketizerToDecoderFrameTransformer(
    9090      rtc::scoped_refptr<FrameTransformerInterface> frame_transformer) override;
     91#if defined(WEBRTC_WEBKIT_BUILD)
     92  void GenerateKeyFrame() override;
     93#endif
    9194
    9295  // RtpReceiverInternal implementation.

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.cc

    r269642 r270294  
    241241}
    242242
     243#if defined(WEBRTC_WEBKIT_BUILD)
     244void VideoSendStream::GenerateKeyFrame()
     245{
     246  RTC_DCHECK_RUN_ON(&thread_checker_);
     247  if (video_stream_encoder_)
     248      video_stream_encoder_->SendKeyFrame();
     249}
     250#endif
     251
    243252}  // namespace internal
    244253}  // namespace webrtc

  • trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.h

    r269642 r270294  
    8787  void ReconfigureVideoEncoder(VideoEncoderConfig) override;
    8888  Stats GetStats() override;
     89#if defined(WEBRTC_WEBKIT_BUILD)
     90  void GenerateKeyFrame() override;
     91#endif
    8992
    9093  void StopPermanentlyAndGetRtpStates(RtpStateMap* rtp_state_map,

  • trunk/Source/WebCore/CMakeLists.txt

    r270257 r270294  
    411411    Modules/mediastream/RTCRtpScriptTransformer.idl
    412412    Modules/mediastream/RTCRtpScriptTransformerConstructor.idl
     413    Modules/mediastream/RTCRtpScriptTransformerContext.idl
    413414    Modules/mediastream/RTCRtpSendParameters.idl
    414415    Modules/mediastream/RTCRtpSender+Transform.idl

  • trunk/Source/WebCore/ChangeLog

    r270293 r270294  
     12020-12-01  Youenn Fablet  <youenn@apple.com>
     2
     3        Allow RTCRtpScriptTransformer to request key frames
     4        https://bugs.webkit.org/show_bug.cgi?id=219199
     5        <rdar://problem/71763010>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Expose RTCRtpScriptTransformerContext to RTCRtpScriptTransformer as part of third parameter of the 'start' callback.
     10        This context exposes some states (sender or receiver transform, audio or video transform).
     11        It also exposes the ability to request key frames for video transforms.
     12
     13        Make RTCRtpTransformableFrame ref counted so that we can keep it even though the frame was piped to the writable stream.
     14        Make LibWebRTCRtpTransformableFrame able to cope with a nullptr internal rtc frame.
     15
     16        Add support for RTCEncodedVideoFrame.type.
     17
     18        Tests: http/wpt/webrtc/audio-script-transform.html
     19               http/wpt/webrtc/video-script-transform.html
     20
     21        * CMakeLists.txt:
     22        * DerivedSources-input.xcfilelist:
     23        * DerivedSources-output.xcfilelist:
     24        * DerivedSources.make:
     25        * Modules/mediastream/RTCEncodedVideoFrame.cpp:
     26        (WebCore::RTCEncodedVideoFrame::RTCEncodedVideoFrame):
     27        * Modules/mediastream/RTCEncodedVideoFrame.h:
     28        (WebCore::RTCEncodedVideoFrame::type const):
     29        * Modules/mediastream/RTCEncodedVideoFrame.idl:
     30        * Modules/mediastream/RTCRtpScriptTransformerContext.h: Added.
     31        (WebCore::RTCRtpScriptTransformerContext::side const):
     32        (WebCore::RTCRtpScriptTransformerContext::mediaType const):
     33        (WebCore::RTCRtpScriptTransformerContext::create):
     34        (WebCore::RTCRtpScriptTransformerContext::RTCRtpScriptTransformerContext):
     35        (WebCore::RTCRtpScriptTransformerContext::requestKeyFrame):
     36        * Modules/mediastream/RTCRtpScriptTransformerContext.idl: Added.
     37        * Modules/mediastream/RTCRtpTransformBackend.h:
     38        * Modules/mediastream/RTCRtpTransformableFrame.h:
     39        * Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp:
     40        (WebCore::LibWebRTCRtpReceiverTransformBackend::requestKeyFrame):
     41        * Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h:
     42        * Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp:
     43        (WebCore::LibWebRTCRtpSenderTransformBackend::requestKeyFrame):
     44        * Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h:
     45        * Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp:
     46        (WebCore::LibWebRTCRtpTransformableFrame::isKeyFrame const):
     47        * Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h:
     48        * Sources.txt:
     49        * WebCore.xcodeproj/project.pbxproj:
     50
    1512020-12-01  Rob Buis  <rbuis@igalia.com>
    252

  • trunk/Source/WebCore/DerivedSources-input.xcfilelist

    r270257 r270294  
    245245$(PROJECT_DIR)/Modules/mediastream/RTCRtpScriptTransformer.idl
    246246$(PROJECT_DIR)/Modules/mediastream/RTCRtpScriptTransformerConstructor.idl
     247$(PROJECT_DIR)/Modules/mediastream/RTCRtpScriptTransformerContext.idl
    247248$(PROJECT_DIR)/Modules/mediastream/RTCRtpSendParameters.idl
    248249$(PROJECT_DIR)/Modules/mediastream/RTCRtpSender+Transform.idl

  • trunk/Source/WebCore/DerivedSources-output.xcfilelist

    r270257 r270294  
    17181718$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerConstructor.cpp
    17191719$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerConstructor.h
     1720$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerContext.cpp
     1721$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerContext.h
    17201722$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpSendParameters.cpp
    17211723$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpSendParameters.h

  • trunk/Source/WebCore/DerivedSources.make

    r270257 r270294  
    272272    $(WebCore)/Modules/mediastream/RTCRtpScriptTransformer.idl \
    273273    $(WebCore)/Modules/mediastream/RTCRtpScriptTransformerConstructor.idl \
     274    $(WebCore)/Modules/mediastream/RTCRtpScriptTransformerContext.idl \
    274275    $(WebCore)/Modules/mediastream/RTCRtpSynchronizationSource.idl \
    275276    $(WebCore)/Modules/mediastream/RTCRtpTransceiver.idl \

  • trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.cpp

    r270290 r270294  
    3333RTCEncodedVideoFrame::RTCEncodedVideoFrame(Ref<RTCRtpTransformableFrame>&& frame)
    3434    : RTCEncodedFrame(WTFMove(frame))
     35    , m_type(m_frame->isKeyFrame() ? Type::Key : Type::Delta)
    3536{
    3637}

  • trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.h

    r270290 r270294  
    3737    ~RTCEncodedVideoFrame();
    3838
     39    enum class Type { Empty, Key, Delta };
     40    Type type() const { return m_type; }
     41
    3942private:
    4043    explicit RTCEncodedVideoFrame(Ref<RTCRtpTransformableFrame>&&);
     44
     45    Type m_type;
    4146};
    4247

  • trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.idl

    r270107 r270294  
    2424 */
    2525
     26enum RTCEncodedVideoFrameType { "empty", "key", "delta" };
     27
    2628[
    2729    Conditional=WEB_RTC,
    … …  
    2931    ImplementationLacksVTable,
    3032] interface RTCEncodedVideoFrame {
    31     // readonly attribute RTCEncodedVideoFrameType type;
     33    readonly attribute RTCEncodedVideoFrameType type;
    3234    // readonly attribute unsigned long long timestamp;
    3335    attribute ArrayBuffer data;

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformer.cpp

    r270290 r270294  
    3333#include "JSRTCEncodedAudioFrame.h"
    3434#include "JSRTCEncodedVideoFrame.h"
     35#include "JSRTCRtpScriptTransformerContext.h"
    3536#include "RTCRtpTransformableFrame.h"
    3637#include "ReadableStream.h"
    … …  
    103104    args.append(toJSNewlyCreated(&globalObject, &globalObject, readableStream.releaseReturnValue()));
    104105    args.append(toJSNewlyCreated(&globalObject, &globalObject, writableStream.releaseReturnValue()));
     106    args.append(toJSNewlyCreated(&globalObject, &globalObject, RTCRtpScriptTransformerContext::create(makeRef(backend))));
    105107
    106108    NakedPtr<JSC::Exception> returnedException;

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.h

    r270293 r270294  
    2828#if ENABLE(WEB_RTC)
    2929
    30 #include "RTCEncodedFrame.h"
     30#include "RTCRtpTransformBackend.h"
     31#include <wtf/Ref.h>
    3132
    3233namespace WebCore {
    3334
    34 class RTCEncodedVideoFrame : public RTCEncodedFrame {
     35class RTCRtpTransformBackend;
     36
     37class RTCRtpScriptTransformerContext
     38    : public RefCounted<RTCRtpScriptTransformerContext> {
    3539public:
    36     static Ref<RTCEncodedVideoFrame> create(Ref<RTCRtpTransformableFrame>&& frame) { return adoptRef(*new RTCEncodedVideoFrame(WTFMove(frame))); }
    37     ~RTCEncodedVideoFrame();
     40    static Ref<RTCRtpScriptTransformerContext> create(Ref<RTCRtpTransformBackend>&&);
     41    ~RTCRtpScriptTransformerContext() = default;
     42
     43    using Side = RTCRtpTransformBackend::Side;
     44    Side side() const { return m_backend->side(); }
     45
     46    using MediaType = RTCRtpTransformBackend::MediaType;
     47    MediaType mediaType() const { return m_backend->mediaType(); }
     48
     49    ExceptionOr<void> requestKeyFrame();
    3850
    3951private:
    40     explicit RTCEncodedVideoFrame(Ref<RTCRtpTransformableFrame>&&);
     52    explicit RTCRtpScriptTransformerContext(Ref<RTCRtpTransformBackend>&&);
     53
     54    Ref<RTCRtpTransformBackend> m_backend;
    4155};
     56
     57inline Ref<RTCRtpScriptTransformerContext> RTCRtpScriptTransformerContext::create(Ref<RTCRtpTransformBackend>&& backend)
     58{
     59    return adoptRef(*new RTCRtpScriptTransformerContext(WTFMove(backend)));
     60}
     61
     62inline RTCRtpScriptTransformerContext::RTCRtpScriptTransformerContext(Ref<RTCRtpTransformBackend>&& backend)
     63    : m_backend(WTFMove(backend))
     64{
     65}
     66
     67inline ExceptionOr<void> RTCRtpScriptTransformerContext::requestKeyFrame()
     68{
     69    switch (mediaType()) {
     70    case MediaType::Audio:
     71        return Exception { InvalidStateError, "Cannot request key frame on audio sender"_s };
     72    case MediaType::Video:
     73        m_backend->requestKeyFrame();
     74        return { };
     75    }
     76}
    4277
    4378} // namespace WebCore

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.idl

    r270293 r270294  
    2424 */
    2525
     26enum RTCRtpScriptTransformerContextSide { "receiver", "sender" };
     27enum RTCRtpScriptTransformerContextMediaType { "audio", "video" };
     28
    2629[
    2730    Conditional=WEB_RTC,
    2831    Exposed=DedicatedWorker,
    2932    ImplementationLacksVTable,
    30 ] interface RTCEncodedVideoFrame {
    31     // readonly attribute RTCEncodedVideoFrameType type;
    32     // readonly attribute unsigned long long timestamp;
    33     attribute ArrayBuffer data;
    34     // RTCVideoFrameMetadata getMetadata();
     33] interface RTCRtpScriptTransformerContext {
     34    readonly attribute RTCRtpScriptTransformerContextSide side;
     35    readonly attribute RTCRtpScriptTransformerContextMediaType mediaType;
     36
     37    [MayThrowException] undefined requestKeyFrame();
    3538};

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformBackend.h

    r270290 r270294  
    4747    enum class Side { Receiver, Sender };
    4848    virtual Side side() const = 0;
     49
     50    virtual void requestKeyFrame() = 0;
    4951};
    5052

  • trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformableFrame.h

    r270290 r270294  
    4141    virtual Data data() const = 0;
    4242    virtual void setData(Data) = 0;
     43
     44    virtual bool isKeyFrame() const = 0;
    4345};
    4446

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp

    r269764 r270294  
    5353}
    5454
     55void LibWebRTCRtpReceiverTransformBackend::requestKeyFrame()
     56{
     57    ASSERT(mediaType() == MediaType::Video);
     58    m_rtcReceiver->GenerateKeyFrame();
     59}
     60
    5561} // namespace WebCore
    5662

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h

    r269789 r270294  
    5454    // RTCRtpTransformBackend
    5555    void setTransformableFrameCallback(Callback&&) final;
     56    void requestKeyFrame() final;
    5657
    5758    rtc::scoped_refptr<webrtc::RtpReceiverInterface> m_rtcReceiver;

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp

    r270290 r270294  
    5353}
    5454
     55void LibWebRTCRtpSenderTransformBackend::requestKeyFrame()
     56{
     57    ASSERT(mediaType() == MediaType::Video);
     58    m_rtcSender->GenerateKeyFrame();
     59}
     60
    5561} // namespace WebCore
    5662

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h

    r269789 r270294  
    5757    // RTCRtpTransformBackend
    5858    void setTransformableFrameCallback(Callback&&) final;
     59    void requestKeyFrame() final;
    5960
    6061    rtc::scoped_refptr<webrtc::RtpSenderInterface> m_rtcSender;

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp

    r270290 r270294  
    6666}
    6767
     68bool LibWebRTCRtpTransformableFrame::isKeyFrame() const
     69{
     70    ASSERT(m_rtcFrame);
     71    auto* videoFrame = static_cast<webrtc::TransformableVideoFrameInterface*>(m_rtcFrame.get());
     72    return videoFrame && videoFrame->IsKeyFrame();
     73}
     74
    6875} // namespace WebCore
    6976

  • trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h

    r270290 r270294  
    4747    explicit LibWebRTCRtpTransformableFrame(std::unique_ptr<webrtc::TransformableFrameInterface>&&);
    4848
     49    // RTCRtpTransformableFrame
    4950    Data data() const final;
    5051    void setData(Data) final;
     52    bool isKeyFrame() const final;
    5153
    5254    std::unique_ptr<webrtc::TransformableFrameInterface> m_rtcFrame;

  • trunk/Source/WebCore/Sources.txt

    r270290 r270294  
    33483348JSRTCRtpScriptTransformer.cpp
    33493349JSRTCRtpScriptTransformerConstructor.cpp
     3350JSRTCRtpScriptTransformerContext.cpp
    33503351JSRTCRtpSendParameters.cpp
    33513352JSRTCRtpSender.cpp

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r270290 r270294  
    77967796                419E6EC62566D213002B5010 /* RTCPeerConnectionIceErrorEvent.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = RTCPeerConnectionIceErrorEvent.cpp; sourceTree = "<group>"; };
    77977797                419E6EC72566D214002B5010 /* RTCPeerConnectionIceErrorEvent.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = RTCPeerConnectionIceErrorEvent.h; sourceTree = "<group>"; };
     7798                419E6ED22567BF69002B5010 /* RTCRtpScriptTransformerContext.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RTCRtpScriptTransformerContext.idl; sourceTree = "<group>"; };
     7799                419E6ED52567BF6A002B5010 /* RTCRtpScriptTransformerContext.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RTCRtpScriptTransformerContext.h; sourceTree = "<group>"; };
     7800                419E6ED82567D8D9002B5010 /* LibWebRTCRtpTransformableFrame.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = LibWebRTCRtpTransformableFrame.cpp; path = libwebrtc/LibWebRTCRtpTransformableFrame.cpp; sourceTree = "<group>"; };
    77987801                419FAFAD1ABABCD5005B3572 /* ReadableStreamBYOBReader.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = ReadableStreamBYOBReader.idl; sourceTree = "<group>"; };
    77997802                419FAFAD1ABABCD5005B828B /* ReadableStreamDefaultReader.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = ReadableStreamDefaultReader.idl; sourceTree = "<group>"; };
    … …  
    1718717190                                414189F62562E67A00363417 /* RTCRtpScriptTransformerConstructor.h */,
    1718817191                                414189F72562E67A00363417 /* RTCRtpScriptTransformerConstructor.idl */,
     17192                                419E6ED52567BF6A002B5010 /* RTCRtpScriptTransformerContext.h */,
     17193                                419E6ED22567BF69002B5010 /* RTCRtpScriptTransformerContext.idl */,
    1718917194                                414189EF2562DF0C00363417 /* RTCRtpScriptTransformProvider.idl */,
    1719017195                                413C8B262552EEA900E65055 /* RTCRtpSender+Transform.idl */,
Note: See TracChangeset for help on using the changeset viewer.