⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 246644 in webkit


Ignore:
Timestamp:
Jun 20, 2019, 11:55:32 AM (7 years ago)
Author:
youenn@apple.com
Message:

Changing settings of a MediaStreamTrack clone should not alter the settings of the original track
​https://bugs.webkit.org/show_bug.cgi?id=198840

Reviewed by Eric Carlson.

Source/WebCore:

Rename RealtimeVideoSource in RealtimeVideoCaptureSource.
Introduce RealtimeVideoSource as a class wrapping RealtimeVideoCaptureSource.
Its goal is to be able to have independent settings from its underlying RealtimeVideoCaptureSource.
It can also adapt size based on its settings if different than its RealtimeVideoCaptureSource.
Apply this wrapping for AV video sources as well as mock video sources.
Test: fast/mediastream/mediastreamtrack-video-clone.html

  • Sources.txt:
  • WebCore.xcodeproj/project.pbxproj:
  • platform/mediastream/MediaStreamTrackPrivate.cpp:

(WebCore::MediaStreamTrackPrivate::clone):

  • platform/mediastream/RealtimeMediaSource.h:
  • platform/mediastream/RealtimeVideoCaptureSource.cpp: Added.

(WebCore::RealtimeVideoCaptureSource::RealtimeVideoCaptureSource):
(WebCore::RealtimeVideoCaptureSource::~RealtimeVideoCaptureSource):
(WebCore::RealtimeVideoCaptureSource::prepareToProduceData):
(WebCore::RealtimeVideoCaptureSource::presets):
(WebCore::RealtimeVideoCaptureSource::setSupportedPresets):
(WebCore::RealtimeVideoCaptureSource::standardVideoSizes):
(WebCore::updateMinMax):
(WebCore::RealtimeVideoCaptureSource::updateCapabilities):
(WebCore::RealtimeVideoCaptureSource::supportsSizeAndFrameRate):
(WebCore::RealtimeVideoCaptureSource::frameRateRangeIncludesRate):
(WebCore::RealtimeVideoCaptureSource::presetSupportsFrameRate):
(WebCore::RealtimeVideoCaptureSource::supportsCaptureSize):
(WebCore::RealtimeVideoCaptureSource::shouldUsePreset):
(WebCore::RealtimeVideoCaptureSource::bestSupportedSizeAndFrameRate):
(WebCore::RealtimeVideoCaptureSource::setSizeAndFrameRate):
(WebCore::RealtimeVideoCaptureSource::adaptVideoSample):
(WebCore::RealtimeVideoCaptureSource::dispatchMediaSampleToObservers):
(WebCore::RealtimeVideoCaptureSource::clientUpdatedSizeAndFrameRate):
(WebCore::SizeAndFrameRate::toJSONObject const):
(WebCore::SizeAndFrameRate::toJSONString const):

  • platform/mediastream/RealtimeVideoCaptureSource.h: Added.

(WebCore::RealtimeVideoCaptureSource::sampleRotation const):
(WebCore::RealtimeVideoCaptureSource::prefersPreset):
(WebCore::RealtimeVideoCaptureSource::setFrameRateWithPreset):
(WebCore::RealtimeVideoCaptureSource::canResizeVideoFrames const):
(WebCore::RealtimeVideoCaptureSource::setDefaultSize):
(WebCore::RealtimeVideoCaptureSource::observedFrameRate const):
(WTF::LogArgument<WebCore::SizeAndFrameRate>::toString):

  • platform/mediastream/RealtimeVideoSource.cpp:

(WebCore::RealtimeVideoSource::RealtimeVideoSource):
(WebCore::m_source):
(WebCore::RealtimeVideoSource::~RealtimeVideoSource):
(WebCore::RealtimeVideoSource::startProducingData):
(WebCore::RealtimeVideoSource::stopProducingData):
(WebCore::RealtimeVideoSource::supportsSizeAndFrameRate):
(WebCore::RealtimeVideoSource::setSizeAndFrameRate):
(WebCore::RealtimeVideoSource::sourceMutedChanged):
(WebCore::RealtimeVideoSource::sourceSettingsChanged):
(WebCore::RealtimeVideoSource::preventSourceFromStopping):
(WebCore::RealtimeVideoSource::sourceStopped):
(WebCore::RealtimeVideoSource::videoSampleAvailable):
(WebCore::RealtimeVideoSource::clone):

  • platform/mediastream/RealtimeVideoSource.h:
  • platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp:

(WebCore::GStreamerVideoCaptureSource::GStreamerVideoCaptureSource):

  • platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h:
  • platform/mediastream/mac/AVVideoCaptureSource.h:
  • platform/mediastream/mac/AVVideoCaptureSource.mm:

(WebCore::AVVideoCaptureSource::create):
(WebCore::AVVideoCaptureSource::AVVideoCaptureSource):

  • platform/mediastream/mac/MockRealtimeVideoSourceMac.h:
  • platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:

(WebCore::MockRealtimeVideoSource::create):

  • platform/mock/MockRealtimeVideoSource.cpp:

(WebCore::MockRealtimeVideoSource::create):
(WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource):
(WebCore::MockRealtimeVideoSource::supportsSizeAndFrameRate):
(WebCore::MockRealtimeVideoSource::setSizeAndFrameRate):

  • platform/mock/MockRealtimeVideoSource.h:

LayoutTests:

  • fast/mediastream/mediastreamtrack-video-clone-expected.txt: Added.
  • fast/mediastream/mediastreamtrack-video-clone.html: Added.
Location:
trunk
Files:
2 added
17 edited
2 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r246642 r246644  
     12019-06-20  Youenn Fablet  <youenn@apple.com>
     2
     3        Changing settings of a MediaStreamTrack clone should not alter the settings of the original track
     4        https://bugs.webkit.org/show_bug.cgi?id=198840
     5
     6        Reviewed by Eric Carlson.
     7
     8        * fast/mediastream/mediastreamtrack-video-clone-expected.txt: Added.
     9        * fast/mediastream/mediastreamtrack-video-clone.html: Added.
     10
    1112019-06-20  Russell Epstein  <russell_e@apple.com>
    212

  • trunk/Source/WebCore/ChangeLog

    r246643 r246644  
     12019-06-20  Youenn Fablet  <youenn@apple.com>
     2
     3        Changing settings of a MediaStreamTrack clone should not alter the settings of the original track
     4        https://bugs.webkit.org/show_bug.cgi?id=198840
     5
     6        Reviewed by Eric Carlson.
     7
     8        Rename RealtimeVideoSource in RealtimeVideoCaptureSource.
     9        Introduce RealtimeVideoSource as a class wrapping RealtimeVideoCaptureSource.
     10        Its goal is to be able to have independent settings from its underlying RealtimeVideoCaptureSource.
     11        It can also adapt size based on its settings if different than its RealtimeVideoCaptureSource.
     12        Apply this wrapping for AV video sources as well as mock video sources.
     13        Test: fast/mediastream/mediastreamtrack-video-clone.html
     14
     15        * Sources.txt:
     16        * WebCore.xcodeproj/project.pbxproj:
     17        * platform/mediastream/MediaStreamTrackPrivate.cpp:
     18        (WebCore::MediaStreamTrackPrivate::clone):
     19        * platform/mediastream/RealtimeMediaSource.h:
     20        * platform/mediastream/RealtimeVideoCaptureSource.cpp: Added.
     21        (WebCore::RealtimeVideoCaptureSource::RealtimeVideoCaptureSource):
     22        (WebCore::RealtimeVideoCaptureSource::~RealtimeVideoCaptureSource):
     23        (WebCore::RealtimeVideoCaptureSource::prepareToProduceData):
     24        (WebCore::RealtimeVideoCaptureSource::presets):
     25        (WebCore::RealtimeVideoCaptureSource::setSupportedPresets):
     26        (WebCore::RealtimeVideoCaptureSource::standardVideoSizes):
     27        (WebCore::updateMinMax):
     28        (WebCore::RealtimeVideoCaptureSource::updateCapabilities):
     29        (WebCore::RealtimeVideoCaptureSource::supportsSizeAndFrameRate):
     30        (WebCore::RealtimeVideoCaptureSource::frameRateRangeIncludesRate):
     31        (WebCore::RealtimeVideoCaptureSource::presetSupportsFrameRate):
     32        (WebCore::RealtimeVideoCaptureSource::supportsCaptureSize):
     33        (WebCore::RealtimeVideoCaptureSource::shouldUsePreset):
     34        (WebCore::RealtimeVideoCaptureSource::bestSupportedSizeAndFrameRate):
     35        (WebCore::RealtimeVideoCaptureSource::setSizeAndFrameRate):
     36        (WebCore::RealtimeVideoCaptureSource::adaptVideoSample):
     37        (WebCore::RealtimeVideoCaptureSource::dispatchMediaSampleToObservers):
     38        (WebCore::RealtimeVideoCaptureSource::clientUpdatedSizeAndFrameRate):
     39        (WebCore::SizeAndFrameRate::toJSONObject const):
     40        (WebCore::SizeAndFrameRate::toJSONString const):
     41        * platform/mediastream/RealtimeVideoCaptureSource.h: Added.
     42        (WebCore::RealtimeVideoCaptureSource::sampleRotation const):
     43        (WebCore::RealtimeVideoCaptureSource::prefersPreset):
     44        (WebCore::RealtimeVideoCaptureSource::setFrameRateWithPreset):
     45        (WebCore::RealtimeVideoCaptureSource::canResizeVideoFrames const):
     46        (WebCore::RealtimeVideoCaptureSource::setDefaultSize):
     47        (WebCore::RealtimeVideoCaptureSource::observedFrameRate const):
     48        (WTF::LogArgument<WebCore::SizeAndFrameRate>::toString):
     49        * platform/mediastream/RealtimeVideoSource.cpp:
     50        (WebCore::RealtimeVideoSource::RealtimeVideoSource):
     51        (WebCore::m_source):
     52        (WebCore::RealtimeVideoSource::~RealtimeVideoSource):
     53        (WebCore::RealtimeVideoSource::startProducingData):
     54        (WebCore::RealtimeVideoSource::stopProducingData):
     55        (WebCore::RealtimeVideoSource::supportsSizeAndFrameRate):
     56        (WebCore::RealtimeVideoSource::setSizeAndFrameRate):
     57        (WebCore::RealtimeVideoSource::sourceMutedChanged):
     58        (WebCore::RealtimeVideoSource::sourceSettingsChanged):
     59        (WebCore::RealtimeVideoSource::preventSourceFromStopping):
     60        (WebCore::RealtimeVideoSource::sourceStopped):
     61        (WebCore::RealtimeVideoSource::videoSampleAvailable):
     62        (WebCore::RealtimeVideoSource::clone):
     63        * platform/mediastream/RealtimeVideoSource.h:
     64        * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp:
     65        (WebCore::GStreamerVideoCaptureSource::GStreamerVideoCaptureSource):
     66        * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h:
     67        * platform/mediastream/mac/AVVideoCaptureSource.h:
     68        * platform/mediastream/mac/AVVideoCaptureSource.mm:
     69        (WebCore::AVVideoCaptureSource::create):
     70        (WebCore::AVVideoCaptureSource::AVVideoCaptureSource):
     71        * platform/mediastream/mac/MockRealtimeVideoSourceMac.h:
     72        * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
     73        (WebCore::MockRealtimeVideoSource::create):
     74        * platform/mock/MockRealtimeVideoSource.cpp:
     75        (WebCore::MockRealtimeVideoSource::create):
     76        (WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource):
     77        (WebCore::MockRealtimeVideoSource::supportsSizeAndFrameRate):
     78        (WebCore::MockRealtimeVideoSource::setSizeAndFrameRate):
     79        * platform/mock/MockRealtimeVideoSource.h:
     80
    1812019-06-20  Saam Barati  <sbarati@apple.com>
    282

  • trunk/Source/WebCore/Headers.cmake

    r246388 r246644  
    11461146    platform/mediastream/RealtimeMediaSourceSettings.h
    11471147    platform/mediastream/RealtimeMediaSourceSupportedConstraints.h
     1148    platform/mediastream/RealtimeVideoCaptureSource.h
    11481149    platform/mediastream/RealtimeVideoSource.h
    11491150    platform/mediastream/VideoPreset.h

  • trunk/Source/WebCore/Sources.txt

    r246631 r246644  
    18731873platform/mediastream/RealtimeOutgoingAudioSource.cpp
    18741874platform/mediastream/RealtimeOutgoingVideoSource.cpp
     1875platform/mediastream/RealtimeVideoCaptureSource.cpp
    18751876platform/mediastream/RealtimeVideoSource.cpp
    18761877

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r246631 r246644  
    113113                07277E5317D018CC0015534D /* JSMediaStreamTrack.h in Headers */ = {isa = PBXBuildFile; fileRef = 07277E4717D018CC0015534D /* JSMediaStreamTrack.h */; };
    114114                07277E5517D018CC0015534D /* JSMediaStreamTrackEvent.h in Headers */ = {isa = PBXBuildFile; fileRef = 07277E4917D018CC0015534D /* JSMediaStreamTrackEvent.h */; };
    115                 072880D12010F1F60071B255 /* RealtimeVideoSource.h in Headers */ = {isa = PBXBuildFile; fileRef = 072880D02010EED70071B255 /* RealtimeVideoSource.h */; settings = {ATTRIBUTES = (Private, ); }; };
     115                072880D12010F1F60071B255 /* RealtimeVideoCaptureSource.h in Headers */ = {isa = PBXBuildFile; fileRef = 072880D02010EED70071B255 /* RealtimeVideoCaptureSource.h */; settings = {ATTRIBUTES = (Private, ); }; };
    116116                072A70401D6E8F6200DF0AFC /* OverconstrainedErrorEvent.h in Headers */ = {isa = PBXBuildFile; fileRef = 072A703E1D6E8F6200DF0AFC /* OverconstrainedErrorEvent.h */; };
    117117                072AE1E5183C0741000A5988 /* PluginReplacement.h in Headers */ = {isa = PBXBuildFile; fileRef = 072AE1DF183C0741000A5988 /* PluginReplacement.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    11181118                41B28B3D1F860EF300FB52AC /* LibWebRTCProviderCocoa.h in Headers */ = {isa = PBXBuildFile; fileRef = 41B28B361F860BD000FB52AC /* LibWebRTCProviderCocoa.h */; settings = {ATTRIBUTES = (Private, ); }; };
    11191119                41B2A6261EF1BF6D002B9D7A /* WebAudioSourceProvider.h in Headers */ = {isa = PBXBuildFile; fileRef = 41B2A6251EF1BF60002B9D7A /* WebAudioSourceProvider.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1120                41BF204922BA7BE80004F812 /* RealtimeVideoSource.h in Headers */ = {isa = PBXBuildFile; fileRef = 41BF204022B947160004F812 /* RealtimeVideoSource.h */; settings = {ATTRIBUTES = (Private, ); }; };
    11201121                41C760B10EDE03D300C1655F /* ScriptState.h in Headers */ = {isa = PBXBuildFile; fileRef = 41C760B00EDE03D300C1655F /* ScriptState.h */; settings = {ATTRIBUTES = (Private, ); }; };
    11211122                41D015CA0F4B5C71004A662F /* ContentType.h in Headers */ = {isa = PBXBuildFile; fileRef = 41D015C80F4B5C71004A662F /* ContentType.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    53825383                07277E4917D018CC0015534D /* JSMediaStreamTrackEvent.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSMediaStreamTrackEvent.h; sourceTree = "<group>"; };
    53835384                072847E216EBC5B00043CFA4 /* PlatformTextTrack.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = PlatformTextTrack.h; sourceTree = "<group>"; };
    5384                 072880CE2010EED60071B255 /* RealtimeVideoSource.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RealtimeVideoSource.cpp; sourceTree = "<group>"; };
    5385                 072880D02010EED70071B255 /* RealtimeVideoSource.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RealtimeVideoSource.h; sourceTree = "<group>"; };
     5385                072880CE2010EED60071B255 /* RealtimeVideoCaptureSource.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RealtimeVideoCaptureSource.cpp; sourceTree = "<group>"; };
     5386                072880D02010EED70071B255 /* RealtimeVideoCaptureSource.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RealtimeVideoCaptureSource.h; sourceTree = "<group>"; };
    53865387                072A703E1D6E8F6200DF0AFC /* OverconstrainedErrorEvent.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = OverconstrainedErrorEvent.h; sourceTree = "<group>"; };
    53875388                072A703F1D6E8F6200DF0AFC /* OverconstrainedErrorEvent.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = OverconstrainedErrorEvent.idl; sourceTree = "<group>"; };
    … …  
    73897390                41B459DA1F4CADB90000F6FD /* ReadableStream.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ReadableStream.h; sourceTree = "<group>"; };
    73907391                41B459ED1F55EBC70000F6FD /* ReadableStream.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ReadableStream.cpp; sourceTree = "<group>"; };
     7392                41BF204022B947160004F812 /* RealtimeVideoSource.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = RealtimeVideoSource.h; sourceTree = "<group>"; };
     7393                41BF204222B947170004F812 /* RealtimeVideoSource.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = RealtimeVideoSource.cpp; sourceTree = "<group>"; };
    73917394                41C760B00EDE03D300C1655F /* ScriptState.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ScriptState.h; sourceTree = "<group>"; };
    73927395                41C7E1051E6A54360027B4DE /* CanvasCaptureMediaStreamTrack.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CanvasCaptureMediaStreamTrack.cpp; sourceTree = "<group>"; };
    … …  
    1585015853                                5CDD833B1E4324BB00621E92 /* RealtimeOutgoingVideoSource.cpp */,
    1585115854                                5CDD833C1E4324BB00621E92 /* RealtimeOutgoingVideoSource.h */,
    15852                                 072880CE2010EED60071B255 /* RealtimeVideoSource.cpp */,
    15853                                 072880D02010EED70071B255 /* RealtimeVideoSource.h */,
     15855                                072880CE2010EED60071B255 /* RealtimeVideoCaptureSource.cpp */,
     15856                                072880D02010EED70071B255 /* RealtimeVideoCaptureSource.h */,
     15857                                41BF204222B947170004F812 /* RealtimeVideoSource.cpp */,
     15858                                41BF204022B947160004F812 /* RealtimeVideoSource.h */,
    1585415859                                3135910C1E7DDCB600F30630 /* RTCBundlePolicy.h */,
    1585515860                                07221BA217CF0AD400848E51 /* RTCDataChannelHandler.h */,
    … …  
    3107231077                                41103AAC1E39791000769F03 /* RealtimeOutgoingAudioSource.h in Headers */,
    3107331078                                41103AAC1E39791000769F14 /* RealtimeOutgoingAudioSourceCocoa.h in Headers */,
    31074                                 072880D12010F1F60071B255 /* RealtimeVideoSource.h in Headers */,
     31079                                072880D12010F1F60071B255 /* RealtimeVideoCaptureSource.h in Headers */,
     31080                                41BF204922BA7BE80004F812 /* RealtimeVideoSource.h in Headers */,
    3107531081                                91B952241F58A58F00931DC2 /* RecordingSwizzleTypes.h in Headers */,
    3107631082                                BC4368E80C226E32005EFB5F /* Rect.h in Headers */,

  • trunk/Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp

    r246436 r246644  
    156156Ref<MediaStreamTrackPrivate> MediaStreamTrackPrivate::clone()
    157157{
    158     auto clonedMediaStreamTrackPrivate = create(m_logger.copyRef(), m_source.copyRef());
     158    auto clonedMediaStreamTrackPrivate = create(m_logger.copyRef(), m_source->clone());
    159159
    160160    clonedMediaStreamTrackPrivate->m_isEnabled = this->m_isEnabled;
    … …  
    162162    clonedMediaStreamTrackPrivate->m_contentHint = this->m_contentHint;
    163163    clonedMediaStreamTrackPrivate->updateReadyState();
     164
     165    if (isProducingData())
     166        clonedMediaStreamTrackPrivate->startProducingData();
    164167
    165168    return clonedMediaStreamTrackPrivate;

  • trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.h

    r243033 r246644  
    9797    virtual ~RealtimeMediaSource() = default;
    9898
     99    virtual Ref<RealtimeMediaSource> clone() { return *this; }
     100
    99101    const String& hashedId() const;
    100102    String deviceIDHashSalt() const;
    … …  
    225227    void audioSamplesAvailable(const MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t);
    226228
     229    void forEachObserver(const WTF::Function<void(Observer&)>&) const;
     230
    227231private:
    228232    virtual void startProducingData() { }
    … …  
    231235
    232236    virtual void hasEnded() { }
    233 
    234     void forEachObserver(const WTF::Function<void(Observer&)>&) const;
    235237
    236238#if !RELEASE_LOG_DISABLED

  • trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp

    r246643 r246644  
    2525
    2626#include "config.h"
    27 #include "RealtimeVideoSource.h"
     27#include "RealtimeVideoCaptureSource.h"
    2828
    2929#if ENABLE(MEDIA_STREAM)
    … …  
    4141namespace WebCore {
    4242
    43 RealtimeVideoSource::RealtimeVideoSource(String&& name, String&& id, String&& hashSalt)
     43RealtimeVideoCaptureSource::RealtimeVideoCaptureSource(String&& name, String&& id, String&& hashSalt)
    4444    : RealtimeMediaSource(Type::Video, WTFMove(name), WTFMove(id), WTFMove(hashSalt))
    4545{
    4646}
    4747
    48 RealtimeVideoSource::~RealtimeVideoSource()
     48RealtimeVideoCaptureSource::~RealtimeVideoCaptureSource()
    4949{
    5050#if PLATFORM(IOS_FAMILY)
    … …  
    5353}
    5454
    55 void RealtimeVideoSource::prepareToProduceData()
     55void RealtimeVideoCaptureSource::prepareToProduceData()
    5656{
    5757    ASSERT(frameRate());
    … …  
    6565}
    6666
    67 const Vector<Ref<VideoPreset>>& RealtimeVideoSource::presets()
     67const Vector<Ref<VideoPreset>>& RealtimeVideoCaptureSource::presets()
    6868{
    6969    if (m_presets.isEmpty())
    … …  
    7474}
    7575
    76 void RealtimeVideoSource::setSupportedPresets(Vector<VideoPresetData>&& presetData)
     76void RealtimeVideoCaptureSource::setSupportedPresets(Vector<VideoPresetData>&& presetData)
    7777{
    7878    Vector<Ref<VideoPreset>> presets;
    … …  
    8484}
    8585
    86 void RealtimeVideoSource::setSupportedPresets(const Vector<Ref<VideoPreset>>& presets)
     86void RealtimeVideoCaptureSource::setSupportedPresets(const Vector<Ref<VideoPreset>>& presets)
    8787{
    8888    m_presets = WTF::map(presets, [](auto& preset) {
    … …  
    9898}
    9999
    100 const Vector<IntSize>& RealtimeVideoSource::standardVideoSizes()
     100const Vector<IntSize>& RealtimeVideoCaptureSource::standardVideoSizes()
    101101{
    102102    static const auto sizes = makeNeverDestroyed([] {
    … …  
    153153}
    154154
    155 void RealtimeVideoSource::updateCapabilities(RealtimeMediaSourceCapabilities& capabilities)
     155void RealtimeVideoCaptureSource::updateCapabilities(RealtimeMediaSourceCapabilities& capabilities)
    156156{
    157157    ASSERT(!presets().isEmpty());
    … …  
    180180        minimumWidth = 1;
    181181        minimumHeight = 1;
    182         for (auto& size : standardVideoSizes()) {
    183             if (size.width() < minimumWidth || size.height() < minimumHeight)
    184                 minimumAspectRatio = std::min(minimumAspectRatio, static_cast<double>(size.width()) / size.height());
    185         }
    186182    }
    187183
    … …  
    192188}
    193189
    194 bool RealtimeVideoSource::supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate)
     190bool RealtimeVideoCaptureSource::supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate)
    195191{
    196192    if (!width && !height && !frameRate)
    … …  
    200196}
    201197
    202 bool RealtimeVideoSource::frameRateRangeIncludesRate(const FrameRateRange& range, double frameRate)
     198bool RealtimeVideoCaptureSource::frameRateRangeIncludesRate(const FrameRateRange& range, double frameRate)
    203199{
    204200    const double epsilon = 0.001;
    … …  
    206202}
    207203
    208 bool RealtimeVideoSource::presetSupportsFrameRate(RefPtr<VideoPreset> preset, double frameRate)
     204bool RealtimeVideoCaptureSource::presetSupportsFrameRate(RefPtr<VideoPreset> preset, double frameRate)
    209205{
    210206    for (const auto& range : preset->frameRateRanges) {
    … …  
    216212}
    217213
    218 bool RealtimeVideoSource::supportsCaptureSize(Optional<int> width, Optional<int> height, const Function<bool(const IntSize&)>&& function)
     214bool RealtimeVideoCaptureSource::supportsCaptureSize(Optional<int> width, Optional<int> height, const Function<bool(const IntSize&)>&& function)
    219215{
    220216    if (width && height)
    … …  
    238234}
    239235
    240 bool RealtimeVideoSource::shouldUsePreset(VideoPreset& current, VideoPreset& candidate)
     236bool RealtimeVideoCaptureSource::shouldUsePreset(VideoPreset& current, VideoPreset& candidate)
    241237{
    242238    return candidate.size.width() <= current.size.width() && candidate.size.height() <= current.size.height() && prefersPreset(candidate);
    243239}
    244240
    245 Optional<RealtimeVideoSource::CaptureSizeAndFrameRate> RealtimeVideoSource::bestSupportedSizeAndFrameRate(Optional<int> requestedWidth, Optional<int> requestedHeight, Optional<double> requestedFrameRate)
     241Optional<RealtimeVideoCaptureSource::CaptureSizeAndFrameRate> RealtimeVideoCaptureSource::bestSupportedSizeAndFrameRate(Optional<int> requestedWidth, Optional<int> requestedHeight, Optional<double> requestedFrameRate)
    246242{
    247243    if (!requestedWidth && !requestedHeight && !requestedFrameRate)
    … …  
    359355}
    360356
    361 void RealtimeVideoSource::setSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate)
     357void RealtimeVideoCaptureSource::setSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate)
    362358{
    363359    ALWAYS_LOG_IF(loggerPtr(), LOGIDENTIFIER, SizeAndFrameRate { width, height, frameRate });
    … …  
    369365    }
    370366
    371     Optional<RealtimeVideoSource::CaptureSizeAndFrameRate> match = bestSupportedSizeAndFrameRate(width, height, frameRate);
     367    Optional<RealtimeVideoCaptureSource::CaptureSizeAndFrameRate> match = bestSupportedSizeAndFrameRate(width, height, frameRate);
    372368    ASSERT(match);
    373369    if (!match)
    … …  
    381377}
    382378
    383 void RealtimeVideoSource::dispatchMediaSampleToObservers(MediaSample& sample)
     379RefPtr<MediaSample> RealtimeVideoCaptureSource::adaptVideoSample(MediaSample& sample)
    384380{
    385381    MediaTime sampleTime = sample.outputPresentationTime();
    … …  
    398394
    399395    auto mediaSample = makeRefPtr(&sample);
     396
    400397#if PLATFORM(COCOA)
    401398    if (!isRemote()) {
    … …  
    410407                if (!mediaSample) {
    411408                    ASSERT_NOT_REACHED();
    412                     return;
     409                    return nullptr;
    413410                }
    414411            }
    … …  
    417414#endif
    418415
    419     videoSampleAvailable(mediaSample.releaseNonNull());
     416    return mediaSample.releaseNonNull();
     417}
     418
     419void RealtimeVideoCaptureSource::dispatchMediaSampleToObservers(MediaSample& sample)
     420{
     421    if (auto mediaSample = adaptVideoSample(sample))
     422        videoSampleAvailable(*mediaSample);
     423}
     424
     425void RealtimeVideoCaptureSource::clientUpdatedSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate)
     426{
     427    // FIXME: We only change settings if capture resolution is below requested one. We should get the best preset for all clients.
     428    auto& settings = this->settings();
     429    if (width && *width < static_cast<int>(settings.width()))
     430        width = { };
     431    if (height && *height < static_cast<int>(settings.height()))
     432        height = { };
     433
     434    // FIXME: handle frameRate potential increase.
     435    if (!width && !height)
     436        return;
     437
     438    auto match = bestSupportedSizeAndFrameRate(width, height, frameRate);
     439    ERROR_LOG_IF(loggerPtr() && !match, LOGIDENTIFIER, "unable to find a preset that would match the size and frame rate");
     440    if (!match)
     441        return;
     442
     443    setFrameRateWithPreset(match->requestedFrameRate, match->encodingPreset);
     444    setSize(match->encodingPreset->size);
     445    setFrameRate(match->requestedFrameRate);
    420446}
    421447

  • trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h

    r246643 r246644  
    3939class ImageTransferSessionVT;
    4040
    41 class RealtimeVideoSource : public RealtimeMediaSource {
     41class RealtimeVideoCaptureSource : public RealtimeMediaSource {
    4242public:
    43     virtual ~RealtimeVideoSource();
     43    virtual ~RealtimeVideoCaptureSource();
     44
     45    void clientUpdatedSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate);
     46
     47    bool supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>) override;
     48    virtual void generatePresets() = 0;
     49    virtual MediaSample::VideoRotation sampleRotation() const { return MediaSample::VideoRotation::None; }
    4450
    4551protected:
    46     RealtimeVideoSource(String&& name, String&& id, String&& hashSalt);
     52    RealtimeVideoCaptureSource(String&& name, String&& id, String&& hashSalt);
    4753
    4854    void prepareToProduceData();
    49     bool supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>) override;
    5055    void setSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>) override;
    5156
    52     virtual void generatePresets() = 0;
    5357    virtual bool prefersPreset(VideoPreset&) { return true; }
    5458    virtual void setFrameRateWithPreset(double, RefPtr<VideoPreset>) { };
    … …  
    7074    void dispatchMediaSampleToObservers(MediaSample&);
    7175    const Vector<IntSize>& standardVideoSizes();
     76    RefPtr<MediaSample> adaptVideoSample(MediaSample&);
    7277
    7378private:
    … …  
    8287
    8388#if !RELEASE_LOG_DISABLED
    84     const char* logClassName() const override { return "RealtimeVideoSource"; }
     89    const char* logClassName() const override { return "RealtimeVideoCaptureSource"; }
    8590#endif
    8691

  • trunk/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp

    r246049 r246644  
    11/*
    2  * Copyright (C) 2018-2019 Apple Inc. All rights reserved.
     2 * Copyright (C) 2019 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2828
    2929#if ENABLE(MEDIA_STREAM)
    30 #include "CaptureDevice.h"
    31 #include "Logging.h"
    32 #include "RealtimeMediaSourceCenter.h"
    33 #include "RealtimeMediaSourceSettings.h"
    34 #include "RemoteVideoSample.h"
    35 #include <wtf/JSONValues.h>
    36 
    37 #if PLATFORM(COCOA)
    38 #include "ImageTransferSessionVT.h"
    39 #endif
    4030
    4131namespace WebCore {
    4232
    43 RealtimeVideoSource::RealtimeVideoSource(String&& name, String&& id, String&& hashSalt)
    44     : RealtimeMediaSource(Type::Video, WTFMove(name), WTFMove(id), WTFMove(hashSalt))
     33RealtimeVideoSource::RealtimeVideoSource(Ref<RealtimeVideoCaptureSource>&& source)
     34    : RealtimeVideoCaptureSource(String { source->name() }, String { source->persistentID() }, String { source->deviceIDHashSalt() })
     35    , m_source(WTFMove(source))
    4536{
     37    m_source->addObserver(*this);
     38    m_currentSettings = m_source->settings();
    4639}
    4740
    4841RealtimeVideoSource::~RealtimeVideoSource()
    4942{
    50 #if PLATFORM(IOS_FAMILY)
    51     RealtimeMediaSourceCenter::singleton().videoCaptureFactory().unsetActiveSource(*this);
    52 #endif
     43    m_source->removeObserver(*this);
    5344}
    5445
    55 void RealtimeVideoSource::prepareToProduceData()
     46void RealtimeVideoSource::startProducingData()
    5647{
    57     ASSERT(frameRate());
    58 
    59 #if PLATFORM(IOS_FAMILY)
    60     RealtimeMediaSourceCenter::singleton().videoCaptureFactory().setActiveSource(*this);
    61 #endif
    62 
    63     if (size().isEmpty() && !m_defaultSize.isEmpty())
    64         setSize(m_defaultSize);
     48    m_source->start();
    6549}
    6650
    67 const Vector<Ref<VideoPreset>>& RealtimeVideoSource::presets()
     51void RealtimeVideoSource::stopProducingData()
    6852{
    69     if (m_presets.isEmpty())
    70         generatePresets();
    71 
    72     ASSERT(!m_presets.isEmpty());
    73     return m_presets;
    74 }
    75 
    76 void RealtimeVideoSource::setSupportedPresets(Vector<VideoPresetData>&& presetData)
    77 {
    78     Vector<Ref<VideoPreset>> presets;
    79 
    80     for (auto& data : presetData)
    81         presets.append(VideoPreset::create(WTFMove(data)));
    82 
    83     setSupportedPresets(WTFMove(presets));
    84 }
    85 
    86 void RealtimeVideoSource::setSupportedPresets(const Vector<Ref<VideoPreset>>& presets)
    87 {
    88     m_presets = WTF::map(presets, [](auto& preset) {
    89         return preset.copyRef();
    90     });
    91 
    92     for (auto& preset : m_presets) {
    93         std::sort(preset->frameRateRanges.begin(), preset->frameRateRanges.end(),
    94             [&] (const auto& a, const auto& b) -> bool {
    95                 return a.minimum < b.minimum;
    96         });
    97     }
    98 }
    99 
    100 const Vector<IntSize>& RealtimeVideoSource::standardVideoSizes()
    101 {
    102     static const auto sizes = makeNeverDestroyed([] {
    103         static IntSize videoSizes[] = {
    104             { 112, 112 },
    105             { 160, 160 },
    106             { 160, 120 }, // 4:3, QQVGA
    107             { 176, 144 }, // 4:3, QCIF
    108             { 192, 192 },
    109             { 192, 112 }, // 16:9
    110             { 192, 144 }, // 3:4
    111             { 240, 240 },
    112             { 240, 160 }, // 3:2, HQVGA
    113             { 320, 320 },
    114             { 320, 180 }, // 16:9
    115             { 320, 240 }, // 4:3, QVGA
    116             { 352, 288 }, // CIF
    117             { 480, 272 }, // 16:9
    118             { 480, 360 }, // 4:3
    119             { 480, 480 },
    120             { 640, 640 },
    121             { 640, 360 }, // 16:9, 360p nHD
    122             { 640, 480 }, // 4:3
    123             { 720, 720 },
    124             { 800, 600 }, // 4:3, SVGA
    125             { 960, 540 }, // 16:9, qHD
    126             { 1024, 600 }, // 16:9, WSVGA
    127             { 1024, 768 }, // 4:3, XGA
    128             { 1280, 960 }, // 4:3
    129             { 1280, 1024 }, // 5:4, SXGA
    130             { 1280, 720 }, // 16:9, WXGA
    131             { 1366, 768 }, // 16:9, HD
    132             { 1600, 1200}, // 4:3, UXGA
    133             { 1920, 1080 }, // 16:9, 1080p FHD
    134             { 2560, 1440 }, // 16:9, QHD
    135             { 2592, 1936 },
    136             { 3264, 2448 }, // 3:4
    137             { 3840, 2160 }, // 16:9, 4K UHD
    138         };
    139         Vector<IntSize> sizes;
    140         for (auto& size : videoSizes)
    141             sizes.append(size);
    142 
    143         return sizes;
    144     }());
    145 
    146     return sizes.get();
    147 }
    148 template <typename ValueType>
    149 static void updateMinMax(ValueType& min, ValueType& max, ValueType value)
    150 {
    151     min = std::min<ValueType>(min, value);
    152     max = std::max<ValueType>(max, value);
    153 }
    154 
    155 void RealtimeVideoSource::updateCapabilities(RealtimeMediaSourceCapabilities& capabilities)
    156 {
    157     ASSERT(!presets().isEmpty());
    158 
    159     int minimumWidth = std::numeric_limits<int>::max();
    160     int maximumWidth = 0;
    161     int minimumHeight = std::numeric_limits<int>::max();
    162     int maximumHeight = 0;
    163     double minimumAspectRatio = std::numeric_limits<double>::max();
    164     double maximumAspectRatio = 0;
    165     double minimumFrameRate = std::numeric_limits<double>::max();
    166     double maximumFrameRate = 0;
    167     for (const auto& preset : presets()) {
    168         const auto& size = preset->size;
    169         updateMinMax(minimumWidth, maximumWidth, size.width());
    170         updateMinMax(minimumHeight, maximumHeight, size.height());
    171         updateMinMax(minimumAspectRatio, maximumAspectRatio, static_cast<double>(size.width()) / size.height());
    172 
    173         for (const auto& rate : preset->frameRateRanges) {
    174             updateMinMax(minimumFrameRate, maximumFrameRate, rate.minimum);
    175             updateMinMax(minimumFrameRate, maximumFrameRate, rate.maximum);
    176         }
    177     }
    178 
    179     if (canResizeVideoFrames()) {
    180         minimumWidth = 1;
    181         minimumHeight = 1;
    182         for (auto& size : standardVideoSizes()) {
    183             if (size.width() < minimumWidth || size.height() < minimumHeight)
    184                 minimumAspectRatio = std::min(minimumAspectRatio, static_cast<double>(size.width()) / size.height());
    185         }
    186     }
    187 
    188     capabilities.setWidth({ minimumWidth, maximumWidth });
    189     capabilities.setHeight({ minimumHeight, maximumHeight });
    190     capabilities.setAspectRatio({ minimumAspectRatio, maximumAspectRatio });
    191     capabilities.setFrameRate({ minimumFrameRate, maximumFrameRate });
     53    m_source->stop();
    19254}
    19355
    19456bool RealtimeVideoSource::supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate)
    19557{
    196     if (!width && !height && !frameRate)
    197         return true;
    198 
    199     return !!bestSupportedSizeAndFrameRate(width, height, frameRate);
    200 }
    201 
    202 bool RealtimeVideoSource::frameRateRangeIncludesRate(const FrameRateRange& range, double frameRate)
    203 {
    204     const double epsilon = 0.001;
    205     return frameRate + epsilon >= range.minimum && frameRate - epsilon <= range.maximum;
    206 }
    207 
    208 bool RealtimeVideoSource::presetSupportsFrameRate(RefPtr<VideoPreset> preset, double frameRate)
    209 {
    210     for (const auto& range : preset->frameRateRanges) {
    211         if (frameRateRangeIncludesRate(range, frameRate))
    212             return true;
    213     }
    214 
    215     return false;
    216 }
    217 
    218 bool RealtimeVideoSource::supportsCaptureSize(Optional<int> width, Optional<int> height, const Function<bool(const IntSize&)>&& function)
    219 {
    220     if (width && height)
    221         return function({ width.value(), height.value() });
    222 
    223     if (width) {
    224         for (auto& size : standardVideoSizes()) {
    225             if (width.value() == size.width() && function({ size.width(), size.height() }))
    226                 return true;
    227         }
    228 
    229         return false;
    230     }
    231 
    232     for (auto& size : standardVideoSizes()) {
    233         if (height.value() == size.height() && function({ size.width(), size.height() }))
    234             return true;
    235     }
    236 
    237     return false;
    238 }
    239 
    240 bool RealtimeVideoSource::shouldUsePreset(VideoPreset& current, VideoPreset& candidate)
    241 {
    242     return candidate.size.width() <= current.size.width() && candidate.size.height() <= current.size.height() && prefersPreset(candidate);
    243 }
    244 
    245 Optional<RealtimeVideoSource::CaptureSizeAndFrameRate> RealtimeVideoSource::bestSupportedSizeAndFrameRate(Optional<int> requestedWidth, Optional<int> requestedHeight, Optional<double> requestedFrameRate)
    246 {
    247     if (!requestedWidth && !requestedHeight && !requestedFrameRate)
    248         return { };
    249 
    250     if (!requestedWidth && !requestedHeight && !size().isEmpty()) {
    251         requestedWidth = size().width();
    252         requestedHeight = size().height();
    253     }
    254     if (!requestedFrameRate)
    255         requestedFrameRate = frameRate();
    256 
    257     CaptureSizeAndFrameRate result;
    258     RefPtr<VideoPreset> exactSizePreset;
    259     RefPtr<VideoPreset> aspectRatioPreset;
    260     IntSize aspectRatioMatchSize;
    261     RefPtr<VideoPreset> resizePreset;
    262     IntSize resizeSize;
    263 
    264     for (const auto& preset : presets()) {
    265         const auto& presetSize = preset->size;
    266 
    267         if (!presetSupportsFrameRate(&preset.get(), requestedFrameRate.value()))
    268             continue;
    269 
    270         if (!requestedWidth && !requestedHeight) {
    271             result.requestedFrameRate = requestedFrameRate.value();
    272             return result;
    273         }
    274 
    275         // Don't look at presets smaller than the requested resolution because we never want to resize larger.
    276         if ((requestedWidth && presetSize.width() < requestedWidth.value()) || (requestedHeight && presetSize.height() < requestedHeight.value()))
    277             continue;
    278 
    279         auto lookForExactSizeMatch = [&] (const IntSize& size) -> bool {
    280             return preset->size == size;
    281         };
    282         if (supportsCaptureSize(requestedWidth, requestedHeight, WTFMove(lookForExactSizeMatch))) {
    283             if (!exactSizePreset || prefersPreset(preset))
    284                 exactSizePreset = &preset.get();
    285             continue;
    286         }
    287 
    288         IntSize encodingSize;
    289         auto lookForAspectRatioMatch = [this, &preset, &encodingSize] (const IntSize& size) -> bool {
    290             auto aspectRatio = [] (const IntSize size) -> double {
    291                 return size.width() / static_cast<double>(size.height());
    292             };
    293             if (std::abs(aspectRatio(preset->size) - aspectRatio(size)) > 10e-7 || !canResizeVideoFrames())
    294                 return false;
    295 
    296             encodingSize = size;
    297             return true;
    298         };
    299         if (supportsCaptureSize(requestedWidth, requestedHeight, WTFMove(lookForAspectRatioMatch))) {
    300             if (!aspectRatioPreset || shouldUsePreset(*aspectRatioPreset, preset)) {
    301                 aspectRatioPreset = &preset.get();
    302                 aspectRatioMatchSize = encodingSize;
    303             }
    304         }
    305 
    306         if (exactSizePreset || aspectRatioPreset)
    307             continue;
    308 
    309         if ((requestedWidth && requestedWidth.value() > preset->size.width()) || (requestedHeight && requestedHeight.value() > preset->size.height()))
    310             continue;
    311 
    312         if (requestedWidth && requestedHeight) {
    313             if (!resizePreset || shouldUsePreset(*resizePreset, preset)) {
    314                 resizePreset = &preset.get();
    315                 resizeSize = { requestedWidth.value(), requestedHeight.value() };
    316             }
    317         } else {
    318             for (auto& standardSize : standardVideoSizes()) {
    319                 if (standardSize.width() > preset->size.width() || standardSize.height() > preset->size.height())
    320                     break;
    321                 if ((requestedWidth && requestedWidth.value() != standardSize.width()) || (requestedHeight && requestedHeight.value() != standardSize.height()))
    322                     continue;
    323 
    324                 if (!resizePreset || shouldUsePreset(*resizePreset, preset)) {
    325                     resizePreset = &preset.get();
    326                     resizeSize = standardSize;
    327                 }
    328             }
    329 
    330             if (!resizePreset || shouldUsePreset(*resizePreset, preset)) {
    331                 resizePreset = &preset.get();
    332                 if (requestedWidth)
    333                     resizeSize = { requestedWidth.value(), requestedWidth.value() * preset->size.height() / preset->size.width()};
    334                 else
    335                     resizeSize = { requestedHeight.value() * preset->size.width() / preset->size.height(), requestedHeight.value() };
    336             }
    337         }
    338     }
    339 
    340     if (!exactSizePreset && !aspectRatioPreset && !resizePreset)
    341         return { };
    342 
    343     result.requestedFrameRate = requestedFrameRate.value();
    344     if (exactSizePreset) {
    345         result.encodingPreset = exactSizePreset;
    346         result.requestedSize = exactSizePreset->size;
    347         return result;
    348     }
    349 
    350     if (aspectRatioPreset) {
    351         result.encodingPreset = aspectRatioPreset;
    352         result.requestedSize = aspectRatioMatchSize;
    353         return result;
    354     }
    355 
    356     result.encodingPreset = resizePreset;
    357     result.requestedSize = resizeSize;
    358     return result;
     58    return m_source->supportsSizeAndFrameRate(width, height, frameRate);
    35959}
    36060
    36161void RealtimeVideoSource::setSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate)
    36262{
    363     ALWAYS_LOG_IF(loggerPtr(), LOGIDENTIFIER, SizeAndFrameRate { width, height, frameRate });
    364 
    365     auto size = this->size();
    366     if (!width && !height && !size.isEmpty()) {
    367         width = size.width();
    368         height = size.height();
     63    if (!width && !height) {
     64        width = size().width();
     65        height = size().height();
    36966    }
    37067
    371     Optional<RealtimeVideoSource::CaptureSizeAndFrameRate> match = bestSupportedSizeAndFrameRate(width, height, frameRate);
    372     ASSERT(match);
    373     if (!match)
     68    m_source->clientUpdatedSizeAndFrameRate(width, height, frameRate);
     69    auto sourceSize = m_source->size();
     70    ASSERT(sourceSize.height());
     71    ASSERT(sourceSize.width());
     72
     73    if (!width)
     74        width = sourceSize.width() * height.value() / sourceSize.height();
     75    m_currentSettings.setWidth(*width);
     76
     77    if (!height)
     78        height = sourceSize.height() * width.value() / sourceSize.width();
     79    m_currentSettings.setHeight(*height);
     80
     81    if (frameRate)
     82        m_currentSettings.setFrameRate(static_cast<float>(*frameRate));
     83
     84    RealtimeMediaSource::setSizeAndFrameRate(width, height, frameRate);
     85}
     86
     87void RealtimeVideoSource::sourceMutedChanged()
     88{
     89    notifyMutedChange(m_source->muted());
     90}
     91
     92void RealtimeVideoSource::sourceSettingsChanged()
     93{
     94    auto rotation = m_source->sampleRotation();
     95    if (rotation == MediaSample::VideoRotation::Left || rotation == MediaSample::VideoRotation::Right) {
     96        auto size = this->size();
     97        size = size.transposedSize();
     98        m_currentSettings.setWidth(size.width());
     99        m_currentSettings.setHeight(size.height());
     100    }
     101
     102    forEachObserver([&](auto& observer) {
     103        observer.sourceSettingsChanged();
     104    });
     105}
     106
     107bool RealtimeVideoSource::preventSourceFromStopping()
     108{
     109    if (!isProducingData())
     110        return false;
     111
     112    bool hasObserverPreventingStopping = false;
     113    forEachObserver([&](auto& observer) {
     114        if (observer.preventSourceFromStopping())
     115            hasObserverPreventingStopping = true;
     116    });
     117    return hasObserverPreventingStopping;
     118}
     119
     120void RealtimeVideoSource::sourceStopped()
     121{
     122    if (m_source->captureDidFail()) {
     123        captureFailed();
     124        return;
     125    }
     126    stop();
     127    forEachObserver([](auto& observer) {
     128        observer.sourceStopped();
     129    });
     130}
     131
     132void RealtimeVideoSource::videoSampleAvailable(MediaSample& sample)
     133{
     134    if (!isProducingData())
    374135        return;
    375136
    376     setFrameRateWithPreset(match->requestedFrameRate, match->encodingPreset);
    377 
    378     if (!match->requestedSize.isEmpty())
    379         setSize(match->requestedSize);
    380     setFrameRate(match->requestedFrameRate);
     137    if (auto mediaSample = adaptVideoSample(sample))
     138        RealtimeMediaSource::videoSampleAvailable(*mediaSample);
    381139}
    382140
    383 void RealtimeVideoSource::dispatchMediaSampleToObservers(MediaSample& sample)
     141Ref<RealtimeMediaSource> RealtimeVideoSource::clone()
    384142{
    385     MediaTime sampleTime = sample.outputPresentationTime();
    386     if (!sampleTime || !sampleTime.isValid())
    387         sampleTime = sample.presentationTime();
     143    auto source = create(m_source.copyRef());
     144    source->m_currentSettings = m_currentSettings;
    388145
    389     auto frameTime = sampleTime.toDouble();
    390     m_observedFrameTimeStamps.append(frameTime);
    391     m_observedFrameTimeStamps.removeAllMatching([&](auto time) {
    392         return time <= frameTime - 2;
    393     });
    394 
    395     auto interval = m_observedFrameTimeStamps.last() - m_observedFrameTimeStamps.first();
    396     if (interval > 1)
    397         m_observedFrameRate = (m_observedFrameTimeStamps.size() / interval);
    398 
    399     auto mediaSample = makeRefPtr(&sample);
    400 #if PLATFORM(COCOA)
    401     if (!isRemote()) {
    402         auto size = this->size();
    403         if (!size.isEmpty() && size != expandedIntSize(sample.presentationSize())) {
    404 
    405             if (!m_imageTransferSession || m_imageTransferSession->pixelFormat() != sample.videoPixelFormat())
    406                 m_imageTransferSession = ImageTransferSessionVT::create(sample.videoPixelFormat());
    407 
    408             if (m_imageTransferSession) {
    409                 mediaSample = m_imageTransferSession->convertMediaSample(sample, size);
    410                 if (!mediaSample) {
    411                     ASSERT_NOT_REACHED();
    412                     return;
    413                 }
    414             }
    415         }
    416     }
    417 #endif
    418 
    419     videoSampleAvailable(mediaSample.releaseNonNull());
     146    return source;
    420147}
    421 
    422 #if !RELEASE_LOG_DISABLED
    423 Ref<JSON::Object> SizeAndFrameRate::toJSONObject() const
    424 {
    425     auto object = JSON::Object::create();
    426 
    427     object->setDouble("width"_s, width ? width.value() : 0);
    428     object->setDouble("height"_s, height ? height.value() : 0);
    429     object->setDouble("frameRate"_s, frameRate ? frameRate.value() : 0);
    430 
    431     return object;
    432 }
    433 
    434 String SizeAndFrameRate::toJSONString() const
    435 {
    436     return toJSONObject()->toJSONString();
    437 }
    438 #endif
    439148
    440149} // namespace WebCore

  • trunk/Source/WebCore/platform/mediastream/RealtimeVideoSource.h

    r245296 r246644  
    11/*
    2  * Copyright (C) 2018-2019 Apple Inc. All rights reserved.
     2 * Copyright (C) 2019 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2828#if ENABLE(MEDIA_STREAM)
    2929
    30 #include "ImageBuffer.h"
    31 #include "MediaSample.h"
    32 #include "RealtimeMediaSource.h"
    33 #include "VideoPreset.h"
    34 #include <wtf/Lock.h>
    35 #include <wtf/RunLoop.h>
     30#include "RealtimeVideoCaptureSource.h"
    3631
    3732namespace WebCore {
    3833
    39 class ImageTransferSessionVT;
    40 
    41 class RealtimeVideoSource : public RealtimeMediaSource {
     34// FIXME: Make RealtimeVideoSource derive from RealtimeMediaSource directly.
     35class RealtimeVideoSource final : public RealtimeVideoCaptureSource, public RealtimeMediaSource::Observer {
    4236public:
    43     virtual ~RealtimeVideoSource();
    44 
    45 protected:
    46     RealtimeVideoSource(String&& name, String&& id, String&& hashSalt);
    47 
    48     void prepareToProduceData();
    49     bool supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>) override;
    50     void setSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>) override;
    51 
    52     virtual void generatePresets() = 0;
    53     virtual bool prefersPreset(VideoPreset&) { return true; }
    54     virtual void setFrameRateWithPreset(double, RefPtr<VideoPreset>) { };
    55     virtual bool canResizeVideoFrames() const { return false; }
    56     bool shouldUsePreset(VideoPreset& current, VideoPreset& candidate);
    57 
    58     void setSupportedPresets(const Vector<Ref<VideoPreset>>&);
    59     void setSupportedPresets(Vector<VideoPresetData>&&);
    60     const Vector<Ref<VideoPreset>>& presets();
    61 
    62     bool frameRateRangeIncludesRate(const FrameRateRange&, double);
    63 
    64     void updateCapabilities(RealtimeMediaSourceCapabilities&);
    65 
    66     void setDefaultSize(const IntSize& size) { m_defaultSize = size; }
    67 
    68     double observedFrameRate() const { return m_observedFrameRate; }
    69 
    70     void dispatchMediaSampleToObservers(MediaSample&);
    71     const Vector<IntSize>& standardVideoSizes();
     37    static Ref<RealtimeVideoSource> create(Ref<RealtimeVideoCaptureSource>&& source) { return adoptRef(*new RealtimeVideoSource(WTFMove(source))); }
    7238
    7339private:
    74     struct CaptureSizeAndFrameRate {
    75         RefPtr<VideoPreset> encodingPreset;
    76         IntSize requestedSize;
    77         double requestedFrameRate { 0 };
    78     };
    79     bool supportsCaptureSize(Optional<int>, Optional<int>, const Function<bool(const IntSize&)>&&);
    80     Optional<CaptureSizeAndFrameRate> bestSupportedSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>);
    81     bool presetSupportsFrameRate(RefPtr<VideoPreset>, double);
     40    explicit RealtimeVideoSource(Ref<RealtimeVideoCaptureSource>&&);
     41    ~RealtimeVideoSource();
    8242
    83 #if !RELEASE_LOG_DISABLED
    84     const char* logClassName() const override { return "RealtimeVideoSource"; }
    85 #endif
     43    // RealtimeVideoCaptureSource
     44    void startProducingData() final;
     45    void stopProducingData() final;
     46    bool supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate) final;
     47    void setSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double> frameRate) final;
     48    Ref<RealtimeMediaSource> clone() final;
    8649
    87     Vector<Ref<VideoPreset>> m_presets;
    88     Deque<double> m_observedFrameTimeStamps;
    89     double m_observedFrameRate { 0 };
    90     IntSize m_defaultSize;
    91 #if PLATFORM(COCOA)
    92     std::unique_ptr<ImageTransferSessionVT> m_imageTransferSession;
    93 #endif
    94 };
     50    const RealtimeMediaSourceCapabilities& capabilities() final { return m_source->capabilities(); }
     51    const RealtimeMediaSourceSettings& settings() final { return m_currentSettings; }
     52    void generatePresets() final { m_source->generatePresets(); }
     53    bool isCaptureSource() const final { return m_source->isCaptureSource(); }
     54    CaptureDevice::DeviceType deviceType() const final { return m_source->deviceType(); }
     55    void monitorOrientation(OrientationNotifier& notifier) final { m_source->monitorOrientation(notifier); }
     56    bool interrupted() const final { return m_source->interrupted(); }
    9557
    96 struct SizeAndFrameRate {
    97     Optional<int> width;
    98     Optional<int> height;
    99     Optional<double> frameRate;
     58    // Observer
     59    void sourceMutedChanged() final;
     60    void sourceSettingsChanged() final;
     61    void sourceStopped() final;
     62    bool preventSourceFromStopping() final;
     63    void videoSampleAvailable(MediaSample&) final;
    10064
    101     String toJSONString() const;
    102     Ref<JSON::Object> toJSONObject() const;
     65    Ref<RealtimeVideoCaptureSource> m_source;
     66    RealtimeMediaSourceSettings m_currentSettings;
    10367};
    10468
    10569} // namespace WebCore
    10670
    107 namespace WTF {
    108 template<typename Type> struct LogArgument;
    109 template <>
    110 struct LogArgument<WebCore::SizeAndFrameRate> {
    111     static String toString(const WebCore::SizeAndFrameRate& size)
    112     {
    113         return size.toJSONString();
    114     }
    115 };
    116 }; // namespace WTF
    117 
    11871#endif // ENABLE(MEDIA_STREAM)
    11972

  • trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp

    r241587 r246644  
    127127
    128128GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(String&& deviceID, String&& name, String&& hashSalt, const gchar *source_factory)
    129     : RealtimeVideoSource(WTFMove(deviceID), WTFMove(name), WTFMove(hashSalt))
     129    : RealtimeVideoCaptureSource(WTFMove(deviceID), WTFMove(name), WTFMove(hashSalt))
    130130    , m_capturer(std::make_unique<GStreamerVideoCapturer>(source_factory))
    131131{
    … …  
    134134
    135135GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(GStreamerCaptureDevice device, String&& hashSalt)
    136     : RealtimeVideoSource(String { device.persistentId() }, String { device.label() }, WTFMove(hashSalt))
     136    : RealtimeVideoCaptureSource(String { device.persistentId() }, String { device.label() }, WTFMove(hashSalt))
    137137    , m_capturer(std::make_unique<GStreamerVideoCapturer>(device))
    138138{

  • trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h

    r239925 r246644  
    2525#include "CaptureDevice.h"
    2626#include "GStreamerVideoCapturer.h"
    27 #include "RealtimeVideoSource.h"
     27#include "RealtimeVideoCaptureSource.h"
    2828
    2929namespace WebCore {
    3030
    31 class GStreamerVideoCaptureSource : public RealtimeVideoSource {
     31class GStreamerVideoCaptureSource : public RealtimeVideoCaptureSource {
    3232public:
    3333    static CaptureSourceOrError create(String&& deviceID, String&& hashSalt, const MediaConstraints*);

  • trunk/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h

    r245296 r246644  
    3030#include "IntSizeHash.h"
    3131#include "OrientationNotifier.h"
    32 #include "RealtimeVideoSource.h"
     32#include "RealtimeVideoCaptureSource.h"
    3333#include <wtf/Lock.h>
    3434#include <wtf/text/StringHash.h>
    … …  
    5252class ImageTransferSessionVT;
    5353
    54 class AVVideoCaptureSource : public RealtimeVideoSource, private OrientationNotifier::Observer {
     54class AVVideoCaptureSource : public RealtimeVideoCaptureSource, private OrientationNotifier::Observer {
    5555public:
    5656    static CaptureSourceOrError create(String&& id, String&& hashSalt, const MediaConstraints*);
    … …  
    9292    bool interrupted() const final;
    9393
     94    MediaSample::VideoRotation sampleRotation() const final { return m_sampleRotation; }
    9495    void setFrameRateWithPreset(double, RefPtr<VideoPreset>) final;
    9596    bool prefersPreset(VideoPreset&) final;

  • trunk/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm

    r246489 r246644  
    3838#import "RealtimeMediaSourceCenter.h"
    3939#import "RealtimeMediaSourceSettings.h"
     40#import "RealtimeVideoSource.h"
    4041#import "RealtimeVideoUtilities.h"
    4142#import <AVFoundation/AVCaptureDevice.h>
    … …  
    122123    }
    123124
    124     return CaptureSourceOrError(WTFMove(source));
     125    return CaptureSourceOrError(RealtimeVideoSource::create(WTFMove(source)));
    125126}
    126127
    127128AVVideoCaptureSource::AVVideoCaptureSource(AVCaptureDevice* device, String&& id, String&& hashSalt)
    128     : RealtimeVideoSource(device.localizedName, WTFMove(id), WTFMove(hashSalt))
     129    : RealtimeVideoCaptureSource(device.localizedName, WTFMove(id), WTFMove(hashSalt))
    129130    , m_objcObserver(adoptNS([[WebCoreAVVideoCaptureSourceObserver alloc] initWithCallback:this]))
    130131    , m_device(device)
    … …  
    638639}
    639640
    640 
    641641} // namespace WebCore
    642642

  • trunk/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h

    r237236 r246644  
    6060    void orientationChanged(int orientation) final;
    6161    void monitorOrientation(OrientationNotifier&) final;
     62    MediaSample::VideoRotation sampleRotation() const final { return m_deviceOrientation; }
    6263
    6364    MediaSample::VideoRotation m_deviceOrientation { MediaSample::VideoRotation::None };

  • trunk/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm

    r244704 r246644  
    4242#import "PlatformLayer.h"
    4343#import "RealtimeMediaSourceSettings.h"
     44#import "RealtimeVideoSource.h"
    4445#import "RealtimeVideoUtilities.h"
    4546#import <QuartzCore/CALayer.h>
    … …  
    6768        return { };
    6869
    69     return CaptureSourceOrError(WTFMove(source));
     70    return CaptureSourceOrError(RealtimeVideoSource::create(WTFMove(source)));
    7071}
    7172

  • trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp

    r245296 r246644  
    4343#include "PlatformLayer.h"
    4444#include "RealtimeMediaSourceSettings.h"
     45#include "RealtimeVideoSource.h"
    4546#include <math.h>
    4647#include <wtf/UUID.h>
    … …  
    6364        return { };
    6465
    65     return CaptureSourceOrError(WTFMove(source));
     66    return CaptureSourceOrError(RealtimeVideoSource::create(WTFMove(source)));
    6667}
    6768#endif
    6869
    6970MockRealtimeVideoSource::MockRealtimeVideoSource(String&& deviceID, String&& name, String&& hashSalt)
    70     : RealtimeVideoSource(WTFMove(name), WTFMove(deviceID), WTFMove(hashSalt))
     71    : RealtimeVideoCaptureSource(WTFMove(name), WTFMove(deviceID), WTFMove(hashSalt))
    7172    , m_emitFrameTimer(RunLoop::current(), this, &MockRealtimeVideoSource::generateFrame)
    7273{
    … …  
    9899    // because of the RealtimeVideoSource inheritance.
    99100    if (mockCamera())
    100         return RealtimeVideoSource::supportsSizeAndFrameRate(width, height, rate);
     101        return RealtimeVideoCaptureSource::supportsSizeAndFrameRate(width, height, rate);
    101102
    102103    return RealtimeMediaSource::supportsSizeAndFrameRate(width, height, rate);
    … …  
    108109    // because of the RealtimeVideoSource inheritance.
    109110    if (mockCamera()) {
    110         RealtimeVideoSource::setSizeAndFrameRate(width, height, rate);
     111        RealtimeVideoCaptureSource::setSizeAndFrameRate(width, height, rate);
    111112        return;
    112113    }

  • trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h

    r245296 r246644  
    3838#include "MockMediaDevice.h"
    3939#include "RealtimeMediaSourceFactory.h"
    40 #include "RealtimeVideoSource.h"
     40#include "RealtimeVideoCaptureSource.h"
    4141#include <wtf/RunLoop.h>
    4242
    … …  
    4646class GraphicsContext;
    4747
    48 class MockRealtimeVideoSource : public RealtimeVideoSource {
     48class MockRealtimeVideoSource : public RealtimeVideoCaptureSource {
    4949public:
    5050
Note: See TracChangeset for help on using the changeset viewer.