⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 248301 in webkit


Ignore:
Timestamp:
Aug 6, 2019, 10:05:13 AM (7 years ago)
Author:
jer.noble@apple.com
Message:

Adopt -expectMinimumUpcomingSampleBufferPresentationTime:
​https://bugs.webkit.org/show_bug.cgi?id=200457
<rdar://problem/53961130>

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/media-source/media-source-minimumupcomingpresentationtime.html

Adopt a new API vended by AVSampleBufferDisplayLayer, piped from SourceBuffer down
through SourceBufferPrivate to SourceBufferPrivateAVFObjC. This value should be
reset and updated when new samples are appended.

  • Modules/mediasource/SourceBuffer.cpp:

(WebCore::SourceBuffer::sourceBufferPrivateDidReceiveSample):
(WebCore::SourceBuffer::provideMediaData):
(WebCore::SourceBuffer::updateMinimumUpcomingPresentationTime):
(WebCore::SourceBuffer::resetMinimumUpcomingPresentationTime):
(WebCore::SourceBuffer::minimumUpcomingPresentationTimeForTrackID):
(WebCore::SourceBuffer::setMaximumQueueDepthForTrackID):

  • Modules/mediasource/SourceBuffer.h:
  • platform/graphics/SourceBufferPrivate.h:

(WebCore::SourceBufferPrivate::canSetMinimumUpcomingPresentationTime const):
(WebCore::SourceBufferPrivate::setMinimumUpcomingPresentationTime):
(WebCore::SourceBufferPrivate::clearMinimumUpcomingPresentationTime):
(WebCore::SourceBufferPrivate::enqueuedSamplesForTrackID):
(WebCore::SourceBufferPrivate::minimumUpcomingPresentationTimeForTrackID):
(WebCore::SourceBufferPrivate::setMaximumQueueDepthForTrackID):

  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:

(WebCore::SourceBufferPrivateAVFObjC::canSetMinimumUpcomingPresentationTime const):
(WebCore::SourceBufferPrivateAVFObjC::setMinimumUpcomingPresentationTime):
(WebCore::SourceBufferPrivateAVFObjC::clearMinimumUpcomingPresentationTime):

  • platform/mock/mediasource/MockSourceBufferPrivate.cpp:

(WebCore::MockSourceBufferPrivate::minimumUpcomingPresentationTimeForTrackID):
(WebCore::MockSourceBufferPrivate::setMaximumQueueDepthForTrackID):
(WebCore::MockSourceBufferPrivate::canSetMinimumUpcomingPresentationTime const):
(WebCore::MockSourceBufferPrivate::setMinimumUpcomingPresentationTime):
(WebCore::MockSourceBufferPrivate::clearMinimumUpcomingPresentationTime):

  • platform/mock/mediasource/MockSourceBufferPrivate.h:
  • testing/Internals.cpp:

(WebCore::Internals::minimumUpcomingPresentationTimeForTrackID):
(WebCore::Internals::setMaximumQueueDepthForTrackID):

  • testing/Internals.h:
  • testing/Internals.idl:

LayoutTests:

  • media/media-source/media-source-minimumupcomingpresentationtime-expected.txt: Added.
  • media/media-source/media-source-minimumupcomingpresentationtime.html: Added.
Location:
trunk
Files:
2 added
12 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r248300 r248301  
     12019-08-06  Jer Noble  <jer.noble@apple.com>
     2
     3        Adopt -expectMinimumUpcomingSampleBufferPresentationTime:
     4        https://bugs.webkit.org/show_bug.cgi?id=200457
     5        <rdar://problem/53961130>
     6
     7        Reviewed by Eric Carlson.
     8
     9        * media/media-source/media-source-minimumupcomingpresentationtime-expected.txt: Added.
     10        * media/media-source/media-source-minimumupcomingpresentationtime.html: Added.
     11
    1122019-08-06  Russell Epstein  <repstein@apple.com>
    213

  • trunk/Source/WebCore/ChangeLog

    r248297 r248301  
     12019-08-06  Jer Noble  <jer.noble@apple.com>
     2
     3        Adopt -expectMinimumUpcomingSampleBufferPresentationTime:
     4        https://bugs.webkit.org/show_bug.cgi?id=200457
     5        <rdar://problem/53961130>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Test: media/media-source/media-source-minimumupcomingpresentationtime.html
     10
     11        Adopt a new API vended by AVSampleBufferDisplayLayer, piped from SourceBuffer down
     12        through SourceBufferPrivate to SourceBufferPrivateAVFObjC. This value should be
     13        reset and updated when new samples are appended.
     14
     15        * Modules/mediasource/SourceBuffer.cpp:
     16        (WebCore::SourceBuffer::sourceBufferPrivateDidReceiveSample):
     17        (WebCore::SourceBuffer::provideMediaData):
     18        (WebCore::SourceBuffer::updateMinimumUpcomingPresentationTime):
     19        (WebCore::SourceBuffer::resetMinimumUpcomingPresentationTime):
     20        (WebCore::SourceBuffer::minimumUpcomingPresentationTimeForTrackID):
     21        (WebCore::SourceBuffer::setMaximumQueueDepthForTrackID):
     22        * Modules/mediasource/SourceBuffer.h:
     23        * platform/graphics/SourceBufferPrivate.h:
     24        (WebCore::SourceBufferPrivate::canSetMinimumUpcomingPresentationTime const):
     25        (WebCore::SourceBufferPrivate::setMinimumUpcomingPresentationTime):
     26        (WebCore::SourceBufferPrivate::clearMinimumUpcomingPresentationTime):
     27        (WebCore::SourceBufferPrivate::enqueuedSamplesForTrackID):
     28        (WebCore::SourceBufferPrivate::minimumUpcomingPresentationTimeForTrackID):
     29        (WebCore::SourceBufferPrivate::setMaximumQueueDepthForTrackID):
     30        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
     31        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
     32        (WebCore::SourceBufferPrivateAVFObjC::canSetMinimumUpcomingPresentationTime const):
     33        (WebCore::SourceBufferPrivateAVFObjC::setMinimumUpcomingPresentationTime):
     34        (WebCore::SourceBufferPrivateAVFObjC::clearMinimumUpcomingPresentationTime):
     35        * platform/mock/mediasource/MockSourceBufferPrivate.cpp:
     36        (WebCore::MockSourceBufferPrivate::minimumUpcomingPresentationTimeForTrackID):
     37        (WebCore::MockSourceBufferPrivate::setMaximumQueueDepthForTrackID):
     38        (WebCore::MockSourceBufferPrivate::canSetMinimumUpcomingPresentationTime const):
     39        (WebCore::MockSourceBufferPrivate::setMinimumUpcomingPresentationTime):
     40        (WebCore::MockSourceBufferPrivate::clearMinimumUpcomingPresentationTime):
     41        * platform/mock/mediasource/MockSourceBufferPrivate.h:
     42        * testing/Internals.cpp:
     43        (WebCore::Internals::minimumUpcomingPresentationTimeForTrackID):
     44        (WebCore::Internals::setMaximumQueueDepthForTrackID):
     45        * testing/Internals.h:
     46        * testing/Internals.idl:
     47
    1482019-08-06  Antti Koivisto  <antti@apple.com>
    249

  • trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp

    r247783 r248301  
    7171    MediaTime highestPresentationTimestamp;
    7272    MediaTime lastEnqueuedPresentationTime;
     73    MediaTime minimumEnqueuedPresentationTime;
    7374    DecodeOrderSampleMap::KeyType lastEnqueuedDecodeKey;
    7475    MediaTime lastEnqueuedDecodeDuration;
    … …  
    7879    bool enabled { false };
    7980    bool needsReenqueueing { false };
     81    bool needsMinimumUpcomingPresentationTimeUpdating { false };
    8082    SampleMap samples;
    8183    DecodeOrderSampleMap::MapType decodeQueue;
    … …  
    17891791        if (trackBuffer.lastEnqueuedDecodeKey.first.isInvalid() || decodeKey > trackBuffer.lastEnqueuedDecodeKey) {
    17901792            trackBuffer.decodeQueue.insert(DecodeOrderSampleMap::MapType::value_type(decodeKey, &sample));
     1793
     1794            if (trackBuffer.minimumEnqueuedPresentationTime.isValid() && sample.presentationTime() < trackBuffer.minimumEnqueuedPresentationTime)
     1795                trackBuffer.needsMinimumUpcomingPresentationTimeUpdating = true;
    17911796        }
    17921797
    … …  
    20222027#endif
    20232028
     2029    if (trackBuffer.needsMinimumUpcomingPresentationTimeUpdating)
     2030        resetMinimumUpcomingPresentationTime(trackBuffer, trackID);
     2031
    20242032    while (!trackBuffer.decodeQueue.empty()) {
    20252033        if (!m_private->isReadyForMoreSamples(trackID)) {
    … …  
    20622070    }
    20632071
     2072    updateMinimumUpcomingPresentationTime(trackBuffer, trackID);
     2073
    20642074#if !RELEASE_LOG_DISABLED
    20652075    DEBUG_LOG(LOGIDENTIFIER, "enqueued ", enqueuedSamples, " samples, ", static_cast<size_t>(trackBuffer.decodeQueue.size()), " remaining");
    … …  
    20672077
    20682078    trySignalAllSamplesInTrackEnqueued(trackID);
     2079}
     2080
     2081void SourceBuffer::updateMinimumUpcomingPresentationTime(TrackBuffer& trackBuffer, const AtomString& trackID)
     2082{
     2083    if (!m_private->canSetMinimumUpcomingPresentationTime(trackID))
     2084        return;
     2085
     2086    if (trackBuffer.decodeQueue.empty()) {
     2087        trackBuffer.minimumEnqueuedPresentationTime = MediaTime::invalidTime();
     2088        m_private->clearMinimumUpcomingPresentationTime(trackID);
     2089        return;
     2090    }
     2091
     2092    auto minPts = std::min_element(trackBuffer.decodeQueue.begin(), trackBuffer.decodeQueue.end(), [](auto& left, auto& right) -> bool {
     2093        return left.second->outputPresentationTime() < right.second->outputPresentationTime();
     2094    });
     2095
     2096    if (minPts == trackBuffer.decodeQueue.end()) {
     2097        trackBuffer.minimumEnqueuedPresentationTime = MediaTime::invalidTime();
     2098        m_private->clearMinimumUpcomingPresentationTime(trackID);
     2099        return;
     2100    }
     2101
     2102    trackBuffer.minimumEnqueuedPresentationTime = minPts->second->outputPresentationTime();
     2103    m_private->setMinimumUpcomingPresentationTime(trackID, trackBuffer.minimumEnqueuedPresentationTime);
     2104}
     2105
     2106
     2107void SourceBuffer::resetMinimumUpcomingPresentationTime(TrackBuffer& trackBuffer, const AtomString& trackID)
     2108{
     2109    if (!m_private->canSetMinimumUpcomingPresentationTime(trackID))
     2110        return;
     2111
     2112    trackBuffer.minimumEnqueuedPresentationTime = MediaTime::invalidTime();
     2113    m_private->clearMinimumUpcomingPresentationTime(trackID);
    20692114}
    20702115
    … …  
    22722317}
    22732318
     2319MediaTime SourceBuffer::minimumUpcomingPresentationTimeForTrackID(const AtomString& trackID)
     2320{
     2321    return m_private->minimumUpcomingPresentationTimeForTrackID(trackID);
     2322}
     2323
     2324void SourceBuffer::setMaximumQueueDepthForTrackID(const AtomString& trackID, size_t maxQueueDepth)
     2325{
     2326    m_private->setMaximumQueueDepthForTrackID(trackID, maxQueueDepth);
     2327}
     2328
    22742329Document& SourceBuffer::document() const
    22752330{

  • trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h

    r246490 r248301  
    199199
    200200    void updateBufferedFromTrackBuffers();
     201    void updateMinimumUpcomingPresentationTime(TrackBuffer&, const AtomString& trackID);
     202    void resetMinimumUpcomingPresentationTime(TrackBuffer&, const AtomString& trackID);
    201203
    202204    void appendError(bool);
    … …  
    211213    WEBCORE_EXPORT Vector<String> bufferedSamplesForTrackID(const AtomString&);
    212214    WEBCORE_EXPORT Vector<String> enqueuedSamplesForTrackID(const AtomString&);
     215    WEBCORE_EXPORT MediaTime minimumUpcomingPresentationTimeForTrackID(const AtomString&);
     216    WEBCORE_EXPORT void setMaximumQueueDepthForTrackID(const AtomString&, size_t);
    213217
    214218    Ref<SourceBufferPrivate> m_private;

  • trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h

    r246490 r248301  
    6363    virtual void notifyClientWhenReadyForMoreSamples(const AtomString&) { }
    6464
     65    virtual bool canSetMinimumUpcomingPresentationTime(const AtomString&) const { return false; }
     66    virtual void setMinimumUpcomingPresentationTime(const AtomString&, const MediaTime&) { }
     67    virtual void clearMinimumUpcomingPresentationTime(const AtomString&) { }
     68    virtual bool canSwitchToType(const ContentType&) { return false; }
     69
     70    // Internals Utility methods:
    6571    virtual Vector<String> enqueuedSamplesForTrackID(const AtomString&) { return { }; }
    66 
    67     virtual bool canSwitchToType(const ContentType&) { return false; }
     72    virtual MediaTime minimumUpcomingPresentationTimeForTrackID(const AtomString&) { return MediaTime::invalidTime(); }
     73    virtual void setMaximumQueueDepthForTrackID(const AtomString&, size_t) { }
    6874
    6975#if !RELEASE_LOG_DISABLED

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h

    r246490 r248301  
    164164    void setActive(bool) final;
    165165    void notifyClientWhenReadyForMoreSamples(const AtomString& trackID) final;
     166    bool canSetMinimumUpcomingPresentationTime(const AtomString&) const override;
     167    void setMinimumUpcomingPresentationTime(const AtomString&, const MediaTime&) override;
     168    void clearMinimumUpcomingPresentationTime(const AtomString&) override;
    166169    bool canSwitchToType(const ContentType&) final;
    167170

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm

    r248190 r248301  
    7070@interface AVSampleBufferDisplayLayer (WebCoreAVSampleBufferDisplayLayerQueueManagementPrivate)
    7171- (void)prerollDecodeWithCompletionHandler:(void (^)(BOOL success))block;
     72- (void)expectMinimumUpcomingSampleBufferPresentationTime: (CMTime)minimumUpcomingPresentationTime;
     73- (void)resetUpcomingSampleBufferPresentationTimeExpectations;
    7274@end
    7375
    … …  
    12591261}
    12601262
     1263bool SourceBufferPrivateAVFObjC::canSetMinimumUpcomingPresentationTime(const AtomString& trackIDString) const
     1264{
     1265    int trackID = trackIDString.toInt();
     1266    if (trackID == m_enabledVideoTrackID)
     1267        return [getAVSampleBufferDisplayLayerClass() instancesRespondToSelector:@selector(expectMinimumUpcomingSampleBufferPresentationTime:)];
     1268    return false;
     1269}
     1270
     1271void SourceBufferPrivateAVFObjC::setMinimumUpcomingPresentationTime(const AtomString& trackIDString, const MediaTime& presentationTime)
     1272{
     1273    ASSERT(canSetMinimumUpcomingPresentationTime(trackIDString));
     1274    if (canSetMinimumUpcomingPresentationTime(trackIDString))
     1275        [m_displayLayer expectMinimumUpcomingSampleBufferPresentationTime:toCMTime(presentationTime)];
     1276}
     1277
     1278void SourceBufferPrivateAVFObjC::clearMinimumUpcomingPresentationTime(const AtomString& trackIDString)
     1279{
     1280    ASSERT(canSetMinimumUpcomingPresentationTime(trackIDString));
     1281    if (canSetMinimumUpcomingPresentationTime(trackIDString))
     1282        [m_displayLayer resetUpcomingSampleBufferPresentationTimeExpectations];
     1283}
     1284
    12611285bool SourceBufferPrivateAVFObjC::canSwitchToType(const ContentType& contentType)
    12621286{

  • trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp

    r246490 r248301  
    241241}
    242242
     243MediaTime MockSourceBufferPrivate::minimumUpcomingPresentationTimeForTrackID(const AtomString&)
     244{
     245    return m_minimumUpcomingPresentationTime;
     246}
     247
     248void MockSourceBufferPrivate::setMaximumQueueDepthForTrackID(const AtomString&, size_t maxQueueDepth)
     249{
     250    m_maxQueueDepth = maxQueueDepth;
     251}
     252
     253bool MockSourceBufferPrivate::canSetMinimumUpcomingPresentationTime(const AtomString&) const
     254{
     255    return true;
     256}
     257
     258void MockSourceBufferPrivate::setMinimumUpcomingPresentationTime(const AtomString&, const MediaTime& presentationTime)
     259{
     260    m_minimumUpcomingPresentationTime = presentationTime;
     261}
     262
     263void MockSourceBufferPrivate::clearMinimumUpcomingPresentationTime(const AtomString&)
     264{
     265    m_minimumUpcomingPresentationTime = MediaTime::invalidTime();
     266}
     267
    243268bool MockSourceBufferPrivate::canSwitchToType(const ContentType& contentType)
    244269{

  • trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h

    r246490 r248301  
    6363    MediaPlayer::ReadyState readyState() const final;
    6464    void setReadyState(MediaPlayer::ReadyState) final;
     65    bool canSetMinimumUpcomingPresentationTime(const AtomString&) const final;
     66    void setMinimumUpcomingPresentationTime(const AtomString&, const MediaTime&) final;
     67    void clearMinimumUpcomingPresentationTime(const AtomString&) final;
    6568    bool canSwitchToType(const ContentType&) final;
    6669
    67     void flush(const AtomString&) final { m_enqueuedSamples.clear(); }
     70    void flush(const AtomString&) final { m_enqueuedSamples.clear(); m_minimumUpcomingPresentationTime = MediaTime::invalidTime(); }
    6871    void enqueueSample(Ref<MediaSample>&&, const AtomString&) final;
    69     bool isReadyForMoreSamples(const AtomString&) final { return true; }
     72    bool isReadyForMoreSamples(const AtomString&) final { return !m_maxQueueDepth || m_enqueuedSamples.size() < m_maxQueueDepth.value(); }
    7073    void setActive(bool) final;
    7174
    7275    Vector<String> enqueuedSamplesForTrackID(const AtomString&) final;
     76    MediaTime minimumUpcomingPresentationTimeForTrackID(const AtomString&) final;
     77    void setMaximumQueueDepthForTrackID(const AtomString&, size_t) final;
    7378
    7479    void didReceiveInitializationSegment(const MockInitializationBox&);
    … …  
    8287    MockMediaSourcePrivate* m_mediaSource;
    8388    SourceBufferPrivateClient* m_client;
     89    MediaTime m_minimumUpcomingPresentationTime;
    8490    Vector<String> m_enqueuedSamples;
    85 
     91    Optional<size_t> m_maxQueueDepth;
    8692    Vector<char> m_inputBuffer;
    8793};

  • trunk/Source/WebCore/testing/Internals.cpp

    r248046 r248301  
    37513751}
    37523752
     3753double Internals::minimumUpcomingPresentationTimeForTrackID(SourceBuffer& buffer, const AtomString& trackID)
     3754{
     3755    return buffer.minimumUpcomingPresentationTimeForTrackID(trackID).toDouble();
     3756}
     3757
    37533758void Internals::setShouldGenerateTimestamps(SourceBuffer& buffer, bool flag)
    37543759{
    37553760    buffer.setShouldGenerateTimestamps(flag);
     3761}
     3762
     3763void Internals::setMaximumQueueDepthForTrackID(SourceBuffer& buffer, const AtomString& trackID, size_t maxQueueDepth)
     3764{
     3765    buffer.setMaximumQueueDepthForTrackID(trackID, maxQueueDepth);
    37563766}
    37573767

  • trunk/Source/WebCore/testing/Internals.h

    r248046 r248301  
    584584    Vector<String> bufferedSamplesForTrackID(SourceBuffer&, const AtomString&);
    585585    Vector<String> enqueuedSamplesForTrackID(SourceBuffer&, const AtomString&);
     586    double minimumUpcomingPresentationTimeForTrackID(SourceBuffer&, const AtomString&);
    586587    void setShouldGenerateTimestamps(SourceBuffer&, bool);
     588    void setMaximumQueueDepthForTrackID(SourceBuffer&, const AtomString&, size_t);
    587589#endif
    588590

  • trunk/Source/WebCore/testing/Internals.idl

    r248046 r248301  
    586586    [Conditional=MEDIA_SOURCE] sequence<DOMString> enqueuedSamplesForTrackID(SourceBuffer buffer, DOMString trackID);
    587587    [Conditional=MEDIA_SOURCE] void setShouldGenerateTimestamps(SourceBuffer buffer, boolean flag);
     588    [Conditional=MEDIA_SOURCE] double minimumUpcomingPresentationTimeForTrackID(SourceBuffer buffer, DOMString trackID);
     589    [Conditional=MEDIA_SOURCE] void setMaximumQueueDepthForTrackID(SourceBuffer buffer, DOMString trackID, unsigned long maxQueueDepth);
    588590
    589591    [Conditional=VIDEO, MayThrowException] void beginMediaSessionInterruption(DOMString interruptionType);
Note: See TracChangeset for help on using the changeset viewer.