Changeset 217098 in webkit


Ignore:
Timestamp:
May 18, 2017 10:50:35 PM (7 years ago)
Author:
jer.noble@apple.com
Message:

[MSE][Mac] Support painting MSE video-element to canvas
​https://bugs.webkit.org/show_bug.cgi?id=125157
<rdar://problem/23062016>

Reviewed by Eric Carlson.

Test: media/media-source/media-source-paint-to-canvas.html

In order to have access to decoded video data for painting, decode the encoded samples manually
instead of adding them to the AVSampleBufferDisplayLayer. To facilitate doing so, add a new
utility class WebCoreDecompressionSession, which can decode samples and store them.

For the purposes of this patch, to avoid double-decoding of video data and to avoid severe complication
of our sample delivery pipeline, we will only support painting of decoded video samples when the video is
not displayed in the DOM.

  • Modules/mediasource/MediaSource.cpp:

(WebCore::MediaSource::seekToTime): Always send waitForSeekCompleted() to give private a chance to delay seek completion.

  • Modules/mediasource/SourceBuffer.cpp:

(WebCore::SourceBuffer::sourceBufferPrivateReenqueSamples): Added.

  • Modules/mediasource/SourceBuffer.h:
  • WebCore.xcodeproj/project.pbxproj:
  • platform/cf/CoreMediaSoftLink.cpp: Added new soft link macros.
  • platform/cf/CoreMediaSoftLink.h: Ditto.
  • platform/cocoa/CoreVideoSoftLink.cpp: Ditto.
  • platform/cocoa/CoreVideoSoftLink.h: Ditto.
  • platform/graphics/SourceBufferPrivateClient.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:

(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::sampleBufferDisplayLayer): Simple accessor.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::decompressionSession): Ditto.

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:

(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::MediaPlayerPrivateMediaSourceAVFObjC):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load): Update whether we should be displaying in a layer or decompression session..
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVisible): Ditto.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::waitForSeekCompleted): m_seeking is now an enum.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seeking): Ditto.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted): Ditto. If waiting for a video frame, delay completing seek.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::nativeImageForCurrentTime): Call updateLastImage() and return result.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastImage): Fetch the image for the current time.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paint): Pass to paintCurrentFrameInCanvas.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paintCurrentFrameInContext): Get a native image, and render it.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged): Create or destroy a layer or decompression session as appropriate.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer): Creates a layer.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer): Destroys a layer.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession): Creates a decompression session.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession): Destroys a decompression session.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setHasAvailableVideoFrame): If seek completion delayed, complete now. Ditto for ready state change.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setReadyState): If waiting for a video frame, delay ready state change.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::addDisplayLayer): Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::removeDisplayLayer): Deleted.

  • platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
  • platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:

(WebCore::MediaSourcePrivateAVFObjC::hasVideo): Promote to a class function.
(WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideo): Return whether any of the active source buffers have video and are selected.
(WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideoChanged): Call setSourceBufferWithSelectedVideo().
(WebCore::MediaSourcePrivateAVFObjC::setVideoLayer): Set (or clear) the layer on the selected buffer.
(WebCore::MediaSourcePrivateAVFObjC::setDecompressionSession): Ditto for decompression session.
(WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo): Remove the layer and decompression session from the unselected

buffer and add the decompression session or layer to the newly selected buffer.

(WebCore::MediaSourcePrivateAVFObjCHasVideo): Deleted.

  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:

(WebCore::SourceBufferPrivateAVFObjC::destroyRenderers): Clear the videoLayer and decompressionSession.
(WebCore::SourceBufferPrivateAVFObjC::hasSelectedVideo): Return whether the buffer has a selected video track.
(WebCore::SourceBufferPrivateAVFObjC::trackDidChangeEnabled): The media player now manages the video layer and decompression session lifetimes.
(WebCore::SourceBufferPrivateAVFObjC::flush): Flush the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::enqueueSample): Enqueue to the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::isReadyForMoreSamples): As the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples): Tell the decompression session to stop requesting data, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::notifyClientWhenReadyForMoreSamples): Request media data from the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::setVideoLayer): Added.
(WebCore::SourceBufferPrivateAVFObjC::setDecompressionSession): Added.

  • platform/graphics/cocoa/WebCoreDecompressionSession.h: Added.

(WebCore::WebCoreDecompressionSession::create):
(WebCore::WebCoreDecompressionSession::isInvalidated):
(WebCore::WebCoreDecompressionSession::createWeakPtr):

  • platform/graphics/cocoa/WebCoreDecompressionSession.mm: Added.

(WebCore::WebCoreDecompressionSession::WebCoreDecompressionSession): Register for media data requests.
(WebCore::WebCoreDecompressionSession::invalidate): Unregister for same.
(WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaDataCallback): Pass to maybeBecomeReadyForMoreMediaData.
(WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaData): Check in-flight decodes, and decoded frame counts.
(WebCore::WebCoreDecompressionSession::enqueueSample): Pass the sample to be decoded on a background queue.
(WebCore::WebCoreDecompressionSession::decodeSample): Decode the sample.
(WebCore::WebCoreDecompressionSession::decompressionOutputCallback): Call handleDecompressionOutput.
(WebCore::WebCoreDecompressionSession::handleDecompressionOutput): Pass decoded sample to be enqueued on the main thread.
(WebCore::WebCoreDecompressionSession::getFirstVideoFrame):
(WebCore::WebCoreDecompressionSession::enqueueDecodedSample): Enqueue the frame (if it's a displayed frame).
(WebCore::WebCoreDecompressionSession::isReadyForMoreMediaData): Return whether we've hit our high water sample count.
(WebCore::WebCoreDecompressionSession::requestMediaDataWhenReady):
(WebCore::WebCoreDecompressionSession::stopRequestingMediaData): Unset the same.
(WebCore::WebCoreDecompressionSession::notifyWhenHasAvailableVideoFrame): Set a callback to notify when a decoded frame has been enqueued.
(WebCore::WebCoreDecompressionSession::imageForTime): Successively dequeue images until reaching one at or beyond the requested time.
(WebCore::WebCoreDecompressionSession::flush): Synchronously empty the producer and consumer queues.
(WebCore::WebCoreDecompressionSession::getDecodeTime): Utility method.
(WebCore::WebCoreDecompressionSession::getPresentationTime): Ditto.
(WebCore::WebCoreDecompressionSession::getDuration): Ditto.
(WebCore::WebCoreDecompressionSession::compareBuffers): Ditto.

  • platform/cocoa/VideoToolboxSoftLink.cpp: Added.
  • platform/cocoa/VideoToolboxSoftLink.h: Added.
Location:
trunk/Source/WebCore
Files:
4 added
16 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r217096 r217098  
     12017-05-18  Jer Noble  <jer.noble@apple.com>
     2
     3        [MSE][Mac] Support painting MSE video-element to canvas
     4        https://bugs.webkit.org/show_bug.cgi?id=125157
     5        <rdar://problem/23062016>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Test: media/media-source/media-source-paint-to-canvas.html
     10
     11        In order to have access to decoded video data for painting, decode the encoded samples manually
     12        instead of adding them to the AVSampleBufferDisplayLayer. To facilitate doing so, add a new
     13        utility class WebCoreDecompressionSession, which can decode samples and store them.
     14
     15        For the purposes of this patch, to avoid double-decoding of video data and to avoid severe complication
     16        of our sample delivery pipeline, we will only support painting of decoded video samples when the video is
     17        not displayed in the DOM.
     18
     19        * Modules/mediasource/MediaSource.cpp:
     20        (WebCore::MediaSource::seekToTime): Always send waitForSeekCompleted() to give private a chance to delay seek completion.
     21        * Modules/mediasource/SourceBuffer.cpp:
     22        (WebCore::SourceBuffer::sourceBufferPrivateReenqueSamples): Added.
     23        * Modules/mediasource/SourceBuffer.h:
     24        * WebCore.xcodeproj/project.pbxproj:
     25        * platform/cf/CoreMediaSoftLink.cpp: Added new soft link macros.
     26        * platform/cf/CoreMediaSoftLink.h: Ditto.
     27        * platform/cocoa/CoreVideoSoftLink.cpp: Ditto.
     28        * platform/cocoa/CoreVideoSoftLink.h: Ditto.
     29        * platform/graphics/SourceBufferPrivateClient.h:
     30        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
     31        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::sampleBufferDisplayLayer): Simple accessor.
     32        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::decompressionSession): Ditto.
     33        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
     34        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::MediaPlayerPrivateMediaSourceAVFObjC):
     35        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load): Update whether we should be displaying in a layer or decompression session..
     36        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVisible): Ditto.
     37        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::waitForSeekCompleted): m_seeking is now an enum.
     38        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seeking): Ditto.
     39        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted): Ditto. If waiting for a video frame, delay completing seek.
     40        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::nativeImageForCurrentTime): Call updateLastImage() and return result.
     41        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastImage): Fetch the image for the current time.
     42        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paint): Pass to paintCurrentFrameInCanvas.
     43        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paintCurrentFrameInContext): Get a native image, and render it.
     44        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged): Create or destroy a layer or decompression session as appropriate.
     45        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer): Creates a layer.
     46        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer): Destroys a layer.
     47        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession): Creates a decompression session.
     48        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession): Destroys a decompression session.
     49        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setHasAvailableVideoFrame): If seek completion delayed, complete now. Ditto for ready state change.
     50        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setReadyState): If waiting for a video frame, delay ready state change.
     51        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::addDisplayLayer): Deleted.
     52        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::removeDisplayLayer): Deleted.
     53        * platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
     54        * platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
     55        (WebCore::MediaSourcePrivateAVFObjC::hasVideo): Promote to a class function.
     56        (WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideo): Return whether any of the active source buffers have video and are selected.
     57        (WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideoChanged): Call setSourceBufferWithSelectedVideo().
     58        (WebCore::MediaSourcePrivateAVFObjC::setVideoLayer): Set (or clear) the layer on the selected buffer.
     59        (WebCore::MediaSourcePrivateAVFObjC::setDecompressionSession): Ditto for decompression session.
     60        (WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo): Remove the layer and decompression session from the unselected
     61
     62                buffer and add the decompression session or layer to the newly selected buffer.
     63        (WebCore::MediaSourcePrivateAVFObjCHasVideo): Deleted.
     64        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
     65        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
     66        (WebCore::SourceBufferPrivateAVFObjC::destroyRenderers): Clear the videoLayer and decompressionSession.
     67        (WebCore::SourceBufferPrivateAVFObjC::hasSelectedVideo): Return whether the buffer has a selected video track.
     68        (WebCore::SourceBufferPrivateAVFObjC::trackDidChangeEnabled): The media player now manages the video layer and decompression session lifetimes.
     69        (WebCore::SourceBufferPrivateAVFObjC::flush): Flush the decompression session, if it exists.
     70        (WebCore::SourceBufferPrivateAVFObjC::enqueueSample): Enqueue to the decompression session, if it exists.
     71        (WebCore::SourceBufferPrivateAVFObjC::isReadyForMoreSamples): As the decompression session, if it exists.
     72        (WebCore::SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples): Tell the decompression session to stop requesting data, if it exists.
     73        (WebCore::SourceBufferPrivateAVFObjC::notifyClientWhenReadyForMoreSamples): Request media data from the decompression session, if it exists.
     74        (WebCore::SourceBufferPrivateAVFObjC::setVideoLayer): Added.
     75        (WebCore::SourceBufferPrivateAVFObjC::setDecompressionSession): Added.
     76        * platform/graphics/cocoa/WebCoreDecompressionSession.h: Added.
     77        (WebCore::WebCoreDecompressionSession::create):
     78        (WebCore::WebCoreDecompressionSession::isInvalidated):
     79        (WebCore::WebCoreDecompressionSession::createWeakPtr):
     80        * platform/graphics/cocoa/WebCoreDecompressionSession.mm: Added.
     81        (WebCore::WebCoreDecompressionSession::WebCoreDecompressionSession): Register for media data requests.
     82        (WebCore::WebCoreDecompressionSession::invalidate):  Unregister for same.
     83        (WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaDataCallback): Pass to maybeBecomeReadyForMoreMediaData.
     84        (WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaData): Check in-flight decodes, and decoded frame counts.
     85        (WebCore::WebCoreDecompressionSession::enqueueSample): Pass the sample to be decoded on a background queue.
     86        (WebCore::WebCoreDecompressionSession::decodeSample): Decode the sample.
     87        (WebCore::WebCoreDecompressionSession::decompressionOutputCallback): Call handleDecompressionOutput.
     88        (WebCore::WebCoreDecompressionSession::handleDecompressionOutput): Pass decoded sample to be enqueued on the main thread.
     89        (WebCore::WebCoreDecompressionSession::getFirstVideoFrame):
     90        (WebCore::WebCoreDecompressionSession::enqueueDecodedSample): Enqueue the frame (if it's a displayed frame).
     91        (WebCore::WebCoreDecompressionSession::isReadyForMoreMediaData): Return whether we've hit our high water sample count.
     92        (WebCore::WebCoreDecompressionSession::requestMediaDataWhenReady):
     93        (WebCore::WebCoreDecompressionSession::stopRequestingMediaData): Unset the same.
     94        (WebCore::WebCoreDecompressionSession::notifyWhenHasAvailableVideoFrame): Set a callback to notify when a decoded frame has been enqueued.
     95        (WebCore::WebCoreDecompressionSession::imageForTime): Successively dequeue images until reaching one at or beyond the requested time.
     96        (WebCore::WebCoreDecompressionSession::flush): Synchronously empty the producer and consumer queues.
     97        (WebCore::WebCoreDecompressionSession::getDecodeTime): Utility method.
     98        (WebCore::WebCoreDecompressionSession::getPresentationTime): Ditto.
     99        (WebCore::WebCoreDecompressionSession::getDuration): Ditto.
     100        (WebCore::WebCoreDecompressionSession::compareBuffers): Ditto.
     101        * platform/cocoa/VideoToolboxSoftLink.cpp: Added.
     102        * platform/cocoa/VideoToolboxSoftLink.h: Added.
     103
    11042017-05-18  Said Abou-Hallawa  <sabouhallawa@apple.com>
    2105

  • trunk/Source/WebCore/Modules/mediasource/MediaSource.cpp

    r216860 r217098  
    234234    // Continue
    235235
     236    m_private->waitForSeekCompleted();
    236237    completeSeek();
    237238}

  • trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp

    r215160 r217098  
    17791779}
    17801780
     1781void SourceBuffer::sourceBufferPrivateReenqueSamples(const AtomicString& trackID)
     1782{
     1783    if (isRemoved())
     1784        return;
     1785
     1786    LOG(MediaSource, "SourceBuffer::sourceBufferPrivateReenqueSamples(%p)", this);
     1787    auto it = m_trackBufferMap.find(trackID);
     1788    if (it == m_trackBufferMap.end())
     1789        return;
     1790
     1791    auto& trackBuffer = it->value;
     1792    trackBuffer.needsReenqueueing = true;
     1793    reenqueueMediaForTime(trackBuffer, trackID, m_source->currentTime());
     1794}
     1795
    17811796void SourceBuffer::sourceBufferPrivateDidBecomeReadyForMoreSamples(const AtomicString& trackID)
    17821797{
     1798    if (isRemoved())
     1799        return;
     1800
    17831801    LOG(MediaSource, "SourceBuffer::sourceBufferPrivateDidBecomeReadyForMoreSamples(%p)", this);
    17841802    auto it = m_trackBufferMap.find(trackID);

  • trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h

    r210319 r217098  
    130130    bool sourceBufferPrivateHasAudio() const final;
    131131    bool sourceBufferPrivateHasVideo() const final;
     132    void sourceBufferPrivateReenqueSamples(const AtomicString& trackID) final;
    132133    void sourceBufferPrivateDidBecomeReadyForMoreSamples(const AtomicString& trackID) final;
    133134    MediaTime sourceBufferPrivateFastSeekTimeForMediaTime(const MediaTime&, const MediaTime& negativeThreshold, const MediaTime& positiveThreshold) final;

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r217049 r217098  
    61296129                CD5896E11CD2B15100B3BCC8 /* WebPlaybackControlsManager.mm in Sources */ = {isa = PBXBuildFile; fileRef = CD5896DF1CD2B15100B3BCC8 /* WebPlaybackControlsManager.mm */; };
    61306130                CD5896E21CD2B15100B3BCC8 /* WebPlaybackControlsManager.h in Headers */ = {isa = PBXBuildFile; fileRef = CD5896E01CD2B15100B3BCC8 /* WebPlaybackControlsManager.h */; settings = {ATTRIBUTES = (Private, ); }; };
     6131                CD5D27771E8318E000D80A3D /* WebCoreDecompressionSession.mm in Sources */ = {isa = PBXBuildFile; fileRef = CD5D27751E8318E000D80A3D /* WebCoreDecompressionSession.mm */; };
     6132                CD5D27781E8318E000D80A3D /* WebCoreDecompressionSession.h in Headers */ = {isa = PBXBuildFile; fileRef = CD5D27761E8318E000D80A3D /* WebCoreDecompressionSession.h */; };
    61316133                CD5E5B5F1A15CE54000C609E /* PageConfiguration.h in Headers */ = {isa = PBXBuildFile; fileRef = CD5E5B5E1A15CE54000C609E /* PageConfiguration.h */; settings = {ATTRIBUTES = (Private, ); }; };
    61326134                CD5E5B611A15F156000C609E /* PageConfiguration.cpp in Sources */ = {isa = PBXBuildFile; fileRef = CD5E5B601A15F156000C609E /* PageConfiguration.cpp */; };
    … …  
    62306232                CDC8B5AB18047FF10016E685 /* SourceBufferPrivateAVFObjC.h in Headers */ = {isa = PBXBuildFile; fileRef = CDC8B5A918047FF10016E685 /* SourceBufferPrivateAVFObjC.h */; };
    62316233                CDC8B5AD1804AE5D0016E685 /* SourceBufferPrivateClient.h in Headers */ = {isa = PBXBuildFile; fileRef = CDC8B5AC1804AE5D0016E685 /* SourceBufferPrivateClient.h */; };
     6234                CDC939A71E9BDFB100BB768D /* VideoToolboxSoftLink.cpp in Sources */ = {isa = PBXBuildFile; fileRef = CDC939A51E9BDFB100BB768D /* VideoToolboxSoftLink.cpp */; };
     6235                CDC939A81E9BDFB100BB768D /* VideoToolboxSoftLink.h in Headers */ = {isa = PBXBuildFile; fileRef = CDC939A61E9BDFB100BB768D /* VideoToolboxSoftLink.h */; };
    62326236                CDC979F41C498C0900DB50D4 /* WebCoreNSErrorExtras.mm in Sources */ = {isa = PBXBuildFile; fileRef = CDC979F21C498C0900DB50D4 /* WebCoreNSErrorExtras.mm */; };
    62336237                CDC979F51C498C0900DB50D4 /* WebCoreNSErrorExtras.h in Headers */ = {isa = PBXBuildFile; fileRef = CDC979F31C498C0900DB50D4 /* WebCoreNSErrorExtras.h */; };
    … …  
    1461914623                CD5896DF1CD2B15100B3BCC8 /* WebPlaybackControlsManager.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebPlaybackControlsManager.mm; sourceTree = "<group>"; };
    1462014624                CD5896E01CD2B15100B3BCC8 /* WebPlaybackControlsManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebPlaybackControlsManager.h; sourceTree = "<group>"; };
     14625                CD5D27751E8318E000D80A3D /* WebCoreDecompressionSession.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebCoreDecompressionSession.mm; sourceTree = "<group>"; };
     14626                CD5D27761E8318E000D80A3D /* WebCoreDecompressionSession.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebCoreDecompressionSession.h; sourceTree = "<group>"; };
    1462114627                CD5E5B5E1A15CE54000C609E /* PageConfiguration.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PageConfiguration.h; sourceTree = "<group>"; };
    1462214628                CD5E5B601A15F156000C609E /* PageConfiguration.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PageConfiguration.cpp; sourceTree = "<group>"; };
    … …  
    1473914745                CDC8B5A918047FF10016E685 /* SourceBufferPrivateAVFObjC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SourceBufferPrivateAVFObjC.h; sourceTree = "<group>"; };
    1474014746                CDC8B5AC1804AE5D0016E685 /* SourceBufferPrivateClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SourceBufferPrivateClient.h; sourceTree = "<group>"; };
     14747                CDC939A51E9BDFB100BB768D /* VideoToolboxSoftLink.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = VideoToolboxSoftLink.cpp; sourceTree = "<group>"; };
     14748                CDC939A61E9BDFB100BB768D /* VideoToolboxSoftLink.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = VideoToolboxSoftLink.h; sourceTree = "<group>"; };
    1474114749                CDC979F21C498C0900DB50D4 /* WebCoreNSErrorExtras.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebCoreNSErrorExtras.mm; sourceTree = "<group>"; };
    1474214750                CDC979F31C498C0900DB50D4 /* WebCoreNSErrorExtras.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebCoreNSErrorExtras.h; sourceTree = "<group>"; };
    … …  
    2111621124                                52D5A1A51C57488900DE34A3 /* WebVideoFullscreenModelVideoElement.h */,
    2111721125                                52D5A1A61C57488900DE34A3 /* WebVideoFullscreenModelVideoElement.mm */,
     21126                                CDC939A51E9BDFB100BB768D /* VideoToolboxSoftLink.cpp */,
     21127                                CDC939A61E9BDFB100BB768D /* VideoToolboxSoftLink.h */,
    2111821128                        );
    2111921129                        path = cocoa;
    … …  
    2335723367                                2D3EF4461917915C00034184 /* WebCoreCALayerExtras.h */,
    2335823368                                2D3EF4471917915C00034184 /* WebCoreCALayerExtras.mm */,
     23369                                CD5D27751E8318E000D80A3D /* WebCoreDecompressionSession.mm */,
     23370                                CD5D27761E8318E000D80A3D /* WebCoreDecompressionSession.h */,
    2335923371                                316BDB8A1E6E153000DE0D5A /* WebGPULayer.h */,
    2336023372                                316BDB891E6E153000DE0D5A /* WebGPULayer.mm */,
    … …  
    2706527077                                E440AA961C68420800A265CC /* ElementAndTextDescendantIterator.h in Headers */,
    2706627078                                E46A2B1E17CA76B1000DBCD8 /* ElementChildIterator.h in Headers */,
     27079                                CD5D27781E8318E000D80A3D /* WebCoreDecompressionSession.h in Headers */,
    2706727080                                B5B7A17117C10AC000E4AA0A /* ElementData.h in Headers */,
    2706827081                                93D437A11D57B3F400AB85EA /* ElementDescendantIterator.h in Headers */,
    … …  
    2855928572                                413E00791DB0E4F2002341D2 /* MemoryRelease.h in Headers */,
    2856028573                                93309DFA099E64920056E581 /* MergeIdenticalElementsCommand.h in Headers */,
     28574                                CDC939A81E9BDFB100BB768D /* VideoToolboxSoftLink.h in Headers */,
    2856128575                                E1ADECCE0E76AD8B004A1A5E /* MessageChannel.h in Headers */,
    2856228576                                75793E840D0CE0B3007FC0AC /* MessageEvent.h in Headers */,
    … …  
    3094430958                                FD31603012B0267600C1A359 /* DelayProcessor.cpp in Sources */,
    3094530959                                93309DDE099E64920056E581 /* DeleteFromTextNodeCommand.cpp in Sources */,
     30960                                CDC939A71E9BDFB100BB768D /* VideoToolboxSoftLink.cpp in Sources */,
    3094630961                                93309DE0099E64920056E581 /* DeleteSelectionCommand.cpp in Sources */,
    3094730962                                9479493C1E045CF300018D85 /* DeprecatedCSSOMPrimitiveValue.cpp in Sources */,
    … …  
    3337533390                                B2227AB70D00BF220071B782 /* SVGStyleElement.cpp in Sources */,
    3337633391                                B2227ABA0D00BF220071B782 /* SVGSVGElement.cpp in Sources */,
     33392                                CD5D27771E8318E000D80A3D /* WebCoreDecompressionSession.mm in Sources */,
    3337733393                                B2227ABD0D00BF220071B782 /* SVGSwitchElement.cpp in Sources */,
    3337833394                                B2227AC00D00BF220071B782 /* SVGSymbolElement.cpp in Sources */,

  • trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.cpp

    r210621 r217098  
    3030#include "CoreMediaSPI.h"
    3131#include "SoftLinking.h"
     32#include <CoreVideo/CoreVideo.h>
    3233
    3334SOFT_LINK_FRAMEWORK_FOR_SOURCE(WebCore, CoreMedia)
    … …  
    4748SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimeRangeGetEnd, CMTime, (CMTimeRange range), (range))
    4849SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimeRangeMake, CMTimeRange, (CMTime start, CMTime duration), (start, duration))
     50SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueReset, OSStatus, (CMBufferQueueRef queue), (queue))
     51SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueCreate, OSStatus, (CFAllocatorRef allocator, CMItemCount capacity, const CMBufferCallbacks* callbacks, CMBufferQueueRef* queueOut), (allocator, capacity, callbacks, queueOut))
     52SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueGetHead, CMBufferRef, (CMBufferQueueRef queue), (queue))
     53SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueDequeueAndRetain, CMBufferRef, (CMBufferQueueRef queue), (queue))
     54SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueEnqueue, OSStatus, (CMBufferQueueRef queue, CMBufferRef buffer), (queue, buffer))
     55SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueIsEmpty, Boolean, (CMBufferQueueRef queue), (queue))
     56SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueGetBufferCount, CMItemCount, (CMBufferQueueRef queue), (queue))
     57SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueGetFirstPresentationTimeStamp, CMTime, (CMBufferQueueRef queue), (queue))
     58SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMBufferQueueInstallTriggerWithIntegerThreshold, OSStatus, (CMBufferQueueRef queue, CMBufferQueueTriggerCallback triggerCallback, void* triggerRefcon, CMBufferQueueTriggerCondition triggerCondition, CMItemCount triggerThreshold, CMBufferQueueTriggerToken* triggerTokenOut), (queue, triggerCallback, triggerRefcon, triggerCondition, triggerThreshold, triggerTokenOut))
    4959
    5060SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, CoreMedia, kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms, CFStringRef)
    … …  
    98108SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimebaseSetTime, OSStatus, (CMTimebaseRef timebase, CMTime time), (timebase, time))
    99109SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimebaseGetEffectiveRate, Float64, (CMTimebaseRef timebase), (timebase))
     110SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimebaseAddTimerDispatchSource, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource), (timebase, timerSource))
     111SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimebaseRemoveTimerDispatchSource, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource), (timebase, timerSource))
     112SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimebaseSetTimerDispatchSourceNextFireTime, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource, CMTime fireTime, uint32_t flags), (timebase, timerSource, fireTime, flags))
     113SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimebaseSetTimerDispatchSourceToFireImmediately, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource), (timebase, timerSource))
     114
    100115SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMTimeCopyAsDictionary, CFDictionaryRef, (CMTime time, CFAllocatorRef allocator), (time, allocator))
    101116SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMVideoFormatDescriptionCreateForImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, CMVideoFormatDescriptionRef* outDesc), (allocator, imageBuffer, outDesc))

  • trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.h

    r210621 r217098  
    6464SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMTimeRangeMake, CMTimeRange, (CMTime start, CMTime duration), (start, duration))
    6565#define CMTimeRangeMake softLink_CoreMedia_CMTimeRangeMake
     66SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueCreate, OSStatus, (CFAllocatorRef allocator, CMItemCount capacity, const CMBufferCallbacks* callbacks, CMBufferQueueRef* queueOut), (allocator, capacity, callbacks, queueOut))
     67#define CMBufferQueueCreate softLink_CoreMedia_CMBufferQueueCreate
     68SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueReset, OSStatus, (CMBufferQueueRef queue), (queue))
     69#define CMBufferQueueReset softLink_CoreMedia_CMBufferQueueReset
     70SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueGetHead, CMBufferRef, (CMBufferQueueRef queue), (queue))
     71#define CMBufferQueueGetHead softLink_CoreMedia_CMBufferQueueGetHead
     72SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueDequeueAndRetain, CMBufferRef, (CMBufferQueueRef queue), (queue))
     73#define CMBufferQueueDequeueAndRetain softLink_CoreMedia_CMBufferQueueDequeueAndRetain
     74SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueEnqueue, OSStatus, (CMBufferQueueRef queue, CMBufferRef buffer), (queue, buffer))
     75#define CMBufferQueueEnqueue softLink_CoreMedia_CMBufferQueueEnqueue
     76SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueIsEmpty, Boolean, (CMBufferQueueRef queue), (queue))
     77#define CMBufferQueueIsEmpty softLink_CoreMedia_CMBufferQueueIsEmpty
     78SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueGetBufferCount, CMItemCount, (CMBufferQueueRef queue), (queue))
     79#define CMBufferQueueGetBufferCount softLink_CoreMedia_CMBufferQueueGetBufferCount
     80SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueGetFirstPresentationTimeStamp, CMTime, (CMBufferQueueRef queue), (queue))
     81#define CMBufferQueueGetFirstPresentationTimeStamp softLink_CoreMedia_CMBufferQueueGetFirstPresentationTimeStamp
     82SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMBufferQueueInstallTriggerWithIntegerThreshold, OSStatus, (CMBufferQueueRef queue, CMBufferQueueTriggerCallback triggerCallback, void* triggerRefcon, CMBufferQueueTriggerCondition triggerCondition, CMItemCount triggerThreshold, CMBufferQueueTriggerToken* triggerTokenOut), (queue, triggerCallback, triggerRefcon, triggerCondition, triggerThreshold, triggerTokenOut))
     83#define CMBufferQueueInstallTriggerWithIntegerThreshold softLink_CoreMedia_CMBufferQueueInstallTriggerWithIntegerThreshold
    6684
    6785SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreMedia, kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms, CFStringRef)
    … …  
    162180SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMTimebaseGetEffectiveRate, Float64, (CMTimebaseRef timebase), (timebase))
    163181#define CMTimebaseGetEffectiveRate softLink_CoreMedia_CMTimebaseGetEffectiveRate
     182SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMTimebaseAddTimerDispatchSource, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource), (timebase, timerSource))
     183#define CMTimebaseAddTimerDispatchSource softLink_CoreMedia_CMTimebaseAddTimerDispatchSource
     184SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMTimebaseRemoveTimerDispatchSource, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource), (timebase, timerSource))
     185#define CMTimebaseRemoveTimerDispatchSource softLink_CoreMedia_CMTimebaseRemoveTimerDispatchSource
     186SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMTimebaseSetTimerDispatchSourceNextFireTime, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource, CMTime fireTime, uint32_t flags), (timebase, timerSource, fireTime, flags))
     187#define CMTimebaseSetTimerDispatchSourceNextFireTime softLink_CoreMedia_CMTimebaseSetTimerDispatchSourceNextFireTime
     188SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMTimebaseSetTimerDispatchSourceToFireImmediately, OSStatus, (CMTimebaseRef timebase, dispatch_source_t timerSource), (timebase, timerSource))
     189#define CMTimebaseSetTimerDispatchSourceToFireImmediately softLink_CoreMedia_CMTimebaseSetTimerDispatchSourceToFireImmediately
     190
    164191SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMTimeCopyAsDictionary, CFDictionaryRef, (CMTime time, CFAllocatorRef allocator), (time, allocator))
    165192#define CMTimeCopyAsDictionary softLink_CoreMedia_CMTimeCopyAsDictionary

  • trunk/Source/WebCore/platform/cocoa/CoreVideoSoftLink.cpp

    r214302 r217098  
    3131SOFT_LINK_FRAMEWORK_FOR_SOURCE(WebCore, CoreVideo)
    3232
     33SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferGetTypeID, CFTypeID, (), ())
    3334SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferGetWidth, size_t, (CVPixelBufferRef pixelBuffer), (pixelBuffer))
    3435SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferGetHeight, size_t, (CVPixelBufferRef pixelBuffer), (pixelBuffer))

  • trunk/Source/WebCore/platform/cocoa/CoreVideoSoftLink.h

    r214302 r217098  
    3232SOFT_LINK_FRAMEWORK_FOR_HEADER(WebCore, CoreVideo)
    3333
     34SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferGetTypeID, CFTypeID, (), ())
     35#define CVPixelBufferGetTypeID softLink_CoreVideo_CVPixelBufferGetTypeID
    3436SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferGetWidth, size_t, (CVPixelBufferRef pixelBuffer), (pixelBuffer))
    3537#define CVPixelBufferGetWidth softLink_CoreVideo_CVPixelBufferGetWidth

  • trunk/Source/WebCore/platform/graphics/SourceBufferPrivateClient.h

    r210319 r217098  
    6969    virtual bool sourceBufferPrivateHasVideo() const = 0;
    7070
     71    virtual void sourceBufferPrivateReenqueSamples(const AtomicString& trackID) = 0;
    7172    virtual void sourceBufferPrivateDidBecomeReadyForMoreSamples(const AtomicString& trackID) = 0;
    7273

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h

    r216951 r217098  
    4242
    4343typedef struct OpaqueCMTimebase* CMTimebaseRef;
     44typedef struct __CVBuffer *CVPixelBufferRef;
     45typedef struct __CVBuffer *CVOpenGLTextureRef;
    4446
    4547namespace WebCore {
    4648
    4749class CDMSessionMediaSourceAVFObjC;
     50class MediaSourcePrivateAVFObjC;
     51class PixelBufferConformerCV;
    4852class PlatformClockCM;
    49 class MediaSourcePrivateAVFObjC;
     53class TextureCacheCV;
     54class VideoTextureCopierCV;
     55class WebCoreDecompressionSession;
    5056
    5157#if PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE)
    … …  
    6470    static void getSupportedTypes(HashSet<String, ASCIICaseInsensitiveHash>& types);
    6571    static MediaPlayer::SupportsType supportsType(const MediaEngineSupportParameters&);
    66 
    67     void addDisplayLayer(AVSampleBufferDisplayLayer*);
    68     void removeDisplayLayer(AVSampleBufferDisplayLayer*);
    6972
    7073    void addAudioRenderer(AVSampleBufferAudioRenderer*);
    … …  
    9295    void characteristicsChanged();
    9396
     97    MediaTime currentMediaTime() const override;
     98    AVSampleBufferDisplayLayer* sampleBufferDisplayLayer() const { return m_sampleBufferDisplayLayer.get(); }
     99    WebCoreDecompressionSession* decompressionSession() const { return m_decompressionSession.get(); }
     100
    94101#if PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE)
    95102    void setVideoFullscreenLayer(PlatformLayer*, std::function<void()> completionHandler) override;
    … …  
    150157
    151158    MediaTime durationMediaTime() const override;
    152     MediaTime currentMediaTime() const override;
    153159    MediaTime startTime() const override;
    154160    MediaTime initialTime() const override;
    … …  
    169175    void setSize(const IntSize&) override;
    170176
     177    NativeImagePtr nativeImageForCurrentTime() override;
     178    bool updateLastPixelBuffer();
     179    bool updateLastImage();
    171180    void paint(GraphicsContext&, const FloatRect&) override;
    172181    void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&) override;
    173 
     182    bool copyVideoTextureToPlatformTexture(GraphicsContext3D*, Platform3DObject, GC3Denum target, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY) override;
     183   
    174184    bool hasAvailableVideoFrame() const override;
    175185
    … …  
    178188    void acceleratedRenderingStateChanged() override;
    179189    void notifyActiveSourceBuffersChanged() override;
     190
     191    // NOTE: Because the only way for MSE to recieve data is through an ArrayBuffer provided by
     192    // javascript running in the page, the video will, by necessity, always be CORS correct and
     193    // in the page's origin.
     194    bool hasSingleSecurityOrigin() const override { return true; }
     195    bool didPassCORSAccessCheck() const override { return true; }
    180196
    181197    MediaPlayer::MovieLoadType movieLoadType() const override;
    … …  
    203219    void ensureLayer();
    204220    void destroyLayer();
     221    void ensureDecompressionSession();
     222    void destroyDecompressionSession();
     223
    205224    bool shouldBePlaying() const;
    206225
    … …  
    235254    RetainPtr<id> m_durationObserver;
    236255    RetainPtr<AVStreamSession> m_streamSession;
     256    RetainPtr<CVPixelBufferRef> m_lastPixelBuffer;
     257    RetainPtr<CGImageRef> m_lastImage;
     258    std::unique_ptr<PixelBufferConformerCV> m_rgbConformer;
     259    RefPtr<WebCoreDecompressionSession> m_decompressionSession;
    237260    Deque<RetainPtr<id>> m_sizeChangeObservers;
    238261    Timer m_seekTimer;
    … …  
    240263    MediaPlayer::NetworkState m_networkState;
    241264    MediaPlayer::ReadyState m_readyState;
     265    bool m_readyStateIsWaitingForAvailableFrame { false };
    242266    MediaTime m_lastSeekTime;
    243267    FloatSize m_naturalSize;
    … …  
    245269    bool m_playing;
    246270    bool m_seeking;
    247     bool m_seekCompleted;
     271    enum SeekState {
     272        Seeking,
     273        WaitingForAvailableFame,
     274        SeekCompleted,
     275    };
     276    SeekState m_seekCompleted { SeekCompleted };
    248277    mutable bool m_loadingProgressed;
    249     bool m_hasAvailableVideoFrame;
     278    bool m_hasBeenAskedToPaintGL { false };
     279    bool m_hasAvailableVideoFrame { false };
    250280    bool m_allRenderersHaveAvailableSamples { false };
    251281    RetainPtr<PlatformLayer> m_textTrackRepresentationLayer;
     282    std::unique_ptr<TextureCacheCV> m_textureCache;
     283    std::unique_ptr<VideoTextureCopierCV> m_videoTextureCopier;
     284    RetainPtr<CVOpenGLTextureRef> m_lastTexture;
    252285#if ENABLE(WIRELESS_PLAYBACK_TARGET)
    253286    RefPtr<MediaPlaybackTarget> m_playbackTarget;

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm

    r216951 r217098  
    3434#import "CDMSessionMediaSourceAVFObjC.h"
    3535#import "FileSystem.h"
     36#import "GraphicsContextCG.h"
    3637#import "Logging.h"
    3738#import "MediaSourcePrivateAVFObjC.h"
    3839#import "MediaSourcePrivateClient.h"
    3940#import "MediaTimeAVFoundation.h"
     41#import "PixelBufferConformerCV.h"
    4042#import "PlatformClockCM.h"
    4143#import "TextTrackRepresentation.h"
     44#import "TextureCacheCV.h"
     45#import "VideoTextureCopierCV.h"
     46#import "WebCoreDecompressionSession.h"
    4247#import "WebCoreSystemInterface.h"
    4348#import <AVFoundation/AVAsset.h>
    … …  
    122127    , m_playing(0)
    123128    , m_seeking(false)
    124     , m_seekCompleted(true)
    125129    , m_loadingProgressed(false)
    126130#if PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE)
    … …  
    149153            if (shouldBePlaying())
    150154                [m_synchronizer setRate:m_rate];
    151             if (!seeking())
     155            if (!seeking() && m_seekCompleted == SeekCompleted)
    152156                m_player->timeChanged();
    153157        }
    … …  
    242246
    243247    m_mediaSourcePrivate = MediaSourcePrivateAVFObjC::create(this, client);
     248    m_mediaSourcePrivate->setVideoLayer(m_sampleBufferDisplayLayer.get());
     249    m_mediaSourcePrivate->setDecompressionSession(m_decompressionSession.get());
     250
     251    acceleratedRenderingStateChanged();
    244252}
    245253
    … …  
    355363void MediaPlayerPrivateMediaSourceAVFObjC::setVisible(bool)
    356364{
    357     // No-op.
     365    acceleratedRenderingStateChanged();
    358366}
    359367
    … …  
    438446        return;
    439447    LOG(MediaSource, "MediaPlayerPrivateMediaSourceAVFObjC::waitForSeekCompleted(%p)", this);
    440     m_seekCompleted = false;
     448    m_seekCompleted = Seeking;
    441449}
    442450
    443451void MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted()
    444452{
    445     if (m_seekCompleted)
    446         return;
     453    if (m_seekCompleted == SeekCompleted)
     454        return;
     455    if (hasVideo() && !m_hasAvailableVideoFrame) {
     456        m_seekCompleted = WaitingForAvailableFame;
     457        return;
     458    }
    447459    LOG(MediaSource, "MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted(%p)", this);
    448     m_seekCompleted = true;
     460    m_seekCompleted = SeekCompleted;
    449461    if (shouldBePlaying())
    450462        [m_synchronizer setRate:m_rate];
    … …  
    455467bool MediaPlayerPrivateMediaSourceAVFObjC::seeking() const
    456468{
    457     return m_seeking || !m_seekCompleted;
     469    return m_seeking || m_seekCompleted != SeekCompleted;
    458470}
    459471
    … …  
    515527}
    516528
    517 void MediaPlayerPrivateMediaSourceAVFObjC::paint(GraphicsContext&, const FloatRect&)
    518 {
    519     // FIXME(125157): Implement painting.
    520 }
    521 
    522 void MediaPlayerPrivateMediaSourceAVFObjC::paintCurrentFrameInContext(GraphicsContext&, const FloatRect&)
    523 {
    524     // FIXME(125157): Implement painting.
     529NativeImagePtr MediaPlayerPrivateMediaSourceAVFObjC::nativeImageForCurrentTime()
     530{
     531    updateLastImage();
     532    return m_lastImage.get();
     533}
     534
     535bool MediaPlayerPrivateMediaSourceAVFObjC::updateLastPixelBuffer()
     536{
     537    if (m_sampleBufferDisplayLayer || !m_decompressionSession)
     538        return false;
     539
     540    auto flags = !m_lastPixelBuffer ? WebCoreDecompressionSession::AllowLater : WebCoreDecompressionSession::ExactTime;
     541    auto newPixelBuffer = m_decompressionSession->imageForTime(currentMediaTime(), flags);
     542    if (!newPixelBuffer)
     543        return false;
     544
     545    m_lastPixelBuffer = newPixelBuffer;
     546    return true;
     547}
     548
     549bool MediaPlayerPrivateMediaSourceAVFObjC::updateLastImage()
     550{
     551    if (!updateLastPixelBuffer())
     552        return false;
     553
     554    ASSERT(m_lastPixelBuffer);
     555
     556    if (!m_rgbConformer) {
     557        NSDictionary *attributes = @{ (NSString *)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_32BGRA) };
     558        m_rgbConformer = std::make_unique<PixelBufferConformerCV>((CFDictionaryRef)attributes);
     559    }
     560
     561    m_lastImage = m_rgbConformer->createImageFromPixelBuffer(m_lastPixelBuffer.get());
     562    return true;
     563}
     564
     565void MediaPlayerPrivateMediaSourceAVFObjC::paint(GraphicsContext& context, const FloatRect& rect)
     566{
     567    paintCurrentFrameInContext(context, rect);
     568}
     569
     570void MediaPlayerPrivateMediaSourceAVFObjC::paintCurrentFrameInContext(GraphicsContext& context, const FloatRect& outputRect)
     571{
     572    if (context.paintingDisabled())
     573        return;
     574
     575    auto image = nativeImageForCurrentTime();
     576    if (!image)
     577        return;
     578
     579    GraphicsContextStateSaver stateSaver(context);
     580    FloatRect imageRect(0, 0, CGImageGetWidth(image.get()), CGImageGetHeight(image.get()));
     581    context.drawNativeImage(image, imageRect.size(), outputRect, imageRect);
     582}
     583
     584bool MediaPlayerPrivateMediaSourceAVFObjC::copyVideoTextureToPlatformTexture(GraphicsContext3D* context, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
     585{
     586    if (flipY || premultiplyAlpha)
     587        return false;
     588
     589    // We have been asked to paint into a WebGL canvas, so take that as a signal to create
     590    // a decompression session, even if that means the native video can't also be displayed
     591    // in page.
     592    if (!m_hasBeenAskedToPaintGL) {
     593        m_hasBeenAskedToPaintGL = true;
     594        acceleratedRenderingStateChanged();
     595    }
     596
     597    ASSERT(context);
     598
     599    if (updateLastPixelBuffer()) {
     600        if (!m_lastPixelBuffer)
     601            return false;
     602
     603        if (!m_textureCache) {
     604            m_textureCache = TextureCacheCV::create(*context);
     605            if (!m_textureCache)
     606                return false;
     607        }
     608
     609        m_lastTexture = m_textureCache->textureFromImage(m_lastPixelBuffer.get(), outputTarget, level, internalFormat, format, type);
     610    }
     611
     612    size_t width = CVPixelBufferGetWidth(m_lastPixelBuffer.get());
     613    size_t height = CVPixelBufferGetHeight(m_lastPixelBuffer.get());
     614
     615    if (!m_videoTextureCopier)
     616        m_videoTextureCopier = std::make_unique<VideoTextureCopierCV>(*context);
     617
     618    return m_videoTextureCopier->copyVideoTextureToPlatformTexture(m_lastTexture.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY);
    525619}
    526620
    … …  
    537631void MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged()
    538632{
    539     if (m_player->client().mediaPlayerRenderingCanBeAccelerated(m_player))
     633    if (!m_hasBeenAskedToPaintGL && m_player->visible() && m_player->client().mediaPlayerRenderingCanBeAccelerated(m_player)) {
     634        destroyDecompressionSession();
    540635        ensureLayer();
    541     else
     636    } else {
    542637        destroyLayer();
     638        ensureDecompressionSession();
     639    }
    543640}
    544641
    … …  
    609706
    610707    [m_synchronizer addRenderer:m_sampleBufferDisplayLayer.get()];
    611    
    612 #if PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE)
     708    if (m_mediaSourcePrivate)
     709        m_mediaSourcePrivate->setVideoLayer(m_sampleBufferDisplayLayer.get());
     710#if PLATFORM(IOS) || (PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE))
    613711    m_videoFullscreenLayerManager->setVideoLayer(m_sampleBufferDisplayLayer.get(), snappedIntRect(m_player->client().mediaPlayerContentBoxRect()).size());
    614712#endif
     713    m_player->client().mediaPlayerRenderingModeChanged(m_player);
    615714}
    616715
    … …  
    624723        // No-op.
    625724    }];
     725
     726    if (m_mediaSourcePrivate)
     727        m_mediaSourcePrivate->setVideoLayer(nullptr);
     728#if PLATFORM(IOS) || (PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE))
     729    m_videoFullscreenLayerManager->didDestroyVideoLayer();
     730#endif
    626731    m_sampleBufferDisplayLayer = nullptr;
     732    setHasAvailableVideoFrame(false);
     733    m_player->client().mediaPlayerRenderingModeChanged(m_player);
     734}
     735
     736void MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession()
     737{
     738    if (m_decompressionSession)
     739        return;
     740
     741    m_decompressionSession = WebCoreDecompressionSession::create();
     742    m_decompressionSession->setTimebase([m_synchronizer timebase]);
     743    m_mediaSourcePrivate->setDecompressionSession(m_decompressionSession.get());
     744
     745    m_player->client().mediaPlayerRenderingModeChanged(m_player);
     746}
     747
     748void MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession()
     749{
     750    if (!m_decompressionSession)
     751        return;
     752
     753    m_mediaSourcePrivate->setDecompressionSession(nullptr);
     754    m_decompressionSession->invalidate();
     755    m_decompressionSession = nullptr;
     756    setHasAvailableVideoFrame(false);
    627757}
    628758
    … …  
    638768    m_hasAvailableVideoFrame = flag;
    639769    updateAllRenderersHaveAvailableSamples();
     770
     771    if (!m_hasAvailableVideoFrame)
     772        return;
     773
     774    m_player->firstVideoFrameAvailable();
     775    if (m_seekCompleted == WaitingForAvailableFame)
     776        seekCompleted();
     777
     778    if (m_readyStateIsWaitingForAvailableFrame) {
     779        m_readyStateIsWaitingForAvailableFrame = false;
     780        m_player->readyStateChanged();
     781    }
    640782}
    641783
    … …  
    658800
    659801    do {
    660         if (m_sampleBufferDisplayLayer && !m_hasAvailableVideoFrame) {
     802        if (hasVideo() && !m_hasAvailableVideoFrame) {
    661803            allRenderersHaveAvailableSamples = false;
    662804            break;
    … …  
    815957        [m_synchronizer setRate:0];
    816958
     959    if (m_readyState >= MediaPlayerEnums::HaveCurrentData && hasVideo() && !m_hasAvailableVideoFrame) {
     960        m_readyStateIsWaitingForAvailableFrame = true;
     961        return;
     962    }
     963
    817964    m_player->readyStateChanged();
    818965}
    … …  
    825972    m_networkState = networkState;
    826973    m_player->networkStateChanged();
    827 }
    828 
    829 void MediaPlayerPrivateMediaSourceAVFObjC::addDisplayLayer(AVSampleBufferDisplayLayer* displayLayer)
    830 {
    831     ASSERT(displayLayer);
    832     if (displayLayer == m_sampleBufferDisplayLayer)
    833         return;
    834 
    835     m_sampleBufferDisplayLayer = displayLayer;
    836     [m_synchronizer addRenderer:m_sampleBufferDisplayLayer.get()];
    837     m_player->client().mediaPlayerRenderingModeChanged(m_player);
    838 
    839     // FIXME: move this somewhere appropriate:
    840     m_player->firstVideoFrameAvailable();
    841 
    842 #if PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE)
    843     m_videoFullscreenLayerManager->setVideoLayer(m_sampleBufferDisplayLayer.get(), snappedIntRect(m_player->client().mediaPlayerContentBoxRect()).size());
    844 #endif
    845 }
    846 
    847 void MediaPlayerPrivateMediaSourceAVFObjC::removeDisplayLayer(AVSampleBufferDisplayLayer* displayLayer)
    848 {
    849     if (displayLayer != m_sampleBufferDisplayLayer)
    850         return;
    851 
    852     CMTime currentTime = CMTimebaseGetTime([m_synchronizer timebase]);
    853     [m_synchronizer removeRenderer:m_sampleBufferDisplayLayer.get() atTime:currentTime withCompletionHandler:^(BOOL){
    854         // No-op.
    855     }];
    856 
    857     m_sampleBufferDisplayLayer = nullptr;
    858     m_player->client().mediaPlayerRenderingModeChanged(m_player);
    859 
    860 #if PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE)
    861     m_videoFullscreenLayerManager->didDestroyVideoLayer();
    862 #endif
    863974}
    864975
    … …  
    8951006void MediaPlayerPrivateMediaSourceAVFObjC::characteristicsChanged()
    8961007{
     1008    updateAllRenderersHaveAvailableSamples();
    8971009    m_player->characteristicChanged();
    8981010}

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h

    r207694 r217098  
    3636
    3737OBJC_CLASS AVAsset;
     38OBJC_CLASS AVSampleBufferDisplayLayer;
    3839OBJC_CLASS AVStreamDataParser;
    3940OBJC_CLASS NSError;
    … …  
    4849class SourceBufferPrivateAVFObjC;
    4950class TimeRanges;
     51class WebCoreDecompressionSession;
    5052
    5153class MediaSourcePrivateAVFObjC final : public MediaSourcePrivate {
    … …  
    7274    bool hasAudio() const;
    7375    bool hasVideo() const;
     76    bool hasSelectedVideo() const;
    7477
    7578    void willSeek();
    … …  
    7780    MediaTime fastSeekTimeForMediaTime(const MediaTime&, const MediaTime& negativeThreshold, const MediaTime& positiveThreshold);
    7881    FloatSize naturalSize() const;
     82
     83    void hasSelectedVideoChanged(SourceBufferPrivateAVFObjC&);
     84    void setVideoLayer(AVSampleBufferDisplayLayer*);
     85    void setDecompressionSession(WebCoreDecompressionSession*);
    7986
    8087private:
    … …  
    8996    void removeSourceBuffer(SourceBufferPrivate*);
    9097
     98    void setSourceBufferWithSelectedVideo(SourceBufferPrivateAVFObjC*);
     99
    91100    friend class SourceBufferPrivateAVFObjC;
    92101
    … …  
    96105    Vector<SourceBufferPrivateAVFObjC*> m_activeSourceBuffers;
    97106    Deque<SourceBufferPrivateAVFObjC*> m_sourceBuffersNeedingSessions;
     107    SourceBufferPrivateAVFObjC* m_sourceBufferWithSelectedVideo { nullptr };
    98108    bool m_isEnded;
    99109};

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm

    r208658 r217098  
    175175}
    176176
    177 static bool MediaSourcePrivateAVFObjCHasVideo(SourceBufferPrivateAVFObjC* sourceBuffer)
    178 {
    179     return sourceBuffer->hasVideo();
    180 }
    181 
    182177bool MediaSourcePrivateAVFObjC::hasVideo() const
    183178{
    184     return std::any_of(m_activeSourceBuffers.begin(), m_activeSourceBuffers.end(), MediaSourcePrivateAVFObjCHasVideo);
     179    return std::any_of(m_activeSourceBuffers.begin(), m_activeSourceBuffers.end(), [] (SourceBufferPrivateAVFObjC* sourceBuffer) {
     180        return sourceBuffer->hasVideo();
     181    });
     182}
     183
     184bool MediaSourcePrivateAVFObjC::hasSelectedVideo() const
     185{
     186    return std::any_of(m_activeSourceBuffers.begin(), m_activeSourceBuffers.end(), [] (SourceBufferPrivateAVFObjC* sourceBuffer) {
     187        return sourceBuffer->hasSelectedVideo();
     188    });
    185189}
    186190
    … …  
    219223}
    220224
     225void MediaSourcePrivateAVFObjC::hasSelectedVideoChanged(SourceBufferPrivateAVFObjC& sourceBuffer)
     226{
     227    bool hasSelectedVideo = sourceBuffer.hasSelectedVideo();
     228    if (m_sourceBufferWithSelectedVideo == &sourceBuffer && !hasSelectedVideo)
     229        setSourceBufferWithSelectedVideo(nullptr);
     230    else if (m_sourceBufferWithSelectedVideo != &sourceBuffer && hasSelectedVideo)
     231        setSourceBufferWithSelectedVideo(&sourceBuffer);
     232}
     233
     234void MediaSourcePrivateAVFObjC::setVideoLayer(AVSampleBufferDisplayLayer* layer)
     235{
     236    if (m_sourceBufferWithSelectedVideo)
     237        m_sourceBufferWithSelectedVideo->setVideoLayer(layer);
     238}
     239
     240void MediaSourcePrivateAVFObjC::setDecompressionSession(WebCoreDecompressionSession* decompressionSession)
     241{
     242    if (m_sourceBufferWithSelectedVideo)
     243        m_sourceBufferWithSelectedVideo->setDecompressionSession(decompressionSession);
     244}
     245
     246void MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo(SourceBufferPrivateAVFObjC* sourceBuffer)
     247{
     248    if (m_sourceBufferWithSelectedVideo) {
     249        m_sourceBufferWithSelectedVideo->setVideoLayer(nullptr);
     250        m_sourceBufferWithSelectedVideo->setDecompressionSession(nullptr);
     251    }
     252
     253    m_sourceBufferWithSelectedVideo = sourceBuffer;
     254
     255    if (m_sourceBufferWithSelectedVideo) {
     256        m_sourceBufferWithSelectedVideo->setVideoLayer(m_player->sampleBufferDisplayLayer());
     257        m_sourceBufferWithSelectedVideo->setDecompressionSession(m_player->decompressionSession());
     258    }
     259}
     260
    221261}
    222262

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h

    r210319 r217098  
    6363class AudioTrackPrivateMediaSourceAVFObjC;
    6464class VideoTrackPrivateMediaSourceAVFObjC;
     65class WebCoreDecompressionSession;
    6566
    6667class SourceBufferPrivateAVFObjCErrorClient {
    … …  
    8990
    9091    bool hasVideo() const;
     92    bool hasSelectedVideo() const;
    9193    bool hasAudio() const;
    9294
    … …  
    109111    void layerDidReceiveError(AVSampleBufferDisplayLayer *, NSError *);
    110112    void rendererDidReceiveError(AVSampleBufferAudioRenderer *, NSError *);
     113
     114    void setVideoLayer(AVSampleBufferDisplayLayer*);
     115    void setDecompressionSession(WebCoreDecompressionSession*);
    111116
    112117private:
    … …  
    153158    OSObjectPtr<dispatch_semaphore_t> m_hasSessionSemaphore;
    154159    OSObjectPtr<dispatch_group_t> m_isAppendingGroup;
     160    RefPtr<WebCoreDecompressionSession> m_decompressionSession;
    155161
    156162    MediaSourcePrivateAVFObjC* m_mediaSource;

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm

    r214930 r217098  
    3030
    3131#import "AVFoundationSPI.h"
     32#import "AudioTrackPrivateMediaSourceAVFObjC.h"
    3233#import "CDMSessionAVContentKeySession.h"
    3334#import "CDMSessionMediaSourceAVFObjC.h"
     35#import "InbandTextTrackPrivateAVFObjC.h"
    3436#import "Logging.h"
    3537#import "MediaDescription.h"
    … …  
    4345#import "SourceBufferPrivateClient.h"
    4446#import "TimeRanges.h"
    45 #import "AudioTrackPrivateMediaSourceAVFObjC.h"
    4647#import "VideoTrackPrivateMediaSourceAVFObjC.h"
    47 #import "InbandTextTrackPrivateAVFObjC.h"
     48#import "WebCoreDecompressionSession.h"
    4849#import <AVFoundation/AVAssetTrack.h>
    4950#import <QuartzCore/CALayer.h>
     51#import <map>
    5052#import <objc/runtime.h>
    5153#import <runtime/TypedArrayInlines.h>
    52 #import <wtf/text/AtomicString.h>
    53 #import <wtf/text/CString.h>
    5454#import <wtf/BlockObjCExceptions.h>
    5555#import <wtf/HashCountedSet.h>
    5656#import <wtf/MainThread.h>
    5757#import <wtf/WeakPtr.h>
    58 #import <map>
     58#import <wtf/text/AtomicString.h>
     59#import <wtf/text/CString.h>
    5960
    6061#pragma mark - Soft Linking
    … …  
    652653void SourceBufferPrivateAVFObjC::destroyRenderers()
    653654{
    654     if (m_displayLayer) {
    655         if (m_mediaSource)
    656             m_mediaSource->player()->removeDisplayLayer(m_displayLayer.get());
    657         [m_displayLayer flush];
    658         [m_displayLayer stopRequestingMediaData];
    659         [m_errorListener stopObservingLayer:m_displayLayer.get()];
    660         m_displayLayer = nullptr;
    661     }
     655    if (m_displayLayer)
     656        setVideoLayer(nullptr);
     657
     658    if (m_decompressionSession)
     659        setDecompressionSession(nullptr);
    662660
    663661    for (auto& renderer : m_audioRenderers.values()) {
    … …  
    697695}
    698696
     697bool SourceBufferPrivateAVFObjC::hasSelectedVideo() const
     698{
     699    return m_enabledVideoTrackID != -1;
     700}
     701
    699702bool SourceBufferPrivateAVFObjC::hasAudio() const
    700703{
    … …  
    708711        m_enabledVideoTrackID = -1;
    709712        [m_parser setShouldProvideMediaData:NO forTrackID:trackID];
    710         if (m_mediaSource)
    711             m_mediaSource->player()->removeDisplayLayer(m_displayLayer.get());
     713
     714        if (m_decompressionSession)
     715            m_decompressionSession->stopRequestingMediaData();
    712716    } else if (track->selected()) {
    713717        m_enabledVideoTrackID = trackID;
    714718        [m_parser setShouldProvideMediaData:YES forTrackID:trackID];
    715         if (!m_displayLayer) {
    716             m_displayLayer = adoptNS([allocAVSampleBufferDisplayLayerInstance() init]);
    717 #ifndef NDEBUG
    718             [m_displayLayer setName:@"SourceBufferPrivateAVFObjC AVSampleBufferDisplayLayer"];
    719 #endif
    720             [m_displayLayer requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:^{
     719
     720        if (m_decompressionSession) {
     721            m_decompressionSession->requestMediaDataWhenReady([this, trackID] {
    721722                didBecomeReadyForMoreSamples(trackID);
    722             }];
    723             [m_errorListener beginObservingLayer:m_displayLayer.get()];
     723            });
    724724        }
    725         if (m_mediaSource)
    726             m_mediaSource->player()->addDisplayLayer(m_displayLayer.get());
    727     }
     725    }
     726
     727    m_mediaSource->hasSelectedVideoChanged(*this);
    728728}
    729729
    … …  
    794794        [m_displayLayer flushAndRemoveImage];
    795795
     796    if (m_decompressionSession) {
     797        m_decompressionSession->flush();
     798        m_decompressionSession->notifyWhenHasAvailableVideoFrame([weakThis = createWeakPtr()] {
     799            if (weakThis && weakThis->m_mediaSource)
     800                weakThis->m_mediaSource->player()->setHasAvailableVideoFrame(true);
     801        });
     802    }
     803
    796804    for (auto& renderer : m_audioRenderers.values())
    797805        [renderer flush];
    … …  
    853861    LOG(MediaSource, "SourceBufferPrivateAVFObjC::flush(%p) - trackId: %d", this, trackID);
    854862
    855     if (trackID == m_enabledVideoTrackID)
     863    if (trackID == m_enabledVideoTrackID) {
    856864        flush(m_displayLayer.get());
    857     else if (m_audioRenderers.contains(trackID))
     865        if (m_decompressionSession) {
     866            m_decompressionSession->flush();
     867            m_decompressionSession->notifyWhenHasAvailableVideoFrame([weakThis = createWeakPtr()] {
     868                if (weakThis && weakThis->m_mediaSource)
     869                    weakThis->m_mediaSource->player()->setHasAvailableVideoFrame(true);
     870            });
     871        }
     872    } else if (m_audioRenderers.contains(trackID))
    858873        flush(m_audioRenderers.get(trackID).get());
    859874}
    … …  
    905920        }
    906921
    907         [m_displayLayer enqueueSampleBuffer:platformSample.sample.cmSampleBuffer];
    908         if (m_mediaSource)
    909             m_mediaSource->player()->setHasAvailableVideoFrame(!sample->isNonDisplaying());
     922        if (m_decompressionSession)
     923            m_decompressionSession->enqueueSample(platformSample.sample.cmSampleBuffer);
     924
     925        if (m_displayLayer) {
     926            [m_displayLayer enqueueSampleBuffer:platformSample.sample.cmSampleBuffer];
     927            if (m_mediaSource)
     928                m_mediaSource->player()->setHasAvailableVideoFrame(!sample->isNonDisplaying());
     929        }
    910930    } else {
    911931        auto renderer = m_audioRenderers.get(trackID);
    … …  
    920940    int trackID = trackIDString.toInt();
    921941    if (trackID == m_enabledVideoTrackID)
    922         return [m_displayLayer isReadyForMoreMediaData];
    923     else if (m_audioRenderers.contains(trackID))
     942        return !m_decompressionSession || m_decompressionSession->isReadyForMoreMediaData();
     943
     944    if (m_audioRenderers.contains(trackID))
    924945        return [m_audioRenderers.get(trackID) isReadyForMoreMediaData];
    925946
    … …  
    962983void SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples(int trackID)
    963984{
    964     if (trackID == m_enabledVideoTrackID)
     985    LOG(Media, "SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples(%p) - track(%d)", this, trackID);
     986    if (trackID == m_enabledVideoTrackID) {
     987        if (m_decompressionSession)
     988            m_decompressionSession->stopRequestingMediaData();
    965989        [m_displayLayer stopRequestingMediaData];
    966     else if (m_audioRenderers.contains(trackID))
     990    } else if (m_audioRenderers.contains(trackID))
    967991        [m_audioRenderers.get(trackID) stopRequestingMediaData];
    968992    else {
    … …  
    9791003    int trackID = trackIDString.toInt();
    9801004    if (trackID == m_enabledVideoTrackID) {
    981         [m_displayLayer requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:^{
     1005        if (m_decompressionSession) {
     1006            m_decompressionSession->requestMediaDataWhenReady([this, trackID] {
     1007                didBecomeReadyForMoreSamples(trackID);
     1008            });
     1009        }
     1010        if (m_displayLayer) {
     1011            [m_displayLayer requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:^ {
     1012                didBecomeReadyForMoreSamples(trackID);
     1013            }];
     1014        }
     1015    } else if (m_audioRenderers.contains(trackID)) {
     1016        [m_audioRenderers.get(trackID) requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:^ {
    9821017            didBecomeReadyForMoreSamples(trackID);
    9831018        }];
    984     } else if (m_audioRenderers.contains(trackID)) {
    985         [m_audioRenderers.get(trackID) requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:^{
    986             didBecomeReadyForMoreSamples(trackID);
     1019    }
     1020}
     1021
     1022void SourceBufferPrivateAVFObjC::setVideoLayer(AVSampleBufferDisplayLayer* layer)
     1023{
     1024    if (layer == m_displayLayer)
     1025        return;
     1026
     1027    ASSERT(!layer || !m_decompressionSession || hasSelectedVideo());
     1028
     1029    if (m_displayLayer) {
     1030        [m_displayLayer flush];
     1031        [m_displayLayer stopRequestingMediaData];
     1032        [m_errorListener stopObservingLayer:m_displayLayer.get()];
     1033    }
     1034
     1035    m_displayLayer = layer;
     1036
     1037    if (m_displayLayer) {
     1038        [m_displayLayer requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:^ {
     1039            didBecomeReadyForMoreSamples(m_enabledVideoTrackID);
    9871040        }];
    988     }
     1041        [m_errorListener beginObservingLayer:m_displayLayer.get()];
     1042        if (m_client)
     1043            m_client->sourceBufferPrivateReenqueSamples(AtomicString::number(m_enabledVideoTrackID));
     1044    }
     1045}
     1046
     1047void SourceBufferPrivateAVFObjC::setDecompressionSession(WebCoreDecompressionSession* decompressionSession)
     1048{
     1049    if (m_decompressionSession == decompressionSession)
     1050        return;
     1051
     1052    if (m_decompressionSession) {
     1053        m_decompressionSession->stopRequestingMediaData();
     1054        m_decompressionSession->invalidate();
     1055    }
     1056
     1057    m_decompressionSession = decompressionSession;
     1058
     1059    if (!m_decompressionSession)
     1060        return;
     1061
     1062    WeakPtr<SourceBufferPrivateAVFObjC> weakThis = createWeakPtr();
     1063    m_decompressionSession->requestMediaDataWhenReady([weakThis] {
     1064        if (weakThis)
     1065            weakThis->didBecomeReadyForMoreSamples(weakThis->m_enabledVideoTrackID);
     1066    });
     1067    m_decompressionSession->notifyWhenHasAvailableVideoFrame([weakThis = createWeakPtr()] {
     1068        if (weakThis && weakThis->m_mediaSource)
     1069            weakThis->m_mediaSource->player()->setHasAvailableVideoFrame(true);
     1070    });
     1071    if (m_client)
     1072        m_client->sourceBufferPrivateReenqueSamples(AtomicString::number(m_enabledVideoTrackID));
    9891073}
    9901074
Note: See TracChangeset for help on using the changeset viewer.