Changeset 203826 in webkit


Ignore:
Timestamp:
Jul 28, 2016 11:26:21 AM (8 years ago)
Author:
Ryan Haddad
Message:

HTMLVideoElement with MediaStream src shows paused image when all video tracks are disabled
​https://bugs.webkit.org/show_bug.cgi?id=160222
<rdar://problem/27557313>

Patch by George Ruan <​gruan@apple.com> on 2016-07-28
Reviewed by Eric Carlson.

Source/WebCore:

Tests: fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html

fast/mediastream/MediaStream-video-element-video-tracks-disabled.html

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:

(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Change criteria to enqueue a
Sample Buffer to the AVSampleBufferDisplayLayer to allow an initial frame to be shown.
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::shouldEnqueueVideoSampleBuffer): Allow an initial frame to be shown.
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::flushAndRemoveVideoSampleBuffers): Removes all buffers from the
AVSampleBufferDisplayLayer.
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Make the AVSampleBufferDisplayLayer's background black.
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Remove all buffers from the AVSampleBufferDisplayLayer
when state of MediaPlayerPrivateMediaStreamAVFObjC is changed to None of PaintItBlack.
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updatePausedImage): Updates paused image.
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::pause): Calls updatePausedImage.
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::paintCurrentFrameInContext): Allow an initial frame to be painted to canvas.

LayoutTests:

  • fast/mediastream/MediaStream-video-element-video-tracks-disabled-expected.html: Added.
  • fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled-expected.txt: Added.
  • fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html: Added. Checks

that the video frames display captured media if all video tracks were disabled and then a single
video track is re-enabled. This test also checks that an initial frame is painted to
canvas if the video has not yet been played.

  • fast/mediastream/MediaStream-video-element-video-tracks-disabled.html: Added. Reference tests the

frames of the video to be black, since the canvas is painted black regardless of the state of the video frames
if displayMode of MediaPlayerPrivateMediaStreamAVFObjC is PaintItBlack.

Location:
trunk
Files:
4 added
4 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r203823 r203826  
     12016-07-28  George Ruan  <gruan@apple.com>
     2
     3        HTMLVideoElement with MediaStream src shows paused image when all video tracks are disabled
     4        https://bugs.webkit.org/show_bug.cgi?id=160222
     5        <rdar://problem/27557313>
     6
     7        Reviewed by Eric Carlson.
     8
     9        * fast/mediastream/MediaStream-video-element-video-tracks-disabled-expected.html: Added.
     10        * fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled-expected.txt: Added.
     11        * fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html: Added. Checks
     12        that the video frames display captured media if all video tracks were disabled and then a single
     13        video track is re-enabled. This test also checks that an initial frame is painted to
     14        canvas if the video has not yet been played.
     15        * fast/mediastream/MediaStream-video-element-video-tracks-disabled.html: Added. Reference tests the
     16        frames of the video to be black, since the canvas is painted black regardless of the state of the video frames
     17        if displayMode of MediaPlayerPrivateMediaStreamAVFObjC is PaintItBlack.
     18
    1192016-07-28  Ryan Haddad  <ryanhaddad@apple.com>
    220

  • trunk/Source/WebCore/ChangeLog

    r203825 r203826  
     12016-07-28  George Ruan  <gruan@apple.com>
     2
     3        HTMLVideoElement with MediaStream src shows paused image when all video tracks are disabled
     4        https://bugs.webkit.org/show_bug.cgi?id=160222
     5        <rdar://problem/27557313>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Tests: fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html
     10               fast/mediastream/MediaStream-video-element-video-tracks-disabled.html
     11
     12        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
     13        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
     14        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoSampleBufferFromTrack): Change criteria to enqueue a
     15        Sample Buffer to the AVSampleBufferDisplayLayer to allow an initial frame to be shown.
     16        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::shouldEnqueueVideoSampleBuffer): Allow an initial frame to be shown.
     17        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::flushAndRemoveVideoSampleBuffers): Removes all buffers from the
     18        AVSampleBufferDisplayLayer.
     19        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayer): Make the AVSampleBufferDisplayLayer's background black.
     20        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayMode): Remove all buffers from the AVSampleBufferDisplayLayer
     21        when state of MediaPlayerPrivateMediaStreamAVFObjC is changed to None of PaintItBlack.
     22        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updatePausedImage): Updates paused image.
     23        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::pause): Calls updatePausedImage.
     24        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::paintCurrentFrameInContext): Allow an initial frame to be painted to canvas.
     25
    1262016-07-28  Dean Jackson  <dino@apple.com>
    227

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h

    r203739 r203826  
    125125    void enqueueAudioSampleBufferFromTrack(MediaStreamTrackPrivate&, PlatformSample);
    126126    void enqueueVideoSampleBufferFromTrack(MediaStreamTrackPrivate&, PlatformSample);
     127    bool shouldEnqueueVideoSampleBuffer() const;
     128    void flushAndRemoveVideoSampleBuffers();
    127129
    128130    void paint(GraphicsContext&, const FloatRect&) override;
    … …  
    162164    DisplayMode currentDisplayMode() const;
    163165    void updateDisplayMode();
     166    void updatePausedImage();
    164167
    165168    // MediaStreamPrivate::Observer
    … …  
    202205    bool m_hasEverEnqueuedVideoFrame { false };
    203206    bool m_hasReceivedMedia { false };
     207    bool m_isFrameDisplayed { false };
    204208
    205209#if PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE)

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm

    r203739 r203826  
    130130        return;
    131131
    132     if (m_displayMode == LivePreview && [m_sampleBufferDisplayLayer isReadyForMoreMediaData]) {
     132    if (shouldEnqueueVideoSampleBuffer()) {
    133133        [m_sampleBufferDisplayLayer enqueueSampleBuffer:platformSample.sample.cmSampleBuffer];
     134        m_isFrameDisplayed = true;
    134135       
    135136        if (!m_hasEverEnqueuedVideoFrame) {
    136137            m_hasEverEnqueuedVideoFrame = true;
    137138            m_player->firstVideoFrameAvailable();
     139
     140            updatePausedImage();
    138141        }
    139142    }
     143}
     144
     145bool MediaPlayerPrivateMediaStreamAVFObjC::shouldEnqueueVideoSampleBuffer() const
     146{
     147    if (![m_sampleBufferDisplayLayer isReadyForMoreMediaData])
     148        return false;
     149
     150    if (m_displayMode == LivePreview)
     151        return true;
     152
     153    if (m_displayMode == PausedImage && !m_isFrameDisplayed)
     154        return true;
     155
     156    return false;
     157}
     158
     159void MediaPlayerPrivateMediaStreamAVFObjC::flushAndRemoveVideoSampleBuffers()
     160{
     161    [m_sampleBufferDisplayLayer flushAndRemoveImage];
     162    m_isFrameDisplayed = false;
    140163}
    141164
    … …  
    149172    [m_sampleBufferDisplayLayer setName:@"MediaPlayerPrivateMediaStreamAVFObjC AVSampleBufferDisplayLayer"];
    150173#endif
     174    m_sampleBufferDisplayLayer.get().backgroundColor = cachedCGColor(Color::black);
    151175   
    152176    renderingModeChanged();
    … …  
    258282    m_displayMode = displayMode;
    259283
    260     if (m_displayMode == None)
    261         return;
     284    if (m_displayMode < PausedImage && m_sampleBufferDisplayLayer)
     285        flushAndRemoveVideoSampleBuffers();
     286}
     287
     288void MediaPlayerPrivateMediaStreamAVFObjC::updatePausedImage()
     289{
     290    ASSERT(m_displayMode == currentDisplayMode());
     291
     292    if (m_displayMode < PausedImage)
     293        return;
     294
     295    RefPtr<Image> image = m_mediaStreamPrivate->currentFrameImage();
     296    ASSERT(image);
     297    if (!image)
     298        return;
     299
     300    m_pausedImage = image->getCGImageRef();
     301    ASSERT(m_pausedImage);
    262302}
    263303
    … …  
    287327    m_playing = false;
    288328    updateDisplayMode();
     329    updatePausedImage();
    289330}
    290331
    … …  
    578619void MediaPlayerPrivateMediaStreamAVFObjC::paintCurrentFrameInContext(GraphicsContext& context, const FloatRect& rect)
    579620{
    580     if (m_displayMode == None || !metaDataAvailable() || context.paintingDisabled() || !m_haveEverPlayed)
    581         return;
    582 
     621    if (m_displayMode == None || !metaDataAvailable() || context.paintingDisabled())
     622        return;
    583623
    584624    if (m_displayMode == LivePreview)
Note: See TracChangeset for help on using the changeset viewer.