Changeset 193836 in webkit


Ignore:
Timestamp:
Dec 9, 2015 9:37:29 AM (8 years ago)
Author:
yoon@igalia.com
Message:

[ThreadedCompositor] Support HTML5 Video
​https://bugs.webkit.org/show_bug.cgi?id=143301

Reviewed by Žan Doberšek.

This patch implements HTML5 Video supports in Threaded Compositor.

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:

(WebCore::GstVideoFrameHolder::GstVideoFrameHolder):
Added to support GStreamer GL by ensuring unmapping of the swapped
GstVideoFrame performed at GStreamer GL's gl thread.
(WebCore::MediaPlayerPrivateGStreamerBase::updateTexture):
Modified to upload decoded frame to the given texture instead of
creating a texture itself because we should use a texture from the
proxy when we are using the threaded compositor.
(WebCore::MediaPlayerPrivateGStreamerBase::pushTextureToCompositor):
Implements two ways to send a texture from GStreamer to the compositor.

  1. If we are not using GStreamer GL, we are going to acquire a free texture

from a TextureMapperPlatformLayerProxy and upload the decoded frame to the
texture. This should be done at the compositing thread because we
don't have a Gst's GL thread.

  1. If we are using GStreamer GL, we map a texture for the given frame

and passes it to the compositing thread. The mapped frame will be
freed if it is swapped out or the layer is removed.

(WebCore::MediaPlayerPrivateGStreamerBase::paintToTextureMapper):
Modified to aquire a new texture itself.

  • platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp:

Adds a way to pass a function to the compositing thread to allocate /
upload textures at the compositing thread.

Location:
trunk/Source/WebCore
Files:
5 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r193832 r193836  
     12015-12-09  Gwang Yoon Hwang  <yoon@igalia.com>
     2
     3        [ThreadedCompositor] Support HTML5 Video
     4        https://bugs.webkit.org/show_bug.cgi?id=143301
     5
     6        Reviewed by Žan Doberšek.
     7
     8        This patch implements HTML5 Video supports in Threaded Compositor.
     9
     10        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
     11        (WebCore::GstVideoFrameHolder::GstVideoFrameHolder):
     12        Added to support GStreamer GL by ensuring unmapping of the swapped
     13        GstVideoFrame performed at GStreamer GL's gl thread.
     14        (WebCore::MediaPlayerPrivateGStreamerBase::updateTexture):
     15        Modified to upload decoded frame to the given texture instead of
     16        creating a texture itself because we should use a texture from the
     17        proxy when we are using the threaded compositor.
     18        (WebCore::MediaPlayerPrivateGStreamerBase::pushTextureToCompositor):
     19        Implements two ways to send a texture from GStreamer to the compositor.
     20        1. If we are not using GStreamer GL, we are going to acquire a free texture
     21        from a TextureMapperPlatformLayerProxy and upload the decoded frame to the
     22        texture. This should be done at the compositing thread because we
     23        don't have a Gst's GL thread.
     24        2. If we are using GStreamer GL, we map a texture for the given frame
     25        and passes it to the compositing thread. The mapped frame will be
     26        freed if it is swapped out or the layer is removed.
     27
     28        (WebCore::MediaPlayerPrivateGStreamerBase::paintToTextureMapper):
     29        Modified to aquire a new texture itself.
     30
     31        * platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp:
     32        Adds a way to pass a function to the compositing thread to allocate /
     33        upload textures at the compositing thread.
     34
     35
    1362015-12-09  Xabier Rodriguez Calvar  <calvaris@igalia.com>
    237

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp

    r193621 r193836  
    5656#include "TextureMapperGL.h"
    5757#endif
     58#if USE(COORDINATED_GRAPHICS_THREADED)
     59#include "TextureMapperPlatformLayerBuffer.h"
     60#endif
    5861
    5962#if USE(GSTREAMER_GL)
    … …  
    103106}
    104107
     108#if USE(COORDINATED_GRAPHICS_THREADED) && USE(GSTREAMER_GL)
     109class GstVideoFrameHolder : public TextureMapperPlatformLayerBuffer::UnmanagedBufferDataHolder {
     110public:
     111    explicit GstVideoFrameHolder(GstSample* sample)
     112    {
     113        GstVideoInfo videoInfo;
     114        gst_video_info_init(&videoInfo);
     115        GstCaps* caps = gst_sample_get_caps(sample);
     116        if (UNLIKELY(!gst_video_info_from_caps(&videoInfo, caps)))
     117            return;
     118
     119        m_size = IntSize(GST_VIDEO_INFO_WIDTH(&videoInfo), GST_VIDEO_INFO_HEIGHT(&videoInfo));
     120        m_flags = GST_VIDEO_INFO_HAS_ALPHA(&videoInfo) ? TextureMapperGL::ShouldBlend : 0;
     121
     122        GstBuffer* buffer = gst_sample_get_buffer(sample);
     123        m_videoFrame = new GstVideoFrame();
     124        if (UNLIKELY(!gst_video_frame_map(m_videoFrame, &videoInfo, buffer, static_cast<GstMapFlags>(GST_MAP_READ | GST_MAP_GL)))) {
     125            delete m_videoFrame;
     126            return;
     127        }
     128
     129        m_textureID = *reinterpret_cast<GLuint*>(m_videoFrame->data[0]);
     130        m_isValid = true;
     131    }
     132
     133    virtual ~GstVideoFrameHolder()
     134    {
     135        if (UNLIKELY(!m_isValid))
     136            return;
     137
     138        GstMapInfo* info = &m_videoFrame->map[0];
     139        GstGLBaseBuffer* mem = (GstGLBaseBuffer*)info->memory;
     140        GstGLWindow* gstWindow = gst_gl_context_get_window(mem->context);
     141
     142        gst_gl_window_send_message_async(gstWindow, (GstGLWindowCB)unmapVideoFrameCallback, m_videoFrame, (GDestroyNotify)freeVideoFrameCallback);
     143    }
     144
     145    static void unmapVideoFrameCallback(GstVideoFrame* videoFrame)
     146    {
     147        gst_video_frame_unmap(videoFrame);
     148    }
     149
     150    static void freeVideoFrameCallback(GstVideoFrame* videoFrame)
     151    {
     152        delete videoFrame;
     153    }
     154
     155    const IntSize& size() const { return m_size; }
     156    TextureMapperGL::Flags flags() const { return m_flags; }
     157    GLuint textureID() const { return m_textureID; }
     158    bool isValid() const { return m_isValid; }
     159
     160private:
     161    GstVideoFrame* m_videoFrame;
     162    IntSize m_size;
     163    TextureMapperGL::Flags m_flags;
     164    GLuint m_textureID;
     165    bool m_isValid { false };
     166};
     167#endif
     168
    105169MediaPlayerPrivateGStreamerBase::MediaPlayerPrivateGStreamerBase(MediaPlayer* player)
    106170    : m_player(player)
    … …  
    114178{
    115179    g_mutex_init(&m_sampleMutex);
     180#if USE(COORDINATED_GRAPHICS_THREADED)
     181    m_platformLayerProxy = adoptRef(new TextureMapperPlatformLayerProxy());
     182#endif
    116183}
    117184
    … …  
    361428}
    362429
    363 #if USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS)
    364 PassRefPtr<BitmapTexture> MediaPlayerPrivateGStreamerBase::updateTexture(TextureMapper* textureMapper)
    365 {
    366     WTF::GMutexLocker<GMutex> lock(m_sampleMutex);
    367     if (!GST_IS_SAMPLE(m_sample.get()))
    368         return nullptr;
    369 
    370     GstCaps* caps = gst_sample_get_caps(m_sample.get());
    371     if (!caps)
    372         return nullptr;
    373 
    374     GstVideoInfo videoInfo;
    375     gst_video_info_init(&videoInfo);
    376     if (!gst_video_info_from_caps(&videoInfo, caps))
    377         return nullptr;
    378 
    379     IntSize size = IntSize(GST_VIDEO_INFO_WIDTH(&videoInfo), GST_VIDEO_INFO_HEIGHT(&videoInfo));
    380     RefPtr<BitmapTexture> texture = textureMapper->acquireTextureFromPool(size, GST_VIDEO_INFO_HAS_ALPHA(&videoInfo) ? BitmapTexture::SupportsAlpha : BitmapTexture::NoFlag);
     430#if USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS_MULTIPROCESS)
     431void MediaPlayerPrivateGStreamerBase::updateTexture(BitmapTextureGL& texture, GstVideoInfo& videoInfo)
     432{
    381433    GstBuffer* buffer = gst_sample_get_buffer(m_sample.get());
    382434
    … …  
    385437    if ((meta = gst_buffer_get_video_gl_texture_upload_meta(buffer))) {
    386438        if (meta->n_textures == 1) { // BRGx & BGRA formats use only one texture.
    387             const BitmapTextureGL* textureGL = static_cast<const BitmapTextureGL*>(texture.get());
    388             guint ids[4] = { textureGL->id(), 0, 0, 0 };
     439            guint ids[4] = { texture.id(), 0, 0, 0 };
    389440
    390441            if (gst_video_gl_texture_upload_meta_upload(meta, ids))
    391                 return texture;
     442                return;
    392443        }
    393444    }
    … …  
    399450    GstVideoFrame videoFrame;
    400451    if (!gst_video_frame_map(&videoFrame, &videoInfo, buffer, GST_MAP_READ))
    401         return nullptr;
     452        return;
    402453
    403454    int stride = GST_VIDEO_FRAME_PLANE_STRIDE(&videoFrame, 0);
    404455    const void* srcData = GST_VIDEO_FRAME_PLANE_DATA(&videoFrame, 0);
    405     texture->updateContents(srcData, WebCore::IntRect(WebCore::IntPoint(0, 0), size), WebCore::IntPoint(0, 0), stride, BitmapTexture::UpdateCannotModifyOriginalImageData);
     456    texture.updateContents(srcData, WebCore::IntRect(0, 0, GST_VIDEO_INFO_WIDTH(&videoInfo), GST_VIDEO_INFO_HEIGHT(&videoInfo)), WebCore::IntPoint(0, 0), stride, BitmapTexture::UpdateCannotModifyOriginalImageData);
    406457    gst_video_frame_unmap(&videoFrame);
    407 
    408     return texture;
     458}
     459#endif
     460
     461#if USE(COORDINATED_GRAPHICS_THREADED)
     462void MediaPlayerPrivateGStreamerBase::pushTextureToCompositor()
     463{
     464#if !USE(GSTREAMER_GL)
     465    class ConditionNotifier {
     466    public:
     467        ConditionNotifier(Lock& lock, Condition& condition)
     468            : m_locker(lock), m_condition(condition)
     469        {
     470        }
     471        ~ConditionNotifier()
     472        {
     473            m_condition.notifyOne();
     474        }
     475    private:
     476        LockHolder m_locker;
     477        Condition& m_condition;
     478    };
     479    ConditionNotifier notifier(m_drawMutex, m_drawCondition);
     480#endif
     481
     482    WTF::GMutexLocker<GMutex> lock(m_sampleMutex);
     483    if (!GST_IS_SAMPLE(m_sample.get()))
     484        return;
     485
     486    LockHolder holder(m_platformLayerProxy->lock());
     487
     488    if (!m_platformLayerProxy->isActive())
     489        return;
     490
     491#if USE(GSTREAMER_GL)
     492    std::unique_ptr<GstVideoFrameHolder> frameHolder = std::make_unique<GstVideoFrameHolder>(m_sample.get());
     493    if (UNLIKELY(!frameHolder->isValid()))
     494        return;
     495
     496    std::unique_ptr<TextureMapperPlatformLayerBuffer> layerBuffer = std::make_unique<TextureMapperPlatformLayerBuffer>(frameHolder->textureID(), frameHolder->size(), frameHolder->flags());
     497    layerBuffer->setUnmanagedBufferDataHolder(WTF::move(frameHolder));
     498    m_platformLayerProxy->pushNextBuffer(WTF::move(layerBuffer));
     499#else
     500    GstCaps* caps = gst_sample_get_caps(m_sample.get());
     501    if (UNLIKELY(!caps))
     502        return;
     503
     504    GstVideoInfo videoInfo;
     505    gst_video_info_init(&videoInfo);
     506    if (UNLIKELY(!gst_video_info_from_caps(&videoInfo, caps)))
     507        return;
     508
     509    IntSize size = IntSize(GST_VIDEO_INFO_WIDTH(&videoInfo), GST_VIDEO_INFO_HEIGHT(&videoInfo));
     510
     511    std::unique_ptr<TextureMapperPlatformLayerBuffer> buffer = m_platformLayerProxy->getAvailableBuffer(size, GraphicsContext3D::DONT_CARE);
     512    if (UNLIKELY(!buffer)) {
     513        if (UNLIKELY(!m_context3D))
     514            m_context3D = GraphicsContext3D::create(GraphicsContext3D::Attributes(), nullptr, GraphicsContext3D::RenderToCurrentGLContext);
     515
     516        RefPtr<BitmapTexture> texture = adoptRef(new BitmapTextureGL(m_context3D));
     517        texture->reset(size, GST_VIDEO_INFO_HAS_ALPHA(&videoInfo) ? BitmapTexture::SupportsAlpha : BitmapTexture::NoFlag);
     518        buffer = std::make_unique<TextureMapperPlatformLayerBuffer>(WTF::move(texture));
     519    }
     520    updateTexture(buffer->textureGL(), videoInfo);
     521    m_platformLayerProxy->pushNextBuffer(WTF::move(buffer));
     522#endif
    409523}
    410524#endif
    … …  
    439553    }
    440554
     555#if USE(COORDINATED_GRAPHICS_THREADED)
     556#if USE(GSTREAMER_GL)
     557    pushTextureToCompositor();
     558#else
     559    {
     560        LockHolder lock(m_drawMutex);
     561        if (!m_platformLayerProxy->scheduleUpdateOnCompositorThread([this] { this->pushTextureToCompositor(); }))
     562            return;
     563        m_drawCondition.wait(m_drawMutex);
     564    }
     565#endif
     566    return;
     567#else
    441568#if USE(GSTREAMER_GL)
    442569    {
    … …  
    450577    repaint();
    451578#endif
     579#endif
    452580}
    453581
    … …  
    472600void MediaPlayerPrivateGStreamerBase::paint(GraphicsContext& context, const FloatRect& rect)
    473601{
    474 #if USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS)
     602#if USE(COORDINATED_GRAPHICS_THREADED)
     603    ASSERT_NOT_REACHED();
     604    return;
     605#elif USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS)
    475606    if (client())
    476607        return;
    … …  
    502633
    503634    if (m_usingFallbackVideoSink) {
    504         if (RefPtr<BitmapTexture> texture = updateTexture(textureMapper))
    505             textureMapper->drawTexture(*texture.get(), targetRect, matrix, opacity);
     635        RefPtr<BitmapTexture> texture;
     636        {
     637            WTF::GMutexLocker<GMutex> lock(m_sampleMutex);
     638
     639            if (!m_sample)
     640                return;
     641
     642            GstCaps* caps = gst_sample_get_caps(m_sample.get());
     643            if (UNLIKELY(!caps))
     644                return;
     645
     646            GstVideoInfo videoInfo;
     647            gst_video_info_init(&videoInfo);
     648            if (UNLIKELY(!gst_video_info_from_caps(&videoInfo, caps)))
     649                return;
     650
     651            IntSize size = IntSize(GST_VIDEO_INFO_WIDTH(&videoInfo), GST_VIDEO_INFO_HEIGHT(&videoInfo));
     652            texture = textureMapper->acquireTextureFromPool(size, GST_VIDEO_INFO_HAS_ALPHA(&videoInfo) ? BitmapTexture::SupportsAlpha : BitmapTexture::NoFlag);
     653            updateTexture(static_cast<BitmapTextureGL&>(*texture), videoInfo);
     654        }
     655        textureMapper->drawTexture(*texture, targetRect, matrix, opacity);
    506656        return;
    507657    }

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h

    r193651 r193836  
    3636#include "TextureMapperPlatformLayer.h"
    3737#endif
     38#if USE(COORDINATED_GRAPHICS_THREADED)
     39#include "TextureMapperPlatformLayerProxy.h"
     40#endif
    3841
    3942typedef struct _GstMessage GstMessage;
    4043typedef struct _GstStreamVolume GstStreamVolume;
     44typedef struct _GstVideoInfo GstVideoInfo;
    4145typedef struct _GstGLContext GstGLContext;
    4246typedef struct _GstGLDisplay GstGLDisplay;
    … …  
    4448namespace WebCore {
    4549
     50class BitmapTextureGL;
    4651class GraphicsContext;
     52class GraphicsContext3D;
    4753class IntSize;
    4854class IntRect;
    … …  
    5157#if USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS)
    5258    , public TextureMapperPlatformLayer
     59#elif USE(COORDINATED_GRAPHICS_THREADED)
     60    , public TextureMapperPlatformLayerProxyProvider
    5361#endif
    5462{
    … …  
    106114#endif
    107115
     116#if USE(COORDINATED_GRAPHICS_THREADED)
     117    virtual PlatformLayer* platformLayer() const override { return const_cast<MediaPlayerPrivateGStreamerBase*>(this); }
     118    virtual bool supportsAcceleratedRendering() const override { return true; }
     119#endif
     120
    108121protected:
    109122    MediaPlayerPrivateGStreamerBase(MediaPlayer*);
    … …  
    156169#if USE(GSTREAMER_GL)
    157170    RunLoop::Timer<MediaPlayerPrivateGStreamerBase> m_drawTimer;
    158     Condition m_drawCondition;
    159     Lock m_drawMutex;
    160171#endif
    161172    mutable FloatSize m_videoSize;
    162173    bool m_usingFallbackVideoSink;
    163 #if USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS)
    164     PassRefPtr<BitmapTexture> updateTexture(TextureMapper*);
     174#if USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS_MULTIPROCESS)
     175    void updateTexture(BitmapTextureGL&, GstVideoInfo&);
    165176#endif
    166177#if USE(GSTREAMER_GL)
    167178    GRefPtr<GstGLContext> m_glContext;
    168179    GRefPtr<GstGLDisplay> m_glDisplay;
     180#endif
     181
     182#if USE(COORDINATED_GRAPHICS_THREADED)
     183    virtual RefPtr<TextureMapperPlatformLayerProxy> proxy() const override { return m_platformLayerProxy.copyRef(); }
     184    virtual void swapBuffersIfNeeded() override { };
     185    void pushTextureToCompositor();
     186    RefPtr<TextureMapperPlatformLayerProxy> m_platformLayerProxy;
     187#endif
     188
     189#if USE(GSTREAMER_GL) || USE(COORDINATED_GRAPHICS_THREADED)
     190    RefPtr<GraphicsContext3D> m_context3D;
     191    Condition m_drawCondition;
     192    Lock m_drawMutex;
    169193#endif
    170194};

  • trunk/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp

    r193630 r193836  
    6464    m_compositor = compositor;
    6565    m_targetLayer = targetLayer;
     66    m_compositorThreadUpdateTimer = std::make_unique<RunLoop::Timer<TextureMapperPlatformLayerProxy>>(RunLoop::current(), this, &TextureMapperPlatformLayerProxy::compositorThreadUpdateTimerFired);
    6667}
    6768
    … …  
    153154}
    154155
     156bool TextureMapperPlatformLayerProxy::scheduleUpdateOnCompositorThread(std::function<void()>&& updateFunction)
     157{
     158    LockHolder locker(m_lock);
     159    if (!m_compositorThreadUpdateTimer)
     160        return false;
     161
     162    m_compositorThreadUpdateFunction = WTF::move(updateFunction);
     163    m_compositorThreadUpdateTimer->startOneShot(0);
     164    return true;
     165}
     166
     167void TextureMapperPlatformLayerProxy::compositorThreadUpdateTimerFired()
     168{
     169    std::function<void()> updateFunction;
     170    {
     171        LockHolder locker(m_lock);
     172        if (!m_compositorThreadUpdateFunction)
     173            return;
     174        updateFunction = WTF::move(m_compositorThreadUpdateFunction);
     175    }
     176
     177    updateFunction();
     178}
     179
    155180} // namespace WebCore
    156181

  • trunk/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.h

    r193630 r193836  
    7676    void swapBuffer();
    7777
     78    bool scheduleUpdateOnCompositorThread(std::function<void()>&&);
     79
    7880private:
    7981    void scheduleReleaseUnusedBuffers();
    … …  
    9496    ThreadIdentifier m_compositorThreadID { 0 };
    9597#endif
     98
     99    void compositorThreadUpdateTimerFired();
     100    std::unique_ptr<RunLoop::Timer<TextureMapperPlatformLayerProxy>> m_compositorThreadUpdateTimer;
     101    std::function<void()> m_compositorThreadUpdateFunction;
    96102};
    97103
Note: See TracChangeset for help on using the changeset viewer.