Changeset 168060 in webkit


Ignore:
Timestamp:
Apr 30, 2014 4:17:36 PM (10 years ago)
Author:
vjaquez@igalia.com
Message:

[GStreamer] Use GstMetaVideo
​https://bugs.webkit.org/show_bug.cgi?id=132247

Reviewed by Philippe Normand.

In WebKitVideoSink we announce the usage of GstMetaVideo, but we do
not use it when handling the video frames. This might break
some decoders and filters that rely on buffer's meta, rather
that in the caps structures.

This patch enables the use of GstMetaVideo through the GstVideoFrame
API. And it is used everywhere the buffer mapping is required.

Also this patch changes to nullptr where zeros were used.

Also, compile conditionally the video buffer conversion when it is
ARGB/BGRA, since it is only required for the Cairo backend.

No new tests, already covered by current tests.

  • platform/graphics/gstreamer/GStreamerUtilities.cpp:

(WebCore::getVideoSizeAndFormatFromCaps): init the GstVideoInfo before
used and remove caps fixate check since it is done by
gst_video_info_from_caps().

  • platform/graphics/gstreamer/ImageGStreamer.h:
  • platform/graphics/gstreamer/ImageGStreamerCairo.cpp:

(ImageGStreamer::ImageGStreamer): use GstVideoFrame for buffer mapping
and unmapping.
(ImageGStreamer::~ImageGStreamer): ditto.

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:

(WebCore::MediaPlayerPrivateGStreamerBase::updateTexture): ditto.
(WebCore::MediaPlayerPrivateGStreamerBase::currentVideoSinkCaps):
return nullptr if failed.

  • platform/graphics/gstreamer/VideoSinkGStreamer.cpp:

(webkitVideoSinkRender): rely on GstVideoInfo rather than on the
caps. Use GstVideoFrame for buffer mapping and unmapping. Add guards
for buffer transformation, since it's only used by Cairo.
(webkitVideoSinkDispose): remove glib version guards.
(webkitVideoSinkSetCaps): update the value of the private
GstVideoInfo.

Location:
trunk/Source/WebCore
Files:
6 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r168059 r168060  
     12014-04-30  Víctor Manuel Jáquez Leal  <vjaquez@igalia.com>
     2
     3        [GStreamer] Use GstMetaVideo
     4        https://bugs.webkit.org/show_bug.cgi?id=132247
     5
     6        Reviewed by Philippe Normand.
     7
     8        In WebKitVideoSink we announce the usage of GstMetaVideo, but we do
     9        not use it when handling the video frames. This might break
     10        some decoders and filters that rely on buffer's meta, rather
     11        that in the caps structures.
     12
     13        This patch enables the use of GstMetaVideo through the GstVideoFrame
     14        API. And it is used everywhere the buffer mapping is required.
     15
     16        Also this patch changes to nullptr where zeros were used.
     17
     18        Also, compile conditionally the video buffer conversion when it is
     19        ARGB/BGRA, since it is only required for the Cairo backend.
     20
     21        No new tests, already covered by current tests.
     22
     23        * platform/graphics/gstreamer/GStreamerUtilities.cpp:
     24        (WebCore::getVideoSizeAndFormatFromCaps): init the GstVideoInfo before
     25        used and remove caps fixate check since it is done by
     26        gst_video_info_from_caps().
     27        * platform/graphics/gstreamer/ImageGStreamer.h:
     28        * platform/graphics/gstreamer/ImageGStreamerCairo.cpp:
     29        (ImageGStreamer::ImageGStreamer): use GstVideoFrame for buffer mapping
     30        and unmapping.
     31        (ImageGStreamer::~ImageGStreamer): ditto.
     32        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
     33        (WebCore::MediaPlayerPrivateGStreamerBase::updateTexture): ditto.
     34        (WebCore::MediaPlayerPrivateGStreamerBase::currentVideoSinkCaps):
     35        return nullptr if failed.
     36        * platform/graphics/gstreamer/VideoSinkGStreamer.cpp:
     37        (webkitVideoSinkRender): rely on GstVideoInfo rather than on the
     38        caps. Use GstVideoFrame for buffer mapping and unmapping. Add guards
     39        for buffer transformation, since it's only used by Cairo.
     40        (webkitVideoSinkDispose): remove glib version guards.
     41        (webkitVideoSinkSetCaps): update the value of the private
     42        GstVideoInfo.
     43
    1442014-04-30  Víctor Manuel Jáquez Leal  <vjaquez@igalia.com>
    245

  • trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerUtilities.cpp

    r164362 r168060  
    5353    GstVideoInfo info;
    5454
    55     if (!gst_caps_is_fixed(caps) || !gst_video_info_from_caps(&info, caps))
     55    gst_video_info_init(&info);
     56    if (!gst_video_info_from_caps(&info, caps))
    5657        return false;
    5758

  • trunk/Source/WebCore/platform/graphics/gstreamer/ImageGStreamer.h

    r159730 r168060  
    2828
    2929#include <gst/gst.h>
     30#include <gst/video/video.h>
    3031
    3132#include <wtf/PassRefPtr.h>
    … …  
    6566
    6667#if USE(CAIRO)
    67         GRefPtr<GstBuffer> m_buffer;
    68         GstMapInfo m_mapInfo;
     68        GstVideoFrame m_videoFrame;
    6969#endif
    7070    };

  • trunk/Source/WebCore/platform/graphics/gstreamer/ImageGStreamerCairo.cpp

    r162599 r168060  
    3434
    3535ImageGStreamer::ImageGStreamer(GstBuffer* buffer, GstCaps* caps)
    36     : m_buffer(buffer)
    3736{
    38     GstVideoFormat format;
    39     IntSize size;
    40     int pixelAspectRatioNumerator, pixelAspectRatioDenominator, stride;
    41     getVideoSizeAndFormatFromCaps(caps, size, format, pixelAspectRatioNumerator, pixelAspectRatioDenominator, stride);
     37    GstVideoInfo videoInfo;
     38    gst_video_info_init(&videoInfo);
     39    if (!gst_video_info_from_caps(&videoInfo, caps))
     40        return;
    4241
    43     gst_buffer_map(buffer, &m_mapInfo, GST_MAP_READ);
    44     unsigned char* bufferData = reinterpret_cast<unsigned char*>(m_mapInfo.data);
     42    // Right now the TextureMapper only supports chromas with one plane
     43    ASSERT(GST_VIDEO_INFO_N_PLANES(&videoInfo) == 1);
     44
     45    if (!gst_video_frame_map(&m_videoFrame, &videoInfo, buffer, GST_MAP_READ))
     46        return;
     47
     48    unsigned char* bufferData = reinterpret_cast<unsigned char*>(GST_VIDEO_FRAME_PLANE_DATA(&m_videoFrame, 0));
    4549
    4650    cairo_format_t cairoFormat;
    4751#if G_BYTE_ORDER == G_LITTLE_ENDIAN
    48     cairoFormat = (format == GST_VIDEO_FORMAT_BGRA) ? CAIRO_FORMAT_ARGB32 : CAIRO_FORMAT_RGB24;
     52    cairoFormat = (GST_VIDEO_FRAME_FORMAT(&m_videoFrame) == GST_VIDEO_FORMAT_BGRA) ? CAIRO_FORMAT_ARGB32 : CAIRO_FORMAT_RGB24;
    4953#else
    50     cairoFormat = (format == GST_VIDEO_FORMAT_ARGB) ? CAIRO_FORMAT_ARGB32 : CAIRO_FORMAT_RGB24;
     54    cairoFormat = (GST_VIDEO_FRAME_FORMAT(&m_videoFrame) == GST_VIDEO_FORMAT_ARGB) ? CAIRO_FORMAT_ARGB32 : CAIRO_FORMAT_RGB24;
    5155#endif
    5256
    53     RefPtr<cairo_surface_t> surface = adoptRef(cairo_image_surface_create_for_data(bufferData, cairoFormat, size.width(), size.height(), stride));
     57    int stride = GST_VIDEO_FRAME_PLANE_STRIDE(&m_videoFrame, 0);
     58    int width = GST_VIDEO_FRAME_WIDTH(&m_videoFrame);
     59    int height = GST_VIDEO_FRAME_HEIGHT(&m_videoFrame);
     60
     61    RefPtr<cairo_surface_t> surface = adoptRef(cairo_image_surface_create_for_data(bufferData, cairoFormat, width, height, stride));
    5462    ASSERT(cairo_surface_status(surface.get()) == CAIRO_STATUS_SUCCESS);
    5563    m_image = BitmapImage::create(surface.release());
    … …  
    6876    // We keep the buffer memory mapped until the image is destroyed because the internal
    6977    // cairo_surface_t was created using cairo_image_surface_create_for_data().
    70     gst_buffer_unmap(m_buffer.get(), &m_mapInfo);
     78    gst_video_frame_unmap(&m_videoFrame);
    7179}
    7280#endif // USE(GSTREAMER)

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp

    r168059 r168060  
    4343
    4444#include <gst/audio/streamvolume.h>
     45#include <gst/video/gstvideometa.h>
    4546
    4647#if GST_CHECK_VERSION(1, 1, 0) && USE(TEXTURE_MAPPER_GL)
    … …  
    271272    GMutexLocker lock(m_bufferMutex);
    272273    if (!m_buffer)
    273         return 0;
    274 
    275     const void* srcData = 0;
     274        return nullptr;
     275
    276276    GRefPtr<GstCaps> caps = currentVideoSinkCaps();
    277277    if (!caps)
    278         return 0;
    279 
    280     IntSize size;
    281     GstVideoFormat format;
    282     int pixelAspectRatioNumerator, pixelAspectRatioDenominator, stride;
    283     if (!getVideoSizeAndFormatFromCaps(caps.get(), size, format, pixelAspectRatioNumerator, pixelAspectRatioDenominator, stride))
    284         return 0;
    285 
    286     const GstVideoFormatInfo* formatInfo = gst_video_format_get_info(format);
    287 
    288     RefPtr<BitmapTexture> texture = textureMapper->acquireTextureFromPool(size, GST_VIDEO_FORMAT_INFO_HAS_ALPHA(formatInfo) ? BitmapTexture::SupportsAlpha : BitmapTexture::NoFlag);
     278        return nullptr;
     279
     280    GstVideoInfo videoInfo;
     281    gst_video_info_init(&videoInfo);
     282    if (!gst_video_info_from_caps(&videoInfo, caps.get()))
     283        return nullptr;
     284
     285    IntSize size = IntSize(GST_VIDEO_INFO_WIDTH(&videoInfo), GST_VIDEO_INFO_HEIGHT(&videoInfo));
     286    RefPtr<BitmapTexture> texture = textureMapper->acquireTextureFromPool(size, GST_VIDEO_INFO_HAS_ALPHA(&videoInfo) ? BitmapTexture::SupportsAlpha : BitmapTexture::NoFlag);
    289287
    290288#if GST_CHECK_VERSION(1, 1, 0)
    … …  
    301299#endif
    302300
    303     GstMapInfo srcInfo;
    304     gst_buffer_map(m_buffer, &srcInfo, GST_MAP_READ);
    305     srcData = srcInfo.data;
    306 
     301    // Right now the TextureMapper only supports chromas with one plane
     302    ASSERT(GST_VIDEO_INFO_N_PLANES(&videoInfo) == 1);
     303
     304    GstVideoFrame videoFrame;
     305    if (!gst_video_frame_map(&videoFrame, &videoInfo, m_buffer, GST_MAP_READ))
     306        return nullptr;
     307
     308    int stride = GST_VIDEO_FRAME_PLANE_STRIDE(&videoFrame, 0);
     309    const void* srcData = GST_VIDEO_FRAME_PLANE_DATA(&videoFrame, 0);
    307310    texture->updateContents(srcData, WebCore::IntRect(WebCore::IntPoint(0, 0), size), WebCore::IntPoint(0, 0), stride, BitmapTexture::UpdateCannotModifyOriginalImageData);
    308 
    309     gst_buffer_unmap(m_buffer, &srcInfo);
     311    gst_video_frame_unmap(&videoFrame);
     312
    310313    return texture;
    311314}
    … …  
    404407{
    405408    if (!m_webkitVideoSink)
    406         return 0;
     409        return nullptr;
    407410
    408411    GRefPtr<GstCaps> currentCaps;

  • trunk/Source/WebCore/platform/graphics/gstreamer/VideoSinkGStreamer.cpp

    r168059 r168060  
    141141    priv->buffer = gst_buffer_ref(buffer);
    142142
    143     GRefPtr<GstCaps> caps;
    144143    // The video info structure is valid only if the sink handled an allocation query.
    145     if (GST_VIDEO_INFO_FORMAT(&priv->info) != GST_VIDEO_FORMAT_UNKNOWN)
    146         caps = adoptGRef(gst_video_info_to_caps(&priv->info));
    147     else
    148         caps = priv->currentCaps;
    149 
    150     GstVideoFormat format;
    151     WebCore::IntSize size;
    152     int pixelAspectRatioNumerator, pixelAspectRatioDenominator, stride;
    153     if (!getVideoSizeAndFormatFromCaps(caps.get(), size, format, pixelAspectRatioNumerator, pixelAspectRatioDenominator, stride)) {
     144    GstVideoFormat format = GST_VIDEO_INFO_FORMAT(&priv->info);
     145    if (format == GST_VIDEO_FORMAT_UNKNOWN) {
    154146        gst_buffer_unref(buffer);
    155147        return GST_FLOW_ERROR;
    156148    }
    157149
     150#if !(USE(TEXTURE_MAPPER_GL) && !USE(COORDINATED_GRAPHICS))
    158151    // Cairo's ARGB has pre-multiplied alpha while GStreamer's doesn't.
    159152    // Here we convert to Cairo's ARGB.
    … …  
    167160
    168161        // Check if allocation failed.
    169         if (UNLIKELY(!newBuffer))
     162        if (UNLIKELY(!newBuffer)) {
     163            gst_buffer_unref(buffer);
    170164            return GST_FLOW_ERROR;
     165        }
    171166
    172167        // We don't use Color::premultipliedARGBFromColor() here because
    … …  
    174169        // For 720p/PAL for example this means 1280*720*25=23040000
    175170        // function calls per second!
    176         GstMapInfo sourceInfo;
    177         GstMapInfo destinationInfo;
    178         gst_buffer_map(buffer, &sourceInfo, GST_MAP_READ);
    179         const guint8* source = const_cast<guint8*>(sourceInfo.data);
    180         gst_buffer_map(newBuffer, &destinationInfo, GST_MAP_WRITE);
    181         guint8* destination = static_cast<guint8*>(destinationInfo.data);
    182 
    183         for (int x = 0; x < size.height(); x++) {
    184             for (int y = 0; y < size.width(); y++) {
     171        GstVideoFrame sourceFrame;
     172        GstVideoFrame destinationFrame;
     173
     174        if (!gst_video_frame_map(&sourceFrame, &priv->info, buffer, GST_MAP_READ)) {
     175            gst_buffer_unref(buffer);
     176            gst_buffer_unref(newBuffer);
     177            return GST_FLOW_ERROR;
     178        }
     179        if (!gst_video_frame_map(&destinationFrame, &priv->info, newBuffer, GST_MAP_WRITE)) {
     180            gst_video_frame_unmap(&sourceFrame);
     181            gst_buffer_unref(buffer);
     182            gst_buffer_unref(newBuffer);
     183            return GST_FLOW_ERROR;
     184        }
     185
     186        const guint8* source = static_cast<guint8*>(GST_VIDEO_FRAME_PLANE_DATA(&sourceFrame, 0));
     187        guint8* destination = static_cast<guint8*>(GST_VIDEO_FRAME_PLANE_DATA(&destinationFrame, 0));
     188
     189        for (int x = 0; x < GST_VIDEO_FRAME_HEIGHT(&sourceFrame); x++) {
     190            for (int y = 0; y < GST_VIDEO_FRAME_WIDTH(&sourceFrame); y++) {
    185191#if G_BYTE_ORDER == G_LITTLE_ENDIAN
    186192                unsigned short alpha = source[3];
    … …  
    201207        }
    202208
    203         gst_buffer_unmap(buffer, &sourceInfo);
    204         gst_buffer_unmap(newBuffer, &destinationInfo);
     209        gst_video_frame_unmap(&sourceFrame);
     210        gst_video_frame_unmap(&destinationFrame);
    205211        gst_buffer_unref(buffer);
    206212        buffer = priv->buffer = newBuffer;
    207213    }
     214#endif
    208215
    209216    // This should likely use a lower priority, but glib currently starves
    … …  
    327334    GST_DEBUG_OBJECT(sink, "Current caps %" GST_PTR_FORMAT ", setting caps %" GST_PTR_FORMAT, priv->currentCaps, caps);
    328335
    329     GstVideoInfo info;
    330     if (!gst_video_info_from_caps(&info, caps)) {
     336    GstVideoInfo videoInfo;
     337    gst_video_info_init(&videoInfo);
     338    if (!gst_video_info_from_caps(&videoInfo, caps)) {
    331339        GST_ERROR_OBJECT(sink, "Invalid caps %" GST_PTR_FORMAT, caps);
    332340        return FALSE;
    333341    }
    334342
     343    priv->info = videoInfo;
    335344    gst_caps_replace(&priv->currentCaps, caps);
    336345    return TRUE;
Note: See TracChangeset for help on using the changeset viewer.