Changeset 284439 in webkit


Ignore:
Timestamp:
Oct 19, 2021 1:35:40 AM (9 months ago)
Author:
Cameron McCormack
Message:

Ensure CanvasRenderingContext2D.drawImage(video) uses the right color space
​https://bugs.webkit.org/show_bug.cgi?id=229025
<rdar://problem/81828490>

Reviewed by Sam Weinig.

LayoutTests/imported/w3c:

Add some new WPTs for drawImage(video).

  • web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-drawImage.html:
  • web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-flipY.html:

Updated tests to check that video color space conversion is done
correctly.

  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video-expected.txt: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video-expected.txt: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3.js:

(pixelsApproximatelyEqual): Tweaked threshold to pass video tests on iOS.

  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-222000000.mp4: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-222000000.webm: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-3FF000000.mp4: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-3FF000000.webm: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-BB0000.mp4: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-BB0000.webm: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-FF0100.mp4: Added.
  • web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-FF0100.webm: Added.

Source/WebCore:

When GPU process media handling is disabled (and in the GPU process
when it is handling a NativeImageForCurrentTime message),
MediaPlayer::nativeImageForCurrentTime is used to get the video frame
image. We make this return an image with the right color space on
Apple platforms in PixelBufferConformerCV::createImageFromPixelBuffer.

For now, we avoid using the correct color space when passing
HTMLVideoElements to WebGLRenderingContext.texImage2D. The
specification is unclear about what color space the texture data
should be in, and implementations are currently interoperable in not
converting the RGB color space of the video (such as Rec.601, Rec.709,
etc.) into sRGB. To avoid the conversion, and effectively reinterpret
the video RGB colors as sRGB, we make the ImageBuffer the video is
drawn into have a matching color space, as it's the raw RGB values
from the ImageBuffer that are uploade as the texture.

(An alternative to this could be passing an argument through
nativeImageForCurrentTime and paintCurrentFrameInContext, which says
to avoid doing the conversion from video RGB to sRGB, but that needs a
fair amount of plumbing.)

If ​https://github.com/KhronosGroup/WebGL/issues/2165 is resolved, the
changes to LRUImageBufferCache in this commit can be reverted, so that
video frames are drawn into sRGB ImageBuffers again. (We'd also need
something to make the copyToPlatformTexture() call in
texImageSourceHelper do the correct color conversion somehow.)

The now differing behavior between CanvasRenderingContext2D.drawImage
and WebGLRenderingContext.texImage2D means that the WebGL conformance
suite textures/misc/texture-corner-case-videos.html test fails, since
it checks for incorrect 2D canvas video frame color conversion.

Tests: imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html

imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html

  • html/HTMLVideoElement.cpp:

(WebCore::HTMLVideoElement::colorSpace const): New function that
exposes the color space of the current frame of the video.

  • html/HTMLVideoElement.h:
  • html/ImageBitmap.cpp:

(WebCore::ImageBitmap::createPromise): Create the ImageBuffer in a
color space that matches the video, not always sRGB.

  • html/canvas/WebGLRenderingContextBase.cpp:

(WebCore::WebGLRenderingContextBase::drawImageIntoBuffer):
(WebCore::WebGLRenderingContextBase::videoFrameToImage): Use an
ImageBuffer whose color space matches the video frame's color space.
(WebCore::WebGLRenderingContextBase::LRUImageBufferCache::imageBuffer):
Add a DestinationColorSpace argument, so that the LRU cache is keyed
off both size and color space.

  • html/canvas/WebGLRenderingContextBase.h:
  • platform/cocoa/CoreVideoSoftLink.cpp:
  • platform/cocoa/CoreVideoSoftLink.h: Add SPIs that let us get or

compute a CGColorSpace from a CVPixelBuffer. Remove
CVImageBufferGetColorSpace, which is unsused.

  • platform/graphics/MediaPlayer.cpp:

(WebCore::MediaPlayer::colorSpace):

  • platform/graphics/MediaPlayer.h:
  • platform/graphics/MediaPlayerPrivate.h:

(WebCore::MediaPlayerPrivateInterface::colorSpace):

  • platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
  • platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp:

(WebCore::MediaPlayerPrivateAVFoundationCF::colorSpace):

  • platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::colorSpace): Expose the
color space of the current image. While we strictly only need the pixel
buffer to be up to date, we call updateLastImage() since it has the
UpdateSynchronously behavior, and the caller to colorSpace() will use
the image soon anyway.

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:

(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::colorSpace):

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:

(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::colorSpace):

  • platform/graphics/cg/ColorSpaceCG.h:
  • platform/graphics/cv/CVUtilities.h:
  • platform/graphics/cv/CVUtilities.mm:

(WebCore::createCGColorSpaceForCVPixelBuffer): New function to produce
a CGColorSpace for a CVPixelBuffer. On macOS, this is stored as an
attachment on the CVPixelBuffer. On iOS, we must call SPI to create a
CGColorSpace based on the other color space information in the
attachments. (It's possible we should cache the results of this
function for a given set of color space attachments, since
CVImageBufferCreateColorSpaceFromAttachments can end up creating a new
CGColorSpace object every time it's called.)

  • platform/graphics/cv/PixelBufferConformerCV.cpp:

(WebCore::PixelBufferConformerCV::createImageFromPixelBuffer): Create
the CGImage in a color space matching the CVPixelBuffer.

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:

(WebCore::MediaPlayerPrivateGStreamer::colorSpace):

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
  • platform/graphics/holepunch/MediaPlayerPrivateHolePunch.h:
  • platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp:

(WebCore::MediaPlayerPrivateMediaFoundation::colorSpace):

  • platform/graphics/win/MediaPlayerPrivateMediaFoundation.h:
  • platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:

(WebCore::MockMediaPlayerMediaSource::colorSpace):

  • platform/mock/mediasource/MockMediaPlayerMediaSource.h:
  • svg/graphics/SVGImage.cpp:

(WebCore::SVGImage::drawInternal):

Source/WebKit:

When GPU process media handling is enabled, ctx.drawImage(video)
calls work by sending a NativeImageForCurrentTime message to the GPU
process, which returns an IOSurface around which a CGImage is created.
There's no good way to get the color space off the IOSurface, so we
add a second return argument to NativeImageForCurrentTime to indicate
the color space used.

ImageBitmap creates an ImageBuffer for that image to be painted into,
and for that we also need to know the color space of the video. The
code that sends NativeImageForCurrentTime and then paints the
resulting image into the GraphicsContext is further down. So we add
a separate IPC message that gets the color space of the video's
current frame. (It would be better if we could avoid a second
synchronous IPC message call, but either ImageBitmap's scaling and
cropping code, or the ImageBuffer creation call, would need to be moved
down into MediaPlayerPrivateRemote::paintCurrentFrameInContext, which
doesn't seem like the right place for it.)

  • GPUProcess/media/RemoteImageDecoderAVFProxy.cpp:

(WebKit::RemoteImageDecoderAVFProxy::createFrameImageAtIndex):

  • GPUProcess/media/RemoteImageDecoderAVFProxy.h:
  • GPUProcess/media/RemoteMediaPlayerProxy.h:
  • GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
  • GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:

(WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime):
(WebKit::RemoteMediaPlayerProxy::colorSpace):

  • WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:

(WebKit::MediaPlayerPrivateRemote::colorSpace):

  • WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
  • WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:

(WebKit::MediaPlayerPrivateRemote::nativeImageForCurrentTime): Add a
second return argument to NativeImageForCurrentTime to indicate the
color space of the returned IOSurface.
(WebKit::MediaPlayerPrivateRemote::colorSpace): Add a new IPC message to
get the color space of a video's current frame.

Source/WTF:

  • wtf/PlatformHave.h: Add HAVE_CVBUFFERCOPYATTACHMENTS.

LayoutTests:

texture-corner-case-videos.html relies on incorrect color space handling
with drawImage() on a 2D canvas and so now fails.

  • platform/glib/TestExpectations:
  • platform/ios-14/TestExpectations:
  • platform/ios-device/TestExpectations:
  • platform/ios/TestExpectations:
  • platform/mac-wk2/TestExpectations:
  • platform/mac/TestExpectations:

New tests pass only on macOS Monterey+ and iOS 15+.

  • fast/canvas/canvas-createPattern-video-loading-expected.txt:
  • fast/canvas/canvas-createPattern-video-loading.html:
  • fast/canvas/canvas-createPattern-video-modify-expected.txt:
  • fast/canvas/canvas-createPattern-video-modify.html:
  • media/video-canvas-createPattern-expected.txt:
  • media/video-canvas-createPattern.html:
  • media/video-canvas-drawing-output-expected.txt:
  • media/video-canvas-drawing-output.html:
  • media/video-orientation-canvas-expected.txt:
  • media/video-orientation-canvas.html:

Update some tests to check for correctly converted color values.

  • platform/mac/media/video-canvas-createPattern-expected.png: Removed.
  • platform/wpe/media/video-canvas-createPattern-expected.png: Removed.
  • platform/mac-wk1/imported/w3c/web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-flipY-expected.txt:

An existing failure that changed a bit. This is
​https://bugs.webkit.org/show_bug.cgi?id=231001.

Location:
trunk
Files:
22 added
2 deleted
66 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r284434 r284439  
     12021-10-19  Cameron McCormack  <heycam@apple.com>
     2
     3        Ensure CanvasRenderingContext2D.drawImage(video) uses the right color space
     4        https://bugs.webkit.org/show_bug.cgi?id=229025
     5        <rdar://problem/81828490>
     6
     7        Reviewed by Sam Weinig.
     8
     9        * TestExpectations:
     10        texture-corner-case-videos.html relies on incorrect color space handling
     11        with drawImage() on a 2D canvas and so now fails.
     12
     13        * platform/glib/TestExpectations:
     14        * platform/ios-14/TestExpectations:
     15        * platform/ios-device/TestExpectations:
     16        * platform/ios/TestExpectations:
     17        * platform/mac-wk2/TestExpectations:
     18        * platform/mac/TestExpectations:
     19        New tests pass only on macOS Monterey+ and iOS 15+.
     20
     21        * fast/canvas/canvas-createPattern-video-loading-expected.txt:
     22        * fast/canvas/canvas-createPattern-video-loading.html:
     23        * fast/canvas/canvas-createPattern-video-modify-expected.txt:
     24        * fast/canvas/canvas-createPattern-video-modify.html:
     25        * media/video-canvas-createPattern-expected.txt:
     26        * media/video-canvas-createPattern.html:
     27        * media/video-canvas-drawing-output-expected.txt:
     28        * media/video-canvas-drawing-output.html:
     29        * media/video-orientation-canvas-expected.txt:
     30        * media/video-orientation-canvas.html:
     31        Update some tests to check for correctly converted color values.
     32
     33        * platform/mac/media/video-canvas-createPattern-expected.png: Removed.
     34        * platform/wpe/media/video-canvas-createPattern-expected.png: Removed.
     35
     36        * platform/mac-wk1/imported/w3c/web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-flipY-expected.txt:
     37        An existing failure that changed a bit. This is
     38        https://bugs.webkit.org/show_bug.cgi?id=231001.
     39
    1402021-10-18  Jean-Yves Avenard  <jya@apple.com>
    241

  • trunk/LayoutTests/TestExpectations

    r284397 r284439  
    35263526webgl/1.0.x/conformance/misc/invalid-passed-params.html [ Pass ]
    35273527webgl/1.0.x/conformance/glsl/bugs/character-set.html [ Pass ]
    3528 webgl/1.0.x/conformance/textures/misc/texture-corner-case-videos.html [ Pass ]
    35293528webgl/1.0.x/conformance/glsl/misc/fragcolor-fragdata-invariant.html [ Pass ]
    35303529webgl/1.0.x/conformance/context/constants-and-properties.html [ Pass ]
    … …  
    35383537webgl/2.0.y/conformance/misc/invalid-passed-params.html [ Pass ]
    35393538webgl/2.0.y/conformance/glsl/bugs/character-set.html [ Pass ]
    3540 webgl/2.0.y/conformance/textures/misc/texture-corner-case-videos.html [ Pass ]
    35413539webgl/2.0.y/conformance/glsl/misc/fragcolor-fragdata-invariant.html [ Pass ]
    35423540webgl/2.0.y/conformance2/context/constants-and-properties-2.html [ Pass ]
    … …  
    35563554webgl/2.0.0/conformance/glsl/misc/shader-with-define-line-continuation.frag.html [ Skip ]
    35573555webgl/2.0.0/conformance/misc/invalid-passed-params.html [ Skip ]
     3556
     3557# https://github.com/KhronosGroup/WebGL/issues/3341
     3558webgl/1.0.x/conformance/textures/misc/texture-corner-case-videos.html [ Failure ]
     3559webgl/2.0.y/conformance/textures/misc/texture-corner-case-videos.html [ Failure ]
    35583560
    35593561# pre-wrap progression. Other rendering engines agree with the result.
    … …  
    50555057imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-cloned.html [ Failure ]
    50565058imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-image.html [ Failure ]
     5059imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html [ Failure ]
     5060imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html [ Failure ]
    50575061imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-canvas.html [ Failure ]
    50585062imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-image.html [ Failure ]

  • trunk/LayoutTests/fast/canvas/canvas-createPattern-video-loading-expected.txt

    r201724 r284439  
    55
    66PASS document.createElement('canvas').getContext('2d').createPattern(video, 'repeat') is null
    7 PASS buffer[0] is within 2 of 194
    8 PASS buffer[1] is within 2 of 193
    9 PASS buffer[2] is within 2 of 14
    10 PASS buffer[0] is within 2 of 194
    11 PASS buffer[1] is within 2 of 193
    12 PASS buffer[2] is within 2 of 14
    13 PASS buffer[0] is within 2 of 194
    14 PASS buffer[1] is within 2 of 193
    15 PASS buffer[2] is within 2 of 14
    16 PASS buffer[0] is within 2 of 194
    17 PASS buffer[1] is within 2 of 193
    18 PASS buffer[2] is within 2 of 14
    19 PASS buffer[0] is within 2 of 17
    20 PASS buffer[1] is within 2 of 197
    21 PASS buffer[2] is within 2 of 195
    22 PASS buffer[0] is within 2 of 17
    23 PASS buffer[1] is within 2 of 197
    24 PASS buffer[2] is within 2 of 195
    25 PASS buffer[0] is within 2 of 17
    26 PASS buffer[1] is within 2 of 197
    27 PASS buffer[2] is within 2 of 195
    28 PASS buffer[0] is within 2 of 17
    29 PASS buffer[1] is within 2 of 197
    30 PASS buffer[2] is within 2 of 195
    31 PASS buffer[0] is within 2 of 20
     7PASS buffer[0] is within 2 of 200
    328PASS buffer[1] is within 2 of 200
    33 PASS buffer[2] is within 2 of 18
    34 PASS buffer[0] is within 2 of 20
     9PASS buffer[2] is within 2 of 0
     10PASS buffer[0] is within 2 of 200
    3511PASS buffer[1] is within 2 of 200
    36 PASS buffer[2] is within 2 of 18
    37 PASS buffer[0] is within 2 of 20
     12PASS buffer[2] is within 2 of 0
     13PASS buffer[0] is within 2 of 200
    3814PASS buffer[1] is within 2 of 200
    39 PASS buffer[2] is within 2 of 18
    40 PASS buffer[0] is within 2 of 20
     15PASS buffer[2] is within 2 of 0
     16PASS buffer[0] is within 2 of 200
    4117PASS buffer[1] is within 2 of 200
    42 PASS buffer[2] is within 2 of 18
    43 PASS buffer[0] is within 2 of 194
    44 PASS buffer[1] is within 2 of 193
    45 PASS buffer[2] is within 2 of 14
    46 PASS buffer[0] is within 2 of 194
    47 PASS buffer[1] is within 2 of 193
    48 PASS buffer[2] is within 2 of 14
    49 PASS buffer[0] is within 2 of 194
    50 PASS buffer[1] is within 2 of 193
    51 PASS buffer[2] is within 2 of 14
    52 PASS buffer[0] is within 2 of 194
    53 PASS buffer[1] is within 2 of 193
    54 PASS buffer[2] is within 2 of 14
    55 PASS buffer[0] is within 2 of 17
    56 PASS buffer[1] is within 2 of 197
    57 PASS buffer[2] is within 2 of 195
    58 PASS buffer[0] is within 2 of 17
    59 PASS buffer[1] is within 2 of 197
    60 PASS buffer[2] is within 2 of 195
    61 PASS buffer[0] is within 2 of 17
    62 PASS buffer[1] is within 2 of 197
    63 PASS buffer[2] is within 2 of 195
    64 PASS buffer[0] is within 2 of 17
    65 PASS buffer[1] is within 2 of 197
    66 PASS buffer[2] is within 2 of 195
    67 PASS buffer[0] is within 2 of 20
     18PASS buffer[2] is within 2 of 0
     19PASS buffer[0] is within 2 of 57
     20PASS buffer[1] is within 2 of 202
     21PASS buffer[2] is within 2 of 203
     22PASS buffer[0] is within 2 of 57
     23PASS buffer[1] is within 2 of 202
     24PASS buffer[2] is within 2 of 203
     25PASS buffer[0] is within 2 of 57
     26PASS buffer[1] is within 2 of 202
     27PASS buffer[2] is within 2 of 203
     28PASS buffer[0] is within 2 of 57
     29PASS buffer[1] is within 2 of 202
     30PASS buffer[2] is within 2 of 203
     31PASS buffer[0] is within 2 of 53
     32PASS buffer[1] is within 2 of 203
     33PASS buffer[2] is within 2 of 9
     34PASS buffer[0] is within 2 of 53
     35PASS buffer[1] is within 2 of 203
     36PASS buffer[2] is within 2 of 9
     37PASS buffer[0] is within 2 of 53
     38PASS buffer[1] is within 2 of 203
     39PASS buffer[2] is within 2 of 9
     40PASS buffer[0] is within 2 of 53
     41PASS buffer[1] is within 2 of 203
     42PASS buffer[2] is within 2 of 9
     43PASS buffer[0] is within 2 of 200
    6844PASS buffer[1] is within 2 of 200
    69 PASS buffer[2] is within 2 of 18
    70 PASS buffer[0] is within 2 of 20
     45PASS buffer[2] is within 2 of 0
     46PASS buffer[0] is within 2 of 200
    7147PASS buffer[1] is within 2 of 200
    72 PASS buffer[2] is within 2 of 18
    73 PASS buffer[0] is within 2 of 20
     48PASS buffer[2] is within 2 of 0
     49PASS buffer[0] is within 2 of 200
    7450PASS buffer[1] is within 2 of 200
    75 PASS buffer[2] is within 2 of 18
    76 PASS buffer[0] is within 2 of 20
     51PASS buffer[2] is within 2 of 0
     52PASS buffer[0] is within 2 of 200
    7753PASS buffer[1] is within 2 of 200
    78 PASS buffer[2] is within 2 of 18
     54PASS buffer[2] is within 2 of 0
     55PASS buffer[0] is within 2 of 57
     56PASS buffer[1] is within 2 of 202
     57PASS buffer[2] is within 2 of 203
     58PASS buffer[0] is within 2 of 57
     59PASS buffer[1] is within 2 of 202
     60PASS buffer[2] is within 2 of 203
     61PASS buffer[0] is within 2 of 57
     62PASS buffer[1] is within 2 of 202
     63PASS buffer[2] is within 2 of 203
     64PASS buffer[0] is within 2 of 57
     65PASS buffer[1] is within 2 of 202
     66PASS buffer[2] is within 2 of 203
     67PASS buffer[0] is within 2 of 53
     68PASS buffer[1] is within 2 of 203
     69PASS buffer[2] is within 2 of 9
     70PASS buffer[0] is within 2 of 53
     71PASS buffer[1] is within 2 of 203
     72PASS buffer[2] is within 2 of 9
     73PASS buffer[0] is within 2 of 53
     74PASS buffer[1] is within 2 of 203
     75PASS buffer[2] is within 2 of 9
     76PASS buffer[0] is within 2 of 53
     77PASS buffer[1] is within 2 of 203
     78PASS buffer[2] is within 2 of 9
    7979PASS successfullyParsed is true
    8080

  • trunk/LayoutTests/fast/canvas/canvas-createPattern-video-loading.html

    r203777 r284439  
    1313    var expectedResults = [
    1414        // Each entry is formatted as [x, y, r, g, b].
    15         [40, 165, 194, 193, 14], // Represents yellow north west tile.
    16         [355, 165, 194, 193, 14], // Represents yellow north east tile.
    17         [40, 403, 194, 193, 14], // Represents yellow south west tile.
    18         [355, 403, 194, 193, 14], // Represents yellow south east tile.
     15        [40, 165, 200, 200, 0], // Represents yellow north west tile.
     16        [355, 165, 200, 200, 0], // Represents yellow north east tile.
     17        [40, 403, 200, 200, 0], // Represents yellow south west tile.
     18        [355, 403, 200, 200, 0], // Represents yellow south east tile.
    1919
    20         [56, 165, 17, 197, 195], // Represents blue/cyan north west tile.
    21         [371, 165, 17, 197, 195], // Represents blue/cyan north east tile.
    22         [56, 403, 17, 197, 195], // Represents blue/cyan south west tile.
    23         [371, 403, 17, 197, 195], // Represents blue/cyan south east tile.
     20        [56, 165, 57, 202, 203], // Represents blue/cyan north west tile.
     21        [371, 165, 57, 202, 203], // Represents blue/cyan north east tile.
     22        [56, 403, 57, 202, 203], // Represents blue/cyan south west tile.
     23        [371, 403, 57, 202, 203], // Represents blue/cyan south east tile.
    2424
    25         [73, 165, 20, 200, 18], // Represents green north west tile.
    26         [388, 165, 20, 200, 18], // Represents green north east tile.
    27         [73, 403, 20, 200, 18], // Represents green south west tile.
    28         [388, 403, 20, 200, 18], // Represents green south east tile.
     25        [73, 165, 53, 203, 9], // Represents green north west tile.
     26        [388, 165, 53, 203, 9], // Represents green north east tile.
     27        [73, 403, 53, 203, 9], // Represents green south west tile.
     28        [388, 403, 53, 203, 9], // Represents green south east tile.
    2929    ];
    3030

  • trunk/LayoutTests/fast/canvas/canvas-createPattern-video-modify-expected.txt

    r201724 r284439  
    44
    55
    6 PASS buffer[0] is within 2 of 194
    7 PASS buffer[1] is within 2 of 193
    8 PASS buffer[2] is within 2 of 14
    9 PASS buffer[0] is within 2 of 194
    10 PASS buffer[1] is within 2 of 193
    11 PASS buffer[2] is within 2 of 14
    12 PASS buffer[0] is within 2 of 194
    13 PASS buffer[1] is within 2 of 193
    14 PASS buffer[2] is within 2 of 14
    15 PASS buffer[0] is within 2 of 194
    16 PASS buffer[1] is within 2 of 193
    17 PASS buffer[2] is within 2 of 14
     6PASS buffer[0] is within 2 of 200
     7PASS buffer[1] is within 2 of 200
     8PASS buffer[2] is within 2 of 0
     9PASS buffer[0] is within 2 of 200
     10PASS buffer[1] is within 2 of 200
     11PASS buffer[2] is within 2 of 0
     12PASS buffer[0] is within 2 of 200
     13PASS buffer[1] is within 2 of 200
     14PASS buffer[2] is within 2 of 0
     15PASS buffer[0] is within 2 of 200
     16PASS buffer[1] is within 2 of 200
     17PASS buffer[2] is within 2 of 0
    1818PASS successfullyParsed is true
    1919

  • trunk/LayoutTests/fast/canvas/canvas-createPattern-video-modify.html

    r203777 r284439  
    1515    var expectedResults = [
    1616        // Each entry is formatted as [x, y, r, g, b].
    17         [40, 165, 194, 193, 14], // Represents yellow north west tile.
    18         [355, 165, 194, 193, 14], // Represents yellow north east tile.
    19         [40, 403, 194, 193, 14], // Represents yellow south west tile.
    20         [355, 403, 194, 193, 14], // Represents yellow south east tile.
     17        [40, 165, 200, 200, 0], // Represents yellow north west tile.
     18        [355, 165, 200, 200, 0], // Represents yellow north east tile.
     19        [40, 403, 200, 200, 0], // Represents yellow south west tile.
     20        [355, 403, 200, 200, 0], // Represents yellow south east tile.
    2121    ];
    2222

  • trunk/LayoutTests/imported/w3c/ChangeLog

    r284437 r284439  
     12021-10-19  Cameron McCormack  <heycam@apple.com>
     2
     3        Ensure CanvasRenderingContext2D.drawImage(video) uses the right color space
     4        https://bugs.webkit.org/show_bug.cgi?id=229025
     5        <rdar://problem/81828490>
     6
     7        Reviewed by Sam Weinig.
     8
     9        Add some new WPTs for drawImage(video).
     10
     11        * web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-drawImage.html:
     12        * web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-flipY.html:
     13        Updated tests to check that video color space conversion is done
     14        correctly.
     15
     16        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video-expected.txt: Added.
     17        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html: Added.
     18        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video-expected.txt: Added.
     19        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html: Added.
     20        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3.js:
     21        (pixelsApproximatelyEqual): Tweaked threshold to pass video tests on iOS.
     22        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-222000000.mp4: Added.
     23        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-222000000.webm: Added.
     24        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-3FF000000.mp4: Added.
     25        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/Rec2020-3FF000000.webm: Added.
     26        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-BB0000.mp4: Added.
     27        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-BB0000.webm: Added.
     28        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-FF0100.mp4: Added.
     29        * web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/resources/sRGB-FF0100.webm: Added.
     30
    1312021-10-18  Antoine Quint  <graouts@webkit.org>
    232

  • trunk/LayoutTests/imported/w3c/web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-drawImage.html

    r282203 r284439  
    1010<body>
    1111<script>
    12 function testCanvasDisplayingPattern(canvas, width, height)
     12function testCanvasDisplayingPattern(canvas, width, height, sourceIsVideo)
    1313{
    14     var tolerance = 10; // for creating ImageBitmap from a video, the tolerance needs to be high
    15     const check = (x, y, r, g, b, a) =>
     14    var tolerance = 1;
     15    let topLeft = [255, 0, 0, 255];
     16    let topRight = [0, 255, 0, 255];
     17    let bottomLeft = [0, 0, 255, 255];
     18    let bottomRight = [0, 0, 0, 255];
     19    if (sourceIsVideo) {
     20        // The source video uses colors in the Rec.601 color space whose
     21        // values are close to full red, full green, full blue, and black,
     22        // but when converted to sRGB, are somewhat different.
     23        topLeft = [247, 37, 0, 255];
     24        topRight = [63, 251, 0, 255];
     25        bottomLeft = [28, 35, 255, 255];
     26        bottomRight = [5, 0, 2, 255];
     27    }
     28    const check = (x, y, [r, g, b, a]) =>
    1629        _assertPixelApprox(canvas, x,y, r,g,b,a, `${x},${y}`, `${r},${g},${b},${a}`, tolerance);
    17     check(1 * width / 4, 1 * height / 4, 255,0,0,255);
    18     check(3 * width / 4, 1 * height / 4, 0,255,0,255);
    19     check(1 * width / 4, 3 * height / 4, 0,0,255,255);
    20     check(3 * width / 4, 3 * height / 4, 0,0,0,255);
     30    check(1 * width / 4, 1 * height / 4, topLeft);
     31    check(3 * width / 4, 1 * height / 4, topRight);
     32    check(1 * width / 4, 3 * height / 4, bottomLeft);
     33    check(3 * width / 4, 3 * height / 4, bottomRight);
    2134}
    2235
    2336function testDrawImageBitmap(source, args = [], { resizeWidth = 20, resizeHeight = 20 } = {})
    2437{
     38    let sourceIsVideo = source instanceof HTMLVideoElement;
    2539    var canvas = document.createElement("canvas");
    2640    canvas.width = resizeWidth;
    … …  
    3145        assert_equals(imageBitmap.height, resizeHeight);
    3246        ctx.drawImage(imageBitmap, 0, 0);
    33         testCanvasDisplayingPattern(canvas, resizeWidth, resizeHeight);
     47        testCanvasDisplayingPattern(canvas, resizeWidth, resizeHeight, sourceIsVideo);
    3448    });
    3549}

  • trunk/LayoutTests/imported/w3c/web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-flipY.html

    r282203 r284439  
    1010<body>
    1111<script>
    12 function testCanvasDisplayingPattern(canvas, width, height, flipped)
     12function testCanvasDisplayingPattern(canvas, width, height, sourceIsVideo, flipped)
    1313{
    14     var tolerance = 10; // for creating ImageBitmap from a video, the tolerance needs to be high
    15     const check = (x, y, r, g, b, a) =>
     14    var tolerance = 1;
     15    let topLeft = [255, 0, 0, 255];
     16    let topRight = [0, 255, 0, 255];
     17    let bottomLeft = [0, 0, 255, 255];
     18    let bottomRight = [0, 0, 0, 255];
     19    if (sourceIsVideo) {
     20        // The source video uses colors in the Rec.601 color space whose
     21        // values are close to full red, full green, full blue, and black,
     22        // but when converted to sRGB, are somewhat different.
     23        topLeft = [247, 37, 0, 255];
     24        topRight = [63, 251, 0, 255];
     25        bottomLeft = [28, 35, 255, 255];
     26        bottomRight = [5, 0, 2, 255];
     27    }
     28    if (flipped) {
     29        [topLeft, bottomLeft] = [bottomLeft, topLeft];
     30        [topRight, bottomRight] = [bottomRight, topRight];
     31    }
     32    const check = (x, y, [r, g, b, a]) =>
    1633        _assertPixelApprox(canvas, x,y, r,g,b,a, `${x},${y}`, `${r},${g},${b},${a}`, tolerance);
    17     if (flipped) {
    18         check(1 * width / 4, 3 * height / 4, 255,0,0,255);
    19         check(3 * width / 4, 3 * height / 4, 0,255,0,255);
    20         check(1 * width / 4, 1 * height / 4, 0,0,255,255);
    21         check(3 * width / 4, 1 * height / 4, 0,0,0,255);
    22     } else {
    23         check(1 * width / 4, 1 * height / 4, 255,0,0,255);
    24         check(3 * width / 4, 1 * height / 4, 0,255,0,255);
    25         check(1 * width / 4, 3 * height / 4, 0,0,255,255);
    26         check(3 * width / 4, 3 * height / 4, 0,0,0,255);
    27     }
     34    check(1 * width / 4, 1 * height / 4, topLeft);
     35    check(3 * width / 4, 1 * height / 4, topRight);
     36    check(1 * width / 4, 3 * height / 4, bottomLeft);
     37    check(3 * width / 4, 3 * height / 4, bottomRight);
    2838}
    2939
    3040function testDrawImageBitmap(source, args = [], flipped, { resizeWidth = 20, resizeHeight = 20 } = {})
    3141{
     42    let sourceIsVideo = source instanceof HTMLVideoElement;
    3243    var canvas = document.createElement("canvas");
    3344    canvas.width = resizeWidth;
    … …  
    3849        assert_equals(imageBitmap.height, resizeHeight);
    3950        ctx.drawImage(imageBitmap, 0, 0);
    40         testCanvasDisplayingPattern(canvas, resizeWidth, resizeHeight, flipped);
     51        testCanvasDisplayingPattern(canvas, resizeWidth, resizeHeight, sourceIsVideo, flipped);
    4152    });
    4253}

  • trunk/LayoutTests/imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3.js

    r283531 r284439  
    208208};
    209209
     210// Each video:
     211//  * is 300x200 and has a single color
     212//  * has a filename base that indicates its contents:
     213//
     214//      <color-space>-<8-or-10-bit-color-value>
     215//
     216//  * was generated using commands like:
     217//
     218//      W=300 H=200 Y=3F Cb=66 Cr=F0 ; \
     219//        perl -e "print pack('c', 0x$Y) x ($W * $H), pack('c', 0x$Cb) x ($W * $H / 4), pack('c', 0x$Cr) x ($W * $H / 4)" | \
     220//        ffmpeg -f rawvideo -pix_fmt yuv420p -s:v ${W}x$H -r 25 -i - -pix_fmt yuv420p -colorspace bt709 -color_primaries bt709 -color_trc iec61966_2_1 sRGB-FF0100.webm
     221//
     222//      W=300 H=200 Y=0BB Cb=1BD Cr=2EF ; \
     223//        perl -e "print pack('s', 0x$Y) x ($W * $H), pack('s', 0x$Cb) x ($W * $H / 4), pack('s', 0x$Cr) x ($W * $H / 4)" | \
     224//        ffmpeg -f rawvideo -pix_fmt yuv420p10le -s:v ${W}x$H -r 25 -i - -c:v libx265 -vtag hvc1 -pix_fmt yuv420p10le -colorspace bt2020nc -color_primaries bt2020 -color_trc bt2020-10 Rec2020-222000000.mp4
     225//
     226//      W=300 H=200 Y=0BB Cb=1BD Cr=2EF ; \
     227//        perl -e "print pack('s', 0x$Y) x ($W * $H), pack('s', 0x$Cb) x ($W * $H / 4), pack('s', 0x$Cr) x ($W * $H / 4)" | \
     228//        ffmpeg -f rawvideo -pix_fmt yuv420p10le -s:v ${W}x$H -r 25 -i - -vcodec libvpx-vp9 -profile:v 2 -pix_fmt yuv420p10le -colorspace bt2020nc -color_primaries bt2020 -color_trc bt2020-10 Rec2020-222000000.webm
     229//
     230//    where the Y'CbCr values were computed using https://jdashg.github.io/misc/colors/from-coeffs.html.
     231const videoTests = {
     232    // Rec.709 Y'CbCr (0x3F, 0x66, 0xF0) = sRGB (0xFF, 0x01, 0x00)
     233    "sRGB-FF0100": {
     234        "srgb srgb": [255, 1, 0, 255],
     235        "srgb display-p3": [234, 51, 35, 255],
     236        "display-p3 srgb": [255, 0, 0, 255],
     237        "display-p3 display-p3": [234, 51, 35, 255],
     238    },
     239    // Rec.709 Y'CbCr (0x32, 0x6D, 0xD2) = sRGB (0xBB, 0x00, 0x00)
     240    "sRGB-BB0000": {
     241        "srgb srgb": [187, 0, 0, 255],
     242        "srgb display-p3": [171, 35, 23, 255],
     243        "display-p3 srgb": [187, 1, 0, 255],
     244        "display-p3 display-p3": [171, 35, 23, 255],
     245    },
     246
     247    // 10 bit Rec.2020 Y'CbCr (0x126, 0x183, 0x3C0) = Rec.2020 (0x3FF, 0x000, 0x000)
     248    "Rec2020-3FF000000": {
     249        "srgb srgb": [255, 0, 0, 255],
     250        "srgb display-p3": [234, 51, 35, 255],
     251        "display-p3 srgb": [255, 0, 0, 255],
     252        "display-p3 display-p3": [255, 0, 9, 255],
     253    },
     254    // 10 bit Rec.2020 Y'CbCr (0x0BB, 0x1BD, 0x2EF) = Rec.2020 (0x222, 0x000, 0x000)
     255    "Rec2020-222000000": {
     256        "srgb srgb": [186, 0, 0, 255],
     257        "srgb display-p3": [170, 34, 23, 255],
     258        "display-p3 srgb": [186, 0, 0, 255],
     259        "display-p3 display-p3": [169, 0, 3, 255],
     260    },
     261};
     262
    210263const fromSRGBToDisplayP3 = {
    211264    "255,0,0,255": [234, 51, 35, 255],
    … …  
    224277function pixelsApproximatelyEqual(p1, p2) {
    225278    for (let i = 0; i < 4; ++i) {
    226         if (Math.abs(p1[i] - p2[i]) > 2)
     279        if (Math.abs(p1[i] - p2[i]) > 3)
    227280            return false;
    228281    }

  • trunk/LayoutTests/media/video-canvas-createPattern-expected.txt

    r201724 r284439  
    22EVENT(canplaythrough)
    33EVENT(seeked)
    4 Saw close enough to (194, 193, 14) at (40, 165) OK
    5 Saw close enough to (194, 193, 14) at (355, 165) OK
    6 Saw close enough to (194, 193, 14) at (40, 403) OK
    7 Saw close enough to (194, 193, 14) at (355, 403) OK
    8 Saw close enough to (17, 197, 195) at (56, 165) OK
    9 Saw close enough to (17, 197, 195) at (371, 165) OK
    10 Saw close enough to (17, 197, 195) at (56, 403) OK
    11 Saw close enough to (17, 197, 195) at (371, 403) OK
    12 Saw close enough to (20, 200, 18) at (73, 165) OK
    13 Saw close enough to (20, 200, 18) at (388, 165) OK
    14 Saw close enough to (20, 200, 18) at (73, 403) OK
    15 Saw close enough to (20, 200, 18) at (388, 403) OK
     4Saw close enough to (200, 200, 0) at (40, 165) OK
     5Saw close enough to (200, 200, 0) at (355, 165) OK
     6Saw close enough to (200, 200, 0) at (40, 403) OK
     7Saw close enough to (200, 200, 0) at (355, 403) OK
     8Saw close enough to (57, 202, 203) at (56, 165) OK
     9Saw close enough to (57, 202, 203) at (371, 165) OK
     10Saw close enough to (57, 202, 203) at (56, 403) OK
     11Saw close enough to (57, 202, 203) at (371, 403) OK
     12Saw close enough to (53, 203, 9) at (73, 165) OK
     13Saw close enough to (53, 203, 9) at (388, 165) OK
     14Saw close enough to (53, 203, 9) at (73, 403) OK
     15Saw close enough to (53, 203, 9) at (388, 403) OK
    1616EVENT(seeked)
    17 Saw close enough to (194, 193, 14) at (40, 165) OK
    18 Saw close enough to (194, 193, 14) at (355, 165) OK
    19 Saw close enough to (194, 193, 14) at (40, 403) OK
    20 Saw close enough to (194, 193, 14) at (355, 403) OK
    21 Saw close enough to (17, 197, 195) at (56, 165) OK
    22 Saw close enough to (17, 197, 195) at (371, 165) OK
    23 Saw close enough to (17, 197, 195) at (56, 403) OK
    24 Saw close enough to (17, 197, 195) at (371, 403) OK
    25 Saw close enough to (20, 200, 18) at (73, 165) OK
    26 Saw close enough to (20, 200, 18) at (388, 165) OK
    27 Saw close enough to (20, 200, 18) at (73, 403) OK
    28 Saw close enough to (20, 200, 18) at (388, 403) OK
     17Saw close enough to (200, 200, 0) at (40, 165) OK
     18Saw close enough to (200, 200, 0) at (355, 165) OK
     19Saw close enough to (200, 200, 0) at (40, 403) OK
     20Saw close enough to (200, 200, 0) at (355, 403) OK
     21Saw close enough to (57, 202, 203) at (56, 165) OK
     22Saw close enough to (57, 202, 203) at (371, 165) OK
     23Saw close enough to (57, 202, 203) at (56, 403) OK
     24Saw close enough to (57, 202, 203) at (371, 403) OK
     25Saw close enough to (53, 203, 9) at (73, 165) OK
     26Saw close enough to (53, 203, 9) at (388, 165) OK
     27Saw close enough to (53, 203, 9) at (73, 403) OK
     28Saw close enough to (53, 203, 9) at (388, 403) OK
    2929EVENT(seeked)
    30 Saw close enough to (194, 193, 14) at (40, 165) OK
    31 Saw close enough to (194, 193, 14) at (355, 165) OK
    32 Saw close enough to (194, 193, 14) at (40, 403) OK
    33 Saw close enough to (194, 193, 14) at (355, 403) OK
    34 Saw close enough to (17, 197, 195) at (56, 165) OK
    35 Saw close enough to (17, 197, 195) at (371, 165) OK
    36 Saw close enough to (17, 197, 195) at (56, 403) OK
    37 Saw close enough to (17, 197, 195) at (371, 403) OK
    38 Saw close enough to (20, 200, 18) at (73, 165) OK
    39 Saw close enough to (20, 200, 18) at (388, 165) OK
    40 Saw close enough to (20, 200, 18) at (73, 403) OK
    41 Saw close enough to (20, 200, 18) at (388, 403) OK
     30Saw close enough to (200, 200, 0) at (40, 165) OK
     31Saw close enough to (200, 200, 0) at (355, 165) OK
     32Saw close enough to (200, 200, 0) at (40, 403) OK
     33Saw close enough to (200, 200, 0) at (355, 403) OK
     34Saw close enough to (57, 202, 203) at (56, 165) OK
     35Saw close enough to (57, 202, 203) at (371, 165) OK
     36Saw close enough to (57, 202, 203) at (56, 403) OK
     37Saw close enough to (57, 202, 203) at (371, 403) OK
     38Saw close enough to (53, 203, 9) at (73, 165) OK
     39Saw close enough to (53, 203, 9) at (388, 165) OK
     40Saw close enough to (53, 203, 9) at (73, 403) OK
     41Saw close enough to (53, 203, 9) at (388, 403) OK
    4242EVENT(seeked)
    43 Saw close enough to (194, 193, 14) at (40, 165) OK
    44 Saw close enough to (194, 193, 14) at (355, 165) OK
    45 Saw close enough to (194, 193, 14) at (40, 403) OK
    46 Saw close enough to (194, 193, 14) at (355, 403) OK
    47 Saw close enough to (17, 197, 195) at (56, 165) OK
    48 Saw close enough to (17, 197, 195) at (371, 165) OK
    49 Saw close enough to (17, 197, 195) at (56, 403) OK
    50 Saw close enough to (17, 197, 195) at (371, 403) OK
    51 Saw close enough to (20, 200, 18) at (73, 165) OK
    52 Saw close enough to (20, 200, 18) at (388, 165) OK
    53 Saw close enough to (20, 200, 18) at (73, 403) OK
    54 Saw close enough to (20, 200, 18) at (388, 403) OK
     43Saw close enough to (200, 200, 0) at (40, 165) OK
     44Saw close enough to (200, 200, 0) at (355, 165) OK
     45Saw close enough to (200, 200, 0) at (40, 403) OK
     46Saw close enough to (200, 200, 0) at (355, 403) OK
     47Saw close enough to (57, 202, 203) at (56, 165) OK
     48Saw close enough to (57, 202, 203) at (371, 165) OK
     49Saw close enough to (57, 202, 203) at (56, 403) OK
     50Saw close enough to (57, 202, 203) at (371, 403) OK
     51Saw close enough to (53, 203, 9) at (73, 165) OK
     52Saw close enough to (53, 203, 9) at (388, 165) OK
     53Saw close enough to (53, 203, 9) at (73, 403) OK
     54Saw close enough to (53, 203, 9) at (388, 403) OK
    5555END OF TEST
    5656

  • trunk/LayoutTests/media/video-canvas-createPattern.html

    r203777 r284439  
    1111            var expectedResults = [
    1212                // Each entry is formatted as [x, y, r, g, b].
    13                 [40, 165, 194, 193, 14], // Represents yellow north west tile.
    14                 [355, 165, 194, 193, 14], // Represents yellow north east tile.
    15                 [40, 403, 194, 193, 14], // Represents yellow south west tile.
    16                 [355, 403, 194, 193, 14], // Represents yellow south east tile.
     13                [40, 165, 200, 200, 0], // Represents yellow north west tile.
     14                [355, 165, 200, 200, 0], // Represents yellow north east tile.
     15                [40, 403, 200, 200, 0], // Represents yellow south west tile.
     16                [355, 403, 200, 200, 0], // Represents yellow south east tile.
    1717
    18                 [56, 165, 17, 197, 195], // Represents blue/cyan north west tile.
    19                 [371, 165, 17, 197, 195], // Represents blue/cyan north east tile.
    20                 [56, 403, 17, 197, 195], // Represents blue/cyan south west tile.
    21                 [371, 403, 17, 197, 195], // Represents blue/cyan south east tile.
     18                [56, 165, 57, 202, 203], // Represents blue/cyan north west tile.
     19                [371, 165, 57, 202, 203], // Represents blue/cyan north east tile.
     20                [56, 403, 57, 202, 203], // Represents blue/cyan south west tile.
     21                [371, 403, 57, 202, 203], // Represents blue/cyan south east tile.
    2222
    23                 [73, 165, 20, 200, 18], // Represents green north west tile.
    24                 [388, 165, 20, 200, 18], // Represents green north east tile.
    25                 [73, 403, 20, 200, 18], // Represents green south west tile.
    26                 [388, 403, 20, 200, 18], // Represents green south east tile.
     23                [73, 165, 53, 203, 9], // Represents green north west tile.
     24                [388, 165, 53, 203, 9], // Represents green north east tile.
     25                [73, 403, 53, 203, 9], // Represents green south west tile.
     26                [388, 403, 53, 203, 9], // Represents green south east tile.
    2727            ]
    2828

  • trunk/LayoutTests/media/video-canvas-drawing-output-expected.txt

    r180441 r284439  
    22EVENT(canplaythrough)
    33EVENT(seeked)
    4 Saw close enough to (194, 193, 14) at (40, 165) OK
    5 Saw close enough to (17, 197, 195) at (56, 165) OK
    6 Saw close enough to (20, 200, 18) at (73, 165) OK
     4Saw close enough to (200, 200, 0) at (40, 165) OK
     5Saw close enough to (57, 202, 203) at (56, 165) OK
     6Saw close enough to (53, 203, 9) at (73, 165) OK
    77EVENT(seeked)
    8 Saw close enough to (194, 193, 14) at (40, 165) OK
    9 Saw close enough to (17, 197, 195) at (56, 165) OK
    10 Saw close enough to (20, 200, 18) at (73, 165) OK
     8Saw close enough to (200, 200, 0) at (40, 165) OK
     9Saw close enough to (57, 202, 203) at (56, 165) OK
     10Saw close enough to (53, 203, 9) at (73, 165) OK
    1111EVENT(seeked)
    12 Saw close enough to (194, 193, 14) at (40, 165) OK
    13 Saw close enough to (17, 197, 195) at (56, 165) OK
    14 Saw close enough to (20, 200, 18) at (73, 165) OK
     12Saw close enough to (200, 200, 0) at (40, 165) OK
     13Saw close enough to (57, 202, 203) at (56, 165) OK
     14Saw close enough to (53, 203, 9) at (73, 165) OK
    1515EVENT(seeked)
    16 Saw close enough to (194, 193, 14) at (40, 165) OK
    17 Saw close enough to (17, 197, 195) at (56, 165) OK
    18 Saw close enough to (20, 200, 18) at (73, 165) OK
     16Saw close enough to (200, 200, 0) at (40, 165) OK
     17Saw close enough to (57, 202, 203) at (56, 165) OK
     18Saw close enough to (53, 203, 9) at (73, 165) OK
    1919END OF TEST
    2020

  • trunk/LayoutTests/media/video-canvas-drawing-output.html

    r221591 r284439  
    99            var expectedResults = [
    1010                // [x, y, r, g, b]
    11                 [40, 165, 194, 193, 14], // yellow
    12                 [56, 165, 17, 197, 195], // blue/cyan
    13                 [73, 165, 20, 200, 18], // green
     11                [40, 165, 200, 200, 0], // yellow
     12                [56, 165, 57, 202, 203], // blue/cyan
     13                [73, 165, 53, 203, 9], // green
    1414            ];
    1515

  • trunk/LayoutTests/media/video-orientation-canvas-expected.txt

    r267644 r284439  
    11
    2 EXPECTED (context.getImageData(88, 115, 1, 1).data == [208,0,0,255] with tolerance 5 OK
    3 EXPECTED (context.getImageData(237, 88, 1, 1).data == [208,0,0,255] with tolerance 5 OK
    4 EXPECTED (context.getImageData(200, 237, 1, 1).data == [208,0,0,255] with tolerance 5 OK
     2EXPECTED (context.getImageData(88, 115, 1, 1).data == [208,30,0,255] with tolerance 5 OK
     3EXPECTED (context.getImageData(237, 88, 1, 1).data == [208,30,0,255] with tolerance 5 OK
     4EXPECTED (context.getImageData(200, 237, 1, 1).data == [208,30,0,255] with tolerance 5 OK
    55END OF TEST
    66

  • trunk/LayoutTests/media/video-orientation-canvas.html

    r274076 r284439  
    55    <script src="video-test.js"></script>
    66    <script>
    7         let redColor = new Uint8ClampedArray([208, 0, 0, 255]);
     7        let redColor = new Uint8ClampedArray([208, 30, 0, 255]);
    88        var context;
    99

  • trunk/LayoutTests/platform/glib/TestExpectations

    r284368 r284439  
    233233media/media-controller-unpause.html [ Pass ]
    234234media/mediacapabilities/mediacapabilities-types.html [ Pass ]
    235 media/video-canvas-createPattern.html [ Pass ]
    236 media/video-canvas-drawing-output.html [ Pass ]
    237235media/video-controller-currentTime-rate.html [ Pass ]
    238236webaudio/codec-tests/aac/vbr-128kbps-44khz.html [ Pass ]

  • trunk/LayoutTests/platform/ios-14/TestExpectations

    r283531 r284439  
    7373imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-cloned.html [ Failure ]
    7474imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-image.html [ Failure ]
     75imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html [ Failure ]
     76imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html [ Failure ]
    7577imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-canvas.html [ Failure ]
    7678imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-image.html [ Failure ]

  • trunk/LayoutTests/platform/ios-device/TestExpectations

    r280375 r284439  
    100100loader/go-back-to-different-window-size.html [ Crash ]
    101101
    102 media/video-canvas-createPattern.html [ Failure ]
    103 media/video-canvas-drawing-output.html [ Failure ]
    104102media/video-load-preload-metadata.html [ Failure ]
    105103media/video-played-collapse.html [ Failure ]

  • trunk/LayoutTests/platform/ios/TestExpectations

    r284287 r284439  
    34893489imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-cloned.html [ Pass ]
    34903490imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-image.html [ Pass ]
     3491imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html [ Pass ]
     3492imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html [ Pass ]
    34913493imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-canvas.html [ Pass ]
    34923494imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-image.html [ Pass ]

  • trunk/LayoutTests/platform/mac-wk1/imported/w3c/web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-flipY-expected.txt

    r282203 r284439  
    99PASS createImageBitmap from a bitmap HTMLImageElement imageOrientation: "flipY", and drawImage on the created ImageBitmap
    1010PASS createImageBitmap from a vector HTMLImageElement imageOrientation: "none", and drawImage on the created ImageBitmap
    11 FAIL createImageBitmap from a vector HTMLImageElement imageOrientation: "flipY", and drawImage on the created ImageBitmap assert_approx_equals: Red channel of the pixel at (5, 15) expected 255 +/- 10 but got 0
     11FAIL createImageBitmap from a vector HTMLImageElement imageOrientation: "flipY", and drawImage on the created ImageBitmap assert_approx_equals: Red channel of the pixel at (5, 5) expected 0 +/- 1 but got 255
    1212FAIL createImageBitmap from a bitmap SVGImageElement imageOrientation: "none", and drawImage on the created ImageBitmap promise_test: Unhandled rejection with value: object "TypeError: Type error"
    1313FAIL createImageBitmap from a bitmap SVGImageElement imageOrientation: "flipY", and drawImage on the created ImageBitmap promise_test: Unhandled rejection with value: object "TypeError: Type error"

  • trunk/LayoutTests/platform/mac-wk2/TestExpectations

    r284328 r284439  
    12171217[ arm64 ] imported/w3c/web-platform-tests/html/canvas/element/path-objects/2d.path.arc.shape.3.html [ Failure ]
    12181218[ arm64 ] inspector/canvas/memory.html [ Failure ]
    1219 [ arm64 ] media/video-canvas-createPattern.html [ Failure ]
    12201219
    12211220webkit.org/b/229671 [ BigSur+ Debug arm64 ] imported/w3c/web-platform-tests/html/rendering/replaced-elements/svg-embedded-sizing/svg-in-img-auto.html [ Pass Crash ]

  • trunk/LayoutTests/platform/mac/TestExpectations

    r284415 r284439  
    19311931webkit.org/b/227891 http/tests/appcache/fail-on-update.html [ Pass Timeout DumpJSConsoleLogInStdErr ]
    19321932
    1933 # rdar://66660680 (REGRESSION (20A2327a-20A2348b): media/video-canvas-createPattern.html is failing)
    1934 [ BigSur+ ] media/video-canvas-createPattern.html [ Failure ]
    1935 
    19361933# Remove this once <rdar://problem/66734187> hits the bots
    19371934[ BigSur+ ] fast/text/locale-shaping.html [ Pass ImageOnlyFailure ]
    … …  
    23662363[ Monterey+ ] imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-cloned.html [ Pass ]
    23672364[ Monterey+ ] imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-image.html [ Pass ]
     2365[ Monterey+ ] imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html [ Pass ]
     2366[ Monterey+ ] imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html [ Pass ]
    23682367[ Monterey+ ] imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-canvas.html [ Pass ]
    23692368[ Monterey+ ] imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-pattern-image.html [ Pass ]

  • trunk/Source/WTF/ChangeLog

    r284428 r284439  
     12021-10-19  Cameron McCormack  <heycam@apple.com>
     2
     3        Ensure CanvasRenderingContext2D.drawImage(video) uses the right color space
     4        https://bugs.webkit.org/show_bug.cgi?id=229025
     5        <rdar://problem/81828490>
     6
     7        Reviewed by Sam Weinig.
     8
     9        * wtf/PlatformHave.h: Add HAVE_CVBUFFERCOPYATTACHMENTS.
     10
    1112021-10-18  Commit Queue  <commit-queue@webkit.org>
    212

  • trunk/Source/WTF/wtf/PlatformHave.h

    r284418 r284439  
    10661066#define HAVE_PUACTIVITYPROGRESSCONTROLLER 1
    10671067#endif
     1068
     1069#if (PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 120000) \
     1070    || (((PLATFORM(IOS) && !PLATFORM(IOS_FAMILY_SIMULATOR)) || PLATFORM(MACCATALYST)) && __IPHONE_OS_VERSION_MIN_REQUIRED >= 150000) \
     1071    || (PLATFORM(WATCHOS) && !PLATFORM(IOS_FAMILY_SIMULATOR) && __WATCH_OS_VERSION_MIN_REQUIRED >= 80000) \
     1072    || (PLATFORM(APPLETV) && !PLATFORM(IOS_FAMILY_SIMULATOR) && __TV_OS_VERSION_MIN_REQUIRED >= 150000)
     1073#define HAVE_CVBUFFERCOPYATTACHMENTS 1
     1074#endif

  • trunk/Source/WebCore/ChangeLog

    r284438 r284439  
     12021-10-19  Cameron McCormack  <heycam@apple.com>
     2
     3        Ensure CanvasRenderingContext2D.drawImage(video) uses the right color space
     4        https://bugs.webkit.org/show_bug.cgi?id=229025
     5        <rdar://problem/81828490>
     6
     7        Reviewed by Sam Weinig.
     8
     9        When GPU process media handling is disabled (and in the GPU process
     10        when it is handling a NativeImageForCurrentTime message),
     11        MediaPlayer::nativeImageForCurrentTime is used to get the video frame
     12        image. We make this return an image with the right color space on
     13        Apple platforms in PixelBufferConformerCV::createImageFromPixelBuffer.
     14
     15        For now, we avoid using the correct color space when passing
     16        HTMLVideoElements to WebGLRenderingContext.texImage2D. The
     17        specification is unclear about what color space the texture data
     18        should be in, and implementations are currently interoperable in not
     19        converting the RGB color space of the video (such as Rec.601, Rec.709,
     20        etc.) into sRGB. To avoid the conversion, and effectively reinterpret
     21        the video RGB colors as sRGB, we make the ImageBuffer the video is
     22        drawn into have a matching color space, as it's the raw RGB values
     23        from the ImageBuffer that are uploade as the texture.
     24
     25        (An alternative to this could be passing an argument through
     26        nativeImageForCurrentTime and paintCurrentFrameInContext, which says
     27        to avoid doing the conversion from video RGB to sRGB, but that needs a
     28        fair amount of plumbing.)
     29
     30        If https://github.com/KhronosGroup/WebGL/issues/2165 is resolved, the
     31        changes to LRUImageBufferCache in this commit can be reverted, so that
     32        video frames are drawn into sRGB ImageBuffers again. (We'd also need
     33        something to make the copyToPlatformTexture() call in
     34        texImageSourceHelper do the correct color conversion somehow.)
     35
     36        The now differing behavior between CanvasRenderingContext2D.drawImage
     37        and WebGLRenderingContext.texImage2D means that the WebGL conformance
     38        suite textures/misc/texture-corner-case-videos.html test fails, since
     39        it checks for incorrect 2D canvas video frame color conversion.
     40
     41        Tests: imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-ImageBitmap-video.html
     42               imported/w3c/web-platform-tests/html/canvas/element/manual/wide-gamut-canvas/canvas-display-p3-drawImage-video.html
     43
     44        * html/HTMLVideoElement.cpp:
     45        (WebCore::HTMLVideoElement::colorSpace const): New function that
     46        exposes the color space of the current frame of the video.
     47        * html/HTMLVideoElement.h:
     48        * html/ImageBitmap.cpp:
     49        (WebCore::ImageBitmap::createPromise): Create the ImageBuffer in a
     50        color space that matches the video, not always sRGB.
     51        * html/canvas/WebGLRenderingContextBase.cpp:
     52        (WebCore::WebGLRenderingContextBase::drawImageIntoBuffer):
     53        (WebCore::WebGLRenderingContextBase::videoFrameToImage): Use an
     54        ImageBuffer whose color space matches the video frame's color space.
     55        (WebCore::WebGLRenderingContextBase::LRUImageBufferCache::imageBuffer):
     56        Add a DestinationColorSpace argument, so that the LRU cache is keyed
     57        off both size and color space.
     58        * html/canvas/WebGLRenderingContextBase.h:
     59        * platform/cocoa/CoreVideoSoftLink.cpp:
     60        * platform/cocoa/CoreVideoSoftLink.h: Add SPIs that let us get or
     61        compute a CGColorSpace from a CVPixelBuffer. Remove
     62        CVImageBufferGetColorSpace, which is unsused.
     63        * platform/graphics/MediaPlayer.cpp:
     64        (WebCore::MediaPlayer::colorSpace):
     65        * platform/graphics/MediaPlayer.h:
     66        * platform/graphics/MediaPlayerPrivate.h:
     67        (WebCore::MediaPlayerPrivateInterface::colorSpace):
     68        * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
     69        * platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp:
     70        (WebCore::MediaPlayerPrivateAVFoundationCF::colorSpace):
     71        * platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.h:
     72        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
     73        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     74        (WebCore::MediaPlayerPrivateAVFoundationObjC::colorSpace): Expose the
     75        color space of the current image. While we strictly only need the pixel
     76        buffer to be up to date, we call updateLastImage() since it has the
     77        UpdateSynchronously behavior, and the caller to colorSpace() will use
     78        the image soon anyway.
     79        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
     80        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
     81        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::colorSpace):
     82        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
     83        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
     84        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::colorSpace):
     85        * platform/graphics/cg/ColorSpaceCG.h:
     86        * platform/graphics/cv/CVUtilities.h:
     87        * platform/graphics/cv/CVUtilities.mm:
     88        (WebCore::createCGColorSpaceForCVPixelBuffer): New function to produce
     89        a CGColorSpace for a CVPixelBuffer. On macOS, this is stored as an
     90        attachment on the CVPixelBuffer. On iOS, we must call SPI to create a
     91        CGColorSpace based on the other color space information in the
     92        attachments. (It's possible we should cache the results of this
     93        function for a given set of color space attachments, since
     94        CVImageBufferCreateColorSpaceFromAttachments can end up creating a new
     95        CGColorSpace object every time it's called.)
     96        * platform/graphics/cv/PixelBufferConformerCV.cpp:
     97        (WebCore::PixelBufferConformerCV::createImageFromPixelBuffer): Create
     98        the CGImage in a color space matching the CVPixelBuffer.
     99        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
     100        (WebCore::MediaPlayerPrivateGStreamer::colorSpace):
     101        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
     102        * platform/graphics/holepunch/MediaPlayerPrivateHolePunch.h:
     103        * platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp:
     104        (WebCore::MediaPlayerPrivateMediaFoundation::colorSpace):
     105        * platform/graphics/win/MediaPlayerPrivateMediaFoundation.h:
     106        * platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:
     107        (WebCore::MockMediaPlayerMediaSource::colorSpace):
     108        * platform/mock/mediasource/MockMediaPlayerMediaSource.h:
     109        * svg/graphics/SVGImage.cpp:
     110        (WebCore::SVGImage::drawInternal):
     111
    11122021-10-19  Kyle Piddington  <kpiddington@apple.com>
    2113        Query EGL_BIND_TO_TEXTURE_TARGET_ANGLE to determine 2D/rectangular texture usage

  • trunk/Source/WebCore/html/HTMLVideoElement.cpp

    r283851 r284439  
    285285}
    286286
     287std::optional<DestinationColorSpace> HTMLVideoElement::colorSpace() const
     288{
     289    RefPtr<MediaPlayer> player = HTMLMediaElement::player();
     290    if (!player)
     291        return std::nullopt;
     292
     293    return player->colorSpace();
     294}
     295
    287296RefPtr<ImageBuffer> HTMLVideoElement::createBufferForPainting(const FloatSize& size, RenderingMode renderingMode, const DestinationColorSpace& colorSpace, PixelFormat pixelFormat) const
    288297{

  • trunk/Source/WebCore/html/HTMLVideoElement.h

    r282740 r284439  
    7979
    8080    WEBCORE_EXPORT RefPtr<NativeImage> nativeImageForCurrentTime();
     81    std::optional<DestinationColorSpace> colorSpace() const;
    8182
    8283    WEBCORE_EXPORT bool shouldDisplayPosterImage() const;

  • trunk/Source/WebCore/html/ImageBitmap.cpp

    r284436 r284439  
    510510    auto outputSize = outputSizeForSourceRectangle(sourceRectangle, options);
    511511
    512     // FIXME: Add support for color spaces / pixel formats to ImageBitmap.
    513     auto bitmapData = video->createBufferForPainting(outputSize, bufferRenderingMode, DestinationColorSpace::SRGB(), PixelFormat::BGRA8);
     512    auto colorSpace = video->colorSpace();
     513    if (!colorSpace)
     514        colorSpace = DestinationColorSpace::SRGB();
     515
     516    // FIXME: Add support for pixel formats to ImageBitmap.
     517    auto bitmapData = video->createBufferForPainting(outputSize, bufferRenderingMode, *colorSpace, PixelFormat::BGRA8);
    514518    if (!bitmapData) {
    515519        resolveWithBlankImageBuffer(scriptExecutionContext, !taintsOrigin(scriptExecutionContext.securityOrigin(), *video), WTFMove(promise));

  • trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp

    r284095 r284439  
    58865886    IntSize size(width, height);
    58875887    size.scale(deviceScaleFactor);
    5888     ImageBuffer* buf = m_generatedImageCache.imageBuffer(size);
     5888    ImageBuffer* buf = m_generatedImageCache.imageBuffer(size, DestinationColorSpace::SRGB());
    58895889    if (!buf) {
    58905890        synthesizeGLError(GraphicsContextGL::OUT_OF_MEMORY, functionName, "out of memory");
    … …  
    59025902RefPtr<Image> WebGLRenderingContextBase::videoFrameToImage(HTMLVideoElement* video, BackingStoreCopy backingStoreCopy, const char* functionName)
    59035903{
     5904    // FIXME: When texImage2D is passed an HTMLVideoElement, implementations
     5905    // interoperably use the native RGB color values of the video frame (e.g.
     5906    // Rec.601 color space values) for the texture. But nativeImageForCurrentTime
     5907    // and paintCurrentFrameInContext return and use an image with its color space
     5908    // correctly matching the video.
     5909    //
     5910    // https://github.com/KhronosGroup/WebGL/issues/2165 is open on converting
     5911    // the video element image source to sRGB instead of leaving it in its
     5912    // native RGB color space. For now, we make sure to paint into an
     5913    // ImageBuffer with a matching color space, to avoid the conversion.
    59045914#if USE(AVFOUNDATION)
    59055915    auto nativeImage = video->nativeImageForCurrentTime();
    … …  
    59145924    }
    59155925    FloatRect imageRect { { }, imageSize };
    5916     ImageBuffer* imageBuffer = m_generatedImageCache.imageBuffer(imageSize, CompositeOperator::Copy);
     5926    ImageBuffer* imageBuffer = m_generatedImageCache.imageBuffer(imageSize, nativeImage->colorSpace(), CompositeOperator::Copy);
    59175927    if (!imageBuffer) {
    59185928        synthesizeGLError(GraphicsContextGL::OUT_OF_MEMORY, functionName, "out of memory");
    … …  
    59255935    // once all platforms implement nativeImageForCurrentTime().
    59265936    IntSize videoSize { static_cast<int>(video->videoWidth()), static_cast<int>(video->videoHeight()) };
    5927     ImageBuffer* imageBuffer = m_generatedImageCache.imageBuffer(videoSize);
     5937    auto colorSpace = video->colorSpace();
     5938    if (!colorSpace)
     5939        colorSpace = DestinationColorSpace::SRGB();
     5940    ImageBuffer* imageBuffer = m_generatedImageCache.imageBuffer(videoSize, *colorSpace);
    59285941    if (!imageBuffer) {
    59295942        synthesizeGLError(GraphicsContextGL::OUT_OF_MEMORY, functionName, "out of memory");
    … …  
    77797792}
    77807793
    7781 ImageBuffer* WebGLRenderingContextBase::LRUImageBufferCache::imageBuffer(const IntSize& size, CompositeOperator fillOperator)
     7794ImageBuffer* WebGLRenderingContextBase::LRUImageBufferCache::imageBuffer(const IntSize& size, DestinationColorSpace colorSpace, CompositeOperator fillOperator)
    77827795{
    77837796    size_t i;
    77847797    for (i = 0; i < m_buffers.size(); ++i) {
    7785         ImageBuffer* buf = m_buffers[i].get();
    7786         if (!buf)
     7798        if (!m_buffers[i])
    77877799            break;
    7788         if (buf->logicalSize() != size)
     7800        ImageBuffer& buf = m_buffers[i]->second.get();
     7801        if (m_buffers[i]->first != colorSpace || buf.logicalSize() != size)
    77897802            continue;
    77907803        bubbleToFront(i);
    77917804        if (fillOperator != CompositeOperator::Copy && fillOperator != CompositeOperator::Clear)
    7792             buf->context().clearRect({ { }, size });
    7793         return buf;
     7805            buf.context().clearRect({ { }, size });
     7806        return &buf;
    77947807    }
    77957808
    77967809    // FIXME (149423): Should this ImageBuffer be unconditionally unaccelerated?
    7797     auto temp = ImageBuffer::create(size, RenderingMode::Unaccelerated, 1, DestinationColorSpace::SRGB(), PixelFormat::BGRA8);
     7810    auto temp = ImageBuffer::create(size, RenderingMode::Unaccelerated, 1, colorSpace, PixelFormat::BGRA8);
    77987811    if (!temp)
    77997812        return nullptr;
    78007813    ASSERT(m_buffers.size() > 0);
    78017814    i = std::min(m_buffers.size() - 1, i);
    7802     m_buffers[i] = WTFMove(temp);
    7803 
    7804     ImageBuffer* buf = m_buffers[i].get();
     7815    m_buffers[i] = { colorSpace, temp.releaseNonNull() };
     7816
     7817    ImageBuffer& buf = m_buffers[i]->second.get();
    78057818    bubbleToFront(i);
    7806     return buf;
     7819    return &buf;
    78077820}
    78087821

  • trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.h

    r284075 r284439  
    622622        // Returns pointer to a cleared image buffer that is owned by the cache. The pointer is valid until next call.
    623623        // Using fillOperator == CompositeOperator::Copy can be used to omit the clear of the buffer.
    624         ImageBuffer* imageBuffer(const IntSize&, CompositeOperator fillOperator = CompositeOperator::SourceOver);
     624        ImageBuffer* imageBuffer(const IntSize&, DestinationColorSpace, CompositeOperator fillOperator = CompositeOperator::SourceOver);
    625625    private:
    626626        void bubbleToFront(size_t idx);
    627         Vector<RefPtr<ImageBuffer>> m_buffers;
     627        Vector<std::optional<std::pair<DestinationColorSpace, Ref<ImageBuffer>>>> m_buffers;
    628628    };
    629629    LRUImageBufferCache m_generatedImageCache { 0 };

  • trunk/Source/WebCore/platform/cocoa/CoreVideoSoftLink.cpp

    r279221 r284439  
    3333SOFT_LINK_FRAMEWORK_FOR_SOURCE(WebCore, CoreVideo)
    3434
     35#if HAVE(CVBUFFERCOPYATTACHMENTS)
     36SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVBufferCopyAttachments, CFDictionaryRef, (CVBufferRef buffer, CVAttachmentMode mode), (buffer, mode))
     37#endif
    3538SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVBufferGetAttachment, CFTypeRef, (CVBufferRef buffer, CFStringRef key, CVAttachmentMode* attachmentMode), (buffer, key, attachmentMode))
     39SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVBufferGetAttachments, CFDictionaryRef, (CVBufferRef buffer, CVAttachmentMode mode), (buffer, mode))
    3640SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferGetTypeID, CFTypeID, (), ())
    3741SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferGetWidth, size_t, (CVPixelBufferRef pixelBuffer), (pixelBuffer))
    … …  
    4751SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferPoolCreatePixelBuffer, CVReturn, (CFAllocatorRef allocator, CVPixelBufferPoolRef pixelBufferPool, CVPixelBufferRef* pixelBufferOut), (allocator, pixelBufferPool, pixelBufferOut))
    4852SOFT_LINK_FUNCTION_FOR_SOURCE_WITH_EXPORT(WebCore, CoreVideo, CVPixelBufferGetIOSurface, IOSurfaceRef, (CVPixelBufferRef pixelBuffer), (pixelBuffer), WEBCORE_EXPORT)
    49 SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVImageBufferGetColorSpace, CGColorSpaceRef, (CVImageBufferRef pixelBuffer), (pixelBuffer))
     53SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVImageBufferCreateColorSpaceFromAttachments, CGColorSpaceRef, (CFDictionaryRef attachments), (attachments))
    5054
    5155SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(WebCore, CoreVideo, kCVPixelBufferPixelFormatTypeKey, CFStringRef, WEBCORE_EXPORT)
    … …  
    7377SOFT_LINK_CONSTANT_MAY_FAIL_FOR_SOURCE(WebCore, CoreVideo, kCVImageBufferYCbCrMatrix_P3_D65, CFStringRef)
    7478SOFT_LINK_CONSTANT_MAY_FAIL_FOR_SOURCE(WebCore, CoreVideo, kCVImageBufferYCbCrMatrix_ITU_R_2020, CFStringRef)
     79SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, CoreVideo, kCVImageBufferCGColorSpaceKey, CFStringRef)
    7580
    7681#if USE(OPENGL_ES)

  • trunk/Source/WebCore/platform/cocoa/CoreVideoSoftLink.h

    r273568 r284439  
    3333SOFT_LINK_FRAMEWORK_FOR_HEADER(WebCore, CoreVideo)
    3434
     35#if HAVE(CVBUFFERCOPYATTACHMENTS)
     36SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVBufferCopyAttachments, CFDictionaryRef, (CVBufferRef buffer, CVAttachmentMode mode), (buffer, mode))
     37#define CVBufferCopyAttachments softLink_CoreVideo_CVBufferCopyAttachments
     38#endif
    3539SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVBufferGetAttachment, CFTypeRef, (CVBufferRef buffer, CFStringRef key, CVAttachmentMode* attachmentMode), (buffer, key, attachmentMode))
    3640#define CVBufferGetAttachment softLink_CoreVideo_CVBufferGetAttachment
     41SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVBufferGetAttachments, CFDictionaryRef, (CVBufferRef buffer, CVAttachmentMode mode), (buffer, mode))
     42#define CVBufferGetAttachments softLink_CoreVideo_CVBufferGetAttachments
    3743SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferGetTypeID, CFTypeID, (), ())
    3844#define CVPixelBufferGetTypeID softLink_CoreVideo_CVPixelBufferGetTypeID
    … …  
    6369SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferPoolGetPixelBufferAttributes, CFDictionaryRef, (CVPixelBufferPoolRef pool), (pool))
    6470#define CVPixelBufferPoolGetPixelBufferAttributes softLink_CoreVideo_CVPixelBufferPoolGetPixelBufferAttributes
    65 SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVImageBufferGetColorSpace, CGColorSpaceRef, (CVImageBufferRef pixelBuffer), (pixelBuffer))
    66 #define CVImageBufferGetColorSpace softLink_CoreVideo_CVImageBufferGetColorSpace
     71SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVImageBufferCreateColorSpaceFromAttachments, CGColorSpaceRef, (CFDictionaryRef attachments), (attachments))
     72#define CVImageBufferCreateColorSpaceFromAttachments softLink_CoreVideo_CVImageBufferCreateColorSpaceFromAttachments
    6773
    6874SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreVideo, kCVPixelBufferPixelFormatTypeKey, CFStringRef)
    … …  
    112118SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreVideo, kCVImageBufferTransferFunction_SMPTE_240M_1995, CFStringRef)
    113119#define kCVImageBufferTransferFunction_SMPTE_240M_1995 get_CoreVideo_kCVImageBufferTransferFunction_SMPTE_240M_1995()
     120SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreVideo, kCVImageBufferCGColorSpaceKey, CFStringRef)
     121#define kCVImageBufferCGColorSpaceKey get_CoreVideo_kCVImageBufferCGColorSpaceKey()
    114122
    115123#if USE(OPENGL_ES)

  • trunk/Source/WebCore/platform/graphics/MediaPlayer.cpp

    r284080 r284439  
    158158
    159159    void paint(GraphicsContext&, const FloatRect&) final { }
     160    DestinationColorSpace colorSpace() final { return DestinationColorSpace::SRGB(); }
    160161
    161162    bool hasSingleSecurityOrigin() const final { return true; }
    … …  
    10741075}
    10751076
     1077DestinationColorSpace MediaPlayer::colorSpace()
     1078{
     1079    return m_private->colorSpace();
     1080}
     1081
    10761082MediaPlayer::SupportsType MediaPlayer::supportsType(const MediaEngineSupportParameters& parameters)
    10771083{

  • trunk/Source/WebCore/platform/graphics/MediaPlayer.h

    r283605 r284439  
    470470
    471471    RefPtr<NativeImage> nativeImageForCurrentTime();
     472    DestinationColorSpace colorSpace();
    472473
    473474    using MediaPlayerEnums::NetworkState;

  • trunk/Source/WebCore/platform/graphics/MediaPlayerPrivate.h

    r283237 r284439  
    183183#endif
    184184    virtual RefPtr<NativeImage> nativeImageForCurrentTime() { return nullptr; }
     185    virtual DestinationColorSpace colorSpace() = 0;
    185186
    186187    virtual void setPreload(MediaPlayer::Preload) { }

  • trunk/Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h

    r284093 r284439  
    192192    bool didLoadingProgress() const override;
    193193    void paint(GraphicsContext&, const FloatRect&) override = 0;
     194    DestinationColorSpace colorSpace() override = 0;
    194195    void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&) override = 0;
    195196    void setPreload(MediaPlayer::Preload) override;

  • trunk/Source/WebCore/platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp

    r280776 r284439  
    906906   
    907907    m_videoFrameHasDrawn = true;
     908}
     909
     910DestinationColorSpace MediaPlayerPrivateAVFoundationCF::colorSpace()
     911{
     912    return DestinationColorSpace::SRGB();
    908913}
    909914

  • trunk/Source/WebCore/platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.h

    r268607 r284439  
    7777    void setClosedCaptionsVisible(bool) override;
    7878    virtual void paint(GraphicsContext&, const FloatRect&);
     79    DestinationColorSpace colorSpace() override;
    7980    virtual void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&);
    8081    virtual PlatformLayer* platformLayer() const;

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h

    r283585 r284439  
    260260    void paintWithVideoOutput(GraphicsContext&, const FloatRect&);
    261261    RefPtr<NativeImage> nativeImageForCurrentTime() final;
     262    DestinationColorSpace colorSpace() final;
    262263    void waitForVideoOutputMediaDataWillChange();
    263264

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r284093 r284439  
    3737#import "CDMInstanceFairPlayStreamingAVFObjC.h"
    3838#import "CDMSessionAVFoundationObjC.h"
     39#import "CVUtilities.h"
    3940#import "ColorSpaceCG.h"
    4041#import "Cookie.h"
    … …  
    25922593}
    25932594
     2595DestinationColorSpace MediaPlayerPrivateAVFoundationObjC::colorSpace()
     2596{
     2597    updateLastImage(UpdateType::UpdateSynchronously);
     2598    if (!m_lastPixelBuffer)
     2599        return DestinationColorSpace::SRGB();
     2600
     2601    return DestinationColorSpace(createCGColorSpaceForCVPixelBuffer(m_lastPixelBuffer.get()));
     2602}
     2603
    25942604void MediaPlayerPrivateAVFoundationObjC::waitForVideoOutputMediaDataWillChange()
    25952605{

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h

    r281227 r284439  
    223223    void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&) override;
    224224    RetainPtr<CVPixelBufferRef> pixelBufferForCurrentTime() final;
     225    DestinationColorSpace colorSpace() final;
    225226
    226227    bool supportsAcceleratedRendering() const override;

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm

    r284093 r284439  
    732732}
    733733
     734DestinationColorSpace MediaPlayerPrivateMediaSourceAVFObjC::colorSpace()
     735{
     736    updateLastImage();
     737    return m_lastImage ? m_lastImage->colorSpace() : DestinationColorSpace::SRGB();
     738}
     739
    734740bool MediaPlayerPrivateMediaSourceAVFObjC::hasAvailableVideoFrame() const
    735741{

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h

    r281227 r284439  
    143143    void paint(GraphicsContext&, const FloatRect&) override;
    144144    void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&) override;
     145    DestinationColorSpace colorSpace() override;
    145146    bool metaDataAvailable() const { return m_mediaStreamPrivate && m_readyState >= MediaPlayer::ReadyState::HaveMetadata; }
    146147

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm

    r284057 r284439  
    10081008}
    10091009
     1010DestinationColorSpace MediaPlayerPrivateMediaStreamAVFObjC::colorSpace()
     1011{
     1012    updateCurrentFrameImage();
     1013    return m_imagePainter.cgImage ? m_imagePainter.cgImage->colorSpace() : DestinationColorSpace::SRGB();
     1014}
     1015
    10101016void MediaPlayerPrivateMediaStreamAVFObjC::updateLayersAsNeeded()
    10111017{

  • trunk/Source/WebCore/platform/graphics/cg/ColorSpaceCG.h

    r281470 r284439  
    2727
    2828#include "ColorSpace.h"
     29#include <CoreGraphics/CoreGraphics.h>
    2930#include <optional>
     31#include <wtf/cf/TypeCastsCF.h>
    3032
    31 typedef struct CGColorSpace *CGColorSpaceRef;
     33WTF_DECLARE_CF_TYPE_TRAIT(CGColorSpace);
    3234
    3335namespace WebCore {

  • trunk/Source/WebCore/platform/graphics/cv/CVUtilities.h

    r283036 r284439  
    3131#include <wtf/RetainPtr.h>
    3232
     33using CGColorSpaceRef = struct CGColorSpace*;
    3334using CVPixelBufferPoolRef = struct __CVPixelBufferPool*;
    3435using CVPixelBufferRef = struct __CVBuffer*;
    … …  
    4445WEBCORE_EXPORT Expected<RetainPtr<CVPixelBufferRef>, CVReturn> createCVPixelBuffer(IOSurfaceRef);
    4546
     47RetainPtr<CGColorSpaceRef> createCGColorSpaceForCVPixelBuffer(CVPixelBufferRef);
    4648}

  • trunk/Source/WebCore/platform/graphics/cv/CVUtilities.mm

    r283036 r284439  
    108108}
    109109
     110RetainPtr<CGColorSpaceRef> createCGColorSpaceForCVPixelBuffer(CVPixelBufferRef buffer)
     111{
     112    if (CGColorSpaceRef colorSpace = dynamic_cf_cast<CGColorSpaceRef>(CVBufferGetAttachment(buffer, kCVImageBufferCGColorSpaceKey, nullptr)))
     113        return colorSpace;
     114
     115    RetainPtr<CFDictionaryRef> attachments;
     116#if HAVE(CVBUFFERCOPYATTACHMENTS)
     117    attachments = adoptCF(CVBufferCopyAttachments(buffer, kCVAttachmentMode_ShouldPropagate));
     118#else
     119    attachments = CVBufferGetAttachments(buffer, kCVAttachmentMode_ShouldPropagate);
     120#endif
     121    if (auto colorSpace = adoptCF(CVImageBufferCreateColorSpaceFromAttachments(attachments.get())))
     122        return colorSpace;
     123
     124    // We should only get here with content that has a broken embedded ICC
     125    // profile; in all other cases VideoToolbox should have put either known
     126    // accurate or guessed color space attachments on the pixel buffer. Content
     127    // that requires an embedded ICC profile is unlikely to be presented
     128    // correctly with any particular fallback color space we choose, so we
     129    // choose sRGB for ease.
     130    return sRGBColorSpaceRef();
    110131}
     132
     133}

  • trunk/Source/WebCore/platform/graphics/cv/PixelBufferConformerCV.cpp

    r261460 r284439  
    2727#include "PixelBufferConformerCV.h"
    2828
     29#include "CVUtilities.h"
    2930#include "GraphicsContextCG.h"
    3031#include "ImageBufferUtilitiesCG.h"
    … …  
    166167        return nullptr;
    167168
     169    auto colorSpace = createCGColorSpaceForCVPixelBuffer(rawBuffer);
     170
    168171    CVPixelBufferInfo* info = new CVPixelBufferInfo();
    169172    info->pixelBuffer = WTFMove(buffer);
    … …  
    173176    RetainPtr<CGDataProviderRef> provider = adoptCF(CGDataProviderCreateDirect(info, byteLength, &providerCallbacks));
    174177
    175     return adoptCF(CGImageCreate(width, height, 8, 32, bytesPerRow, sRGBColorSpaceRef(), bitmapInfo, provider.get(), nullptr, false, kCGRenderingIntentDefault));
     178    return adoptCF(CGImageCreate(width, height, 8, 32, bytesPerRow, colorSpace.get(), bitmapInfo, provider.get(), nullptr, false, kCGRenderingIntentDefault));
    176179}
    177180

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp

    r284368 r284439  
    32833283}
    32843284
     3285DestinationColorSpace MediaPlayerPrivateGStreamer::colorSpace()
     3286{
     3287    return DestinationColorSpace::SRGB();
     3288}
     3289
    32853290#if USE(GSTREAMER_GL)
    32863291bool MediaPlayerPrivateGStreamer::copyVideoTextureToPlatformTexture(GraphicsContextGL* context, PlatformGLObject outputTexture, GCGLenum outputTarget, GCGLint level, GCGLenum internalFormat, GCGLenum format, GCGLenum type, bool premultiplyAlpha, bool flipY)

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h

    r284368 r284439  
    182182#endif
    183183    void paint(GraphicsContext&, const FloatRect&) final;
     184    DestinationColorSpace colorSpace() final;
    184185    bool supportsFullscreen() const final;
    185186    MediaPlayer::MovieLoadType movieLoadType() const final;

  • trunk/Source/WebCore/platform/graphics/holepunch/MediaPlayerPrivateHolePunch.h

    r281227 r284439  
    8888    void paint(GraphicsContext&, const FloatRect&) final { };
    8989
     90    DestinationColorSpace colorSpace() final { return DestinationColorSpace::SRGB(); }
     91
    9092    bool supportsAcceleratedRendering() const final { return true; }
    9193

  • trunk/Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp

    r284093 r284439  
    359359}
    360360
     361DestinationColorSpace MediaPlayerPrivateMediaFoundation::colorSpace()
     362{
     363    return DestinationColorSpace::SRGB();
     364}
     365
    361366bool MediaPlayerPrivateMediaFoundation::createSession()
    362367{

  • trunk/Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.h

    r281227 r284439  
    100100
    101101    void paint(GraphicsContext&, const FloatRect&) final;
     102
     103    DestinationColorSpace colorSpace() final;
    102104
    103105private:

  • trunk/Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp

    r284057 r284439  
    299299}
    300300
     301DestinationColorSpace MockMediaPlayerMediaSource::colorSpace()
     302{
     303    return DestinationColorSpace::SRGB();
     304}
     305
    301306}
    302307

  • trunk/Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h

    r281227 r284439  
    9090    void seekWithTolerance(const MediaTime&, const MediaTime&, const MediaTime&) override;
    9191    std::optional<VideoPlaybackQualityMetrics> videoPlaybackQualityMetrics() override;
     92    DestinationColorSpace colorSpace() override;
    9293
    9394    MediaPlayer* m_player;

  • trunk/Source/WebCore/svg/graphics/SVGImage.cpp

    r283851 r284439  
    321321    }
    322322
     323    // FIXME: We should honor options.orientation(), since ImageBitmap's flipY handling relies on it. https://bugs.webkit.org/show_bug.cgi?id=231001
    323324    FloatSize scale(dstRect.size() / srcRect.size());
    324325   

  • trunk/Source/WebKit/ChangeLog

    r284429 r284439  
     12021-10-19  Cameron McCormack  <heycam@apple.com>
     2
     3        Ensure CanvasRenderingContext2D.drawImage(video) uses the right color space
     4        https://bugs.webkit.org/show_bug.cgi?id=229025
     5        <rdar://problem/81828490>
     6
     7        Reviewed by Sam Weinig.
     8
     9        When GPU process media handling is enabled, ctx.drawImage(video)
     10        calls work by sending a NativeImageForCurrentTime message to the GPU
     11        process, which returns an IOSurface around which a CGImage is created.
     12        There's no good way to get the color space off the IOSurface, so we
     13        add a second return argument to NativeImageForCurrentTime to indicate
     14        the color space used.
     15
     16        ImageBitmap creates an ImageBuffer for that image to be painted into,
     17        and for that we also need to know the color space of the video. The
     18        code that sends NativeImageForCurrentTime and then paints the
     19        resulting image into the GraphicsContext is further down. So we add
     20        a separate IPC message that gets the color space of the video's
     21        current frame. (It would be better if we could avoid a second
     22        synchronous IPC message call, but either ImageBitmap's scaling and
     23        cropping code, or the ImageBuffer creation call, would need to be moved
     24        down into MediaPlayerPrivateRemote::paintCurrentFrameInContext, which
     25        doesn't seem like the right place for it.)
     26
     27        * GPUProcess/media/RemoteImageDecoderAVFProxy.cpp:
     28        (WebKit::RemoteImageDecoderAVFProxy::createFrameImageAtIndex):
     29        * GPUProcess/media/RemoteImageDecoderAVFProxy.h:
     30        * GPUProcess/media/RemoteMediaPlayerProxy.h:
     31        * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
     32        * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
     33        (WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime):
     34        (WebKit::RemoteMediaPlayerProxy::colorSpace):
     35        * WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
     36        (WebKit::MediaPlayerPrivateRemote::colorSpace):
     37        * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
     38        * WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
     39        (WebKit::MediaPlayerPrivateRemote::nativeImageForCurrentTime): Add a
     40        second return argument to NativeImageForCurrentTime to indicate the
     41        color space of the returned IOSurface.
     42        (WebKit::MediaPlayerPrivateRemote::colorSpace): Add a new IPC message to
     43        get the color space of a video's current frame.
     44
    1452021-10-18  Commit Queue  <commit-queue@webkit.org>
    246

  • trunk/Source/WebKit/GPUProcess/media/RemoteImageDecoderAVFProxy.cpp

    r284142 r284439  
    114114}
    115115
    116 void RemoteImageDecoderAVFProxy::createFrameImageAtIndex(ImageDecoderIdentifier identifier, size_t index, CompletionHandler<void(std::optional<WTF::MachSendRight>&&, std::optional<WebCore::DestinationColorSpace>&&)>&& completionHandler)
     116void RemoteImageDecoderAVFProxy::createFrameImageAtIndex(ImageDecoderIdentifier identifier, size_t index, CompletionHandler<void(std::optional<WTF::MachSendRight>&&, WebCore::DestinationColorSpace)>&& completionHandler)
    117117{
    118118    ASSERT(m_imageDecoders.contains(identifier));
    119     std::optional<WTF::MachSendRight> sendRight;
    120     std::optional<WebCore::DestinationColorSpace> colorSpace;
    121119    if (!m_imageDecoders.contains(identifier)) {
    122         completionHandler(WTFMove(sendRight), WTFMove(colorSpace));
     120        completionHandler(std::nullopt, DestinationColorSpace::SRGB());
    123121        return;
    124122    }
    … …  
    126124    auto frameImage = m_imageDecoders.get(identifier)->createFrameImageAtIndex(index);
    127125    if (!frameImage) {
    128         completionHandler(WTFMove(sendRight), WTFMove(colorSpace));
     126        completionHandler(std::nullopt, DestinationColorSpace::SRGB());
    129127        return;
    130128    }
    131129
    132     if (auto surface = IOSurface::createFromImage(frameImage.get())) {
    133         sendRight = surface->createSendRight();
    134         colorSpace = surface->colorSpace();
     130    auto surface = IOSurface::createFromImage(frameImage.get());
     131    if (!surface) {
     132        completionHandler(std::nullopt, DestinationColorSpace::SRGB());
     133        return;
    135134    }
    136135
    137     completionHandler(WTFMove(sendRight), WTFMove(colorSpace));
     136    completionHandler(surface->createSendRight(), surface->colorSpace());
    138137}
    139138

  • trunk/Source/WebKit/GPUProcess/media/RemoteImageDecoderAVFProxy.h

    r278253 r284439  
    5757    void setExpectedContentSize(WebCore::ImageDecoderIdentifier, long long expectedContentSize);
    5858    void setData(WebCore::ImageDecoderIdentifier, const IPC::DataReference&, bool allDataReceived, CompletionHandler<void(size_t frameCount, const WebCore::IntSize& size, bool hasTrack, std::optional<Vector<WebCore::ImageDecoder::FrameInfo>>&&)>&&);
    59     void createFrameImageAtIndex(WebCore::ImageDecoderIdentifier, size_t index, CompletionHandler<void(std::optional<WTF::MachSendRight>&&, std::optional<WebCore::DestinationColorSpace>&&)>&&);
     59    void createFrameImageAtIndex(WebCore::ImageDecoderIdentifier, size_t index, CompletionHandler<void(std::optional<WTF::MachSendRight>&&, WebCore::DestinationColorSpace)>&&);
    6060    void clearFrameBufferCache(WebCore::ImageDecoderIdentifier, size_t index);
    6161

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h

    r284294 r284439  
    312312
    313313#if PLATFORM(COCOA)
    314     void nativeImageForCurrentTime(CompletionHandler<void(std::optional<WTF::MachSendRight>&&)>&&);
     314    void nativeImageForCurrentTime(CompletionHandler<void(std::optional<WTF::MachSendRight>&&, WebCore::DestinationColorSpace)>&&);
     315    void colorSpace(CompletionHandler<void(WebCore::DestinationColorSpace)>&&);
    315316#endif
    316317#if USE(AVFOUNDATION)

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in

    r284294 r284439  
    124124
    125125#if PLATFORM(COCOA)
    126     NativeImageForCurrentTime() -> (std::optional<MachSendRight> sendRight) Synchronous
     126    NativeImageForCurrentTime() -> (std::optional<MachSendRight> sendRight, WebCore::DestinationColorSpace colorSpace) Synchronous
     127    ColorSpace() -> (WebCore::DestinationColorSpace colorSpace) Synchronous
    127128#endif
    128129#if USE(AVFOUNDATION)

  • trunk/Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm

    r284102 r284439  
    8484}
    8585
    86 void RemoteMediaPlayerProxy::nativeImageForCurrentTime(CompletionHandler<void(std::optional<WTF::MachSendRight>&&)>&& completionHandler)
     86void RemoteMediaPlayerProxy::nativeImageForCurrentTime(CompletionHandler<void(std::optional<WTF::MachSendRight>&&, WebCore::DestinationColorSpace)>&& completionHandler)
    8787{
    8888    if (!m_player) {
    89         completionHandler(std::nullopt);
     89        completionHandler(std::nullopt, DestinationColorSpace::SRGB());
    9090        return;
    9191    }
    … …  
    9393    auto nativeImage = m_player->nativeImageForCurrentTime();
    9494    if (!nativeImage) {
    95         completionHandler(std::nullopt);
     95        completionHandler(std::nullopt, DestinationColorSpace::SRGB());
    9696        return;
    9797    }
    … …  
    9999    auto platformImage = nativeImage->platformImage();
    100100    if (!platformImage) {
    101         completionHandler(std::nullopt);
     101        completionHandler(std::nullopt, DestinationColorSpace::SRGB());
    102102        return;
    103103    }
    … …  
    105105    auto surface = WebCore::IOSurface::createFromImage(platformImage.get());
    106106    if (!surface) {
    107         completionHandler(std::nullopt);
     107        completionHandler(std::nullopt, DestinationColorSpace::SRGB());
    108108        return;
    109109    }
    110110
    111     completionHandler(surface->createSendRight());
     111    completionHandler(surface->createSendRight(), nativeImage->colorSpace());
     112}
     113
     114void RemoteMediaPlayerProxy::colorSpace(CompletionHandler<void(WebCore::DestinationColorSpace)>&& completionHandler)
     115{
     116    if (!m_player) {
     117        completionHandler(DestinationColorSpace::SRGB());
     118        return;
     119    }
     120
     121    completionHandler(m_player->colorSpace());
    112122}
    113123

  • trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp

    r284294 r284439  
    997997    return nullptr;
    998998}
     999
     1000DestinationColorSpace MediaPlayerPrivateRemote::colorSpace()
     1001{
     1002    notImplemented();
     1003    return DestinationColorSpace::SRGB();
     1004}
    9991005#endif
    10001006

  • trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h

    r284102 r284439  
    291291#endif
    292292    RefPtr<WebCore::NativeImage> nativeImageForCurrentTime() final;
     293    WebCore::DestinationColorSpace colorSpace() final;
    293294
    294295    WebCore::MediaPlayerIdentifier identifier() const final;

  • trunk/Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm

    r284102 r284439  
    5050{
    5151    std::optional<MachSendRight> sendRight;
    52     if (!connection().sendSync(Messages::RemoteMediaPlayerProxy::NativeImageForCurrentTime(), Messages::RemoteMediaPlayerProxy::NativeImageForCurrentTime::Reply(sendRight), m_id))
     52    auto colorSpace = DestinationColorSpace::SRGB();
     53    if (!connection().sendSync(Messages::RemoteMediaPlayerProxy::NativeImageForCurrentTime(), Messages::RemoteMediaPlayerProxy::NativeImageForCurrentTime::Reply(sendRight, colorSpace), m_id))
    5354        return nullptr;
    5455
    … …  
    5657        return nullptr;
    5758
    58     auto surface = WebCore::IOSurface::createFromSendRight(WTFMove(*sendRight), WebCore::DestinationColorSpace::SRGB());
     59    auto surface = WebCore::IOSurface::createFromSendRight(WTFMove(*sendRight), colorSpace);
    5960    if (!surface)
    6061        return nullptr;
    … …  
    6566
    6667    return NativeImage::create(WTFMove(platformImage));
     68}
     69
     70WebCore::DestinationColorSpace MediaPlayerPrivateRemote::colorSpace()
     71{
     72    auto colorSpace = DestinationColorSpace::SRGB();
     73    connection().sendSync(Messages::RemoteMediaPlayerProxy::ColorSpace(), Messages::RemoteMediaPlayerProxy::ColorSpace::Reply(colorSpace), m_id);
     74    return colorSpace;
    6775}
    6876
Note: See TracChangeset for help on using the changeset viewer.