Changeset 246611 in webkit


Ignore:
Timestamp:
Jun 19, 2019 2:33:35 PM (5 years ago)
Author:
jer.noble@apple.com
Message:

iOS 12.2 Drawing portrait video to canvas is sideways
https://bugs.webkit.org/show_bug.cgi?id=196772
<rdar://problem/49781802>

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/video-orientation-canvas.html

Move rotation code into its own ImageRotationSessionVT class for re-use across
all existing classes with rotation operations. Should slightly increase performance
for painting rotated media files, as the rotation only occurs once per frame, rather
than once per drawing operation.

  • WebCore.xcodeproj/project.pbxproj:
  • platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.h:

(WebCore::ImageDecoderAVFObjC::RotationProperties::isIdentity const): Deleted.

  • platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.mm:

(WebCore::ImageDecoderAVFObjC::readTrackMetadata):
(WebCore::ImageDecoderAVFObjC::storeSampleBuffer):
(WebCore::ImageDecoderAVFObjC::setTrack):
(WebCore::transformToRotationProperties): Deleted.

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::tracksChanged):
(WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastPixelBuffer):
(WebCore::MediaPlayerPrivateAVFoundationObjC::paintWithVideoOutput):

  • platform/graphics/cv/ImageRotationSessionVT.h: Added.

(WebCore::ImageRotationSessionVT::RotationProperties::isIdentity const):
(WebCore::ImageRotationSessionVT::rotationProperties const):
(WebCore::ImageRotationSessionVT::rotatedSize):

  • platform/graphics/cv/ImageRotationSessionVT.mm: Added.

(WebCore::transformToRotationProperties):
(WebCore::ImageRotationSessionVT::ImageRotationSessionVT):
(WebCore::ImageRotationSessionVT::rotate):

  • platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp:
  • platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.h:
  • platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.mm:

(WebCore::rotationToAngle):
(WebCore::RealtimeOutgoingVideoSourceCocoa::rotatePixelBuffer):
(WebCore::computeRotatedWidthAndHeight): Deleted.

LayoutTests:

  • media/content/no-rotation.mp4:
  • media/media-source/only-bcp47-language-tags-accepted-as-valid-expected.txt:
  • media/video-orientation-canvas-expected.html: Added.
  • media/video-orientation-canvas.html: Added.
  • media/video-test.js:

(waitFor):

Location:
trunk
Files:
4 added
13 edited

Legend:

Unmodified
Added
Removed
  • trunk/LayoutTests/ChangeLog

    r246604 r246611  
     12019-06-19  Jer Noble  <jer.noble@apple.com>
     2
     3        iOS 12.2 Drawing portrait video to canvas is sideways
     4        https://bugs.webkit.org/show_bug.cgi?id=196772
     5        <rdar://problem/49781802>
     6
     7        Reviewed by Eric Carlson.
     8
     9        * media/content/no-rotation.mp4:
     10        * media/media-source/only-bcp47-language-tags-accepted-as-valid-expected.txt:
     11        * media/video-orientation-canvas-expected.html: Added.
     12        * media/video-orientation-canvas.html: Added.
     13        * media/video-test.js:
     14        (waitFor):
     15
    1162019-06-19  Alicia Boya García  <aboya@igalia.com>
    217
  • trunk/LayoutTests/media/media-source/only-bcp47-language-tags-accepted-as-valid-expected.txt

    r231817 r246611  
    1 CONSOLE MESSAGE: line 221: The language 'a' is not a valid BCP 47 language tag.
     1CONSOLE MESSAGE: line 222: The language 'a' is not a valid BCP 47 language tag.
    22CONSOLE MESSAGE: line 106: The language 'a' is not a valid BCP 47 language tag.
    3 CONSOLE MESSAGE: line 221: The language 'a' is not a valid BCP 47 language tag.
    4 CONSOLE MESSAGE: line 221: The language '1' is not a valid BCP 47 language tag.
     3CONSOLE MESSAGE: line 222: The language 'a' is not a valid BCP 47 language tag.
     4CONSOLE MESSAGE: line 222: The language '1' is not a valid BCP 47 language tag.
    55CONSOLE MESSAGE: line 106: The language '1' is not a valid BCP 47 language tag.
    6 CONSOLE MESSAGE: line 221: The language '1' is not a valid BCP 47 language tag.
    7 CONSOLE MESSAGE: line 221: The language 'ab-abcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghij' is not a valid BCP 47 language tag.
     6CONSOLE MESSAGE: line 222: The language '1' is not a valid BCP 47 language tag.
     7CONSOLE MESSAGE: line 222: The language 'ab-abcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghij' is not a valid BCP 47 language tag.
    88CONSOLE MESSAGE: line 106: The language 'ab-abcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghij' is not a valid BCP 47 language tag.
    9 CONSOLE MESSAGE: line 221: The language 'ab-abcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghij' is not a valid BCP 47 language tag.
    10 CONSOLE MESSAGE: line 221: The language '1a' is not a valid BCP 47 language tag.
     9CONSOLE MESSAGE: line 222: The language 'ab-abcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghij' is not a valid BCP 47 language tag.
     10CONSOLE MESSAGE: line 222: The language '1a' is not a valid BCP 47 language tag.
    1111CONSOLE MESSAGE: line 106: The language '1a' is not a valid BCP 47 language tag.
    12 CONSOLE MESSAGE: line 221: The language '1a' is not a valid BCP 47 language tag.
    13 CONSOLE MESSAGE: line 221: The language '-a' is not a valid BCP 47 language tag.
     12CONSOLE MESSAGE: line 222: The language '1a' is not a valid BCP 47 language tag.
     13CONSOLE MESSAGE: line 222: The language '-a' is not a valid BCP 47 language tag.
    1414CONSOLE MESSAGE: line 106: The language '-a' is not a valid BCP 47 language tag.
    15 CONSOLE MESSAGE: line 221: The language '-a' is not a valid BCP 47 language tag.
    16 CONSOLE MESSAGE: line 221: The language 'a-' is not a valid BCP 47 language tag.
     15CONSOLE MESSAGE: line 222: The language '-a' is not a valid BCP 47 language tag.
     16CONSOLE MESSAGE: line 222: The language 'a-' is not a valid BCP 47 language tag.
    1717CONSOLE MESSAGE: line 106: The language 'a-' is not a valid BCP 47 language tag.
    18 CONSOLE MESSAGE: line 221: The language 'a-' is not a valid BCP 47 language tag.
    19 CONSOLE MESSAGE: line 221: The language 'a1' is not a valid BCP 47 language tag.
     18CONSOLE MESSAGE: line 222: The language 'a-' is not a valid BCP 47 language tag.
     19CONSOLE MESSAGE: line 222: The language 'a1' is not a valid BCP 47 language tag.
    2020CONSOLE MESSAGE: line 106: The language 'a1' is not a valid BCP 47 language tag.
    21 CONSOLE MESSAGE: line 221: The language 'a1' is not a valid BCP 47 language tag.
    22 CONSOLE MESSAGE: line 221: The language 'aa1' is not a valid BCP 47 language tag.
     21CONSOLE MESSAGE: line 222: The language 'a1' is not a valid BCP 47 language tag.
     22CONSOLE MESSAGE: line 222: The language 'aa1' is not a valid BCP 47 language tag.
    2323CONSOLE MESSAGE: line 106: The language 'aa1' is not a valid BCP 47 language tag.
    24 CONSOLE MESSAGE: line 221: The language 'aa1' is not a valid BCP 47 language tag.
    25 CONSOLE MESSAGE: line 221: The language 'aaaa' is not a valid BCP 47 language tag.
     24CONSOLE MESSAGE: line 222: The language 'aa1' is not a valid BCP 47 language tag.
     25CONSOLE MESSAGE: line 222: The language 'aaaa' is not a valid BCP 47 language tag.
    2626CONSOLE MESSAGE: line 106: The language 'aaaa' is not a valid BCP 47 language tag.
    27 CONSOLE MESSAGE: line 221: The language 'aaaa' is not a valid BCP 47 language tag.
    28 CONSOLE MESSAGE: line 221: The language 'aaa1' is not a valid BCP 47 language tag.
     27CONSOLE MESSAGE: line 222: The language 'aaaa' is not a valid BCP 47 language tag.
     28CONSOLE MESSAGE: line 222: The language 'aaa1' is not a valid BCP 47 language tag.
    2929CONSOLE MESSAGE: line 106: The language 'aaa1' is not a valid BCP 47 language tag.
    30 CONSOLE MESSAGE: line 221: The language 'aaa1' is not a valid BCP 47 language tag.
    31 CONSOLE MESSAGE: line 221: The language 'inv-alid-char space' is not a valid BCP 47 language tag.
     30CONSOLE MESSAGE: line 222: The language 'aaa1' is not a valid BCP 47 language tag.
     31CONSOLE MESSAGE: line 222: The language 'inv-alid-char space' is not a valid BCP 47 language tag.
    3232CONSOLE MESSAGE: line 106: The language 'inv-alid-char space' is not a valid BCP 47 language tag.
    33 CONSOLE MESSAGE: line 221: The language 'inv-alid-char space' is not a valid BCP 47 language tag.
    34 CONSOLE MESSAGE: line 221: The language 'inv-alid-char–longDash' is not a valid BCP 47 language tag.
     33CONSOLE MESSAGE: line 222: The language 'inv-alid-char space' is not a valid BCP 47 language tag.
     34CONSOLE MESSAGE: line 222: The language 'inv-alid-char–longDash' is not a valid BCP 47 language tag.
    3535CONSOLE MESSAGE: line 106: The language 'inv-alid-char–longDash' is not a valid BCP 47 language tag.
    36 CONSOLE MESSAGE: line 221: The language 'inv-alid-char–longDash' is not a valid BCP 47 language tag.
    37 CONSOLE MESSAGE: line 221: The language 'inv-alid-char-PÃ¥lska' is not a valid BCP 47 language tag.
     36CONSOLE MESSAGE: line 222: The language 'inv-alid-char–longDash' is not a valid BCP 47 language tag.
     37CONSOLE MESSAGE: line 222: The language 'inv-alid-char-PÃ¥lska' is not a valid BCP 47 language tag.
    3838CONSOLE MESSAGE: line 106: The language 'inv-alid-char-PÃ¥lska' is not a valid BCP 47 language tag.
    39 CONSOLE MESSAGE: line 221: The language 'inv-alid-char-PÃ¥lska' is not a valid BCP 47 language tag.
    40 CONSOLE MESSAGE: line 221: The language 'inv-alid-char-*' is not a valid BCP 47 language tag.
     39CONSOLE MESSAGE: line 222: The language 'inv-alid-char-PÃ¥lska' is not a valid BCP 47 language tag.
     40CONSOLE MESSAGE: line 222: The language 'inv-alid-char-*' is not a valid BCP 47 language tag.
    4141CONSOLE MESSAGE: line 106: The language 'inv-alid-char-*' is not a valid BCP 47 language tag.
    42 CONSOLE MESSAGE: line 221: The language 'inv-alid-char-*' is not a valid BCP 47 language tag.
    43 CONSOLE MESSAGE: line 221: The language 'inv-alid-char-' is not a valid BCP 47 language tag.
     42CONSOLE MESSAGE: line 222: The language 'inv-alid-char-*' is not a valid BCP 47 language tag.
     43CONSOLE MESSAGE: line 222: The language 'inv-alid-char-' is not a valid BCP 47 language tag.
    4444CONSOLE MESSAGE: line 106: The language 'inv-alid-char-' is not a valid BCP 47 language tag.
    45 CONSOLE MESSAGE: line 221: The language 'inv-alid-char-' is not a valid BCP 47 language tag.
     45CONSOLE MESSAGE: line 222: The language 'inv-alid-char-' is not a valid BCP 47 language tag.
    4646Test that only BCP47 language tags are accepted as valid but still reflected.
    4747
  • trunk/LayoutTests/media/video-test.js

    r236531 r246611  
    187187}
    188188
    189 function waitFor(element, type) {
     189function waitFor(element, type, silent) {
    190190    return new Promise(resolve => {
    191191        element.addEventListener(type, event => {
    192             consoleWrite(`EVENT(${event.type})`);
     192            if (!silent)
     193                consoleWrite(`EVENT(${event.type})`);
    193194            resolve(event);
    194195        }, { once: true });
  • trunk/Source/WebCore/ChangeLog

    r246596 r246611  
     12019-06-19  Jer Noble  <jer.noble@apple.com>
     2
     3        iOS 12.2 Drawing portrait video to canvas is sideways
     4        https://bugs.webkit.org/show_bug.cgi?id=196772
     5        <rdar://problem/49781802>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Test: media/video-orientation-canvas.html
     10
     11        Move rotation code into its own ImageRotationSessionVT class for re-use across
     12        all existing classes with rotation operations. Should slightly increase performance
     13        for painting rotated media files, as the rotation only occurs once per frame, rather
     14        than once per drawing operation.
     15
     16        * WebCore.xcodeproj/project.pbxproj:
     17        * platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.h:
     18        (WebCore::ImageDecoderAVFObjC::RotationProperties::isIdentity const): Deleted.
     19        * platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.mm:
     20        (WebCore::ImageDecoderAVFObjC::readTrackMetadata):
     21        (WebCore::ImageDecoderAVFObjC::storeSampleBuffer):
     22        (WebCore::ImageDecoderAVFObjC::setTrack):
     23        (WebCore::transformToRotationProperties): Deleted.
     24        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
     25        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     26        (WebCore::MediaPlayerPrivateAVFoundationObjC::tracksChanged):
     27        (WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastPixelBuffer):
     28        (WebCore::MediaPlayerPrivateAVFoundationObjC::paintWithVideoOutput):
     29        * platform/graphics/cv/ImageRotationSessionVT.h: Added.
     30        (WebCore::ImageRotationSessionVT::RotationProperties::isIdentity const):
     31        (WebCore::ImageRotationSessionVT::rotationProperties const):
     32        (WebCore::ImageRotationSessionVT::rotatedSize):
     33        * platform/graphics/cv/ImageRotationSessionVT.mm: Added.
     34        (WebCore::transformToRotationProperties):
     35        (WebCore::ImageRotationSessionVT::ImageRotationSessionVT):
     36        (WebCore::ImageRotationSessionVT::rotate):
     37        * platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp:
     38        * platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.h:
     39        * platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.mm:
     40        (WebCore::rotationToAngle):
     41        (WebCore::RealtimeOutgoingVideoSourceCocoa::rotatePixelBuffer):
     42        (WebCore::computeRotatedWidthAndHeight): Deleted.
     43
    1442019-06-19  Adrian Perez de Castro  <aperez@igalia.com>
    245
  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r246554 r246611  
    40274027                CD225C0B1C46FBF400140761 /* WebCoreNSURLSession.mm in Sources */ = {isa = PBXBuildFile; fileRef = CD225C091C46FBF400140761 /* WebCoreNSURLSession.mm */; };
    40284028                CD225C0C1C46FBF400140761 /* WebCoreNSURLSession.h in Headers */ = {isa = PBXBuildFile; fileRef = CD225C0A1C46FBF400140761 /* WebCoreNSURLSession.h */; settings = {ATTRIBUTES = (Private, ); }; };
     4029                CD27AE5022A9868700947FF9 /* ImageRotationSessionVT.h in Headers */ = {isa = PBXBuildFile; fileRef = CD27AE4E22A9868700947FF9 /* ImageRotationSessionVT.h */; };
     4030                CD27AE5122A9868700947FF9 /* ImageRotationSessionVT.mm in Sources */ = {isa = PBXBuildFile; fileRef = CD27AE4F22A9868700947FF9 /* ImageRotationSessionVT.mm */; };
    40294031                CD2F4A2418D89F700063746D /* AudioHardwareListener.h in Headers */ = {isa = PBXBuildFile; fileRef = CD2F4A2218D89F700063746D /* AudioHardwareListener.h */; settings = {ATTRIBUTES = (Private, ); }; };
    40304032                CD2F4A2818D8A3490063746D /* AudioHardwareListenerMac.h in Headers */ = {isa = PBXBuildFile; fileRef = CD2F4A2618D8A3490063746D /* AudioHardwareListenerMac.h */; };
     
    1351213514                CD225C091C46FBF400140761 /* WebCoreNSURLSession.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebCoreNSURLSession.mm; sourceTree = "<group>"; };
    1351313515                CD225C0A1C46FBF400140761 /* WebCoreNSURLSession.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebCoreNSURLSession.h; sourceTree = "<group>"; };
     13516                CD27AE4E22A9868700947FF9 /* ImageRotationSessionVT.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = ImageRotationSessionVT.h; sourceTree = "<group>"; };
     13517                CD27AE4F22A9868700947FF9 /* ImageRotationSessionVT.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = ImageRotationSessionVT.mm; sourceTree = "<group>"; };
    1351413518                CD27F6E014575C1B0078207D /* MediaController.idl */ = {isa = PBXFileReference; lastKnownFileType = text; path = MediaController.idl; sourceTree = "<group>"; };
    1351513519                CD27F6E2145767580078207D /* JSMediaController.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSMediaController.cpp; sourceTree = "<group>"; };
     
    2567025674                        isa = PBXGroup;
    2567125675                        children = (
     25676                                CD27AE4E22A9868700947FF9 /* ImageRotationSessionVT.h */,
     25677                                CD27AE4F22A9868700947FF9 /* ImageRotationSessionVT.mm */,
    2567225678                                0746D30C2146EA38003DDF84 /* ImageTransferSessionVT.h */,
    2567325679                                0746D30A2146EA37003DDF84 /* ImageTransferSessionVT.mm */,
     
    2950929515                                B51A2F3F17D7D3AE0072517A /* ImageQualityController.h in Headers */,
    2951029516                                49291E4B134172C800E753DE /* ImageRenderingMode.h in Headers */,
     29517                                CD27AE5022A9868700947FF9 /* ImageRotationSessionVT.h in Headers */,
    2951129518                                7C193BC11F5E0EED0088F3E6 /* ImageSmoothingQuality.h in Headers */,
    2951229519                                5546757B1FD212A9003B10B0 /* ImageSource.h in Headers */,
     
    3276332770                                1AC900C31943C0FC008625B5 /* HTTPHeaderNames.cpp in Sources */,
    3276432771                                CD19FEAF1F574B6D000C42FB /* ImageDecoderAVFObjC.mm in Sources */,
     32772                                CD27AE5122A9868700947FF9 /* ImageRotationSessionVT.mm in Sources */,
    3276532773                                BE961C5418AD338500D07DC5 /* InbandDataTextTrack.cpp in Sources */,
    3276632774                                BE16C59217CFE17200852C04 /* InbandGenericTextTrack.cpp in Sources */,
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.h

    r239636 r246611  
    3939OBJC_CLASS WebCoreSharedBufferResourceLoaderDelegate;
    4040typedef struct opaqueCMSampleBuffer* CMSampleBufferRef;
    41 typedef struct OpaqueVTImageRotationSession* VTImageRotationSessionRef;
    42 typedef struct __CVPixelBufferPool* CVPixelBufferPoolRef;
    4341
    4442namespace WTF {
     
    5048class ContentType;
    5149class ImageDecoderAVFObjCSample;
     50class ImageRotationSessionVT;
    5251class PixelBufferConformerCV;
    5352class WebCoreDecompressionSession;
     
    9190    void clearFrameBufferCache(size_t) final;
    9291
    93     struct RotationProperties {
    94         bool flipX { false };
    95         bool flipY { false };
    96         unsigned angle { 0 };
    97 
    98         bool isIdentity() const { return !flipX && !flipY && !angle; }
    99     };
    100 
    10192private:
    10293    ImageDecoderAVFObjC(SharedBuffer&, const String& mimeType, AlphaOption, GammaAndColorProfileOption);
     
    117108    RetainPtr<AVAssetTrack> m_track;
    118109    RetainPtr<WebCoreSharedBufferResourceLoaderDelegate> m_loader;
    119     RetainPtr<VTImageRotationSessionRef> m_rotationSession;
    120     RetainPtr<CVPixelBufferPoolRef> m_rotationPool;
     110    std::unique_ptr<ImageRotationSessionVT> m_imageRotationSession;
    121111    Ref<WebCoreDecompressionSession> m_decompressionSession;
    122112    WTF::Function<void(EncodedDataStatus)> m_encodedDataStatusChangedCallback;
     
    127117    bool m_isAllDataReceived { false };
    128118    Optional<IntSize> m_size;
    129     Optional<RotationProperties> m_rotation;
    130119};
    131120
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.mm

    r244980 r246611  
    3535#import "FloatRect.h"
    3636#import "FloatSize.h"
     37#import "ImageRotationSessionVT.h"
    3738#import "Logging.h"
    3839#import "MIMETypeRegistry.h"
     
    234235}
    235236
    236 static ImageDecoderAVFObjC::RotationProperties transformToRotationProperties(AffineTransform inTransform)
    237 {
    238     ImageDecoderAVFObjC::RotationProperties rotation;
    239     if (inTransform.isIdentity())
    240         return rotation;
    241 
    242     AffineTransform::DecomposedType decomposed { };
    243     if (!inTransform.decompose(decomposed))
    244         return rotation;
    245 
    246     rotation.flipY = WTF::areEssentiallyEqual(decomposed.scaleX, -1.);
    247     rotation.flipX = WTF::areEssentiallyEqual(decomposed.scaleY, -1.);
    248     auto degrees = rad2deg(decomposed.angle);
    249     while (degrees < 0)
    250         degrees += 360;
    251 
    252     // Only support rotation in multiples of 90º:
    253     if (WTF::areEssentiallyEqual(fmod(degrees, 90.), 0.))
    254         rotation.angle = clampToUnsigned(degrees);
    255 
    256     return rotation;
    257 }
    258 
    259237class ImageDecoderAVFObjCSample : public MediaSampleAVFObjC {
    260238public:
     
    423401void ImageDecoderAVFObjC::readTrackMetadata()
    424402{
    425     if (!m_rotation)
    426         m_rotation = transformToRotationProperties(CGAffineTransformConcat(m_asset.get().preferredTransform, m_track.get().preferredTransform));
    427 
    428     if (!m_size) {
    429         auto size = FloatSize(m_track.get().naturalSize);
    430         auto angle = m_rotation.value().angle;
    431         if (angle == 90 || angle == 270)
    432             size = size.transposedSize();
    433 
    434         m_size = expandedIntSize(size);
    435     }
     403    AffineTransform finalTransform = CGAffineTransformConcat(m_asset.get().preferredTransform, m_track.get().preferredTransform);
     404    auto size = expandedIntSize(FloatSize(m_track.get().naturalSize));
     405    if (finalTransform.isIdentity()) {
     406        m_size = size;
     407        m_imageRotationSession = nullptr;
     408        return;
     409    }
     410
     411    if (!m_imageRotationSession
     412        || !m_imageRotationSession->transform()
     413        || m_imageRotationSession->transform().value() != finalTransform
     414        || m_imageRotationSession->size() != size)
     415        m_imageRotationSession = std::make_unique<ImageRotationSessionVT>(WTFMove(finalTransform), size, kCVPixelFormatType_32BGRA, ImageRotationSessionVT::IsCGImageCompatible::Yes);
     416
     417    m_size = expandedIntSize(m_imageRotationSession->rotatedSize());
    436418}
    437419
     
    447429    auto iter = m_sampleData.presentationOrder().findSampleWithPresentationTime(presentationTime);
    448430
    449     if (m_rotation && !m_rotation.value().isIdentity()) {
    450         auto& rotation = m_rotation.value();
    451         if (!m_rotationSession) {
    452             VTImageRotationSessionRef rawRotationSession = nullptr;
    453             VTImageRotationSessionCreate(kCFAllocatorDefault, rotation.angle, &rawRotationSession);
    454             m_rotationSession = rawRotationSession;
    455             VTImageRotationSessionSetProperty(m_rotationSession.get(), kVTImageRotationPropertyKey_EnableHighSpeedTransfer, kCFBooleanTrue);
    456 
    457             if (rotation.flipY)
    458                 VTImageRotationSessionSetProperty(m_rotationSession.get(), kVTImageRotationPropertyKey_FlipVerticalOrientation, kCFBooleanTrue);
    459             if (rotation.flipX)
    460                 VTImageRotationSessionSetProperty(m_rotationSession.get(), kVTImageRotationPropertyKey_FlipHorizontalOrientation, kCFBooleanTrue);
    461         }
    462 
    463         if (!m_rotationPool) {
    464             auto pixelAttributes = @{
    465                 (__bridge NSString *)kCVPixelBufferWidthKey: @(m_size.value().width()),
    466                 (__bridge NSString *)kCVPixelBufferHeightKey: @(m_size.value().height()),
    467                 (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_32BGRA),
    468                 (__bridge NSString *)kCVPixelBufferCGImageCompatibilityKey: @YES,
    469             };
    470             CVPixelBufferPoolRef rawPool = nullptr;
    471             CVPixelBufferPoolCreate(kCFAllocatorDefault, nullptr, (__bridge CFDictionaryRef)pixelAttributes, &rawPool);
    472             m_rotationPool = adoptCF(rawPool);
    473         }
    474 
    475         CVPixelBufferRef rawRotatedBuffer = nullptr;
    476         CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, m_rotationPool.get(), &rawRotatedBuffer);
    477         auto status = VTImageRotationSessionTransferImage(m_rotationSession.get(), pixelBuffer.get(), rawRotatedBuffer);
    478         if (status == noErr)
    479             pixelBuffer = adoptCF(rawRotatedBuffer);
    480     }
     431    if (m_imageRotationSession)
     432        pixelBuffer = m_imageRotationSession->rotate(pixelBuffer.get());
    481433
    482434    CGImageRef rawImage = nullptr;
     
    507459    m_sampleData.clear();
    508460    m_size.reset();
    509     m_rotation.reset();
    510461    m_cursor = m_sampleData.decodeOrder().end();
    511     m_rotationSession = nullptr;
     462    m_imageRotationSession = nullptr;
    512463
    513464    [track loadValuesAsynchronouslyForKeys:@[@"naturalSize", @"preferredTransform"] completionHandler:[protectedThis = makeRefPtr(this)] () mutable {
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h

    r245868 r246611  
    5959class CDMInstanceFairPlayStreamingAVFObjC;
    6060class CDMSessionAVFoundationObjC;
     61class ImageRotationSessionVT;
    6162class InbandMetadataTextTrackPrivateAVF;
    6263class MediaSelectionGroupAVFObjC;
     
    326327
    327328    void setShouldDisableSleep(bool) override;
     329    void updateRotationSession();
    328330
    329331    Optional<VideoPlaybackQualityMetrics> videoPlaybackQualityMetrics() final;
     
    361363    RetainPtr<CGImageRef> m_lastImage;
    362364    BinarySemaphore m_videoOutputSemaphore;
     365    std::unique_ptr<ImageRotationSessionVT> m_imageRotationSession;
    363366    std::unique_ptr<VideoTextureCopierCV> m_videoTextureCopier;
    364367#endif
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r246490 r246611  
    4444#import "GraphicsContext3D.h"
    4545#import "GraphicsContextCG.h"
     46#import "ImageRotationSessionVT.h"
    4647#import "InbandMetadataTextTrackPrivateAVF.h"
    4748#import "InbandTextTrackPrivateAVFObjC.h"
     
    18641865}
    18651866
     1867void MediaPlayerPrivateAVFoundationObjC::updateRotationSession()
     1868{
     1869    AffineTransform finalTransform = m_avAsset.get().preferredTransform;
     1870    FloatSize naturalSize;
     1871    if (auto* firstEnabledVideoTrack = firstEnabledTrack([m_avAsset.get() tracksWithMediaCharacteristic:AVMediaCharacteristicVisual])) {
     1872        naturalSize = FloatSize(firstEnabledVideoTrack.naturalSize);
     1873        finalTransform *= firstEnabledVideoTrack.preferredTransform;
     1874    }
     1875
     1876    if (finalTransform.isIdentity()) {
     1877        m_imageRotationSession = nullptr;
     1878        return;
     1879    }
     1880
     1881    if (m_imageRotationSession
     1882        && m_imageRotationSession->transform()
     1883        && m_imageRotationSession->transform().value() == finalTransform
     1884        && m_imageRotationSession->size() == naturalSize)
     1885        return;
     1886
     1887    m_imageRotationSession = std::make_unique<ImageRotationSessionVT>(WTFMove(finalTransform), naturalSize, kCVPixelFormatType_32BGRA, ImageRotationSessionVT::IsCGImageCompatible::Yes);
     1888}
     1889
    18661890#if ENABLE(VIDEO_TRACK)
    18671891
     
    20542078        return;
    20552079
     2080    updateRotationSession();
    20562081    setNaturalSize(m_cachedPresentationSize);
    20572082}
     
    21412166
    21422167    m_lastPixelBuffer = adoptCF([m_videoOutput.get() copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil]);
     2168
     2169    if (m_imageRotationSession)
     2170        m_lastPixelBuffer = m_imageRotationSession->rotate(m_lastPixelBuffer.get());
     2171
    21432172    m_lastImage = nullptr;
    21442173    return true;
     
    22072236    INFO_LOG(LOGIDENTIFIER);
    22082237
    2209     GraphicsContextStateSaver stateSaver(context);
    22102238    FloatRect imageRect(0, 0, CGImageGetWidth(m_lastImage.get()), CGImageGetHeight(m_lastImage.get()));
    2211     AffineTransform videoTransform = [firstEnabledVideoTrack preferredTransform];
    2212     FloatRect transformedOutputRect = videoTransform.inverse().valueOr(AffineTransform()).mapRect(outputRect);
    2213 
    2214     context.concatCTM(videoTransform);
    2215     context.drawNativeImage(m_lastImage.get(), imageRect.size(), transformedOutputRect, imageRect);
     2239    context.drawNativeImage(m_lastImage.get(), imageRect.size(), outputRect, imageRect);
    22162240
    22172241    // If we have created an AVAssetImageGenerator in the past due to m_videoOutput not having an available
  • trunk/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp

    r240120 r246611  
    2929#if USE(LIBWEBRTC)
    3030
     31#include "ImageRotationSessionVT.h"
    3132#include "Logging.h"
    3233#include "RealtimeIncomingVideoSourceCocoa.h"
  • trunk/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.h

    r245033 r246611  
    3232#include <webrtc/api/video/video_rotation.h>
    3333
    34 typedef struct OpaqueVTImageRotationSession* VTImageRotationSessionRef;
    35 typedef struct __CVPixelBufferPool* CVPixelBufferPoolRef;
     34namespace WebCore {
    3635
    37 namespace WebCore {
     36class ImageRotationSessionVT;
    3837
    3938class RealtimeOutgoingVideoSourceCocoa final : public RealtimeOutgoingVideoSource {
     
    5352
    5453    std::unique_ptr<PixelBufferConformerCV> m_pixelBufferConformer;
    55     RetainPtr<VTImageRotationSessionRef> m_rotationSession;
    56     RetainPtr<CVPixelBufferPoolRef> m_rotationPool;
     54    std::unique_ptr<ImageRotationSessionVT> m_rotationSession;
    5755    webrtc::VideoRotation m_currentRotationSessionAngle { webrtc::kVideoRotation_0 };
    5856    size_t m_rotatedWidth { 0 };
  • trunk/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.mm

    r245033 r246611  
    2929#if USE(LIBWEBRTC)
    3030
     31#import "AffineTransform.h"
     32#import "ImageRotationSessionVT.h"
    3133#import "Logging.h"
    3234#import "MediaSample.h"
     
    5052}
    5153
    52 static inline void computeRotatedWidthAndHeight(CVPixelBufferRef pixelBuffer, webrtc::VideoRotation rotation, size_t& width, size_t& height)
     54static inline unsigned rotationToAngle(webrtc::VideoRotation rotation)
    5355{
    5456    switch (rotation) {
    5557    case webrtc::kVideoRotation_0:
     58        return 0;
     59    case webrtc::kVideoRotation_90:
     60        return 90;
    5661    case webrtc::kVideoRotation_180:
    57         width = CVPixelBufferGetWidth(pixelBuffer);
    58         height = CVPixelBufferGetHeight(pixelBuffer);
    59         return;
    60     case webrtc::kVideoRotation_90:
     62        return 180;
    6163    case webrtc::kVideoRotation_270:
    62         width = CVPixelBufferGetHeight(pixelBuffer);
    63         height = CVPixelBufferGetWidth(pixelBuffer);
    64         return;
     64        return 270;
    6565    }
    6666}
     
    7373
    7474    if (!m_rotationSession || rotation != m_currentRotationSessionAngle) {
    75         VTImageRotationSessionRef rawRotationSession = nullptr;
    76         auto status = VTImageRotationSessionCreate(kCFAllocatorDefault, rotation, &rawRotationSession);
    77         if (status != noErr) {
    78             ERROR_LOG(LOGIDENTIFIER, "Failed creating a rotation session with error ", status);
    79             return nullptr;
    80         }
    81 
    82         m_rotationSession = adoptCF(rawRotationSession);
    83         m_currentRotationSessionAngle = rotation;
    84 
    85         VTImageRotationSessionSetProperty(rawRotationSession, kVTImageRotationPropertyKey_EnableHighSpeedTransfer, kCFBooleanTrue);
     75        IntSize size = { (int)CVPixelBufferGetWidth(pixelBuffer) , (int)CVPixelBufferGetHeight(pixelBuffer) };
     76        AffineTransform transform;
     77        transform.rotate(rotationToAngle(rotation));
     78        m_rotationSession = std::make_unique<ImageRotationSessionVT>(WTFMove(transform), size, CVPixelBufferGetPixelFormatType(pixelBuffer), ImageRotationSessionVT::IsCGImageCompatible::No);
    8679    }
    8780
    88     size_t rotatedWidth, rotatedHeight;
    89     computeRotatedWidthAndHeight(pixelBuffer, rotation, rotatedWidth, rotatedHeight);
    90     auto format = CVPixelBufferGetPixelFormatType(pixelBuffer);
    91     if (!m_rotationPool || rotatedWidth != m_rotatedWidth || rotatedHeight != m_rotatedHeight || format != m_rotatedFormat) {
    92         auto pixelAttributes = @{
    93             (__bridge NSString *)kCVPixelBufferWidthKey: @(rotatedWidth),
    94             (__bridge NSString *)kCVPixelBufferHeightKey: @(rotatedHeight),
    95             (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(format),
    96             (__bridge NSString *)kCVPixelBufferCGImageCompatibilityKey: @NO,
    97         };
    98 
    99         CVPixelBufferPoolRef pool = nullptr;
    100         auto status = CVPixelBufferPoolCreate(kCFAllocatorDefault, nullptr, (__bridge CFDictionaryRef)pixelAttributes, &pool);
    101 
    102         if (status != kCVReturnSuccess) {
    103             ERROR_LOG(LOGIDENTIFIER, "Failed creating a pixel buffer pool with error ", status);
    104             return nullptr;
    105         }
    106         m_rotationPool = adoptCF(pool);
    107 
    108         m_rotatedWidth = rotatedWidth;
    109         m_rotatedHeight = rotatedHeight;
    110         m_rotatedFormat = format;
    111     }
    112 
    113     CVPixelBufferRef rawRotatedBuffer = nullptr;
    114     auto status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, m_rotationPool.get(), &rawRotatedBuffer);
    115 
    116     if (status != kCVReturnSuccess) {
    117         ERROR_LOG(LOGIDENTIFIER, "Failed creating a pixel buffer with error ", status);
    118         return nullptr;
    119     }
    120     RetainPtr<CVPixelBufferRef> rotatedBuffer = adoptCF(rawRotatedBuffer);
    121 
    122     status = VTImageRotationSessionTransferImage(m_rotationSession.get(), pixelBuffer, rotatedBuffer.get());
    123 
    124     if (status != noErr) {
    125         ERROR_LOG(LOGIDENTIFIER, "Failed rotating with error ", status, " for rotation ", m_currentRotation);
    126         return nullptr;
    127     }
    128     return rotatedBuffer;
     81    return m_rotationSession->rotate(pixelBuffer);
    12982}
    13083
Note: See TracChangeset for help on using the changeset viewer.