Changeset 173702 in webkit

Timestamp:

Sep 17, 2014 2:18:04 PM (10 years ago)

Author:

Simon Fraser

Message:

Fix an issue that caused video playback to not always use high resolution streams
​https://bugs.webkit.org/show_bug.cgi?id=136879
rdar://problem/18316542

Reviewed by Eric Carlson.

The underlying video framework expects to be able to use -[CALayer convertRect:toLayer:nil] on
the video layer to obtain information about the display resolution of the video,
including the device scale factor. This doesn't work for layers in the web process,
which are remotely hosted into the UI process.

For now, use a workaround which involves putting a scale transform equivalent to
the device scale factor on the web process layer, and the inverse scale on the
hosting layer in the UI process.

We have to do this in two places, for in-page video, and for fullscreen video.

• Shared/mac/RemoteLayerTreeTransaction.h: Add hostingDeviceScaleFactor to the LayerCreation

properties.

• Shared/mac/RemoteLayerTreeTransaction.mm:

(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::LayerCreationProperties):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::encode):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::decode):

• UIProcess/ios/RemoteLayerTreeHostIOS.mm:

(-[WKRemoteView initWithFrame:contextID:hostingDeviceScaleFactor:]): Set the inverse scale
transform on the layer.
(WebKit::RemoteLayerTreeHost::createLayer):
(-[WKRemoteView initWithFrame:contextID:]): Deleted.

• UIProcess/ios/WebVideoFullscreenManagerProxy.h:
• UIProcess/ios/WebVideoFullscreenManagerProxy.messages.in:
• UIProcess/ios/WebVideoFullscreenManagerProxy.mm:

(WebKit::WebVideoFullscreenManagerProxy::setupFullscreenWithID):

• WebProcess/WebPage/mac/PlatformCALayerRemoteCustom.mm:

(WebKit::PlatformCALayerRemoteCustom::PlatformCALayerRemoteCustom):

• WebProcess/WebPage/mac/RemoteLayerTreeContext.h:

(WebKit::RemoteLayerTreeContext::deviceScaleFactor):

• WebProcess/WebPage/mac/RemoteLayerTreeContext.mm:

(WebKit::RemoteLayerTreeContext::layerWasCreated):

• WebProcess/ios/WebVideoFullscreenManager.mm:

(WebKit::WebVideoFullscreenManager::enterFullscreenForNode):
(WebKit::WebVideoFullscreenManager::didSetupFullscreen):

Location:

trunk/Source/WebKit2

Files:

• ChangeLog (modified) (1 diff)
• Shared/mac/RemoteLayerTreeTransaction.h (modified) (1 diff)
• Shared/mac/RemoteLayerTreeTransaction.mm (modified) (3 diffs)
• UIProcess/ios/RemoteLayerTreeHostIOS.mm (modified) (2 diffs)
• UIProcess/ios/WebVideoFullscreenManagerProxy.h (modified) (1 diff)
• UIProcess/ios/WebVideoFullscreenManagerProxy.messages.in (modified) (1 diff)
• UIProcess/ios/WebVideoFullscreenManagerProxy.mm (modified) (1 diff)
• WebProcess/WebPage/mac/PlatformCALayerRemoteCustom.mm (modified) (1 diff)
• WebProcess/WebPage/mac/RemoteLayerTreeContext.h (modified) (1 diff)
• WebProcess/WebPage/mac/RemoteLayerTreeContext.mm (modified) (1 diff)
• WebProcess/ios/WebVideoFullscreenManager.mm (modified) (2 diffs)

trunk/Source/WebKit2/ChangeLog

@@ +1 @@
+2014-09-17  Simon Fraser  <simon.fraser@apple.com>
+
+        Fix an issue that caused video playback to not always use high resolution streams
+        https://bugs.webkit.org/show_bug.cgi?id=136879
+        rdar://problem/18316542
+
+        Reviewed by Eric Carlson.
+        
+        The underlying video framework expects to be able to use -[CALayer convertRect:toLayer:nil] on
+        the video layer to obtain information about the display resolution of the video,
+        including the device scale factor. This doesn't work for layers in the web process,
+        which are remotely hosted into the UI process.
+        
+        For now, use a workaround which involves putting a scale transform equivalent to
+        the device scale factor on the web process layer, and the inverse scale on the 
+        hosting layer in the UI process.
+        
+        We have to do this in two places, for in-page video, and for fullscreen video.
+
+        * Shared/mac/RemoteLayerTreeTransaction.h: Add hostingDeviceScaleFactor to the LayerCreation
+        properties.
+        * Shared/mac/RemoteLayerTreeTransaction.mm:
+        (WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::LayerCreationProperties):
+        (WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::encode):
+        (WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::decode):
+        * UIProcess/ios/RemoteLayerTreeHostIOS.mm:
+        (-[WKRemoteView initWithFrame:contextID:hostingDeviceScaleFactor:]): Set the inverse scale
+        transform on the layer.
+        (WebKit::RemoteLayerTreeHost::createLayer):
+        (-[WKRemoteView initWithFrame:contextID:]): Deleted.
+        * UIProcess/ios/WebVideoFullscreenManagerProxy.h:
+        * UIProcess/ios/WebVideoFullscreenManagerProxy.messages.in:
+        * UIProcess/ios/WebVideoFullscreenManagerProxy.mm:
+        (WebKit::WebVideoFullscreenManagerProxy::setupFullscreenWithID):
+        * WebProcess/WebPage/mac/PlatformCALayerRemoteCustom.mm:
+        (WebKit::PlatformCALayerRemoteCustom::PlatformCALayerRemoteCustom):
+        * WebProcess/WebPage/mac/RemoteLayerTreeContext.h:
+        (WebKit::RemoteLayerTreeContext::deviceScaleFactor):
+        * WebProcess/WebPage/mac/RemoteLayerTreeContext.mm:
+        (WebKit::RemoteLayerTreeContext::layerWasCreated):
+        * WebProcess/ios/WebVideoFullscreenManager.mm:
+        (WebKit::WebVideoFullscreenManager::enterFullscreenForNode):
+        (WebKit::WebVideoFullscreenManager::didSetupFullscreen):
+
 2014-09-17  Gavin Barraclough  <baraclough@apple.com>
 

trunk/Source/WebKit2/Shared/mac/RemoteLayerTreeTransaction.h

@@ -96 +96 @@
 
         uint32_t hostingContextID;
+        float hostingDeviceScaleFactor;
     };
 

trunk/Source/WebKit2/Shared/mac/RemoteLayerTreeTransaction.mm

@@ -48 +48 @@
     , type(PlatformCALayer::LayerTypeLayer)
     , hostingContextID(0)
+    , hostingDeviceScaleFactor(1)
 {
 }
@@ -56 +57 @@
     encoder.encodeEnum(type);
     encoder << hostingContextID;
+    encoder << hostingDeviceScaleFactor;
 }
 
@@ -67 +69 @@
 
     if (!decoder.decode(result.hostingContextID))
+        return false;
+
+    if (!decoder.decode(result.hostingDeviceScaleFactor))
         return false;
 

trunk/Source/WebKit2/UIProcess/ios/RemoteLayerTreeHostIOS.mm

@@ -109 +109 @@
 
 @implementation WKRemoteView
-- (instancetype)initWithFrame:(CGRect)frame contextID:(uint32_t)contextID
+- (instancetype)initWithFrame:(CGRect)frame contextID:(uint32_t)contextID hostingDeviceScaleFactor:(float)scaleFactor
 {
     if ((self = [super initWithFrame:frame])) {
         [[self layer] setContextId:contextID];
+        // Invert the scale transform added in the WebProcess to fix <rdar://problem/18316542>.
+        float inverseScale = 1 / scaleFactor;
+        [[self layer] setTransform:CATransform3DMakeScale(inverseScale, inverseScale, 1)];
     }
     
@@ -156 +159 @@
     case PlatformCALayer::LayerTypeWebGLLayer:
         if (!m_isDebugLayerTreeHost)
-            view = adoptNS([[WKRemoteView alloc] initWithFrame:CGRectZero contextID:properties.hostingContextID]);
+            view = adoptNS([[WKRemoteView alloc] initWithFrame:CGRectZero contextID:properties.hostingContextID hostingDeviceScaleFactor:properties.hostingDeviceScaleFactor]);
         else
             view = adoptNS([[WKCompositingView alloc] init]);

trunk/Source/WebKit2/UIProcess/ios/WebVideoFullscreenManagerProxy.h

@@ -56 +56 @@
 
     // Translate to FullscreenInterface
-    void setupFullscreenWithID(uint32_t, WebCore::IntRect initialRect);
+    void setupFullscreenWithID(uint32_t, WebCore::IntRect initialRect, float hostingDeviceScaleFactor);
     void setSeekableRangesVector(Vector<std::pair<double, double>>&);
     void setExternalPlaybackProperties(bool enabled, uint32_t targetType, String localizedDeviceName);

trunk/Source/WebKit2/UIProcess/ios/WebVideoFullscreenManagerProxy.messages.in

@@ -32 +32 @@
     SetDuration(double duration)
     SetRate(bool isPlaying, double rate)
-    SetupFullscreenWithID(uint32_t videoLayerID, WebCore::IntRect initialRect)
+    SetupFullscreenWithID(uint32_t videoLayerID, WebCore::IntRect initialRect, float hostingScaleFactor)
     EnterFullscreen()
     ExitFullscreen(WebCore::IntRect finalRect)

trunk/Source/WebKit2/UIProcess/ios/WebVideoFullscreenManagerProxy.mm

@@ -73 +73 @@
 }
 
-void WebVideoFullscreenManagerProxy::setupFullscreenWithID(uint32_t videoLayerID, WebCore::IntRect initialRect)
+void WebVideoFullscreenManagerProxy::setupFullscreenWithID(uint32_t videoLayerID, WebCore::IntRect initialRect, float hostingDeviceScaleFactor)
 {
     ASSERT(videoLayerID);
     m_layerHost = WKMakeRenderLayer(videoLayerID);
+    if (hostingDeviceScaleFactor != 1) {
+        // Invert the scale transform added in the WebProcess to fix <rdar://problem/18316542>.
+        float inverseScale = 1 / hostingDeviceScaleFactor;
+        [m_layerHost setTransform:CATransform3DMakeScale(inverseScale, inverseScale, 1)];
+    }
+
     UIView *parentView = toRemoteLayerTreeDrawingAreaProxy(m_page->drawingArea())->remoteLayerTreeHost().rootLayer();
     setupFullscreen(*m_layerHost.get(), initialRect, parentView);

trunk/Source/WebKit2/WebProcess/WebPage/mac/PlatformCALayerRemoteCustom.mm

@@ -65 +65 @@
     case LayerHostingMode::OutOfProcess:
         m_layerHostingContext = LayerHostingContext::createForExternalHostingProcess();
+#if PLATFORM(IOS)
+        float scaleFactor = context.deviceScaleFactor();
+        // Set a scale factor here to make convertRect:toLayer:nil take scale factor into account. <rdar://problem/18316542>.
+        // This scale factor is inverted in the hosting process.
+        [customLayer setTransform:CATransform3DMakeScale(scaleFactor, scaleFactor, 1)];
+#endif
         break;
 #endif

trunk/Source/WebKit2/WebProcess/WebPage/mac/RemoteLayerTreeContext.h

@@ -56 +56 @@
     WebCore::LayerPool& layerPool() { return m_layerPool; }
 
+    float deviceScaleFactor() const { return m_webPage.deviceScaleFactor(); }
+
     LayerHostingMode layerHostingMode() const { return m_webPage.layerHostingMode(); }
 

trunk/Source/WebKit2/WebProcess/WebPage/mac/RemoteLayerTreeContext.mm

@@ -62 +62 @@
     creationProperties.type = type;
 
-    if (layer.isPlatformCALayerRemoteCustom())
+    if (layer.isPlatformCALayerRemoteCustom()) {
         creationProperties.hostingContextID = layer.hostingContextID();
+        creationProperties.hostingDeviceScaleFactor = deviceScaleFactor();
+    }
 
     m_createdLayers.append(creationProperties);

trunk/Source/WebKit2/WebProcess/ios/WebVideoFullscreenManager.mm

@@ -98 +98 @@
     m_layerHostingContext = LayerHostingContext::createForExternalHostingProcess();
     
-    m_page->send(Messages::WebVideoFullscreenManagerProxy::SetupFullscreenWithID(m_layerHostingContext->contextID(), clientRectForElement(videoElement)), m_page->pageID());
+    m_page->send(Messages::WebVideoFullscreenManagerProxy::SetupFullscreenWithID(m_layerHostingContext->contextID(), clientRectForElement(videoElement), m_page->deviceScaleFactor()), m_page->pageID());
 }
 
@@ -176 +176 @@
     [CATransaction begin];
     [CATransaction setDisableActions:YES];
+
     [videoLayer setBackgroundColor:cachedCGColor(WebCore::Color::transparent, WebCore::ColorSpaceDeviceRGB)];
+
+    // Set a scale factor here to make convertRect:toLayer:nil take scale factor into account. <rdar://problem/18316542>.
+    // This scale factor is inverted in the hosting process.
+    float hostingScaleFactor = m_page->deviceScaleFactor();
+    [videoLayer setTransform:CATransform3DMakeScale(hostingScaleFactor, hostingScaleFactor, 1)];
     m_layerHostingContext->setRootLayer(videoLayer);
+
     setVideoFullscreenLayer(videoLayer);
     [CATransaction commit];
+
     m_page->send(Messages::WebVideoFullscreenManagerProxy::EnterFullscreen(), m_page->pageID());
 }