⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 267491 in webkit


Ignore:
Timestamp:
Sep 23, 2020, 11:45:11 AM (6 years ago)
Author:
Peng Liu
Message:

[Media in GPU Process] Implement caption support in video fullscreen and PiP
​https://bugs.webkit.org/show_bug.cgi?id=208752

Reviewed by Eric Carlson.

Source/WebCore:

Export TextTrackRepresentation so that WebKit can use it. Also, update the
interface of VideoLayerManagerObjC related to text track representation to
use PlatformLayer* instead of TextTrackRepresentation*.

Manually tested.

  • WebCore.xcodeproj/project.pbxproj:

Change target membership of TextTrackRepresentation.h.

  • platform/graphics/avfoundation/objc/VideoLayerManagerObjC.h:
  • platform/graphics/avfoundation/objc/VideoLayerManagerObjC.mm:

(WebCore::VideoLayerManagerObjC::setTextTrackRepresentationLayer):
(WebCore::VideoLayerManagerObjC::setTextTrackRepresentation): Deleted.
Replace setTextTrackRepresentation() with setTextTrackRepresentationLayer().

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::setTextTrackRepresentation):
Use VideoLayerManagerObjC::setTextTrackRepresentationLayer() instead of
VideoLayerManagerObjC::setTextTrackRepresentation().

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:

(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setTextTrackRepresentation): Ditto.

Source/WebKit:

  • GPUProcess/media/RemoteMediaPlayerProxy.cpp:

(WebKit::RemoteMediaPlayerProxy::syncTextTrackBounds): Deleted.

  • GPUProcess/media/RemoteMediaPlayerProxy.h:
  • GPUProcess/media/RemoteMediaPlayerProxy.messages.in:

Remove syncTextTrackBounds() because the GPU process does not need to deal with
the text track bounds.

  • WebProcess/GPU/GPUProcessConnection.cpp:

(WebKit::GPUProcessConnection::dispatchMessage):
Add a missing IPC message receiver.

  • WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:

(WebKit::MediaPlayerPrivateRemote::setVideoFullscreenLayer): Set the flag
m_requiresTextTrackRepresentation when changing video presentation mode.
(WebKit::MediaPlayerPrivateRemote::requiresTextTrackRepresentation const): Deleted.
(WebKit::MediaPlayerPrivateRemote::setTextTrackRepresentation): Deleted.
(WebKit::MediaPlayerPrivateRemote::syncTextTrackBounds): Deleted.
Move the implementation of those functions to MediaPlayerPrivateRemoteCocoa.

  • WebProcess/GPU/media/MediaPlayerPrivateRemote.h:

Add the flag m_requiresTextTrackRepresentation and add a reference
to the layer used to render text track (in fullscreen or picture-in-picture).

  • WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:

(WebKit::MediaPlayerPrivateRemote::setVideoFullscreenFrame):
(WebKit::MediaPlayerPrivateRemote::setTextTrackRepresentation):
(WebKit::MediaPlayerPrivateRemote::syncTextTrackBounds):
Those functions share a similar implementation as VideoLayerManagerObjC.

Location:
trunk/Source
Files:
14 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r267486 r267491  
     12020-09-23  Peng Liu  <peng.liu6@apple.com>
     2
     3        [Media in GPU Process] Implement caption support in video fullscreen and PiP
     4        https://bugs.webkit.org/show_bug.cgi?id=208752
     5
     6        Reviewed by Eric Carlson.
     7
     8        Export TextTrackRepresentation so that WebKit can use it. Also, update the
     9        interface of VideoLayerManagerObjC related to text track representation to
     10        use PlatformLayer* instead of TextTrackRepresentation*.
     11
     12        Manually tested.
     13
     14        * WebCore.xcodeproj/project.pbxproj:
     15        Change target membership of TextTrackRepresentation.h.
     16
     17        * platform/graphics/avfoundation/objc/VideoLayerManagerObjC.h:
     18        * platform/graphics/avfoundation/objc/VideoLayerManagerObjC.mm:
     19        (WebCore::VideoLayerManagerObjC::setTextTrackRepresentationLayer):
     20        (WebCore::VideoLayerManagerObjC::setTextTrackRepresentation): Deleted.
     21        Replace setTextTrackRepresentation() with setTextTrackRepresentationLayer().
     22
     23        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     24        (WebCore::MediaPlayerPrivateAVFoundationObjC::setTextTrackRepresentation):
     25        Use VideoLayerManagerObjC::setTextTrackRepresentationLayer() instead of
     26        VideoLayerManagerObjC::setTextTrackRepresentation().
     27
     28        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
     29        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setTextTrackRepresentation): Ditto.
     30
    1312020-09-23  Chris Dumez  <cdumez@apple.com>
    232

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r267472 r267491  
    635635                1D9F0FC12122029B005D8FD4 /* ShareData.h in Headers */ = {isa = PBXBuildFile; fileRef = 1DC55400211BA8C8004B780E /* ShareData.h */; settings = {ATTRIBUTES = (Private, ); }; };
    636636                1DBC1B562347B3D200B901AF /* PictureInPictureObserver.h in Headers */ = {isa = PBXBuildFile; fileRef = 1DBC1B552347B3D200B901AF /* PictureInPictureObserver.h */; settings = {ATTRIBUTES = (Private, ); }; };
     637                1DF7E81F251A9E0600DB8F61 /* TextTrackRepresentation.h in Headers */ = {isa = PBXBuildFile; fileRef = CDD1E525167BA56400CE820B /* TextTrackRepresentation.h */; settings = {ATTRIBUTES = (Private, ); }; };
    637638                1F36EA9C1E21BA1700621E25 /* WebBackgroundTaskController.h in Headers */ = {isa = PBXBuildFile; fileRef = 1F36EA9A1E21BA1700621E25 /* WebBackgroundTaskController.h */; settings = {ATTRIBUTES = (Private, ); }; };
    638639                1F72BF0B187FD45C0009BCB3 /* TileControllerMemoryHandlerIOS.h in Headers */ = {isa = PBXBuildFile; fileRef = 1F72BF09187FD4270009BCB3 /* TileControllerMemoryHandlerIOS.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    3422934230                                B1AD4E7413A12A4600846B27 /* TextTrackLoader.h in Headers */,
    3423034231                                CD8B5A49180E138B008B8E65 /* TextTrackMediaSource.h in Headers */,
     34232                                1DF7E81F251A9E0600DB8F61 /* TextTrackRepresentation.h in Headers */,
    3423134233                                1C73A7132185757E004CCEA5 /* TextUnderlineOffset.h in Headers */,
    3423234234                                29FAF4B6195AB08900A522DC /* TextUndoInsertionMarkupMac.h in Headers */,

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r267472 r267491  
    21132113void MediaPlayerPrivateAVFoundationObjC::setTextTrackRepresentation(TextTrackRepresentation* representation)
    21142114{
    2115     m_videoLayerManager->setTextTrackRepresentation(representation);
     2115    auto* representationLayer = representation ? representation->platformLayer() : nil;
     2116    m_videoLayerManager->setTextTrackRepresentationLayer(representationLayer);
    21162117}
    21172118

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm

    r267472 r267491  
    12411241void MediaPlayerPrivateMediaSourceAVFObjC::setTextTrackRepresentation(TextTrackRepresentation* representation)
    12421242{
    1243     m_videoLayerManager->setTextTrackRepresentation(representation);
     1243    auto* representationLayer = representation ? representation->platformLayer() : nil;
     1244    m_videoLayerManager->setTextTrackRepresentationLayer(representationLayer);
    12441245}
    12451246

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/VideoLayerManagerObjC.h

    r260412 r267491  
    4848    PlatformLayer *videoInlineLayer() const { return m_videoInlineLayer.get(); }
    4949    void setVideoLayer(PlatformLayer *, IntSize contentSize);
     50    void didDestroyVideoLayer();
    5051
    5152#if ENABLE(VIDEO_PRESENTATION_MODE)
    … …  
    5758#endif
    5859
    59     void didDestroyVideoLayer();
    60 
    6160    bool requiresTextTrackRepresentation() const;
    62     void setTextTrackRepresentation(TextTrackRepresentation*);
     61    void setTextTrackRepresentationLayer(PlatformLayer*);
    6362    void syncTextTrackBounds();
    6463

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/VideoLayerManagerObjC.mm

    r263323 r267491  
    183183}
    184184
    185 void VideoLayerManagerObjC::setTextTrackRepresentation(TextTrackRepresentation* representation)
     185void VideoLayerManagerObjC::setTextTrackRepresentationLayer(PlatformLayer* representationLayer)
    186186{
    187187#if !ENABLE(VIDEO_PRESENTATION_MODE)
    188     UNUSED_PARAM(representation);
     188    UNUSED_PARAM(representationLayer);
    189189#else
    190190    ALWAYS_LOG(LOGIDENTIFIER);
    191191
    192     PlatformLayer* representationLayer = representation ? representation->platformLayer() : nil;
    193192    if (representationLayer == m_textTrackRepresentationLayer) {
    194193        syncTextTrackBounds();

  • trunk/Source/WebKit/ChangeLog

    r267490 r267491  
     12020-09-23  Peng Liu  <peng.liu6@apple.com>
     2
     3        [Media in GPU Process] Implement caption support in video fullscreen and PiP
     4        https://bugs.webkit.org/show_bug.cgi?id=208752
     5
     6        Reviewed by Eric Carlson.
     7
     8        * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     9        (WebKit::RemoteMediaPlayerProxy::syncTextTrackBounds): Deleted.
     10        * GPUProcess/media/RemoteMediaPlayerProxy.h:
     11        * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
     12        Remove syncTextTrackBounds() because the GPU process does not need to deal with
     13        the text track bounds.
     14
     15        * WebProcess/GPU/GPUProcessConnection.cpp:
     16        (WebKit::GPUProcessConnection::dispatchMessage):
     17        Add a missing IPC message receiver.
     18
     19        * WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
     20        (WebKit::MediaPlayerPrivateRemote::setVideoFullscreenLayer): Set the flag
     21        m_requiresTextTrackRepresentation when changing video presentation mode.
     22        (WebKit::MediaPlayerPrivateRemote::requiresTextTrackRepresentation const): Deleted.
     23        (WebKit::MediaPlayerPrivateRemote::setTextTrackRepresentation): Deleted.
     24        (WebKit::MediaPlayerPrivateRemote::syncTextTrackBounds): Deleted.
     25        Move the implementation of those functions to MediaPlayerPrivateRemoteCocoa.
     26
     27        * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
     28        Add the flag m_requiresTextTrackRepresentation and add a reference
     29        to the layer used to render text track (in fullscreen or picture-in-picture).
     30
     31        * WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
     32        (WebKit::MediaPlayerPrivateRemote::setVideoFullscreenFrame):
     33        (WebKit::MediaPlayerPrivateRemote::setTextTrackRepresentation):
     34        (WebKit::MediaPlayerPrivateRemote::syncTextTrackBounds):
     35        Those functions share a similar implementation as VideoLayerManagerObjC.
     36
    1372020-09-23  Per Arne Vollan  <pvollan@apple.com>
    238

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp

    r264021 r267491  
    837837}
    838838
    839 void RemoteMediaPlayerProxy::syncTextTrackBounds()
    840 {
    841     m_player->syncTextTrackBounds();
    842 }
    843 
    844839void RemoteMediaPlayerProxy::performTaskAtMediaTime(const MediaTime& taskTime, WallTime messageTime, CompletionHandler<void(Optional<MediaTime>)>&& completionHandler)
    845840{

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h

    r264021 r267491  
    176176    void notifyTrackModeChanged();
    177177    void tracksChanged();
    178     void syncTextTrackBounds();
    179178
    180179    void performTaskAtMediaTime(const MediaTime&, WallTime, CompletionHandler<void(Optional<MediaTime>)>&&);

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in

    r260412 r267491  
    113113    NotifyTrackModeChanged()
    114114    TracksChanged()
    115     SyncTextTrackBounds()
    116115
    117116#if ENABLE(WIRELESS_PLAYBACK_TARGET)

  • trunk/Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp

    r267427 r267491  
    5454#include "RemoteAudioSession.h"
    5555#include "RemoteAudioSessionMessages.h"
     56#endif
     57
     58#if PLATFORM(IOS_FAMILY)
     59#include "RemoteMediaSessionHelper.h"
     60#include "RemoteMediaSessionHelperMessages.h"
    5661#endif
    5762
    … …  
    142147    }
    143148#endif
     149#if PLATFORM(IOS_FAMILY)
     150    if (decoder.messageReceiverName() == Messages::RemoteMediaSessionHelper::messageReceiverName()) {
     151        // FIXME
     152        return true;
     153    }
     154#endif
    144155#if ENABLE(ENCRYPTED_MEDIA)
    145156    if (decoder.messageReceiverName() == Messages::RemoteCDMInstanceSession::messageReceiverName()) {

  • trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp

    r264297 r267491  
    652652{
    653653    if (!videoFullscreenLayer) {
    654         connection().sendWithAsyncReply(Messages::RemoteMediaPlayerProxy::ExitFullscreen(), WTFMove(completionHandler), m_id);
     654        connection().sendWithAsyncReply(Messages::RemoteMediaPlayerProxy::ExitFullscreen(), [this, weakThis = makeWeakPtr(*this), completionHandler = WTFMove(completionHandler)]() mutable {
     655            if (!weakThis)
     656                return;
     657
     658            m_requiresTextTrackRepresentation = false;
     659            completionHandler();
     660        }, m_id);
    655661        return;
    656662    }
    657663
    658664    ASSERT(m_videoFullscreenLayer.get() == videoFullscreenLayer);
    659     connection().sendWithAsyncReply(Messages::RemoteMediaPlayerProxy::EnterFullscreen(), WTFMove(completionHandler), m_id);
     665    connection().sendWithAsyncReply(Messages::RemoteMediaPlayerProxy::EnterFullscreen(), [this, weakThis = makeWeakPtr(*this), completionHandler = WTFMove(completionHandler)]() mutable {
     666        if (!weakThis)
     667            return;
     668
     669        m_requiresTextTrackRepresentation = true;
     670        completionHandler();
     671    }, m_id);
    660672}
    661673
    … …  
    10101022#endif
    10111023
    1012 bool MediaPlayerPrivateRemote::requiresTextTrackRepresentation() const
    1013 {
    1014     notImplemented();
    1015     return false;
    1016 }
    1017 
    1018 void MediaPlayerPrivateRemote::setTextTrackRepresentation(TextTrackRepresentation*)
    1019 {
    1020     notImplemented();
    1021 }
    1022 
    1023 void MediaPlayerPrivateRemote::syncTextTrackBounds()
    1024 {
    1025     connection().send(Messages::RemoteMediaPlayerProxy::SyncTextTrackBounds(), m_id);
    1026 }
    1027 
    10281024void MediaPlayerPrivateRemote::tracksChanged()
    10291025{

  • trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h

    r264021 r267491  
    4949class ISOWebVTTCue;
    5050class SerializedPlatformDataCueValue;
     51class TextTrackRepresentation;
    5152}
    5253
    … …  
    324325#endif
    325326
    326     bool requiresTextTrackRepresentation() const final;
     327    bool requiresTextTrackRepresentation() const final { return m_requiresTextTrackRepresentation; }
     328#if PLATFORM(COCOA)
    327329    void setTextTrackRepresentation(WebCore::TextTrackRepresentation*) final;
    328330    void syncTextTrackBounds() final;
     331#endif
    329332    void tracksChanged() final;
    330333
    … …  
    357360    WebCore::MediaPlayer* m_player { nullptr };
    358361    RefPtr<WebCore::PlatformMediaResourceLoader> m_mediaResourceLoader;
     362    bool m_requiresTextTrackRepresentation { false };
    359363    PlatformLayerContainer m_videoInlineLayer;
    360364    PlatformLayerContainer m_videoFullscreenLayer;
     365#if PLATFORM(COCOA)
     366    RetainPtr<PlatformLayer> m_textTrackRepresentationLayer;
     367#endif
    361368    Optional<LayerHostingContextID> m_fullscreenLayerHostingContextId;
    362369    RemoteMediaPlayerManager& m_manager;

  • trunk/Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm

    r260366 r267491  
    3030
    3131#import <WebCore/FloatRect.h>
     32#import <WebCore/TextTrackRepresentation.h>
    3233#import <pal/spi/cocoa/QuartzCoreSPI.h>
    3334#import <wtf/MachSendRight.h>
    … …  
    6667
    6768    [CATransaction commit];
     69
     70    syncTextTrackBounds();
    6871}
    6972
    7073#endif
    7174
     75void MediaPlayerPrivateRemote::setTextTrackRepresentation(WebCore::TextTrackRepresentation* representation)
     76{
     77#if !ENABLE(VIDEO_PRESENTATION_MODE)
     78    UNUSED_PARAM(representation);
     79#else
     80    PlatformLayer* representationLayer = representation ? representation->platformLayer() : nil;
     81
     82    if (representationLayer == m_textTrackRepresentationLayer) {
     83        syncTextTrackBounds();
     84        return;
     85    }
     86
     87    [CATransaction begin];
     88    [CATransaction setDisableActions:YES];
     89
     90    if (m_textTrackRepresentationLayer)
     91        [m_textTrackRepresentationLayer removeFromSuperlayer];
     92
     93    m_textTrackRepresentationLayer = representationLayer;
     94
     95    if (m_videoFullscreenLayer && m_textTrackRepresentationLayer) {
     96        syncTextTrackBounds();
     97        [m_videoFullscreenLayer addSublayer:m_textTrackRepresentationLayer.get()];
     98    }
     99
     100    [CATransaction commit];
     101#endif
     102}
     103
     104void MediaPlayerPrivateRemote::syncTextTrackBounds()
     105{
     106#if ENABLE(VIDEO_PRESENTATION_MODE)
     107    if (!m_videoFullscreenLayer || !m_textTrackRepresentationLayer)
     108        return;
     109
     110    [CATransaction begin];
     111    [CATransaction setDisableActions:YES];
     112
     113    [m_textTrackRepresentationLayer setFrame:m_videoFullscreenLayer.get().bounds];
     114
     115    [CATransaction commit];
     116#endif
     117}
     118
    72119} // namespace WebKit
    73120
Note: See TracChangeset for help on using the changeset viewer.