Changeset 257125 in webkit


Ignore:
Timestamp:
Feb 20, 2020 7:35:50 PM (4 years ago)
Author:
eric.carlson@apple.com
Message:

Support in-band metadata cues when loading media in the GPU Process
​https://bugs.webkit.org/show_bug.cgi?id=207904
<rdar://problem/59561647>

Reviewed by Dean Jackson.

Source/WebCore:

No new tests, unskipped existing tests.

  • WebCore.xcodeproj/project.pbxproj:
  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::selectMediaResource):
(WebCore::HTMLMediaElement::play):
(WebCore::HTMLMediaElement::mediaEngineWasUpdated):
(WebCore::HTMLMediaElement::mediaPlayerEngineUpdated):

  • html/track/InbandDataTextTrack.cpp:

(WebCore::InbandDataTextTrack::addDataCue):
(WebCore::InbandDataTextTrack::findIncompleteCue):
(WebCore::InbandDataTextTrack::updateDataCue):
(WebCore::InbandDataTextTrack::removeDataCue):
(WebCore::InbandDataTextTrack::removeCue):

  • html/track/InbandDataTextTrack.h:
  • html/track/TextTrack.cpp:

(WebCore::TextTrack::addCue):

  • html/track/TextTrack.h:

(WebCore::TextTrack::cuesInternal const):

  • html/track/TextTrackCue.cpp:

(WebCore::TextTrackCue::cueIndex const):

  • platform/SerializedPlatformDataCue.h:

(WebCore::SerializedPlatformDataCue::deserialize const):
(WebCore::SerializedPlatformDataCue::data const):
(WebCore::SerializedPlatformDataCue::isEqual const):
(WebCore::SerializedPlatformDataCue::platformType const):
(WebCore::SerializedPlatformDataCue::encodingRequiresPlatformData const):
(WebCore::SerializedPlatformDataCue::encodableValue const):

  • platform/graphics/MediaPlayer.cpp:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::metadataDidArrive):

  • platform/mac/SerializedPlatformDataCueMac.h:
  • platform/mac/SerializedPlatformDataCueMac.mm:

(WebCore::SerializedPlatformDataCue::create):
(WebCore::SerializedPlatformDataCueMac::SerializedPlatformDataCueMac):
(WebCore::SerializedPlatformDataCueMac::~SerializedPlatformDataCueMac):
(WebCore::SerializedPlatformDataCueMac::isEqual const):
(WebCore::toSerializedPlatformDataCueMac):
(WebCore::SerializedPlatformDataCueMac::allowedClassesForNativeValues):
(WebCore::SerializedPlatformDataCueMac::encodableValue const):
(WebCore::jsValueWithAVMetadataItemInContext):
(WebCore::NSDictionaryWithAVMetadataItem):
(WebCore::SerializedPlatformDataCueMac::create): Deleted.

Source/WebKit:

  • GPUProcess/media/RemoteMediaPlayerManagerProxy.messages.in:
  • GPUProcess/media/RemoteTextTrackProxy.cpp:

(WebKit::RemoteTextTrackProxy::addDataCue):
(WebKit::RemoteTextTrackProxy::updateDataCue):
(WebKit::RemoteTextTrackProxy::removeDataCue):

  • Shared/WebCoreArgumentCoders.cpp:

(IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::encode):
(IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::decode):

  • Shared/WebCoreArgumentCoders.h:
  • Shared/mac/WebCoreArgumentCodersMac.mm:

(IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::encodePlatformData):
(IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::decodePlatformData):

  • WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:

(WebKit::MediaPlayerPrivateRemote::MediaPlayerPrivateRemote):
(WebKit::MediaPlayerPrivateRemote::setConfiguration):
(WebKit::MediaPlayerPrivateRemote::addDataCue):
(WebKit::MediaPlayerPrivateRemote::addDataCueWithType):
(WebKit::MediaPlayerPrivateRemote::updateDataCue):
(WebKit::MediaPlayerPrivateRemote::removeDataCue):

  • WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
  • WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in:
  • WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:

(WebKit::RemoteMediaPlayerManager::createRemoteMediaPlayer):

  • WebProcess/GPU/media/RemoteMediaPlayerManager.h:
  • WebProcess/GPU/media/TextTrackPrivateRemote.cpp:

(WebKit::TextTrackPrivateRemote::addGenericCue):
(WebKit::TextTrackPrivateRemote::updateGenericCue):
(WebKit::TextTrackPrivateRemote::removeGenericCue):
(WebKit::TextTrackPrivateRemote::parseWebVTTFileHeader):
(WebKit::TextTrackPrivateRemote::parseWebVTTCueData):
(WebKit::TextTrackPrivateRemote::parseWebVTTCueDataStruct):
(WebKit::TextTrackPrivateRemote::addDataCue):
(WebKit::TextTrackPrivateRemote::addDataCueWithType):
(WebKit::TextTrackPrivateRemote::updateDataCue):
(WebKit::TextTrackPrivateRemote::removeDataCue):

  • WebProcess/GPU/media/TextTrackPrivateRemote.h:

LayoutTests:

  • gpu-process/TestExpectations:
Location:
trunk
Files:
32 edited
2 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r257101 r257125  
     12020-02-20  Eric Carlson  <eric.carlson@apple.com>
     2
     3        Support in-band metadata cues when loading media in the GPU Process
     4        https://bugs.webkit.org/show_bug.cgi?id=207904
     5        <rdar://problem/59561647>
     6
     7        Reviewed by Dean Jackson.
     8
     9        * gpu-process/TestExpectations:
     10
    1112020-02-20  Jason Lawrence  <lawrence.j@apple.com>
    212

  • trunk/LayoutTests/gpu-process/TestExpectations

    r256353 r257125  
    230230http/tests/media/track/track-webvtt-slow-loading-2.html [ Pass ]
    231231http/tests/media/hls/hls-webvtt-tracks.html [ Pass ]
     232http/tests/media/track-in-band-hls-metadata.html [ Pass ]
     233http/tests/media/track-in-band-hls-metadata-crash.html [ Pass ]
    232234
    233235accessibility/media-element.html [ Skip ]

  • trunk/Source/WebCore/ChangeLog

    r257123 r257125  
     12020-02-20  Eric Carlson  <eric.carlson@apple.com>
     2
     3        Support in-band metadata cues when loading media in the GPU Process
     4        https://bugs.webkit.org/show_bug.cgi?id=207904
     5        <rdar://problem/59561647>
     6
     7        Reviewed by Dean Jackson.
     8
     9        No new tests, unskipped existing tests.
     10
     11        * WebCore.xcodeproj/project.pbxproj:
     12        * html/HTMLMediaElement.cpp:
     13        (WebCore::HTMLMediaElement::selectMediaResource):
     14        (WebCore::HTMLMediaElement::play):
     15        (WebCore::HTMLMediaElement::mediaEngineWasUpdated):
     16        (WebCore::HTMLMediaElement::mediaPlayerEngineUpdated):
     17        * html/track/InbandDataTextTrack.cpp:
     18        (WebCore::InbandDataTextTrack::addDataCue):
     19        (WebCore::InbandDataTextTrack::findIncompleteCue):
     20        (WebCore::InbandDataTextTrack::updateDataCue):
     21        (WebCore::InbandDataTextTrack::removeDataCue):
     22        (WebCore::InbandDataTextTrack::removeCue):
     23        * html/track/InbandDataTextTrack.h:
     24        * html/track/TextTrack.cpp:
     25        (WebCore::TextTrack::addCue):
     26        * html/track/TextTrack.h:
     27        (WebCore::TextTrack::cuesInternal const):
     28        * html/track/TextTrackCue.cpp:
     29        (WebCore::TextTrackCue::cueIndex const):
     30        * platform/SerializedPlatformDataCue.h:
     31        (WebCore::SerializedPlatformDataCue::deserialize const):
     32        (WebCore::SerializedPlatformDataCue::data const):
     33        (WebCore::SerializedPlatformDataCue::isEqual const):
     34        (WebCore::SerializedPlatformDataCue::platformType const):
     35        (WebCore::SerializedPlatformDataCue::encodingRequiresPlatformData const):
     36        (WebCore::SerializedPlatformDataCue::encodableValue const):
     37        * platform/graphics/MediaPlayer.cpp:
     38        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     39        (WebCore::MediaPlayerPrivateAVFoundationObjC::metadataDidArrive):
     40        * platform/mac/SerializedPlatformDataCueMac.h:
     41        * platform/mac/SerializedPlatformDataCueMac.mm:
     42        (WebCore::SerializedPlatformDataCue::create):
     43        (WebCore::SerializedPlatformDataCueMac::SerializedPlatformDataCueMac):
     44        (WebCore::SerializedPlatformDataCueMac::~SerializedPlatformDataCueMac):
     45        (WebCore::SerializedPlatformDataCueMac::isEqual const):
     46        (WebCore::toSerializedPlatformDataCueMac):
     47        (WebCore::SerializedPlatformDataCueMac::allowedClassesForNativeValues):
     48        (WebCore::SerializedPlatformDataCueMac::encodableValue const):
     49        (WebCore::jsValueWithAVMetadataItemInContext):
     50        (WebCore::NSDictionaryWithAVMetadataItem):
     51        (WebCore::SerializedPlatformDataCueMac::create): Deleted.
     52
    1532020-02-20  Sunny He  <sunny_he@apple.com>
    254

  • trunk/Source/WebCore/Headers.cmake

    r256892 r257125  
    999999    platform/SearchPopupMenu.h
    10001000    platform/SerializedPlatformDataCue.h
     1001    platform/SerializedPlatformDataCueValue.h
    10011002    platform/SharedBuffer.h
    10021003    platform/SharedStringHash.h

  • trunk/Source/WebCore/Sources.txt

    r256892 r257125  
    17651765platform/ScrollbarTheme.cpp
    17661766platform/ScrollbarThemeComposite.cpp
     1767platform/SerializedPlatformDataCue.cpp
    17671768platform/SharedBuffer.cpp
    17681769platform/SharedBufferChunkReader.cpp

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r256892 r257125  
    136136                076970861463AD8700F502CF /* TextTrackList.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 076970841463AD8700F502CF /* TextTrackList.cpp */; };
    137137                076970871463AD8700F502CF /* TextTrackList.h in Headers */ = {isa = PBXBuildFile; fileRef = 076970851463AD8700F502CF /* TextTrackList.h */; };
     138                076D1C1F23F6EB5B00D95B06 /* SerializedPlatformDataCueValue.h in Headers */ = {isa = PBXBuildFile; fileRef = 076D1C1D23F6D9D800D95B06 /* SerializedPlatformDataCueValue.h */; settings = {ATTRIBUTES = (Private, ); }; };
    138139                076F0D0E12B8192700C26AA4 /* MediaPlayerPrivateAVFoundation.h in Headers */ = {isa = PBXBuildFile; fileRef = 076F0D0A12B8192700C26AA4 /* MediaPlayerPrivateAVFoundation.h */; };
    139140                077664FD183E6B5C00133B92 /* JSQuickTimePluginReplacement.h in Headers */ = {isa = PBXBuildFile; fileRef = 077664FB183E6B5C00133B92 /* JSQuickTimePluginReplacement.h */; };
    140141                077AF14018F4AE400001ED61 /* SerializedPlatformDataCue.h in Headers */ = {isa = PBXBuildFile; fileRef = 077AF13E18F4AE400001ED61 /* SerializedPlatformDataCue.h */; settings = {ATTRIBUTES = (Private, ); }; };
    141                 077AF14318F4B1BB0001ED61 /* SerializedPlatformDataCueMac.h in Headers */ = {isa = PBXBuildFile; fileRef = 077AF14118F4B1BB0001ED61 /* SerializedPlatformDataCueMac.h */; };
     142                077AF14318F4B1BB0001ED61 /* SerializedPlatformDataCueMac.h in Headers */ = {isa = PBXBuildFile; fileRef = 077AF14118F4B1BB0001ED61 /* SerializedPlatformDataCueMac.h */; settings = {ATTRIBUTES = (Private, ); }; };
    142143                077B64131B94F12E003E9AD5 /* MediaPlaybackTargetPickerMock.h in Headers */ = {isa = PBXBuildFile; fileRef = 077B64111B94F12E003E9AD5 /* MediaPlaybackTargetPickerMock.h */; settings = {ATTRIBUTES = (Private, ); }; };
    143144                077B64171B95F703003E9AD5 /* MediaPlaybackTargetMock.h in Headers */ = {isa = PBXBuildFile; fileRef = 077B64151B95F703003E9AD5 /* MediaPlaybackTargetMock.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    54745475                076970841463AD8700F502CF /* TextTrackList.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = TextTrackList.cpp; sourceTree = "<group>"; };
    54755476                076970851463AD8700F502CF /* TextTrackList.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = TextTrackList.h; sourceTree = "<group>"; };
     5477                076D1C1D23F6D9D800D95B06 /* SerializedPlatformDataCueValue.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = SerializedPlatformDataCueValue.h; sourceTree = "<group>"; };
    54765478                076E11BE1F683E0D00177395 /* TrackPrivateBase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = TrackPrivateBase.cpp; sourceTree = "<group>"; };
    54775479                076F0D0912B8192700C26AA4 /* MediaPlayerPrivateAVFoundation.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MediaPlayerPrivateAVFoundation.cpp; sourceTree = "<group>"; };
    … …  
    54915493                07846341145B151A00A58DF1 /* JSTrackEvent.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSTrackEvent.h; sourceTree = "<group>"; };
    54925494                07846384145B1B8E00A58DF1 /* JSTrackCustom.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSTrackCustom.h; sourceTree = "<group>"; };
     5495                078633D723FE09640087AE21 /* SerializedPlatformDataCue.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = SerializedPlatformDataCue.cpp; sourceTree = "<group>"; };
    54935496                0787C4671BFBDF6F006DCD7F /* JSMediaTrackSupportedConstraints.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSMediaTrackSupportedConstraints.cpp; sourceTree = "<group>"; };
    54945497                0787C4681BFBDF6F006DCD7F /* JSMediaTrackSupportedConstraints.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSMediaTrackSupportedConstraints.h; sourceTree = "<group>"; };
    … …  
    2599525998                                BC6D6E2509AF943500F59759 /* ScrollView.h */,
    2599625999                                AB7170880B3118080017123E /* SearchPopupMenu.h */,
     26000                                078633D723FE09640087AE21 /* SerializedPlatformDataCue.cpp */,
    2599726001                                077AF13E18F4AE400001ED61 /* SerializedPlatformDataCue.h */,
     26002                                076D1C1D23F6D9D800D95B06 /* SerializedPlatformDataCueValue.h */,
    2599826003                                1A4A954B0B4EDCCB002D8C3C /* SharedBuffer.cpp */,
    2599926004                                1A4A954C0B4EDCCB002D8C3C /* SharedBuffer.h */,
    … …  
    3225532260                                077AF14018F4AE400001ED61 /* SerializedPlatformDataCue.h in Headers */,
    3225632261                                077AF14318F4B1BB0001ED61 /* SerializedPlatformDataCueMac.h in Headers */,
     32262                                076D1C1F23F6EB5B00D95B06 /* SerializedPlatformDataCueValue.h in Headers */,
    3225732263                                A75E497610752ACB00C9B896 /* SerializedScriptValue.h in Headers */,
    3225832264                                516F7F6D1C31E39A00F111DC /* ServerOpenDBRequest.h in Headers */,

  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r257039 r257125  
    13331333            ASSERT(m_player);
    13341334            if (!m_player) {
    1335                 ERROR_LOG(logSiteIdentifier, " has srcAttr but m_player is not created");
     1335                ERROR_LOG(logSiteIdentifier, "has srcAttr but m_player is not created");
    13361336                return;
    13371337            }
    … …  
    34743474        if (success.value() == MediaPlaybackDenialReason::UserGestureRequired)
    34753475            setAutoplayEventPlaybackState(AutoplayEventPlaybackState::PreventedAutoplay);
     3476        ERROR_LOG(LOGIDENTIFIER, "rejecting promise: ", success.value());
    34763477        promise.reject(NotAllowedError);
    34773478        return;
    … …  
    34793480
    34803481    if (m_error && m_error->code() == MediaError::MEDIA_ERR_SRC_NOT_SUPPORTED) {
     3482        ERROR_LOG(LOGIDENTIFIER, "rejecting promise because of error");
    34813483        promise.reject(NotSupportedError, "The operation is not supported.");
    34823484        return;
    … …  
    34963498    auto success = m_mediaSession->playbackPermitted();
    34973499    if (!success) {
     3500        ERROR_LOG(LOGIDENTIFIER, "playback not permitted: ", success.value());
    34983501        if (success.value() == MediaPlaybackDenialReason::UserGestureRequired)
    34993502            setAutoplayEventPlaybackState(AutoplayEventPlaybackState::PreventedAutoplay);
    … …  
    50955098void HTMLMediaElement::mediaEngineWasUpdated()
    50965099{
    5097     INFO_LOG(LOGIDENTIFIER);
     5100    ALWAYS_LOG(LOGIDENTIFIER);
     5101
    50985102    beginProcessingMediaPlayerCallback();
    50995103    updateRenderer();
    … …  
    51305134void HTMLMediaElement::mediaPlayerEngineUpdated()
    51315135{
    5132     INFO_LOG(LOGIDENTIFIER);
     5136    INFO_LOG(LOGIDENTIFIER, m_player->engineDescription());
    51335137
    51345138#if ENABLE(MEDIA_SOURCE)

  • trunk/Source/WebCore/html/track/InbandDataTextTrack.cpp

    r256536 r257125  
    6060void InbandDataTextTrack::addDataCue(const MediaTime& start, const MediaTime& end, Ref<SerializedPlatformDataCue>&& platformValue, const String& type)
    6161{
    62     if (m_incompleteCueMap.contains(platformValue.ptr()))
     62    if (findIncompleteCue(platformValue))
    6363        return;
    6464
    … …  
    7171    if (end.isPositiveInfinite() && mediaElement()) {
    7272        cue->setEndTime(mediaElement()->durationMediaTime());
    73         m_incompleteCueMap.add(WTFMove(platformValue), cue.copyRef());
     73        m_incompleteCueMap.append(&cue.get());
    7474    }
    7575
    … …  
    7979}
    8080
     81RefPtr<DataCue> InbandDataTextTrack::findIncompleteCue(const SerializedPlatformDataCue& cueToFind)
     82{
     83    auto index = m_incompleteCueMap.findMatching([&](const auto& cue) {
     84        return cueToFind.isEqual(*cue->platformValue());
     85    });
     86
     87    if (index == notFound)
     88        return nullptr;
     89
     90    return m_incompleteCueMap[index];
     91}
     92
    8193void InbandDataTextTrack::updateDataCue(const MediaTime& start, const MediaTime& inEnd, SerializedPlatformDataCue& platformValue)
    8294{
    83     RefPtr<DataCue> cue = m_incompleteCueMap.get(&platformValue);
     95    auto cue = findIncompleteCue(platformValue);
    8496    if (!cue)
    8597        return;
    … …  
    91103        end = mediaElement()->durationMediaTime();
    92104    else
    93         m_incompleteCueMap.remove(&platformValue);
     105        m_incompleteCueMap.removeFirst(cue);
    94106
    95107    INFO_LOG(LOGIDENTIFIER, "was start = ", cue->startMediaTime(), ", end = ", cue->endMediaTime(), ", will be start = ", start, ", end = ", end);
    … …  
    103115void InbandDataTextTrack::removeDataCue(const MediaTime&, const MediaTime&, SerializedPlatformDataCue& platformValue)
    104116{
    105     if (auto cue = m_incompleteCueMap.take(&platformValue)) {
     117    if (auto cue = findIncompleteCue(platformValue)) {
    106118        INFO_LOG(LOGIDENTIFIER, "removing: ", *cue);
     119        m_incompleteCueMap.removeFirst(cue);
    107120        InbandTextTrack::removeCue(*cue);
    108121    }
    … …  
    113126    ASSERT(cue.cueType() == TextTrackCue::Data);
    114127
    115     m_incompleteCueMap.remove(const_cast<SerializedPlatformDataCue*>(toDataCue(&cue)->platformValue()));
     128    if (auto platformValue = const_cast<SerializedPlatformDataCue*>(toDataCue(&cue)->platformValue()))
     129        removeDataCue({ }, { }, *platformValue);
    116130
    117131    return InbandTextTrack::removeCue(cue);

  • trunk/Source/WebCore/html/track/InbandDataTextTrack.h

    r256536 r257125  
    5656    ExceptionOr<void> removeCue(TextTrackCue&) final;
    5757
    58     HashMap<RefPtr<SerializedPlatformDataCue>, RefPtr<DataCue>> m_incompleteCueMap;
     58    RefPtr<DataCue> findIncompleteCue(const SerializedPlatformDataCue&);
     59
     60    Vector<RefPtr<DataCue>> m_incompleteCueMap;
    5961#endif
    6062};

  • trunk/Source/WebCore/html/track/TextTrack.h

    r256353 r257125  
    9797    TextTrackCueList* activeCues() const;
    9898
     99    TextTrackCueList* cuesInternal() const { return m_cues.get(); }
     100
    99101    void clearClient() override { m_client = nullptr; }
    100102    TextTrackClient* client() { return m_client; }

  • trunk/Source/WebCore/html/track/TextTrackCue.cpp

    r254767 r257125  
    342342unsigned TextTrackCue::cueIndex() const
    343343{
    344     ASSERT(m_track && m_track->cues());
    345     if (!m_track || !m_track->cues())
     344    ASSERT(m_track && m_track->cuesInternal());
     345    if (!m_track || !m_track->cuesInternal())
    346346        return std::numeric_limits<unsigned>::max();
    347347
    348     return m_track->cues()->cueIndex(*this);
     348    return m_track->cuesInternal()->cueIndex(*this);
    349349}
    350350

  • trunk/Source/WebCore/platform/SerializedPlatformDataCue.cpp

    r257124 r257125  
    11/*
    2  * Copyright (C) 2017-2020 Apple Inc. All rights reserved.
     2 * Copyright (C) 2020 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2525
    2626#include "config.h"
    27 #include "TrackPrivateBase.h"
     27#include "SerializedPlatformDataCue.h"
    2828
    29 #if ENABLE(VIDEO_TRACK)
    30 
    31 #include "Logging.h"
    32 #include <wtf/LoggerHelper.h>
     29#if ENABLE(VIDEO_TRACK) && ENABLE(DATACUE_VALUE)
    3330
    3431namespace WebCore {
    3532
    36 static uint64_t s_uniqueId = 0;
    37 
    38 #if !RELEASE_LOG_DISABLED
    39 void TrackPrivateBase::setLogger(const Logger& logger, const void* logIdentifier)
     33#if !PLATFORM(COCOA)
     34Ref<SerializedPlatformDataCue> SerializedPlatformDataCue::create(SerializedPlatformDataCueValue&&)
    4035{
    41     m_logger = &logger;
    42     m_logIdentifier = childLogIdentifier(logIdentifier, ++s_uniqueId);
    43 }
    44 
    45 WTFLogChannel& TrackPrivateBase::logChannel() const
    46 {
    47     return LogMedia;
     36    return adoptRef(*new SerializedPlatformDataCue());
    4837}
    4938#endif

  • trunk/Source/WebCore/platform/SerializedPlatformDataCue.h

    r256536 r257125  
    2828#if ENABLE(VIDEO)
    2929
     30#include "SerializedPlatformDataCueValue.h"
    3031#include <JavaScriptCore/JSCInlines.h>
    3132#include <wtf/RefCounted.h>
    … …  
    3536class SerializedPlatformDataCue : public RefCounted<SerializedPlatformDataCue> {
    3637public:
     38    WEBCORE_EXPORT static Ref<SerializedPlatformDataCue> create(SerializedPlatformDataCueValue&&);
     39
    3740    virtual ~SerializedPlatformDataCue() = default;
    3841
    39     virtual JSC::JSValue deserialize(JSC::JSGlobalObject*) const = 0;
    40     virtual RefPtr<JSC::ArrayBuffer> data() const = 0;
    41     virtual bool isEqual(const SerializedPlatformDataCue&) const = 0;
     42    virtual JSC::JSValue deserialize(JSC::JSGlobalObject*) const { return JSC::jsNull(); }
     43    virtual RefPtr<JSC::ArrayBuffer> data() const { return { }; }
     44    virtual bool isEqual(const SerializedPlatformDataCue&) const { return false; }
    4245
    43     enum PlatformType {
     46    enum class PlatformType {
     47        None,
    4448        ObjC,
    4549    };
    46     virtual PlatformType platformType() const = 0;
     50    virtual PlatformType platformType() const { return PlatformType::None; }
     51
     52    virtual bool encodingRequiresPlatformData() const { return false; }
     53
     54    virtual SerializedPlatformDataCueValue encodableValue() const { return { }; }
    4755
    4856protected:

  • trunk/Source/WebCore/platform/SerializedPlatformDataCueValue.h

    r257124 r257125  
    11/*
    2  * Copyright (C) 2014-2020 Apple Inc. All rights reserved.
     2 * Copyright (C) 2020 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2828#if ENABLE(VIDEO)
    2929
    30 #include <JavaScriptCore/JSCInlines.h>
    31 #include <wtf/RefCounted.h>
     30#include <wtf/IsoMalloc.h>
     31#include <wtf/RetainPtr.h>
     32
     33#if !defined(__OBJC__)
     34typedef struct objc_object *id;
     35#endif
     36
     37#if PLATFORM(COCOA)
     38using PlatformNativeValue = id;
     39#else
     40using PlatformNativeValue = void*;
     41#endif
    3242
    3343namespace WebCore {
    3444
    35 class SerializedPlatformDataCue : public RefCounted<SerializedPlatformDataCue> {
     45class SerializedPlatformDataCueValue {
     46    WTF_MAKE_ISO_ALLOCATED(SerializedPlatformDataCueValue);
    3647public:
    37     virtual ~SerializedPlatformDataCue() = default;
    38 
    39     virtual JSC::JSValue deserialize(JSC::JSGlobalObject*) const = 0;
    40     virtual RefPtr<JSC::ArrayBuffer> data() const = 0;
    41     virtual bool isEqual(const SerializedPlatformDataCue&) const = 0;
    42 
    43     enum PlatformType {
     48    enum class PlatformType {
     49        None,
    4450        ObjC,
    4551    };
    46     virtual PlatformType platformType() const = 0;
     52
     53    SerializedPlatformDataCueValue(PlatformType platformType, PlatformNativeValue nativeValue)
     54        : m_nativeValue(nativeValue)
     55        , m_type(platformType)
     56    {
     57    }
     58    SerializedPlatformDataCueValue() = default;
     59    ~SerializedPlatformDataCueValue() = default;
     60
     61    PlatformType platformType() const { return m_type; }
     62
     63    PlatformNativeValue nativeValue() const { return m_nativeValue; }
     64
     65    bool encodingRequiresPlatformData() const { return m_type == PlatformType::ObjC; }
    4766
    4867protected:
    49     SerializedPlatformDataCue() = default;
     68    PlatformNativeValue m_nativeValue { nullptr };
     69    PlatformType m_type { PlatformType::None };
    5070};
    5171

  • trunk/Source/WebCore/platform/graphics/MediaPlayer.cpp

    r254964 r257125  
    110110    void pause() final { }
    111111
     112    String engineDescription() const final { return "NullMediaPlayer"; }
     113
    112114    PlatformLayer* platformLayer() const final { return nullptr; }
    113115

  • trunk/Source/WebCore/platform/graphics/TrackPrivateBase.cpp

    r256353 r257125  
    3434namespace WebCore {
    3535
     36#if !RELEASE_LOG_DISABLED
     37
    3638static uint64_t s_uniqueId = 0;
    3739
    38 #if !RELEASE_LOG_DISABLED
    3940void TrackPrivateBase::setLogger(const Logger& logger, const void* logIdentifier)
    4041{

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r256839 r257125  
    30753075            type = metadataType(item.keySpace);
    30763076
    3077         m_metadataTrack->addDataCue(start, end, SerializedPlatformDataCueMac::create(item), type);
     3077        m_metadataTrack->addDataCue(start, end, SerializedPlatformDataCue::create({ WebCore::SerializedPlatformDataCueValue::PlatformType::ObjC, item }), type);
    30783078    }
    30793079#endif

  • trunk/Source/WebCore/platform/mac/SerializedPlatformDataCueMac.h

    r256536 r257125  
    3030#include "SerializedPlatformDataCue.h"
    3131
    32 #if USE(FOUNDATION) && !defined(__OBJC__)
    33 typedef struct objc_object *id;
    34 #endif
    35 
    3632namespace WebCore {
    3733
    3834class SerializedPlatformDataCueMac final : public SerializedPlatformDataCue {
    3935public:
    40     virtual ~SerializedPlatformDataCueMac();
    41     static Ref<SerializedPlatformDataCue> create(id);
     36    SerializedPlatformDataCueMac(SerializedPlatformDataCueValue&&);
     37    virtual ~SerializedPlatformDataCueMac() = default;
    4238
    4339    JSC::JSValue deserialize(JSC::JSGlobalObject*) const final;
    4440    RefPtr<ArrayBuffer> data() const final;
     41    bool isEqual(const SerializedPlatformDataCue&) const final;
     42    PlatformType platformType() const final { return PlatformType::ObjC; }
     43    bool encodingRequiresPlatformData() const final { return true; }
    4544
    46     bool isEqual(const SerializedPlatformDataCue&) const final;
    47 
    48     PlatformType platformType() const final { return SerializedPlatformDataCue::ObjC; }
     45    WEBCORE_EXPORT SerializedPlatformDataCueValue encodableValue() const final;
    4946
    5047    id nativeValue() const { return m_nativeValue.get(); }
    5148
     49    WEBCORE_EXPORT static NSArray *allowedClassesForNativeValues();
     50
    5251private:
    53     explicit SerializedPlatformDataCueMac(id nativeValue);
    5452
    5553    RetainPtr<id> m_nativeValue;

  • trunk/Source/WebCore/platform/mac/SerializedPlatformDataCueMac.mm

    r256536 r257125  
    4949static JSValue *jsValueWithAVMetadataItemInContext(AVMetadataItem *, JSContext *);
    5050static JSValue *jsValueWithValueInContext(id, JSContext *);
    51 #endif
    52 
    53 SerializedPlatformDataCueMac::SerializedPlatformDataCueMac(id nativeValue)
     51static NSDictionary *NSDictionaryWithAVMetadataItem(AVMetadataItem *);
     52#endif
     53
     54Ref<SerializedPlatformDataCue> SerializedPlatformDataCue::create(SerializedPlatformDataCueValue&& value)
     55{
     56    return adoptRef(*new SerializedPlatformDataCueMac(WTFMove(value)));
     57}
     58
     59SerializedPlatformDataCueMac::SerializedPlatformDataCueMac(SerializedPlatformDataCueValue&& value)
    5460    : SerializedPlatformDataCue()
    55     , m_nativeValue(nativeValue)
    56 {
    57 }
    58 
    59 SerializedPlatformDataCueMac::~SerializedPlatformDataCueMac()
    60 {
    61 }
    62 
    63 Ref<SerializedPlatformDataCue> SerializedPlatformDataCueMac::create(id nativeValue)
    64 {
    65     return adoptRef(*new SerializedPlatformDataCueMac(nativeValue));
     61    , m_nativeValue(value.nativeValue())
     62{
     63    RELEASE_ASSERT_WITH_SECURITY_IMPLICATION(value.platformType() == SerializedPlatformDataCueValue::PlatformType::ObjC);
    6664}
    6765
    … …  
    9088bool SerializedPlatformDataCueMac::isEqual(const SerializedPlatformDataCue& other) const
    9189{
    92     if (other.platformType() != SerializedPlatformDataCue::ObjC)
     90    if (other.platformType() != PlatformType::ObjC)
    9391        return false;
    9492
    … …  
    108106const SerializedPlatformDataCueMac* toSerializedPlatformDataCueMac(const SerializedPlatformDataCue* rep)
    109107{
    110     RELEASE_ASSERT_WITH_SECURITY_IMPLICATION(rep->platformType() == SerializedPlatformDataCue::ObjC);
     108    RELEASE_ASSERT_WITH_SECURITY_IMPLICATION(rep->platformType() == SerializedPlatformDataCue::PlatformType::ObjC);
    111109    return static_cast<const SerializedPlatformDataCueMac*>(rep);
     110}
     111
     112NSArray *SerializedPlatformDataCueMac::allowedClassesForNativeValues()
     113{
     114    static NeverDestroyed<RetainPtr<NSArray>> allowedClasses(@[ [NSString class], [NSNumber class], [NSLocale class], [NSDictionary class], [NSArray class], [NSData class] ]);
     115    return allowedClasses.get().get();
     116}
     117
     118SerializedPlatformDataCueValue SerializedPlatformDataCueMac::encodableValue() const
     119{
     120    if ([m_nativeValue.get() isKindOfClass:PAL::getAVMetadataItemClass()])
     121        return { SerializedPlatformDataCueValue::PlatformType::ObjC, NSDictionaryWithAVMetadataItem(m_nativeValue.get()) };
     122
     123    return { SerializedPlatformDataCueValue::PlatformType::ObjC, m_nativeValue.get() };
    112124}
    113125
    … …  
    194206
    195207static JSValue *jsValueWithAVMetadataItemInContext(AVMetadataItem *item, JSContext *context)
     208{
     209    return jsValueWithDictionaryInContext(NSDictionaryWithAVMetadataItem(item), context);
     210}
     211
     212static NSDictionary *NSDictionaryWithAVMetadataItem(AVMetadataItem *item)
    196213{
    197214    NSMutableDictionary *dictionary = [NSMutableDictionary dictionary];
    … …  
    231248        [dictionary setObject:item.value forKey:@"data"];
    232249
    233     return jsValueWithDictionaryInContext(dictionary, context);
     250    return dictionary;
    234251}
    235252#endif

  • trunk/Source/WebKit/ChangeLog

    r257106 r257125  
     12020-02-20  Eric Carlson  <eric.carlson@apple.com>
     2
     3        Support in-band metadata cues when loading media in the GPU Process
     4        https://bugs.webkit.org/show_bug.cgi?id=207904
     5        <rdar://problem/59561647>
     6
     7        Reviewed by Dean Jackson.
     8
     9        * GPUProcess/media/RemoteMediaPlayerManagerProxy.messages.in:
     10        * GPUProcess/media/RemoteTextTrackProxy.cpp:
     11        (WebKit::RemoteTextTrackProxy::addDataCue):
     12        (WebKit::RemoteTextTrackProxy::updateDataCue):
     13        (WebKit::RemoteTextTrackProxy::removeDataCue):
     14        * Shared/WebCoreArgumentCoders.cpp:
     15        (IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::encode):
     16        (IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::decode):
     17        * Shared/WebCoreArgumentCoders.h:
     18        * Shared/mac/WebCoreArgumentCodersMac.mm:
     19        (IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::encodePlatformData):
     20        (IPC::ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::decodePlatformData):
     21        * WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
     22        (WebKit::MediaPlayerPrivateRemote::MediaPlayerPrivateRemote):
     23        (WebKit::MediaPlayerPrivateRemote::setConfiguration):
     24        (WebKit::MediaPlayerPrivateRemote::addDataCue):
     25        (WebKit::MediaPlayerPrivateRemote::addDataCueWithType):
     26        (WebKit::MediaPlayerPrivateRemote::updateDataCue):
     27        (WebKit::MediaPlayerPrivateRemote::removeDataCue):
     28        * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
     29        * WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in:
     30        * WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
     31        (WebKit::RemoteMediaPlayerManager::createRemoteMediaPlayer):
     32        * WebProcess/GPU/media/RemoteMediaPlayerManager.h:
     33        * WebProcess/GPU/media/TextTrackPrivateRemote.cpp:
     34        (WebKit::TextTrackPrivateRemote::addGenericCue):
     35        (WebKit::TextTrackPrivateRemote::updateGenericCue):
     36        (WebKit::TextTrackPrivateRemote::removeGenericCue):
     37        (WebKit::TextTrackPrivateRemote::parseWebVTTFileHeader):
     38        (WebKit::TextTrackPrivateRemote::parseWebVTTCueData):
     39        (WebKit::TextTrackPrivateRemote::parseWebVTTCueDataStruct):
     40        (WebKit::TextTrackPrivateRemote::addDataCue):
     41        (WebKit::TextTrackPrivateRemote::addDataCueWithType):
     42        (WebKit::TextTrackPrivateRemote::updateDataCue):
     43        (WebKit::TextTrackPrivateRemote::removeDataCue):
     44        * WebProcess/GPU/media/TextTrackPrivateRemote.h:
     45
    1462020-02-20  John Wilander  <wilander@apple.com>
    247

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.messages.in

    r254784 r257125  
    2525
    2626messages -> RemoteMediaPlayerManagerProxy NotRefCounted {
    27     CreateMediaPlayer(WebKit::MediaPlayerPrivateRemoteIdentifier id, enum:uint8_t WebCore::MediaPlayerEnums::MediaEngineIdentifier remoteEngineIdentifier, struct WebKit::RemoteMediaPlayerProxyConfiguration proxyConfiguration) -> (struct WebKit::RemoteMediaPlayerConfiguration playerConfiguration) Synchronous
    28     DeleteMediaPlayer(WebKit::MediaPlayerPrivateRemoteIdentifier id)
     27    CreateMediaPlayer(WebKit::MediaPlayerPrivateRemoteIdentifier id, enum:uint8_t WebCore::MediaPlayerEnums::MediaEngineIdentifier remoteEngineIdentifier, struct WebKit::RemoteMediaPlayerProxyConfiguration proxyConfiguration) -> (struct WebKit::RemoteMediaPlayerConfiguration playerConfiguration) Async
     28     DeleteMediaPlayer(WebKit::MediaPlayerPrivateRemoteIdentifier id)
    2929
    3030    GetSupportedTypes(enum:uint8_t WebCore::MediaPlayerEnums::MediaEngineIdentifier remoteEngineIdentifier) -> (Vector<String> types) Synchronous

  • trunk/Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.cpp

    r256536 r257125  
    9797}
    9898
    99 void RemoteTextTrackProxy::addDataCue(const MediaTime&, const MediaTime&, const void*, unsigned)
     99void RemoteTextTrackProxy::addDataCue(const MediaTime& start, const MediaTime& end, const void* data, unsigned length)
    100100{
    101     notImplemented();
     101    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::AddDataCue(m_identifier, start, end, IPC::DataReference(reinterpret_cast<const uint8_t*>(data), length)), m_player.idendifier());
    102102}
    103103
    104104#if ENABLE(DATACUE_VALUE)
    105 void RemoteTextTrackProxy::addDataCue(const MediaTime&, const MediaTime&, Ref<SerializedPlatformDataCue>&&, const String&)
     105void RemoteTextTrackProxy::addDataCue(const MediaTime& start, const MediaTime& end, Ref<SerializedPlatformDataCue>&& cueData, const String& type)
    106106{
    107     notImplemented();
     107    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::AddDataCueWithType(m_identifier, start, end, cueData->encodableValue(), type), m_player.idendifier());
    108108}
    109109
    110 void RemoteTextTrackProxy::updateDataCue(const MediaTime&, const MediaTime&, SerializedPlatformDataCue&)
     110void RemoteTextTrackProxy::updateDataCue(const MediaTime& start, const MediaTime& end, SerializedPlatformDataCue& cueData)
    111111{
    112     notImplemented();
     112    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::UpdateDataCue(m_identifier, start, end, cueData.encodableValue()), m_player.idendifier());
    113113}
    114114
    115 void RemoteTextTrackProxy::removeDataCue(const MediaTime&, const MediaTime&, SerializedPlatformDataCue&)
     115void RemoteTextTrackProxy::removeDataCue(const MediaTime& start, const MediaTime& end, SerializedPlatformDataCue& cueData)
    116116{
    117     notImplemented();
     117    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::RemoveDataCue(m_identifier, start, end, cueData.encodableValue()), m_player.idendifier());
    118118}
    119119#endif

  • trunk/Source/WebKit/Shared/WebCoreArgumentCoders.cpp

    r255785 r257125  
    7676#include <WebCore/SecurityOrigin.h>
    7777#include <WebCore/SerializedAttachmentData.h>
     78#include <WebCore/SerializedPlatformDataCueValue.h>
    7879#include <WebCore/ServiceWorkerClientData.h>
    7980#include <WebCore/ServiceWorkerClientIdentifier.h>
    … …  
    31753176#endif // ENABLE(ATTACHMENT_ELEMENT)
    31763177
     3178#if ENABLE(VIDEO)
     3179void ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::encode(Encoder& encoder, const SerializedPlatformDataCueValue& value)
     3180{
     3181    bool hasPlatformData = value.encodingRequiresPlatformData();
     3182    encoder << hasPlatformData;
     3183
     3184    encoder.encodeEnum(value.platformType());
     3185    if (hasPlatformData)
     3186        encodePlatformData(encoder, value);
     3187}
     3188
     3189Optional<SerializedPlatformDataCueValue> ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::decode(IPC::Decoder& decoder)
     3190{
     3191    bool hasPlatformData;
     3192    if (!decoder.decode(hasPlatformData))
     3193        return WTF::nullopt;
     3194
     3195    WebCore::SerializedPlatformDataCueValue::PlatformType type;
     3196    if (!decoder.decodeEnum(type))
     3197        return WTF::nullopt;
     3198
     3199    if (hasPlatformData)
     3200        return decodePlatformData(decoder, type);
     3201
     3202    return {{ }};
     3203}
     3204#endif
     3205
    31773206} // namespace IPC

  • trunk/Source/WebKit/Shared/WebCoreArgumentCoders.h

    r254817 r257125  
    3838#include <WebCore/RealtimeMediaSource.h>
    3939#include <WebCore/ScrollSnapOffsetsInfo.h>
     40#include <WebCore/SerializedPlatformDataCueValue.h>
    4041#include <WebCore/ServiceWorkerTypes.h>
    4142#include <WebCore/StoredCredentialsPolicy.h>
    … …  
    820821#endif // ENABLE(ATTACHMENT_ELEMENT)
    821822
     823#if ENABLE(VIDEO)
     824template<> struct ArgumentCoder<WebCore::SerializedPlatformDataCueValue> {
     825    static void encode(Encoder&, const WebCore::SerializedPlatformDataCueValue&);
     826    static Optional<WebCore::SerializedPlatformDataCueValue> decode(Decoder&);
     827    static void encodePlatformData(Encoder&, const WebCore::SerializedPlatformDataCueValue&);
     828    static Optional<WebCore::SerializedPlatformDataCueValue> decodePlatformData(Decoder&, WebCore::SerializedPlatformDataCueValue::PlatformType);
     829};
     830#endif
     831
    822832} // namespace IPC
    823833

  • trunk/Source/WebKit/Shared/curl/WebCoreArgumentCodersCurl.cpp

    r254202 r257125  
    247247}
    248248
    249 }
     249#if ENABLE(VIDEO)
     250void ArgumentCoder<SerializedPlatformDataCueValue>::encodePlatformData(Encoder& encoder, const SerializedPlatformDataCueValue& value)
     251{
     252    ASSERT_NOT_REACHED();
     253}
     254
     255Optional<SerializedPlatformDataCueValue>  ArgumentCoder<SerializedPlatformDataCueValue>::decodePlatformData(Decoder& decoder, WebCore::SerializedPlatformDataCueValue::PlatformType platformType)
     256{
     257    ASSERT_NOT_REACHED();
     258    return WTF::nullopt;
     259}
     260#endif
     261
     262}

  • trunk/Source/WebKit/Shared/mac/WebCoreArgumentCodersMac.mm

    r255784 r257125  
    3838#import <WebCore/ResourceError.h>
    3939#import <WebCore/ResourceRequest.h>
     40#import <WebCore/SerializedPlatformDataCueMac.h>
    4041#import <pal/spi/cf/CFNetworkSPI.h>
    4142#import <wtf/MachSendRight.h>
    … …  
    670671#endif
    671672
     673#if ENABLE(VIDEO)
     674void ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::encodePlatformData(Encoder& encoder, const WebCore::SerializedPlatformDataCueValue& value)
     675{
     676    ASSERT(value.platformType() == WebCore::SerializedPlatformDataCueValue::PlatformType::ObjC);
     677    if (value.platformType() == WebCore::SerializedPlatformDataCueValue::PlatformType::ObjC)
     678        encodeObject(encoder, value.nativeValue());
     679}
     680
     681Optional<WebCore::SerializedPlatformDataCueValue>  ArgumentCoder<WebCore::SerializedPlatformDataCueValue>::decodePlatformData(Decoder& decoder, WebCore::SerializedPlatformDataCueValue::PlatformType platformType)
     682{
     683    ASSERT(platformType == WebCore::SerializedPlatformDataCueValue::PlatformType::ObjC);
     684
     685    if (platformType != WebCore::SerializedPlatformDataCueValue::PlatformType::ObjC)
     686        return WTF::nullopt;
     687
     688    auto object = decodeObject(decoder, WebCore::SerializedPlatformDataCueMac::allowedClassesForNativeValues());
     689    if (!object)
     690        return WTF::nullopt;
     691
     692    return WebCore::SerializedPlatformDataCueValue { platformType, object.value().get() };
     693}
     694#endif
     695
    672696} // namespace IPC

  • trunk/Source/WebKit/Shared/soup/WebCoreArgumentCodersSoup.cpp

    r254202 r257125  
    264264}
    265265
    266 }
    267 
     266#if ENABLE(VIDEO)
     267void ArgumentCoder<SerializedPlatformDataCueValue>::encodePlatformData(Encoder& encoder, const SerializedPlatformDataCueValue& value)
     268{
     269    ASSERT_NOT_REACHED();
     270}
     271
     272Optional<SerializedPlatformDataCueValue>  ArgumentCoder<SerializedPlatformDataCueValue>::decodePlatformData(Decoder& decoder, WebCore::SerializedPlatformDataCueValue::PlatformType platformType)
     273{
     274    ASSERT_NOT_REACHED();
     275    return WTF::nullopt;
     276}
     277#endif
     278
     279}
     280

  • trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp

    r256353 r257125  
    6868#endif
    6969
    70 MediaPlayerPrivateRemote::MediaPlayerPrivateRemote(MediaPlayer* player, MediaPlayerEnums::MediaEngineIdentifier engineIdentifier, MediaPlayerPrivateRemoteIdentifier playerIdentifier, RemoteMediaPlayerManager& manager, const RemoteMediaPlayerConfiguration& configuration)
     70MediaPlayerPrivateRemote::MediaPlayerPrivateRemote(MediaPlayer* player, MediaPlayerEnums::MediaEngineIdentifier engineIdentifier, MediaPlayerPrivateRemoteIdentifier playerIdentifier, RemoteMediaPlayerManager& manager)
    7171    : m_player(player)
    7272    , m_mediaResourceLoader(player->createResourceLoader())
    … …  
    7474    , m_remoteEngineIdentifier(engineIdentifier)
    7575    , m_id(playerIdentifier)
    76     , m_configuration(configuration)
    7776#if !RELEASE_LOG_DISABLED
    7877    , m_logger(&player->mediaPlayerLogger())
    … …  
    8180{
    8281    INFO_LOG(LOGIDENTIFIER);
     82}
     83
     84void MediaPlayerPrivateRemote::setConfiguration(RemoteMediaPlayerConfiguration&& configuration)
     85{
     86    m_configuration = WTFMove(configuration);
     87    m_player->mediaEngineUpdated();
    8388}
    8489
    … …  
    485490}
    486491
     492void MediaPlayerPrivateRemote::addDataCue(TrackPrivateRemoteIdentifier identifier, MediaTime&& start, MediaTime&& end, IPC::DataReference&& data)
     493{
     494    ASSERT(m_textTracks.contains(identifier));
     495
     496    if (const auto& track = m_textTracks.get(identifier))
     497        track->addDataCue(WTFMove(start), WTFMove(end), WTFMove(data));
     498}
     499
     500#if ENABLE(DATACUE_VALUE)
     501void MediaPlayerPrivateRemote::addDataCueWithType(TrackPrivateRemoteIdentifier identifier, MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&& data, String&& type)
     502{
     503    ASSERT(m_textTracks.contains(identifier));
     504
     505    if (const auto& track = m_textTracks.get(identifier))
     506        track->addDataCueWithType(WTFMove(start), WTFMove(end), WTFMove(data), WTFMove(type));
     507}
     508
     509void MediaPlayerPrivateRemote::updateDataCue(TrackPrivateRemoteIdentifier identifier, MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&& data)
     510{
     511    ASSERT(m_textTracks.contains(identifier));
     512
     513    if (const auto& track = m_textTracks.get(identifier))
     514        track->updateDataCue(WTFMove(start), WTFMove(end), WTFMove(data));
     515}
     516
     517void MediaPlayerPrivateRemote::removeDataCue(TrackPrivateRemoteIdentifier identifier, MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&& data)
     518{
     519    ASSERT(m_textTracks.contains(identifier));
     520
     521    if (const auto& track = m_textTracks.get(identifier))
     522        track->removeDataCue(WTFMove(start), WTFMove(end), WTFMove(data));
     523}
     524#endif
     525
    487526void MediaPlayerPrivateRemote::addRemoteVideoTrack(TrackPrivateRemoteIdentifier identifier, TrackPrivateRemoteConfiguration&& configuration)
    488527{

  • trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h

    r256353 r257125  
    4141namespace WebCore {
    4242class ISOWebVTTCue;
     43class SerializedPlatformDataCueValue;
    4344}
    4445
    … …  
    4748class AudioTrackPrivateRemote;
    4849class TextTrackPrivateRemote;
     50class UserData;
    4951class VideoTrackPrivateRemote;
    5052struct TextTrackPrivateRemoteConfiguration;
    … …  
    6062{
    6163public:
    62     static std::unique_ptr<MediaPlayerPrivateRemote> create(WebCore::MediaPlayer* player, WebCore::MediaPlayerEnums::MediaEngineIdentifier remoteEngineIdentifier, MediaPlayerPrivateRemoteIdentifier identifier, RemoteMediaPlayerManager& manager, const RemoteMediaPlayerConfiguration& configuration)
     64    static std::unique_ptr<MediaPlayerPrivateRemote> create(WebCore::MediaPlayer* player, WebCore::MediaPlayerEnums::MediaEngineIdentifier remoteEngineIdentifier, MediaPlayerPrivateRemoteIdentifier identifier, RemoteMediaPlayerManager& manager)
    6365    {
    64         return makeUnique<MediaPlayerPrivateRemote>(player, remoteEngineIdentifier, identifier, manager, configuration);
     66        return makeUnique<MediaPlayerPrivateRemote>(player, remoteEngineIdentifier, identifier, manager);
    6567    }
    6668
    67     MediaPlayerPrivateRemote(WebCore::MediaPlayer*, WebCore::MediaPlayerEnums::MediaEngineIdentifier, MediaPlayerPrivateRemoteIdentifier, RemoteMediaPlayerManager&, const RemoteMediaPlayerConfiguration&);
     69    MediaPlayerPrivateRemote(WebCore::MediaPlayer*, WebCore::MediaPlayerEnums::MediaEngineIdentifier, MediaPlayerPrivateRemoteIdentifier, RemoteMediaPlayerManager&);
    6870    ~MediaPlayerPrivateRemote();
     71
     72    void setConfiguration(RemoteMediaPlayerConfiguration&&);
    6973
    7074    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
    … …  
    105109    void parseWebVTTCueDataStruct(TrackPrivateRemoteIdentifier, WebCore::ISOWebVTTCue&&);
    106110
     111    void addDataCue(TrackPrivateRemoteIdentifier, MediaTime&& start, MediaTime&& end, IPC::DataReference&&);
     112#if ENABLE(DATACUE_VALUE)
     113    void addDataCueWithType(TrackPrivateRemoteIdentifier, MediaTime&& start, MediaTime&& end, WebCore::SerializedPlatformDataCueValue&&, String&&);
     114    void updateDataCue(TrackPrivateRemoteIdentifier, MediaTime&& start, MediaTime&& end, WebCore::SerializedPlatformDataCueValue&&);
     115    void removeDataCue(TrackPrivateRemoteIdentifier, MediaTime&& start, MediaTime&& end, WebCore::SerializedPlatformDataCueValue&&);
     116#endif
     117
    107118    void requestResource(RemoteMediaResourceIdentifier, WebCore::ResourceRequest&&, WebCore::PlatformMediaResourceLoader::LoadOptions, CompletionHandler<void()>&&);
    108119    void removeResource(RemoteMediaResourceIdentifier);

  • trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in

    r256536 r257125  
    5151    ParseWebVTTCueDataStruct(WebKit::TrackPrivateRemoteIdentifier trackID, WebCore::ISOWebVTTCue cueData)
    5252
    53     ## AddDataCue(MediaTime start, MediaTime end, IPC::DataReference data)
    54     ## AddDataCue(MediaTime start, MediaTime end, WebCore::SerializedPlatformDataCue data, String type)
    55     ## UpdateDataCue(MediaTime start, MediaTime end, WebCore::SerializedPlatformDataCue data)
    56     ## RemoveDataCue(MediaTime start, MediaTime end, WebCore::SerializedPlatformDataCue data)
     53    AddDataCue(WebKit::TrackPrivateRemoteIdentifier trackID, MediaTime start, MediaTime end, IPC::DataReference data)
     54#if ENABLE(DATACUE_VALUE)
     55    AddDataCueWithType(WebKit::TrackPrivateRemoteIdentifier trackID, MediaTime start, MediaTime end, WebCore::SerializedPlatformDataCueValue cueData, String type)
     56    UpdateDataCue(WebKit::TrackPrivateRemoteIdentifier trackID, MediaTime start, MediaTime end, WebCore::SerializedPlatformDataCueValue cueData)
     57    RemoveDataCue(WebKit::TrackPrivateRemoteIdentifier trackID, MediaTime start, MediaTime end, WebCore::SerializedPlatformDataCueValue cueData)
     58#endif
    5759
    5860    ## AddGenericCue(WebCore::GenericCueData cue)

  • trunk/Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.cpp

    r255346 r257125  
    145145std::unique_ptr<MediaPlayerPrivateInterface> RemoteMediaPlayerManager::createRemoteMediaPlayer(MediaPlayer* player, MediaPlayerEnums::MediaEngineIdentifier remoteEngineIdentifier)
    146146{
    147     auto id = MediaPlayerPrivateRemoteIdentifier::generate();
    148147
    149148    RemoteMediaPlayerProxyConfiguration proxyConfiguration;
    … …  
    160159    proxyConfiguration.isVideo = player->isVideoPlayer();
    161160
     161    auto identifier = MediaPlayerPrivateRemoteIdentifier::generate();
    162162    RemoteMediaPlayerConfiguration playerConfiguration;
    163     bool sendSucceeded = gpuProcessConnection().connection().sendSync(Messages::RemoteMediaPlayerManagerProxy::CreateMediaPlayer(id, remoteEngineIdentifier, proxyConfiguration), Messages::RemoteMediaPlayerManagerProxy::CreateMediaPlayer::Reply(playerConfiguration), 0);
    164     if (!sendSucceeded) {
    165         WTFLogAlways("Failed to create remote media player.");
    166         return nullptr;
    167     }
    168 
    169     auto remotePlayer = MediaPlayerPrivateRemote::create(player, remoteEngineIdentifier, id, *this, playerConfiguration);
    170     m_players.add(id, makeWeakPtr(*remotePlayer));
     163    gpuProcessConnection().connection().sendWithAsyncReply(Messages::RemoteMediaPlayerManagerProxy::CreateMediaPlayer(identifier, remoteEngineIdentifier, proxyConfiguration), [this, weakThis = makeWeakPtr(this), identifier](auto&& playerConfiguration) {
     164        if (!weakThis)
     165            return;
     166
     167        if (const auto& player = m_players.get(identifier))
     168            player->setConfiguration(WTFMove(playerConfiguration));
     169    }, 0);
     170
     171    auto remotePlayer = MediaPlayerPrivateRemote::create(player, remoteEngineIdentifier, identifier, *this);
     172    m_players.add(identifier, makeWeakPtr(*remotePlayer));
     173
    171174    return remotePlayer;
    172175}

  • trunk/Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.h

    r254809 r257125  
    5252
    5353class RemoteMediaPlayerManager
    54     : public WebProcessSupplement {
     54    : public WebProcessSupplement
     55    , public CanMakeWeakPtr<RemoteMediaPlayerManager> {
    5556    WTF_MAKE_FAST_ALLOCATED;
    5657public:

  • trunk/Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.cpp

    r256536 r257125  
    8989}
    9090
    91 void TextTrackPrivateRemote::addDataCue(const MediaTime& start, const MediaTime& end, const void*, unsigned)
    92 {
    93     notImplemented();
    94 }
    95 
    96 #if ENABLE(DATACUE_VALUE)
    97 void TextTrackPrivateRemote::addDataCue(const MediaTime& start, const MediaTime& end, Ref<SerializedPlatformDataCue>&&, const String&)
    98 {
    99     notImplemented();
    100 }
    101 
    102 void TextTrackPrivateRemote::updateDataCue(const MediaTime& start, const MediaTime& end, SerializedPlatformDataCue&)
    103 {
    104     notImplemented();
    105 }
    106 
    107 void TextTrackPrivateRemote::removeDataCue(const MediaTime& start, const MediaTime& end, SerializedPlatformDataCue&)
    108 {
    109     notImplemented();
    110 }
    111 
    112 #endif
    113 
    11491void TextTrackPrivateRemote::addGenericCue(GenericCueData&)
    11592{
    … …  
    149126}
    150127
     128void TextTrackPrivateRemote::addDataCue(MediaTime&& start, MediaTime&& end, IPC::DataReference&& data)
     129{
     130    ASSERT(client());
     131    if (auto* client = this->client())
     132        client->addDataCue(WTFMove(start), WTFMove(end), reinterpret_cast<const char*>(data.data()), data.size());
     133}
     134
     135#if ENABLE(DATACUE_VALUE)
     136void TextTrackPrivateRemote::addDataCueWithType(MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&& dataValue, String&& type)
     137{
     138    ASSERT(client());
     139    if (auto* client = this->client())
     140        client->addDataCue(WTFMove(start), WTFMove(end), SerializedPlatformDataCue::create(WTFMove(dataValue)), type);
     141}
     142
     143void TextTrackPrivateRemote::updateDataCue(MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&& dataValue)
     144{
     145    ASSERT(client());
     146    if (auto* client = this->client())
     147        client->updateDataCue(WTFMove(start), WTFMove(end), SerializedPlatformDataCue::create(WTFMove(dataValue)));
     148}
     149
     150void TextTrackPrivateRemote::removeDataCue(MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&& dataValue)
     151{
     152    ASSERT(client());
     153    if (auto* client = this->client())
     154        client->removeDataCue(WTFMove(start), WTFMove(end), SerializedPlatformDataCue::create(WTFMove(dataValue)));
     155}
     156#endif
    151157
    152158} // namespace WebKit

  • trunk/Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.h

    r256536 r257125  
    3333#include <WebCore/InbandTextTrackPrivate.h>
    3434
     35namespace WebCore {
     36class GenericCueData;
     37class ISOWebVTTCue;
     38}
     39
    3540namespace WebKit {
    3641
    … …  
    4550    }
    4651
    47     void addDataCue(const MediaTime& start, const MediaTime& end, const void*, unsigned);
     52    void addDataCue(MediaTime&& start, MediaTime&& end, IPC::DataReference&&);
    4853
    4954#if ENABLE(DATACUE_VALUE)
    50     void addDataCue(const MediaTime& start, const MediaTime& end, Ref<SerializedPlatformDataCue>&&, const String&);
    51     void updateDataCue(const MediaTime& start, const MediaTime& end, SerializedPlatformDataCue&);
    52     void removeDataCue(const MediaTime& start, const MediaTime& end, SerializedPlatformDataCue&);
     55    void addDataCueWithType(MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&&, String&&);
     56    void updateDataCue(MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&&);
     57    void removeDataCue(MediaTime&& start, MediaTime&& end, SerializedPlatformDataCueValue&&);
    5358#endif
    5459
    55     void addGenericCue(GenericCueData&);
    56     void updateGenericCue(GenericCueData&);
    57     void removeGenericCue(GenericCueData&);
     60    void addGenericCue(WebCore::GenericCueData&);
     61    void updateGenericCue(WebCore::GenericCueData&);
     62    void removeGenericCue(WebCore::GenericCueData&);
    5863
    5964    void parseWebVTTFileHeader(String&&);
    6065    void parseWebVTTCueData(const IPC::DataReference&);
    61     void parseWebVTTCueDataStruct(ISOWebVTTCue&&);
     66    void parseWebVTTCueDataStruct(WebCore::ISOWebVTTCue&&);
    6267
    6368    void updateConfiguration(TextTrackPrivateRemoteConfiguration&&);
Note: See TracChangeset for help on using the changeset viewer.