Changeset 293488 in webkit


Ignore:
Timestamp:
Apr 26, 2022 5:29:15 PM (3 months ago)
Author:
Jean-Yves Avenard
Message:

MediaSession.setPositionState() does not work.
​https://bugs.webkit.org/show_bug.cgi?id=237196

Reviewed by Eric Carlson.

Use MediaSession's positionState if present when constructing NowPlaying
MediaRemote data.
We emulate what the media-session position would have been had the
time progressed according to the playback rate.
As such, we need to tell the media-session when no action handlers are
set that the media was paused or played again.
When seeking, we update the new time position and continue according to
the set playback rate.

Tested manually with the following scenarios:

  • Set a page only defining the media-session's position state once
  • Let the media element play by default, check the Now Playing position
  • Pause/Play using Now Playing control and checks that the time doesn't jump around.
  • Seek using Now Playing and check that the media seeks properly and that the Now Playing positon progress from the seek point according to the media-session's playback rate.
  • Seek using default media control and check the same as above.
  • Seek using default media control, then pause in Now Playing: check that the playback position doesn't jump around.

We don't have a way to properly test the MediaRemote
external framework and with existing MediaSession API tests currently
disabled following bug 221514.

  • Modules/mediasession/MediaSession.cpp:

(WebCore::MediaSession::setPlaybackState):
(WebCore::MediaSession::updateReportedPosition):
(WebCore::MediaSession::willBeginPlayback):
(WebCore::MediaSession::willPausePlayback):
(WebCore::MediaSession::positionOverridden):

  • Modules/mediasession/MediaSession.h:
  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::parseAttribute):
(WebCore::HTMLMediaElement::removedFromAncestor):
(WebCore::HTMLMediaElement::mediaLoadingFailed):
(WebCore::HTMLMediaElement::setReadyState):
(WebCore::HTMLMediaElement::finishSeek):
(WebCore::HTMLMediaElement::clearMediaPlayer):

  • html/MediaElementSession.cpp:

(WebCore::MediaElementSession::clientWillBeginPlayback):
(WebCore::MediaElementSession::clientWillPausePlayback):
(WebCore::MediaElementSession::nowPlayingInfo const):
(WebCore::MediaElementSession::metadataChanged):
(WebCore::MediaElementSession::positionStateChanged):
(WebCore::MediaElementSession::clientCharacteristicsChanged):

  • html/MediaElementSession.h:
  • platform/audio/NowPlayingInfo.h:

(WebCore::NowPlayingInfo::operator== const):
(WebCore::NowPlayingInfo::encode const):
(WebCore::NowPlayingInfo::decode):

  • platform/audio/PlatformMediaSession.cpp:

(WebCore::PlatformMediaSession::clientCharacteristicsChanged):

  • platform/audio/PlatformMediaSession.h:
  • platform/audio/PlatformMediaSessionManager.h:

(WebCore::PlatformMediaSessionManager::clientCharacteristicsChanged):

  • platform/audio/cocoa/MediaSessionManagerCocoa.h:
  • platform/audio/cocoa/MediaSessionManagerCocoa.mm:

(WebCore::MediaSessionManagerCocoa::clientCharacteristicsChanged):
(WebCore::MediaSessionManagerCocoa::setNowPlayingInfo):

Location:
trunk/Source/WebCore
Files:
14 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r293486 r293488  
     12022-04-26  Jean-Yves Avenard  <jya@apple.com>
     2
     3        MediaSession.setPositionState() does not work.
     4        https://bugs.webkit.org/show_bug.cgi?id=237196
     5
     6        Reviewed by Eric Carlson.
     7
     8        Use MediaSession's positionState if present when constructing NowPlaying
     9        MediaRemote data.
     10        We emulate what the media-session position would have been had the
     11        time progressed according to the playback rate.
     12        As such, we need to tell the media-session when no action handlers are
     13        set that the media was paused or played again.
     14        When seeking, we update the new time position and continue according to
     15        the set playback rate.
     16
     17        Tested manually with the following scenarios:
     18        - Set a page only defining the media-session's position state once
     19        - Let the media element play by default, check the Now Playing position
     20        - Pause/Play using Now Playing control and checks that the time doesn't
     21          jump around.
     22        - Seek using Now Playing and check that the media seeks properly and that
     23          the Now Playing positon progress from the seek point according to the
     24          media-session's playback rate.
     25        - Seek using default media control and check the same as above.
     26        - Seek using default media control, then pause in Now Playing: check that
     27          the playback position doesn't jump around.
     28
     29        We don't have a way to properly test the MediaRemote
     30        external framework and with existing MediaSession API tests currently
     31        disabled following bug 221514.
     32
     33        * Modules/mediasession/MediaSession.cpp:
     34        (WebCore::MediaSession::setPlaybackState):
     35        (WebCore::MediaSession::updateReportedPosition):
     36        (WebCore::MediaSession::willBeginPlayback):
     37        (WebCore::MediaSession::willPausePlayback):
     38        (WebCore::MediaSession::positionOverridden):
     39        * Modules/mediasession/MediaSession.h:
     40        * html/HTMLMediaElement.cpp:
     41        (WebCore::HTMLMediaElement::parseAttribute):
     42        (WebCore::HTMLMediaElement::removedFromAncestor):
     43        (WebCore::HTMLMediaElement::mediaLoadingFailed):
     44        (WebCore::HTMLMediaElement::setReadyState):
     45        (WebCore::HTMLMediaElement::finishSeek):
     46        (WebCore::HTMLMediaElement::clearMediaPlayer):
     47        * html/MediaElementSession.cpp:
     48        (WebCore::MediaElementSession::clientWillBeginPlayback):
     49        (WebCore::MediaElementSession::clientWillPausePlayback):
     50        (WebCore::MediaElementSession::nowPlayingInfo const):
     51        (WebCore::MediaElementSession::metadataChanged):
     52        (WebCore::MediaElementSession::positionStateChanged):
     53        (WebCore::MediaElementSession::clientCharacteristicsChanged):
     54        * html/MediaElementSession.h:
     55        * platform/audio/NowPlayingInfo.h:
     56        (WebCore::NowPlayingInfo::operator== const):
     57        (WebCore::NowPlayingInfo::encode const):
     58        (WebCore::NowPlayingInfo::decode):
     59        * platform/audio/PlatformMediaSession.cpp:
     60        (WebCore::PlatformMediaSession::clientCharacteristicsChanged):
     61        * platform/audio/PlatformMediaSession.h:
     62        * platform/audio/PlatformMediaSessionManager.h:
     63        (WebCore::PlatformMediaSessionManager::clientCharacteristicsChanged):
     64        * platform/audio/cocoa/MediaSessionManagerCocoa.h:
     65        * platform/audio/cocoa/MediaSessionManagerCocoa.mm:
     66        (WebCore::MediaSessionManagerCocoa::clientCharacteristicsChanged):
     67        (WebCore::MediaSessionManagerCocoa::setNowPlayingInfo):
     68
    1692022-04-26  Elliott Williams  <emw@apple.com>
    270

  • trunk/Source/WebCore/Modules/mediasession/MediaSession.cpp

    r287138 r293488  
    229229    ALWAYS_LOG(LOGIDENTIFIER, state);
    230230
    231     auto currentPosition = this->currentPosition();
    232     if (m_positionState && currentPosition) {
    233         m_positionState->position = *currentPosition;
    234         m_timeAtLastPositionUpdate = MonotonicTime::now();
    235     }
     231    updateReportedPosition();
     232
    236233    m_playbackState = state;
    237234    notifyPlaybackStateObservers();
    … …  
    397394}
    398395
     396void MediaSession::updateReportedPosition()
     397{
     398    auto currentPosition = this->currentPosition();
     399    if (m_positionState && currentPosition) {
     400        m_lastReportedPosition = m_positionState->position = *currentPosition;
     401        m_timeAtLastPositionUpdate = MonotonicTime::now();
     402    }
     403}
     404
     405void MediaSession::willBeginPlayback()
     406{
     407    updateReportedPosition();
     408    m_playbackState = MediaSessionPlaybackState::Playing;
     409    notifyPositionStateObservers();
     410}
     411
     412void MediaSession::willPausePlayback()
     413{
     414    updateReportedPosition();
     415    m_playbackState = MediaSessionPlaybackState::Paused;
     416    notifyPositionStateObservers();
     417}
     418
    399419#if ENABLE(MEDIA_SESSION_COORDINATOR)
    400420void MediaSession::notifyReadyStateObservers()

  • trunk/Source/WebCore/Modules/mediasession/MediaSession.h

    r284857 r293488  
    7272
    7373    WEBCORE_EXPORT std::optional<double> currentPosition() const;
     74    void willBeginPlayback();
     75    void willPausePlayback();
    7476
    7577    Document* document() const;
    … …  
    120122
    121123    const void* logIdentifier() const { return m_logIdentifier; }
     124
     125    void updateReportedPosition();
    122126
    123127    void forEachObserver(const Function<void(Observer&)>&);

  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r293484 r293488  
    790790    } else if (name == titleAttr) {
    791791        if (m_mediaSession)
    792             m_mediaSession->clientCharacteristicsChanged();
     792            m_mediaSession->clientCharacteristicsChanged(false);
    793793    }
    794794    else
    … …  
    894894
    895895    if (m_mediaSession)
    896         m_mediaSession->clientCharacteristicsChanged();
     896        m_mediaSession->clientCharacteristicsChanged(false);
    897897
    898898    HTMLElement::removedFromAncestor(removalType, oldParentOfRemovedTree);
    … …  
    23532353    logMediaLoadRequest(document().page(), String(), convertEnumerationToString(error), false);
    23542354
    2355     mediaSession().clientCharacteristicsChanged();
     2355    mediaSession().clientCharacteristicsChanged(false);
    23562356#if ENABLE(WIRELESS_PLAYBACK_TARGET)
    23572357    if (!m_hasPlaybackTargetAvailabilityListeners)
    … …  
    25782578#endif
    25792579
    2580             mediaSession().clientCharacteristicsChanged();
     2580            mediaSession().clientCharacteristicsChanged(false);
    25812581
    25822582            // As the spec only mentiones HAVE_METADATA, run the later
    … …  
    33203320
    33213321    if (m_mediaSession)
    3322         m_mediaSession->clientCharacteristicsChanged();
     3322        m_mediaSession->clientCharacteristicsChanged(true);
    33233323
    33243324#if ENABLE(MEDIA_SOURCE)
    … …  
    59195919    queueTaskKeepingObjectAlive(*this, TaskSource::MediaElement, [this] {
    59205920        if (m_mediaSession) {
    5921             m_mediaSession->clientCharacteristicsChanged();
     5921            m_mediaSession->clientCharacteristicsChanged(false);
    59225922            m_mediaSession->canProduceAudioChanged();
    59235923        }

  • trunk/Source/WebCore/html/MediaElementSession.cpp

    r293484 r293488  
    235235    m_elementIsHiddenBecauseItWasRemovedFromDOM = false;
    236236    updateClientDataBuffering();
     237
     238#if ENABLE(MEDIA_SESSION)
     239    if (auto* session = mediaSession())
     240        session->willBeginPlayback();
     241#endif
     242
    237243    return true;
    238244}
    … …  
    244250
    245251    updateClientDataBuffering();
     252
     253#if ENABLE(MEDIA_SESSION)
     254    if (auto* session = mediaSession())
     255        session->willPausePlayback();
     256#endif
     257
    246258    return true;
    247259}
    … …  
    11771189    bool isPlaying = state() == PlatformMediaSession::Playing;
    11781190    bool supportsSeeking = m_element.supportsSeeking();
     1191    double rate = 1.0;
    11791192    double duration = supportsSeeking ? m_element.duration() : MediaPlayer::invalidTime();
    11801193    double currentTime = m_element.currentTime();
    … …  
    11841197#if ENABLE(MEDIA_SESSION)
    11851198    auto* session = mediaSession();
     1199    auto positionState = session ? session->positionState() : std::nullopt;
     1200    auto currentPosition = session ? session->currentPosition() : std::nullopt;
     1201    if (positionState) {
     1202        duration = positionState->duration;
     1203        rate = positionState->playbackRate;
     1204    }
     1205    if (currentPosition)
     1206        currentTime = *currentPosition;
    11861207    auto* sessionMetadata = session ? session->metadata() : nullptr;
    11871208    if (sessionMetadata) {
    … …  
    11911212            artwork = NowPlayingInfoArtwork { sessionMetadata->artworkSrc(), sessionMetadata->artworkImage()->mimeType(), sessionMetadata->artworkImage()->data() };
    11921213        }
    1193         return NowPlayingInfo { sessionMetadata->title(), sessionMetadata->artist(), sessionMetadata->album(), m_element.sourceApplicationIdentifier(), duration, currentTime, supportsSeeking, m_element.mediaUniqueIdentifier(), isPlaying, allowsNowPlayingControlsVisibility, WTFMove(artwork) };
    1194     }
    1195 #endif
    1196 
    1197     return NowPlayingInfo { m_element.mediaSessionTitle(), emptyString(), emptyString(), m_element.sourceApplicationIdentifier(), duration, currentTime, supportsSeeking, m_element.mediaUniqueIdentifier(), isPlaying, allowsNowPlayingControlsVisibility, { }};
     1214        return NowPlayingInfo { sessionMetadata->title(), sessionMetadata->artist(), sessionMetadata->album(), m_element.sourceApplicationIdentifier(), duration, currentTime, rate, supportsSeeking, m_element.mediaUniqueIdentifier(), isPlaying, allowsNowPlayingControlsVisibility, WTFMove(artwork) };
     1215    }
     1216#endif
     1217
     1218    return NowPlayingInfo { m_element.mediaSessionTitle(), emptyString(), emptyString(), m_element.sourceApplicationIdentifier(), duration, currentTime, rate, supportsSeeking, m_element.mediaUniqueIdentifier(), isPlaying, allowsNowPlayingControlsVisibility, { } };
    11981219}
    11991220
    … …  
    13001321void MediaElementSession::metadataChanged(const RefPtr<MediaMetadata>&)
    13011322{
    1302     clientCharacteristicsChanged();
    1303 }
    1304 
    1305 void MediaElementSession::positionStateChanged(const std::optional<MediaPositionState>&) { }
     1323    clientCharacteristicsChanged(false);
     1324}
     1325
     1326void MediaElementSession::positionStateChanged(const std::optional<MediaPositionState>&)
     1327{
     1328    clientCharacteristicsChanged(false);
     1329}
    13061330
    13071331void MediaElementSession::playbackStateChanged(MediaSessionPlaybackState) { }
    … …  
    13091333void MediaElementSession::actionHandlersChanged() { }
    13101334
     1335void MediaElementSession::clientCharacteristicsChanged(bool positionChanged)
     1336{
     1337#if ENABLE(MEDIA_SESSION)
     1338    auto* session = mediaSession();
     1339    if (positionChanged && session) {
     1340        auto positionState = session->positionState();
     1341        if (positionState)
     1342            session->setPositionState(MediaPositionState { positionState->duration, positionState->playbackRate, m_element.currentTime() });
     1343    }
     1344#endif
     1345    PlatformMediaSession::clientCharacteristicsChanged(positionChanged);
     1346}
     1347
    13111348}
    13121349

  • trunk/Source/WebCore/html/MediaElementSession.h

    r284080 r293488  
    7171    bool clientWillBeginPlayback() final;
    7272    bool clientWillPausePlayback() final;
     73    void clientCharacteristicsChanged(bool) final;
    7374
    7475    void visibilityChanged();

  • trunk/Source/WebCore/platform/audio/NowPlayingInfo.h

    r287021 r293488  
    8282    double duration { 0 };
    8383    double currentTime { 0 };
     84    double rate { 1.0 };
    8485    bool supportsSeeking { false };
    8586    MediaUniqueIdentifier uniqueIdentifier;
    … …  
    9697            && duration == other.duration
    9798            && currentTime == other.currentTime
     99            && rate == other.rate
    98100            && supportsSeeking == other.supportsSeeking
    99101            && uniqueIdentifier == other.uniqueIdentifier
    … …  
    114116template<class Encoder> inline void NowPlayingInfo::encode(Encoder& encoder) const
    115117{
    116     encoder << title << artist << album << sourceApplicationIdentifier << duration << currentTime << supportsSeeking << uniqueIdentifier << isPlaying << allowsNowPlayingControlsVisibility << artwork;
     118    encoder << title << artist << album << sourceApplicationIdentifier << duration << currentTime << rate << supportsSeeking << uniqueIdentifier << isPlaying << allowsNowPlayingControlsVisibility << artwork;
    117119}
    118120
    … …  
    143145        return { };
    144146
     147    double rate;
     148    if (!decoder.decode(rate))
     149        return { };
     150
    145151    bool supportsSeeking;
    146152    if (!decoder.decode(supportsSeeking))
    … …  
    163169        return { };
    164170
    165     return NowPlayingInfo { WTFMove(title), WTFMove(artist), WTFMove(album), WTFMove(sourceApplicationIdentifier), duration, currentTime, supportsSeeking, uniqueIdentifier, isPlaying, allowsNowPlayingControlsVisibility, WTFMove(artwork) };
     171    return NowPlayingInfo { WTFMove(title), WTFMove(artist), WTFMove(album), WTFMove(sourceApplicationIdentifier), duration, currentTime, rate, supportsSeeking, uniqueIdentifier, isPlaying, allowsNowPlayingControlsVisibility, WTFMove(artwork) };
    166172}
    167173

  • trunk/Source/WebCore/platform/audio/PlatformMediaSession.cpp

    r292563 r293488  
    376376}
    377377
    378 void PlatformMediaSession::clientCharacteristicsChanged()
    379 {
    380     PlatformMediaSessionManager::sharedManager().clientCharacteristicsChanged(*this);
     378void PlatformMediaSession::clientCharacteristicsChanged(bool positionChanged)
     379{
     380    PlatformMediaSessionManager::sharedManager().clientCharacteristicsChanged(*this, positionChanged);
    381381}
    382382

  • trunk/Source/WebCore/platform/audio/PlatformMediaSession.h

    r292563 r293488  
    103103    };
    104104
    105     void clientCharacteristicsChanged();
     105    virtual void clientCharacteristicsChanged(bool);
    106106
    107107    void beginInterruption(InterruptionType);

  • trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h

    r291759 r293488  
    132132    virtual void sessionStateChanged(PlatformMediaSession&);
    133133    virtual void sessionDidEndRemoteScrubbing(PlatformMediaSession&) { };
    134     virtual void clientCharacteristicsChanged(PlatformMediaSession&) { }
     134    virtual void clientCharacteristicsChanged(PlatformMediaSession&, bool) { }
    135135    virtual void sessionCanProduceAudioChanged();
    136136

  • trunk/Source/WebCore/platform/audio/cocoa/MediaSessionManagerCocoa.h

    r291759 r293488  
    8888    void sessionWillEndPlayback(PlatformMediaSession&, DelayCallingUpdateNowPlaying) override;
    8989    void sessionDidEndRemoteScrubbing(PlatformMediaSession&) final;
    90     void clientCharacteristicsChanged(PlatformMediaSession&) final;
     90    void clientCharacteristicsChanged(PlatformMediaSession&, bool) final;
    9191    void sessionCanProduceAudioChanged() final;
    9292

  • trunk/Source/WebCore/platform/audio/cocoa/MediaSessionManagerCocoa.mm

    r292563 r293488  
    310310}
    311311
    312 void MediaSessionManagerCocoa::clientCharacteristicsChanged(PlatformMediaSession& session)
     312void MediaSessionManagerCocoa::clientCharacteristicsChanged(PlatformMediaSession& session, bool)
    313313{
    314314    ALWAYS_LOG(LOGIDENTIFIER, session.logIdentifier());
    … …  
    377377    }
    378378
    379     double rate = nowPlayingInfo.isPlaying ? 1 : 0;
     379    double rate = nowPlayingInfo.isPlaying ? nowPlayingInfo.rate : 0;
    380380    auto cfRate = adoptCF(CFNumberCreate(kCFAllocatorDefault, kCFNumberDoubleType, &rate));
    381381    CFDictionarySetValue(info.get(), kMRMediaRemoteNowPlayingInfoPlaybackRate, cfRate.get());

  • trunk/Source/WebCore/platform/audio/glib/MediaSessionManagerGLib.cpp

    r285654 r293488  
    228228}
    229229
    230 void MediaSessionManagerGLib::clientCharacteristicsChanged(PlatformMediaSession& platformSession)
     230void MediaSessionManagerGLib::clientCharacteristicsChanged(PlatformMediaSession& platformSession, bool)
    231231{
    232232    ALWAYS_LOG(LOGIDENTIFIER, platformSession.logIdentifier());

  • trunk/Source/WebCore/platform/audio/glib/MediaSessionManagerGLib.h

    r283437 r293488  
    6666    void sessionStateChanged(PlatformMediaSession&) override;
    6767    void sessionDidEndRemoteScrubbing(PlatformMediaSession&) final;
    68     void clientCharacteristicsChanged(PlatformMediaSession&) final;
     68    void clientCharacteristicsChanged(PlatformMediaSession&, bool) final;
    6969    void sessionCanProduceAudioChanged() final;
    7070
Note: See TracChangeset for help on using the changeset viewer.