Changeset 206350 in webkit


Ignore:
Timestamp:
Sep 23, 2016, 6:38:52 PM (9 years ago)
Author:
Wenson Hsieh
Message:

Media controls playhead does not animate smoothly while playing
​https://bugs.webkit.org/show_bug.cgi?id=162399
<rdar://problem/28115680>

Reviewed by Beth Dakin.

Source/WebCore:

The media controls playhead currently does not animate smoothly during playback because we don't specify a
playback rate when updating the WebPlaybackControlsManager's timing value. However, simply setting this timing
value to the current playback rate (as known to the UI process) results in the UI process receiving multiple
updates from the web process where the current time is equal (or even less than) the time at which media began
to play, which results in the playhead seeking backwards to the start time multiple times when playing or
resuming media.

To address this, in WebCore, we inform the playback session model of the media time when playback begins (i.e.
a playing or play event is fired). This message precedes both the "rate changed" and "current time changed"
messages.

Unit tests to be added in a future patch.

  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::notifyAboutPlaying):
(WebCore::HTMLMediaElement::setReadyState):
(WebCore::HTMLMediaElement::playInternal):

  • html/HTMLMediaElement.h:

(WebCore::HTMLMediaElement::playbackStartedTime):

  • platform/cocoa/WebPlaybackSessionModel.h:

(WebCore::WebPlaybackSessionModelClient::playbackStartedTimeChanged):
(WebCore::WebPlaybackSessionModelClient::bufferedTimeChanged): Deleted.

  • platform/cocoa/WebPlaybackSessionModelMediaElement.h:
  • platform/cocoa/WebPlaybackSessionModelMediaElement.mm:

(WebPlaybackSessionModelMediaElement::updateForEventName):
(WebPlaybackSessionModelMediaElement::playbackStartedTime):

  • platform/ios/WebVideoFullscreenControllerAVKit.mm:
  • platform/mac/WebPlaybackSessionInterfaceMac.h:
  • platform/mac/WebPlaybackSessionInterfaceMac.mm:

(WebCore::WebPlaybackSessionInterfaceMac::currentTimeChanged):
(WebCore::WebPlaybackSessionInterfaceMac::rateChanged):
(WebCore::WebPlaybackSessionInterfaceMac::beginScrubbing):
(WebCore::WebPlaybackSessionInterfaceMac::endScrubbing):
(WebCore::WebPlaybackSessionInterfaceMac::updatePlaybackControlsManagerTiming):

Source/WebKit2:

See WebCore ChangeLog for more details.

In the UI process, we update the WebPlaybackSessionManager's timing value when the rate or current time changes.
Each AVValueTiming is generated from the current time, system anchor time, and playback rate. The behavior of
the first two properties is unaffected. However, the rate used to update the timing value is the effective
playback rate, which is equal to the playback rate unless we are (1) not playing, (2) interacting with the media
controls in such a way that the media is essentially not playing, or (3) the current time precedes the playback
start time, accounting for playback direction. In these cases, our effective playback rate is 0, which means
that we do not animate the playhead.

  • UIProcess/Cocoa/WebPlaybackSessionManagerProxy.h:
  • UIProcess/Cocoa/WebPlaybackSessionManagerProxy.messages.in:
  • UIProcess/Cocoa/WebPlaybackSessionManagerProxy.mm:

(WebKit::WebPlaybackSessionModelContext::beginScrubbing):
(WebKit::WebPlaybackSessionModelContext::endScrubbing):
(WebKit::WebPlaybackSessionModelContext::setPlaybackStartedTime):
(WebKit::WebPlaybackSessionModelContext::setCurrentTime):
(WebKit::WebPlaybackSessionManagerProxy::setPlaybackStartedTime):

  • WebProcess/cocoa/WebPlaybackSessionManager.h:
  • WebProcess/cocoa/WebPlaybackSessionManager.mm:

(WebKit::WebPlaybackSessionInterfaceContext::playbackStartedTimeChanged):
(WebKit::WebPlaybackSessionManager::playbackStartedTimeChanged):

Location:
trunk/Source
Files:
15 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r206343 r206350  
     12016-09-23  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Media controls playhead does not animate smoothly while playing
     4        https://bugs.webkit.org/show_bug.cgi?id=162399
     5        <rdar://problem/28115680>
     6
     7        Reviewed by Beth Dakin.
     8
     9        The media controls playhead currently does not animate smoothly during playback because we don't specify a
     10        playback rate when updating the WebPlaybackControlsManager's timing value. However, simply setting this timing
     11        value to the current playback rate (as known to the UI process) results in the UI process receiving multiple
     12        updates from the web process where the current time is equal (or even less than) the time at which media began
     13        to play, which results in the playhead seeking backwards to the start time multiple times when playing or
     14        resuming media.
     15
     16        To address this, in WebCore, we inform the playback session model of the media time when playback begins (i.e.
     17        a `playing` or `play` event is fired). This message precedes both the "rate changed" and "current time changed"
     18        messages.
     19
     20        Unit tests to be added in a future patch.
     21
     22        * html/HTMLMediaElement.cpp:
     23        (WebCore::HTMLMediaElement::notifyAboutPlaying):
     24        (WebCore::HTMLMediaElement::setReadyState):
     25        (WebCore::HTMLMediaElement::playInternal):
     26        * html/HTMLMediaElement.h:
     27        (WebCore::HTMLMediaElement::playbackStartedTime):
     28        * platform/cocoa/WebPlaybackSessionModel.h:
     29        (WebCore::WebPlaybackSessionModelClient::playbackStartedTimeChanged):
     30        (WebCore::WebPlaybackSessionModelClient::bufferedTimeChanged): Deleted.
     31        * platform/cocoa/WebPlaybackSessionModelMediaElement.h:
     32        * platform/cocoa/WebPlaybackSessionModelMediaElement.mm:
     33        (WebPlaybackSessionModelMediaElement::updateForEventName):
     34        (WebPlaybackSessionModelMediaElement::playbackStartedTime):
     35        * platform/ios/WebVideoFullscreenControllerAVKit.mm:
     36        * platform/mac/WebPlaybackSessionInterfaceMac.h:
     37        * platform/mac/WebPlaybackSessionInterfaceMac.mm:
     38        (WebCore::WebPlaybackSessionInterfaceMac::currentTimeChanged):
     39        (WebCore::WebPlaybackSessionInterfaceMac::rateChanged):
     40        (WebCore::WebPlaybackSessionInterfaceMac::beginScrubbing):
     41        (WebCore::WebPlaybackSessionInterfaceMac::endScrubbing):
     42        (WebCore::WebPlaybackSessionInterfaceMac::updatePlaybackControlsManagerTiming):
     43
    1442016-09-23  Zalan Bujtas  <zalan@apple.com>
    245

  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r206315 r206350  
    10291029void HTMLMediaElement::notifyAboutPlaying()
    10301030{
     1031    m_playbackStartedTime = currentMediaTime().toDouble();
    10311032    dispatchEvent(Event::create(eventNames().playingEvent, false, true));
    10321033    resolvePendingPlayPromises();
    … …  
    24222423            m_paused = false;
    24232424            invalidateCachedTime();
     2425            m_playbackStartedTime = currentMediaTime().toDouble();
    24242426            scheduleEvent(eventNames().playEvent);
    24252427            scheduleNotifyAboutPlaying();
    … …  
    32113213        m_paused = false;
    32123214        invalidateCachedTime();
     3215        m_playbackStartedTime = currentMediaTime().toDouble();
    32133216        scheduleEvent(eventNames().playEvent);
    32143217

  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r206315 r206350  
    479479    bool hasEverHadVideo() const { return m_hasEverHadVideo; }
    480480
     481    double playbackStartedTime() const { return m_playbackStartedTime; }
     482
    481483protected:
    482484    HTMLMediaElement(const QualifiedName&, Document&, bool createdByParser);
    … …  
    859861   
    860862    double m_previousProgressTime;
     863    double m_playbackStartedTime { 0 };
    861864
    862865    // The last time a timeupdate event was sent (based on monotonic clock).

  • trunk/Source/WebCore/platform/cocoa/WebPlaybackSessionModel.h

    r205365 r206350  
    6060    enum ExternalPlaybackTargetType { TargetTypeNone, TargetTypeAirPlay, TargetTypeTVOut };
    6161
     62    virtual double playbackStartedTime() const = 0;
    6263    virtual double duration() const = 0;
    6364    virtual double currentTime() const = 0;
    6465    virtual double bufferedTime() const = 0;
    6566    virtual bool isPlaying() const = 0;
     67    virtual bool isScrubbing() const = 0;
    6668    virtual float playbackRate() const = 0;
    6769    virtual Ref<TimeRanges> seekableRanges() const = 0;
    … …  
    8385    virtual void currentTimeChanged(double /* currentTime */, double /* anchorTime */) { }
    8486    virtual void bufferedTimeChanged(double) { }
     87    virtual void playbackStartedTimeChanged(double /* playbackStartedTime */) { }
    8588    virtual void rateChanged(bool /* isPlaying */, float /* playbackRate */) { }
    8689    virtual void seekableRangesChanged(const TimeRanges&) { }

  • trunk/Source/WebCore/platform/cocoa/WebPlaybackSessionModelMediaElement.h

    r206245 r206350  
    7474    double bufferedTime() const final;
    7575    bool isPlaying() const final;
     76    bool isScrubbing() const final { return false; }
    7677    float playbackRate() const final;
    7778    Ref<TimeRanges> seekableRanges() const final;
    … …  
    99100    Vector<RefPtr<AudioTrack>> m_audioTracksForMenu;
    100101   
     102    double playbackStartedTime() const;
    101103    void updateLegibleOptions();
    102104};

  • trunk/Source/WebCore/platform/cocoa/WebPlaybackSessionModelMediaElement.mm

    r205412 r206350  
    107107
    108108    if (all
     109        || eventName == eventNames().playEvent
     110        || eventName == eventNames().playingEvent) {
     111        for (auto client : m_clients)
     112            client->playbackStartedTimeChanged(playbackStartedTime());
     113    }
     114
     115    if (all
    109116        || eventName == eventNames().pauseEvent
    110117        || eventName == eventNames().playEvent
    … …  
    286293        client->legibleMediaSelectionOptionsChanged(legibleOptions, legibleIndex);
    287294    }
     295}
     296
     297double WebPlaybackSessionModelMediaElement::playbackStartedTime() const
     298{
     299    if (!m_mediaElement)
     300        return 0;
     301
     302    return m_mediaElement->playbackStartedTime();
    288303}
    289304

  • trunk/Source/WebCore/platform/ios/WebVideoFullscreenControllerAVKit.mm

    r205365 r206350  
    148148    void selectLegibleMediaOption(uint64_t) override;
    149149    double duration() const override;
     150    double playbackStartedTime() const override { return 0; }
    150151    double currentTime() const override;
    151152    double bufferedTime() const override;
    152153    bool isPlaying() const override;
     154    bool isScrubbing() const override { return false; }
    153155    float playbackRate() const override;
    154156    Ref<TimeRanges> seekableRanges() const override;

  • trunk/Source/WebCore/platform/mac/WebPlaybackSessionInterfaceMac.h

    r205365 r206350  
    6464    WEBCORE_EXPORT void ensureControlsManager();
    6565    WEBCORE_EXPORT void setPlayBackControlsManager(WebPlaybackControlsManager *);
     66    WEBCORE_EXPORT void beginScrubbing();
     67    WEBCORE_EXPORT void endScrubbing();
    6668    WEBCORE_EXPORT WebPlaybackControlsManager *playBackControlsManager();
    6769
    … …  
    7072    WebPlaybackSessionModel* m_playbackSessionModel { nullptr };
    7173    WebPlaybackControlsManager *m_playbackControlsManager  { nullptr };
     74
     75    void updatePlaybackControlsManagerTiming(double currentTime, double anchorTime, double playbackRate, bool isPlaying);
    7276};
    7377

  • trunk/Source/WebCore/platform/mac/WebPlaybackSessionInterfaceMac.mm

    r205365 r206350  
    7777{
    7878    WebPlaybackControlsManager* controlsManager = playBackControlsManager();
    79 
    80     NSTimeInterval anchorTimeStamp = ![controlsManager rate] ? NAN : anchorTime;
    81     AVValueTiming *timing = [getAVValueTimingClass() valueTimingWithAnchorValue:currentTime
    82         anchorTimeStamp:anchorTimeStamp rate:0];
    83 
    84     [controlsManager setTiming:timing];
     79    updatePlaybackControlsManagerTiming(currentTime, anchorTime, controlsManager.rate, controlsManager.playing);
    8580}
    8681
    … …  
    9085    [controlsManager setRate:isPlaying ? playbackRate : 0.];
    9186    [controlsManager setPlaying:isPlaying];
     87    updatePlaybackControlsManagerTiming(m_playbackSessionModel ? m_playbackSessionModel->currentTime() : 0, [[NSProcessInfo processInfo] systemUptime], playbackRate, isPlaying);
     88}
     89
     90void WebPlaybackSessionInterfaceMac::beginScrubbing()
     91{
     92    updatePlaybackControlsManagerTiming(m_playbackSessionModel ? m_playbackSessionModel->currentTime() : 0, [[NSProcessInfo processInfo] systemUptime], 0, false);
     93    webPlaybackSessionModel()->beginScrubbing();
     94}
     95
     96void WebPlaybackSessionInterfaceMac::endScrubbing()
     97{
     98    webPlaybackSessionModel()->endScrubbing();
    9299}
    93100
    … …  
    164171}
    165172
     173void WebPlaybackSessionInterfaceMac::updatePlaybackControlsManagerTiming(double currentTime, double anchorTime, double playbackRate, bool isPlaying)
     174{
     175    WebPlaybackControlsManager *manager = playBackControlsManager();
     176    if (!manager)
     177        return;
     178
     179    WebPlaybackSessionModel *model = webPlaybackSessionModel();
     180    if (!model)
     181        return;
     182
     183    double effectiveAnchorTime = playbackRate ? anchorTime : NAN;
     184    double effectivePlaybackRate = playbackRate;
     185    if (!isPlaying
     186        || model->isScrubbing()
     187        || (manager.rate > 0 && model->playbackStartedTime() >= currentTime)
     188        || (manager.rate < 0 && model->playbackStartedTime() <= currentTime))
     189        effectivePlaybackRate = 0;
     190
     191    manager.timing = [getAVValueTimingClass() valueTimingWithAnchorValue:currentTime anchorTimeStamp:effectiveAnchorTime rate:effectivePlaybackRate];
     192}
     193
    166194}
    167195

  • trunk/Source/WebKit2/ChangeLog

    r206333 r206350  
     12016-09-23  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Media controls playhead does not animate smoothly while playing
     4        https://bugs.webkit.org/show_bug.cgi?id=162399
     5        <rdar://problem/28115680>
     6
     7        Reviewed by Beth Dakin.
     8
     9        See WebCore ChangeLog for more details.
     10
     11        In the UI process, we update the WebPlaybackSessionManager's timing value when the rate or current time changes.
     12        Each AVValueTiming is generated from the current time, system anchor time, and playback rate. The behavior of
     13        the first two properties is unaffected. However, the rate used to update the timing value is the effective
     14        playback rate, which is equal to the playback rate unless we are (1) not playing, (2) interacting with the media
     15        controls in such a way that the media is essentially not playing, or (3) the current time precedes the playback
     16        start time, accounting for playback direction. In these cases, our effective playback rate is 0, which means
     17        that we do not animate the playhead.
     18
     19        * UIProcess/Cocoa/WebPlaybackSessionManagerProxy.h:
     20        * UIProcess/Cocoa/WebPlaybackSessionManagerProxy.messages.in:
     21        * UIProcess/Cocoa/WebPlaybackSessionManagerProxy.mm:
     22        (WebKit::WebPlaybackSessionModelContext::beginScrubbing):
     23        (WebKit::WebPlaybackSessionModelContext::endScrubbing):
     24        (WebKit::WebPlaybackSessionModelContext::setPlaybackStartedTime):
     25        (WebKit::WebPlaybackSessionModelContext::setCurrentTime):
     26        (WebKit::WebPlaybackSessionManagerProxy::setPlaybackStartedTime):
     27        * WebProcess/cocoa/WebPlaybackSessionManager.h:
     28        * WebProcess/cocoa/WebPlaybackSessionManager.mm:
     29        (WebKit::WebPlaybackSessionInterfaceContext::playbackStartedTimeChanged):
     30        (WebKit::WebPlaybackSessionManager::playbackStartedTimeChanged):
     31
    1322016-09-23  Caitlin Potter  <caitp@igalia.com>
    233

  • trunk/Source/WebKit2/UIProcess/Cocoa/WebPlaybackSessionManagerProxy.h

    r205412 r206350  
    7272    void setCurrentTime(double);
    7373    void setBufferedTime(double);
     74    void setPlaybackStartedTime(double);
    7475    void setRate(bool isPlaying, float playbackRate);
    7576    void setSeekableRanges(WebCore::TimeRanges&);
    … …  
    103104    void selectLegibleMediaOption(uint64_t) final;
    104105
     106    double playbackStartedTime() const final { return m_playbackStartedTime; }
    105107    double duration() const final { return m_duration; }
    106108    double currentTime() const final { return m_currentTime; }
    107109    double bufferedTime() const final { return m_bufferedTime; }
    108110    bool isPlaying() const final { return m_isPlaying; }
     111    bool isScrubbing() const final { return m_isScrubbing; }
    109112    float playbackRate() const final { return m_playbackRate; }
    110113    Ref<WebCore::TimeRanges> seekableRanges() const final { return m_seekableRanges.copyRef(); }
    … …  
    122125    uint64_t m_contextId;
    123126    HashSet<WebCore::WebPlaybackSessionModelClient*> m_clients;
     127    double m_playbackStartedTime { 0 };
     128    bool m_playbackStartedTimeNeedsUpdate { false };
    124129    double m_duration { 0 };
    125130    double m_currentTime { 0 };
    126131    double m_bufferedTime { 0 };
    127132    bool m_isPlaying { false };
     133    bool m_isScrubbing { false };
    128134    float m_playbackRate { 0 };
    129135    Ref<WebCore::TimeRanges> m_seekableRanges { WebCore::TimeRanges::create() };
    … …  
    177183    void setWirelessVideoPlaybackDisabled(uint64_t contextId, bool);
    178184    void setDuration(uint64_t contextId, double duration);
     185    void setPlaybackStartedTime(uint64_t contextId, double playbackStartedTime);
    179186    void setRate(uint64_t contextId, bool isPlaying, double rate);
    180187    void handleControlledElementIDResponse(uint64_t, String) const;

  • trunk/Source/WebKit2/UIProcess/Cocoa/WebPlaybackSessionManagerProxy.messages.in

    r205412 r206350  
    3333    SetWirelessVideoPlaybackDisabled(uint64_t contextId, bool disabled)
    3434    SetDuration(uint64_t contextId, double duration)
     35    SetPlaybackStartedTime(uint64_t contextId, double playbackStartedTime)
    3536    SetRate(uint64_t contextId, bool isPlaying, double rate)
    3637    SetUpPlaybackControlsManagerWithID(uint64_t contextId)

  • trunk/Source/WebKit2/UIProcess/Cocoa/WebPlaybackSessionManagerProxy.mm

    r205412 r206350  
    7575    if (m_manager)
    7676        m_manager->beginScrubbing(m_contextId);
     77
     78    m_isScrubbing = true;
    7779}
    7880
    … …  
    8183    if (m_manager)
    8284        m_manager->endScrubbing(m_contextId);
     85
     86    m_isScrubbing = false;
     87    m_playbackStartedTimeNeedsUpdate = isPlaying();
    8388}
    8489
    … …  
    123128    if (m_manager)
    124129        m_manager->selectLegibleMediaOption(m_contextId, optionId);
     130}
     131
     132void WebPlaybackSessionModelContext::setPlaybackStartedTime(double playbackStartedTime)
     133{
     134    m_playbackStartedTime = playbackStartedTime;
     135    m_playbackStartedTimeNeedsUpdate = false;
    125136}
    126137
    … …  
    136147    m_currentTime = currentTime;
    137148    auto anchorTime = [[NSProcessInfo processInfo] systemUptime];
     149    if (m_playbackStartedTimeNeedsUpdate)
     150        setPlaybackStartedTime(currentTime);
    138151
    139152    for (auto* client : m_clients)
    … …  
    390403}
    391404
     405void WebPlaybackSessionManagerProxy::setPlaybackStartedTime(uint64_t contextId, double playbackStartedTime)
     406{
     407    ensureModel(contextId).setPlaybackStartedTime(playbackStartedTime);
     408}
     409
    392410void WebPlaybackSessionManagerProxy::setRate(uint64_t contextId, bool isPlaying, double rate)
    393411{

  • trunk/Source/WebKit2/WebProcess/cocoa/WebPlaybackSessionManager.h

    r205412 r206350  
    7878    void currentTimeChanged(double currentTime, double anchorTime) final;
    7979    void bufferedTimeChanged(double) final;
     80    void playbackStartedTimeChanged(double playbackStartedTime) final;
    8081    void rateChanged(bool isPlaying, float playbackRate) final;
    8182    void seekableRangesChanged(const WebCore::TimeRanges&) final;
    … …  
    123124    void currentTimeChanged(uint64_t contextId, double currentTime, double anchorTime);
    124125    void bufferedTimeChanged(uint64_t contextId, double bufferedTime);
     126    void playbackStartedTimeChanged(uint64_t contextId, double playbackStartedTime);
    125127    void rateChanged(uint64_t contextId, bool isPlaying, float playbackRate);
    126128    void seekableRangesChanged(uint64_t contextId, const WebCore::TimeRanges&);

  • trunk/Source/WebKit2/WebProcess/cocoa/WebPlaybackSessionManager.mm

    r205412 r206350  
    9797}
    9898
     99void WebPlaybackSessionInterfaceContext::playbackStartedTimeChanged(double playbackStartedTime)
     100{
     101    if (m_manager)
     102        m_manager->playbackStartedTimeChanged(m_contextId, playbackStartedTime);
     103}
     104
    99105void WebPlaybackSessionInterfaceContext::seekableRangesChanged(const WebCore::TimeRanges& ranges)
    100106{
    … …  
    298304}
    299305
     306void WebPlaybackSessionManager::playbackStartedTimeChanged(uint64_t contextId, double playbackStartedTime)
     307{
     308    m_page->send(Messages::WebPlaybackSessionManagerProxy::SetPlaybackStartedTime(contextId, playbackStartedTime), m_page->pageID());
     309}
     310
    300311void WebPlaybackSessionManager::rateChanged(uint64_t contextId, bool isPlaying, float playbackRate)
    301312{
Note: See TracChangeset for help on using the changeset viewer.