Changeset 266844 in webkit


Ignore:
Timestamp:
Sep 10, 2020 11:23:51 AM (4 years ago)
Author:
jer.noble@apple.com
Message:

[Cocoa] PERF: Don't instantiate AVPlayer-based audio decoders or renderers if an element is initially muted.
https://bugs.webkit.org/show_bug.cgi?id=216299

Reviewed by Eric Carlson.

Source/WebCore:

When an AVPlayer is created, even if muted, it will still instantiate an audio decoder and renderer if the
AVAsset in the current player item has an audio track. Ostensibly, this is so that an unmute operation is
instantaneous, as it's merely applying a zero gain to the decoded audio. However for web content, there's
many autoplaying, muted <video> elements which may never be un-muted before being destroyed.

Implement a policy where, if an AVPlayer is initially muted, we adopt AVFoundation SPI to forcibly prevent
audio decoding and rendering until the first time the AVPlayer is unmuted. This means the first un-mute may
not be instantaneous, as an audio decoder will have to be created and fed before any audio is rendered.

There's some incorrect caching of mute state at the MediaPlayer level; so MediaPlayer and MPPAVFoundationObjC
can get their respective m_muted states out of sync. Make sure that HTMLMediaElement always sets muted state
after creating a MediaPlayer and that, respectively, MPPAVFoundationObjC always queries it's parent MediaPlayer's
mute state when it in turn is created.

  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::createMediaPlayer):

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::MediaPlayerPrivateAVFoundationObjC):
(WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayer):
(WebCore::MediaPlayerPrivateAVFoundationObjC::setMuted):

Source/WebCore/PAL:

  • pal/spi/cocoa/AVFoundationSPI.h:

Source/WTF:

  • wtf/PlatformHave.h:
Location:
trunk/Source
Files:
7 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WTF/ChangeLog

    r266836 r266844  
     12020-09-10  Jer Noble  <jer.noble@apple.com>
     2
     3        [Cocoa] PERF: Don't instantiate AVPlayer-based audio decoders or renderers if an element is initially muted.
     4        https://bugs.webkit.org/show_bug.cgi?id=216299
     5
     6        Reviewed by Eric Carlson.
     7
     8        * wtf/PlatformHave.h:
     9
    1102020-09-10  Tim Horton  <timothy_horton@apple.com>
    211
  • trunk/Source/WTF/wtf/PlatformHave.h

    r266827 r266844  
    491491#endif
    492492
     493#if (PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 110000) || (PLATFORM(IOS_FAMILY) && __IPHONE_OS_VERSION_MIN_REQUIRED >= 140000)
     494#define HAVE_AVPLAYER_SUPRESSES_AUDIO_RENDERING 1
     495#endif
     496
    493497#if (PLATFORM(MAC) && __MAC_OS_X_VERSION_MAX_ALLOWED < 101404)
    494498#define HAVE_VIDEO_PERFORMANCE_METRICS 1
  • trunk/Source/WebCore/ChangeLog

    r266842 r266844  
     12020-09-10  Jer Noble  <jer.noble@apple.com>
     2
     3        [Cocoa] PERF: Don't instantiate AVPlayer-based audio decoders or renderers if an element is initially muted.
     4        https://bugs.webkit.org/show_bug.cgi?id=216299
     5
     6        Reviewed by Eric Carlson.
     7
     8        When an AVPlayer is created, even if muted, it will still instantiate an audio decoder and renderer if the
     9        AVAsset in the current player item has an audio track. Ostensibly, this is so that an unmute operation is
     10        instantaneous, as it's merely applying a zero gain to the decoded audio. However for web content, there's
     11        many autoplaying, muted <video> elements which may never be un-muted before being destroyed.
     12
     13        Implement a policy where, if an AVPlayer is initially muted, we adopt AVFoundation SPI to forcibly prevent
     14        audio decoding and rendering until the first time the AVPlayer is unmuted. This means the first un-mute may
     15        not be instantaneous, as an audio decoder will have to be created and fed before any audio is rendered.
     16
     17        There's some incorrect caching of mute state at the MediaPlayer level; so MediaPlayer and MPPAVFoundationObjC
     18        can get their respective m_muted states out of sync. Make sure that HTMLMediaElement always sets muted state
     19        after creating a MediaPlayer and that, respectively, MPPAVFoundationObjC always queries it's parent MediaPlayer's
     20        mute state when it in turn is created.
     21
     22        * html/HTMLMediaElement.cpp:
     23        (WebCore::HTMLMediaElement::createMediaPlayer):
     24        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     25        (WebCore::MediaPlayerPrivateAVFoundationObjC::MediaPlayerPrivateAVFoundationObjC):
     26        (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayer):
     27        (WebCore::MediaPlayerPrivateAVFoundationObjC::setMuted):
     28
    1292020-09-10  Chris Dumez  <cdumez@apple.com>
    230
  • trunk/Source/WebCore/PAL/ChangeLog

    r266693 r266844  
     12020-09-10  Jer Noble  <jer.noble@apple.com>
     2
     3        [Cocoa] PERF: Don't instantiate AVPlayer-based audio decoders or renderers if an element is initially muted.
     4        https://bugs.webkit.org/show_bug.cgi?id=216299
     5
     6        Reviewed by Eric Carlson.
     7
     8        * pal/spi/cocoa/AVFoundationSPI.h:
     9
    1102020-09-06  Myles C. Maxfield  <mmaxfield@apple.com>
    211
  • trunk/Source/WebCore/PAL/pal/spi/cocoa/AVFoundationSPI.h

    r266312 r266844  
    7373#endif
    7474
     75#if HAVE(AVPLAYER_SUPRESSES_AUDIO_RENDERING)
     76@interface AVPlayer (AVPlayerSupressesAudioRendering)
     77@property (nonatomic, getter=_suppressesAudioRendering, setter=_setSuppressesAudioRendering:) BOOL suppressesAudioRendering;
     78@end
     79#endif
     80
    7581#if ENABLE(WIRELESS_PLAYBACK_TARGET) || PLATFORM(IOS_FAMILY)
    7682
  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r266752 r266844  
    65576557    m_player->setBufferingPolicy(m_bufferingPolicy);
    65586558    m_player->setPreferredDynamicRangeMode(preferredDynamicRangeMode(document().view()));
     6559    m_player->setMuted(effectiveMuted());
    65596560    schedulePlaybackControlsManagerUpdate();
    65606561
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r266410 r266844  
    433433    , m_cachedItemStatus(MediaPlayerAVPlayerItemStatusDoesNotExist)
    434434{
     435    m_muted = player->muted();
    435436}
    436437
     
    941942        m_muted = false;
    942943        [m_avPlayer.get() setMuted:m_muted];
     944
     945#if HAVE(AVPLAYER_SUPRESSES_AUDIO_RENDERING)
     946        m_avPlayer.get().suppressesAudioRendering = YES;
     947#endif
    943948    }
    944949
     
    13251330
    13261331    [m_avPlayer.get() setMuted:m_muted];
     1332#if HAVE(AVPLAYER_SUPRESSES_AUDIO_RENDERING)
     1333    if (!m_muted)
     1334        m_avPlayer.get().suppressesAudioRendering = NO;
     1335#endif
    13271336}
    13281337
Note: See TracChangeset for help on using the changeset viewer.