Changeset 182555 in webkit


Ignore:
Timestamp:
Apr 8, 2015 11:12:56 AM (9 years ago)
Author:
jer.noble@apple.com
Message:

[Mac][WebAudio] Update the AVAudioMix in the AudioSourceProviderAVFObjC when the list of enabled audio tracks change.
​https://bugs.webkit.org/show_bug.cgi?id=143332

Reviewed by Eric Carlson.

Some media assets (notably, mp3s) will not have an enabled audio track when the AVAsset is
first loaded, so the AVAudioMix will have no trackID in it's parameters. Whenever the list
of enabled tracks change, recreate the AVAudioMix with the new first enabled audio trackID.

To facilitate this, add a new setter to AudioSourceProviderAVFObjC taking an AVAssetTrack to
use with the AVAudioMix. Whenever this parameter changes, the AVAudioMix is destroyed and
recreated.

  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::mediaPlayerEngineUpdated): Drive-by fix: when the media

player switches engines, re-associate the audio source node with its provider.

  • platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
  • platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:

(WebCore::AudioSourceProviderAVFObjC::setPlayerItem): Only create the mix if there is a valid AVPlayerItem and AVAssetTrack
(WebCore::AudioSourceProviderAVFObjC::setAudioTrack): Ditto.
(WebCore::AudioSourceProviderAVFObjC::createMix): Don't iterate over the AVPlayerItem's tracks,

just use the one passed in through setAudioTrack().

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::cancelLoad): Clear the provider's track.
(WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem): Set the provider's track.
(WebCore::MediaPlayerPrivateAVFoundationObjC::tracksChanged): Ditto.
(WebCore::MediaPlayerPrivateAVFoundationObjC::audioSourceProvider): Ditto.

Location:
trunk/Source/WebCore
Files:
5 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r182551 r182555  
     12015-04-08  Jer Noble  <jer.noble@apple.com>
     2
     3        [Mac][WebAudio] Update the AVAudioMix in the AudioSourceProviderAVFObjC when the list of enabled audio tracks change.
     4        https://bugs.webkit.org/show_bug.cgi?id=143332
     5
     6        Reviewed by Eric Carlson.
     7
     8        Some media assets (notably, mp3s) will not have an enabled audio track when the AVAsset is
     9        first loaded, so the AVAudioMix will have no trackID in it's parameters. Whenever the list
     10        of enabled tracks change, recreate the AVAudioMix with the new first enabled audio trackID.
     11
     12        To facilitate this, add a new setter to AudioSourceProviderAVFObjC taking an AVAssetTrack to
     13        use with the AVAudioMix. Whenever this parameter changes, the AVAudioMix is destroyed and
     14        recreated.
     15
     16        * html/HTMLMediaElement.cpp:
     17        (WebCore::HTMLMediaElement::mediaPlayerEngineUpdated): Drive-by fix: when the media
     18            player switches engines, re-associate the audio source node with its provider.
     19        * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
     20        * platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:
     21        (WebCore::AudioSourceProviderAVFObjC::setPlayerItem): Only create the mix if there is a valid AVPlayerItem and AVAssetTrack
     22        (WebCore::AudioSourceProviderAVFObjC::setAudioTrack): Ditto.
     23        (WebCore::AudioSourceProviderAVFObjC::createMix): Don't iterate over the AVPlayerItem's tracks,
     24            just use the one passed in through setAudioTrack().
     25        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     26        (WebCore::MediaPlayerPrivateAVFoundationObjC::cancelLoad): Clear the provider's track.
     27        (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem): Set the provider's track.
     28        (WebCore::MediaPlayerPrivateAVFoundationObjC::tracksChanged): Ditto.
     29        (WebCore::MediaPlayerPrivateAVFoundationObjC::audioSourceProvider): Ditto.
     30
    1312015-04-08  Anders Carlsson  <andersca@apple.com>
    232

  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r182544 r182555  
    42744274    m_mediaSession->applyMediaPlayerRestrictions(*this);
    42754275
     4276#if ENABLE(WEB_AUDIO)
     4277    if (m_audioSourceNode && audioSourceProvider()) {
     4278        m_audioSourceNode->lock();
     4279        audioSourceProvider()->setClient(m_audioSourceNode);
     4280        m_audioSourceNode->unlock();
     4281    }
     4282#endif
     4283
    42764284#if PLATFORM(IOS)
    42774285    if (!m_player)

  • trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h

    r177733 r182555  
    3535#include <wtf/RetainPtr.h>
    3636
     37OBJC_CLASS AVAssetTrack;
    3738OBJC_CLASS AVPlayerItem;
    3839OBJC_CLASS AVMutableAudioMix;
    … …  
    5556
    5657    void setPlayerItem(AVPlayerItem *);
     58    void setAudioTrack(AVAssetTrack *);
    5759
    5860private:
    … …  
    7981
    8082    RetainPtr<AVPlayerItem> m_avPlayerItem;
     83    RetainPtr<AVAssetTrack> m_avAssetTrack;
    8184    RetainPtr<AVMutableAudioMix> m_avAudioMix;
    8285    RetainPtr<MTAudioProcessingTapRef> m_tap;

  • trunk/Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm

    r181368 r182555  
    3939#import <AVFoundation/AVMediaFormat.h>
    4040#import <AVFoundation/AVPlayerItem.h>
    41 #import <AVFoundation/AVPlayerItemTrack.h>
    4241#import <objc/runtime.h>
    4342#import <wtf/MainThread.h>
    … …  
    152151    m_avPlayerItem = avPlayerItem;
    153152
    154     if (m_client && m_avPlayerItem)
     153    if (m_client && m_avPlayerItem && m_avAssetTrack)
     154        createMix();
     155}
     156
     157void AudioSourceProviderAVFObjC::setAudioTrack(AVAssetTrack *avAssetTrack)
     158{
     159    if (m_avAssetTrack == avAssetTrack)
     160        return;
     161
     162    if (m_avAudioMix)
     163        destroyMix();
     164
     165    m_avAssetTrack = avAssetTrack;
     166
     167    if (m_client && m_avPlayerItem && m_avAssetTrack)
    155168        createMix();
    156169}
    … …  
    191204    [parameters setAudioTapProcessor:m_tap.get()];
    192205
    193     CMPersistentTrackID firstEnabledAudioTrackID = kCMPersistentTrackID_Invalid;
    194     NSArray* tracks = [m_avPlayerItem tracks];
    195     for (AVPlayerItemTrack* track in tracks) {
    196         if ([track.assetTrack hasMediaCharacteristic:AVMediaCharacteristicAudible] && track.enabled) {
    197             firstEnabledAudioTrackID = track.assetTrack.trackID;
    198             break;
    199         }
    200     }
    201     [parameters setTrackID:firstEnabledAudioTrackID];
     206    CMPersistentTrackID trackID = m_avAssetTrack.get().trackID;
     207    [parameters setTrackID:trackID];
    202208   
    203209    [m_avAudioMix setInputParameters:@[parameters.get()]];

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r182486 r182555  
    350350static NSArray *assetTrackMetadataKeyNames();
    351351static NSArray *playerKVOProperties();
     352static AVAssetTrack* firstEnabledTrack(NSArray* tracks);
    352353
    353354#if !LOG_DISABLED
    … …  
    567568
    568569#if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
    569     if (m_provider)
     570    if (m_provider) {
    570571        m_provider->setPlayerItem(nullptr);
     572        m_provider->setAudioTrack(nullptr);
     573    }
    571574#endif
    572575
    … …  
    9991002
    10001003#if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
    1001     if (m_provider)
     1004    if (m_provider) {
    10021005        m_provider->setPlayerItem(m_avPlayerItem.get());
     1006        m_provider->setAudioTrack(firstEnabledTrack([m_avAsset tracksWithMediaCharacteristic:AVMediaCharacteristicAudible]));
     1007    }
    10031008#endif
    10041009
    … …  
    19181923        characteristicsChanged();
    19191924
     1925#if ENABLE(WEB_AUDIO) && USE(MEDIATOOLBOX)
     1926    if (m_provider)
     1927        m_provider->setAudioTrack(firstEnabledTrack([m_avAsset tracksWithMediaCharacteristic:AVMediaCharacteristicAudible]));
     1928#endif
     1929
    19201930    setDelayCharacteristicsChangedNotification(false);
    19211931}
    … …  
    21282138AudioSourceProvider* MediaPlayerPrivateAVFoundationObjC::audioSourceProvider()
    21292139{
    2130     if (!m_provider)
     2140    if (!m_provider) {
    21312141        m_provider = AudioSourceProviderAVFObjC::create(m_avPlayerItem.get());
     2142        m_provider->setAudioTrack(firstEnabledTrack([m_avAsset tracksWithMediaCharacteristic:AVMediaCharacteristicAudible]));
     2143    }
     2144
    21322145    return m_provider.get();
    21332146}
Note: See TracChangeset for help on using the changeset viewer.