Changeset 206518 in webkit


Ignore:
Timestamp:
Sep 28, 2016 7:52:48 AM (8 years ago)
Author:
jer.noble@apple.com
Message:

[MSE][Mac] In SourceBufferPrivateAVFObjC::abort(), support reseting parser to the last appended initialization segment.
​https://bugs.webkit.org/show_bug.cgi?id=135164

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/media-source/media-source-abort-resets-parser.html

Use the -[AVStreamDataParser appendStreamData:withFlags:] to implement "resetting" the parser. In this case,
the parser isn't explicitly reset during resetParserState(), but rather a flag is set so that the next append
signals a data discontinuity, and the parser is reset at that point.

Because a previous append operation may be in-flight during this abort(), care must be taken to invalidate any
operations which may have already started on a background thread. So SourceBufferPrivateAVFObjC will use a
separate WeakPtrFactory for its append operations, will invalidate any outstanding WeakPtrs during an abort(),
and will block until the previous append() operation completes.

This will require the WebAVStreamDataParserListener object to occasionally have it's WeakPtr pointing back to the
SourceBufferPrivateAVFObjC to be reset after an abort(), so make that ivar an @property. Rather than passing a
RetainPtr to itself in all the callbacks it handles, the WebAVStreamDataParserListener can just pass in a copy
of its own WeakPtr (which may be invalidated during an abort()).

Break the distinct operations of "abort()" and "resetParserState()" into their own methods in SourceBufferPrivate
and all its subclasses.

  • Modules/mediasource/SourceBuffer.cpp:

(WebCore::SourceBuffer::resetParserState):
(WebCore::SourceBuffer::abortIfUpdating):

  • platform/graphics/SourceBufferPrivate.h:
  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:

(-[WebAVStreamDataParserListener streamDataParser:didParseStreamDataAsAsset:]):
(-[WebAVStreamDataParserListener streamDataParser:didParseStreamDataAsAsset:withDiscontinuity:]):
(-[WebAVStreamDataParserListener streamDataParser:didFailToParseStreamDataWithError:]):
(-[WebAVStreamDataParserListener streamDataParser:didProvideMediaData:forTrackID:mediaType:flags:]):
(-[WebAVStreamDataParserListener streamDataParser:didReachEndOfTrackWithTrackID:mediaType:]):
(-[WebAVStreamDataParserListener streamDataParserWillProvideContentKeyRequestInitializationData:forTrackID:]):
(-[WebAVStreamDataParserListener streamDataParser:didProvideContentKeyRequestInitializationData:forTrackID:]):
(WebCore::SourceBufferPrivateAVFObjC::SourceBufferPrivateAVFObjC):
(WebCore::SourceBufferPrivateAVFObjC::append):
(WebCore::SourceBufferPrivateAVFObjC::abort):
(WebCore::SourceBufferPrivateAVFObjC::resetParserState):
(-[WebAVStreamDataParserListener initWithParser:parent:]): Deleted.

  • platform/graphics/gstreamer/SourceBufferPrivateGStreamer.cpp:

(WebCore::SourceBufferPrivateGStreamer::resetParserState):

  • platform/graphics/gstreamer/SourceBufferPrivateGStreamer.h:
  • platform/mock/mediasource/MockSourceBufferPrivate.cpp:

(WebCore::MockSourceBufferPrivate::resetParserState):

  • platform/mock/mediasource/MockSourceBufferPrivate.h:
  • platform/spi/mac/AVFoundationSPI.h:

LayoutTests:

  • media/media-source/media-source-abort-resets-parser-expected.txt: Added.
  • media/media-source/media-source-abort-resets-parser.html: Added.
Location:
trunk
Files:
2 added
11 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r206514 r206518  
     12016-09-28  Jer Noble  <jer.noble@apple.com>
     2
     3        [MSE][Mac] In SourceBufferPrivateAVFObjC::abort(), support reseting parser to the last appended initialization segment.
     4        https://bugs.webkit.org/show_bug.cgi?id=135164
     5
     6        Reviewed by Eric Carlson.
     7
     8        * media/media-source/media-source-abort-resets-parser-expected.txt: Added.
     9        * media/media-source/media-source-abort-resets-parser.html: Added.
     10
    1112016-09-28  Alejandro G. Castro  <alex@igalia.com>
    212

  • trunk/Source/WebCore/ChangeLog

    r206517 r206518  
     12016-09-28  Jer Noble  <jer.noble@apple.com>
     2
     3        [MSE][Mac] In SourceBufferPrivateAVFObjC::abort(), support reseting parser to the last appended initialization segment.
     4        https://bugs.webkit.org/show_bug.cgi?id=135164
     5
     6        Reviewed by Eric Carlson.
     7
     8        Test: media/media-source/media-source-abort-resets-parser.html
     9
     10        Use the -[AVStreamDataParser appendStreamData:withFlags:] to implement "resetting" the parser. In this case,
     11        the parser isn't explicitly reset during resetParserState(), but rather a flag is set so that the next append
     12        signals a data discontinuity, and the parser is reset at that point.
     13
     14        Because a previous append operation may be in-flight during this abort(), care must be taken to invalidate any
     15        operations which may have already started on a background thread. So SourceBufferPrivateAVFObjC will use a
     16        separate WeakPtrFactory for its append operations, will invalidate any outstanding WeakPtrs during an abort(),
     17        and will block until the previous append() operation completes.
     18
     19        This will require the WebAVStreamDataParserListener object to occasionally have it's WeakPtr pointing back to the
     20        SourceBufferPrivateAVFObjC to be reset after an abort(), so make that ivar an @property. Rather than passing a
     21        RetainPtr to itself in all the callbacks it handles, the WebAVStreamDataParserListener can just pass in a copy
     22        of its own WeakPtr (which may be invalidated during an abort()).
     23
     24        Break the distinct operations of "abort()" and "resetParserState()" into their own methods in SourceBufferPrivate
     25        and all its subclasses.
     26
     27        * Modules/mediasource/SourceBuffer.cpp:
     28        (WebCore::SourceBuffer::resetParserState):
     29        (WebCore::SourceBuffer::abortIfUpdating):
     30        * platform/graphics/SourceBufferPrivate.h:
     31        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
     32        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
     33        (-[WebAVStreamDataParserListener streamDataParser:didParseStreamDataAsAsset:]):
     34        (-[WebAVStreamDataParserListener streamDataParser:didParseStreamDataAsAsset:withDiscontinuity:]):
     35        (-[WebAVStreamDataParserListener streamDataParser:didFailToParseStreamDataWithError:]):
     36        (-[WebAVStreamDataParserListener streamDataParser:didProvideMediaData:forTrackID:mediaType:flags:]):
     37        (-[WebAVStreamDataParserListener streamDataParser:didReachEndOfTrackWithTrackID:mediaType:]):
     38        (-[WebAVStreamDataParserListener streamDataParserWillProvideContentKeyRequestInitializationData:forTrackID:]):
     39        (-[WebAVStreamDataParserListener streamDataParser:didProvideContentKeyRequestInitializationData:forTrackID:]):
     40        (WebCore::SourceBufferPrivateAVFObjC::SourceBufferPrivateAVFObjC):
     41        (WebCore::SourceBufferPrivateAVFObjC::append):
     42        (WebCore::SourceBufferPrivateAVFObjC::abort):
     43        (WebCore::SourceBufferPrivateAVFObjC::resetParserState):
     44        (-[WebAVStreamDataParserListener initWithParser:parent:]): Deleted.
     45        * platform/graphics/gstreamer/SourceBufferPrivateGStreamer.cpp:
     46        (WebCore::SourceBufferPrivateGStreamer::resetParserState):
     47        * platform/graphics/gstreamer/SourceBufferPrivateGStreamer.h:
     48        * platform/mock/mediasource/MockSourceBufferPrivate.cpp:
     49        (WebCore::MockSourceBufferPrivate::resetParserState):
     50        * platform/mock/mediasource/MockSourceBufferPrivate.h:
     51        * platform/spi/mac/AVFoundationSPI.h:
     52
    1532016-09-28  Michael Catanzaro  <mcatanzaro@igalia.com>
    254

  • trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp

    r206300 r206518  
    279279    m_appendState = WaitingForSegment;
    280280
    281     m_private->abort();
     281    m_private->resetParserState();
    282282}
    283283
    … …  
    384384    m_appendBufferTimer.stop();
    385385    m_pendingAppendData.clear();
     386    m_private->abort();
    386387
    387388    // 4.2. Set the updating attribute to false.

  • trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h

    r206301 r206518  
    5353    virtual void append(const unsigned char* data, unsigned length) = 0;
    5454    virtual void abort() = 0;
     55    virtual void resetParserState() = 0;
    5556    virtual void removedFromMediaSource() = 0;
    5657

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h

    r206025 r206518  
    3030
    3131#include "SourceBufferPrivate.h"
     32#include <dispatch/group.h>
    3233#include <dispatch/semaphore.h>
    3334#include <wtf/Deque.h>
    … …  
    116117    void append(const unsigned char* data, unsigned length) override;
    117118    void abort() override;
     119    void resetParserState() override;
    118120    void removedFromMediaSource() override;
    119121    MediaPlayer::ReadyState readyState() const override;
    … …  
    140142
    141143    WeakPtrFactory<SourceBufferPrivateAVFObjC> m_weakFactory;
     144    WeakPtrFactory<SourceBufferPrivateAVFObjC> m_appendWeakFactory;
    142145
    143146    RetainPtr<AVStreamDataParser> m_parser;
    … …  
    149152    RetainPtr<NSError> m_hdcpError;
    150153    OSObjectPtr<dispatch_semaphore_t> m_hasSessionSemaphore;
     154    OSObjectPtr<dispatch_group_t> m_isAppendingGroup;
    151155
    152156    MediaSourcePrivateAVFObjC* m_mediaSource;
    … …  
    157161    FloatSize m_currentSize;
    158162    bool m_parsingSucceeded;
     163    bool m_parserStateWasReset { false };
    159164    int m_enabledVideoTrackID;
    160165    int m_protectedTrackID;

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm

    r206025 r206518  
    132132    AVStreamDataParser* _parser;
    133133}
     134@property (assign) WeakPtr<WebCore::SourceBufferPrivateAVFObjC> parent;
    134135- (id)initWithParser:(AVStreamDataParser*)parser parent:(WeakPtr<WebCore::SourceBufferPrivateAVFObjC>)parent;
    135136@end
    … …  
    149150}
    150151
     152@synthesize parent=_parent;
     153
    151154- (void)dealloc
    152155{
    … …  
    167170#endif
    168171    ASSERT(streamDataParser == _parser);
    169     RetainPtr<WebAVStreamDataParserListener> protectedSelf = self;
    170172
    171173    RetainPtr<AVAsset*> protectedAsset = asset;
    172     callOnMainThread([protectedSelf = WTFMove(protectedSelf), protectedAsset = WTFMove(protectedAsset)] {
    173         if (protectedSelf->_parent)
    174             protectedSelf->_parent->didParseStreamDataAsAsset(protectedAsset.get());
     174    callOnMainThread([parent = _parent, protectedAsset = WTFMove(protectedAsset)] {
     175        if (parent)
     176            parent->didParseStreamDataAsAsset(protectedAsset.get());
    175177    });
    176178}
    … …  
    183185#endif
    184186    ASSERT(streamDataParser == _parser);
    185     RetainPtr<WebAVStreamDataParserListener> protectedSelf = self;
    186187
    187188    RetainPtr<AVAsset*> protectedAsset = asset;
    188     callOnMainThread([protectedSelf = WTFMove(protectedSelf), protectedAsset = WTFMove(protectedAsset)] {
    189         if (protectedSelf->_parent)
    190             protectedSelf->_parent->didParseStreamDataAsAsset(protectedAsset.get());
     189    callOnMainThread([parent = _parent, protectedAsset = WTFMove(protectedAsset)] {
     190        if (parent)
     191            parent->didParseStreamDataAsAsset(protectedAsset.get());
    191192    });
    192193}
    … …  
    198199#endif
    199200    ASSERT(streamDataParser == _parser);
    200     RetainPtr<WebAVStreamDataParserListener> protectedSelf = self;
    201201
    202202    RetainPtr<NSError> protectedError = error;
    203     callOnMainThread([protectedSelf = WTFMove(protectedSelf), protectedError = WTFMove(protectedError)] {
    204         if (protectedSelf->_parent)
    205             protectedSelf->_parent->didFailToParseStreamDataWithError(protectedError.get());
     203    callOnMainThread([parent = _parent, protectedError = WTFMove(protectedError)] {
     204        if (parent)
     205            parent->didFailToParseStreamDataWithError(protectedError.get());
    206206    });
    207207}
    … …  
    213213#endif
    214214    ASSERT(streamDataParser == _parser);
    215     RetainPtr<WebAVStreamDataParserListener> protectedSelf = self;
    216215
    217216    RetainPtr<CMSampleBufferRef> protectedSample = sample;
    218217    String mediaType = nsMediaType;
    219     callOnMainThread([protectedSelf = WTFMove(protectedSelf), protectedSample = WTFMove(protectedSample), trackID, mediaType, flags] {
    220         if (protectedSelf->_parent)
    221             protectedSelf->_parent->didProvideMediaDataForTrackID(trackID, protectedSample.get(), mediaType, flags);
     218    callOnMainThread([parent = _parent, protectedSample = WTFMove(protectedSample), trackID, mediaType, flags] {
     219        if (parent)
     220            parent->didProvideMediaDataForTrackID(trackID, protectedSample.get(), mediaType, flags);
    222221    });
    223222}
    … …  
    229228#endif
    230229    ASSERT(streamDataParser == _parser);
    231     RetainPtr<WebAVStreamDataParserListener> protectedSelf = self;
    232230
    233231    String mediaType = nsMediaType;
    234     callOnMainThread([protectedSelf = WTFMove(protectedSelf), trackID, mediaType] {
    235         if (protectedSelf->_parent)
    236             protectedSelf->_parent->didReachEndOfTrackWithTrackID(trackID, mediaType);
     232    callOnMainThread([parent = _parent, trackID, mediaType] {
     233        if (parent)
     234            parent->didReachEndOfTrackWithTrackID(trackID, mediaType);
    237235    });
    238236}
    … …  
    247245    // We must call synchronously to the main thread, as the AVStreamSession must be associated
    248246    // with the streamDataParser before the delegate method returns.
    249     RetainPtr<WebAVStreamDataParserListener> strongSelf = self;
    250     dispatch_sync(dispatch_get_main_queue(), [strongSelf, trackID]() {
    251         if (strongSelf->_parent)
    252             strongSelf->_parent->willProvideContentKeyRequestInitializationDataForTrackID(trackID);
     247    dispatch_sync(dispatch_get_main_queue(), [parent = _parent, trackID]() {
     248        if (parent)
     249            parent->willProvideContentKeyRequestInitializationDataForTrackID(trackID);
    253250    });
    254251}
    … …  
    260257#endif
    261258    ASSERT(streamDataParser == _parser);
    262     RetainPtr<WebAVStreamDataParserListener> protectedSelf = self;
    263259
    264260    OSObjectPtr<dispatch_semaphore_t> hasSessionSemaphore = adoptOSObject(dispatch_semaphore_create(0));
    265     callOnMainThread([protectedSelf = WTFMove(protectedSelf), protectedInitData = RetainPtr<NSData>(initData), trackID, hasSessionSemaphore] {
    266         if (protectedSelf->_parent)
    267             protectedSelf->_parent->didProvideContentKeyRequestInitializationDataForTrackID(protectedInitData.get(), trackID, hasSessionSemaphore);
     261    callOnMainThread([parent = _parent, protectedInitData = RetainPtr<NSData>(initData), trackID, hasSessionSemaphore] {
     262        if (parent)
     263            parent->didProvideContentKeyRequestInitializationDataForTrackID(protectedInitData.get(), trackID, hasSessionSemaphore);
    268264    });
    269265    dispatch_semaphore_wait(hasSessionSemaphore.get(), DISPATCH_TIME_FOREVER);
    … …  
    462458SourceBufferPrivateAVFObjC::SourceBufferPrivateAVFObjC(MediaSourcePrivateAVFObjC* parent)
    463459    : m_weakFactory(this)
     460    , m_appendWeakFactory(this)
    464461    , m_parser(adoptNS([allocAVStreamDataParserInstance() init]))
    465462    , m_delegate(adoptNS([[WebAVStreamDataParserListener alloc] initWithParser:m_parser.get() parent:createWeakPtr()]))
    466463    , m_errorListener(adoptNS([[WebAVSampleBufferErrorListener alloc] initWithParent:this]))
     464    , m_isAppendingGroup(adoptOSObject(dispatch_group_create()))
    467465    , m_mediaSource(parent)
    468466    , m_client(0)
    … …  
    642640
    643641    RetainPtr<NSData> nsData = adoptNS([[NSData alloc] initWithBytes:data length:length]);
    644     WeakPtr<SourceBufferPrivateAVFObjC> weakThis = createWeakPtr();
     642    WeakPtr<SourceBufferPrivateAVFObjC> weakThis = m_appendWeakFactory.createWeakPtr();
    645643    RetainPtr<AVStreamDataParser> parser = m_parser;
    646644    RetainPtr<WebAVStreamDataParserListener> delegate = m_delegate;
    647645
    648646    m_parsingSucceeded = true;
    649 
    650     dispatch_async(globalDataParserQueue(), [nsData, weakThis, parser, delegate] {
    651 
    652         [parser appendStreamData:nsData.get()];
     647    dispatch_group_enter(m_isAppendingGroup.get());
     648
     649    dispatch_async(globalDataParserQueue(), [nsData, weakThis, parser, delegate, isAppendingGroup = m_isAppendingGroup, parserStateWasReset = m_parserStateWasReset] {
     650        if (parserStateWasReset)
     651            [parser appendStreamData:nsData.get() withFlags:AVStreamDataParserStreamDataDiscontinuity];
     652        else
     653            [parser appendStreamData:nsData.get()];
    653654
    654655        callOnMainThread([weakThis] {
    … …  
    656657                weakThis->appendCompleted();
    657658        });
     659        dispatch_group_leave(isAppendingGroup.get());
    658660    });
     661    m_parserStateWasReset = false;
    659662}
    660663
    … …  
    670673void SourceBufferPrivateAVFObjC::abort()
    671674{
    672     // The parser does not have a mechanism for resetting to a clean state, so destroy and re-create it.
    673     // FIXME(135164): Support resetting parser to the last appended initialization segment.
    674     destroyParser();
    675 
    676     m_parser = adoptNS([allocAVStreamDataParserInstance() init]);
    677     m_delegate = adoptNS([[WebAVStreamDataParserListener alloc] initWithParser:m_parser.get() parent:createWeakPtr()]);
     675    // The parsing queue may be blocked waiting for the main thread to provide it a AVStreamSession. We
     676    // were asked to abort, and that cancels all outstanding append operations. Without cancelling this
     677    // semaphore, the m_isAppendingGroup wait operation will deadlock.
     678    if (m_hasSessionSemaphore)
     679        dispatch_semaphore_signal(m_hasSessionSemaphore.get());
     680    dispatch_group_wait(m_isAppendingGroup.get(), DISPATCH_TIME_FOREVER);
     681    m_appendWeakFactory.revokeAll();
     682    m_delegate.get().parent = m_appendWeakFactory.createWeakPtr();
     683}
     684
     685void SourceBufferPrivateAVFObjC::resetParserState()
     686{
     687    m_parserStateWasReset = true;
    678688}
    679689

  • trunk/Source/WebCore/platform/graphics/gstreamer/SourceBufferPrivateGStreamer.cpp

    r191174 r206518  
    7676}
    7777
     78void SourceBufferPrivateGStreamer::resetParserState()
     79{
     80    notImplemented();
     81}
     82
    7883void SourceBufferPrivateGStreamer::removedFromMediaSource()
    7984{

  • trunk/Source/WebCore/platform/graphics/gstreamer/SourceBufferPrivateGStreamer.h

    r191174 r206518  
    5151    virtual void append(const unsigned char* data, unsigned length);
    5252    virtual void abort();
     53    virtual void resetParserState();
    5354    virtual void removedFromMediaSource();
    5455

  • trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp

    r206306 r206518  
    204204}
    205205
     206void MockSourceBufferPrivate::resetParserState()
     207{
     208}
     209
    206210void MockSourceBufferPrivate::removedFromMediaSource()
    207211{

  • trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h

    r206300 r206518  
    6464    void append(const unsigned char* data, unsigned length) override;
    6565    void abort() override;
     66    void resetParserState() override;
    6667    void removedFromMediaSource() override;
    6768    MediaPlayer::ReadyState readyState() const override;

  • trunk/Source/WebCore/platform/spi/mac/AVFoundationSPI.h

    r202434 r206518  
    117117};
    118118
     119typedef NS_ENUM(NSUInteger, AVStreamDataParserStreamDataFlags) {
     120    AVStreamDataParserStreamDataDiscontinuity = 1 << 0,
     121};
     122
    119123@interface AVStreamDataParser : NSObject
    120124- (void)setDelegate:(nullable id<AVStreamDataParserOutputHandling>)delegate;
    121125- (void)appendStreamData:(NSData *)data;
     126- (void)appendStreamData:(NSData *)data withFlags:(AVStreamDataParserStreamDataFlags)flags;
    122127- (void)setShouldProvideMediaData:(BOOL)shouldProvideMediaData forTrackID:(CMPersistentTrackID)trackID;
    123128- (BOOL)shouldProvideMediaDataForTrackID:(CMPersistentTrackID)trackID;
Note: See TracChangeset for help on using the changeset viewer.