Changeset 279542 in webkit


Ignore:
Timestamp:
Jul 3, 2021 12:01:52 AM (13 months ago)
Author:
Jean-Yves Avenard
Message:

SourceBuffer.abort() doesn't go back to state WAITING_FOR_SEGMENT properly
​https://bugs.webkit.org/show_bug.cgi?id=227559
<rdar://problem/79996056>

Reviewed by Eric Carlson.

Source/WebCore:

Per spec, calling sourcebuffer.abort method should allow you to add a new segment
immediately after, as abort moves the append state back to WAITING_FOR_SEGMENT.
A segment can be either an init segment or a media segment.
We used to discard all further content until an init segment was encountered.
This could be problematic due to the typical use case of abort:
1- Seek to a location
2- Append a partial media segment long enough to finish seeking and display the

new content at the new position.

If multiple seeks were done in rapid succession, abort() is called right after
before starting the new seek so that we can add the new segment, regardless of what
was appended before.
YouTube applies a workaround for Safari where it will always append an init segment
after calling abort, this is different to what they do with Firefox (and likely Chrome).
To be able to resume after appending a partial media segment we must ensure that the
SourceBufferParser is always left in a sane context, and not be interrupted at some
random points. The abort() call used to interrupt the buffer parsing on the fly and
then reset things which would require a new init segment to restart.
Instead we always fully parse the pending buffer received befofe the call to abort.
The SourceBufferParserAVFObjC could already properly deal with discontinuity unlike
SourceBufferParserWebM.
To ensure that buffers sent after the call to abort() are only ever processed once
the pending ones have been parsed, and in order to avoid having blocking calls
we play with the order in which tasks are scheduled.

Fly-by fixes:

  • The SourceBufferParser handle two types of parser: SourceBufferParser and the

platform specific AVStreamDataParser. Rename the accessor method to more clearly
differentate which parser we are dealing with.

  • The SourceBufferParserWebM and SourceBufferPrivateAVFObjC used different task dispatching

mechanisms. We make them both share the same one now found in the base class.

Tests: media/media-source/media-mp4-h264-partial-abort.html

media/media-source/media-webm-opus-partial-abort.html

  • platform/graphics/avfoundation/objc/CDMSessionAVContentKeySession.mm:

(WebCore::CDMSessionAVContentKeySession::~CDMSessionAVContentKeySession): rename method.

  • platform/graphics/avfoundation/objc/CDMSessionAVStreamSession.mm:

(WebCore::CDMSessionAVStreamSession::~CDMSessionAVStreamSession): rename method.
(WebCore::CDMSessionAVStreamSession::update): rename method.

  • platform/graphics/avfoundation/objc/CDMSessionMediaSourceAVFObjC.mm: rename method.

(WebCore::CDMSessionMediaSourceAVFObjC::addSourceBuffer): rename method.
(WebCore::CDMSessionMediaSourceAVFObjC::removeSourceBuffer): rename method.

  • platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.h: rename method, remove

now unused member.

  • platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.mm:

(WebCore::SourceBufferPrivateAVFObjC::removeCodedFrames): Postpone call to ensure
we are running the remove frame algorithm once all pending frames have been processed.
(WebCore::SourceBufferParserAVFObjC::resetParserState): Remove use of m_discardSamplesUntilNextInitializationSegment.
(WebCore::SourceBufferParserAVFObjC::didParseStreamDataAsAsset): Change to base dispatch method.
(WebCore::SourceBufferParserAVFObjC::didFailToParseStreamDataWithError): Change to base dispatch method.
(WebCore::SourceBufferParserAVFObjC::didProvideMediaDataForTrackID): Change to base dispatch method.

  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h: Rename methods. Remove no longer used

members.

  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:

(WebCore::SourceBufferPrivateAVFObjC::~SourceBufferPrivateAVFObjC):
(WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData): No longer use a Cancellable task
as we never cancel it anymore.
(WebCore::SourceBufferPrivateAVFObjC::append): re-schedule the append task immediately to ensure
that processed packets flushed on the parser queue during abort are handled in the right order on
the main thread.
(WebCore::SourceBufferPrivateAVFObjC::appendCompleted): Check that abort wasn't called since append started
to ensure that no updateend event is incorrectly fired twice.
(WebCore::SourceBufferPrivateAVFObjC::abort): Abort is now a no-op that only set the m_abortCalled member.
(WebCore::SourceBufferPrivateAVFObjC::resetParserState): Change the order of operations so that the
SourceBufferParser is only reset after it has finished processing its data.
(WebCore::SourceBufferPrivateAVFObjC::destroyStreamDataParser): Use rename method.
(WebCore::SourceBufferPrivateAVFObjC::removedFromMediaSource): Use rename method.
(WebCore::SourceBufferPrivateAVFObjC::streamDataParser const): Renamed method from "parser"
(WebCore::SourceBufferPrivateAVFObjC::attemptToDecrypt): Use renamed method.

  • platform/graphics/cocoa/SourceBufferParser.cpp:

(WebCore::callOnMainThreadCallback): Move dispatch method from SourceBufferParserWebM
(WebCore::SourceBufferParser::setCallOnClientThreadCallback):
(WebCore::SourceBufferParser::SourceBufferParser):

  • platform/graphics/cocoa/SourceBufferParser.h:
  • platform/graphics/cocoa/SourceBufferParserWebM.cpp:

(WebCore::SourceBufferParserWebM::SourceBufferParserWebM):
(WebCore::SourceBufferParserWebM::resetParserState): Don't clear the data set by parsing
the previous init segment. Set the parsing state to waiting for a new segment if an
init segment has been fully parsed.
(WebCore::SourceBufferParserWebM::OnElementEnd):
(WebCore::SourceBufferParserWebM::OnEbml):
(WebCore::SourceBufferParserWebM::VideoTrackData::reset):
(WebCore::SourceBufferParserWebM::VideoTrackData::consumeFrameData):
(WebCore::SourceBufferParserWebM::AudioTrackData::reset):

  • platform/graphics/cocoa/SourceBufferParserWebM.h:

LayoutTests:

  • media/media-source/content/test-fragmented-video-manifest.json: Added.
  • media/media-source/content/test-fragmented-video.mp4: Added.
  • media/media-source/media-mp4-h264-partial-abort-expected.txt: Added.
  • media/media-source/media-mp4-h264-partial-abort.html: Added.
  • media/media-source/media-webm-opus-partial-abort-expected.txt: Added.
  • media/media-source/media-webm-opus-partial-abort.html: Added.
  • media/media-source/media-webm-opus-partial.html: fix title.
  • media/media-source/media-webm-vorbis-partial.html: fix title.
  • platform/mac/TestExpectations:
Location:
trunk
Files:
6 added
16 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r279522 r279542  
     12021-07-03  Jean-Yves Avenard  <jya@apple.com>
     2
     3        SourceBuffer.abort() doesn't go back to state WAITING_FOR_SEGMENT properly
     4        https://bugs.webkit.org/show_bug.cgi?id=227559
     5        <rdar://problem/79996056>
     6
     7        Reviewed by Eric Carlson.
     8
     9        * media/media-source/content/test-fragmented-video-manifest.json: Added.
     10        * media/media-source/content/test-fragmented-video.mp4: Added.
     11        * media/media-source/media-mp4-h264-partial-abort-expected.txt: Added.
     12        * media/media-source/media-mp4-h264-partial-abort.html: Added.
     13        * media/media-source/media-webm-opus-partial-abort-expected.txt: Added.
     14        * media/media-source/media-webm-opus-partial-abort.html: Added.
     15        * media/media-source/media-webm-opus-partial.html: fix title.
     16        * media/media-source/media-webm-vorbis-partial.html: fix title.
     17        * platform/mac/TestExpectations:
     18
    1192021-07-02  Eric Hutchison  <ehutchison@apple.com>
    220

  • trunk/LayoutTests/media/media-source/media-webm-opus-partial.html

    r279492 r279542  
    22<html>
    33<head>
    4     <title>media-opus-partial</title>
     4    <title>media-webm-opus-partial</title>
    55    <script src="../../media/media-source/media-source-loader.js"></script>
    66    <script src="../../media/video-test.js"></script>

  • trunk/LayoutTests/media/media-source/media-webm-vorbis-partial.html

    r279492 r279542  
    22<html>
    33<head>
    4     <title>media-vorbis-partial</title>
     4    <title>media-webm-vorbis-partial</title>
    55    <script src="../../media/media-source/media-source-loader.js"></script>
    66    <script src="../../media/video-test.js"></script>

  • trunk/LayoutTests/platform/mac/TestExpectations

    r279516 r279542  
    17691769# These tests require macOS Monterey.
    17701770[ Catalina Mojave BigSur ] media/media-source/media-webm-vorbis-partial.html [ Skip ]
     1771[ Catalina Mojave BigSur ] media/media-source/media-source-webm-vorbis-partial.html [ Skip ]
    17711772[ Catalina Mojave BigSur ] media/media-source/media-webm-opus-partial.html [ Skip ]
     1773[ Catalina Mojave BigSur ] media/media-source/media-webm-opus-partial-abort.html [ Skip ]
    17721774
    17731775webkit.org/b/214422 imported/w3c/web-platform-tests/webaudio/the-audio-api/the-audiocontext-interface/suspend-after-construct.html [ Pass Failure ]
    … …  
    21642166
    21652167webkit.org/b/222573 media/media-fullscreen-pause-inline.html [ Pass Failure ]
    2166 
    2167 webkit.org/b/222495 media/media-source/media-source-webm-vorbis-partial.html [ Failure ]
    21682168
    21692169webkit.org/b/222692 inspector/page/empty-or-missing-resources.html [ Pass Timeout ]

  • trunk/Source/WebCore/ChangeLog

    r279530 r279542  
     12021-07-03  Jean-Yves Avenard  <jya@apple.com>
     2
     3        SourceBuffer.abort() doesn't go back to state WAITING_FOR_SEGMENT properly
     4        https://bugs.webkit.org/show_bug.cgi?id=227559
     5        <rdar://problem/79996056>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Per spec, calling sourcebuffer.abort method should allow you to add a new segment
     10        immediately after, as abort moves the append state back to WAITING_FOR_SEGMENT.
     11        A segment can be either an init segment or a media segment.
     12        We used to discard all further content until an init segment was encountered.
     13        This could be problematic due to the typical use case of abort:
     14        1- Seek to a location
     15        2- Append a partial media segment long enough to finish seeking and display the
     16         new content at the new position.
     17        If multiple seeks were done in rapid succession, abort() is called right after
     18        before starting the new seek so that we can add the new segment, regardless of what
     19        was appended before.
     20        YouTube applies a workaround for Safari where it will always append an init segment
     21        after calling abort, this is different to what they do with Firefox (and likely Chrome).
     22        To be able to resume after appending a partial media segment we must ensure that the
     23        SourceBufferParser is always left in a sane context, and not be interrupted at some
     24        random points. The abort() call used to interrupt the buffer parsing on the fly and
     25        then reset things which would require a new init segment to restart.
     26        Instead we always fully parse the pending buffer received befofe the call to abort.
     27        The SourceBufferParserAVFObjC could already properly deal with discontinuity unlike
     28        SourceBufferParserWebM.
     29        To ensure that buffers sent after the call to abort() are only ever processed once
     30        the pending ones have been parsed, and in order to avoid having blocking calls
     31        we play with the order in which tasks are scheduled.
     32
     33        Fly-by fixes:
     34        - The SourceBufferParser handle two types of parser: SourceBufferParser and the
     35        platform specific AVStreamDataParser. Rename the accessor method to more clearly
     36        differentate which parser we are dealing with.
     37        - The SourceBufferParserWebM and SourceBufferPrivateAVFObjC used different task dispatching
     38        mechanisms. We make them both share the same one now found in the base class.
     39
     40        Tests: media/media-source/media-mp4-h264-partial-abort.html
     41               media/media-source/media-webm-opus-partial-abort.html
     42
     43        * platform/graphics/avfoundation/objc/CDMSessionAVContentKeySession.mm:
     44        (WebCore::CDMSessionAVContentKeySession::~CDMSessionAVContentKeySession): rename method.
     45        * platform/graphics/avfoundation/objc/CDMSessionAVStreamSession.mm:
     46        (WebCore::CDMSessionAVStreamSession::~CDMSessionAVStreamSession): rename method.
     47        (WebCore::CDMSessionAVStreamSession::update): rename method.
     48        * platform/graphics/avfoundation/objc/CDMSessionMediaSourceAVFObjC.mm: rename method.
     49        (WebCore::CDMSessionMediaSourceAVFObjC::addSourceBuffer): rename method.
     50        (WebCore::CDMSessionMediaSourceAVFObjC::removeSourceBuffer): rename method.
     51        * platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.h: rename method, remove
     52        now unused member.
     53        * platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.mm:
     54        (WebCore::SourceBufferPrivateAVFObjC::removeCodedFrames): Postpone call to ensure
     55        we are running the remove frame algorithm once all pending frames have been processed.
     56        (WebCore::SourceBufferParserAVFObjC::resetParserState): Remove use of m_discardSamplesUntilNextInitializationSegment.
     57        (WebCore::SourceBufferParserAVFObjC::didParseStreamDataAsAsset): Change to base dispatch method.
     58        (WebCore::SourceBufferParserAVFObjC::didFailToParseStreamDataWithError): Change to base dispatch method.
     59        (WebCore::SourceBufferParserAVFObjC::didProvideMediaDataForTrackID): Change to base dispatch method.
     60        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h: Rename methods. Remove no longer used
     61        members.
     62        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
     63        (WebCore::SourceBufferPrivateAVFObjC::~SourceBufferPrivateAVFObjC):
     64        (WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData): No longer use a Cancellable task
     65        as we never cancel it anymore.
     66        (WebCore::SourceBufferPrivateAVFObjC::append): re-schedule the append task immediately to ensure
     67        that processed packets flushed on the parser queue during abort are handled in the right order on
     68        the main thread.
     69        (WebCore::SourceBufferPrivateAVFObjC::appendCompleted): Check that abort wasn't called since append started
     70        to ensure that no updateend event is incorrectly fired twice.
     71        (WebCore::SourceBufferPrivateAVFObjC::abort): Abort is now a no-op that only set the m_abortCalled member.
     72        (WebCore::SourceBufferPrivateAVFObjC::resetParserState): Change the order of operations so that the
     73        SourceBufferParser is only reset after it has finished processing its data.
     74        (WebCore::SourceBufferPrivateAVFObjC::destroyStreamDataParser): Use rename method.
     75        (WebCore::SourceBufferPrivateAVFObjC::removedFromMediaSource): Use rename method.
     76        (WebCore::SourceBufferPrivateAVFObjC::streamDataParser const): Renamed method from "parser"
     77        (WebCore::SourceBufferPrivateAVFObjC::attemptToDecrypt): Use renamed method.
     78        * platform/graphics/cocoa/SourceBufferParser.cpp:
     79        (WebCore::callOnMainThreadCallback): Move dispatch method from SourceBufferParserWebM
     80        (WebCore::SourceBufferParser::setCallOnClientThreadCallback):
     81        (WebCore::SourceBufferParser::SourceBufferParser):
     82        * platform/graphics/cocoa/SourceBufferParser.h:
     83        * platform/graphics/cocoa/SourceBufferParserWebM.cpp:
     84        (WebCore::SourceBufferParserWebM::SourceBufferParserWebM):
     85        (WebCore::SourceBufferParserWebM::resetParserState): Don't clear the data set by parsing
     86        the previous init segment. Set the parsing state to waiting for a new segment if an
     87        init segment has been fully parsed.
     88        (WebCore::SourceBufferParserWebM::OnElementEnd):
     89        (WebCore::SourceBufferParserWebM::OnEbml):
     90        (WebCore::SourceBufferParserWebM::VideoTrackData::reset):
     91        (WebCore::SourceBufferParserWebM::VideoTrackData::consumeFrameData):
     92        (WebCore::SourceBufferParserWebM::AudioTrackData::reset):
     93        * platform/graphics/cocoa/SourceBufferParserWebM.h:
     94
    1952021-07-02  Joonghun Park  <jh718.park@samsung.com>
    296

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/CDMSessionAVContentKeySession.mm

    r277449 r279542  
    121121
    122122    for (auto& sourceBuffer : m_sourceBuffers)
    123         removeParser(sourceBuffer->parser());
     123        removeParser(sourceBuffer->streamDataParser());
    124124}
    125125

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/CDMSessionAVStreamSession.mm

    r264710 r279542  
    9797
    9898    for (auto& sourceBuffer : m_sourceBuffers)
    99         removeParser(sourceBuffer->parser());
     99        removeParser(sourceBuffer->streamDataParser());
    100100}
    101101
    … …  
    235235        NSError* error = nil;
    236236        ALLOW_DEPRECATED_DECLARATIONS_BEGIN
    237         RetainPtr<NSData> request = [protectedSourceBuffer->parser() streamingContentKeyRequestDataForApp:certificateData.get() contentIdentifier:initData.get() trackID:protectedSourceBuffer->protectedTrackID() options:options.get() error:&error];
     237        RetainPtr<NSData> request = [protectedSourceBuffer->streamDataParser() streamingContentKeyRequestDataForApp:certificateData.get() contentIdentifier:initData.get() trackID:protectedSourceBuffer->protectedTrackID() options:options.get() error:&error];
    238238        ALLOW_DEPRECATED_DECLARATIONS_END
    239239
    240         if (![protectedSourceBuffer->parser() respondsToSelector:@selector(contentProtectionSessionIdentifier)])
     240        if (![protectedSourceBuffer->streamDataParser() respondsToSelector:@selector(contentProtectionSessionIdentifier)])
    241241            m_sessionId = createCanonicalUUIDString();
    242242
    … …  
    264264    RetainPtr<NSData> keyData = adoptNS([[NSData alloc] initWithBytes:key->data() length:key->length()]);
    265265    ALLOW_DEPRECATED_DECLARATIONS_BEGIN
    266     [protectedSourceBuffer->parser() processContentKeyResponseData:keyData.get() forTrackID:protectedSourceBuffer->protectedTrackID()];
     266    [protectedSourceBuffer->streamDataParser() processContentKeyResponseData:keyData.get() forTrackID:protectedSourceBuffer->protectedTrackID()];
    267267    ALLOW_DEPRECATED_DECLARATIONS_END
    268268

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/CDMSessionMediaSourceAVFObjC.mm

    r240437 r279542  
    8484    ASSERT(sourceBuffer);
    8585
    86     addParser(sourceBuffer->parser());
     86    addParser(sourceBuffer->streamDataParser());
    8787
    8888    m_sourceBuffers.append(sourceBuffer);
    … …  
    9595    ASSERT(sourceBuffer);
    9696
    97     removeParser(sourceBuffer->parser());
     97    removeParser(sourceBuffer->streamDataParser());
    9898
    9999    sourceBuffer->unregisterForErrorNotifications(this);

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.h

    r272039 r279542  
    5151    virtual ~SourceBufferParserAVFObjC();
    5252
    53     AVStreamDataParser* parser() const { return m_parser.get(); }
     53    AVStreamDataParser* streamDataParser() const { return m_parser.get(); }
    5454
    5555    Type type() const { return Type::AVFObjC; }
    … …  
    7878    RetainPtr<AVStreamDataParser> m_parser;
    7979    RetainPtr<WebAVStreamDataParserListener> m_delegate;
    80     bool m_discardSamplesUntilNextInitializationSegment { false };
    8180    bool m_parserStateWasReset { false };
    8281

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.mm

    r278253 r279542  
    242242{
    243243    m_parserStateWasReset = true;
    244     m_discardSamplesUntilNextInitializationSegment = true;
    245244}
    246245
    … …  
    262261void SourceBufferParserAVFObjC::didParseStreamDataAsAsset(AVAsset* asset)
    263262{
    264     callOnMainThread([this, strongThis = makeRef(*this), asset = retainPtr(asset)] {
    265         m_discardSamplesUntilNextInitializationSegment = false;
    266 
     263    m_callOnClientThreadCallback([this, strongThis = makeRef(*this), asset = retainPtr(asset)] {
    267264        if (!m_didParseInitializationDataCallback)
    268265            return;
    … …  
    305302void SourceBufferParserAVFObjC::didFailToParseStreamDataWithError(NSError* error)
    306303{
    307     callOnMainThread([this, strongThis = makeRef(*this), error = retainPtr(error)] {
     304    m_callOnClientThreadCallback([this, strongThis = makeRef(*this), error = retainPtr(error)] {
    308305        if (m_didEncounterErrorDuringParsingCallback)
    309306            m_didEncounterErrorDuringParsingCallback(error.get().code);
    … …  
    314311{
    315312    UNUSED_PARAM(flags);
    316     callOnMainThread([this, strongThis = makeRef(*this), sampleBuffer = retainPtr(sampleBuffer), trackID, mediaType = mediaType] {
     313    m_callOnClientThreadCallback([this, strongThis = makeRef(*this), sampleBuffer = retainPtr(sampleBuffer), trackID, mediaType = mediaType] {
    317314        if (!m_didProvideMediaDataCallback)
    318             return;
    319 
    320         if (m_discardSamplesUntilNextInitializationSegment)
    321315            return;
    322316

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h

    r278580 r279542  
    103103
    104104    uint64_t protectedTrackID() const { return m_protectedTrackID; }
    105     AVStreamDataParser* parser() const;
     105    AVStreamDataParser* streamDataParser() const;
    106106    void setCDMSession(CDMSessionMediaSourceAVFObjC*);
    107107    void setCDMInstance(CDMInstance*);
    … …  
    151151    // SourceBufferPrivate overrides
    152152    void append(Vector<unsigned char>&&) final;
     153    void removeCodedFrames(const MediaTime& start, const MediaTime& end, const MediaTime& currentMediaTime, bool isEnded, CompletionHandler<void()>&&) final;
    153154    void abort() final;
    154155    void resetParserState() final;
    … …  
    173174    void didBecomeReadyForMoreSamples(uint64_t trackID);
    174175    void appendCompleted();
    175     void destroyParser();
     176    void destroyStreamDataParser();
    176177    void destroyRenderers();
    177178    void clearTracks();
    … …  
    194195    bool m_hasPendingAppendCompletedCallback { false };
    195196    Vector<std::pair<uint64_t, Ref<MediaSample>>> m_mediaSamples;
    196     TaskCancellationGroup m_mediaSampleTaskCancellationGroup;
    197197
    198198    RetainPtr<AVSampleBufferDisplayLayer> m_displayLayer;
    … …  
    228228    uint64_t m_protectedTrackID { notFound };
    229229    uint64_t m_mapID;
     230    bool m_abortCalled { false };
    230231
    231232#if !RELEASE_LOG_DISABLED

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm

    r279335 r279542  
    349349    ASSERT(!m_client);
    350350    sourceBufferMap().remove(m_mapID);
    351     destroyParser();
     351    destroyStreamDataParser();
    352352    destroyRenderers();
    353353    clearTracks();
    … …  
    356356        PAL::CMNotificationCenterRemoveListener(PAL::CMNotificationCenterGetDefaultLocalCenter(), this, bufferWasConsumedCallback, PAL::kCMSampleBufferConsumerNotification_BufferConsumed, nullptr);
    357357
    358     if (m_hasSessionSemaphore)
    359         m_hasSessionSemaphore->signal();
    360 
    361     m_mediaSampleTaskCancellationGroup.cancel();
     358    resetParserState();
    362359}
    363360
    … …  
    418415        }
    419416
    420         callOnMainThread(CancellableTask(m_mediaSampleTaskCancellationGroup, [this, weakThis = WTFMove(weakThis)] {
     417        callOnMainThread([this, weakThis = WTFMove(weakThis)] {
    421418            if (!weakThis)
    422419                return;
    … …  
    439436                appendCompleted();
    440437            }
    441         }));
     438        });
    442439    });
    443440}
    … …  
    482479    m_protectedTrackID = trackID;
    483480
    484     auto parser = this->parser();
     481    auto parser = this->streamDataParser();
    485482    if (!parser)
    486483        return;
    … …  
    515512    m_mediaSource->sourceBufferKeyNeeded(this, m_initData.get());
    516513    if (auto session = player->cdmSession()) {
    517         if (auto parser = this->parser())
     514        if (auto parser = this->streamDataParser())
    518515            session->addParser(parser);
    519516        hasSessionSemaphore->signal();
    … …  
    534531    if (m_cdmInstance) {
    535532        if (auto instanceSession = m_cdmInstance->sessionForKeyIDs(keyIDs.value())) {
    536             if (auto parser = this->parser())
     533            if (auto parser = this->streamDataParser())
    537534                [instanceSession->contentKeySession() addContentKeyRecipient:parser];
    538535            if (m_hasSessionSemaphore) {
    … …  
    571568    ALWAYS_LOG(LOGIDENTIFIER, "data length = ", data.size());
    572569
    573     ASSERT(!m_hasSessionSemaphore);
    574     ASSERT(!m_abortSemaphore);
    575 
    576     if (m_client)
    577         m_client->sourceBufferPrivateReportExtraMemoryCost(totalTrackBufferSizeInBytes());
    578 
    579     m_abortSemaphore = Box<Semaphore>::create(0);
    580     m_parser->setWillProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis = makeWeakPtr(this), abortSemaphore = m_abortSemaphore] (uint64_t trackID) mutable {
    581         // We must call synchronously to the main thread, as the AVStreamSession must be associated
    582         // with the streamDataParser before the delegate method returns.
    583         Box<BinarySemaphore> respondedSemaphore = Box<BinarySemaphore>::create();
    584         callOnMainThread([weakThis = WTFMove(weakThis), trackID, respondedSemaphore]() {
    585             if (weakThis)
    586                 weakThis->willProvideContentKeyRequestInitializationDataForTrackID(trackID);
    587             respondedSemaphore->signal();
     570    // Queue a task to preserve the ordering of operations started by the
     571    // abort process and ensure that the new appendBuffer will only
     572    // ever deal with a sane parsing context.
     573    callOnMainThread([weakThis = makeWeakPtr(*this), data = WTFMove(data), this]() mutable {
     574        if (!weakThis)
     575            return;
     576
     577        ASSERT(!m_hasSessionSemaphore);
     578        ASSERT(!m_abortSemaphore);
     579
     580        if (m_client)
     581            m_client->sourceBufferPrivateReportExtraMemoryCost(totalTrackBufferSizeInBytes());
     582
     583        m_abortSemaphore = Box<Semaphore>::create(0);
     584        m_parser->setWillProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis, abortSemaphore = m_abortSemaphore](uint64_t trackID) mutable {
     585            // We must call synchronously to the main thread, as the AVStreamSession must be associated
     586            // with the streamDataParser before the delegate method returns.
     587            Box<BinarySemaphore> respondedSemaphore = Box<BinarySemaphore>::create();
     588            callOnMainThread([weakThis = WTFMove(weakThis), trackID, respondedSemaphore]() {
     589                if (weakThis)
     590                    weakThis->willProvideContentKeyRequestInitializationDataForTrackID(trackID);
     591                respondedSemaphore->signal();
     592            });
     593
     594            while (true) {
     595                if (respondedSemaphore->waitFor(100_ms))
     596                    return;
     597
     598                if (abortSemaphore->waitFor(100_ms)) {
     599                    abortSemaphore->signal();
     600                    return;
     601                }
     602            }
    588603        });
    589604
    590         while (true) {
    591             if (respondedSemaphore->waitFor(100_ms))
    592                 return;
    593 
    594             if (abortSemaphore->waitFor(100_ms)) {
    595                 abortSemaphore->signal();
    596                 return;
     605        m_parser->setDidProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis, abortSemaphore = m_abortSemaphore](Ref<Uint8Array>&& initData, uint64_t trackID) mutable {
     606            // Called on the data parser queue.
     607            Box<BinarySemaphore> hasSessionSemaphore = Box<BinarySemaphore>::create();
     608            callOnMainThread([weakThis = WTFMove(weakThis), initData = WTFMove(initData), trackID, hasSessionSemaphore]() mutable {
     609                if (weakThis)
     610                    weakThis->didProvideContentKeyRequestInitializationDataForTrackID(WTFMove(initData), trackID, hasSessionSemaphore);
     611            });
     612
     613            while (true) {
     614                if (hasSessionSemaphore->waitFor(100_ms))
     615                    return;
     616
     617                if (abortSemaphore->waitFor(100_ms)) {
     618                    abortSemaphore->signal();
     619                    return;
     620                }
    597621            }
    598         }
    599     });
    600 
    601     m_parser->setDidProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis = makeWeakPtr(this), abortSemaphore = m_abortSemaphore] (Ref<Uint8Array>&& initData, uint64_t trackID) mutable {
    602         Box<BinarySemaphore> hasSessionSemaphore = Box<BinarySemaphore>::create();
    603         callOnMainThread([weakThis = WTFMove(weakThis), initData = WTFMove(initData), trackID, hasSessionSemaphore] () mutable {
    604             if (weakThis)
    605                 weakThis->didProvideContentKeyRequestInitializationDataForTrackID(WTFMove(initData), trackID, hasSessionSemaphore);
    606622        });
    607623
    608         while (true) {
    609             if (hasSessionSemaphore->waitFor(100_ms))
    610                 return;
    611 
    612             if (abortSemaphore->waitFor(100_ms)) {
    613                 abortSemaphore->signal();
    614                 return;
    615             }
    616         }
    617     });
    618 
    619     m_parsingSucceeded = true;
    620     dispatch_group_enter(m_isAppendingGroup.get());
    621 
    622     dispatch_async(globalDataParserQueue(), [data = WTFMove(data), weakThis = m_appendWeakFactory.createWeakPtr(*this), parser = m_parser, isAppendingGroup = m_isAppendingGroup]() mutable {
    623         parser->appendData(WTFMove(data), [weakThis = WTFMove(weakThis)]() mutable {
    624             callOnMainThread([weakThis = WTFMove(weakThis)] {
     624        m_parsingSucceeded = true;
     625        dispatch_group_enter(m_isAppendingGroup.get());
     626
     627        dispatch_async(globalDataParserQueue(), [data = WTFMove(data), weakThis = m_appendWeakFactory.createWeakPtr(*this), parser = m_parser, isAppendingGroup = m_isAppendingGroup]() mutable {
     628            parser->appendData(WTFMove(data), [weakThis = WTFMove(weakThis)]() mutable {
     629                callOnMainThread([weakThis = WTFMove(weakThis)] {
    625630                if (!weakThis)
    626631                    return;
    … …  
    632637
    633638                weakThis->appendCompleted();
     639                });
    634640            });
     641            dispatch_group_leave(isAppendingGroup.get());
    635642        });
    636         dispatch_group_leave(isAppendingGroup.get());
    637643    });
    638644}
    … …  
    651657    }
    652658
     659    if (m_abortCalled)
     660        return;
     661
    653662    if (auto player = this->player(); player && m_parsingSucceeded)
    654663        player->setLoadingProgresssed(true);
    … …  
    657666}
    658667
     668void SourceBufferPrivateAVFObjC::removeCodedFrames(const MediaTime& start, const MediaTime& end, const MediaTime& currentMediaTime, bool isEnded, CompletionHandler<void()>&& completionHandler)
     669{
     670    // Queue a task to preserve the ordering of operations started by any
     671    // earlier call to abort and that pending frames will be removed if needed.
     672    callOnMainThread([weakThis = makeWeakPtr(*this), start, end, currentMediaTime, isEnded, completionHandler = WTFMove(completionHandler)]() mutable {
     673        if (!weakThis) {
     674            completionHandler();
     675            return;
     676        }
     677        weakThis->SourceBufferPrivate::removeCodedFrames(start, end, currentMediaTime, isEnded, WTFMove(completionHandler));
     678    });
     679}
     680
     681// The MSE spec requires that we abort the current buffer append algorithm
     682// https://w3c.github.io/media-source/#dfn-buffer-append
     683// which is then followed by a call to resetParserState
     684// as per https://w3c.github.io/media-source/#dom-sourcebuffer-abort
     685// However due to our asynchronous design this causes inherent difficulties.
     686// The SourceBuffe's abortIfUpdating method would have already cancelled any
     687// pending update not yet despatched by its timer.
     688// As the spec behaviour is non deterministic anyway, we instead process all
     689// pending frames found in the input buffer.
    659690void SourceBufferPrivateAVFObjC::abort()
    660691{
    661692    ALWAYS_LOG(LOGIDENTIFIER);
    662693
    663     // The parsing queue may be blocked waiting for the main thread to provide it a AVStreamSession. We
     694    // Queue a task to preserve the ordering of operations started by any
     695    // earlier call to abort.
     696    callOnMainThread([weakThis = makeWeakPtr(*this), this]() {
     697        if (!weakThis)
     698            return;
     699        ASSERT(!m_abortCalled, "Abort should only be called if we were currently updating, resetParserState must have been called in between");
     700        m_abortCalled = true;
     701    });
     702}
     703
     704void SourceBufferPrivateAVFObjC::resetParserState()
     705{
     706    ALWAYS_LOG(LOGIDENTIFIER);
     707
     708    // The parsing queue may be blocked waiting for the main thread to provide it an AVStreamSession. We
    664709    // were asked to abort, and that cancels all outstanding append operations. Without cancelling this
    665710    // semaphore, the m_isAppendingGroup wait operation will deadlock.
    … …  
    673718    }
    674719
    675     m_parser->resetParserState();
    676     m_mediaSamples.clear();
    677     m_initializationSegmentIsHandled = false;
    678     m_mediaSampleTaskCancellationGroup.cancel();
    679 
     720    // Wait for any pending parsing to complete.
    680721    dispatch_group_wait(m_isAppendingGroup.get(), DISPATCH_TIME_FOREVER);
    681 }
    682 
    683 void SourceBufferPrivateAVFObjC::resetParserState()
    684 {
    685     ALWAYS_LOG(LOGIDENTIFIER);
    686 
    687     m_parser->resetParserState();
    688 }
    689 
    690 void SourceBufferPrivateAVFObjC::destroyParser()
    691 {
    692     auto parser = this->parser();
     722
     723    // Dispatch a task to complete all tasks that may have been queued by the
     724    // appending group get to run first.
     725    // We keep a strong reference to the parser so that it can't be destructed
     726    // and can perform all required cleaning operations at the end of all operations.
     727    callOnMainThread([weakThis = makeWeakPtr(*this), parser = m_parser, this]() {
     728        parser->resetParserState();
     729        if (!weakThis)
     730            return;
     731        ASSERT(m_mediaSamples.isEmpty(), "All pending frames should have been processed");
     732        m_abortCalled = false;
     733    });
     734}
     735
     736void SourceBufferPrivateAVFObjC::destroyStreamDataParser()
     737{
     738    auto parser = this->streamDataParser();
    693739    if (!parser)
    694740        return;
    … …  
    743789
    744790    clearTrackBuffers();
    745     destroyParser();
     791    destroyStreamDataParser();
    746792    destroyRenderers();
    747793
    … …  
    841887}
    842888
    843 AVStreamDataParser* SourceBufferPrivateAVFObjC::parser() const
     889AVStreamDataParser* SourceBufferPrivateAVFObjC::streamDataParser() const
    844890{
    845891    if (is<SourceBufferParserAVFObjC>(m_parser.get()))
    846         return downcast<SourceBufferParserAVFObjC>(m_parser.get()).parser();
     892        return downcast<SourceBufferParserAVFObjC>(m_parser.get()).streamDataParser();
    847893    return nil;
    848894}
    … …  
    909955        return;
    910956
    911     if (auto parser = this->parser())
     957    if (auto parser = this->streamDataParser())
    912958        [instanceSession->contentKeySession() addContentKeyRecipient:parser];
    913959    if (m_hasSessionSemaphore) {

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParser.cpp

    r278338 r279542  
    5454
    5555    return nullptr;
     56}
     57
     58static SourceBufferParser::CallOnClientThreadCallback callOnMainThreadCallback()
     59{
     60    return [](Function<void()>&& function) {
     61        callOnMainThread(WTFMove(function));
     62    };
     63}
     64
     65void SourceBufferParser::setCallOnClientThreadCallback(CallOnClientThreadCallback&& callback)
     66{
     67    ASSERT(callback);
     68    m_callOnClientThreadCallback = WTFMove(callback);
     69}
     70
     71SourceBufferParser::SourceBufferParser()
     72    : m_callOnClientThreadCallback(callOnMainThreadCallback())
     73{
    5674}
    5775

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParser.h

    r272039 r279542  
    8383    };
    8484
     85    using CallOnClientThreadCallback = WTF::Function<void(WTF::Function<void()>&&)>;
     86    void setCallOnClientThreadCallback(CallOnClientThreadCallback&&);
     87
     88    // appendData will be called on the SourceBufferPrivateAVFObjC data parser queue.
     89    // Other methods will be called on the main thread, but only once appendData has returned.
    8590    virtual void appendData(Segment&&, CompletionHandler<void()>&& = [] { }, AppendFlags = AppendFlags::None) = 0;
    8691    virtual void flushPendingMediaData() = 0;
    … …  
    131136
    132137protected:
    133     SourceBufferParser() = default;
     138    SourceBufferParser();
    134139
     140    CallOnClientThreadCallback m_callOnClientThreadCallback;
    135141    DidParseInitializationDataCallback m_didParseInitializationDataCallback;
    136142    DidEncounterErrorDuringParsingCallback m_didEncounterErrorDuringParsingCallback;

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp

    r279492 r279542  
    593593}
    594594
    595 static SourceBufferParserWebM::CallOnClientThreadCallback callOnMainThreadCallback()
    596 {
    597     return [](Function<void()>&& function) {
    598         callOnMainThread(WTFMove(function));
    599     };
    600 }
    601 
    602595SourceBufferParserWebM::SourceBufferParserWebM()
    603596    : m_reader(WTF::makeUniqueRef<StreamingVectorReader>())
    604     , m_callOnClientThreadCallback(callOnMainThreadCallback())
    605597{
    606598    if (isWebmParserAvailable())
    … …  
    690682{
    691683    INFO_LOG_IF_POSSIBLE(LOGIDENTIFIER);
    692     flushPendingAudioBuffers();
    693684    if (m_parser)
    694685        m_parser->DidSeek();
    695     m_state = State::None;
    696     m_tracks.clear();
     686    m_reader->reset();
     687    m_state = m_initializationSegmentProcessed ? State::ReadingSegment : State::None;
    697688    m_initializationSegment = nullptr;
    698689    m_initializationSegmentEncountered = false;
    699690    m_currentBlock.reset();
     691    for (auto& track : m_tracks)
     692        track->reset();
    700693}
    701694
    … …  
    794787        m_initializationSegmentEncountered = false;
    795788        m_initializationSegment = nullptr;
     789        m_initializationSegmentProcessed = true;
    796790
    797791        if (!m_keyIds.isEmpty()) {
    … …  
    815809    m_initializationSegmentEncountered = true;
    816810    m_initializationSegment = WTF::makeUniqueWithoutFastMallocCheck<InitializationSegment>();
     811    // TODO: Setting this to false here, will prevent adding a new media segment should a
     812    // partial init segment be encountered after a call to sourceBuffer.abort().
     813    // It's probably fine as no-one in their right mind should send partial init segment only
     814    // to immediately abort it. We do it this way mostly to avoid getting into a rabbit hole
     815    // of ensuring that libwebm does something sane with rubbish input.
     816    m_initializationSegmentProcessed = false;
    817817
    818818    return Status(Status::kOkCompleted);
    … …  
    10841084#define PARSER_LOG_ERROR_IF_POSSIBLE(...) if (parser().loggerPtr()) parser().loggerPtr()->error(logChannel(), WTF::Logger::LogSiteIdentifier(logClassName(), __func__, parser().logIdentifier()), __VA_ARGS__)
    10851085
     1086#if ENABLE(VP9)
     1087void SourceBufferParserWebM::VideoTrackData::reset()
     1088{
     1089    m_currentBlockBuffer = nullptr;
     1090    TrackData::reset();
     1091}
     1092#endif
     1093
    10861094webm::Status SourceBufferParserWebM::VideoTrackData::consumeFrameData(webm::Reader& reader, const FrameMetadata& metadata, uint64_t* bytesRemaining, const CMTime& presentationTime, int sampleCount)
    10871095{
    … …  
    11491157    createSampleBuffer(presentationTime, sampleCount, metadata);
    11501158
    1151     m_currentBlockBuffer = nullptr;
    1152     m_partialBytesRead = 0;
    1153     m_currentPacketSize = std::nullopt;
     1159    reset();
    11541160#else
    11551161    UNUSED_PARAM(metadata);
    … …  
    12401246    UNUSED_PARAM(metadata);
    12411247#endif // ENABLE(VP9)
     1248}
     1249
     1250void SourceBufferParserWebM::AudioTrackData::reset()
     1251{
     1252    m_packetDescriptions.clear();
     1253    m_packetsData.clear();
     1254    m_currentPacketByteOffset = std::nullopt;
     1255    TrackData::reset();
    12421256}
    12431257
    … …  
    14021416}
    14031417
    1404 void SourceBufferParserWebM::setCallOnClientThreadCallback(CallOnClientThreadCallback&& callback)
    1405 {
    1406     ASSERT(callback);
    1407     m_callOnClientThreadCallback = WTFMove(callback);
    1408 }
    1409 
    14101418const MemoryCompactLookupOnlyRobinHoodHashSet<String>& SourceBufferParserWebM::supportedVideoCodecs()
    14111419{

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h

    r279492 r279542  
    8484    void setMinimumAudioSampleDuration(float);
    8585   
    86     using CallOnClientThreadCallback = WTF::Function<void(WTF::Function<void()>&&)>;
    87     void setCallOnClientThreadCallback(CallOnClientThreadCallback&&);
    88 
    8986    void setLogger(const WTF::Logger&, const void* identifier) final;
    9087
    … …  
    161158        }
    162159
     160        virtual void reset()
     161        {
     162            m_currentPacketSize = std::nullopt;
     163            m_partialBytesRead = 0;
     164        }
     165
    163166    protected:
    164167        std::optional<size_t> m_currentPacketSize;
    … …  
    186189        }
    187190
     191#if ENABLE(VP9)
     192        void reset() final;
     193#endif
    188194        webm::Status consumeFrameData(webm::Reader&, const webm::FrameMetadata&, uint64_t*, const CMTime&, int) final;
    189        
     195
    190196    private:
    191197        void createSampleBuffer(const CMTime&, int, const webm::FrameMetadata&);
    … …  
    212218
    213219        webm::Status consumeFrameData(webm::Reader&, const webm::FrameMetadata&, uint64_t*, const CMTime&, int) final;
     220        void reset() final;
    214221        void createSampleBuffer(std::optional<size_t> latestByteRangeOffset = std::nullopt);
    215222
    … …  
    264271    std::unique_ptr<webm::WebmParser> m_parser;
    265272    bool m_initializationSegmentEncountered { false };
     273    bool m_initializationSegmentProcessed { false };
    266274    uint32_t m_timescale { 1000 };
    267275    uint64_t m_currentTimecode { 0 };
    … …  
    277285    float m_minimumAudioSampleDuration { 2 };
    278286
    279     CallOnClientThreadCallback m_callOnClientThreadCallback;
    280 
    281287    RefPtr<const WTF::Logger> m_logger;
    282288    const void* m_logIdentifier { nullptr };
Note: See TracChangeset for help on using the changeset viewer.