Changeset 279904 in webkit


Ignore:
Timestamp:
Jul 13, 2021, 10:12:26 PM (4 years ago)
Author:
Jean-Yves Avenard
Message:

SourceBuffer.abort() doesn't go back to state WAITING_FOR_SEGMENT properly
​https://bugs.webkit.org/show_bug.cgi?id=227559
<rdar://problem/79996056>

Reviewed by Jer Noble.

Source/WebCore:

Per spec, calling sourcebuffer.abort method should allow you to add a new segment
immediately after, as abort moves the append state back to WAITING_FOR_SEGMENT.
A segment can be either an init segment or a media segment.
We used to discard all further content until an init segment was encountered.
This could be problematic due to the typical use case of abort:
1- Seek to a location
2- Append a partial media segment long enough to finish seeking and display the

new content at the new position.

If multiple seeks were done in rapid succession, abort() is called right after
before starting the new seek so that we can add the new segment, regardless of what
was appended before.
YouTube applies a workaround for Safari where it will always append an init segment
after calling abort, this is different to what they do with Firefox (and likely Chrome).
To be able to resume after appending a partial media segment we must ensure that the
SourceBufferParser is always left in a sane context, and not be interrupted at some
random points. The abort() call used to interrupt the buffer parsing on the fly and
then reset things which would require a new init segment to restart.
Instead we always fully parse the pending buffer received befofe the call to abort
and drop all demuxed samples.
The SourceBufferParserAVFObjC could already properly deal with discontinuity unlike
SourceBufferParserWebM.
To ensure that buffers sent after the call to abort() are only ever processed once
the pending ones have been parsed, and in order to avoid having blocking calls
we play with the order in which tasks are scheduled.

Fly-by fixes:

  • The SourceBufferParser handle two type of parser: SourceBufferParser and the

platform specific AVStreamDataParser. Rename the accessor method to more clearly
differentate which parser we are dealing with.

  • The SourceBufferParserWebM and SourceBufferPrivateAVFObjC used different task dispatching

mechanisms. We make them both share the same one now found in the base class.

  • If SourceBufferPrivateAVFObjC::m_hasPendingAppendCompletedCallback had been set prior an

abort() it wouldn't have been reset, causing the need for a new init segment.

  • If abort() had been called while samples were pending, the source buffer content was

undefined if timestamp offset or append windows start/end were changed immediately after.

  • When an error occurs during the Segment Parser Loop, we should abort and run the append

error algorithm. We instead fired an error event for each sample found in the media segment.
In a debug built it was have asserted. We can't actually abort, so instead we ignore all
further samples once an error is encountered.

Tests: media/media-source/media-mp4-h264-partial-abort.html

media/media-source/media-webm-opus-partial-abort.html

  • platform/graphics/avfoundation/objc/CDMSessionAVContentKeySession.mm:

(WebCore::CDMSessionAVContentKeySession::~CDMSessionAVContentKeySession): rename method.

  • platform/graphics/avfoundation/objc/CDMSessionAVStreamSession.mm:

(WebCore::CDMSessionAVStreamSession::~CDMSessionAVStreamSession): rename method.
(WebCore::CDMSessionAVStreamSession::update): rename method.

  • platform/graphics/avfoundation/objc/CDMSessionMediaSourceAVFObjC.mm: rename method.

(WebCore::CDMSessionMediaSourceAVFObjC::addSourceBuffer): rename method.
(WebCore::CDMSessionMediaSourceAVFObjC::removeSourceBuffer): rename method.

  • platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.h: rename method, remove

now unused member.

  • platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.mm:

(WebCore::SourceBufferPrivateAVFObjC::removeCodedFrames): Postpone call to ensure
we are running the remove frame algorithm once all pending frames have been processed.
(WebCore::SourceBufferParserAVFObjC::resetParserState): Remove use of m_discardSamplesUntilNextInitializationSegment.
(WebCore::SourceBufferParserAVFObjC::didParseStreamDataAsAsset): Change to base dispatch method.
(WebCore::SourceBufferParserAVFObjC::didFailToParseStreamDataWithError): Change to base dispatch method.
(WebCore::SourceBufferParserAVFObjC::didProvideMediaDataForTrackID): Change to base dispatch method.

  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h: Rename methods. Remove no longer used

members.

  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:

(WebCore::SourceBufferPrivateAVFObjC::~SourceBufferPrivateAVFObjC):
(WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData): No longer use a Cancellable task
as we never cancel it anymore.
(WebCore::SourceBufferPrivateAVFObjC::append): re-schedule the append task immediately to ensure
that processed packets flushed on the parser queue during abort are handled in the right order on
the main thread.
(WebCore::SourceBufferPrivateAVFObjC::appendCompleted): Check that abort wasn't called since append started
to ensure that no updateend event is incorrectly fired twice.
(WebCore::SourceBufferPrivateAVFObjC::abort): Abort is now a no-op that only set the m_abortCalled member.
(WebCore::SourceBufferPrivateAVFObjC::resetParserState): Change the order of operations so that the
SourceBufferParser is only reset after it has finished processing its data.
(WebCore::SourceBufferPrivateAVFObjC::destroyStreamDataParser): Use rename method.
(WebCore::SourceBufferPrivateAVFObjC::removedFromMediaSource): Use rename method.
(WebCore::SourceBufferPrivateAVFObjC::streamDataParser const): Renamed method from "parser"
(WebCore::SourceBufferPrivateAVFObjC::attemptToDecrypt): Use renamed method.

  • platform/graphics/cocoa/SourceBufferParser.cpp:

(WebCore::callOnMainThreadCallback): Move dispatch method from SourceBufferParserWebM
(WebCore::SourceBufferParser::setCallOnClientThreadCallback):
(WebCore::SourceBufferParser::SourceBufferParser):

  • platform/graphics/cocoa/SourceBufferParser.h:
  • platform/graphics/cocoa/SourceBufferParserWebM.cpp:

(WebCore::SourceBufferParserWebM::SourceBufferParserWebM):
(WebCore::SourceBufferParserWebM::resetParserState): Don't clear the data set by parsing
the previous init segment. Set the parsing state to waiting for a new segment if an
init segment has been fully parsed.
(WebCore::SourceBufferParserWebM::OnElementEnd):
(WebCore::SourceBufferParserWebM::OnEbml):
(WebCore::SourceBufferParserWebM::VideoTrackData::reset):
(WebCore::SourceBufferParserWebM::VideoTrackData::consumeFrameData):
(WebCore::SourceBufferParserWebM::AudioTrackData::reset):

  • platform/graphics/cocoa/SourceBufferParserWebM.h:

LayoutTests:

  • media/media-source/content/test-fragmented-video-manifest.json: Added.
  • media/media-source/content/test-fragmented-video.mp4: Added.
  • media/media-source/media-mp4-h264-partial-abort-expected.txt: Added.
  • media/media-source/media-mp4-h264-partial-abort.html: Added.
  • media/media-source/media-webm-opus-partial-abort-expected.txt: Added.
  • media/media-source/media-webm-opus-partial-abort.html: Added.
  • media/media-source/media-webm-opus-partial.html: fix title.
  • media/media-source/media-webm-vorbis-partial.html: fix title.
  • platform/mac/TestExpectations:
Location:
trunk
Files:
6 added
19 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r279901 r279904  
     12021-07-13  Jean-Yves Avenard  <jya@apple.com>
     2
     3        SourceBuffer.abort() doesn't go back to state WAITING_FOR_SEGMENT properly
     4        https://bugs.webkit.org/show_bug.cgi?id=227559
     5        <rdar://problem/79996056>
     6
     7        Reviewed by Jer Noble.
     8
     9        * media/media-source/content/test-fragmented-video-manifest.json: Added.
     10        * media/media-source/content/test-fragmented-video.mp4: Added.
     11        * media/media-source/media-mp4-h264-partial-abort-expected.txt: Added.
     12        * media/media-source/media-mp4-h264-partial-abort.html: Added.
     13        * media/media-source/media-webm-opus-partial-abort-expected.txt: Added.
     14        * media/media-source/media-webm-opus-partial-abort.html: Added.
     15        * media/media-source/media-webm-opus-partial.html: fix title.
     16        * media/media-source/media-webm-vorbis-partial.html: fix title.
     17        * platform/mac/TestExpectations:
     18
    1192021-07-13  Eric Hutchison  <ehutchison@apple.com>
    220

  • trunk/LayoutTests/media/media-source/media-webm-opus-partial.html

    r279622 r279904  
    22<html>
    33<head>
    4     <title>media-opus-partial</title>
     4    <title>media-webm-opus-partial</title>
    55    <script src="../../media/media-source/media-source-loader.js"></script>
    66    <script src="../../media/video-test.js"></script>

  • trunk/LayoutTests/media/media-source/media-webm-vorbis-partial.html

    r279622 r279904  
    22<html>
    33<head>
    4     <title>media-vorbis-partial</title>
     4    <title>media-webm-vorbis-partial</title>
    55    <script src="../../media/media-source/media-source-loader.js"></script>
    66    <script src="../../media/video-test.js"></script>

  • trunk/LayoutTests/platform/mac-wk2/TestExpectations

    r279900 r279904  
    12511251imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-error.html [ Pass Failure ]
    12521252
    1253 webkit.org/b/222500 fast/canvas/webgl/texImage2D-video-flipY-false.html [ Timeout ]
     1253webkit.org/b/222500 fast/canvas/webgl/texImage2D-video-flipY-false.html [ Pass Timeout ]
    12541254
    12551255webkit.org/b/222677 inspector/model/auditTestCase.html [ Pass Timeout ]

  • trunk/LayoutTests/platform/mac/TestExpectations

    r279894 r279904  
    17651765# These tests require macOS Monterey.
    17661766[ Catalina Mojave BigSur ] media/media-source/media-webm-vorbis-partial.html [ Skip ]
     1767[ Catalina Mojave BigSur ] media/media-source/media-source-webm-vorbis-partial.html [ Skip ]
    17671768[ Catalina Mojave BigSur ] media/media-source/media-webm-opus-partial.html [ Skip ]
     1769[ Catalina Mojave BigSur ] media/media-source/media-webm-opus-partial-abort.html [ Skip ]
    17681770
    17691771webkit.org/b/214422 imported/w3c/web-platform-tests/webaudio/the-audio-api/the-audiocontext-interface/suspend-after-construct.html [ Pass Failure ]
    … …  
    21602162
    21612163webkit.org/b/222573 media/media-fullscreen-pause-inline.html [ Pass Failure ]
    2162 
    2163 webkit.org/b/222495 media/media-source/media-source-webm-vorbis-partial.html [ Failure ]
    21642164
    21652165webkit.org/b/222692 inspector/page/empty-or-missing-resources.html [ Pass Timeout ]

  • trunk/Source/WebCore/ChangeLog

    r279886 r279904  
     12021-07-13  Jean-Yves Avenard  <jya@apple.com>
     2
     3        SourceBuffer.abort() doesn't go back to state WAITING_FOR_SEGMENT properly
     4        https://bugs.webkit.org/show_bug.cgi?id=227559
     5        <rdar://problem/79996056>
     6
     7        Reviewed by Jer Noble.
     8
     9        Per spec, calling sourcebuffer.abort method should allow you to add a new segment
     10        immediately after, as abort moves the append state back to WAITING_FOR_SEGMENT.
     11        A segment can be either an init segment or a media segment.
     12        We used to discard all further content until an init segment was encountered.
     13        This could be problematic due to the typical use case of abort:
     14        1- Seek to a location
     15        2- Append a partial media segment long enough to finish seeking and display the
     16         new content at the new position.
     17        If multiple seeks were done in rapid succession, abort() is called right after
     18        before starting the new seek so that we can add the new segment, regardless of what
     19        was appended before.
     20        YouTube applies a workaround for Safari where it will always append an init segment
     21        after calling abort, this is different to what they do with Firefox (and likely Chrome).
     22        To be able to resume after appending a partial media segment we must ensure that the
     23        SourceBufferParser is always left in a sane context, and not be interrupted at some
     24        random points. The abort() call used to interrupt the buffer parsing on the fly and
     25        then reset things which would require a new init segment to restart.
     26        Instead we always fully parse the pending buffer received befofe the call to abort
     27        and drop all demuxed samples.
     28        The SourceBufferParserAVFObjC could already properly deal with discontinuity unlike
     29        SourceBufferParserWebM.
     30        To ensure that buffers sent after the call to abort() are only ever processed once
     31        the pending ones have been parsed, and in order to avoid having blocking calls
     32        we play with the order in which tasks are scheduled.
     33
     34        Fly-by fixes:
     35        - The SourceBufferParser handle two type of parser: SourceBufferParser and the
     36        platform specific AVStreamDataParser. Rename the accessor method to more clearly
     37        differentate which parser we are dealing with.
     38        - The SourceBufferParserWebM and SourceBufferPrivateAVFObjC used different task dispatching
     39        mechanisms. We make them both share the same one now found in the base class.
     40        - If SourceBufferPrivateAVFObjC::m_hasPendingAppendCompletedCallback had been set prior an
     41        abort() it wouldn't have been reset, causing the need for a new init segment.
     42        - If abort() had been called while samples were pending, the source buffer content was
     43        undefined if timestamp offset or append windows start/end were changed immediately after.
     44        - When an error occurs during the Segment Parser Loop, we should abort and run the append
     45        error algorithm. We instead fired an error event for each sample found in the media segment.
     46        In a debug built it was have asserted. We can't actually abort, so instead we ignore all
     47        further samples once an error is encountered.
     48
     49        Tests: media/media-source/media-mp4-h264-partial-abort.html
     50               media/media-source/media-webm-opus-partial-abort.html
     51
     52        * platform/graphics/avfoundation/objc/CDMSessionAVContentKeySession.mm:
     53        (WebCore::CDMSessionAVContentKeySession::~CDMSessionAVContentKeySession): rename method.
     54        * platform/graphics/avfoundation/objc/CDMSessionAVStreamSession.mm:
     55        (WebCore::CDMSessionAVStreamSession::~CDMSessionAVStreamSession): rename method.
     56        (WebCore::CDMSessionAVStreamSession::update): rename method.
     57        * platform/graphics/avfoundation/objc/CDMSessionMediaSourceAVFObjC.mm: rename method.
     58        (WebCore::CDMSessionMediaSourceAVFObjC::addSourceBuffer): rename method.
     59        (WebCore::CDMSessionMediaSourceAVFObjC::removeSourceBuffer): rename method.
     60        * platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.h: rename method, remove
     61        now unused member.
     62        * platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.mm:
     63        (WebCore::SourceBufferPrivateAVFObjC::removeCodedFrames): Postpone call to ensure
     64        we are running the remove frame algorithm once all pending frames have been processed.
     65        (WebCore::SourceBufferParserAVFObjC::resetParserState): Remove use of m_discardSamplesUntilNextInitializationSegment.
     66        (WebCore::SourceBufferParserAVFObjC::didParseStreamDataAsAsset): Change to base dispatch method.
     67        (WebCore::SourceBufferParserAVFObjC::didFailToParseStreamDataWithError): Change to base dispatch method.
     68        (WebCore::SourceBufferParserAVFObjC::didProvideMediaDataForTrackID): Change to base dispatch method.
     69        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h: Rename methods. Remove no longer used
     70        members.
     71        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
     72        (WebCore::SourceBufferPrivateAVFObjC::~SourceBufferPrivateAVFObjC):
     73        (WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData): No longer use a Cancellable task
     74        as we never cancel it anymore.
     75        (WebCore::SourceBufferPrivateAVFObjC::append): re-schedule the append task immediately to ensure
     76        that processed packets flushed on the parser queue during abort are handled in the right order on
     77        the main thread.
     78        (WebCore::SourceBufferPrivateAVFObjC::appendCompleted): Check that abort wasn't called since append started
     79        to ensure that no updateend event is incorrectly fired twice.
     80        (WebCore::SourceBufferPrivateAVFObjC::abort): Abort is now a no-op that only set the m_abortCalled member.
     81        (WebCore::SourceBufferPrivateAVFObjC::resetParserState): Change the order of operations so that the
     82        SourceBufferParser is only reset after it has finished processing its data.
     83        (WebCore::SourceBufferPrivateAVFObjC::destroyStreamDataParser): Use rename method.
     84        (WebCore::SourceBufferPrivateAVFObjC::removedFromMediaSource): Use rename method.
     85        (WebCore::SourceBufferPrivateAVFObjC::streamDataParser const): Renamed method from "parser"
     86        (WebCore::SourceBufferPrivateAVFObjC::attemptToDecrypt): Use renamed method.
     87        * platform/graphics/cocoa/SourceBufferParser.cpp:
     88        (WebCore::callOnMainThreadCallback): Move dispatch method from SourceBufferParserWebM
     89        (WebCore::SourceBufferParser::setCallOnClientThreadCallback):
     90        (WebCore::SourceBufferParser::SourceBufferParser):
     91        * platform/graphics/cocoa/SourceBufferParser.h:
     92        * platform/graphics/cocoa/SourceBufferParserWebM.cpp:
     93        (WebCore::SourceBufferParserWebM::SourceBufferParserWebM):
     94        (WebCore::SourceBufferParserWebM::resetParserState): Don't clear the data set by parsing
     95        the previous init segment. Set the parsing state to waiting for a new segment if an
     96        init segment has been fully parsed.
     97        (WebCore::SourceBufferParserWebM::OnElementEnd):
     98        (WebCore::SourceBufferParserWebM::OnEbml):
     99        (WebCore::SourceBufferParserWebM::VideoTrackData::reset):
     100        (WebCore::SourceBufferParserWebM::VideoTrackData::consumeFrameData):
     101        (WebCore::SourceBufferParserWebM::AudioTrackData::reset):
     102        * platform/graphics/cocoa/SourceBufferParserWebM.h:
     103
    11042021-07-13  Alex Christensen  <achristensen@webkit.org>
    2105

  • trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.cpp

    r278728 r279904  
    171171
    172172    if (m_client) {
    173         m_client->sourceBufferPrivateAppendComplete(parsingSucceeded ? SourceBufferPrivateClient::AppendResult::AppendSucceeded : SourceBufferPrivateClient::AppendResult::ParsingFailed);
     173        if (!m_didReceiveSampleErrored)
     174            m_client->sourceBufferPrivateAppendComplete(parsingSucceeded ? SourceBufferPrivateClient::AppendResult::AppendSucceeded : SourceBufferPrivateClient::AppendResult::ParsingFailed);
    174175        m_client->sourceBufferPrivateReportExtraMemoryCost(totalTrackBufferSizeInBytes());
    175176    }
    … …  
    830831void SourceBufferPrivate::didReceiveSample(Ref<MediaSample>&& originalSample)
    831832{
    832     if (!m_isAttached)
     833    if (!m_isAttached || m_didReceiveSampleErrored)
    833834        return;
    834835
    … …  
    844845    if ((!m_receivedFirstInitializationSegment || m_pendingInitializationSegmentForChangeType) && m_client) {
    845846        m_client->sourceBufferPrivateAppendError(true);
     847        m_didReceiveSampleErrored = true;
    846848        return;
    847849    }

  • trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h

    r278603 r279904  
    192192    bool m_receivedFirstInitializationSegment { false };
    193193    bool m_pendingInitializationSegmentForChangeType { false };
     194    bool m_didReceiveSampleErrored { false };
    194195
    195196    MediaTime m_timestampOffset;

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/CDMSessionAVContentKeySession.mm

    r279622 r279904  
    121121
    122122    for (auto& sourceBuffer : m_sourceBuffers)
    123         removeParser(sourceBuffer->parser());
     123        removeParser(sourceBuffer->streamDataParser());
    124124}
    125125

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/CDMSessionAVStreamSession.mm

    r279622 r279904  
    9797
    9898    for (auto& sourceBuffer : m_sourceBuffers)
    99         removeParser(sourceBuffer->parser());
     99        removeParser(sourceBuffer->streamDataParser());
    100100}
    101101
    … …  
    235235        NSError* error = nil;
    236236        ALLOW_DEPRECATED_DECLARATIONS_BEGIN
    237         RetainPtr<NSData> request = [protectedSourceBuffer->parser() streamingContentKeyRequestDataForApp:certificateData.get() contentIdentifier:initData.get() trackID:protectedSourceBuffer->protectedTrackID() options:options.get() error:&error];
     237        RetainPtr<NSData> request = [protectedSourceBuffer->streamDataParser() streamingContentKeyRequestDataForApp:certificateData.get() contentIdentifier:initData.get() trackID:protectedSourceBuffer->protectedTrackID() options:options.get() error:&error];
    238238        ALLOW_DEPRECATED_DECLARATIONS_END
    239239
    240         if (![protectedSourceBuffer->parser() respondsToSelector:@selector(contentProtectionSessionIdentifier)])
     240        if (![protectedSourceBuffer->streamDataParser() respondsToSelector:@selector(contentProtectionSessionIdentifier)])
    241241            m_sessionId = createCanonicalUUIDString();
    242242
    … …  
    264264    RetainPtr<NSData> keyData = adoptNS([[NSData alloc] initWithBytes:key->data() length:key->length()]);
    265265    ALLOW_DEPRECATED_DECLARATIONS_BEGIN
    266     [protectedSourceBuffer->parser() processContentKeyResponseData:keyData.get() forTrackID:protectedSourceBuffer->protectedTrackID()];
     266    [protectedSourceBuffer->streamDataParser() processContentKeyResponseData:keyData.get() forTrackID:protectedSourceBuffer->protectedTrackID()];
    267267    ALLOW_DEPRECATED_DECLARATIONS_END
    268268

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/CDMSessionMediaSourceAVFObjC.mm

    r279622 r279904  
    8484    ASSERT(sourceBuffer);
    8585
    86     addParser(sourceBuffer->parser());
     86    addParser(sourceBuffer->streamDataParser());
    8787
    8888    m_sourceBuffers.append(sourceBuffer);
    … …  
    9595    ASSERT(sourceBuffer);
    9696
    97     removeParser(sourceBuffer->parser());
     97    removeParser(sourceBuffer->streamDataParser());
    9898
    9999    sourceBuffer->unregisterForErrorNotifications(this);

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.h

    r279622 r279904  
    5151    virtual ~SourceBufferParserAVFObjC();
    5252
    53     AVStreamDataParser* parser() const { return m_parser.get(); }
     53    AVStreamDataParser* streamDataParser() const { return m_parser.get(); }
    5454
    5555    Type type() const { return Type::AVFObjC; }
    … …  
    7878    RetainPtr<AVStreamDataParser> m_parser;
    7979    RetainPtr<WebAVStreamDataParserListener> m_delegate;
    80     bool m_discardSamplesUntilNextInitializationSegment { false };
    8180    bool m_parserStateWasReset { false };
    8281

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferParserAVFObjC.mm

    r279622 r279904  
    242242{
    243243    m_parserStateWasReset = true;
    244     m_discardSamplesUntilNextInitializationSegment = true;
    245244}
    246245
    … …  
    262261void SourceBufferParserAVFObjC::didParseStreamDataAsAsset(AVAsset* asset)
    263262{
    264     callOnMainThread([this, strongThis = makeRef(*this), asset = retainPtr(asset)] {
    265         m_discardSamplesUntilNextInitializationSegment = false;
    266 
     263    m_callOnClientThreadCallback([this, strongThis = makeRef(*this), asset = retainPtr(asset)] {
    267264        if (!m_didParseInitializationDataCallback)
    268265            return;
    … …  
    305302void SourceBufferParserAVFObjC::didFailToParseStreamDataWithError(NSError* error)
    306303{
    307     callOnMainThread([this, strongThis = makeRef(*this), error = retainPtr(error)] {
     304    m_callOnClientThreadCallback([this, strongThis = makeRef(*this), error = retainPtr(error)] {
    308305        if (m_didEncounterErrorDuringParsingCallback)
    309306            m_didEncounterErrorDuringParsingCallback(error.get().code);
    … …  
    314311{
    315312    UNUSED_PARAM(flags);
    316     callOnMainThread([this, strongThis = makeRef(*this), sampleBuffer = retainPtr(sampleBuffer), trackID, mediaType = mediaType] {
     313    m_callOnClientThreadCallback([this, strongThis = makeRef(*this), sampleBuffer = retainPtr(sampleBuffer), trackID, mediaType = mediaType] {
    317314        if (!m_didProvideMediaDataCallback)
    318             return;
    319 
    320         if (m_discardSamplesUntilNextInitializationSegment)
    321315            return;
    322316

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h

    r279622 r279904  
    103103
    104104    uint64_t protectedTrackID() const { return m_protectedTrackID; }
    105     AVStreamDataParser* parser() const;
     105    AVStreamDataParser* streamDataParser() const;
    106106    void setCDMSession(CDMSessionMediaSourceAVFObjC*);
    107107    void setCDMInstance(CDMInstance*);
    … …  
    173173    void didBecomeReadyForMoreSamples(uint64_t trackID);
    174174    void appendCompleted();
    175     void destroyParser();
     175    void destroyStreamDataParser();
    176176    void destroyRenderers();
    177177    void clearTracks();
    … …  
    191191
    192192    Ref<SourceBufferParser> m_parser;
    193     bool m_initializationSegmentIsHandled { false };
     193    bool m_processingInitializationSegment { false };
    194194    bool m_hasPendingAppendCompletedCallback { false };
    195195    Vector<std::pair<uint64_t, Ref<MediaSample>>> m_mediaSamples;
    196     TaskCancellationGroup m_mediaSampleTaskCancellationGroup;
    197196
    198197    RetainPtr<AVSampleBufferDisplayLayer> m_displayLayer;
    … …  
    228227    uint64_t m_protectedTrackID { notFound };
    229228    uint64_t m_mapID;
     229    uint32_t m_abortCalled { 0 };
    230230
    231231#if !RELEASE_LOG_DISABLED

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm

    r279622 r279904  
    323323
    324324    sourceBufferMap().add(m_mapID, makeWeakPtr(*this));
    325 
    326     m_parser->setDidParseInitializationDataCallback([weakThis = makeWeakPtr(this)] (InitializationSegment&& segment) mutable {
    327         ASSERT(isMainThread());
    328         if (weakThis)
    329             weakThis->didParseInitializationData(WTFMove(segment));
    330     });
    331 
    332     m_parser->setDidEncounterErrorDuringParsingCallback([weakThis = makeWeakPtr(this)] (int32_t errorCode) mutable {
    333         ASSERT(isMainThread());
    334         if (weakThis)
    335             weakThis->didEncounterErrorDuringParsing(errorCode);
    336     });
    337 
    338     m_parser->setDidProvideMediaDataCallback([weakThis = makeWeakPtr(this)] (Ref<MediaSample>&& sample, uint64_t trackId, const String& mediaType) mutable {
    339         ASSERT(isMainThread());
    340         if (weakThis)
    341             weakThis->didProvideMediaDataForTrackId(WTFMove(sample), trackId, mediaType);
    342     });
    343325}
    344326
    … …  
    349331    ASSERT(!m_client);
    350332    sourceBufferMap().remove(m_mapID);
    351     destroyParser();
     333    destroyStreamDataParser();
    352334    destroyRenderers();
    353335    clearTracks();
    … …  
    356338        PAL::CMNotificationCenterRemoveListener(PAL::CMNotificationCenterGetDefaultLocalCenter(), this, bufferWasConsumedCallback, PAL::kCMSampleBufferConsumerNotification_BufferConsumed, nullptr);
    357339
    358     if (m_hasSessionSemaphore)
    359         m_hasSessionSemaphore->signal();
    360 
    361     m_mediaSampleTaskCancellationGroup.cancel();
     340    abort();
     341    resetParserState();
    362342}
    363343
    … …  
    403383        player->characteristicsChanged();
    404384
    405     m_initializationSegmentIsHandled = false;
    406     didReceiveInitializationSegment(WTFMove(segment), [this, weakThis = makeWeakPtr(*this)]() mutable {
     385    m_processingInitializationSegment = true;
     386    didReceiveInitializationSegment(WTFMove(segment), [this, weakThis = makeWeakPtr(*this), abortCalled = m_abortCalled]() {
    407387        ASSERT(isMainThread());
    408         if (!weakThis)
     388        if (!weakThis || abortCalled != weakThis->m_abortCalled)
    409389            return;
     390
     391        m_processingInitializationSegment = false;
    410392
    411393        if (auto player = this->player())
    412394            player->characteristicsChanged();
    413395
    414         if  (m_mediaSamples.isEmpty()) {
    415             m_initializationSegmentIsHandled = true;
    416             ALWAYS_LOG(LOGIDENTIFIER, "initialization segment is handled");
    417             return;
     396        auto mediaSamples = std::exchange(m_mediaSamples, { });
     397        for (auto& trackIdMediaSamplePair : mediaSamples) {
     398            auto trackId = trackIdMediaSamplePair.first;
     399            auto& mediaSample = trackIdMediaSamplePair.second;
     400            if (trackId == m_enabledVideoTrackID || m_audioRenderers.contains(trackId)) {
     401                DEBUG_LOG(LOGIDENTIFIER, mediaSample.get());
     402                didReceiveSample(WTFMove(mediaSample));
     403            }
    418404        }
    419405
    420         callOnMainThread(CancellableTask(m_mediaSampleTaskCancellationGroup, [this, weakThis = WTFMove(weakThis)] {
    421             if (!weakThis)
    422                 return;
    423 
    424             auto mediaSamples = std::exchange(m_mediaSamples, { });
    425             for (auto& trackIdMediaSamplePair : mediaSamples) {
    426                 auto trackId = trackIdMediaSamplePair.first;
    427                 auto& mediaSample = trackIdMediaSamplePair.second;
    428                 if (trackId == m_enabledVideoTrackID || m_audioRenderers.contains(trackId)) {
    429                     DEBUG_LOG(LOGIDENTIFIER, mediaSample.get());
    430                     didReceiveSample(WTFMove(mediaSample));
    431                 }
    432             }
    433 
    434             m_initializationSegmentIsHandled = true;
    435             ALWAYS_LOG(LOGIDENTIFIER, "initialization segment is handled");
    436 
    437             if (m_hasPendingAppendCompletedCallback) {
    438                 m_hasPendingAppendCompletedCallback = false;
    439                 appendCompleted();
    440             }
    441         }));
     406        ALWAYS_LOG(LOGIDENTIFIER, "initialization segment was processed");
     407
     408        if (m_hasPendingAppendCompletedCallback) {
     409            m_hasPendingAppendCompletedCallback = false;
     410            appendCompleted();
     411        }
    442412    });
    443413}
    … …  
    456426{
    457427    UNUSED_PARAM(mediaType);
    458     if (!m_initializationSegmentIsHandled) {
     428
     429    if (m_processingInitializationSegment) {
    459430        DEBUG_LOG(LOGIDENTIFIER, mediaSample.get());
    460431        m_mediaSamples.append(std::make_pair(trackId, WTFMove(mediaSample)));
    … …  
    482453    m_protectedTrackID = trackID;
    483454
    484     auto parser = this->parser();
     455    auto parser = this->streamDataParser();
    485456    if (!parser)
    486457        return;
    … …  
    515486    m_mediaSource->sourceBufferKeyNeeded(this, m_initData.get());
    516487    if (auto session = player->cdmSession()) {
    517         if (auto parser = this->parser())
     488        if (auto parser = this->streamDataParser())
    518489            session->addParser(parser);
    519490        hasSessionSemaphore->signal();
    … …  
    534505    if (m_cdmInstance) {
    535506        if (auto instanceSession = m_cdmInstance->sessionForKeyIDs(keyIDs.value())) {
    536             if (auto parser = this->parser())
     507            if (auto parser = this->streamDataParser())
    537508                [instanceSession->contentKeySession() addContentKeyRecipient:parser];
    538509            if (m_hasSessionSemaphore) {
    … …  
    577548        m_client->sourceBufferPrivateReportExtraMemoryCost(totalTrackBufferSizeInBytes());
    578549
     550    m_parser->setDidParseInitializationDataCallback([weakThis = makeWeakPtr(this), abortCalled = m_abortCalled] (InitializationSegment&& segment) {
     551        ASSERT(isMainThread());
     552        if (!weakThis || abortCalled != weakThis->m_abortCalled)
     553            return;
     554        weakThis->didParseInitializationData(WTFMove(segment));
     555    });
     556
     557    m_parser->setDidEncounterErrorDuringParsingCallback([weakThis = makeWeakPtr(this), abortCalled = m_abortCalled] (int32_t errorCode) {
     558        ASSERT(isMainThread());
     559        if (!weakThis || abortCalled != weakThis->m_abortCalled)
     560            return;
     561        weakThis->didEncounterErrorDuringParsing(errorCode);
     562    });
     563
     564    m_parser->setDidProvideMediaDataCallback([weakThis = makeWeakPtr(this), abortCalled = m_abortCalled] (Ref<MediaSample>&& sample, uint64_t trackId, const String& mediaType) {
     565        ASSERT(isMainThread());
     566        if (!weakThis || abortCalled != weakThis->m_abortCalled)
     567            return;
     568        weakThis->didProvideMediaDataForTrackId(WTFMove(sample), trackId, mediaType);
     569    });
     570
    579571    m_abortSemaphore = Box<Semaphore>::create(0);
    580     m_parser->setWillProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis = makeWeakPtr(this), abortSemaphore = m_abortSemaphore] (uint64_t trackID) mutable {
     572    m_parser->setWillProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis = makeWeakPtr(this), abortSemaphore = m_abortSemaphore, abortCalled = m_abortCalled] (uint64_t trackID) mutable {
    581573        // We must call synchronously to the main thread, as the AVStreamSession must be associated
    582574        // with the streamDataParser before the delegate method returns.
    583575        Box<BinarySemaphore> respondedSemaphore = Box<BinarySemaphore>::create();
    584         callOnMainThread([weakThis = WTFMove(weakThis), trackID, respondedSemaphore]() {
    585             if (weakThis)
     576        callOnMainThread([weakThis = WTFMove(weakThis), abortCalled, trackID, respondedSemaphore]() {
     577            if (weakThis && abortCalled == weakThis->m_abortCalled)
    586578                weakThis->willProvideContentKeyRequestInitializationDataForTrackID(trackID);
    587579            respondedSemaphore->signal();
    … …  
    599591    });
    600592
    601     m_parser->setDidProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis = makeWeakPtr(this), abortSemaphore = m_abortSemaphore] (Ref<Uint8Array>&& initData, uint64_t trackID) mutable {
     593    m_parser->setDidProvideContentKeyRequestInitializationDataForTrackIDCallback([weakThis = makeWeakPtr(this), abortSemaphore = m_abortSemaphore, abortCalled = m_abortCalled](Ref<Uint8Array>&& initData, uint64_t trackID) mutable {
     594        // Called on the data parser queue.
    602595        Box<BinarySemaphore> hasSessionSemaphore = Box<BinarySemaphore>::create();
    603         callOnMainThread([weakThis = WTFMove(weakThis), initData = WTFMove(initData), trackID, hasSessionSemaphore] () mutable {
    604             if (weakThis)
    605                 weakThis->didProvideContentKeyRequestInitializationDataForTrackID(WTFMove(initData), trackID, hasSessionSemaphore);
     596        callOnMainThread([weakThis = WTFMove(weakThis), abortCalled, initData = WTFMove(initData), trackID, hasSessionSemaphore] () mutable {
     597            if (!weakThis || abortCalled != weakThis->m_abortCalled)
     598                return;
     599            weakThis->didProvideContentKeyRequestInitializationDataForTrackID(WTFMove(initData), trackID, hasSessionSemaphore);
    606600        });
    607601
    … …  
    620614    dispatch_group_enter(m_isAppendingGroup.get());
    621615
    622     dispatch_async(globalDataParserQueue(), [data = WTFMove(data), weakThis = m_appendWeakFactory.createWeakPtr(*this), parser = m_parser, isAppendingGroup = m_isAppendingGroup]() mutable {
    623         parser->appendData(WTFMove(data), [weakThis = WTFMove(weakThis)]() mutable {
    624             callOnMainThread([weakThis = WTFMove(weakThis)] {
    625                 if (!weakThis)
     616    dispatch_async(globalDataParserQueue(), [data = WTFMove(data), weakThis = m_appendWeakFactory.createWeakPtr(*this), parser = m_parser, isAppendingGroup = m_isAppendingGroup, abortCalled = m_abortCalled]() mutable {
     617        parser->appendData(WTFMove(data), [weakThis = WTFMove(weakThis), abortCalled]() mutable {
     618            callOnMainThread([weakThis = WTFMove(weakThis), abortCalled] {
     619                if (!weakThis || abortCalled != weakThis->m_abortCalled)
    626620                    return;
    627621
    628                 if (!weakThis->m_mediaSamples.isEmpty()) {
     622                if (weakThis->m_processingInitializationSegment) {
    629623                    weakThis->m_hasPendingAppendCompletedCallback = true;
    630624                    return;
    … …  
    661655    ALWAYS_LOG(LOGIDENTIFIER);
    662656
    663     // The parsing queue may be blocked waiting for the main thread to provide it a AVStreamSession. We
     657    // The parsing queue may be blocked waiting for the main thread to provide it an AVStreamSession. We
    664658    // were asked to abort, and that cancels all outstanding append operations. Without cancelling this
    665659    // semaphore, the m_isAppendingGroup wait operation will deadlock.
    … …  
    673667    }
    674668
     669    m_abortCalled++;
     670}
     671
     672void SourceBufferPrivateAVFObjC::resetParserState()
     673{
     674    ALWAYS_LOG(LOGIDENTIFIER);
     675
     676    dispatch_group_wait(m_isAppendingGroup.get(), DISPATCH_TIME_FOREVER);
     677    m_mediaSamples.clear();
     678    m_hasPendingAppendCompletedCallback = false;
     679    m_processingInitializationSegment = false;
    675680    m_parser->resetParserState();
    676     m_mediaSamples.clear();
    677     m_initializationSegmentIsHandled = false;
    678     m_mediaSampleTaskCancellationGroup.cancel();
    679 
    680     dispatch_group_wait(m_isAppendingGroup.get(), DISPATCH_TIME_FOREVER);
    681 }
    682 
    683 void SourceBufferPrivateAVFObjC::resetParserState()
    684 {
    685     ALWAYS_LOG(LOGIDENTIFIER);
    686 
    687     m_parser->resetParserState();
    688 }
    689 
    690 void SourceBufferPrivateAVFObjC::destroyParser()
    691 {
    692     auto parser = this->parser();
     681}
     682
     683void SourceBufferPrivateAVFObjC::destroyStreamDataParser()
     684{
     685    auto parser = this->streamDataParser();
    693686    if (!parser)
    694687        return;
    … …  
    743736
    744737    clearTrackBuffers();
    745     destroyParser();
     738    destroyStreamDataParser();
    746739    destroyRenderers();
    747740
    … …  
    841834}
    842835
    843 AVStreamDataParser* SourceBufferPrivateAVFObjC::parser() const
     836AVStreamDataParser* SourceBufferPrivateAVFObjC::streamDataParser() const
    844837{
    845838    if (is<SourceBufferParserAVFObjC>(m_parser.get()))
    846         return downcast<SourceBufferParserAVFObjC>(m_parser.get()).parser();
     839        return downcast<SourceBufferParserAVFObjC>(m_parser.get()).streamDataParser();
    847840    return nil;
    848841}
    … …  
    909902        return;
    910903
    911     if (auto parser = this->parser())
     904    if (auto parser = this->streamDataParser())
    912905        [instanceSession->contentKeySession() addContentKeyRecipient:parser];
    913906    if (m_hasSessionSemaphore) {
    … …  
    11651158                    }
    11661159
    1167                     callOnMainThread([weakThis = WTFMove(weakThis)] () mutable {
     1160                    callOnMainThread([weakThis = WTFMove(weakThis)] () {
    11681161                        if (!weakThis)
    11691162                            return;

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParser.cpp

    r279622 r279904  
    5454
    5555    return nullptr;
     56}
     57
     58static SourceBufferParser::CallOnClientThreadCallback callOnMainThreadCallback()
     59{
     60    return [](Function<void()>&& function) {
     61        callOnMainThread(WTFMove(function));
     62    };
     63}
     64
     65void SourceBufferParser::setCallOnClientThreadCallback(CallOnClientThreadCallback&& callback)
     66{
     67    ASSERT(callback);
     68    m_callOnClientThreadCallback = WTFMove(callback);
     69}
     70
     71SourceBufferParser::SourceBufferParser()
     72    : m_callOnClientThreadCallback(callOnMainThreadCallback())
     73{
    5674}
    5775

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParser.h

    r279622 r279904  
    8383    };
    8484
     85    using CallOnClientThreadCallback = WTF::Function<void(WTF::Function<void()>&&)>;
     86    void setCallOnClientThreadCallback(CallOnClientThreadCallback&&);
     87
     88    // appendData will be called on the SourceBufferPrivateAVFObjC data parser queue.
     89    // Other methods will be called on the main thread, but only once appendData has returned.
    8590    virtual void appendData(Segment&&, CompletionHandler<void()>&& = [] { }, AppendFlags = AppendFlags::None) = 0;
    8691    virtual void flushPendingMediaData() = 0;
    … …  
    131136
    132137protected:
    133     SourceBufferParser() = default;
     138    SourceBufferParser();
    134139
     140    CallOnClientThreadCallback m_callOnClientThreadCallback;
    135141    DidParseInitializationDataCallback m_didParseInitializationDataCallback;
    136142    DidEncounterErrorDuringParsingCallback m_didEncounterErrorDuringParsingCallback;

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp

    r279622 r279904  
    593593}
    594594
    595 static SourceBufferParserWebM::CallOnClientThreadCallback callOnMainThreadCallback()
    596 {
    597     return [](Function<void()>&& function) {
    598         callOnMainThread(WTFMove(function));
    599     };
    600 }
    601 
    602595SourceBufferParserWebM::SourceBufferParserWebM()
    603596    : m_reader(WTF::makeUniqueRef<StreamingVectorReader>())
    604     , m_callOnClientThreadCallback(callOnMainThreadCallback())
    605597{
    606598    if (isWebmParserAvailable())
    … …  
    690682{
    691683    INFO_LOG_IF_POSSIBLE(LOGIDENTIFIER);
    692     flushPendingAudioBuffers();
    693684    if (m_parser)
    694685        m_parser->DidSeek();
    695     m_state = State::None;
    696     m_tracks.clear();
     686    m_reader->reset();
     687    m_state = m_initializationSegmentProcessed ? State::ReadingSegment : State::None;
    697688    m_initializationSegment = nullptr;
    698689    m_initializationSegmentEncountered = false;
    699690    m_currentBlock.reset();
     691    for (auto& track : m_tracks)
     692        track->reset();
    700693}
    701694
    … …  
    794787        m_initializationSegmentEncountered = false;
    795788        m_initializationSegment = nullptr;
     789        m_initializationSegmentProcessed = true;
    796790
    797791        if (!m_keyIds.isEmpty()) {
    … …  
    815809    m_initializationSegmentEncountered = true;
    816810    m_initializationSegment = WTF::makeUniqueWithoutFastMallocCheck<InitializationSegment>();
     811    // TODO: Setting this to false here, will prevent adding a new media segment should a
     812    // partial init segment be encountered after a call to sourceBuffer.abort().
     813    // It's probably fine as no-one in their right mind should send partial init segment only
     814    // to immediately abort it. We do it this way mostly to avoid getting into a rabbit hole
     815    // of ensuring that libwebm does something sane with rubbish input.
     816    m_initializationSegmentProcessed = false;
    817817
    818818    return Status(Status::kOkCompleted);
    … …  
    10841084#define PARSER_LOG_ERROR_IF_POSSIBLE(...) if (parser().loggerPtr()) parser().loggerPtr()->error(logChannel(), WTF::Logger::LogSiteIdentifier(logClassName(), __func__, parser().logIdentifier()), __VA_ARGS__)
    10851085
     1086#if ENABLE(VP9)
     1087void SourceBufferParserWebM::VideoTrackData::reset()
     1088{
     1089    m_currentBlockBuffer = nullptr;
     1090    TrackData::reset();
     1091}
     1092#endif
     1093
    10861094webm::Status SourceBufferParserWebM::VideoTrackData::consumeFrameData(webm::Reader& reader, const FrameMetadata& metadata, uint64_t* bytesRemaining, const CMTime& presentationTime, int sampleCount)
    10871095{
    … …  
    11491157    createSampleBuffer(presentationTime, sampleCount, metadata);
    11501158
    1151     m_currentBlockBuffer = nullptr;
    1152     m_partialBytesRead = 0;
    1153     m_currentPacketSize = std::nullopt;
     1159    reset();
    11541160#else
    11551161    UNUSED_PARAM(metadata);
    … …  
    12401246    UNUSED_PARAM(metadata);
    12411247#endif // ENABLE(VP9)
     1248}
     1249
     1250void SourceBufferParserWebM::AudioTrackData::reset()
     1251{
     1252    m_packetDescriptions.clear();
     1253    m_packetsData.clear();
     1254    m_currentPacketByteOffset = std::nullopt;
     1255    TrackData::reset();
    12421256}
    12431257
    … …  
    14021416}
    14031417
    1404 void SourceBufferParserWebM::setCallOnClientThreadCallback(CallOnClientThreadCallback&& callback)
    1405 {
    1406     ASSERT(callback);
    1407     m_callOnClientThreadCallback = WTFMove(callback);
    1408 }
    1409 
    14101418const MemoryCompactLookupOnlyRobinHoodHashSet<String>& SourceBufferParserWebM::supportedVideoCodecs()
    14111419{

  • trunk/Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h

    r279622 r279904  
    8484    void setMinimumAudioSampleDuration(float);
    8585   
    86     using CallOnClientThreadCallback = WTF::Function<void(WTF::Function<void()>&&)>;
    87     void setCallOnClientThreadCallback(CallOnClientThreadCallback&&);
    88 
    8986    void setLogger(const WTF::Logger&, const void* identifier) final;
    9087
    … …  
    161158        }
    162159
     160        virtual void reset()
     161        {
     162            m_currentPacketSize = std::nullopt;
     163            m_partialBytesRead = 0;
     164        }
     165
    163166    protected:
    164167        std::optional<size_t> m_currentPacketSize;
    … …  
    186189        }
    187190
     191#if ENABLE(VP9)
     192        void reset() final;
     193#endif
    188194        webm::Status consumeFrameData(webm::Reader&, const webm::FrameMetadata&, uint64_t*, const CMTime&, int) final;
    189        
     195
    190196    private:
    191197        void createSampleBuffer(const CMTime&, int, const webm::FrameMetadata&);
    … …  
    212218
    213219        webm::Status consumeFrameData(webm::Reader&, const webm::FrameMetadata&, uint64_t*, const CMTime&, int) final;
     220        void reset() final;
    214221        void createSampleBuffer(std::optional<size_t> latestByteRangeOffset = std::nullopt);
    215222
    … …  
    264271    std::unique_ptr<webm::WebmParser> m_parser;
    265272    bool m_initializationSegmentEncountered { false };
     273    bool m_initializationSegmentProcessed { false };
    266274    uint32_t m_timescale { 1000 };
    267275    uint64_t m_currentTimecode { 0 };
    … …  
    277285    float m_minimumAudioSampleDuration { 2 };
    278286
    279     CallOnClientThreadCallback m_callOnClientThreadCallback;
    280 
    281287    RefPtr<const WTF::Logger> m_logger;
    282288    const void* m_logIdentifier { nullptr };
Note: See TracChangeset for help on using the changeset viewer.