Changeset 278603 in webkit
- Timestamp:
- Jun 8, 2021, 5:29:51 AM (5 years ago)
- Location:
- trunk/Source
- Files:
-
- 10 edited
-
WebCore/ChangeLog (modified) (1 diff)
-
WebCore/Modules/mediasource/SourceBuffer.cpp (modified) (2 diffs)
-
WebCore/platform/graphics/SourceBufferPrivate.cpp (modified) (9 diffs)
-
WebCore/platform/graphics/SourceBufferPrivate.h (modified) (5 diffs)
-
WebKit/ChangeLog (modified) (1 diff)
-
WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp (modified) (2 diffs)
-
WebKit/GPUProcess/media/RemoteSourceBufferProxy.h (modified) (1 diff)
-
WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in (modified) (1 diff)
-
WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp (modified) (2 diffs)
-
WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h (modified) (2 diffs)
Legend:
- Unmodified
- Added
- Removed
-
trunk/Source/WebCore/ChangeLog
r278602 r278603 1 2021-06-08 Jean-Yves Avenard <jya@apple.com> 2 3 [MSE] Rework handling of SourceBuffer's buffer full. 4 https://bugs.webkit.org/show_bug.cgi?id=226711 5 <rdar://problem/78937909> 6 7 Reviewed by Jer Noble. 8 9 Bug 225630 modified the handling of the Source Buffer "buffer full" algorithm. 10 Per spec, we are to reject a buffer only once we know that the source buffer is full. 11 The first appendBuffer should always complete. 12 https://w3c.github.io/media-source/#sourcebuffer-buffer-full-flag 13 "The buffer full flag keeps track of whether appendBuffer() is allowed to accept 14 more bytes. It is set to false when the SourceBuffer object is created and gets 15 updated as data is appended and removed." 16 17 "buffer full flag" only gets modified to true in the 3.5.1 Segment Parser Loop 18 algorithm, step 6.3 19 https://w3c.github.io/media-source/#sourcebuffer-segment-parser-loop 20 "If this SourceBuffer is full and cannot accept more media data, then set the 21 buffer full flag to true." 22 23 On the 2nd call to the appendBuffer, in the Prepare Append algorithm, step 3.5.4.6: 24 https://w3c.github.io/media-source/#sourcebuffer-prepare-append 25 26 "If the buffer full flag equals true, then throw a QuotaExceededError exception 27 and abort these steps." 28 29 This change moves the check of the source buffer size back into SourceBufferPrivate 30 so that checking if we have sufficient space or not is hidden from the SourceBuffer. 31 Rather than deal with a single "buffer full" flag, we use instead a isBufferFullFor() 32 method which allows to easily swap between the previous per-spec behaviour and the 33 one introduced by bug 225630 as it does present some advantages. 34 We can always determine if we have sufficient space by checking the current source 35 buffer memory size and the requested limit. 36 The previous algorithm took into consideration the allocated capacity of the 37 temporary SourceBuffer::m_pendingAppendData ; this approach was flawed in that we 38 always checked if m_pendingAppendData.capacity + newDataSize would fit in the 39 source buffer. However newDataSize would always happen to fit within the existing 40 capacity, so it was accounted for twice. 41 We remove check on the capacity allocated as it simplifies the code a great deal, 42 and avoid the piggy-backing of unrelated methods (such as 43 SourceBufferPrivate::reenqueueMediaIfNeeded that ended up also checking capacity) 44 45 This change is already covered with existing tests. 46 47 * Modules/mediasource/SourceBuffer.cpp: call new isBufferFullFor to check if 48 sufficient is available. 49 (WebCore::SourceBuffer::appendBufferInternal): 50 (WebCore::SourceBuffer::sourceBufferPrivateAppendComplete): 51 * platform/graphics/SourceBufferPrivate.h: Add isBufferFullFor method 52 remove m_bufferFull member. 53 * platform/graphics/SourceBufferPrivate.cpp: 54 (WebCore::SourceBufferPrivate::reenqueueMediaIfNeeded): remove reference to 55 buffer capacity 56 (WebCore::SourceBufferPrivate::evictCodedFrames): remove reference to 57 buffer capacity 58 (WebCore::SourceBufferPrivate::isBufferFullFor): To get per-spec behaviour 59 we only need to stop checking the value of the requiredSize argument. 60 1 61 2021-06-08 Antti Koivisto <antti@apple.com> 2 62 -
trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp
r278539 r278603 484 484 485 485 // 4. Run the coded frame eviction algorithm. 486 m_private->evictCodedFrames(size, m _pendingAppendData.size(), maximumBufferSize(), m_source->currentTime(), m_source->duration(), m_source->isEnded());486 m_private->evictCodedFrames(size, maximumBufferSize(), m_source->currentTime(), m_source->duration(), m_source->isEnded()); 487 487 488 488 // 5. If the buffer full flag equals true, then throw a QuotaExceededError exception and abort these step. 489 if (m_private-> bufferFull() || m_private->totalTrackBufferSizeInBytes() + m_pendingAppendData.size() + size >= maximumBufferSize()) {489 if (m_private->isBufferFullFor(size, maximumBufferSize())) { 490 490 ERROR_LOG(LOGIDENTIFIER, "buffer full, failing with QuotaExceededError error"); 491 491 return Exception { QuotaExceededError }; … … 574 574 575 575 m_source->monitorSourceBuffers(); 576 m_private->reenqueueMediaIfNeeded(m_source->currentTime() , m_pendingAppendData.capacity(), maximumBufferSize());576 m_private->reenqueueMediaIfNeeded(m_source->currentTime()); 577 577 578 578 DEBUG_LOG(LOGIDENTIFIER); -
trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
r278147 r278603 36 36 #include "SourceBufferPrivateClient.h" 37 37 #include "TimeRanges.h" 38 #include <wtf/CheckedArithmetic.h> 38 39 #include <wtf/MediaTime.h> 39 40 #include <wtf/StringPrintStream.h> … … 420 421 } 421 422 422 void SourceBufferPrivate::reenqueueMediaIfNeeded(const MediaTime& currentTime , uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize)423 void SourceBufferPrivate::reenqueueMediaIfNeeded(const MediaTime& currentTime) 423 424 { 424 425 for (auto& trackBufferPair : m_trackBufferMap) { … … 432 433 provideMediaData(trackBuffer, trackID); 433 434 } 434 435 if (totalTrackBufferSizeInBytes() + pendingAppendDataCapacity > maximumBufferSize)436 m_bufferFull = true;437 435 } 438 436 … … 629 627 } 630 628 631 void SourceBufferPrivate::evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_tmaximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded)629 void SourceBufferPrivate::evictCodedFrames(uint64_t newDataSize, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded) 632 630 { 633 631 // 3.5.13 Coded Frame Eviction Algorithm … … 640 638 // 1. Let new data equal the data that is about to be appended to this SourceBuffer. 641 639 // 2. If the buffer full flag equals false, then abort these steps. 642 if (! m_bufferFull && totalTrackBufferSizeInBytes() + pendingAppendDataCapacity + newDataSize < maximumBufferSize)640 if (!isBufferFullFor(newDataSize, maximumBufferSize)) 643 641 return; 644 642 … … 662 660 // end equal to the removal range start and end timestamp respectively. 663 661 removeCodedFrames(rangeStart, std::min(rangeEnd, maximumRangeEnd), currentTime, isEnded); 664 if (totalTrackBufferSizeInBytes() + pendingAppendDataCapacity + newDataSize < maximumBufferSize) { 665 m_bufferFull = false; 662 if (!isBufferFullFor(newDataSize, maximumBufferSize)) { 666 663 break; 667 664 } … … 671 668 } 672 669 673 if (! m_bufferFull) {670 if (!isBufferFullFor(newDataSize, maximumBufferSize)) { 674 671 #if !RELEASE_LOG_DISABLED 675 672 DEBUG_LOG(LOGIDENTIFIER, "evicted ", initialBufferedSize - totalTrackBufferSizeInBytes()); … … 706 703 removeCodedFrames(std::max(minimumRangeStart, rangeStart), rangeEnd, currentTime, isEnded); 707 704 708 if (totalTrackBufferSizeInBytes() + pendingAppendDataCapacity + newDataSize < maximumBufferSize) { 709 m_bufferFull = false; 705 if (!isBufferFullFor(newDataSize, maximumBufferSize)) { 710 706 break; 711 707 } … … 716 712 717 713 #if !RELEASE_LOG_DISABLED 718 if ( m_bufferFull)714 if (isBufferFullFor(newDataSize, maximumBufferSize)) 719 715 ERROR_LOG(LOGIDENTIFIER, "FAILED to free enough after evicting ", initialBufferedSize - totalTrackBufferSizeInBytes()); 720 716 else 721 717 DEBUG_LOG(LOGIDENTIFIER, "evicted ", initialBufferedSize - totalTrackBufferSizeInBytes()); 722 718 #endif 719 } 720 721 bool SourceBufferPrivate::isBufferFullFor(uint64_t requiredSize, uint64_t maximumBufferSize) 722 { 723 auto totalRequired = checkedSum<uint64_t>(totalTrackBufferSizeInBytes(), requiredSize); 724 if (totalRequired.hasOverflowed()) 725 return true; 726 727 return totalRequired >= maximumBufferSize; 723 728 } 724 729 -
trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h
r278539 r278603 78 78 WEBCORE_EXPORT virtual void setMediaSourceEnded(bool); 79 79 virtual void setMode(SourceBufferAppendMode mode) { m_appendMode = mode; } 80 WEBCORE_EXPORT virtual void reenqueueMediaIfNeeded(const MediaTime& currentMediaTime , uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize);80 WEBCORE_EXPORT virtual void reenqueueMediaIfNeeded(const MediaTime& currentMediaTime); 81 81 WEBCORE_EXPORT virtual void addTrackBuffer(const AtomString& trackId, RefPtr<MediaDescription>&&); 82 82 WEBCORE_EXPORT virtual void resetTrackBuffers(); … … 88 88 WEBCORE_EXPORT virtual void updateBufferedFromTrackBuffers(bool sourceIsEnded); 89 89 WEBCORE_EXPORT virtual void removeCodedFrames(const MediaTime& start, const MediaTime& end, const MediaTime& currentMediaTime, bool isEnded, CompletionHandler<void()>&& = [] { }); 90 WEBCORE_EXPORT virtual void evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_tmaximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded);90 WEBCORE_EXPORT virtual void evictCodedFrames(uint64_t newDataSize, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded); 91 91 WEBCORE_EXPORT virtual uint64_t totalTrackBufferSizeInBytes() const; 92 92 WEBCORE_EXPORT virtual void resetTimestampOffsetInTrackBuffers(); … … 103 103 const TimeRanges* buffered() const { return m_buffered.get(); } 104 104 105 bool bufferFull() const { return m_bufferFull; }105 bool isBufferFullFor(uint64_t requiredSize, uint64_t maximumBufferSize); 106 106 107 107 // Methods used by MediaSourcePrivate … … 168 168 WEBCORE_EXPORT void didReceiveSample(Ref<MediaSample>&&); 169 169 WEBCORE_EXPORT void setBufferedRanges(const PlatformTimeRanges&); 170 void setBufferFull(bool bufferFull) { m_bufferFull = bufferFull; }171 170 void provideMediaData(const AtomString& trackID); 172 171 … … 202 201 MediaTime m_groupEndTimestamp { MediaTime::zeroTime() }; 203 202 204 bool m_bufferFull { false };205 203 bool m_isMediaSourceEnded { false }; 206 204 RefPtr<TimeRanges> m_buffered; -
trunk/Source/WebKit/ChangeLog
r278597 r278603 1 2021-06-08 Jean-Yves Avenard <jya@apple.com> 2 3 [MSE] Rework handling of SourceBuffer's buffer full. 4 https://bugs.webkit.org/show_bug.cgi?id=226711 5 <rdar://problem/78937909> 6 7 Reviewed by Jer Noble. 8 9 * GPUProcess/media/RemoteSourceBufferProxy.cpp: 10 (WebKit::RemoteSourceBufferProxy::evictCodedFrames): 11 (WebKit::RemoteSourceBufferProxy::reenqueueMediaIfNeeded): 12 * GPUProcess/media/RemoteSourceBufferProxy.h: 13 * GPUProcess/media/RemoteSourceBufferProxy.messages.in: Remove capacity and 14 buffer full references from all methods; relying on the totalTrackBufferSizeInBytes 15 instead. 16 * WebProcess/GPU/media/SourceBufferPrivateRemote.cpp: 17 (WebKit::SourceBufferPrivateRemote::evictCodedFrames): 18 (WebKit::SourceBufferPrivateRemote::reenqueueMediaIfNeeded): 19 * WebProcess/GPU/media/SourceBufferPrivateRemote.h: 20 1 21 2021-06-08 Carlos Garcia Campos <cgarcia@igalia.com> 2 22 -
trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp
r278539 r278603 258 258 } 259 259 260 void RemoteSourceBufferProxy::evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_tmaximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded, EvictCodedFramesDelayedReply&& completionHandler)261 { 262 m_sourceBufferPrivate->evictCodedFrames(newDataSize, pendingAppendDataCapacity,maximumBufferSize, currentTime, duration, isEnded);263 completionHandler(m_sourceBufferPrivate-> bufferFull(), m_sourceBufferPrivate->totalTrackBufferSizeInBytes());260 void RemoteSourceBufferProxy::evictCodedFrames(uint64_t newDataSize, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded, EvictCodedFramesDelayedReply&& completionHandler) 261 { 262 m_sourceBufferPrivate->evictCodedFrames(newDataSize, maximumBufferSize, currentTime, duration, isEnded); 263 completionHandler(m_sourceBufferPrivate->totalTrackBufferSizeInBytes()); 264 264 } 265 265 … … 286 286 } 287 287 288 void RemoteSourceBufferProxy::reenqueueMediaIfNeeded(const MediaTime& currentMediaTime, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize, CompletionHandler<void(bool)>&& completionHandler) 289 { 290 m_sourceBufferPrivate->reenqueueMediaIfNeeded(currentMediaTime, pendingAppendDataCapacity, maximumBufferSize); 291 completionHandler(m_sourceBufferPrivate->bufferFull()); 288 void RemoteSourceBufferProxy::reenqueueMediaIfNeeded(const MediaTime& currentMediaTime) 289 { 290 m_sourceBufferPrivate->reenqueueMediaIfNeeded(currentMediaTime); 292 291 } 293 292 -
trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h
r278539 r278603 99 99 using EvictCodedFramesDelayedReply= Messages::RemoteSourceBufferProxy::EvictCodedFramesDelayedReply; 100 100 void removeCodedFrames(const MediaTime& start, const MediaTime& end, const MediaTime& currentTime, bool isEnded, RemoveCodedFramesAsyncReply&&); 101 void evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_tmaximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded, EvictCodedFramesDelayedReply&&);101 void evictCodedFrames(uint64_t newDataSize, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded, EvictCodedFramesDelayedReply&&); 102 102 void addTrackBuffer(TrackPrivateRemoteIdentifier); 103 103 void resetTrackBuffers(); 104 104 void clearTrackBuffers(); 105 105 void setAllTrackBuffersNeedRandomAccess(); 106 void reenqueueMediaIfNeeded(const MediaTime& currentMediaTime , uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize, CompletionHandler<void(bool)>&&);106 void reenqueueMediaIfNeeded(const MediaTime& currentMediaTime); 107 107 void setGroupStartTimestamp(const MediaTime&); 108 108 void setGroupStartTimestampToEndTimestamp(); -
trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in
r278539 r278603 43 43 SetAllTrackBuffersNeedRandomAccess() 44 44 RemoveCodedFrames(MediaTime start, MediaTime end, MediaTime currentTime, bool isEnded) -> (WebCore::PlatformTimeRanges buffered, uint64_t totalTrackBufferSizeInBytes) Async 45 EvictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize, MediaTime currentTime, MediaTime duration, bool isEnded) -> (bool bufferFull,uint64_t totalTrackBufferSizeInBytes) Synchronous46 ReenqueueMediaIfNeeded(MediaTime currentMediaTime , uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize) -> (bool bufferFull) Async45 EvictCodedFrames(uint64_t newDataSize, uint64_t maximumBufferSize, MediaTime currentTime, MediaTime duration, bool isEnded) -> (uint64_t totalTrackBufferSizeInBytes) Synchronous 46 ReenqueueMediaIfNeeded(MediaTime currentMediaTime) 47 47 SetGroupStartTimestamp(MediaTime timestamp) 48 48 SetGroupStartTimestampToEndTimestamp() -
trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp
r278539 r278603 201 201 } 202 202 203 void SourceBufferPrivateRemote::evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded) 204 { 205 if (!m_gpuProcessConnection) 206 return; 207 208 bool bufferFull = false; 203 void SourceBufferPrivateRemote::evictCodedFrames(uint64_t newDataSize, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded) 204 { 205 if (!m_gpuProcessConnection) 206 return; 207 209 208 uint64_t totalBufferSizeInBytes = 0; 210 if (m_gpuProcessConnection->connection().sendSync(Messages::RemoteSourceBufferProxy::EvictCodedFrames(newDataSize, pendingAppendDataCapacity, maximumBufferSize, currentTime, duration, isEnded), Messages::RemoteSourceBufferProxy::EvictCodedFrames::Reply(bufferFull, totalBufferSizeInBytes), m_remoteSourceBufferIdentifier)) { 211 setBufferFull(bufferFull); 209 if (m_gpuProcessConnection->connection().sendSync(Messages::RemoteSourceBufferProxy::EvictCodedFrames(newDataSize, maximumBufferSize, currentTime, duration, isEnded), Messages::RemoteSourceBufferProxy::EvictCodedFrames::Reply(totalBufferSizeInBytes), m_remoteSourceBufferIdentifier)) { 212 210 m_totalTrackBufferSizeInBytes = totalBufferSizeInBytes; 213 211 } … … 272 270 } 273 271 274 void SourceBufferPrivateRemote::reenqueueMediaIfNeeded(const MediaTime& currentMediaTime, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize) 275 { 276 if (!m_gpuProcessConnection) 277 return; 278 279 m_gpuProcessConnection->connection().sendWithAsyncReply(Messages::RemoteSourceBufferProxy::ReenqueueMediaIfNeeded(currentMediaTime, pendingAppendDataCapacity, maximumBufferSize), [this, protectedThis = makeRef(*this)](auto bufferFull) mutable { 280 setBufferFull(bufferFull); 281 }, m_remoteSourceBufferIdentifier); 272 void SourceBufferPrivateRemote::reenqueueMediaIfNeeded(const MediaTime& currentMediaTime) 273 { 274 if (!m_gpuProcessConnection) 275 return; 276 277 m_gpuProcessConnection->connection().send(Messages::RemoteSourceBufferProxy::ReenqueueMediaIfNeeded(currentMediaTime), m_remoteSourceBufferIdentifier); 282 278 } 283 279 -
trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h
r278539 r278603 82 82 void setMediaSourceEnded(bool) final; 83 83 void setMode(WebCore::SourceBufferAppendMode) final; 84 void reenqueueMediaIfNeeded(const MediaTime& currentMediaTime , uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize) final;84 void reenqueueMediaIfNeeded(const MediaTime& currentMediaTime) final; 85 85 void addTrackBuffer(const AtomString& trackId, RefPtr<WebCore::MediaDescription>&&) final; 86 86 void resetTrackBuffers() final; … … 92 92 void updateBufferedFromTrackBuffers(bool sourceIsEnded) final; 93 93 void removeCodedFrames(const MediaTime& start, const MediaTime& end, const MediaTime& currentMediaTime, bool isEnded, CompletionHandler<void()>&&) final; 94 void evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_tmaximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded) final;94 void evictCodedFrames(uint64_t newDataSize, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded) final; 95 95 void resetTimestampOffsetInTrackBuffers() final; 96 96 void startChangingType() final;
Note:
See TracChangeset
for help on using the changeset viewer.