Changeset 172657 in webkit


Ignore:
Timestamp:
Aug 15, 2014 4:09:19 PM (10 years ago)
Author:
eric.carlson@apple.com
Message:

[MSE] Implement a maximum buffer size for SourceBuffer
https://bugs.webkit.org/show_bug.cgi?id=135614

Reviewed by Jer Noble.

Implement the MSE coded frame eviction algorithm: when new buffers are appended attempt
to keep the amount of data buffered below a maximum size (which is determined by the media
session manager) by freeing coded frames as follows:

1 - Free frames in 30 second chunks from 0 to 30 seconds before the current time.
2 - If there are time ranges after the range with the current time, free frames in

30 second chunks from duration back to the beginning of the time range after
current time.

For now we DO NOT throw a QUOTA_EXCEEDED_ERR when we are not able to free up enough
space to remain below the prescribed quota, because some big name, widely deployed,
code bases ignore the error and continue appending data as though the failed append
succeeded, leading to a corrupted bitstream and failure to play.

  • Modules/mediasource/SampleMap.cpp:

(WebCore::SampleMap::addSample): Drive-by performance optimization: sample->presentationTime()

is used more than once, stash it in a local variable.

(WebCore::SampleMap::removeSample): Ditto.

  • Modules/mediasource/SourceBuffer.cpp:

(WebCore::logRanges): Debug-only function to log TimeRanges.
(WebCore::SourceBuffer::SourceBuffer):
(WebCore::SourceBuffer::remove): Optimize logging. Buffer full and coded frame eviction logic

is in SourceBuffer instead of SourceBufferPrivate.

(WebCore::SourceBuffer::appendBufferInternal): Ditto.
(WebCore::SourceBuffer::sourceBufferPrivateAppendComplete): Set m_bufferFull when more data

has been buffered than allowed.

(WebCore::removeSamplesFromTrackBuffer): Add logging.
(WebCore::SourceBuffer::removeCodedFrames): Improve logging. Avoid debug build assert when

current time is after last enqueued presentation time.

(WebCore::SourceBuffer::evictCodedFrames): The coded frame eviction algorithm.
(WebCore::SourceBuffer::maximumBufferSize): Return the maximum amount of data that can

be buffered.

(WebCore::SourceBuffer::sourceBufferPrivateDidReceiveSample): Improve logging. Don't attempt

to create a PlatformTimeRanges with inverted start and end to avoid an assert in debug.

(WebCore::SourceBuffer::hasAudio): New, return true if there are any audio tracks.
(WebCore::SourceBuffer::hasVideo): New, return true if there are any video tracks.
(WebCore::SourceBuffer::sourceBufferPrivateHasAudio): Call hasAudio.
(WebCore::SourceBuffer::sourceBufferPrivateHasVideo): Call hasVideo.
(WebCore::SourceBuffer::hasCurrentTime): Return true if currentTime is greater than duration.
(WebCore::SourceBuffer::hasFutureTime): Ditto.
(WebCore::SourceBuffer::extraMemoryCost): Return the amount of data buffered: the size of

the input buffer plus the size of all track samples.

(WebCore::SourceBuffer::reportExtraMemoryCost): Move buffered size calculation to extraMemoryCost.
(WebCore::SourceBuffer::document): Document accessor.

  • Modules/mediasource/SourceBuffer.h: Drive-by size optimization by moving all bool member

variables to the end of class.

  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::refreshCachedTime): Drive-by removal of overly chatty logging.
(WebCore::HTMLMediaElement::maximumSourceBufferSize): New, maximum source buffer size.

  • html/HTMLMediaElement.h:
  • html/HTMLMediaSession.cpp:

(WebCore::HTMLMediaSession::maximumMediaSourceBufferSize): New, get maximum source buffer

size from settings, return full amount for an SourceBuffer with video and audio, return 5%
of the maximum for an audio-only SourceBuffer.

  • html/HTMLMediaSession.h:
  • page/Settings.in: Add maximumSourceBufferSize. Default value is enough for approximately

five minutes of 1080p video and stereo audio.

  • platform/graphics/PlatformTimeRanges.cpp:

(WebCore::PlatformTimeRanges::totalDuration): Drive-by optimization.
(WebCore::PlatformTimeRanges::dump): New, allow a PlatformTimeRanges to be printed.

  • platform/graphics/PlatformTimeRanges.h:
  • platform/graphics/SourceBufferPrivate.h: Delete evictCodedFrames and isFull, that logic

is not in SourceBuffer.

  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
  • platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:

(WebCore::SourceBufferPrivateAVFObjC::processCodedFrame): Drive-by logging fix.
(WebCore::SourceBufferPrivateAVFObjC::evictCodedFrames): Deleted.
(WebCore::SourceBufferPrivateAVFObjC::isFull): Deleted.

  • platform/mock/mediasource/MockSourceBufferPrivate.cpp:

(WebCore::MockSourceBufferPrivate::evictCodedFrames): Deleted.
(WebCore::MockSourceBufferPrivate::isFull): Deleted.

  • platform/mock/mediasource/MockSourceBufferPrivate.h:
Location:
trunk/Source/WebCore
Files:
16 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebCore/ChangeLog

    r172656 r172657  
     12014-08-15  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [MSE] Implement a maximum buffer size for SourceBuffer
     4        https://bugs.webkit.org/show_bug.cgi?id=135614
     5
     6        Reviewed by Jer Noble.
     7
     8        Implement the MSE coded frame eviction algorithm: when new buffers are appended attempt
     9        to keep the amount of data buffered below a maximum size (which is determined by the media
     10        session manager) by freeing coded frames as follows:
     11            1 - Free frames in 30 second chunks from 0 to 30 seconds before the current time.
     12            2 - If there are time ranges after the range with the current time, free frames in
     13                30 second chunks from duration back to the beginning of the time range after
     14                current time.
     15
     16        For now we DO NOT throw a QUOTA_EXCEEDED_ERR when we are not able to free up enough
     17        space to remain below the prescribed quota, because some big name, widely deployed,
     18        code bases ignore the error and continue appending data as though the failed append
     19        succeeded, leading to a corrupted bitstream and failure to play.
     20
     21        * Modules/mediasource/SampleMap.cpp:
     22        (WebCore::SampleMap::addSample): Drive-by performance optimization: sample->presentationTime()
     23            is used more than once, stash it in a local variable.
     24        (WebCore::SampleMap::removeSample): Ditto.
     25
     26        * Modules/mediasource/SourceBuffer.cpp:
     27        (WebCore::logRanges): Debug-only function to log TimeRanges.
     28        (WebCore::SourceBuffer::SourceBuffer):
     29        (WebCore::SourceBuffer::remove): Optimize logging. Buffer full and coded frame eviction logic
     30            is in SourceBuffer instead of SourceBufferPrivate.
     31        (WebCore::SourceBuffer::appendBufferInternal): Ditto.
     32        (WebCore::SourceBuffer::sourceBufferPrivateAppendComplete): Set m_bufferFull when more data
     33            has been buffered than allowed.
     34        (WebCore::removeSamplesFromTrackBuffer): Add logging.
     35        (WebCore::SourceBuffer::removeCodedFrames): Improve logging. Avoid debug build assert when
     36            current time is after last enqueued presentation time.
     37        (WebCore::SourceBuffer::evictCodedFrames): The coded frame eviction algorithm.
     38        (WebCore::SourceBuffer::maximumBufferSize): Return the maximum amount of data that can
     39            be buffered.
     40        (WebCore::SourceBuffer::sourceBufferPrivateDidReceiveSample): Improve logging. Don't attempt
     41            to create a PlatformTimeRanges with inverted start and end to avoid an assert in debug.
     42        (WebCore::SourceBuffer::hasAudio): New, return true if there are any audio tracks.
     43        (WebCore::SourceBuffer::hasVideo): New, return true if there are any video tracks.
     44        (WebCore::SourceBuffer::sourceBufferPrivateHasAudio): Call hasAudio.
     45        (WebCore::SourceBuffer::sourceBufferPrivateHasVideo): Call hasVideo.
     46        (WebCore::SourceBuffer::hasCurrentTime): Return true if currentTime is greater than duration.
     47        (WebCore::SourceBuffer::hasFutureTime): Ditto.
     48        (WebCore::SourceBuffer::extraMemoryCost): Return the amount of data buffered: the size of
     49            the input buffer plus the size of all track samples.
     50        (WebCore::SourceBuffer::reportExtraMemoryCost): Move buffered size calculation to extraMemoryCost.
     51        (WebCore::SourceBuffer::document): Document accessor.
     52        * Modules/mediasource/SourceBuffer.h: Drive-by size optimization by moving all bool member
     53            variables to the end of class.
     54
     55        * html/HTMLMediaElement.cpp:
     56        (WebCore::HTMLMediaElement::refreshCachedTime): Drive-by removal of overly chatty logging.
     57        (WebCore::HTMLMediaElement::maximumSourceBufferSize): New, maximum source buffer size.
     58        * html/HTMLMediaElement.h:
     59
     60        * html/HTMLMediaSession.cpp:
     61        (WebCore::HTMLMediaSession::maximumMediaSourceBufferSize): New, get maximum source buffer
     62            size from settings, return full amount for an SourceBuffer with video and audio, return 5%
     63            of the maximum for an audio-only SourceBuffer.
     64        * html/HTMLMediaSession.h:
     65
     66        * page/Settings.in: Add maximumSourceBufferSize. Default value is enough for approximately
     67            five minutes of 1080p video and stereo audio.
     68
     69        * platform/graphics/PlatformTimeRanges.cpp:
     70        (WebCore::PlatformTimeRanges::totalDuration): Drive-by optimization.
     71        (WebCore::PlatformTimeRanges::dump): New, allow a PlatformTimeRanges to be printed.
     72        * platform/graphics/PlatformTimeRanges.h:
     73
     74        * platform/graphics/SourceBufferPrivate.h: Delete evictCodedFrames and isFull, that logic
     75            is not in SourceBuffer.
     76        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
     77        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
     78        (WebCore::SourceBufferPrivateAVFObjC::processCodedFrame): Drive-by logging fix.
     79        (WebCore::SourceBufferPrivateAVFObjC::evictCodedFrames): Deleted.
     80        (WebCore::SourceBufferPrivateAVFObjC::isFull): Deleted.
     81
     82        * platform/mock/mediasource/MockSourceBufferPrivate.cpp:
     83        (WebCore::MockSourceBufferPrivate::evictCodedFrames): Deleted.
     84        (WebCore::MockSourceBufferPrivate::isFull): Deleted.
     85        * platform/mock/mediasource/MockSourceBufferPrivate.h:
     86
    1872014-08-15  Zalan Bujtas  <zalan@apple.com>
    288
  • trunk/Source/WebCore/Modules/mediasource/SampleMap.cpp

    r171616 r172657  
    114114    ASSERT(sample);
    115115
    116     presentationOrder().m_samples.insert(PresentationOrderSampleMap::MapType::value_type(sample->presentationTime(), sample));
    117 
    118     auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), sample->presentationTime());
     116    MediaTime presentationTime = sample->presentationTime();
     117
     118    presentationOrder().m_samples.insert(PresentationOrderSampleMap::MapType::value_type(presentationTime, sample));
     119
     120    auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), presentationTime);
    119121    decodeOrder().m_samples.insert(DecodeOrderSampleMap::MapType::value_type(decodeKey, sample));
    120122
     
    125127{
    126128    ASSERT(sample);
    127     presentationOrder().m_samples.erase(sample->presentationTime());
    128 
    129     auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), sample->presentationTime());
     129    MediaTime presentationTime = sample->presentationTime();
     130
     131    presentationOrder().m_samples.erase(presentationTime);
     132
     133    auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), presentationTime);
    130134    decodeOrder().m_samples.erase(decodeKey);
    131135
  • trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp

    r172584 r172657  
    11/*
    22 * Copyright (C) 2013 Google Inc. All rights reserved.
     3 * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
    34 *
    45 * Redistribution and use in source and binary forms, with or without
     
    4950#include "TimeRanges.h"
    5051#include "VideoTrackList.h"
     52#include <limits>
    5153#include <map>
    5254#include <runtime/JSCInlines.h>
     
    5557#include <wtf/CurrentTime.h>
    5658#include <wtf/NeverDestroyed.h>
     59#if !LOG_DISABLED
     60#include <wtf/text/StringBuilder.h>
     61#endif
    5762
    5863namespace WebCore {
     
    105110    , m_source(source)
    106111    , m_asyncEventQueue(*this)
    107     , m_updating(false)
    108112    , m_appendBufferTimer(this, &SourceBuffer::appendBufferTimerFired)
    109113    , m_highestPresentationEndTimestamp(MediaTime::invalidTime())
    110     , m_receivedFirstInitializationSegment(false)
    111114    , m_buffered(TimeRanges::create())
    112     , m_active(false)
    113115    , m_appendState(WaitingForSegment)
    114116    , m_timeOfBufferingMonitor(monotonicallyIncreasingTime())
     
    119121    , m_pendingRemoveEnd(MediaTime::invalidTime())
    120122    , m_removeTimer(this, &SourceBuffer::removeTimerFired)
     123    , m_updating(false)
     124    , m_receivedFirstInitializationSegment(false)
     125    , m_active(false)
     126    , m_bufferFull(false)
    121127{
    122128    ASSERT(m_source);
     
    236242void SourceBuffer::remove(double start, double end, ExceptionCode& ec)
    237243{
    238     LOG(MediaSource, "SourceBuffer::remove(%p) - start(%s), end(%s)", this, toString(start).utf8().data(), toString(end).utf8().data());
     244    LOG(MediaSource, "SourceBuffer::remove(%p) - start(%lf), end(%lf)", this, start, end);
     245
    239246    // Section 3.2 remove() method steps.
    240247    // 1. If start is negative or greater than duration, then throw an InvalidAccessError exception and abort these steps.
     
    408415
    409416    // 4. Run the coded frame eviction algorithm.
    410     m_private->evictCodedFrames();
    411 
     417    evictCodedFrames(size);
     418
     419    // FIXME: enable this code when MSE libraries have been updated to support it.
     420#if 0
    412421    // 5. If the buffer full flag equals true, then throw a QUOTA_EXCEEDED_ERR exception and abort these step.
    413     if (m_private->isFull()) {
     422    if (m_bufferFull) {
     423        LOG(MediaSource, "SourceBuffer::appendBufferInternal(%p) -  buffer full, failing with QUOTA_EXCEEDED_ERR error", this);
    414424        ec = QUOTA_EXCEEDED_ERR;
    415425        return;
    416426    }
     427#endif
    417428
    418429    // NOTE: Return to 3.2 appendBuffer()
     
    515526
    516527    reportExtraMemoryCost();
     528    if (extraMemoryCost() > this->maximumBufferSize())
     529        m_bufferFull = true;
     530
     531    LOG(Media, "SourceBuffer::sourceBufferPrivateAppendComplete(%p) - buffered = %s", this, toString(m_buffered->ranges()).utf8().data());
    517532}
    518533
     
    528543}
    529544
    530 static PassRefPtr<TimeRanges> removeSamplesFromTrackBuffer(const DecodeOrderSampleMap::MapType& samples, SourceBuffer::TrackBuffer& trackBuffer)
    531 {
     545static PassRefPtr<TimeRanges> removeSamplesFromTrackBuffer(const DecodeOrderSampleMap::MapType& samples, SourceBuffer::TrackBuffer& trackBuffer, const SourceBuffer* buffer, const char* logPrefix)
     546{
     547#if !LOG_DISABLED
     548    double earliestSample = std::numeric_limits<double>::infinity();
     549    double latestSample = 0;
     550    size_t bytesRemoved = 0;
     551#else
     552    UNUSED_PARAM(logPrefix);
     553    UNUSED_PARAM(buffer);
     554#endif
     555
    532556    RefPtr<TimeRanges> erasedRanges = TimeRanges::create();
    533557    MediaTime microsecond(1, 1000000);
    534558    for (auto sampleIt : samples) {
    535559        const DecodeOrderSampleMap::KeyType& decodeKey = sampleIt.first;
     560#if !LOG_DISABLED
     561        size_t startBufferSize = trackBuffer.samples.sizeInBytes();
     562#endif
     563
    536564        RefPtr<MediaSample>& sample = sampleIt.second;
     565        LOG(MediaSource, "SourceBuffer::%s(%p) - removing sample(%s)", logPrefix, buffer, toString(*sampleIt.second).utf8().data());
    537566
    538567        // Remove the erased samples from the TrackBuffer sample map.
     
    545574        double endTime = startTime + (sample->duration() + microsecond).toDouble();
    546575        erasedRanges->add(startTime, endTime);
    547     }
     576
     577#if !LOG_DISABLED
     578        bytesRemoved += startBufferSize - trackBuffer.samples.sizeInBytes();
     579        if (startTime < earliestSample)
     580            earliestSample = startTime;
     581        if (endTime > latestSample)
     582            latestSample = endTime;
     583#endif
     584    }
     585
     586#if !LOG_DISABLED
     587    if (bytesRemoved)
     588        LOG(MediaSource, "SourceBuffer::%s(%p) removed %zu bytes, start(%lf), end(%lf)", logPrefix, buffer, bytesRemoved, earliestSample, latestSample);
     589#endif
     590
    548591    return erasedRanges.release();
    549592}
     
    591634
    592635        DecodeOrderSampleMap::MapType erasedSamples(removeDecodeStart, removeDecodeEnd);
    593         RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(erasedSamples, trackBuffer);
     636        RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(erasedSamples, trackBuffer, this, "removeCodedFrames");
    594637
    595638        // Only force the TrackBuffer to re-enqueue if the removed ranges overlap with enqueued and possibly
    596639        // not yet displayed samples.
    597         PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
    598         possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
    599         if (possiblyEnqueuedRanges.length())
    600             trackBuffer.needsReenqueueing = true;
     640        if (currentMediaTime < trackBuffer.lastEnqueuedPresentationTime) {
     641            PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
     642            possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
     643            if (possiblyEnqueuedRanges.length())
     644                trackBuffer.needsReenqueueing = true;
     645        }
    601646
    602647        erasedRanges->invert();
     
    612657    // 4. If buffer full flag equals true and this object is ready to accept more bytes, then set the buffer full flag to false.
    613658    // No-op
     659
     660    LOG(Media, "SourceBuffer::removeCodedFrames(%p) - buffered = %s", this, toString(m_buffered->ranges()).utf8().data());
    614661}
    615662
     
    636683    // 12. Queue a task to fire a simple event named updateend at this SourceBuffer object.
    637684    scheduleEvent(eventNames().updateendEvent);
     685}
     686
     687void SourceBuffer::evictCodedFrames(size_t newDataSize)
     688{
     689    // 3.5.13 Coded Frame Eviction Algorithm
     690    // http://www.w3.org/TR/media-source/#sourcebuffer-coded-frame-eviction
     691
     692    if (isRemoved())
     693        return;
     694
     695    // This algorithm is run to free up space in this source buffer when new data is appended.
     696    // 1. Let new data equal the data that is about to be appended to this SourceBuffer.
     697    // 2. If the buffer full flag equals false, then abort these steps.
     698    if (!m_bufferFull)
     699        return;
     700
     701    size_t maximumBufferSize = this->maximumBufferSize();
     702
     703    // 3. Let removal ranges equal a list of presentation time ranges that can be evicted from
     704    // the presentation to make room for the new data.
     705
     706    // NOTE: begin by removing data from the beginning of the buffered ranges, 30 seconds at
     707    // a time, up to 30 seconds before currentTime.
     708    MediaTime thirtySeconds = MediaTime(30, 1);
     709    MediaTime currentTime = MediaTime::createWithDouble(m_source->currentTime());
     710    MediaTime maximumRangeEnd = currentTime - thirtySeconds;
     711
     712#if !LOG_DISABLED
     713    LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - currentTime = %lf, require %zu bytes, maximum buffer size is %zu", this, m_source->currentTime(), extraMemoryCost() + newDataSize, maximumBufferSize);
     714    size_t initialBufferedSize = extraMemoryCost();
     715#endif
     716
     717    MediaTime rangeStart = MediaTime::zeroTime();
     718    MediaTime rangeEnd = rangeStart + thirtySeconds;
     719    while (rangeStart < maximumRangeEnd) {
     720        // 4. For each range in removal ranges, run the coded frame removal algorithm with start and
     721        // end equal to the removal range start and end timestamp respectively.
     722        removeCodedFrames(rangeStart, std::min(rangeEnd, maximumRangeEnd));
     723        if (extraMemoryCost() + newDataSize < maximumBufferSize) {
     724            m_bufferFull = false;
     725            break;
     726        }
     727
     728        rangeStart += thirtySeconds;
     729        rangeEnd += thirtySeconds;
     730    }
     731
     732    if (!m_bufferFull) {
     733        LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - evicted %zu bytes", this, initialBufferedSize - extraMemoryCost());
     734        return;
     735    }
     736
     737    // If there still isn't enough free space and there buffers in time ranges after the current range (ie. there is a gap after
     738    // the current buffered range), delete 30 seconds at a time from duration back to the current time range or 30 seconds after
     739    // currenTime whichever we hit first.
     740    auto buffered = m_buffered->ranges();
     741    size_t currentTimeRange = buffered.find(currentTime);
     742    if (currentTimeRange == notFound || currentTimeRange == buffered.length() - 1) {
     743        LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - evicted %zu bytes but FAILED to free enough", this, initialBufferedSize - extraMemoryCost());
     744        return;
     745    }
     746
     747    MediaTime minimumRangeStart = currentTime + thirtySeconds;
     748
     749    rangeEnd = MediaTime::createWithDouble(m_source->duration());
     750    rangeStart = rangeEnd - thirtySeconds;
     751    while (rangeStart > minimumRangeStart) {
     752
     753        // Do not evict data from the time range that contains currentTime.
     754        size_t startTimeRange = buffered.find(rangeStart);
     755        if (startTimeRange == currentTimeRange) {
     756            size_t endTimeRange = buffered.find(rangeEnd);
     757            if (endTimeRange == currentTimeRange)
     758                break;
     759
     760            rangeEnd = buffered.start(endTimeRange);
     761        }
     762
     763        // 4. For each range in removal ranges, run the coded frame removal algorithm with start and
     764        // end equal to the removal range start and end timestamp respectively.
     765        removeCodedFrames(std::max(minimumRangeStart, rangeStart), rangeEnd);
     766        if (extraMemoryCost() + newDataSize < maximumBufferSize) {
     767            m_bufferFull = false;
     768            break;
     769        }
     770
     771        rangeStart -= thirtySeconds;
     772        rangeEnd -= thirtySeconds;
     773    }
     774
     775    LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - evicted %zu bytes%s", this, initialBufferedSize - extraMemoryCost(), m_bufferFull ? "" : " but FAILED to free enough");
     776}
     777
     778size_t SourceBuffer::maximumBufferSize() const
     779{
     780    if (isRemoved())
     781        return 0;
     782
     783    HTMLMediaElement* element = m_source->mediaElement();
     784    if (!element)
     785        return 0;
     786
     787    return element->maximumSourceBufferSize(*this);
    638788}
    639789
     
    11991349            dependentSamples.insert(firstDecodeIter, nextSyncIter);
    12001350
    1201             RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(dependentSamples, trackBuffer);
    1202 #if !LOG_DISABLED
    1203             for (auto& samplePair : dependentSamples)
    1204                 LOG(MediaSource, "SourceBuffer::sourceBufferPrivateDidReceiveSample(%p) - removing sample(%s)", this, toString(*samplePair.second).utf8().data());
    1205 #endif
     1351            RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(dependentSamples, trackBuffer, this, "sourceBufferPrivateDidReceiveSample");
    12061352
    12071353            // Only force the TrackBuffer to re-enqueue if the removed ranges overlap with enqueued and possibly
    12081354            // not yet displayed samples.
    12091355            MediaTime currentMediaTime = MediaTime::createWithDouble(m_source->currentTime());
    1210             PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
    1211             possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
    1212             if (possiblyEnqueuedRanges.length())
    1213                 trackBuffer.needsReenqueueing = true;
     1356            if (currentMediaTime < trackBuffer.lastEnqueuedPresentationTime) {
     1357                PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
     1358                possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
     1359                if (possiblyEnqueuedRanges.length())
     1360                    trackBuffer.needsReenqueueing = true;
     1361            }
    12141362
    12151363            erasedRanges->invert();
     
    12631411}
    12641412
     1413bool SourceBuffer::hasAudio() const
     1414{
     1415    return m_audioTracks && m_audioTracks->length();
     1416}
     1417
     1418bool SourceBuffer::hasVideo() const
     1419{
     1420    return m_videoTracks && m_videoTracks->length();
     1421}
     1422
    12651423bool SourceBuffer::sourceBufferPrivateHasAudio(const SourceBufferPrivate*) const
    12661424{
    1267     return m_audioTracks && m_audioTracks->length();
     1425    return hasAudio();
    12681426}
    12691427
    12701428bool SourceBuffer::sourceBufferPrivateHasVideo(const SourceBufferPrivate*) const
    12711429{
    1272     return m_videoTracks && m_videoTracks->length();
     1430    return hasVideo();
    12731431}
    12741432
     
    15161674
    15171675    MediaTime currentTime = MediaTime::createWithDouble(m_source->currentTime());
     1676    MediaTime duration = MediaTime::createWithDouble(m_source->duration());
     1677    if (currentTime >= duration)
     1678        return true;
     1679
    15181680    std::unique_ptr<PlatformTimeRanges> ranges = bufferedAccountingForEndOfStream();
    15191681    return abs(ranges->nearest(currentTime) - currentTime) <= currentTimeFudgeFactor();
     
    15301692
    15311693    MediaTime currentTime = MediaTime::createWithDouble(m_source->currentTime());
     1694    MediaTime duration = MediaTime::createWithDouble(m_source->duration());
     1695    if (currentTime >= duration)
     1696        return true;
     1697
    15321698    MediaTime nearest = ranges->nearest(currentTime);
    15331699    if (abs(nearest - currentTime) > currentTimeFudgeFactor())
     
    15391705
    15401706    MediaTime localEnd = ranges->end(found);
    1541     MediaTime duration = MediaTime::createWithDouble(m_source->duration());
    15421707    if (localEnd == duration)
    15431708        return true;
     
    15731738}
    15741739
    1575 void SourceBuffer::reportExtraMemoryCost()
     1740size_t SourceBuffer::extraMemoryCost() const
    15761741{
    15771742    size_t extraMemoryCost = m_pendingAppendData.capacity();
     
    15791744        extraMemoryCost += trackBuffer.samples.sizeInBytes();
    15801745
     1746    return extraMemoryCost;
     1747}
     1748
     1749void SourceBuffer::reportExtraMemoryCost()
     1750{
     1751    size_t extraMemoryCost = this->extraMemoryCost();
    15811752    if (extraMemoryCost < m_reportedExtraMemoryCost)
    15821753        return;
     
    16041775}
    16051776
     1777Document& SourceBuffer::document() const
     1778{
     1779    ASSERT(scriptExecutionContext()->isDocument());
     1780    return *static_cast<Document*>(scriptExecutionContext());
     1781}
     1782
    16061783} // namespace WebCore
    16071784
  • trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h

    r172584 r172657  
    11/*
    22 * Copyright (C) 2013 Google Inc. All rights reserved.
     3 * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
    34 *
    45 * Redistribution and use in source and binary forms, with or without
     
    9192    bool canPlayThrough();
    9293
     94    bool hasVideo() const;
     95    bool hasAudio() const;
     96
    9397    bool active() const { return m_active; }
    9498
     
    105109
    106110    struct TrackBuffer;
     111
     112    Document& document() const;
    107113
    108114protected:
     
    155161    void provideMediaData(TrackBuffer&, AtomicString trackID);
    156162    void didDropSample();
     163    void evictCodedFrames(size_t newDataSize);
     164    size_t maximumBufferSize() const;
    157165
    158166    void monitorBufferingRate();
     
    161169    void removeCodedFrames(const MediaTime& start, const MediaTime& end);
    162170
     171    size_t extraMemoryCost() const;
    163172    void reportExtraMemoryCost();
    164173
     
    173182    GenericEventQueue m_asyncEventQueue;
    174183
    175     bool m_updating;
    176 
    177184    Vector<unsigned char> m_pendingAppendData;
    178185    Timer<SourceBuffer> m_appendBufferTimer;
     
    190197
    191198    HashMap<AtomicString, TrackBuffer> m_trackBufferMap;
    192     bool m_receivedFirstInitializationSegment;
    193199    RefPtr<TimeRanges> m_buffered;
    194     bool m_active;
    195200
    196201    enum AppendStateType { WaitingForSegment, ParsingInitSegment, ParsingMediaSegment };
     
    206211    MediaTime m_pendingRemoveEnd;
    207212    Timer<SourceBuffer> m_removeTimer;
     213
     214    bool m_updating;
     215    bool m_receivedFirstInitializationSegment;
     216    bool m_active;
     217    bool m_bufferFull;
    208218};
    209219
  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r172422 r172657  
    25192519    }
    25202520
    2521     LOG(Media, "HTMLMediaElement::refreshCachedTime - caching time %f", m_cachedTime);
    25222521    m_clockTimeAtLastCachedTimeUpdate = monotonicallyIncreasingTime();
    25232522}
     
    59735972}
    59745973
     5974#if ENABLE(MEDIA_SOURCE)
     5975size_t HTMLMediaElement::maximumSourceBufferSize(const SourceBuffer& buffer) const
     5976{
     5977    return m_mediaSession->maximumMediaSourceBufferSize(buffer);
     5978}
     5979#endif
     5980
    59755981void HTMLMediaElement::pausePlayback()
    59765982{
  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r172422 r172657  
    7373#if ENABLE(MEDIA_SOURCE)
    7474class MediaSource;
     75class SourceBuffer;
    7576class VideoPlaybackQuality;
    7677#endif
     
    209210    void closeMediaSource();
    210211    void incrementDroppedFrameCount() { ++m_droppedVideoFrames; }
     212    size_t maximumSourceBufferSize(const SourceBuffer&) const;
    211213#endif
    212214
  • trunk/Source/WebCore/html/HTMLMediaSession.cpp

    r168367 r172657  
    3939#include "Page.h"
    4040#include "ScriptController.h"
     41#include "SourceBuffer.h"
    4142
    4243#if PLATFORM(IOS)
     
    302303}
    303304
     305#if ENABLE(MEDIA_SOURCE)
     306const unsigned fiveMinutesOf1080PVideo = 290 * 1024 * 1024; // 290 MB is approximately 5 minutes of 8Mbps (1080p) content.
     307const unsigned fiveMinutesStereoAudio = 14 * 1024 * 1024; // 14 MB is approximately 5 minutes of 384kbps content.
     308
     309size_t HTMLMediaSession::maximumMediaSourceBufferSize(const SourceBuffer& buffer) const
     310{
     311    // A good quality 1080p video uses 8,000 kbps and stereo audio uses 384 kbps, so assume 95% for video and 5% for audio.
     312    const float bufferBudgetPercentageForVideo = .95;
     313    const float bufferBudgetPercentageForAudio = .05;
     314
     315    size_t maximum;
     316    Settings* settings = buffer.document().settings();
     317    if (settings)
     318        maximum = settings->maximumSourceBufferSize();
     319    else
     320        maximum = fiveMinutesOf1080PVideo + fiveMinutesStereoAudio;
     321
     322    // Allow a SourceBuffer to buffer as though it is audio-only even if it doesn't have any active tracks (yet).
     323    size_t bufferSize = static_cast<size_t>(maximum * bufferBudgetPercentageForAudio);
     324    if (buffer.hasVideo())
     325        bufferSize += static_cast<size_t>(maximum * bufferBudgetPercentageForVideo);
     326
     327    // FIXME: we might want to modify this algorithm to:
     328    // - decrease the maximum size for background tabs
     329    // - decrease the maximum size allowed for inactive elements when a process has more than one
     330    //   element, eg. so a page with many elements which are played one at a time doesn't keep
     331    //   everything buffered after an element has finished playing.
     332
     333    return bufferSize;
     334}
     335#endif
     336
    304337}
    305338
  • trunk/Source/WebCore/html/HTMLMediaSession.h

    r164661 r172657  
    3535
    3636class HTMLMediaElement;
     37class SourceBuffer;
    3738
    3839class HTMLMediaSession : public MediaSession {
     
    8384    void removeBehaviorRestriction(BehaviorRestrictions);
    8485
     86#if ENABLE(MEDIA_SOURCE)
     87    size_t maximumMediaSourceBufferSize(const SourceBuffer&) const;
     88#endif
     89
    8590private:
    8691    BehaviorRestrictions m_restrictions;
  • trunk/Source/WebCore/page/Settings.in

    r172503 r172657  
    231231allowNavigationToInvalidURL initial=false
    232232
     233# Allow SourceBuffers to store up to 304MB each, enough for approximately five minutes
     234# of 1080p video and stereo audio.
     235maximumSourceBufferSize type=int, initial=318767104, conditional=MEDIA_SOURCE
     236
    233237longMousePressEnabled initial=false
  • trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.cpp

    r171624 r172657  
    2828
    2929#include <math.h>
     30#include <wtf/PrintStream.h>
    3031
    3132namespace WebCore {
     
    253254{
    254255    MediaTime total = MediaTime::zeroTime();
    255     bool ignoreInvalid;
    256256
    257257    for (unsigned n = 0; n < length(); n++)
    258         total += abs(end(n, ignoreInvalid) - start(n, ignoreInvalid));
     258        total += abs(end(n) - start(n));
    259259    return total;
    260260}
    261261
    262 }
     262void PlatformTimeRanges::dump(PrintStream& out) const
     263{
     264    if (!length())
     265        return;
     266
     267    for (size_t i = 0; i < length(); ++i)
     268        out.print("[", start(i), "..", end(i), "] ");
     269}
     270
     271}
  • trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.h

    r171624 r172657  
    3232#include <wtf/RefCounted.h>
    3333#include <wtf/Vector.h>
     34
     35namespace WTF {
     36class PrintStream;
     37}
    3438
    3539namespace WebCore {
     
    6973
    7074    MediaTime totalDuration() const;
     75
     76    void dump(WTF::PrintStream&) const;
    7177
    7278private:
  • trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h

    r168508 r172657  
    11/*
    22 * Copyright (C) 2013 Google Inc. All rights reserved.
     3 * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
    34 *
    45 * Redistribution and use in source and binary forms, with or without
     
    5657    virtual MediaPlayer::ReadyState readyState() const = 0;
    5758    virtual void setReadyState(MediaPlayer::ReadyState) = 0;
    58     virtual void evictCodedFrames() = 0;
    59     virtual bool isFull() = 0;
    6059
    6160    virtual void flushAndEnqueueNonDisplayingSamples(Vector<RefPtr<MediaSample>>, AtomicString) { }
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h

    r169830 r172657  
    11/*
    2  * Copyright (C) 2013 Apple Inc. All rights reserved.
     2 * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
     
    113113    virtual MediaPlayer::ReadyState readyState() const override;
    114114    virtual void setReadyState(MediaPlayer::ReadyState) override;
    115     virtual void evictCodedFrames() override;
    116     virtual bool isFull() override;
    117115    virtual void flushAndEnqueueNonDisplayingSamples(Vector<RefPtr<MediaSample>>, AtomicString trackID) override;
    118116    virtual void enqueueSample(PassRefPtr<MediaSample>, AtomicString trackID) override;
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm

    r171478 r172657  
    11/*
    2  * Copyright (C) 2013 Apple Inc. All rights reserved.
     2 * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
     
    633633        FloatSize formatSize = FloatSize(CMVideoFormatDescriptionGetPresentationDimensions(formatDescription, true, true));
    634634        if (formatSize != m_cachedSize) {
    635             LOG(MediaSource, "SourceBufferPrivateAVFObjC::processCodedFrame(%p) - size change detected: {width=%lf, height=%lf", formatSize.width(), formatSize.height());
     635            LOG(MediaSource, "SourceBufferPrivateAVFObjC::processCodedFrame(%p) - size change detected: {width=%lf, height=%lf}", formatSize.width(), formatSize.height());
    636636            m_cachedSize = formatSize;
    637637            if (m_mediaSource)
     
    779779        m_mediaSource->player()->setReadyState(readyState);
    780780}
    781 
    782 void SourceBufferPrivateAVFObjC::evictCodedFrames()
    783 {
    784     notImplemented();
    785 }
    786 
    787 bool SourceBufferPrivateAVFObjC::isFull()
    788 {
    789     notImplemented();
    790     return false;
    791 }
    792 
    793781
    794782bool SourceBufferPrivateAVFObjC::hasVideo() const
  • trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp

    r169523 r172657  
    11/*
    2  * Copyright (C) 2013 Apple Inc. All rights reserved.
     2 * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
     
    214214}
    215215
    216 void MockSourceBufferPrivate::evictCodedFrames()
    217 {
    218     // No-op.
    219 }
    220 
    221 bool MockSourceBufferPrivate::isFull()
    222 {
    223     return false;
    224 }
    225 
    226216void MockSourceBufferPrivate::setActive(bool isActive)
    227217{
  • trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h

    r168508 r172657  
    11/*
    2  * Copyright (C) 2013 Apple Inc. All rights reserved.
     2 * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
     
    6969    virtual MediaPlayer::ReadyState readyState() const override;
    7070    virtual void setReadyState(MediaPlayer::ReadyState) override;
    71     virtual void evictCodedFrames() override;
    72     virtual bool isFull() override;
    7371
    7472    virtual void flushAndEnqueueNonDisplayingSamples(Vector<RefPtr<MediaSample>>, AtomicString) override { }
Note: See TracChangeset for help on using the changeset viewer.