Changeset 212311 in webkit


Ignore:
Timestamp:
Feb 14, 2017, 11:17:14 AM (9 years ago)
Author:
jer.noble@apple.com
Message:

Video elements with MediaSource objects set by srcObject are not cleared when srcObject is set to null
https://bugs.webkit.org/show_bug.cgi?id=168268

Reviewed by Eric Carlson.

Source/WebCore:

Test: fast/mediastream/MediaStream-MediaElement-setObject-null.html

Make the setSrcObject() operation compliant with the HTML spec. Since the specification defines
srcObject in terms of either a MediaSource, MediaStream, or Blob object, add the variant typedef
to HTMLMediaElement and move the definition out of the Modules/mediastream extension IDL and into
HTMLMediaElement.idl. Then bring the "media elements load" and "resource selection" algorithms up
to their most recent definitions in the HTML5 spec.

Drive-by fix: Allow the (admittedly weird) single-element-union type in IDL.

  • CMakeLists.txt:
  • DerivedSources.cpp:
  • DerivedSources.make:
  • Modules/mediastream/HTMLMediaElementMediaStream.cpp: Removed.
  • Modules/mediastream/HTMLMediaElementMediaStream.h: Removed.
  • Modules/mediastream/HTMLMediaElementMediaStream.idl: Removed.
  • WebCore.xcodeproj/project.pbxproj:
  • bindings/scripts/IDLParser.pm:

(parseUnionType):

  • html/HTMLAudioElement.cpp:

(WebCore::HTMLAudioElement::createForJSConstructor):

  • html/HTMLMediaElement.cpp:

(WebCore::actionName):
(WebCore::HTMLMediaElement::parseAttribute):
(WebCore::HTMLMediaElement::insertedInto):
(WebCore::HTMLMediaElement::scheduleDelayedAction):
(WebCore::HTMLMediaElement::scheduleNextSourceChild):
(WebCore::HTMLMediaElement::pendingActionTimerFired):
(WebCore::HTMLMediaElement::setSrcObject):
(WebCore::HTMLMediaElement::load):
(WebCore::HTMLMediaElement::prepareForLoad):
(WebCore::HTMLMediaElement::selectMediaResource):
(WebCore::HTMLMediaElement::loadResource):
(WebCore::HTMLMediaElement::playInternal):
(WebCore::HTMLMediaElement::pauseInternal):
(WebCore::HTMLMediaElement::sourceWasAdded):
(WebCore::HTMLMediaElement::clearMediaPlayer):
(WebCore::HTMLMediaElement::resume):
(WebCore::HTMLMediaElement::mediaCanStart):
(WebCore::HTMLMediaElement::createMediaPlayer):
(WebCore::HTMLMediaElement::loadInternal): Deleted.

  • html/HTMLMediaElement.h:

(WebCore::HTMLMediaElement::srcObject):

  • html/HTMLMediaElement.idl:
  • platform/ContentType.h:

LayoutTests:

  • fast/mediastream/MediaStream-MediaElement-setObject-null-expected.txt: Added.
  • fast/mediastream/MediaStream-MediaElement-setObject-null.html: Added.
  • fast/mediastream/MediaStream-video-element-expected.txt:
  • fast/mediastream/MediaStream-video-element-track-stop-expected.txt:
  • fast/mediastream/MediaStream-video-element-track-stop.html:
  • fast/mediastream/MediaStream-video-element.html:
Location:
trunk
Files:
2 added
3 deleted
16 edited

Legend:

Unmodified
Added
Removed
  • trunk/LayoutTests/ChangeLog

    r212304 r212311  
     12017-02-14  Jer Noble  <jer.noble@apple.com>
     2
     3        Video elements with MediaSource objects set by srcObject are not cleared when srcObject is set to null
     4        https://bugs.webkit.org/show_bug.cgi?id=168268
     5
     6        Reviewed by Eric Carlson.
     7
     8        * fast/mediastream/MediaStream-MediaElement-setObject-null-expected.txt: Added.
     9        * fast/mediastream/MediaStream-MediaElement-setObject-null.html: Added.
     10        * fast/mediastream/MediaStream-video-element-expected.txt:
     11        * fast/mediastream/MediaStream-video-element-track-stop-expected.txt:
     12        * fast/mediastream/MediaStream-video-element-track-stop.html:
     13        * fast/mediastream/MediaStream-video-element.html:
     14
    1152017-02-14  Antoine Quint  <graouts@apple.com>
    216
  • trunk/LayoutTests/fast/mediastream/MediaStream-video-element-expected.txt

    r208559 r212311  
    2525PASS video.videoHeight is mediaStream.getVideoTracks()[0].getSettings().height
    2626PASS video.src.indexOf("blob:") is 0
    27 PASS video.srcObject is mediaStream
    2827
    2928**** check video tracks ****
  • trunk/LayoutTests/fast/mediastream/MediaStream-video-element-track-stop-expected.txt

    r208559 r212311  
    2626PASS video.videoHeight is mediaStream.getVideoTracks()[0].getSettings().height
    2727PASS video.src.indexOf("blob:") is 0
    28 PASS video.srcObject is mediaStream
    2928
    3029PASS video.videoTracks.length is 1
  • trunk/LayoutTests/fast/mediastream/MediaStream-video-element-track-stop.html

    r192954 r212311  
    6464                shouldBe('video.videoHeight', 'mediaStream.getVideoTracks()[0].getSettings().height');
    6565                shouldBe('video.src.indexOf("blob:")', '0');
    66                 shouldBe('video.srcObject', 'mediaStream');
    6766               
    6867                debug("");
  • trunk/LayoutTests/fast/mediastream/MediaStream-video-element.html

    r192954 r212311  
    6060                shouldBe('video.videoHeight', 'mediaStream.getVideoTracks()[0].getSettings().height');
    6161                shouldBe('video.src.indexOf("blob:")', '0');
    62                 shouldBe('video.srcObject', 'mediaStream');
    63                
     62
    6463                debug("<br>**** check video tracks ****");
    6564                shouldBe('video.videoTracks.length', '1');
  • trunk/Source/WebCore/CMakeLists.txt

    r212252 r212311  
    212212    Modules/mediastream/DOMURLMediaStream.idl
    213213    Modules/mediastream/DoubleRange.idl
    214     Modules/mediastream/HTMLMediaElementMediaStream.idl
    215214    Modules/mediastream/LongRange.idl
    216215    Modules/mediastream/MediaDeviceInfo.idl
     
    897896
    898897    Modules/mediastream/DOMURLMediaStream.cpp
    899     Modules/mediastream/HTMLMediaElementMediaStream.cpp
    900898    Modules/mediastream/MediaConstraintsImpl.cpp
    901899    Modules/mediastream/MediaDeviceInfo.cpp
  • trunk/Source/WebCore/ChangeLog

    r212307 r212311  
     12017-02-14  Jer Noble  <jer.noble@apple.com>
     2
     3        Video elements with MediaSource objects set by srcObject are not cleared when srcObject is set to null
     4        https://bugs.webkit.org/show_bug.cgi?id=168268
     5
     6        Reviewed by Eric Carlson.
     7
     8        Test: fast/mediastream/MediaStream-MediaElement-setObject-null.html
     9
     10        Make the setSrcObject() operation compliant with the HTML spec. Since the specification defines
     11        srcObject in terms of either a MediaSource, MediaStream, or Blob object, add the variant typedef
     12        to HTMLMediaElement and move the definition out of the Modules/mediastream extension IDL and into
     13        HTMLMediaElement.idl. Then bring the "media elements load" and "resource selection" algorithms up
     14        to their most recent definitions in the HTML5 spec.
     15
     16        Drive-by fix: Allow the (admittedly weird) single-element-union type in IDL.
     17
     18        * CMakeLists.txt:
     19        * DerivedSources.cpp:
     20        * DerivedSources.make:
     21        * Modules/mediastream/HTMLMediaElementMediaStream.cpp: Removed.
     22        * Modules/mediastream/HTMLMediaElementMediaStream.h: Removed.
     23        * Modules/mediastream/HTMLMediaElementMediaStream.idl: Removed.
     24        * WebCore.xcodeproj/project.pbxproj:
     25        * bindings/scripts/IDLParser.pm:
     26        (parseUnionType):
     27        * html/HTMLAudioElement.cpp:
     28        (WebCore::HTMLAudioElement::createForJSConstructor):
     29        * html/HTMLMediaElement.cpp:
     30        (WebCore::actionName):
     31        (WebCore::HTMLMediaElement::parseAttribute):
     32        (WebCore::HTMLMediaElement::insertedInto):
     33        (WebCore::HTMLMediaElement::scheduleDelayedAction):
     34        (WebCore::HTMLMediaElement::scheduleNextSourceChild):
     35        (WebCore::HTMLMediaElement::pendingActionTimerFired):
     36        (WebCore::HTMLMediaElement::setSrcObject):
     37        (WebCore::HTMLMediaElement::load):
     38        (WebCore::HTMLMediaElement::prepareForLoad):
     39        (WebCore::HTMLMediaElement::selectMediaResource):
     40        (WebCore::HTMLMediaElement::loadResource):
     41        (WebCore::HTMLMediaElement::playInternal):
     42        (WebCore::HTMLMediaElement::pauseInternal):
     43        (WebCore::HTMLMediaElement::sourceWasAdded):
     44        (WebCore::HTMLMediaElement::clearMediaPlayer):
     45        (WebCore::HTMLMediaElement::resume):
     46        (WebCore::HTMLMediaElement::mediaCanStart):
     47        (WebCore::HTMLMediaElement::createMediaPlayer):
     48        (WebCore::HTMLMediaElement::loadInternal): Deleted.
     49        * html/HTMLMediaElement.h:
     50        (WebCore::HTMLMediaElement::srcObject):
     51        * html/HTMLMediaElement.idl:
     52        * platform/ContentType.h:
     53
    1542017-02-14  Aakash Jain  <aakash_jain@apple.com>
    255
  • trunk/Source/WebCore/DerivedSources.cpp

    r212193 r212311  
    264264#include "JSHTMLMediaElement.cpp"
    265265#include "JSHTMLMediaElementMediaSession.cpp"
    266 #include "JSHTMLMediaElementMediaStream.cpp"
    267266#include "JSHTMLMenuElement.cpp"
    268267#include "JSHTMLMetaElement.cpp"
  • trunk/Source/WebCore/DerivedSources.make

    r212193 r212311  
    153153    $(WebCore)/Modules/mediastream/DOMURLMediaStream.idl \
    154154    $(WebCore)/Modules/mediastream/DoubleRange.idl \
    155     $(WebCore)/Modules/mediastream/HTMLMediaElementMediaStream.idl \
    156155    $(WebCore)/Modules/mediastream/LongRange.idl \
    157156    $(WebCore)/Modules/mediastream/MediaDeviceInfo.idl \
  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r212302 r212311  
    163163                077664FC183E6B5C00133B92 /* JSQuickTimePluginReplacement.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 077664FA183E6B5C00133B92 /* JSQuickTimePluginReplacement.cpp */; };
    164164                077664FD183E6B5C00133B92 /* JSQuickTimePluginReplacement.h in Headers */ = {isa = PBXBuildFile; fileRef = 077664FB183E6B5C00133B92 /* JSQuickTimePluginReplacement.h */; };
    165                 0779BF0D18453168000B6AE7 /* HTMLMediaElementMediaStream.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0779BF0A18453168000B6AE7 /* HTMLMediaElementMediaStream.cpp */; };
    166165                0779BF0E18453168000B6AE7 /* HTMLMediaElementMediaStream.h in Headers */ = {isa = PBXBuildFile; fileRef = 0779BF0B18453168000B6AE7 /* HTMLMediaElementMediaStream.h */; };
    167166                077AF14018F4AE400001ED61 /* SerializedPlatformRepresentation.h in Headers */ = {isa = PBXBuildFile; fileRef = 077AF13E18F4AE400001ED61 /* SerializedPlatformRepresentation.h */; settings = {ATTRIBUTES = (Private, ); }; };
     
    72917290                077664FA183E6B5C00133B92 /* JSQuickTimePluginReplacement.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSQuickTimePluginReplacement.cpp; sourceTree = "<group>"; };
    72927291                077664FB183E6B5C00133B92 /* JSQuickTimePluginReplacement.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSQuickTimePluginReplacement.h; sourceTree = "<group>"; };
    7293                 0779BF0A18453168000B6AE7 /* HTMLMediaElementMediaStream.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HTMLMediaElementMediaStream.cpp; sourceTree = "<group>"; };
    72947292                0779BF0B18453168000B6AE7 /* HTMLMediaElementMediaStream.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HTMLMediaElementMediaStream.h; sourceTree = "<group>"; };
    72957293                0779BF0C18453168000B6AE7 /* HTMLMediaElementMediaStream.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = HTMLMediaElementMediaStream.idl; sourceTree = "<group>"; };
     
    1529815296                                93A806111E03B51C008A1F26 /* DoubleRange.h */,
    1529915297                                93A806121E03B51C008A1F26 /* DoubleRange.idl */,
    15300                                 0779BF0A18453168000B6AE7 /* HTMLMediaElementMediaStream.cpp */,
    1530115298                                0779BF0B18453168000B6AE7 /* HTMLMediaElementMediaStream.h */,
    1530215299                                0779BF0C18453168000B6AE7 /* HTMLMediaElementMediaStream.idl */,
     
    3004530042                                E44613A40CD6331000FADA75 /* HTMLMediaElement.cpp in Sources */,
    3004630043                                C968B2E81B1E778100EF1F81 /* HTMLMediaElementMediaSession.cpp in Sources */,
    30047                                 0779BF0D18453168000B6AE7 /* HTMLMediaElementMediaStream.cpp in Sources */,
    3004830044                                A8EA79F80A1916DF00A8EF5F /* HTMLMenuElement.cpp in Sources */,
    3004930045                                2BE8E2C912A58A0100FAD550 /* HTMLMetaCharsetParser.cpp in Sources */,
  • trunk/Source/WebCore/bindings/scripts/IDLParser.pm

    r211409 r212311  
    20192019       
    20202020        push(@{$unionType->subtypes}, $self->parseUnionMemberType());
    2021        
    2022         $self->assertTokenValue($self->getToken(), "or", __LINE__);
    2023        
    2024         push(@{$unionType->subtypes}, $self->parseUnionMemberType());
    20252021        push(@{$unionType->subtypes}, $self->parseUnionMemberTypes());
    20262022       
  • trunk/Source/WebCore/html/HTMLAudioElement.cpp

    r177996 r212311  
    5151    Ref<HTMLAudioElement> audio = adoptRef(*new HTMLAudioElement(audioTag, document, false));
    5252    audio->setPreload("auto");
    53     if (!src.isNull()) {
     53    if (!src.isNull())
    5454        audio->setSrc(src);
    55         audio->scheduleDelayedAction(HTMLMediaElement::LoadMediaResource);
    56     }
    5755    audio->suspendIfNeeded();
    5856    return audio;
  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r212289 r212311  
    3232#include "ApplicationCacheResource.h"
    3333#include "Attribute.h"
     34#include "Blob.h"
    3435#include "CSSPropertyNames.h"
    3536#include "CSSValueKeywords.h"
     
    204205    } \
    205206
    206     ACTION(LoadMediaResource);
    207207    ACTION(ConfigureTextTracks);
    208208    ACTION(TextTrackChangesNotification);
     
    743743{
    744744    if (name == srcAttr) {
     745        // https://html.spec.whatwg.org/multipage/embedded-content.html#location-of-the-media-resource
     746        // Location of the Media Resource
     747        // 12 February 2017
     748
     749        // If a src attribute of a media element is set or changed, the user
     750        // agent must invoke the media element's media element load algorithm.
    745751#if PLATFORM(IOS)
    746752        // Note, unless the restriction on requiring user action has been removed,
     
    748754        if (!value.isNull() && m_mediaSession->dataLoadingPermitted(*this))
    749755#else
    750         // Trigger a reload, as long as the 'src' attribute is present.
    751756        if (!value.isNull())
    752757#endif
    753         {
    754             clearMediaPlayer(LoadMediaResource);
    755             scheduleDelayedAction(LoadMediaResource);
    756         }
     758            prepareForLoad();
    757759    } else if (name == controlsAttr)
    758760        configureMediaControls();
     
    771773
    772774        // The attribute must be ignored if the autoplay attribute is present
    773         if (!autoplay() && m_player)
     775        if (!autoplay() && !m_havePreparedToPlay && m_player)
    774776            m_player->setPreload(m_mediaSession->effectivePreloadForElement(*this));
    775777
     
    831833        if (m_networkState == NETWORK_EMPTY && !attributeWithoutSynchronization(srcAttr).isEmpty())
    832834#endif
    833             scheduleDelayedAction(LoadMediaResource);
     835            prepareForLoad();
    834836    }
    835837
     
    931933    LOG(Media, "HTMLMediaElement::scheduleDelayedAction(%p) - setting %s flag", this, actionName(actionType).utf8().data());
    932934
    933     if ((actionType & LoadMediaResource) && !(m_pendingActionFlags & LoadMediaResource)) {
    934         prepareForLoad();
    935         setFlags(m_pendingActionFlags, LoadMediaResource);
    936     }
    937 
    938935#if ENABLE(VIDEO_TRACK)
    939936    if (actionType & ConfigureTextTracks)
     
    961958{
    962959    // Schedule the timer to try the next <source> element WITHOUT resetting state ala prepareForLoad.
    963     LOG(Media, "HTMLMediaElement::scheduleNextSourceChild(%p) - setting %s flag", this, actionName(LoadMediaResource).utf8().data());
    964     setFlags(m_pendingActionFlags, LoadMediaResource);
    965     m_pendingActionTimer.startOneShot(0);
     960    m_resourceSelectionTaskQueue.enqueueTask([this] {
     961        loadNextSourceChild();
     962    });
    966963}
    967964
     
    10361033        configureTextTracks();
    10371034#endif
    1038 
    1039     if (pendingActions & LoadMediaResource) {
    1040         if (m_loadState == LoadingFromSourceElement)
    1041             loadNextSourceChild();
    1042         else
    1043             loadInternal();
    1044     }
    10451035
    10461036#if ENABLE(WIRELESS_PLAYBACK_TARGET)
     
    10721062}
    10731063
    1074 #if ENABLE(MEDIA_STREAM)
    1075 void HTMLMediaElement::setSrcObject(ScriptExecutionContext& context, MediaStream* mediaStream)
     1064void HTMLMediaElement::setSrcObject(MediaProvider&& mediaProvider)
    10761065{
    10771066    // FIXME: Setting the srcObject attribute may cause other changes to the media element's internal state:
     
    10821071    // https://bugs.webkit.org/show_bug.cgi?id=124896
    10831072
    1084     m_mediaStreamSrcObject = mediaStream;
    1085     if (mediaStream) {
    1086         m_settingMediaStreamSrcObject = true;
    1087         setSrc(DOMURL::createPublicURL(context, *mediaStream));
    1088         m_settingMediaStreamSrcObject = false;
    1089     }
    1090 }
    1091 #endif
     1073
     1074    // https://www.w3.org/TR/html51/semantics-embedded-content.html#dom-htmlmediaelement-srcobject
     1075    // 4.7.14.2. Location of the media resource
     1076    // srcObject: On setting, it must set the element’s assigned media provider object to the new
     1077    // value, and then invoke the element’s media element load algorithm.
     1078    m_mediaProvider = WTFMove(mediaProvider);
     1079    prepareForLoad();
     1080}
    10921081
    10931082void HTMLMediaElement::setCrossOrigin(const AtomicString& value)
     
    11431132void HTMLMediaElement::load()
    11441133{
    1145     Ref<HTMLMediaElement> protectedThis(*this); // loadInternal may result in a 'beforeload' event, which can make arbitrary DOM mutations.
     1134    Ref<HTMLMediaElement> protectedThis(*this); // prepareForLoad may result in a 'beforeload' event, which can make arbitrary DOM mutations.
    11461135   
    11471136    LOG(Media, "HTMLMediaElement::load(%p)", this);
     
    11531142
    11541143    prepareForLoad();
    1155     loadInternal();
    1156     prepareToPlay();
     1144    m_resourceSelectionTaskQueue.enqueueTask([this] {
     1145        prepareToPlay();
     1146    });
    11571147}
    11581148
    11591149void HTMLMediaElement::prepareForLoad()
    11601150{
     1151    // https://html.spec.whatwg.org/multipage/embedded-content.html#media-element-load-algorithm
     1152    // The Media Element Load Algorithm
     1153    // 12 February 2017
     1154
    11611155    LOG(Media, "HTMLMediaElement::prepareForLoad(%p)", this);
    11621156
     1157    // 1 - Abort any already-running instance of the resource selection algorithm for this element.
    11631158    // Perform the cleanup required for the resource load algorithm to run.
    11641159    stopPeriodicTimers();
    11651160    m_pendingActionTimer.stop();
     1161    m_resourceSelectionTaskQueue.cancelAllTasks();
    11661162    // FIXME: Figure out appropriate place to reset LoadTextTrackResource if necessary and set m_pendingActionFlags to 0 here.
    1167     m_pendingActionFlags &= ~LoadMediaResource;
    11681163    m_sentEndEvent = false;
    11691164    m_sentStalledEvent = false;
     
    11781173#endif
    11791174
    1180     // 1 - Abort any already-running instance of the resource selection algorithm for this element.
    11811175    m_loadState = WaitingForSource;
    11821176    m_currentSourceNode = nullptr;
    11831177
    1184     // 2 - If there are any tasks from the media element's media element event task source in
    1185     // one of the task queues, then remove those tasks.
     1178    createMediaPlayer();
     1179
     1180    // 2 - Let pending tasks be a list of all tasks from the media element's media element event task source in one of the task queues.
     1181    // 3 - For each task in pending tasks that would resolve pending play promises or reject pending play promises, immediately resolve or reject those promises in the order the corresponding tasks were queued.
     1182    // 4 - Remove each task in pending tasks from its task queue
    11861183    cancelPendingEventsAndCallbacks();
    11871184
    1188     // 3 - If the media element's networkState is set to NETWORK_LOADING or NETWORK_IDLE, queue
     1185    // 5 - If the media element's networkState is set to NETWORK_LOADING or NETWORK_IDLE, queue
    11891186    // a task to fire a simple event named abort at the media element.
    11901187    if (m_networkState == NETWORK_LOADING || m_networkState == NETWORK_IDLE)
    11911188        scheduleEvent(eventNames().abortEvent);
    11921189
     1190    // 6 - If the media element's networkState is not set to NETWORK_EMPTY, then run these substeps
     1191    if (m_networkState != NETWORK_EMPTY) {
     1192        // 6.1 - Queue a task to fire a simple event named emptied at the media element.
     1193        scheduleEvent(eventNames().emptiedEvent);
     1194
     1195        // 6.2 - If a fetching process is in progress for the media element, the user agent should stop it.
     1196        m_networkState = NETWORK_EMPTY;
     1197
     1198        // 6.3 - If the media element’s assigned media provider object is a MediaSource object, then detach it.
    11931199#if ENABLE(MEDIA_SOURCE)
    1194     detachMediaSource();
    1195 #endif
    1196 
    1197     createMediaPlayer();
    1198 
    1199     // 4 - If the media element's networkState is not set to NETWORK_EMPTY, then run these substeps
    1200     if (m_networkState != NETWORK_EMPTY) {
    1201         // 4.1 - Queue a task to fire a simple event named emptied at the media element.
    1202         scheduleEvent(eventNames().emptiedEvent);
    1203 
    1204         // 4.2 - If a fetching process is in progress for the media element, the user agent should stop it.
    1205         m_networkState = NETWORK_EMPTY;
    1206 
    1207         // 4.3 - Forget the media element's media-resource-specific tracks.
     1200        detachMediaSource();
     1201#endif
     1202
     1203        // 6.4 - Forget the media element's media-resource-specific tracks.
    12081204        forgetResourceSpecificTracks();
    12091205
    1210         // 4.4 - If readyState is not set to HAVE_NOTHING, then set it to that state.
     1206        // 6.5 - If readyState is not set to HAVE_NOTHING, then set it to that state.
    12111207        m_readyState = HAVE_NOTHING;
    12121208        m_readyStateMaximum = HAVE_NOTHING;
    12131209
    1214         // 4.5 - If the paused attribute is false, then set it to true.
     1210        // 6.6 - If the paused attribute is false, then set it to true.
    12151211        m_paused = true;
    12161212
    1217         // 4.6 - If seeking is true, set it to false.
     1213        // 6.7 - If seeking is true, set it to false.
    12181214        clearSeeking();
    12191215
    1220         // 4.7 - Set the current playback position to 0.
     1216        // 6.8 - Set the current playback position to 0.
    12211217        //       Set the official playback position to 0.
    12221218        //       If this changed the official playback position, then queue a task to fire a simple event named timeupdate at the media element.
     1219        m_lastSeekTime = MediaTime::zeroTime();
     1220        m_playedTimeRanges = TimeRanges::create();
    12231221        // FIXME: Add support for firing this event. e.g., scheduleEvent(eventNames().timeUpdateEvent);
    12241222
    1225         // 4.8 - Set the initial playback position to 0.
     1223        // 4.9 - Set the initial playback position to 0.
    12261224        // FIXME: Make this less subtle. The position only becomes 0 because of the createMediaPlayer() call
    12271225        // above.
     
    12301228        invalidateCachedTime();
    12311229
    1232         // 4.9 - Set the timeline offset to Not-a-Number (NaN).
    1233         // 4.10 - Update the duration attribute to Not-a-Number (NaN).
     1230        // 4.10 - Set the timeline offset to Not-a-Number (NaN).
     1231        // 4.11 - Update the duration attribute to Not-a-Number (NaN).
    12341232
    12351233        updateMediaController();
     
    12391237    }
    12401238
    1241     // 5 - Set the playbackRate attribute to the value of the defaultPlaybackRate attribute.
     1239    // 7 - Set the playbackRate attribute to the value of the defaultPlaybackRate attribute.
    12421240    setPlaybackRate(defaultPlaybackRate());
    12431241
    1244     // 6 - Set the error attribute to null and the autoplaying flag to true.
     1242    // 8 - Set the error attribute to null and the autoplaying flag to true.
    12451243    m_error = nullptr;
    12461244    m_autoplaying = true;
    12471245    mediaSession().clientWillBeginAutoplaying();
    12481246
    1249     // 7 - Invoke the media element's resource selection algorithm.
    1250 
    1251     // 8 - Note: Playback of any previously playing media resource for this element stops.
    1252 
    1253     // The resource selection algorithm
    1254     // 1 - Set the networkState to NETWORK_NO_SOURCE
     1247    // 9 - Invoke the media element's resource selection algorithm.
     1248    selectMediaResource();
     1249
     1250    // 10 - Note: Playback of any previously playing media resource for this element stops.
     1251
     1252    configureMediaControls();
     1253}
     1254
     1255void HTMLMediaElement::selectMediaResource()
     1256{
     1257    // https://www.w3.org/TR/2016/REC-html51-20161101/semantics-embedded-content.html#resource-selection-algorithm
     1258    // The Resource Selection Algorithm
     1259
     1260    // 1. Set the element’s networkState attribute to the NETWORK_NO_SOURCE value.
    12551261    m_networkState = NETWORK_NO_SOURCE;
    12561262
    1257     // 2 - Asynchronously await a stable state.
    1258 
    1259     m_playedTimeRanges = TimeRanges::create();
    1260 
    1261     // FIXME: Investigate whether these can be moved into m_networkState != NETWORK_EMPTY block above
    1262     // so they are closer to the relevant spec steps.
    1263     m_lastSeekTime = MediaTime::zeroTime();
    1264 
    1265     // The spec doesn't say to block the load event until we actually run the asynchronous section
    1266     // algorithm, but do it now because we won't start that until after the timer fires and the
    1267     // event may have already fired by then.
    1268     MediaPlayer::Preload effectivePreload = m_mediaSession->effectivePreloadForElement(*this);
    1269     if (effectivePreload != MediaPlayer::None)
    1270         setShouldDelayLoadEvent(true);
    1271 
    1272 #if PLATFORM(IOS)
    1273     if (effectivePreload != MediaPlayer::None && m_mediaSession->allowsAutomaticMediaDataLoading(*this))
    1274         prepareToPlay();
    1275 #endif
    1276 
    1277     configureMediaControls();
    1278 }
    1279 
    1280 void HTMLMediaElement::loadInternal()
    1281 {
    1282     LOG(Media, "HTMLMediaElement::loadInternal(%p)", this);
    1283 
    1284     // Some of the code paths below this function dispatch the BeforeLoad event. This ASSERT helps
    1285     // us catch those bugs more quickly without needing all the branches to align to actually
    1286     // trigger the event.
    1287     ASSERT(NoEventDispatchAssertion::isEventAllowedInMainThread());
    1288 
    1289     // If we can't start a load right away, start it later.
     1263    // 2. Set the element’s show poster flag to true.
     1264    setDisplayMode(Poster);
     1265
     1266    // 3. Set the media element’s delaying-the-load-event flag to true (this delays the load event).
     1267    setShouldDelayLoadEvent(true);
     1268
     1269    // 4. in parallel await a stable state, allowing the task that invoked this algorithm to continue.
     1270    if (m_resourceSelectionTaskQueue.hasPendingTasks())
     1271        return;
     1272
    12901273    if (!m_mediaSession->pageAllowsDataLoading(*this)) {
    1291         LOG(Media, "HTMLMediaElement::loadInternal(%p) - not allowed to load in background, waiting", this);
     1274        LOG(Media, "HTMLMediaElement::selectMediaResource(%p) - not allowed to load in background, waiting", this);
    12921275        setShouldDelayLoadEvent(false);
    12931276        if (m_isWaitingUntilMediaCanStart)
     
    12981281    }
    12991282
    1300     clearFlags(m_pendingActionFlags, LoadMediaResource);
    1301 
    1302     // Once the page has allowed an element to load media, it is free to load at will. This allows a
    1303     // playlist that starts in a foreground tab to continue automatically if the tab is subsequently
     1283    // Once the page has allowed an element to load media, it is free to load at will. This allows a
     1284    // playlist that starts in a foreground tab to continue automatically if the tab is subsequently
    13041285    // put into the background.
    13051286    m_mediaSession->removeBehaviorRestriction(MediaElementSession::RequirePageConsentToLoadMedia);
    13061287
     1288
     1289    m_resourceSelectionTaskQueue.enqueueTask([this]  {
     1290        // 5. If the media element’s blocked-on-parser flag is false, then populate the list of pending text tracks.
    13071291#if ENABLE(VIDEO_TRACK)
    1308     if (hasMediaControls())
    1309         mediaControls()->changedClosedCaptionsVisibility();
    1310 
    1311     // HTMLMediaElement::textTracksAreReady will need "... the text tracks whose mode was not in the
    1312     // disabled state when the element's resource selection algorithm last started".
    1313     m_textTracksWhenResourceSelectionBegan.clear();
    1314     if (m_textTracks) {
    1315         for (unsigned i = 0; i < m_textTracks->length(); ++i) {
    1316             TextTrack* track = m_textTracks->item(i);
    1317             if (track->mode() != TextTrack::Mode::Disabled)
    1318                 m_textTracksWhenResourceSelectionBegan.append(track);
     1292        if (hasMediaControls())
     1293            mediaControls()->changedClosedCaptionsVisibility();
     1294
     1295        // HTMLMediaElement::textTracksAreReady will need "... the text tracks whose mode was not in the
     1296        // disabled state when the element's resource selection algorithm last started".
     1297        // FIXME: Update this to match "populate the list of pending text tracks" step.
     1298        m_textTracksWhenResourceSelectionBegan.clear();
     1299        if (m_textTracks) {
     1300            for (unsigned i = 0; i < m_textTracks->length(); ++i) {
     1301                TextTrack* track = m_textTracks->item(i);
     1302                if (track->mode() != TextTrack::Mode::Disabled)
     1303                    m_textTracksWhenResourceSelectionBegan.append(track);
     1304            }
    13191305        }
    1320     }
    1321 #endif
    1322 
    1323     selectMediaResource();
    1324 }
    1325 
    1326 void HTMLMediaElement::selectMediaResource()
    1327 {
    1328     LOG(Media, "HTMLMediaElement::selectMediaResource(%p)", this);
    1329 
    1330     ASSERT(m_player);
    1331     if (!m_player)
    1332         return;
    1333 
    1334     enum Mode { attribute, children };
    1335 
    1336     // 3 - If the media element has a src attribute, then let mode be attribute.
    1337     Mode mode = attribute;
    1338     if (!hasAttributeWithoutSynchronization(srcAttr)) {
    1339         // Otherwise, if the media element does not have a src attribute but has a source
    1340         // element child, then let mode be children and let candidate be the first such
    1341         // source element child in tree order.
    1342         if (auto firstSource = childrenOfType<HTMLSourceElement>(*this).first()) {
    1343             mode = children;
     1306#endif
     1307
     1308        enum Mode { None, Object, Attribute, Children };
     1309        Mode mode = None;
     1310
     1311        if (m_mediaProvider) {
     1312            // 6. If the media element has an assigned media provider object, then let mode be object.
     1313            mode = Object;
     1314        } else if (hasAttributeWithoutSynchronization(srcAttr)) {
     1315            //    Otherwise, if the media element has no assigned media provider object but has a src attribute, then let mode be attribute.
     1316            mode = Attribute;
     1317        } else if (auto firstSource = childrenOfType<HTMLSourceElement>(*this).first()) {
     1318            //    Otherwise, if the media element does not have an assigned media provider object and does not have a src attribute,
     1319            //    but does have a source element child, then let mode be children and let candidate be the first such source element
     1320            //    child in tree order.
     1321            mode = Children;
    13441322            m_nextChildNodeToConsider = firstSource;
    13451323            m_currentSourceNode = nullptr;
    13461324        } else {
    1347             // Otherwise the media element has neither a src attribute nor a source element
    1348             // child: set the networkState to NETWORK_EMPTY, and abort these steps; the
    1349             // synchronous section ends.
     1325            //  Otherwise the media element has no assigned media provider object and has neither a src attribute nor a source
     1326            //  element child: set the networkState to NETWORK_EMPTY, and abort these steps; the synchronous section ends.
    13501327            m_loadState = WaitingForSource;
    13511328            setShouldDelayLoadEvent(false);
     
    13551332            return;
    13561333        }
    1357     }
    1358 
    1359     // 4 - Set the media element's delaying-the-load-event flag to true (this delays the load event),
    1360     // and set its networkState to NETWORK_LOADING.
    1361     setShouldDelayLoadEvent(true);
    1362     m_networkState = NETWORK_LOADING;
    1363 
    1364     // 5 - Queue a task to fire a simple event named loadstart at the media element.
    1365     scheduleEvent(eventNames().loadstartEvent);
    1366 
    1367     // 6 - If mode is attribute, then run these substeps
    1368     if (mode == attribute) {
    1369         m_loadState = LoadingFromSrcAttr;
    1370 
    1371         // If the src attribute's value is the empty string ... jump down to the failed step below
    1372         URL mediaURL = getNonEmptyURLAttribute(srcAttr);
    1373         if (mediaURL.isEmpty()) {
    1374             mediaLoadingFailed(MediaPlayer::FormatError);
    1375             LOG(Media, "HTMLMediaElement::selectMediaResource(%p) -  empty 'src'", this);
     1334
     1335        // 7. Set the media element’s networkState to NETWORK_LOADING.
     1336        m_networkState = NETWORK_LOADING;
     1337
     1338        // 8. Queue a task to fire a simple event named loadstart at the media element.
     1339        scheduleEvent(eventNames().loadstartEvent);
     1340
     1341        // 9. Run the appropriate steps from the following list:
     1342        // ↳ If mode is object
     1343        if (mode == Object) {
     1344            // 1. Set the currentSrc attribute to the empty string.
     1345            m_currentSrc = URL();
     1346
     1347            // 2. End the synchronous section, continuing the remaining steps in parallel.
     1348            // 3. Run the resource fetch algorithm with the assigned media provider object.
     1349            WTF::visit(WTF::makeVisitor(
     1350#if ENABLE(MEDIA_STREAM)
     1351                [this](RefPtr<MediaStream> stream) { m_mediaStreamSrcObject = stream; },
     1352#endif
     1353#if ENABLE(MEDIA_SOURCE)
     1354                [this](RefPtr<MediaSource> source) { m_mediaSource = source; },
     1355#endif
     1356                [this](RefPtr<Blob> blob) { m_blob = blob; }
     1357            ), m_mediaProvider.value());
     1358
     1359            ContentType contentType;
     1360            loadResource(URL(), contentType, String());
     1361            LOG(Media, "HTMLMediaElement::selectMediaResource(%p) - using 'srcObject' property", this);
     1362
     1363            //    If that algorithm returns without aborting this one, then the load failed.
     1364            // 4. Failed with media provider: Reaching this step indicates that the media resource
     1365            //    failed to load. Queue a task to run the dedicated media source failure steps.
     1366            // 5. Wait for the task queued by the previous step to have executed.
     1367            // 6. Abort these steps. The element won’t attempt to load another resource until this
     1368            //    algorithm is triggered again.
    13761369            return;
    13771370        }
    13781371
    1379         if (!isSafeToLoadURL(mediaURL, Complain) || !dispatchBeforeLoadEvent(mediaURL.string())) {
    1380             mediaLoadingFailed(MediaPlayer::FormatError);
     1372        // ↳ If mode is attribute
     1373        if (mode == Attribute) {
     1374            m_loadState = LoadingFromSrcAttr;
     1375
     1376            // 1. If the src attribute’s value is the empty string, then end the synchronous section,
     1377            //    and jump down to the failed with attribute step below.
     1378            // 2. Let absolute URL be the absolute URL that would have resulted from parsing the URL
     1379            //    specified by the src attribute’s value relative to the media element when the src
     1380            //    attribute was last changed.
     1381            URL absoluteURL = getNonEmptyURLAttribute(srcAttr);
     1382            if (absoluteURL.isEmpty()) {
     1383                mediaLoadingFailed(MediaPlayer::FormatError);
     1384                LOG(Media, "HTMLMediaElement::selectMediaResource(%p) -  empty 'src'", this);
     1385                return;
     1386            }
     1387
     1388            if (!isSafeToLoadURL(absoluteURL, Complain) || !dispatchBeforeLoadEvent(absoluteURL.string())) {
     1389                mediaLoadingFailed(MediaPlayer::FormatError);
     1390                return;
     1391            }
     1392
     1393            // 3. If absolute URL was obtained successfully, set the currentSrc attribute to absolute URL.
     1394            m_currentSrc = absoluteURL;
     1395
     1396            // 4. End the synchronous section, continuing the remaining steps in parallel.
     1397            // 5. If absolute URL was obtained successfully, run the resource fetch algorithm with absolute
     1398            //    URL. If that algorithm returns without aborting this one, then the load failed.
     1399
     1400            // No type or key system information is available when the url comes
     1401            // from the 'src' attribute so MediaPlayer
     1402            // will have to pick a media engine based on the file extension.
     1403            ContentType contentType;
     1404            loadResource(absoluteURL, contentType, String());
     1405            LOG(Media, "HTMLMediaElement::selectMediaResource(%p) - using 'src' attribute url", this);
     1406
     1407            // 6. Failed with attribute: Reaching this step indicates that the media resource failed to load
     1408            //    or that the given URL could not be resolved. Queue a task to run the dedicated media source failure steps.
     1409            // 7. Wait for the task queued by the previous step to have executed.
     1410            // 8. Abort these steps. The element won’t attempt to load another resource until this algorithm is triggered again.
    13811411            return;
    13821412        }
    13831413
    1384         // No type or key system information is available when the url comes
    1385         // from the 'src' attribute so MediaPlayer
    1386         // will have to pick a media engine based on the file extension.
    1387         ContentType contentType((String()));
    1388         loadResource(mediaURL, contentType, String());
    1389         LOG(Media, "HTMLMediaElement::selectMediaResource(%p) - using 'src' attribute url", this);
    1390         return;
    1391     }
    1392 
    1393     // Otherwise, the source elements will be used
    1394     loadNextSourceChild();
     1414        // ↳ Otherwise (mode is children)
     1415        // (Ctd. in loadNextSourceChild())
     1416        loadNextSourceChild();
     1417    });
    13951418}
    13961419
    13971420void HTMLMediaElement::loadNextSourceChild()
    13981421{
    1399     ContentType contentType((String()));
     1422    ContentType contentType;
    14001423    String keySystem;
    14011424    URL mediaURL = selectNextSourceChild(&contentType, &keySystem, Complain);
     
    14141437void HTMLMediaElement::loadResource(const URL& initialURL, ContentType& contentType, const String& keySystem)
    14151438{
    1416     ASSERT(isSafeToLoadURL(initialURL, Complain));
     1439    ASSERT(initialURL.isEmpty() || isSafeToLoadURL(initialURL, Complain));
    14171440
    14181441    LOG(Media, "HTMLMediaElement::loadResource(%p) - %s, %s, %s", this, urlForLoggingMedia(initialURL).utf8().data(), contentType.raw().utf8().data(), keySystem.utf8().data());
     
    14311454
    14321455    URL url = initialURL;
    1433     if (!frame->loader().willLoadMediaElementURL(url)) {
     1456    if (!url.isEmpty() && !frame->loader().willLoadMediaElementURL(url)) {
    14341457        mediaLoadingFailed(MediaPlayer::FormatError);
    14351458        return;
     
    14501473    // If the URL should be loaded from the application cache, pass the URL of the cached file to the media engine.
    14511474    ApplicationCacheResource* resource = nullptr;
    1452     if (frame->loader().documentLoader()->applicationCacheHost().shouldLoadResourceFromApplicationCache(ResourceRequest(url), resource)) {
     1475    if (!url.isEmpty() && frame->loader().documentLoader()->applicationCacheHost().shouldLoadResourceFromApplicationCache(ResourceRequest(url), resource)) {
    14531476        // Resources that are not present in the manifest will always fail to load (at least, after the
    14541477        // cache has been primed the first time), making the testing of offline applications simpler.
     
    14841507    setDisplayMode(Unknown);
    14851508
    1486     if (!autoplay())
     1509    if (!autoplay() && !m_havePreparedToPlay)
    14871510        m_player->setPreload(m_mediaSession->effectivePreloadForElement(*this));
    14881511    m_player->setPreservesPitch(m_webkitPreservesPitch);
     
    14981521    bool loadAttempted = false;
    14991522#if ENABLE(MEDIA_SOURCE)
    1500     ASSERT(!m_mediaSource);
    1501 
    1502     if (url.protocolIs(mediaSourceBlobProtocol))
     1523    if (!m_mediaSource && url.protocolIs(mediaSourceBlobProtocol))
    15031524        m_mediaSource = MediaSource::lookup(url.string());
    15041525
    15051526    if (m_mediaSource) {
    1506         if (m_mediaSource->attachToElement(*this))
    1507             m_player->load(url, contentType, m_mediaSource.get());
    1508         else {
     1527        loadAttempted = true;
     1528        if (!m_mediaSource->attachToElement(*this) || !m_player->load(url, contentType, m_mediaSource.get())) {
    15091529            // Forget our reference to the MediaSource, so we leave it alone
    15101530            // while processing remainder of load failure.
     
    15121532            mediaLoadingFailed(MediaPlayer::FormatError);
    15131533        }
    1514         loadAttempted = true;
    15151534    }
    15161535#endif
     
    15281547    }
    15291548#endif
     1549
     1550    if (!loadAttempted && m_blob) {
     1551        loadAttempted = true;
     1552        if (!m_player->load(m_blob->url(), contentType, keySystem))
     1553            mediaLoadingFailed(MediaPlayer::FormatError);
     1554    }
    15301555
    15311556    if (!loadAttempted && !m_player->load(url, contentType, keySystem))
     
    31223147    // 4.8.10.9. Playing the media resource
    31233148    if (!m_player || m_networkState == NETWORK_EMPTY)
    3124         scheduleDelayedAction(LoadMediaResource);
     3149        prepareForLoad();
    31253150
    31263151    if (endedPlayback())
     
    32103235        if (!m_mediaSession->playbackPermitted(*this))
    32113236            return;
    3212         scheduleDelayedAction(LoadMediaResource);
     3237        prepareForLoad();
    32133238    }
    32143239
     
    42564281    // the media element's resource selection algorithm.
    42574282    if (networkState() == HTMLMediaElement::NETWORK_EMPTY) {
    4258         scheduleDelayedAction(LoadMediaResource);
    42594283        m_nextChildNodeToConsider = source;
     4284        selectMediaResource();
    42604285        return;
    42614286    }
     
    50425067#endif
    50435068
     5069    m_blob = nullptr;
     5070
    50445071#if ENABLE(VIDEO_TRACK)
    50455072    forgetResourceSpecificTracks();
     
    52045231        // This behavior is not specified but it seems like a sensible thing to do.
    52055232        // As it is not safe to immedately start loading now, let's schedule a load.
    5206         scheduleDelayedAction(LoadMediaResource);
     5233        prepareForLoad();
    52075234    }
    52085235
     
    57295756    if (m_isWaitingUntilMediaCanStart) {
    57305757        m_isWaitingUntilMediaCanStart = false;
    5731         loadInternal();
     5758        selectMediaResource();
    57325759    }
    57335760    if (m_pausedInternal)
     
    60036030
    60046031#if ENABLE(MEDIA_SOURCE)
    6005     if (m_mediaSource)
    6006         m_mediaSource->detachFromElement(*this);
     6032    detachMediaSource();
    60076033#endif
    60086034
  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r212289 r212311  
    5858class AudioTrackList;
    5959class AudioTrackPrivate;
     60class Blob;
    6061class DOMError;
    6162class DeferredPromise;
     
    9394#endif
    9495
     96using MediaProvider = std::optional<Variant<
     97#if ENABLE(MEDIA_STREAM)
     98    RefPtr<MediaStream>,
     99#endif
     100#if ENABLE(MEDIA_SOURCE)
     101    RefPtr<MediaSource>,
     102#endif
     103    RefPtr<Blob>>>;
     104
    95105class HTMLMediaElement
    96106    : public HTMLElement
     
    165175    const URL& currentSrc() const { return m_currentSrc; }
    166176
    167 #if ENABLE(MEDIA_STREAM)
    168     MediaStream* srcObject() const { return m_mediaStreamSrcObject.get(); }
    169     void setSrcObject(ScriptExecutionContext&, MediaStream*);
    170 #endif
     177    const MediaProvider& srcObject() const { return m_mediaProvider; }
     178    void setSrcObject(MediaProvider&&);
    171179
    172180    WEBCORE_EXPORT void setCrossOrigin(const AtomicString&);
     
    699707
    700708    // These "internal" functions do not check user gesture restrictions.
    701     void loadInternal();
    702709    bool playInternal();
    703710    void pauseInternal();
     
    829836    GenericTaskQueue<Timer> m_updatePlaybackControlsManagerQueue;
    830837    GenericTaskQueue<Timer> m_playbackControlsManagerBehaviorRestrictionsQueue;
     838    GenericTaskQueue<Timer> m_resourceSelectionTaskQueue;
    831839    RefPtr<TimeRanges> m_playedTimeRanges;
    832840    GenericEventQueue m_asyncEventQueue;
     
    10131021    friend class TrackDisplayUpdateScope;
    10141022
     1023    RefPtr<Blob> m_blob;
     1024    MediaProvider m_mediaProvider;
     1025
    10151026#if ENABLE(LEGACY_ENCRYPTED_MEDIA)
    10161027    RefPtr<WebKitMediaKeys> m_webKitMediaKeys;
  • trunk/Source/WebCore/html/HTMLMediaElement.idl

    r210780 r212311  
    2424 */
    2525
     26
     27typedef (
     28#if defined(ENABLE_MEDIA_STREAM) && ENABLE_MEDIA_STREAM
     29    MediaStream or
     30#endif
     31#if defined(ENABLE_MEDIA_SOURCE) && ENABLE_MEDIA_SOURCE
     32    MediaSource or
     33#endif
     34    Blob) MediaProvider;
     35
    2636[
    2737    ActiveDOMObject,
     
    3646    // network state
    3747    [Reflect, URL] attribute USVString src;
     48    attribute MediaProvider? srcObject;
    3849    [URL] readonly attribute USVString currentSrc;
    3950    attribute DOMString? crossOrigin;
  • trunk/Source/WebCore/platform/ContentType.h

    r165676 r212311  
    3535    public:
    3636        explicit ContentType(const String& type);
     37        ContentType() = default;
    3738
    3839        String parameter(const String& parameterName) const;
Note: See TracChangeset for help on using the changeset viewer.