Changeset 147675 in webkit


Ignore:
Timestamp:
Apr 4, 2013, 3:23:47 PM (12 years ago)
Author:
eric.carlson@apple.com
Message:

[Mac] add "automatic" text track menu item
​https://bugs.webkit.org/show_bug.cgi?id=113822

Reviewed by Jer Noble.

Source/WebCore:

No new tests, existing tests updated.

  • English.lproj/Localizable.strings: Add new track menu item.
  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::HTMLMediaElement): Initialize m_captionDisplayMode.
(WebCore::HTMLMediaElement::finishParsingChildren): LoadTextTrackResource -> ConfigureTextTracks.
(WebCore::HTMLMediaElement::scheduleDelayedAction): Ditto.
(WebCore::HTMLMediaElement::loadTimerFired): Ditto.
(WebCore::HTMLMediaElement::loadInternal): Ditto. Flag the caption menu as invalid so it will

be rebuilt again.

(WebCore::HTMLMediaElement::mediaPlayerDidAddTrack): LoadTextTrackResource -> ConfigureTextTracks.
(WebCore::HTMLMediaElement::didAddTrack): Ditto.
(WebCore::HTMLMediaElement::configureTextTrackGroup): Deal with new track display modes.
(WebCore::HTMLMediaElement::HTMLMediaElement::hasClosedCaptions): Minor cleanup.
(WebCore::HTMLMediaElement::configureTextTrackGroup): Add some logging.
(WebCore::HTMLMediaElement::configureTextTracks): Call updateActiveTextTrackCues so cue display

is updated.

(WebCore::HTMLMediaElement::captionPreferencesChanged): Deal with new track display modes.
(WebCore::HTMLMediaElement::markCaptionAndSubtitleTracksAsUnconfigured): Add some logging.

  • html/HTMLMediaElement.h: Ditto.
  • html/shadow/MediaControlElementTypes.cpp: Remove trackIndexAttributeName and trackListIndexForElement.
  • html/shadow/MediaControlElementTypes.h: Ditto.
  • html/shadow/MediaControlElements.cpp:

(WebCore::MediaControlClosedCaptionsTrackListElement::defaultEventHandler): Don't use track index,

everything we need is in the track<->element map now.

(WebCore::MediaControlClosedCaptionsTrackListElement::updateDisplay): Ditto. Always rebuild the

track list because clear the map each time the menu goes away because it refs every track.

(WebCore::MediaControlClosedCaptionsTrackListElement::rebuildTrackListMenu): Don't set the track

index attribute on the menu items, just use the map.

  • html/shadow/MediaControlElements.h:
  • html/shadow/MediaControlsApple.cpp:

(WebCore::MediaControlsApple::changedClosedCaptionsVisibility): resetTrackListMenu is gone.
(WebCore::MediaControlsApple::reset): Ditto.
(WebCore::MediaControlsApple::closedCaptionTracksChanged): Ditto.

  • html/track/TextTrack.cpp:

(WebCore::TextTrack::captionMenuOffItem): New, static TextTrack used for the menu "Off" item.
(WebCore::TextTrack::captionMenuAutomaticItem): New, static TextTrack used for the menu "Automatic" item.

  • html/track/TextTrack.h:
  • page/CaptionUserPreferences.cpp:

(WebCore::CaptionUserPreferences::shouldShowCaptions): Removed, we use captionDisplayMode instead.
(WebCore::CaptionUserPreferences::captionDisplayMode): New.
(WebCore::CaptionUserPreferences::setCaptionDisplayMode): New.
(WebCore::CaptionUserPreferences::setPreferredLanguage): Make the parameter a const reference.
(WebCore::CaptionUserPreferences::textTrackSelectionScore): Return 0 if the user doesn't want

captions or subtitles.

(WebCore::CaptionUserPreferences::textTrackLanguageSelectionScore): Take language as a parameter

instead of assuming that it is always the entire language list.

  • page/CaptionUserPreferences.h:
  • page/CaptionUserPreferencesMac.h:
  • page/CaptionUserPreferencesMac.mm:

(WebCore::CaptionUserPreferencesMac::shouldShowCaptions): Removed, we use captionDisplayMode instead.
(WebCore::CaptionUserPreferencesMac::captionDisplayMode): New.
(WebCore::CaptionUserPreferencesMac::setCaptionDisplayMode): New.
(WebCore::CaptionUserPreferencesMac::setPreferredLanguage): Make the parameter a const reference.
(WebCore::CaptionUserPreferencesMac::textTrackSelectionScore): Deal with "Automatic" mode.

captions or subtitles.

(WebCore::CaptionUserPreferences::textTrackLanguageSelectionScore): Take language as a parameter

instead of assuming that it is always the entire language list.

(WebCore::CaptionUserPreferencesMac::sortedTrackListForMenu): Insert "Off" and "Automatic" items.

  • platform/LocalizedStrings.cpp:

(WebCore::textTrackAutomaticMenuItemText): New.

  • platform/LocalizedStrings.h:
  • platform/graphics/MediaPlayer.cpp:

(WebCore::MediaPlayer::languageOfPrimaryAudioTrack): New.

  • platform/graphics/MediaPlayer.h:
  • platform/graphics/MediaPlayerPrivate.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::languageOfPrimaryAudioTrack): Return the language

of the main audio track.

LayoutTests:

  • media/media-captions.html: Opportunistic fix - don't look at state until we get the 'canplaythrough'

event because the media engine state may not have been processed until that time.

  • media/track/track-cue-container-rendering-position.html: Don't assume that a cue has been

rendered immediately when the 'canplaythrough' event comes through.

  • media/track/track-cue-rendering-mode-changed.html: Ditto.
  • media/track/track-in-band-expected.txt: Don't check if cues have loaded, it doesn't matter

for this test.

  • media/track/track-in-band.html: Ditto.
  • media/track/track-user-preferences-expected.txt: Changed name of utility function from

menuIndexForLanguage to indexOfMenuItemBeginningWith.

  • media/track/track-user-preferences.html: Move menuIndexForLanguage to trackmenu-test.js and

change name to indexOfMenuItemBeginningWith. Clarify the message logged when a track loads
but it should not.

  • media/trackmenu-test.js: Move some functions here from track-user-preferences.html

(trackMenuList):
(indexOfMenuItemBeginningWith):
(selectCaptionMenuItem):

  • media/video-controls-captions-trackmenu-sorted.html: Update for new menu item.
  • media/video-controls-captions-trackmenu.html: Ditto.
  • platform/mac/media/video-controls-captions-trackmenu-expected.txt: Ditto.
  • platform/mac/media/video-controls-captions-trackmenu-sorted-expected.txt: Ditto.
Location:
trunk
Files:
35 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r147674 r147675  
     12013-04-04  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [Mac] add "automatic" text track menu item
     4        https://bugs.webkit.org/show_bug.cgi?id=113822
     5
     6        Reviewed by Jer Noble.
     7
     8        * media/media-captions.html: Opportunistic fix - don't look at state until we get the 'canplaythrough'
     9            event because the media engine state may not have been processed until that time.
     10        * media/track/track-cue-container-rendering-position.html: Don't assume that a cue has been
     11            rendered immediately when the 'canplaythrough' event comes through.
     12        * media/track/track-cue-rendering-mode-changed.html: Ditto.
     13        * media/track/track-in-band-expected.txt: Don't check if cues have loaded, it doesn't matter
     14            for this test.
     15        * media/track/track-in-band.html: Ditto.
     16        * media/track/track-user-preferences-expected.txt: Changed name of utility function from
     17            menuIndexForLanguage to indexOfMenuItemBeginningWith.
     18        * media/track/track-user-preferences.html: Move menuIndexForLanguage to trackmenu-test.js and
     19            change name to indexOfMenuItemBeginningWith. Clarify the message logged when a track loads
     20            but it should not.
     21        * media/trackmenu-test.js: Move some functions here from track-user-preferences.html
     22        (trackMenuList):
     23        (indexOfMenuItemBeginningWith):
     24        (selectCaptionMenuItem):
     25        * media/video-controls-captions-trackmenu-sorted.html: Update for new menu item.
     26        * media/video-controls-captions-trackmenu.html: Ditto.
     27        * platform/mac/media/video-controls-captions-trackmenu-expected.txt: Ditto.
     28        * platform/mac/media/video-controls-captions-trackmenu-sorted-expected.txt: Ditto.
     29
    1302013-04-04  Glenn Adams  <glenn@skynav.com>
    231

  • trunk/LayoutTests/media/media-captions.html

    r79630 r147675  
    7575                waitForEvent("playing");
    7676
    77                 waitForEvent('loadedmetadata', canplaythrough);
     77                waitForEvent('canplaythrough', canplaythrough);
    7878
    7979                consoleWrite("<b>Test before movie is open:</b>");

  • trunk/LayoutTests/media/track/track-cue-container-rendering-position.html

    r134783 r147675  
    3131            findMediaElement();
    3232            video.src = findMediaFile('video', '../content/test');
    33             waitForEvent('canplaythrough', testPosition);
     33            waitForEvent('canplaythrough', function () {setTimeout(testPosition, 100);});
    3434        }
    3535

  • trunk/LayoutTests/media/track/track-cue-rendering-mode-changed.html

    r139932 r147675  
    1313        var testTrackEnglish;
    1414        var testCueDisplayBox;
     15        var currentTest = 0;
     16
     17        function scheduleNextTest()
     18        {
     19            var test;
     20            switch (++currentTest) {
     21            case 1:
     22                test = addTracks;
     23                break;
     24            case 2:
     25                test = testCueActiveState;
     26                break;
     27            case 3:
     28                test = testCueVisibility1;
     29                break;
     30            case 4:
     31                test = testCueVisibility2;
     32                break;
     33            case 5:
     34                endTest();
     35                return;
     36            }
     37
     38            window.setTimeout(test, 100);
     39        }
    1540
    1641        function addTracks()
    … …  
    3257            consoleWrite("Set the mode of the 'English' track to hidden");
    3358            testTrackEnglish.mode = "hidden";
     59
     60            scheduleNextTest();
    3461        }
    3562
    … …  
    4370            testExpected("testTrackArabic.activeCues.length", 1);
    4471            testExpected("testTrackArabic.activeCues[0].text", "Arabic");
     72
     73            scheduleNextTest();
    4574        }
    4675
    47         function testCueVisibility()
     76        function testCueVisibility1()
    4877        {
    4978            consoleWrite("");
    … …  
    5786            testTrackEnglish.mode = "showing";
    5887
     88            scheduleNextTest();
     89        }
     90
     91        function testCueVisibility2()
     92        {
    5993            consoleWrite("");
    6094            consoleWrite("** Both cues shold be visible. **");
    … …  
    6498            testCueDisplayBox = textTrackDisplayElement(video, 'display', 1);
    6599            testExpected("testCueDisplayBox.innerText", "English");
    66         }
    67100
    68         function runTests() {
    69             addTracks();
    70             testCueActiveState();
    71             testCueVisibility();
    72             endTest();
     101            scheduleNextTest();
    73102        }
    74103
    … …  
    80109            video.src = findMediaFile('video', '../content/test');
    81110
    82             waitForEvent('canplaythrough', runTests);
     111            waitForEvent('canplaythrough', scheduleNextTest);
    83112        }
    84113

  • trunk/LayoutTests/media/track/track-in-band-expected.txt

    r137844 r147675  
    1313RUN(inbandTrack1 = video.textTracks[0])
    1414RUN(inbandTrack2 = video.textTracks[1])
    15 EXPECTED (inbandTrack1.mode == 'disabled') OK
    16 EXPECTED (inbandTrack1.cues == 'null') OK
    1715EXPECTED (inbandTrack1.language == 'en') OK
    1816EXPECTED (inbandTrack1.kind == 'subtitles') OK
    19 EXPECTED (inbandTrack2.mode == 'disabled') OK
    20 EXPECTED (inbandTrack2.cues == 'null') OK
    2117EXPECTED (inbandTrack2.language == 'fr') OK
    2218EXPECTED (inbandTrack2.kind == 'subtitles') OK

  • trunk/LayoutTests/media/track/track-in-band.html

    r137844 r147675  
    3030                run("inbandTrack1 = video.textTracks[0]");
    3131                run("inbandTrack2 = video.textTracks[1]");
    32                 testExpected("inbandTrack1.mode", "disabled");
    33                 testExpected("inbandTrack1.cues", null);
    3432                testExpected("inbandTrack1.language", "en");
    3533                testExpected("inbandTrack1.kind", "subtitles");
    36                 testExpected("inbandTrack2.mode", "disabled");
    37                 testExpected("inbandTrack2.cues", null);
    3834                testExpected("inbandTrack2.language", "fr");
    3935                testExpected("inbandTrack2.kind", "subtitles");

  • trunk/LayoutTests/media/track/track-user-preferences-expected.txt

    r146380 r147675  
    1919Test 3: select 'fr' track from menu
    2020- show captions menu.
    21 EXPECTED (trackMenuItems[menuIndexForLanguage('French')].className == '') OK
    22 EXPECTED (trackMenuItems[menuIndexForLanguage('English')].className == '') OK
    23 EXPECTED (trackMenuItems[menuIndexForLanguage('Norwegian')].className == 'selected') OK
     21EXPECTED (trackMenuItems[indexOfMenuItemBeginningWith('French')].className == '') OK
     22EXPECTED (trackMenuItems[indexOfMenuItemBeginningWith('English')].className == '') OK
     23EXPECTED (trackMenuItems[indexOfMenuItemBeginningWith('Norwegian')].className == 'selected') OK
    2424- click on 'French' menu item.
    2525EVENT(load)

  • trunk/LayoutTests/media/track/track-user-preferences.html

    r146380 r147675  
    2424            }
    2525
    26             function trackMenuList()
    27             {
    28                 trackListElement = getTrackListElement();
    29                 if (!trackListElement)
    30                     return;
    31 
    32                 // Track list should have a <ul> with <li> children (One of them is "Off").
    33                 var trackList = trackListElement.querySelector("ul");
    34                 if (!trackList) {
    35                     failTest("Could not find a child ul element in track list menu");
    36                     return;
    37                 }
    38                 var trackListItems = trackList.querySelectorAll("li");
    39                 if (!trackListItems) {
    40                     failTest("Could not find child li elements in track list menu");
    41                     return;
    42                 }
    43 
    44                 return trackListItems;
    45             }
    46 
    47             function menuIndexForLanguage(language)
    48             {
    49                 var trackMenuItems = trackMenuList();
    50                 for (i = 0; i < trackMenuItems.length; ++i) {
    51                     if (trackMenuItems[i].textContent.indexOf(language) >= 0)
    52                         break;
    53                 }
    54                 return (i < trackMenuItems.length) ? i : -1;
    55             }
    56 
    57             function selectCaptionMenuItem(language)
    58             {
    59                 var trackMenuItems = trackMenuList();
    60                 var index = menuIndexForLanguage(language);
    61                 if (index < 0) {
    62                     failTest("Menu item " + language + " not found in track list menu.");
    63                     return;
    64                 }
    65 
    66                 consoleWrite("- click on '" + language + "' menu item.");
    67                 var selectedTrackItem = trackMenuItems[index];
    68                 var boundingRect = selectedTrackItem.getBoundingClientRect();
    69                 var x = boundingRect.left + boundingRect.width / 2;
    70                 var y = boundingRect.top + boundingRect.height / 2;
    71                 eventSender.mouseMoveTo(x, y);
    72                 eventSender.mouseDown();
    73                 eventSender.mouseUp();
    74             }
    75 
    7626            function trackLoaded()
    7727            {
    7828                consoleWrite("EVENT(load)");
     29                if (expectedLanguage == "") {
     30                    failTest("'" + event.target.srclang + "' track loaded unexpectedly.");
     31                    return;
     32                }
    7933                testExpected("event.target.srclang", expectedLanguage);
    8034                testExpected("event.target.readyState", HTMLTrackElement.LOADED);
    … …  
    13185                case 4:
    13286                    trackMenuItems = trackMenuList();
    133                     testExpected("trackMenuItems[menuIndexForLanguage('French')].className", "");
    134                     testExpected("trackMenuItems[menuIndexForLanguage('English')].className", "");
    135                     testExpected("trackMenuItems[menuIndexForLanguage('Norwegian')].className", "selected");
     87                    testExpected("trackMenuItems[indexOfMenuItemBeginningWith('French')].className", "");
     88                    testExpected("trackMenuItems[indexOfMenuItemBeginningWith('English')].className", "");
     89                    testExpected("trackMenuItems[indexOfMenuItemBeginningWith('Norwegian')].className", "selected");
    13690
    13791                    selectCaptionMenuItem("French");
    … …  
    158112                case 7:
    159113                    selectCaptionMenuItem("Off");
     114                    expectedLanguage = '';
    160115                    createTrackElements([ 'ru', 'jp', 'en']);
    161116                    timer = setTimeout(nextStep, 100);

  • trunk/LayoutTests/media/trackmenu-test.js

    r138071 r147675  
    3636  return trackListElement;
    3737}
     38
     39function trackMenuList()
     40{
     41    trackListElement = getTrackListElement();
     42    if (!trackListElement)
     43        return;
     44
     45    // Track list should have a <ul> with <li> children.
     46    var trackList = trackListElement.querySelector("ul");
     47    if (!trackList) {
     48        failTest("Could not find a child ul element in track list menu");
     49        return;
     50    }
     51    var trackListItems = trackList.querySelectorAll("li");
     52    if (!trackListItems) {
     53        failTest("Could not find child li elements in track list menu");
     54        return;
     55    }
     56
     57    return trackListItems;
     58}
     59
     60function indexOfMenuItemBeginningWith(title)
     61{
     62    var trackMenuItems = trackMenuList();
     63    for (i = 0; i < trackMenuItems.length; ++i) {
     64        if (trackMenuItems[i].textContent.indexOf(title) == 0)
     65            break;
     66    }
     67    return (i < trackMenuItems.length) ? i : -1;
     68}
     69
     70function selectCaptionMenuItem(title)
     71{
     72    var trackMenuItems = trackMenuList();
     73    var index = indexOfMenuItemBeginningWith(title);
     74    if (index < 0) {
     75        failTest("Menu item " + title + " not found in track list menu.");
     76        return;
     77    }
     78
     79    consoleWrite("- click on '" + title + "' menu item.");
     80    var selectedTrackItem = trackMenuItems[index];
     81    var boundingRect = selectedTrackItem.getBoundingClientRect();
     82    var x = boundingRect.left + boundingRect.width / 2;
     83    var y = boundingRect.top + boundingRect.height / 2;
     84    eventSender.mouseMoveTo(x, y);
     85    eventSender.mouseDown();
     86    eventSender.mouseUp();
     87}
     88

  • trunk/LayoutTests/media/video-controls-captions-trackmenu-sorted.html

    r146380 r147675  
    4242                    trackMenuItems = textTrackMenuItems();
    4343                    testExpected("trackMenuItems[0].textContent", "Off");
    44                     testExpected("trackMenuItems[1].textContent", "u (English) SDH");
    45                     testExpected("trackMenuItems[2].textContent", "y (English-Australia) SDH");
    46                     testExpected("trackMenuItems[3].textContent", "v (English)");
    47                     testExpected("trackMenuItems[4].textContent", "x (English-Australia)");
    48                     testExpected("trackMenuItems[5].textContent", "w (Japanese) SDH");
    49                     testExpected("trackMenuItems[6].textContent", "z (Japanese)");
     44                    testExpected("trackMenuItems[1].textContent", "Automatic (English)");
     45                    testExpected("trackMenuItems[2].textContent", "u (English) SDH");
     46                    testExpected("trackMenuItems[3].textContent", "y (English-Australia) SDH");
     47                    testExpected("trackMenuItems[4].textContent", "v (English)");
     48                    testExpected("trackMenuItems[5].textContent", "x (English-Australia)");
     49                    testExpected("trackMenuItems[6].textContent", "w (Japanese) SDH");
     50                    testExpected("trackMenuItems[7].textContent", "z (Japanese)");
    5051                    testExpected("video.textTracks.length", 6);
    51                     testExpected("trackMenuItems.length", 7);
     52                    testExpected("trackMenuItems.length", 8);
    5253
    5354                    consoleWrite("<br><i>** Change preferred language to Japanese, change preference to subtitles<" + "/i>");
    … …  
    6263                    trackMenuItems = textTrackMenuItems();
    6364                    testExpected("trackMenuItems[0].textContent", "Off");
    64                     testExpected("trackMenuItems[1].textContent", "z (Japanese)");
    65                     testExpected("trackMenuItems[2].textContent", "v (English)");
    66                     testExpected("trackMenuItems[3].textContent", "x (English-Australia)");
     65                    testExpected("trackMenuItems[1].textContent", "Automatic (Japanese)");
     66                    testExpected("trackMenuItems[2].textContent", "z (Japanese)");
     67                    testExpected("trackMenuItems[3].textContent", "v (English)");
     68                    testExpected("trackMenuItems[4].textContent", "x (English-Australia)");
    6769                    testExpected("video.textTracks.length", 6);
    68                     testExpected("trackMenuItems.length", 4);
     70                    testExpected("trackMenuItems.length", 5);
    6971
    7072                    consoleWrite("");

  • trunk/LayoutTests/media/video-controls-captions-trackmenu.html

    r146380 r147675  
    4242            {
    4343                trackListItems = trackMenuList();
    44                 var expectedItemCount = video.textTracks.length + 1;
     44                var expectedItemCount = video.textTracks.length + 2;
    4545                consoleWrite("There should be " + expectedItemCount + " items in the menu.");
    4646                testExpected("trackListItems.length", expectedItemCount);
    … …  
    6464            {
    6565                consoleWrite("*** Turning captions on");
    66                 // Click on the second item, which is the second track (Off is the first item in the menu)
    67                 selectCaptionMenuItem(2, testCaptionsVisible);
     66                // Click on the third item, which is the second track (Off is the first item, Automatic is second)
     67                selectCaptionMenuItem(3, testCaptionsVisible);
    6868            }
    6969

  • trunk/LayoutTests/platform/mac/media/video-controls-captions-trackmenu-expected.txt

    r146380 r147675  
    1010
    1111*** Turning captions on
    12 There should be 5 items in the menu.
    13 EXPECTED (trackListItems.length == '5') OK
     12There should be 6 items in the menu.
     13EXPECTED (trackListItems.length == '6') OK
    1414EXPECTED (video.textTracks.length == '4') OK
    1515Track 0 should be showing
    … …  
    2626RUN(video.removeChild(document.querySelectorAll("track")[0]))
    2727*** Turning captions off
    28 There should be 4 items in the menu.
    29 EXPECTED (trackListItems.length == '4') OK
     28There should be 5 items in the menu.
     29EXPECTED (trackListItems.length == '5') OK
    3030EXPECTED (video.textTracks.length == '3') OK
    3131Track 0 should be disabled

  • trunk/LayoutTests/platform/mac/media/video-controls-captions-trackmenu-sorted-expected.txt

    r146380 r147675  
    99RUN(internals.settings.setShouldDisplayTrackKind('Captions', true))
    1010EXPECTED (trackMenuItems[0].textContent == 'Off') OK
    11 EXPECTED (trackMenuItems[1].textContent == 'u (English) SDH') OK
    12 EXPECTED (trackMenuItems[2].textContent == 'y (English-Australia) SDH') OK
    13 EXPECTED (trackMenuItems[3].textContent == 'v (English)') OK
    14 EXPECTED (trackMenuItems[4].textContent == 'x (English-Australia)') OK
    15 EXPECTED (trackMenuItems[5].textContent == 'w (Japanese) SDH') OK
    16 EXPECTED (trackMenuItems[6].textContent == 'z (Japanese)') OK
     11EXPECTED (trackMenuItems[1].textContent == 'Automatic (English)') OK
     12EXPECTED (trackMenuItems[2].textContent == 'u (English) SDH') OK
     13EXPECTED (trackMenuItems[3].textContent == 'y (English-Australia) SDH') OK
     14EXPECTED (trackMenuItems[4].textContent == 'v (English)') OK
     15EXPECTED (trackMenuItems[5].textContent == 'x (English-Australia)') OK
     16EXPECTED (trackMenuItems[6].textContent == 'w (Japanese) SDH') OK
     17EXPECTED (trackMenuItems[7].textContent == 'z (Japanese)') OK
    1718EXPECTED (video.textTracks.length == '6') OK
    18 EXPECTED (trackMenuItems.length == '7') OK
     19EXPECTED (trackMenuItems.length == '8') OK
    1920
    2021** Change preferred language to Japanese, change preference to subtitles
    … …  
    2324RUN(internals.settings.setShouldDisplayTrackKind('Subtitles', true))
    2425EXPECTED (trackMenuItems[0].textContent == 'Off') OK
    25 EXPECTED (trackMenuItems[1].textContent == 'z (Japanese)') OK
    26 EXPECTED (trackMenuItems[2].textContent == 'v (English)') OK
    27 EXPECTED (trackMenuItems[3].textContent == 'x (English-Australia)') OK
     26EXPECTED (trackMenuItems[1].textContent == 'Automatic (Japanese)') OK
     27EXPECTED (trackMenuItems[2].textContent == 'z (Japanese)') OK
     28EXPECTED (trackMenuItems[3].textContent == 'v (English)') OK
     29EXPECTED (trackMenuItems[4].textContent == 'x (English-Australia)') OK
    2830EXPECTED (video.textTracks.length == '6') OK
    29 EXPECTED (trackMenuItems.length == '4') OK
     31EXPECTED (trackMenuItems.length == '5') OK
    3032
    3133END OF TEST

  • trunk/Source/WebCore/ChangeLog

    r147671 r147675  
     12013-04-04  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [Mac] add "automatic" text track menu item
     4        https://bugs.webkit.org/show_bug.cgi?id=113822
     5
     6        Reviewed by Jer Noble.
     7
     8        No new tests, existing tests updated.
     9
     10        * English.lproj/Localizable.strings: Add new track menu item.
     11
     12        * html/HTMLMediaElement.cpp:
     13        (WebCore::HTMLMediaElement::HTMLMediaElement): Initialize m_captionDisplayMode.
     14        (WebCore::HTMLMediaElement::finishParsingChildren): LoadTextTrackResource -> ConfigureTextTracks.
     15        (WebCore::HTMLMediaElement::scheduleDelayedAction): Ditto.
     16        (WebCore::HTMLMediaElement::loadTimerFired): Ditto.
     17        (WebCore::HTMLMediaElement::loadInternal): Ditto. Flag the caption menu as invalid so it will
     18            be rebuilt again.
     19        (WebCore::HTMLMediaElement::mediaPlayerDidAddTrack): LoadTextTrackResource -> ConfigureTextTracks.
     20        (WebCore::HTMLMediaElement::didAddTrack): Ditto.
     21        (WebCore::HTMLMediaElement::configureTextTrackGroup): Deal with new track display modes.
     22        (WebCore::HTMLMediaElement::HTMLMediaElement::hasClosedCaptions): Minor cleanup.
     23        (WebCore::HTMLMediaElement::configureTextTrackGroup): Add some logging.
     24        (WebCore::HTMLMediaElement::configureTextTracks): Call updateActiveTextTrackCues so cue display
     25            is updated.
     26        (WebCore::HTMLMediaElement::captionPreferencesChanged): Deal with new track display modes.
     27        (WebCore::HTMLMediaElement::markCaptionAndSubtitleTracksAsUnconfigured): Add some logging.
     28        * html/HTMLMediaElement.h: Ditto.
     29
     30        * html/shadow/MediaControlElementTypes.cpp: Remove trackIndexAttributeName and trackListIndexForElement.
     31        * html/shadow/MediaControlElementTypes.h: Ditto.
     32
     33        * html/shadow/MediaControlElements.cpp:
     34        (WebCore::MediaControlClosedCaptionsTrackListElement::defaultEventHandler): Don't use track index,
     35            everything we need is in the track<->element map now.
     36        (WebCore::MediaControlClosedCaptionsTrackListElement::updateDisplay): Ditto. Always rebuild the
     37            track list because clear the map each time the menu goes away because it refs every track.
     38        (WebCore::MediaControlClosedCaptionsTrackListElement::rebuildTrackListMenu): Don't set the track
     39            index attribute on the menu items, just use the map.
     40        * html/shadow/MediaControlElements.h:
     41
     42        * html/shadow/MediaControlsApple.cpp:
     43        (WebCore::MediaControlsApple::changedClosedCaptionsVisibility): resetTrackListMenu is gone.
     44        (WebCore::MediaControlsApple::reset): Ditto.
     45        (WebCore::MediaControlsApple::closedCaptionTracksChanged): Ditto.
     46
     47        * html/track/TextTrack.cpp:
     48        (WebCore::TextTrack::captionMenuOffItem): New, static TextTrack used for the menu "Off" item.
     49        (WebCore::TextTrack::captionMenuAutomaticItem): New, static TextTrack used for the menu "Automatic" item.
     50        * html/track/TextTrack.h:
     51
     52        * page/CaptionUserPreferences.cpp:
     53        (WebCore::CaptionUserPreferences::shouldShowCaptions): Removed, we use captionDisplayMode instead.
     54        (WebCore::CaptionUserPreferences::captionDisplayMode): New.
     55        (WebCore::CaptionUserPreferences::setCaptionDisplayMode): New.
     56        (WebCore::CaptionUserPreferences::setPreferredLanguage): Make the parameter a const reference.
     57        (WebCore::CaptionUserPreferences::textTrackSelectionScore): Return 0 if the user doesn't want
     58            captions or subtitles.
     59        (WebCore::CaptionUserPreferences::textTrackLanguageSelectionScore): Take language as a parameter
     60            instead of assuming that it is always the entire language list.
     61        * page/CaptionUserPreferences.h:
     62
     63        * page/CaptionUserPreferencesMac.h:
     64        * page/CaptionUserPreferencesMac.mm:
     65        (WebCore::CaptionUserPreferencesMac::shouldShowCaptions): Removed, we use captionDisplayMode instead.
     66        (WebCore::CaptionUserPreferencesMac::captionDisplayMode): New.
     67        (WebCore::CaptionUserPreferencesMac::setCaptionDisplayMode): New.
     68        (WebCore::CaptionUserPreferencesMac::setPreferredLanguage): Make the parameter a const reference.
     69        (WebCore::CaptionUserPreferencesMac::textTrackSelectionScore): Deal with "Automatic" mode.
     70            captions or subtitles.
     71        (WebCore::CaptionUserPreferences::textTrackLanguageSelectionScore): Take language as a parameter
     72            instead of assuming that it is always the entire language list.
     73        (WebCore::CaptionUserPreferencesMac::sortedTrackListForMenu): Insert "Off" and "Automatic" items.
     74
     75        * platform/LocalizedStrings.cpp:
     76        (WebCore::textTrackAutomaticMenuItemText): New.
     77        * platform/LocalizedStrings.h:
     78
     79        * platform/graphics/MediaPlayer.cpp:
     80        (WebCore::MediaPlayer::languageOfPrimaryAudioTrack): New.
     81        * platform/graphics/MediaPlayer.h:
     82        * platform/graphics/MediaPlayerPrivate.h:
     83
     84        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
     85        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     86        (WebCore::MediaPlayerPrivateAVFoundationObjC::languageOfPrimaryAudioTrack): Return the language
     87            of the main audio track.
     88
    1892013-04-04  Alexey Proskuryakov  <ap@apple.com>
    290

  • trunk/Source/WebCore/English.lproj/Localizable.strings

    r147562 r147675  
    7777"Align Right (Undo action name)" = "Align Right";
    7878
     79/* Menu item label for automatic track selection behavior in the form of 'Automatic (SystemLanguage)' */
     80"Automatic (%@)" = "Automatic (%@)";
     81
    7982/* Back context menu item */
    8083"Back" = "Back";

  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r147521 r147675  
    296296    , m_processingPreferenceChange(false)
    297297    , m_lastTextTrackUpdateTime(-1)
     298    , m_captionDisplayMode(CaptionUserPreferences::Automatic)
    298299    , m_textTracks(0)
    299300    , m_ignoreTrackDisplayUpdate(0)
    … …  
    497498    for (Node* node = firstChild(); node; node = node->nextSibling()) {
    498499        if (node->hasTagName(trackTag)) {
    499             scheduleDelayedAction(LoadTextTrackResource);
     500            scheduleDelayedAction(ConfigureTextTracks);
    500501            break;
    501502        }
    … …  
    611612
    612613#if ENABLE(VIDEO_TRACK)
    613     if (RuntimeEnabledFeatures::webkitVideoTrackEnabled() && (actionType & LoadTextTrackResource))
    614         m_pendingActionFlags |= LoadTextTrackResource;
     614    if (RuntimeEnabledFeatures::webkitVideoTrackEnabled() && (actionType & ConfigureTextTracks))
     615        m_pendingActionFlags |= ConfigureTextTracks;
    615616#endif
    616617
    … …  
    620621#endif
    621622
    622     if (!m_loadTimer.isActive())
    623         m_loadTimer.startOneShot(0);
     623    m_loadTimer.startOneShot(0);
    624624}
    625625
    … …  
    647647
    648648#if ENABLE(VIDEO_TRACK)
    649     if (RuntimeEnabledFeatures::webkitVideoTrackEnabled() && (m_pendingActionFlags & LoadTextTrackResource))
     649    if (RuntimeEnabledFeatures::webkitVideoTrackEnabled() && (m_pendingActionFlags & ConfigureTextTracks))
    650650        configureTextTracks();
    651651#endif
    … …  
    831831
    832832#if ENABLE(VIDEO_TRACK)
     833    if (hasMediaControls())
     834        mediaControls()->changedClosedCaptionsVisibility();
     835
    833836    // HTMLMediaElement::textTracksAreReady will need "... the text tracks whose mode was not in the
    834837    // disabled state when the element's resource selection algorithm last started".
    … …  
    28222825    // the relevant specification for the data.
    28232826    //  - This will happen in configureTextTracks()
    2824     scheduleDelayedAction(LoadTextTrackResource);
     2827    scheduleDelayedAction(ConfigureTextTracks);
    28252828   
    28262829    // 8. Add the new text track to the media element's list of text tracks.
    … …  
    30243027    // in the markup have been added.
    30253028    if (!m_parsingInProgress)
    3026         scheduleDelayedAction(LoadTextTrackResource);
     3029        scheduleDelayedAction(ConfigureTextTracks);
    30273030
    30283031    if (hasMediaControls())
    … …  
    30643067}
    30653068
    3066 bool HTMLMediaElement::userPrefersCaptions() const
    3067 {
    3068     Page* page = document()->page();
    3069     if (!page)
    3070         return false;
    3071 
    3072     CaptionUserPreferences* captionPreferences = page->group().captionPreferences();
    3073     return captionPreferences->userHasCaptionPreferences() && captionPreferences->shouldShowCaptions();
    3074 }
    3075 
    30763069void HTMLMediaElement::configureTextTrackGroup(const TrackGroup& group)
    30773070{
    30783071    ASSERT(group.tracks.size());
     3072
     3073    LOG(Media, "HTMLMediaElement::configureTextTrackGroup");
    30793074
    30803075    Page* page = document()->page();
    … …  
    31483143    if (!trackList || !trackList->length())
    31493144        return;
    3150     if (trackToSelect && !trackList->contains(trackToSelect))
    3151         return;
    3152 
    3153     for (int i = 0, length = trackList->length(); i < length; ++i) {
    3154         TextTrack* track = trackList->item(i);
    3155         if (!trackToSelect || track != trackToSelect)
    3156             track->setMode(TextTrack::disabledKeyword());
    3157         else
    3158             track->setMode(TextTrack::showingKeyword());
     3145
     3146    if (trackToSelect != TextTrack::captionMenuOffItem() && trackToSelect != TextTrack::captionMenuAutomaticItem()) {
     3147        if (!trackList->contains(trackToSelect))
     3148            return;
     3149       
     3150        for (int i = 0, length = trackList->length(); i < length; ++i) {
     3151            TextTrack* track = trackList->item(i);
     3152            if (!trackToSelect || track != trackToSelect)
     3153                track->setMode(TextTrack::disabledKeyword());
     3154            else
     3155                track->setMode(TextTrack::showingKeyword());
     3156        }
    31593157    }
    31603158
    31613159    CaptionUserPreferences* captionPreferences = document()->page() ? document()->page()->group().captionPreferences() : 0;
    3162     if (captionPreferences) {
    3163         captionPreferences->setShouldShowCaptions(trackToSelect);
    3164         if (trackToSelect && trackToSelect->language().length())
     3160    if (!captionPreferences)
     3161        return;
     3162
     3163    CaptionUserPreferences::CaptionDisplayMode displayMode = captionPreferences->captionDisplayMode();
     3164    if (trackToSelect == TextTrack::captionMenuOffItem())
     3165        displayMode = CaptionUserPreferences::ForcedOnly;
     3166    else if (trackToSelect == TextTrack::captionMenuAutomaticItem())
     3167        displayMode = CaptionUserPreferences::Automatic;
     3168    else {
     3169        displayMode = CaptionUserPreferences::AlwaysOn;
     3170        if (trackToSelect->language().length())
    31653171            captionPreferences->setPreferredLanguage(trackToSelect->language());
    3166     }
     3172       
     3173        // Set m_captionDisplayMode here so we don't reconfigure again when the preference changed notification comes through.
     3174        m_captionDisplayMode = displayMode;
     3175    }
     3176
     3177    captionPreferences->setCaptionDisplayMode(displayMode);
    31673178}
    31683179
    … …  
    42494260
    42504261#if ENABLE(VIDEO_TRACK)
    4251     if (RuntimeEnabledFeatures::webkitVideoTrackEnabled() && m_textTracks)
     4262    if (!RuntimeEnabledFeatures::webkitVideoTrackEnabled() || !m_textTracks)
     4263        return false;
     4264
    42524265    for (unsigned i = 0; i < m_textTracks->length(); ++i) {
    42534266        if (m_textTracks->item(i)->readinessState() == TextTrack::FailedToLoad)
    … …  
    42594272    }
    42604273#endif
     4274
    42614275    return false;
    42624276}
    … …  
    44584472    if (m_processingPreferenceChange)
    44594473        return;
     4474
     4475    LOG(Media, "HTMLMediaElement::configureTextTrackDisplay");
    44604476
    44614477    bool haveVisibleTextTrack = false;
    … …  
    44674483    }
    44684484
    4469     if (m_haveVisibleTextTrack == haveVisibleTextTrack)
    4470         return;
     4485    if (m_haveVisibleTextTrack == haveVisibleTextTrack) {
     4486        updateActiveTextTrackCues(currentTime());
     4487        return;
     4488    }
     4489
    44714490    m_haveVisibleTextTrack = haveVisibleTextTrack;
    44724491    m_closedCaptionsVisible = m_haveVisibleTextTrack;
    … …  
    44794498    mediaControls()->changedClosedCaptionsVisibility();
    44804499   
    4481     if (RuntimeEnabledFeatures::webkitVideoTrackEnabled())
     4500    if (RuntimeEnabledFeatures::webkitVideoTrackEnabled()) {
    44824501        updateTextTrackDisplay();
     4502        updateActiveTextTrackCues(currentTime());
     4503    }
    44834504}
    44844505
    … …  
    44914512        mediaControls()->textTrackPreferencesChanged();
    44924513
    4493     setClosedCaptionsVisible(userPrefersCaptions());
     4514    if (!document()->page())
     4515        return;
     4516
     4517    CaptionUserPreferences::CaptionDisplayMode displayMode = document()->page()->group().captionPreferences()->captionDisplayMode();
     4518    if (m_captionDisplayMode == displayMode)
     4519        return;
     4520
     4521    m_captionDisplayMode = displayMode;
     4522    setClosedCaptionsVisible(m_captionDisplayMode == CaptionUserPreferences::AlwaysOn);
    44944523}
    44954524
    … …  
    44984527    if (!m_textTracks)
    44994528        return;
     4529
     4530    LOG(Media, "HTMLMediaElement::markCaptionAndSubtitleTracksAsUnconfigured");
    45004531
    45014532    // Mark all tracks as not "configured" so that configureTextTracks()

  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r147001 r147675  
    4040
    4141#if ENABLE(VIDEO_TRACK)
     42#include "CaptionUserPreferences.h"
    4243#include "PODIntervalTree.h"
    4344#include "TextTrack.h"
    … …  
    113114    enum DelayedActionType {
    114115        LoadMediaResource = 1 << 0,
    115         LoadTextTrackResource = 1 << 1,
    116         TextTrackChangesNotification = 1 << 2
     116        ConfigureTextTracks = 1 << 1,
     117        TextTrackChangesNotification = 1 << 2,
     118        ConfigureTextTrackDisplay = 1 << 3,
    117119    };
    118120    void scheduleDelayedAction(DelayedActionType);
    … …  
    269271
    270272    void setSelectedTextTrack(TextTrack*);
    271     static int textTracksOffIndex() { return -1; }
    272     static int textTracksIndexNotFound() { return -2; }
    273 
    274     bool userPrefersCaptions() const;
     273
    275274    bool textTracksAreReady() const;
    276275    void configureTextTrackDisplay();
    … …  
    697696    bool m_haveVisibleTextTrack : 1;
    698697    bool m_processingPreferenceChange : 1;
     698
    699699    float m_lastTextTrackUpdateTime;
     700    CaptionUserPreferences::CaptionDisplayMode m_captionDisplayMode;
    700701
    701702    RefPtr<TextTrackList> m_textTracks;

  • trunk/Source/WebCore/html/shadow/MediaControlElementTypes.cpp

    r146992 r147675  
    7676}
    7777
    78 #if ENABLE(VIDEO_TRACK)
    79 const AtomicString& trackIndexAttributeName()
    80 {
    81     DEFINE_STATIC_LOCAL(AtomicString, name, ("x-webkit-track-index", AtomicString::ConstructFromLiteral));
    82     return name;
    83 }
    84 
    85 int trackListIndexForElement(Element* element)
    86 {
    87     const AtomicString trackIndexAttributeValue = element->getAttribute(trackIndexAttributeName());
    88     if (trackIndexAttributeValue.isNull() || trackIndexAttributeValue.isEmpty())
    89         return HTMLMediaElement::textTracksIndexNotFound();
    90     bool ok;
    91     int trackIndex = trackIndexAttributeValue.toInt(&ok);
    92     if (!ok)
    93         return HTMLMediaElement::textTracksIndexNotFound();
    94     return trackIndex;
    95 }
    96 #endif
    97 
    9878MediaControlElement::MediaControlElement(MediaControlElementType displayType, HTMLElement* element)
    9979    : m_mediaController(0)

  • trunk/Source/WebCore/html/shadow/MediaControlElementTypes.h

    r146992 r147675  
    8080MediaControlElementType mediaControlElementType(Node*);
    8181
    82 #if ENABLE(VIDEO_TRACK)
    83 const AtomicString& trackIndexAttributeName();
    84 int trackListIndexForElement(Element*);
    85 #endif
    86 
    8782// ----------------------------
    8883

  • trunk/Source/WebCore/html/shadow/MediaControlElements.cpp

    r146992 r147675  
    7171static const AtomicString& getMediaControlTimeRemainingDisplayElementShadowPseudoId();
    7272
    73 #if ENABLE(VIDEO_TRACK)
    74 static const char* textTracksOffAttrValue = "-1"; // This must match HTMLMediaElement::textTracksOffIndex()
    75 #endif
    76 
    7773MediaControlPanelElement::MediaControlPanelElement(Document* document)
    7874    : MediaControlDivElement(document, MediaControlsPanel)
    … …  
    759755    : MediaControlDivElement(document, MediaClosedCaptionsTrackList)
    760756    , m_controls(controls)
    761     , m_trackListHasChanged(true)
    762757{
    763758}
    … …  
    789784        m_menuToTrackMap.clear();
    790785        m_controls->toggleClosedCaptionTrackList();
    791 
    792         int trackIndex = trackListIndexForElement(toElement(target));
    793         if (trackIndex == HTMLMediaElement::textTracksIndexNotFound())
     786        if (!textTrack)
    794787            return;
    795788
    … …  
    798791            return;
    799792
    800         if (textTrack)
    801             mediaElement->setSelectedTextTrack(textTrack.get());
    802         else if (trackIndex == HTMLMediaElement::textTracksOffIndex())
    803             mediaElement->setSelectedTextTrack(0);
     793        mediaElement->setSelectedTextTrack(textTrack.get());
    804794
    805795        updateDisplay();
    … …  
    824814        return;
    825815
     816    if (!document()->page())
     817        return;
     818    CaptionUserPreferences::CaptionDisplayMode displayMode = document()->page()->group().captionPreferences()->captionDisplayMode();
     819    bool trackIsSelected = displayMode != CaptionUserPreferences::Automatic && displayMode != CaptionUserPreferences::ForcedOnly;
     820
    826821    HTMLMediaElement* mediaElement = toParentMediaElement(this);
    827822    if (!mediaElement)
    … …  
    829824
    830825    TextTrackList* trackList = mediaElement->textTracks();
    831 
    832826    if (!trackList || !trackList->length())
    833827        return;
    834828
    835     if (m_trackListHasChanged)
    836         rebuildTrackListMenu();
    837 
    838     bool captionsVisible = mediaElement->closedCaptionsVisible();
     829    rebuildTrackListMenu();
     830
    839831    for (unsigned i = 0, length = m_menuItems.size(); i < length; ++i) {
    840832        RefPtr<Element> trackItem = m_menuItems[i];
    841 
    842         int trackIndex = trackListIndexForElement(trackItem.get());
    843         if (trackIndex == HTMLMediaElement::textTracksIndexNotFound())
    844             continue;
    845 
    846         if (trackIndex == HTMLMediaElement::textTracksOffIndex()) {
    847             if (captionsVisible)
    848                 trackItem->classList()->remove(selectedClassValue, ASSERT_NO_EXCEPTION);
    849             else
    850                 trackItem->classList()->add(selectedClassValue, ASSERT_NO_EXCEPTION);
    851             continue;
    852         }
    853833
    854834        RefPtr<TextTrack> textTrack;
    … …  
    859839        if (!textTrack)
    860840            continue;
    861         if (textTrack->mode() == TextTrack::showingKeyword())
     841
     842        if (textTrack == TextTrack::captionMenuOffItem()) {
     843            if (displayMode == CaptionUserPreferences::ForcedOnly)
     844                trackItem->classList()->add(selectedClassValue, ASSERT_NO_EXCEPTION);
     845            else
     846                trackItem->classList()->remove(selectedClassValue, ASSERT_NO_EXCEPTION);
     847            continue;
     848        }
     849
     850        if (textTrack == TextTrack::captionMenuAutomaticItem()) {
     851            if (displayMode == CaptionUserPreferences::Automatic)
     852                trackItem->classList()->add(selectedClassValue, ASSERT_NO_EXCEPTION);
     853            else
     854                trackItem->classList()->remove(selectedClassValue, ASSERT_NO_EXCEPTION);
     855            continue;
     856        }
     857
     858        if (trackIsSelected && textTrack->mode() == TextTrack::showingKeyword())
    862859            trackItem->classList()->add(selectedClassValue, ASSERT_NO_EXCEPTION);
    863860        else
    … …  
    873870    removeChildren();
    874871    m_menuItems.clear();
    875 
    876     m_trackListHasChanged = false;
    877872    m_menuToTrackMap.clear();
    878873
    … …  
    889884
    890885    Document* doc = document();
    891     CaptionUserPreferences* captionsUserPreferences = doc->page()->group().captionPreferences();
    892     Vector<RefPtr<TextTrack> > tracksForMenu = captionsUserPreferences->sortedTrackListForMenu(trackList);
     886    if (!document()->page())
     887        return;
     888    CaptionUserPreferences* captionPreferences = document()->page()->group().captionPreferences();
     889    Vector<RefPtr<TextTrack> > tracksForMenu = captionPreferences->sortedTrackListForMenu(trackList);
    893890
    894891    RefPtr<Element> captionsHeader = doc->createElement(h3Tag, ASSERT_NO_EXCEPTION);
    … …  
    897894    RefPtr<Element> captionsMenuList = doc->createElement(ulTag, ASSERT_NO_EXCEPTION);
    898895
    899     RefPtr<Element> menuItem;
    900     menuItem = doc->createElement(liTag, ASSERT_NO_EXCEPTION);
    901     menuItem->appendChild(doc->createTextNode(textTrackOffText()));
    902     menuItem->setAttribute(trackIndexAttributeName(), textTracksOffAttrValue, ASSERT_NO_EXCEPTION);
    903     captionsMenuList->appendChild(menuItem);
    904     m_menuItems.append(menuItem);
    905 
    906896    for (unsigned i = 0, length = tracksForMenu.size(); i < length; ++i) {
    907897        RefPtr<TextTrack> textTrack = tracksForMenu[i];
    908         menuItem = doc->createElement(liTag, ASSERT_NO_EXCEPTION);
    909 
    910         // Add a custom attribute to the <li> element which will allow
    911         // us to easily associate the user tapping here with the
    912         // track. Since this list is rebuilt if the tracks change, we
    913         // should always be in sync.
    914         menuItem->setAttribute(trackIndexAttributeName(), String::number(i), ASSERT_NO_EXCEPTION);
    915 
    916         menuItem->appendChild(doc->createTextNode(captionsUserPreferences->displayNameForTrack(textTrack.get())));
    917 
     898        RefPtr<Element> menuItem = doc->createElement(liTag, ASSERT_NO_EXCEPTION);
     899        menuItem->appendChild(doc->createTextNode(captionPreferences->displayNameForTrack(textTrack.get())));
    918900        captionsMenuList->appendChild(menuItem);
    919901        m_menuItems.append(menuItem);
    … …  
    922904
    923905    appendChild(captionsMenuList);
    924 
    925     updateDisplay();
    926906#endif
    927907}

  • trunk/Source/WebCore/html/shadow/MediaControlElements.h

    r146992 r147675  
    308308
    309309    void updateDisplay();
    310     void resetTrackListMenu() { m_trackListHasChanged = true; }
    311310
    312311private:
    … …  
    325324#endif
    326325    MediaControls* m_controls;
    327     bool m_trackListHasChanged;
    328326};
    329327

  • trunk/Source/WebCore/html/shadow/MediaControlsApple.cpp

    r146647 r147675  
    306306{
    307307    MediaControls::changedClosedCaptionsVisibility();
    308     if (m_closedCaptionsTrackList)
    309         m_closedCaptionsTrackList->resetTrackListMenu();
     308    if (m_closedCaptionsContainer && m_closedCaptionsContainer->isShowing())
     309        m_closedCaptionsContainer->hide();
     310
    310311}
    311312
    … …  
    341342
    342343    if (m_toggleClosedCaptionsButton) {
    343         if (m_mediaController->hasClosedCaptions()) {
     344        if (m_mediaController->hasClosedCaptions())
    344345            m_toggleClosedCaptionsButton->show();
    345             if (m_closedCaptionsTrackList)
    346                 m_closedCaptionsTrackList->resetTrackListMenu();
    347         } else
     346        else
    348347            m_toggleClosedCaptionsButton->hide();
    349348    }
    … …  
    511510void MediaControlsApple::closedCaptionTracksChanged()
    512511{
    513     if (m_closedCaptionsTrackList)
    514         m_closedCaptionsTrackList->resetTrackListMenu();
    515512    if (m_toggleClosedCaptionsButton) {
    516513        if (m_mediaController->hasClosedCaptions())

  • trunk/Source/WebCore/html/track/TextTrack.cpp

    r147325 r147675  
    9696}
    9797
     98TextTrack* TextTrack::captionMenuOffItem()
     99{
     100    DEFINE_STATIC_LOCAL(RefPtr<TextTrack>, off, (TextTrack::create(0, 0, "off menu item", "", "")));
     101    return off.get();
     102}
     103
     104TextTrack* TextTrack::captionMenuAutomaticItem()
     105{
     106    DEFINE_STATIC_LOCAL(RefPtr<TextTrack>, automatic, (TextTrack::create(0, 0, "automatic menu item", "", "")));
     107    return automatic.get();
     108}
     109
    98110TextTrack::TextTrack(ScriptExecutionContext* context, TextTrackClient* client, const AtomicString& kind, const AtomicString& label, const AtomicString& language, TextTrackType type)
    99111    : TrackBase(context, TrackBase::TextTrack)

  • trunk/Source/WebCore/html/track/TextTrack.h

    r147325 r147675  
    7373    }
    7474    virtual ~TextTrack();
     75
     76    static TextTrack* captionMenuOffItem();
     77    static TextTrack* captionMenuAutomaticItem();
    7578
    7679    void setMediaElement(HTMLMediaElement* element) { m_mediaElement = element; }

  • trunk/Source/WebCore/page/CaptionUserPreferences.cpp

    r147001 r147675  
    3939CaptionUserPreferences::CaptionUserPreferences(PageGroup* group)
    4040    : m_pageGroup(group)
     41    , m_displayMode(AlwaysOn)
    4142    , m_timer(this, &CaptionUserPreferences::timerFired)
    4243    , m_testingMode(false)
    4344    , m_havePreferences(false)
    44     , m_shouldShowCaptions(false)
    4545{
    4646}
    … …  
    4848CaptionUserPreferences::~CaptionUserPreferences()
    4949{
    50 }
    51 
    52 bool CaptionUserPreferences::shouldShowCaptions() const
    53 {
    54     if (!m_testingMode)
    55         return false;
    56    
    57     return m_shouldShowCaptions || userPrefersCaptions() || userPrefersSubtitles();
    5850}
    5951
    … …  
    7365}
    7466
    75 void CaptionUserPreferences::setShouldShowCaptions(bool preference)
    76 {
    77     m_shouldShowCaptions = preference;
    78     if (m_testingMode && !preference) {
     67CaptionUserPreferences::CaptionDisplayMode CaptionUserPreferences::captionDisplayMode() const
     68{
     69    return m_displayMode;
     70}
     71
     72void CaptionUserPreferences::setCaptionDisplayMode(CaptionUserPreferences::CaptionDisplayMode mode)
     73{
     74    m_displayMode = mode;
     75    if (m_testingMode && mode != AlwaysOn) {
    7976        setUserPrefersCaptions(false);
    8077        setUserPrefersSubtitles(false);
    … …  
    154151}
    155152
    156 void CaptionUserPreferences::setPreferredLanguage(String language)
     153void CaptionUserPreferences::setPreferredLanguage(const String& language)
    157154{
    158155    m_userPreferredLanguage = language;
    … …  
    199196    if (track->kind() != TextTrack::captionsKeyword() && track->kind() != TextTrack::subtitlesKeyword())
    200197        return trackScore;
    201 
     198   
     199    if (!userPrefersSubtitles() && !userPrefersCaptions())
     200        return trackScore;
     201   
    202202    if (track->kind() == TextTrack::subtitlesKeyword() && userPrefersSubtitles())
    203203        trackScore = 1;
    … …  
    205205        trackScore = 1;
    206206   
    207     return trackScore + textTrackLanguageSelectionScore(track);
    208 }
    209 
    210 int CaptionUserPreferences::textTrackLanguageSelectionScore(TextTrack* track) const
     207    return trackScore + textTrackLanguageSelectionScore(track, preferredLanguages());
     208}
     209
     210int CaptionUserPreferences::textTrackLanguageSelectionScore(TextTrack* track, const Vector<String>& preferredLanguages) const
    211211{
    212212    if (track->language().isEmpty())
    213213        return 0;
    214214
    215     Vector<String> languages = preferredLanguages();
    216     size_t languageMatchIndex = indexOfBestMatchingLanguageInList(track->language(), languages);
    217     if (languageMatchIndex >= languages.size())
     215    size_t languageMatchIndex = indexOfBestMatchingLanguageInList(track->language(), preferredLanguages);
     216    if (languageMatchIndex >= preferredLanguages.size())
    218217        return 0;
    219218
    220219    // Matching a track language is more important than matching track type, so this multiplier must be
    221220    // greater than the maximum value returned by textTrackSelectionScore.
    222     return (languages.size() - languageMatchIndex) * 10;
     221    return (preferredLanguages.size() - languageMatchIndex) * 10;
    223222}
    224223

  • trunk/Source/WebCore/page/CaptionUserPreferences.h

    r147001 r147675  
    4747    virtual ~CaptionUserPreferences();
    4848
    49     virtual bool userHasCaptionPreferences() const { return m_testingMode && m_havePreferences; }
    50     virtual bool shouldShowCaptions() const;
    51     virtual void setShouldShowCaptions(bool);
     49    enum CaptionDisplayMode {
     50        Automatic,
     51        ForcedOnly,
     52        AlwaysOn
     53    };
     54    virtual CaptionDisplayMode captionDisplayMode() const;
     55    virtual void setCaptionDisplayMode(CaptionDisplayMode);
    5256
    5357    virtual int textTrackSelectionScore(TextTrack*, HTMLMediaElement*) const;
    54     virtual int textTrackLanguageSelectionScore(TextTrack*) const;
     58    virtual int textTrackLanguageSelectionScore(TextTrack*, const Vector<String>&) const;
    5559
    5660    virtual bool userPrefersCaptions() const;
    … …  
    7074    virtual void captionPreferencesChanged();
    7175
    72     virtual void setPreferredLanguage(String);
     76    virtual void setPreferredLanguage(const String&);
    7377    virtual Vector<String> preferredLanguages() const;
    7478
    … …  
    7882    virtual bool testingMode() const { return m_testingMode; }
    7983    virtual void setTestingMode(bool override) { m_testingMode = override; }
    80 
     84   
    8185    PageGroup* pageGroup() const { return m_pageGroup; }
    8286
    … …  
    8993
    9094    PageGroup* m_pageGroup;
     95    CaptionDisplayMode m_displayMode;
    9196    Timer<CaptionUserPreferences> m_timer;
    9297    String m_userPreferredLanguage;
    9398    bool m_testingMode;
    9499    bool m_havePreferences;
    95     bool m_shouldShowCaptions;
    96100};
    97101   

  • trunk/Source/WebCore/page/CaptionUserPreferencesMac.h

    r147001 r147675  
    4242
    4343#if HAVE(MEDIA_ACCESSIBILITY_FRAMEWORK)
    44     virtual bool userHasCaptionPreferences() const OVERRIDE;
    45     virtual bool shouldShowCaptions() const OVERRIDE;
    46     virtual void setShouldShowCaptions(bool) OVERRIDE;
     44    virtual CaptionDisplayMode captionDisplayMode() const OVERRIDE;
     45    virtual void setCaptionDisplayMode(CaptionDisplayMode) OVERRIDE;
    4746
    4847    virtual bool userPrefersCaptions() const OVERRIDE;
    … …  
    5453    virtual void setInterestedInCaptionPreferenceChanges() OVERRIDE;
    5554
    56     virtual void setPreferredLanguage(String) OVERRIDE;
     55    virtual void setPreferredLanguage(const String&) OVERRIDE;
    5756    virtual Vector<String> preferredLanguages() const OVERRIDE;
    5857

  • trunk/Source/WebCore/page/CaptionUserPreferencesMac.mm

    r147001 r147675  
    6161SOFT_LINK_FRAMEWORK_OPTIONAL(MediaAccessibility)
    6262
    63 SOFT_LINK(MediaAccessibility, MACaptionAppearanceGetShowCaptions, bool, (MACaptionAppearanceDomain domain), (domain))
    64 SOFT_LINK(MediaAccessibility, MACaptionAppearanceSetShowCaptions, void, (MACaptionAppearanceDomain domain, bool showCaptions), (domain, showCaptions))
     63SOFT_LINK(MediaAccessibility, MACaptionAppearanceGetDisplayType, MACaptionAppearanceDisplayType, (MACaptionAppearanceDomain domain), (domain))
     64SOFT_LINK(MediaAccessibility, MACaptionAppearanceSetDisplayType, void, (MACaptionAppearanceDomain domain, MACaptionAppearanceDisplayType displayType), (domain, displayType))
    6565SOFT_LINK(MediaAccessibility, MACaptionAppearanceCopyForegroundColor, CGColorRef, (MACaptionAppearanceDomain domain, MACaptionAppearanceBehavior *behavior), (domain, behavior))
    6666SOFT_LINK(MediaAccessibility, MACaptionAppearanceCopyBackgroundColor, CGColorRef, (MACaptionAppearanceDomain domain, MACaptionAppearanceBehavior *behavior), (domain, behavior))
    … …  
    117117#if HAVE(MEDIA_ACCESSIBILITY_FRAMEWORK)
    118118
    119 bool CaptionUserPreferencesMac::shouldShowCaptions() const
     119CaptionUserPreferences::CaptionDisplayMode CaptionUserPreferencesMac::captionDisplayMode() const
    120120{
    121121    if (testingMode() || !MediaAccessibilityLibrary())
    122         return CaptionUserPreferences::shouldShowCaptions();
    123 
    124     return MACaptionAppearanceGetShowCaptions(kMACaptionAppearanceDomainUser);
    125 }
    126 
    127 void CaptionUserPreferencesMac::setShouldShowCaptions(bool preference)
     122        return CaptionUserPreferences::captionDisplayMode();
     123
     124    MACaptionAppearanceDisplayType displayType = MACaptionAppearanceGetDisplayType(kMACaptionAppearanceDomainUser);
     125    switch (displayType) {
     126    case kMACaptionAppearanceDisplayTypeForcedOnly:
     127        return ForcedOnly;
     128        break;
     129
     130    case kMACaptionAppearanceDisplayTypeAutomatic:
     131        return Automatic;
     132        break;
     133
     134    case kMACaptionAppearanceDisplayTypeAlwaysOn:
     135        return AlwaysOn;
     136        break;
     137    }
     138
     139    ASSERT_NOT_REACHED();
     140    return ForcedOnly;
     141}
     142   
     143void CaptionUserPreferencesMac::setCaptionDisplayMode(CaptionUserPreferences::CaptionDisplayMode mode)
    128144{
    129145    if (testingMode() || !MediaAccessibilityLibrary()) {
    130         CaptionUserPreferences::setShouldShowCaptions(preference);
     146        CaptionUserPreferences::setCaptionDisplayMode(mode);
    131147        return;
    132148    }
    133149
    134     MACaptionAppearanceSetShowCaptions(kMACaptionAppearanceDomainUser, preference);
     150    MACaptionAppearanceDisplayType displayType = kMACaptionAppearanceDisplayTypeForcedOnly;
     151    switch (mode) {
     152        case Automatic:
     153            displayType = kMACaptionAppearanceDisplayTypeAutomatic;
     154            break;
     155        case ForcedOnly:
     156            displayType = kMACaptionAppearanceDisplayTypeForcedOnly;
     157            break;
     158        case AlwaysOn:
     159            displayType = kMACaptionAppearanceDisplayTypeAlwaysOn;
     160            break;
     161        default:
     162            ASSERT_NOT_REACHED();
     163            break;
     164    }
     165
     166    MACaptionAppearanceSetDisplayType(kMACaptionAppearanceDomainUser, displayType);
    135167}
    136168
    … …  
    153185    RetainPtr<CFArrayRef> captioningMediaCharacteristics(AdoptCF, MACaptionAppearanceCopyPreferredCaptioningMediaCharacteristics(kMACaptionAppearanceDomainUser));
    154186    return !(captioningMediaCharacteristics && CFArrayGetCount(captioningMediaCharacteristics.get()));
    155 }
    156 
    157 bool CaptionUserPreferencesMac::userHasCaptionPreferences() const
    158 {
    159     if (testingMode() || !MediaAccessibilityLibrary())
    160         return CaptionUserPreferences::userHasCaptionPreferences();
    161 
    162     return true;
    163187}
    164188
    … …  
    482506}
    483507
    484 void CaptionUserPreferencesMac::setPreferredLanguage(String language)
     508void CaptionUserPreferencesMac::setPreferredLanguage(const String& language)
    485509{
    486510    if (testingMode() || !MediaAccessibilityLibrary()) {
    … …  
    526550}
    527551#endif  // HAVE(MEDIA_ACCESSIBILITY_FRAMEWORK)
    528    
     552
     553static String languageIdentifier(const String& languageCode)
     554{
     555    if (languageCode.isEmpty())
     556        return languageCode;
     557
     558    String lowercaseLanguageCode = languageCode.lower();
     559
     560    // Need 2U here to disambiguate String::operator[] from operator(NSString*, int)[] in a production build.
     561    if (lowercaseLanguageCode.length() >= 3 && (lowercaseLanguageCode[2U] == '_' || lowercaseLanguageCode[2U] == '-'))
     562        lowercaseLanguageCode.truncate(2);
     563
     564    return lowercaseLanguageCode;
     565}
     566
    529567static String trackDisplayName(TextTrack* track)
    530568{
     569    if (track == TextTrack::captionMenuOffItem())
     570        return textTrackOffMenuItemText();
     571    if (track == TextTrack::captionMenuAutomaticItem()) {
     572        String preferredLanguageDisplayName = displayNameForLanguageLocale(languageIdentifier(defaultLanguage()));
     573        return textTrackAutomaticMenuItemText(preferredLanguageDisplayName);
     574    }
     575
    531576    StringBuilder displayName;
    532577    String label = track->label();
    … …  
    588633}
    589634
    590 static String languageIdentifier(const String& languageCode)
    591 {
    592     if (languageCode.isEmpty())
    593         return languageCode;
    594 
    595     String lowercaseLanguageCode = languageCode.lower();
    596 
    597     // Need 2U here to disambiguate String::operator[] from operator(NSString*, int)[] in a production build.
    598     if (lowercaseLanguageCode.length() >= 3 && (lowercaseLanguageCode[2U] == '_' || lowercaseLanguageCode[2U] == '-'))
    599         lowercaseLanguageCode.truncate(2);
    600    
    601     return lowercaseLanguageCode;
    602 }
    603 
    604635int CaptionUserPreferencesMac::textTrackSelectionScore(TextTrack* track, HTMLMediaElement* mediaElement) const
    605636{
    606     if (!shouldShowCaptions() && !mediaElement->webkitClosedCaptionsVisible())
     637    CaptionDisplayMode displayMode = captionDisplayMode();
     638    if (displayMode == ForcedOnly)
     639        return 0;
     640    if (displayMode == AlwaysOn && (!userPrefersSubtitles() && !userPrefersCaptions()))
    607641        return 0;
    608642    if (track->kind() != TextTrack::captionsKeyword() && track->kind() != TextTrack::subtitlesKeyword())
    … …  
    612646    if (!track->isMainProgramContent())
    613647        return 0;
     648
     649    Vector<String> userPreferredCaptionLanguages = preferredLanguages();
     650
     651    if (displayMode == Automatic) {
     652
     653        // Only enable a text track if the current audio track is not in the user's preferred language.
     654        if (!mediaElement || !mediaElement->player())
     655            return 0;
     656
     657        String audioTrackLanguage = mediaElement->player()->languageOfPrimaryAudioTrack();
     658
     659        if (audioTrackLanguage.isEmpty())
     660            return 0;
     661
     662        Vector<String> languages;
     663        languages.append(defaultLanguage());
     664        size_t offset = indexOfBestMatchingLanguageInList(audioTrackLanguage, languages);
     665        if (!offset)
     666            return 0;
     667
     668        userPreferredCaptionLanguages = languages;
     669    }
    614670
    615671    int trackScore = 0;
    … …  
    633689    }
    634690
    635     return trackScore + textTrackLanguageSelectionScore(track);
     691    return trackScore + textTrackLanguageSelectionScore(track, userPreferredCaptionLanguages);
    636692}
    637693
    … …  
    739795    nonCopyingSort(tracksForMenu.begin(), tracksForMenu.end(), textTrackCompare);
    740796
     797    tracksForMenu.insert(0, TextTrack::captionMenuOffItem());
     798    tracksForMenu.insert(1, TextTrack::captionMenuAutomaticItem());
     799
    741800    return tracksForMenu;
    742801}

  • trunk/Source/WebCore/platform/LocalizedStrings.cpp

    r147520 r147675  
    10261026}
    10271027
    1028 String textTrackOffText()
     1028String textTrackOffMenuItemText()
    10291029{
    10301030    return WEB_UI_STRING("Off", "Menu item label for the track that represents disabling closed captions");
     1031}
     1032
     1033String textTrackAutomaticMenuItemText(const String& language)
     1034{
     1035    return formatLocalizedString(WEB_UI_STRING("Automatic (%@)", "Menu item label for automatic track selection behavior in the form of 'Automatic (SystemLanguage)'"), language.createCFString().get());
    10311036}
    10321037

  • trunk/Source/WebCore/platform/LocalizedStrings.h

    r147520 r147675  
    268268#if ENABLE(VIDEO_TRACK)
    269269    String textTrackSubtitlesText();
    270     String textTrackOffText();
     270    String textTrackOffMenuItemText();
     271    String textTrackAutomaticMenuItemText(const String& language);
    271272    String textTrackNoLabelText();
    272273#if PLATFORM(MAC)

  • trunk/Source/WebCore/platform/graphics/MediaPlayer.cpp

    r145811 r147675  
    11551155#endif
    11561156
    1157 }
    1158 
    1159 #endif
     1157String MediaPlayer::languageOfPrimaryAudioTrack() const
     1158{
     1159    if (!m_private)
     1160        return emptyString();
     1161   
     1162    return m_private->languageOfPrimaryAudioTrack();
     1163}
     1164}
     1165
     1166#endif

  • trunk/Source/WebCore/platform/graphics/MediaPlayer.h

    r147628 r147675  
    467467#endif
    468468
     469    String languageOfPrimaryAudioTrack() const;
     470
    469471private:
    470472    MediaPlayer(MediaPlayerClient*);

  • trunk/Source/WebCore/platform/graphics/MediaPlayerPrivate.h

    r145811 r147675  
    192192    virtual void simulateAudioInterruption() { }
    193193#endif
     194   
     195    virtual String languageOfPrimaryAudioTrack() const { return emptyString(); }
    194196};
    195197

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h

    r144565 r147675  
    160160#endif
    161161
     162    virtual String languageOfPrimaryAudioTrack() const OVERRIDE;
     163
    162164#if HAVE(AVFOUNDATION_TEXT_TRACK_SUPPORT)
    163165    virtual void setCurrentTrack(InbandTextTrackPrivateAVF*) OVERRIDE;
    … …  
    173175    RetainPtr<WebCoreAVFMovieObserver> m_objcObserver;
    174176    RetainPtr<id> m_timeObserver;
     177    mutable String m_languageOfPrimaryAudioTrack;
    175178    bool m_videoFrameHasDrawn;
    176179    bool m_haveCheckedPlayability;

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r146563 r147675  
    947947void MediaPlayerPrivateAVFoundationObjC::tracksChanged()
    948948{
     949    m_languageOfPrimaryAudioTrack = String();
     950
    949951    if (!m_avAsset)
    950952        return;
    … …  
    13781380#endif // HAVE(AVFOUNDATION_TEXT_TRACK_SUPPORT)
    13791381
     1382String MediaPlayerPrivateAVFoundationObjC::languageOfPrimaryAudioTrack() const
     1383{
     1384    if (!m_languageOfPrimaryAudioTrack.isNull())
     1385        return m_languageOfPrimaryAudioTrack;
     1386
     1387    if (!m_avPlayerItem.get())
     1388        return emptyString();
     1389
     1390#if HAVE(AVFOUNDATION_TEXT_TRACK_SUPPORT)
     1391    // If AVFoundation has an audible group, return the language of the currently selected audible option.
     1392    AVMediaSelectionGroupType *audibleGroup = [m_avAsset.get() mediaSelectionGroupForMediaCharacteristic:AVMediaCharacteristicAudible];
     1393    AVMediaSelectionOptionType *currentlySelectedAudibleOption = [m_avPlayerItem.get() selectedMediaOptionInMediaSelectionGroup:audibleGroup];
     1394    if (currentlySelectedAudibleOption) {
     1395        m_languageOfPrimaryAudioTrack = [[currentlySelectedAudibleOption locale] localeIdentifier];
     1396        return m_languageOfPrimaryAudioTrack;
     1397    }
     1398#endif // HAVE(AVFOUNDATION_TEXT_TRACK_SUPPORT)
     1399
     1400    // AVFoundation synthesizes an audible group when there is only one ungrouped audio track if there is also a legible group (one or
     1401    // more in-band text tracks). It doesn't know about out-of-band tracks, so if there is a single audio track return its language.
     1402    NSArray *tracks = [m_avAsset.get() tracksWithMediaType:AVMediaTypeAudio];
     1403    if (!tracks || [tracks count] != 1) {
     1404        m_languageOfPrimaryAudioTrack = emptyString();
     1405        return m_languageOfPrimaryAudioTrack;
     1406    }
     1407
     1408    AVAssetTrack *track = [tracks objectAtIndex:0];
     1409    NSString *language = [track extendedLanguageTag];
     1410
     1411    // Some legacy tracks have "und" as a language, treat that the same as no language at all.
     1412    if (language && ![language isEqualToString:@"und"]) {
     1413        m_languageOfPrimaryAudioTrack = language;
     1414        return m_languageOfPrimaryAudioTrack;
     1415    }
     1416
     1417    m_languageOfPrimaryAudioTrack = emptyString();
     1418    return m_languageOfPrimaryAudioTrack;
     1419}
     1420   
    13801421NSArray* assetMetadataKeyNames()
    13811422{
Note: See TracChangeset for help on using the changeset viewer.