Changeset 254615 in webkit


Ignore:
Timestamp:
Jan 15, 2020 11:16:08 AM (4 years ago)
Author:
Alan Coon
Message:

Cherry-pick r254389. rdar://problem/58548648

REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
​https://bugs.webkit.org/show_bug.cgi?id=200043
Source/WTF:

<rdar://problem/44119496>

Reviewed by Dean Jackson.

We ask the system for the current locale using CFLocaleCopyPreferredLanguages(), and then round-trip
it through CFBundleGetLocalizationInfoForLocalization() / CFBundleCopyLocalizationForLocalizationInfo().
This was to work around the fact that CFLocaleCopyPreferredLanguages() previously didn't report BCP47
language codes. However, that round-tripping was introducing errors, such as "zh-Hant-HK" was getting
turned into "zh-Hant-TW" which is clearly wrong. The CFBundle functions were never supposed to be used
in this way.

Instead, we can use CFLocaleCreateCanonicalLanguageIdentifierFromString() which is intended to
canonicalize locale identifiers, and does return BCP47 language codes. However, this function preserves
more fingerprinting entropy than the old code path, so we pass the input through new NSLocale SPI to
minimize the entropy revealed.

  • WTF.xcodeproj/project.pbxproj:
  • wtf/Language.h:
  • wtf/Platform.h:
  • wtf/PlatformMac.cmake:
  • wtf/cf/LanguageCF.cpp: (WTF::httpStyleLanguageCode): (WTF::platformUserPreferredLanguages):
  • wtf/cocoa/LanguageCocoa.mm: Added. (WTF::minimizedLanguagesFromLanguages):
  • wtf/spi/cocoa/NSLocaleSPI.h: Added.

Tools:

Reviewed by Dean Jackson.

Migrate system language tests to LayoutTests, to match the rest of our system language tests.

  • TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
  • TestWebKitAPI/Tests/mac/NavigatorLanguage.mm: Removed.

LayoutTests:

<rdar://problem/44119496>

Reviewed by Dean Jackson.

Migrate and update tests from TestWebKitAPI to LayoutTests, to match the rest of our system language tests.

  • platform/mac/TestExpectations: Mark these tests as possibly failing on older versions of macOS.
  • fast/text/international/system-language/navigator-language/navigator-language-en-GB-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-en-GB.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-en-US-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-en-US.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-en-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-en.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es-419-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es-419.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es-ES-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es-ES.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es-MX-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es-MX.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-es.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-fr-CA-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-fr-CA.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-fr-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-fr.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-hi-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-hi.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-ja-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-ja.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-pt-BR-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-pt-BR.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-pt-PT-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-pt-PT.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-ru-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-ru.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-HK-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-HK.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-Hans-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-Hans.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-Hant.html: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-TW-expected.txt: Added.
  • fast/text/international/system-language/navigator-language/navigator-language-zh-TW.html: Added.
  • js/dom/navigator-language-expected.txt:

git-svn-id: ​https://svn.webkit.org/repository/webkit/trunk@254389 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Location:
branches/safari-609-branch
Files:
41 added
1 deleted
11 edited

Legend:

Unmodified
Added
Removed

  • branches/safari-609-branch/LayoutTests/ChangeLog

    r254612 r254615  
     12020-01-15  Alan Coon  <alancoon@apple.com>
     2
     3        Cherry-pick r254389. rdar://problem/58548648
     4
     5    REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
     6    https://bugs.webkit.org/show_bug.cgi?id=200043
     7    Source/WTF:
     8   
     9    <rdar://problem/44119496>
     10   
     11    Reviewed by Dean Jackson.
     12   
     13    We ask the system for the current locale using CFLocaleCopyPreferredLanguages(), and then round-trip
     14    it through CFBundleGetLocalizationInfoForLocalization() / CFBundleCopyLocalizationForLocalizationInfo().
     15    This was to work around the fact that CFLocaleCopyPreferredLanguages() previously didn't report BCP47
     16    language codes. However, that round-tripping was introducing errors, such as "zh-Hant-HK" was getting
     17    turned into "zh-Hant-TW" which is clearly wrong. The CFBundle functions were never supposed to be used
     18    in this way.
     19   
     20    Instead, we can use CFLocaleCreateCanonicalLanguageIdentifierFromString() which is intended to
     21    canonicalize locale identifiers, and does return BCP47 language codes. However, this function preserves
     22    more fingerprinting entropy than the old code path, so we pass the input through new NSLocale SPI to
     23    minimize the entropy revealed.
     24   
     25    * WTF.xcodeproj/project.pbxproj:
     26    * wtf/Language.h:
     27    * wtf/Platform.h:
     28    * wtf/PlatformMac.cmake:
     29    * wtf/cf/LanguageCF.cpp:
     30    (WTF::httpStyleLanguageCode):
     31    (WTF::platformUserPreferredLanguages):
     32    * wtf/cocoa/LanguageCocoa.mm: Added.
     33    (WTF::minimizedLanguagesFromLanguages):
     34    * wtf/spi/cocoa/NSLocaleSPI.h: Added.
     35   
     36    Tools:
     37   
     38    Reviewed by Dean Jackson.
     39   
     40    Migrate system language tests to LayoutTests, to match the rest of our system language tests.
     41   
     42    * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
     43    * TestWebKitAPI/Tests/mac/NavigatorLanguage.mm: Removed.
     44   
     45    LayoutTests:
     46   
     47    <rdar://problem/44119496>
     48   
     49    Reviewed by Dean Jackson.
     50   
     51    Migrate and update tests from TestWebKitAPI to LayoutTests, to match the rest of our system language tests.
     52   
     53    * platform/mac/TestExpectations: Mark these tests as possibly failing on older versions of macOS.
     54    * fast/text/international/system-language/navigator-language/navigator-language-en-GB-expected.txt: Added.
     55    * fast/text/international/system-language/navigator-language/navigator-language-en-GB.html: Added.
     56    * fast/text/international/system-language/navigator-language/navigator-language-en-US-expected.txt: Added.
     57    * fast/text/international/system-language/navigator-language/navigator-language-en-US.html: Added.
     58    * fast/text/international/system-language/navigator-language/navigator-language-en-expected.txt: Added.
     59    * fast/text/international/system-language/navigator-language/navigator-language-en.html: Added.
     60    * fast/text/international/system-language/navigator-language/navigator-language-es-419-expected.txt: Added.
     61    * fast/text/international/system-language/navigator-language/navigator-language-es-419.html: Added.
     62    * fast/text/international/system-language/navigator-language/navigator-language-es-ES-expected.txt: Added.
     63    * fast/text/international/system-language/navigator-language/navigator-language-es-ES.html: Added.
     64    * fast/text/international/system-language/navigator-language/navigator-language-es-MX-expected.txt: Added.
     65    * fast/text/international/system-language/navigator-language/navigator-language-es-MX.html: Added.
     66    * fast/text/international/system-language/navigator-language/navigator-language-es-expected.txt: Added.
     67    * fast/text/international/system-language/navigator-language/navigator-language-es.html: Added.
     68    * fast/text/international/system-language/navigator-language/navigator-language-fr-CA-expected.txt: Added.
     69    * fast/text/international/system-language/navigator-language/navigator-language-fr-CA.html: Added.
     70    * fast/text/international/system-language/navigator-language/navigator-language-fr-expected.txt: Added.
     71    * fast/text/international/system-language/navigator-language/navigator-language-fr.html: Added.
     72    * fast/text/international/system-language/navigator-language/navigator-language-hi-expected.txt: Added.
     73    * fast/text/international/system-language/navigator-language/navigator-language-hi.html: Added.
     74    * fast/text/international/system-language/navigator-language/navigator-language-ja-expected.txt: Added.
     75    * fast/text/international/system-language/navigator-language/navigator-language-ja.html: Added.
     76    * fast/text/international/system-language/navigator-language/navigator-language-pt-BR-expected.txt: Added.
     77    * fast/text/international/system-language/navigator-language/navigator-language-pt-BR.html: Added.
     78    * fast/text/international/system-language/navigator-language/navigator-language-pt-PT-expected.txt: Added.
     79    * fast/text/international/system-language/navigator-language/navigator-language-pt-PT.html: Added.
     80    * fast/text/international/system-language/navigator-language/navigator-language-ru-expected.txt: Added.
     81    * fast/text/international/system-language/navigator-language/navigator-language-ru.html: Added.
     82    * fast/text/international/system-language/navigator-language/navigator-language-zh-HK-expected.txt: Added.
     83    * fast/text/international/system-language/navigator-language/navigator-language-zh-HK.html: Added.
     84    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans-expected.txt: Added.
     85    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans.html: Added.
     86    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK-expected.txt: Added.
     87    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK.html: Added.
     88    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-expected.txt: Added.
     89    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant.html: Added.
     90    * fast/text/international/system-language/navigator-language/navigator-language-zh-TW-expected.txt: Added.
     91    * fast/text/international/system-language/navigator-language/navigator-language-zh-TW.html: Added.
     92    * js/dom/navigator-language-expected.txt:
     93   
     94    git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254389 268f45cc-cd09-0410-ab3c-d52691b4dbfc
     95
     96    2020-01-10  Myles C. Maxfield  <mmaxfield@apple.com>
     97
     98            REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
     99            https://bugs.webkit.org/show_bug.cgi?id=200043
     100            <rdar://problem/44119496>
     101
     102            Reviewed by Dean Jackson.
     103
     104            Migrate and update tests from TestWebKitAPI to LayoutTests, to match the rest of our system language tests.
     105
     106            * platform/mac/TestExpectations: Mark these tests as possibly failing on older versions of macOS.
     107            * fast/text/international/system-language/navigator-language/navigator-language-en-GB-expected.txt: Added.
     108            * fast/text/international/system-language/navigator-language/navigator-language-en-GB.html: Added.
     109            * fast/text/international/system-language/navigator-language/navigator-language-en-US-expected.txt: Added.
     110            * fast/text/international/system-language/navigator-language/navigator-language-en-US.html: Added.
     111            * fast/text/international/system-language/navigator-language/navigator-language-en-expected.txt: Added.
     112            * fast/text/international/system-language/navigator-language/navigator-language-en.html: Added.
     113            * fast/text/international/system-language/navigator-language/navigator-language-es-419-expected.txt: Added.
     114            * fast/text/international/system-language/navigator-language/navigator-language-es-419.html: Added.
     115            * fast/text/international/system-language/navigator-language/navigator-language-es-ES-expected.txt: Added.
     116            * fast/text/international/system-language/navigator-language/navigator-language-es-ES.html: Added.
     117            * fast/text/international/system-language/navigator-language/navigator-language-es-MX-expected.txt: Added.
     118            * fast/text/international/system-language/navigator-language/navigator-language-es-MX.html: Added.
     119            * fast/text/international/system-language/navigator-language/navigator-language-es-expected.txt: Added.
     120            * fast/text/international/system-language/navigator-language/navigator-language-es.html: Added.
     121            * fast/text/international/system-language/navigator-language/navigator-language-fr-CA-expected.txt: Added.
     122            * fast/text/international/system-language/navigator-language/navigator-language-fr-CA.html: Added.
     123            * fast/text/international/system-language/navigator-language/navigator-language-fr-expected.txt: Added.
     124            * fast/text/international/system-language/navigator-language/navigator-language-fr.html: Added.
     125            * fast/text/international/system-language/navigator-language/navigator-language-hi-expected.txt: Added.
     126            * fast/text/international/system-language/navigator-language/navigator-language-hi.html: Added.
     127            * fast/text/international/system-language/navigator-language/navigator-language-ja-expected.txt: Added.
     128            * fast/text/international/system-language/navigator-language/navigator-language-ja.html: Added.
     129            * fast/text/international/system-language/navigator-language/navigator-language-pt-BR-expected.txt: Added.
     130            * fast/text/international/system-language/navigator-language/navigator-language-pt-BR.html: Added.
     131            * fast/text/international/system-language/navigator-language/navigator-language-pt-PT-expected.txt: Added.
     132            * fast/text/international/system-language/navigator-language/navigator-language-pt-PT.html: Added.
     133            * fast/text/international/system-language/navigator-language/navigator-language-ru-expected.txt: Added.
     134            * fast/text/international/system-language/navigator-language/navigator-language-ru.html: Added.
     135            * fast/text/international/system-language/navigator-language/navigator-language-zh-HK-expected.txt: Added.
     136            * fast/text/international/system-language/navigator-language/navigator-language-zh-HK.html: Added.
     137            * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans-expected.txt: Added.
     138            * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans.html: Added.
     139            * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK-expected.txt: Added.
     140            * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK.html: Added.
     141            * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-expected.txt: Added.
     142            * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant.html: Added.
     143            * fast/text/international/system-language/navigator-language/navigator-language-zh-TW-expected.txt: Added.
     144            * fast/text/international/system-language/navigator-language/navigator-language-zh-TW.html: Added.
     145            * js/dom/navigator-language-expected.txt:
     146
    11472020-01-14  Alan Coon  <alancoon@apple.com>
    2148

  • branches/safari-609-branch/LayoutTests/js/dom/navigator-language-expected.txt

    r206949 r254615  
    33
    44
    5 PASS language is 'en-US'
     5PASS
    66PASS successfullyParsed is true
    77

  • branches/safari-609-branch/LayoutTests/js/dom/navigator-language.html

    r206949 r254615  
    1616
    1717var language = navigator.language;
    18 if (language.length == 2)
    19     shouldBe("language", "'en'");
     18if (language == "en" || language == "en-us" || language == "en-US")
     19    testPassed("");
    2020else
    21     shouldBe("language", "'en-US'");
     21    testFailed("");
    2222</script>
    2323<script src="../../resources/js-test-post.js"></script>

  • branches/safari-609-branch/LayoutTests/platform/mac/TestExpectations

    r254591 r254615  
    19681968webkit.org/b/205757 webgl/1.0.3/conformance/textures/texture-upload-size.html [ Pass Timeout ]
    19691969
     1970# The navigator.language tests rely on functionality only available in recent releases of macOS Catalina and onward.
     1971webkit.org/b/200043 [ Sierra HighSierra Mojave ] fast/text/international/system-language/navigator-language [ Pass Failure ]

  • branches/safari-609-branch/Source/WTF/ChangeLog

    r254014 r254615  
     12020-01-15  Alan Coon  <alancoon@apple.com>
     2
     3        Cherry-pick r254389. rdar://problem/58548648
     4
     5    REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
     6    https://bugs.webkit.org/show_bug.cgi?id=200043
     7    Source/WTF:
     8   
     9    <rdar://problem/44119496>
     10   
     11    Reviewed by Dean Jackson.
     12   
     13    We ask the system for the current locale using CFLocaleCopyPreferredLanguages(), and then round-trip
     14    it through CFBundleGetLocalizationInfoForLocalization() / CFBundleCopyLocalizationForLocalizationInfo().
     15    This was to work around the fact that CFLocaleCopyPreferredLanguages() previously didn't report BCP47
     16    language codes. However, that round-tripping was introducing errors, such as "zh-Hant-HK" was getting
     17    turned into "zh-Hant-TW" which is clearly wrong. The CFBundle functions were never supposed to be used
     18    in this way.
     19   
     20    Instead, we can use CFLocaleCreateCanonicalLanguageIdentifierFromString() which is intended to
     21    canonicalize locale identifiers, and does return BCP47 language codes. However, this function preserves
     22    more fingerprinting entropy than the old code path, so we pass the input through new NSLocale SPI to
     23    minimize the entropy revealed.
     24   
     25    * WTF.xcodeproj/project.pbxproj:
     26    * wtf/Language.h:
     27    * wtf/Platform.h:
     28    * wtf/PlatformMac.cmake:
     29    * wtf/cf/LanguageCF.cpp:
     30    (WTF::httpStyleLanguageCode):
     31    (WTF::platformUserPreferredLanguages):
     32    * wtf/cocoa/LanguageCocoa.mm: Added.
     33    (WTF::minimizedLanguagesFromLanguages):
     34    * wtf/spi/cocoa/NSLocaleSPI.h: Added.
     35   
     36    Tools:
     37   
     38    Reviewed by Dean Jackson.
     39   
     40    Migrate system language tests to LayoutTests, to match the rest of our system language tests.
     41   
     42    * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
     43    * TestWebKitAPI/Tests/mac/NavigatorLanguage.mm: Removed.
     44   
     45    LayoutTests:
     46   
     47    <rdar://problem/44119496>
     48   
     49    Reviewed by Dean Jackson.
     50   
     51    Migrate and update tests from TestWebKitAPI to LayoutTests, to match the rest of our system language tests.
     52   
     53    * platform/mac/TestExpectations: Mark these tests as possibly failing on older versions of macOS.
     54    * fast/text/international/system-language/navigator-language/navigator-language-en-GB-expected.txt: Added.
     55    * fast/text/international/system-language/navigator-language/navigator-language-en-GB.html: Added.
     56    * fast/text/international/system-language/navigator-language/navigator-language-en-US-expected.txt: Added.
     57    * fast/text/international/system-language/navigator-language/navigator-language-en-US.html: Added.
     58    * fast/text/international/system-language/navigator-language/navigator-language-en-expected.txt: Added.
     59    * fast/text/international/system-language/navigator-language/navigator-language-en.html: Added.
     60    * fast/text/international/system-language/navigator-language/navigator-language-es-419-expected.txt: Added.
     61    * fast/text/international/system-language/navigator-language/navigator-language-es-419.html: Added.
     62    * fast/text/international/system-language/navigator-language/navigator-language-es-ES-expected.txt: Added.
     63    * fast/text/international/system-language/navigator-language/navigator-language-es-ES.html: Added.
     64    * fast/text/international/system-language/navigator-language/navigator-language-es-MX-expected.txt: Added.
     65    * fast/text/international/system-language/navigator-language/navigator-language-es-MX.html: Added.
     66    * fast/text/international/system-language/navigator-language/navigator-language-es-expected.txt: Added.
     67    * fast/text/international/system-language/navigator-language/navigator-language-es.html: Added.
     68    * fast/text/international/system-language/navigator-language/navigator-language-fr-CA-expected.txt: Added.
     69    * fast/text/international/system-language/navigator-language/navigator-language-fr-CA.html: Added.
     70    * fast/text/international/system-language/navigator-language/navigator-language-fr-expected.txt: Added.
     71    * fast/text/international/system-language/navigator-language/navigator-language-fr.html: Added.
     72    * fast/text/international/system-language/navigator-language/navigator-language-hi-expected.txt: Added.
     73    * fast/text/international/system-language/navigator-language/navigator-language-hi.html: Added.
     74    * fast/text/international/system-language/navigator-language/navigator-language-ja-expected.txt: Added.
     75    * fast/text/international/system-language/navigator-language/navigator-language-ja.html: Added.
     76    * fast/text/international/system-language/navigator-language/navigator-language-pt-BR-expected.txt: Added.
     77    * fast/text/international/system-language/navigator-language/navigator-language-pt-BR.html: Added.
     78    * fast/text/international/system-language/navigator-language/navigator-language-pt-PT-expected.txt: Added.
     79    * fast/text/international/system-language/navigator-language/navigator-language-pt-PT.html: Added.
     80    * fast/text/international/system-language/navigator-language/navigator-language-ru-expected.txt: Added.
     81    * fast/text/international/system-language/navigator-language/navigator-language-ru.html: Added.
     82    * fast/text/international/system-language/navigator-language/navigator-language-zh-HK-expected.txt: Added.
     83    * fast/text/international/system-language/navigator-language/navigator-language-zh-HK.html: Added.
     84    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans-expected.txt: Added.
     85    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans.html: Added.
     86    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK-expected.txt: Added.
     87    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK.html: Added.
     88    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-expected.txt: Added.
     89    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant.html: Added.
     90    * fast/text/international/system-language/navigator-language/navigator-language-zh-TW-expected.txt: Added.
     91    * fast/text/international/system-language/navigator-language/navigator-language-zh-TW.html: Added.
     92    * js/dom/navigator-language-expected.txt:
     93   
     94    git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254389 268f45cc-cd09-0410-ab3c-d52691b4dbfc
     95
     96    2020-01-10  Myles C. Maxfield  <mmaxfield@apple.com>
     97
     98            REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
     99            https://bugs.webkit.org/show_bug.cgi?id=200043
     100            <rdar://problem/44119496>
     101
     102            Reviewed by Dean Jackson.
     103
     104            We ask the system for the current locale using CFLocaleCopyPreferredLanguages(), and then round-trip
     105            it through CFBundleGetLocalizationInfoForLocalization() / CFBundleCopyLocalizationForLocalizationInfo().
     106            This was to work around the fact that CFLocaleCopyPreferredLanguages() previously didn't report BCP47
     107            language codes. However, that round-tripping was introducing errors, such as "zh-Hant-HK" was getting
     108            turned into "zh-Hant-TW" which is clearly wrong. The CFBundle functions were never supposed to be used
     109            in this way.
     110
     111            Instead, we can use CFLocaleCreateCanonicalLanguageIdentifierFromString() which is intended to
     112            canonicalize locale identifiers, and does return BCP47 language codes. However, this function preserves
     113            more fingerprinting entropy than the old code path, so we pass the input through new NSLocale SPI to
     114            minimize the entropy revealed.
     115
     116            * WTF.xcodeproj/project.pbxproj:
     117            * wtf/Language.h:
     118            * wtf/Platform.h:
     119            * wtf/PlatformMac.cmake:
     120            * wtf/cf/LanguageCF.cpp:
     121            (WTF::httpStyleLanguageCode):
     122            (WTF::platformUserPreferredLanguages):
     123            * wtf/cocoa/LanguageCocoa.mm: Added.
     124            (WTF::minimizedLanguagesFromLanguages):
     125            * wtf/spi/cocoa/NSLocaleSPI.h: Added.
     126
    11272020-01-03  Dean Jackson  <dino@apple.com>
    2128

  • branches/safari-609-branch/Source/WTF/WTF.xcodeproj/project.pbxproj

    r253987 r254615  
    6161                1C181C931D307AB800F5FA16 /* UTextProviderUTF16.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 1C181C8D1D307AB800F5FA16 /* UTextProviderUTF16.cpp */; };
    6262                1C181C961D30800A00F5FA16 /* TextBreakIteratorInternalICUCocoa.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 1C181C951D30800A00F5FA16 /* TextBreakIteratorInternalICUCocoa.cpp */; };
     63                1C503BE623AAE0AE0072E66B /* LanguageCocoa.mm in Sources */ = {isa = PBXBuildFile; fileRef = 1C503BE523AAE0AE0072E66B /* LanguageCocoa.mm */; };
    6364                1FA47C8A152502DA00568D1B /* WebCoreThread.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 1FA47C88152502DA00568D1B /* WebCoreThread.cpp */; };
    6465                2CCD892A15C0390200285083 /* GregorianDateTime.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2CCD892915C0390200285083 /* GregorianDateTime.cpp */; };
    … …  
    340341                1C181C8E1D307AB800F5FA16 /* UTextProviderUTF16.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = UTextProviderUTF16.h; sourceTree = "<group>"; };
    341342                1C181C951D30800A00F5FA16 /* TextBreakIteratorInternalICUCocoa.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = TextBreakIteratorInternalICUCocoa.cpp; sourceTree = "<group>"; };
     343                1C503BE423AADEEA0072E66B /* NSLocaleSPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = NSLocaleSPI.h; sourceTree = "<group>"; };
     344                1C503BE523AAE0AE0072E66B /* LanguageCocoa.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = LanguageCocoa.mm; sourceTree = "<group>"; };
    342345                1CCDB1491E566626006C73C0 /* TextBreakIteratorCF.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = TextBreakIteratorCF.h; sourceTree = "<group>"; };
    343346                1CCDB14D1E566898006C73C0 /* TextBreakIteratorICU.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = TextBreakIteratorICU.h; sourceTree = "<group>"; };
    … …  
    14131416                                CEA072A7236FFA160018839C /* CrashReporterClientSPI.h */,
    14141417                                E31BDE2822E913CC0029B105 /* MachVMSPI.h */,
     1418                                1C503BE423AADEEA0072E66B /* NSLocaleSPI.h */,
    14151419                                93D191CF20CAECE800C51B8E /* objcSPI.h */,
    14161420                                A5098B011C16A4F900087797 /* SecuritySPI.h */,
    … …  
    14761480                                143DDE9520C8BC37007F76FA /* Entitlements.mm */,
    14771481                                A331D95E21F249F6009F02AA /* FileSystemCocoa.mm */,
     1482                                1C503BE523AAE0AE0072E66B /* LanguageCocoa.mm */,
    14781483                                7A6EBA3320746C34004F9C44 /* MachSendRight.cpp */,
    14791484                                A8A472C5151A825A004123FF /* MainThreadCocoa.mm */,
    … …  
    16221627                                C2BCFC401F61D13000C9222C /* Language.cpp in Sources */,
    16231628                                C2BCFC421F61D61600C9222C /* LanguageCF.cpp in Sources */,
     1629                                1C503BE623AAE0AE0072E66B /* LanguageCocoa.mm in Sources */,
    16241630                                C2BCFC551F621F3F00C9222C /* LineEnding.cpp in Sources */,
    16251631                                0FE1646A1B6FFC9600400E7C /* Lock.cpp in Sources */,

  • branches/safari-609-branch/Source/WTF/wtf/Language.h

    r221768 r254615  
    2929#include <wtf/Vector.h>
    3030
     31#if PLATFORM(COCOA)
     32#import <CoreFoundation/CoreFoundation.h>
     33#import <wtf/RetainPtr.h>
     34#endif
     35
    3136namespace WTF {
    3237
    … …  
    4449WTF_EXPORT void addLanguageChangeObserver(void* context, LanguageChangeObserverFunction);
    4550WTF_EXPORT void removeLanguageChangeObserver(void* context);
     51WTF_EXPORT String displayNameForLanguageLocale(const String&);
    4652
    47 WTF_EXPORT String displayNameForLanguageLocale(const String&);
     53#if PLATFORM(COCOA)
     54bool canMinimizeLanguages();
     55RetainPtr<CFArrayRef> minimizedLanguagesFromLanguages(CFArrayRef);
     56#endif
     57
    4858}
    4959

  • branches/safari-609-branch/Source/WTF/wtf/PlatformMac.cmake

    r252065 r254615  
    2727    spi/cocoa/CrashReporterClientSPI.h
    2828    spi/cocoa/MachVMSPI.h
     29    spi/cocoa/NSLocaleSPI.h
    2930    spi/cocoa/SecuritySPI.h
    3031    spi/cocoa/objcSPI.h
    … …  
    5758    cocoa/Entitlements.mm
    5859    cocoa/FileSystemCocoa.mm
     60    cocoa/LanguageCocoa.mm
    5961    cocoa/MachSendRight.cpp
    6062    cocoa/MainThreadCocoa.mm

  • branches/safari-609-branch/Source/WTF/wtf/cf/LanguageCF.cpp

    r237099 r254615  
    4949static String httpStyleLanguageCode(CFStringRef language)
    5050{
    51     SInt32 languageCode;
    52     SInt32 regionCode;
    53     SInt32 scriptCode;
    54     CFStringEncoding stringEncoding;
     51    RetainPtr<CFStringRef> preferredLanguageCode;
     52    // If we can minimize the language list to reduce fingerprinting, we can afford to be more lossless when canonicalizing the locale list.
     53    if (canMinimizeLanguages())
     54        preferredLanguageCode = adoptCF(CFLocaleCreateCanonicalLanguageIdentifierFromString(kCFAllocatorDefault, language));
     55    else {
     56        SInt32 languageCode;
     57        SInt32 regionCode;
     58        SInt32 scriptCode;
     59        CFStringEncoding stringEncoding;
    5560
    56     // FIXME: This transformation is very wrong:
    57     // 1. There is no reason why CFBundle localization names would be at all related to language names as used on the Web.
    58     // 2. Script Manager codes cannot represent all languages that are now supported by the platform, so the conversion is lossy.
    59     // 3. This should probably match what is sent by the network layer as Accept-Language, but currently, that's implemented separately.
    60     CFBundleGetLocalizationInfoForLocalization(language, &languageCode, &regionCode, &scriptCode, &stringEncoding);
    61     RetainPtr<CFStringRef> preferredLanguageCode = adoptCF(CFBundleCopyLocalizationForLocalizationInfo(languageCode, regionCode, scriptCode, stringEncoding));
    62     if (preferredLanguageCode)
    63         language = preferredLanguageCode.get();
     61        // FIXME: This transformation is very wrong:
     62        // 1. There is no reason why CFBundle localization names would be at all related to language names as used on the Web.
     63        // 2. Script Manager codes cannot represent all languages that are now supported by the platform, so the conversion is lossy.
     64        // 3. This should probably match what is sent by the network layer as Accept-Language, but currently, that's implemented separately.
     65        CFBundleGetLocalizationInfoForLocalization(language, &languageCode, &regionCode, &scriptCode, &stringEncoding);
     66        preferredLanguageCode = adoptCF(CFBundleCopyLocalizationForLocalizationInfo(languageCode, regionCode, scriptCode, stringEncoding));
     67    }
     68
     69    if (!preferredLanguageCode)
     70        preferredLanguageCode = language;
     71    auto mutableLanguageCode = adoptCF(CFStringCreateMutableCopy(kCFAllocatorDefault, 0, preferredLanguageCode.get()));
    6472
    6573    // Turn a '_' into a '-' if it appears after a 2-letter language code
    66     if (CFStringGetLength(language) >= 3 && CFStringGetCharacterAtIndex(language, 2) == '_') {
    67         auto mutableLanguageCode = adoptCF(CFStringCreateMutableCopy(kCFAllocatorDefault, 0, language));
     74    if (CFStringGetLength(mutableLanguageCode.get()) >= 3 && CFStringGetCharacterAtIndex(mutableLanguageCode.get(), 2) == '_')
    6875        CFStringReplace(mutableLanguageCode.get(), CFRangeMake(2, 1), CFSTR("-"));
    69         return mutableLanguageCode.get();
    70     }
    7176
    72     return language;
     77    CFStringLowercase(mutableLanguageCode.get(), nullptr);
     78    return mutableLanguageCode.get();
     79
    7380}
    7481
    … …  
    99106    if (userPreferredLanguages.isEmpty()) {
    100107        RetainPtr<CFArrayRef> languages = adoptCF(CFLocaleCopyPreferredLanguages());
     108        languages = minimizedLanguagesFromLanguages(languages.get());
    101109        CFIndex languageCount = CFArrayGetCount(languages.get());
    102110        if (!languageCount)

  • branches/safari-609-branch/Tools/ChangeLog

    r254613 r254615  
     12020-01-15  Alan Coon  <alancoon@apple.com>
     2
     3        Cherry-pick r254389. rdar://problem/58548648
     4
     5    REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
     6    https://bugs.webkit.org/show_bug.cgi?id=200043
     7    Source/WTF:
     8   
     9    <rdar://problem/44119496>
     10   
     11    Reviewed by Dean Jackson.
     12   
     13    We ask the system for the current locale using CFLocaleCopyPreferredLanguages(), and then round-trip
     14    it through CFBundleGetLocalizationInfoForLocalization() / CFBundleCopyLocalizationForLocalizationInfo().
     15    This was to work around the fact that CFLocaleCopyPreferredLanguages() previously didn't report BCP47
     16    language codes. However, that round-tripping was introducing errors, such as "zh-Hant-HK" was getting
     17    turned into "zh-Hant-TW" which is clearly wrong. The CFBundle functions were never supposed to be used
     18    in this way.
     19   
     20    Instead, we can use CFLocaleCreateCanonicalLanguageIdentifierFromString() which is intended to
     21    canonicalize locale identifiers, and does return BCP47 language codes. However, this function preserves
     22    more fingerprinting entropy than the old code path, so we pass the input through new NSLocale SPI to
     23    minimize the entropy revealed.
     24   
     25    * WTF.xcodeproj/project.pbxproj:
     26    * wtf/Language.h:
     27    * wtf/Platform.h:
     28    * wtf/PlatformMac.cmake:
     29    * wtf/cf/LanguageCF.cpp:
     30    (WTF::httpStyleLanguageCode):
     31    (WTF::platformUserPreferredLanguages):
     32    * wtf/cocoa/LanguageCocoa.mm: Added.
     33    (WTF::minimizedLanguagesFromLanguages):
     34    * wtf/spi/cocoa/NSLocaleSPI.h: Added.
     35   
     36    Tools:
     37   
     38    Reviewed by Dean Jackson.
     39   
     40    Migrate system language tests to LayoutTests, to match the rest of our system language tests.
     41   
     42    * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
     43    * TestWebKitAPI/Tests/mac/NavigatorLanguage.mm: Removed.
     44   
     45    LayoutTests:
     46   
     47    <rdar://problem/44119496>
     48   
     49    Reviewed by Dean Jackson.
     50   
     51    Migrate and update tests from TestWebKitAPI to LayoutTests, to match the rest of our system language tests.
     52   
     53    * platform/mac/TestExpectations: Mark these tests as possibly failing on older versions of macOS.
     54    * fast/text/international/system-language/navigator-language/navigator-language-en-GB-expected.txt: Added.
     55    * fast/text/international/system-language/navigator-language/navigator-language-en-GB.html: Added.
     56    * fast/text/international/system-language/navigator-language/navigator-language-en-US-expected.txt: Added.
     57    * fast/text/international/system-language/navigator-language/navigator-language-en-US.html: Added.
     58    * fast/text/international/system-language/navigator-language/navigator-language-en-expected.txt: Added.
     59    * fast/text/international/system-language/navigator-language/navigator-language-en.html: Added.
     60    * fast/text/international/system-language/navigator-language/navigator-language-es-419-expected.txt: Added.
     61    * fast/text/international/system-language/navigator-language/navigator-language-es-419.html: Added.
     62    * fast/text/international/system-language/navigator-language/navigator-language-es-ES-expected.txt: Added.
     63    * fast/text/international/system-language/navigator-language/navigator-language-es-ES.html: Added.
     64    * fast/text/international/system-language/navigator-language/navigator-language-es-MX-expected.txt: Added.
     65    * fast/text/international/system-language/navigator-language/navigator-language-es-MX.html: Added.
     66    * fast/text/international/system-language/navigator-language/navigator-language-es-expected.txt: Added.
     67    * fast/text/international/system-language/navigator-language/navigator-language-es.html: Added.
     68    * fast/text/international/system-language/navigator-language/navigator-language-fr-CA-expected.txt: Added.
     69    * fast/text/international/system-language/navigator-language/navigator-language-fr-CA.html: Added.
     70    * fast/text/international/system-language/navigator-language/navigator-language-fr-expected.txt: Added.
     71    * fast/text/international/system-language/navigator-language/navigator-language-fr.html: Added.
     72    * fast/text/international/system-language/navigator-language/navigator-language-hi-expected.txt: Added.
     73    * fast/text/international/system-language/navigator-language/navigator-language-hi.html: Added.
     74    * fast/text/international/system-language/navigator-language/navigator-language-ja-expected.txt: Added.
     75    * fast/text/international/system-language/navigator-language/navigator-language-ja.html: Added.
     76    * fast/text/international/system-language/navigator-language/navigator-language-pt-BR-expected.txt: Added.
     77    * fast/text/international/system-language/navigator-language/navigator-language-pt-BR.html: Added.
     78    * fast/text/international/system-language/navigator-language/navigator-language-pt-PT-expected.txt: Added.
     79    * fast/text/international/system-language/navigator-language/navigator-language-pt-PT.html: Added.
     80    * fast/text/international/system-language/navigator-language/navigator-language-ru-expected.txt: Added.
     81    * fast/text/international/system-language/navigator-language/navigator-language-ru.html: Added.
     82    * fast/text/international/system-language/navigator-language/navigator-language-zh-HK-expected.txt: Added.
     83    * fast/text/international/system-language/navigator-language/navigator-language-zh-HK.html: Added.
     84    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans-expected.txt: Added.
     85    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hans.html: Added.
     86    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK-expected.txt: Added.
     87    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-HK.html: Added.
     88    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant-expected.txt: Added.
     89    * fast/text/international/system-language/navigator-language/navigator-language-zh-Hant.html: Added.
     90    * fast/text/international/system-language/navigator-language/navigator-language-zh-TW-expected.txt: Added.
     91    * fast/text/international/system-language/navigator-language/navigator-language-zh-TW.html: Added.
     92    * js/dom/navigator-language-expected.txt:
     93   
     94    git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254389 268f45cc-cd09-0410-ab3c-d52691b4dbfc
     95
     96    2020-01-10  Myles C. Maxfield  <mmaxfield@apple.com>
     97
     98            REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
     99            https://bugs.webkit.org/show_bug.cgi?id=200043
     100
     101            Reviewed by Dean Jackson.
     102
     103            Migrate system language tests to LayoutTests, to match the rest of our system language tests.
     104
     105            * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
     106            * TestWebKitAPI/Tests/mac/NavigatorLanguage.mm: Removed.
     107
    11082020-01-14  Alan Coon  <alancoon@apple.com>
    2109

  • branches/safari-609-branch/Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj

    r253950 r254615  
    376376                5C0BF8931DD599BD00B00328 /* IsNavigationActionTrusted.mm in Sources */ = {isa = PBXBuildFile; fileRef = 57F10D921C7E7B3800ECDF30 /* IsNavigationActionTrusted.mm */; };
    377377                5C0BF8941DD599C900B00328 /* MenuTypesForMouseEvents.mm in Sources */ = {isa = PBXBuildFile; fileRef = 7A99D9931AD4A29D00373141 /* MenuTypesForMouseEvents.mm */; };
    378                 5C0BF8951DD599CD00B00328 /* NavigatorLanguage.mm in Sources */ = {isa = PBXBuildFile; fileRef = E19DB9781B32137C00DB38D4 /* NavigatorLanguage.mm */; };
    379378                5C16F8FC230C94370074C4A8 /* TextSize.mm in Sources */ = {isa = PBXBuildFile; fileRef = 5C16F8FB230C942B0074C4A8 /* TextSize.mm */; };
    380379                5C19A5241FD0F60100EEA323 /* CookiePrivateBrowsing.mm in Sources */ = {isa = PBXBuildFile; fileRef = 5C19A5231FD0F32600EEA323 /* CookiePrivateBrowsing.mm */; };
    … …  
    24952494                E194E1BA177E5145009C4D4E /* StopLoadingFromDidReceiveResponse.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = StopLoadingFromDidReceiveResponse.mm; sourceTree = "<group>"; };
    24962495                E194E1BC177E534A009C4D4E /* StopLoadingFromDidReceiveResponse.html */ = {isa = PBXFileReference; lastKnownFileType = text.html; path = StopLoadingFromDidReceiveResponse.html; sourceTree = "<group>"; };
    2497                 E19DB9781B32137C00DB38D4 /* NavigatorLanguage.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = NavigatorLanguage.mm; sourceTree = "<group>"; };
    24982496                E32B549122810AC0008AD702 /* Packed.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = Packed.cpp; sourceTree = "<group>"; };
    24992497                E35FC7B122B82A6D00F32F98 /* JSLockTakesWebThreadLock.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = JSLockTakesWebThreadLock.mm; sourceTree = "<group>"; };
    … …  
    40024000                                5C0BF88C1DD5957400B00328 /* MemoryPressureHandler.mm */,
    40034001                                7A99D9931AD4A29D00373141 /* MenuTypesForMouseEvents.mm */,
    4004                                 E19DB9781B32137C00DB38D4 /* NavigatorLanguage.mm */,
    40054002                                83F22C6320B355EB0034277E /* NoPolicyDelegateResponse.mm */,
    40064003                                A57A34EF16AF677200C2501F /* PageVisibilityStateWithWindowChanges.mm */,
    … …  
    47724769                                7CCE7F241A411AF600447C4C /* Navigation.mm in Sources */,
    47734770                                6351992822275C6A00890AD3 /* NavigationAction.mm in Sources */,
    4774                                 5C0BF8951DD599CD00B00328 /* NavigatorLanguage.mm in Sources */,
    47754771                                5C8BC799218CF44700813886 /* NetworkProcess.mm in Sources */,
    47764772                                5CAE463820193B6A0051610F /* NetworkProcessCrashNonPersistentDataStore.mm in Sources */,
Note: See TracChangeset for help on using the changeset viewer.