Changeset 204074 in webkit

Timestamp:

Aug 2, 2016 11:35:08 PM (8 years ago)

Author:

mmaxfield@apple.com

Message:

Update breaking rules to match ICU 57
​https://bugs.webkit.org/show_bug.cgi?id=160488
<rdar://problem/25856238>

Reviewed by Darin Adler.

Source/WTF:

This patch fixes a typo in the uax14AssignmentsAfter rules which
was causing flag emoji to have line breaking opportunities in their
middles. It also fixes significant language issues for languages
such as Hebrew, and adds correct breaking opportunities around
hyphens.

• wtf/text/TextBreakIterator.cpp:

(WTF::cursorMovementIterator):
(WTF::uax14AssignmentsAfter):
(WTF::uax14Forward):
(WTF::uax14Reverse):

LayoutTests:

• fast/text/flag-emoji-line-break-expected.html: Added.
• fast/text/flag-emoji-line-break.html: Added.
• platform/mac/TestExpectations: Mark the test as failing on Yosemite.

Location:

trunk

Files:

• LayoutTests/ChangeLog (modified) (1 diff)
• LayoutTests/fast/text/flag-emoji-line-break-expected.html (added)
• LayoutTests/fast/text/flag-emoji-line-break.html (added)
• LayoutTests/platform/mac/TestExpectations (modified) (1 diff)
• Source/WTF/ChangeLog (modified) (1 diff)
• Source/WTF/wtf/text/TextBreakIterator.cpp (modified) (5 diffs)

trunk/LayoutTests/ChangeLog

@@ +1 @@
+2016-08-02  Myles C. Maxfield  <mmaxfield@apple.com>
+
+        Update breaking rules to match ICU 57
+        https://bugs.webkit.org/show_bug.cgi?id=160488
+        <rdar://problem/25856238>
+
+        Reviewed by Darin Adler.
+
+        * fast/text/flag-emoji-line-break-expected.html: Added.
+        * fast/text/flag-emoji-line-break.html: Added.
+        * platform/mac/TestExpectations: Mark the test as failing on Yosemite.
+
 2016-08-02  Chris Dumez  <cdumez@apple.com>
 

trunk/LayoutTests/platform/mac/TestExpectations

@@ -1440 +1440 @@
 webkit.org/b/159755 [ Sierra+ ] fast/text/emoji-gender-fe0f-9.html [ Pass ]
 webkit.org/b/159755 [ Sierra+ ] fast/text/emoji-gender.html [ Pass ]
+
+# ADDITIONAL_EMOJI_SUPPORT is not enabled on Yosemite
+webkit.org/b/160488 [ Yosemite ] fast/text/flag-emoji-line-break.html [ ImageOnlyFailure ]

trunk/Source/WTF/ChangeLog

@@ +1 @@
+2016-08-02  Myles C. Maxfield  <mmaxfield@apple.com>
+
+        Update breaking rules to match ICU 57
+        https://bugs.webkit.org/show_bug.cgi?id=160488
+        <rdar://problem/25856238>
+
+        Reviewed by Darin Adler.
+
+        This patch fixes a typo in the uax14AssignmentsAfter rules which
+        was causing flag emoji to have line breaking opportunities in their
+        middles. It also fixes significant language issues for languages
+        such as Hebrew, and adds correct breaking opportunities around
+        hyphens.
+
+        * wtf/text/TextBreakIterator.cpp:
+        (WTF::cursorMovementIterator):
+        (WTF::uax14AssignmentsAfter):
+        (WTF::uax14Forward):
+        (WTF::uax14Reverse):
+
 2016-08-02  Benjamin Poulain  <benjamin@webkit.org>
 

trunk/Source/WTF/wtf/text/TextBreakIterator.cpp

@@ -165 +165 @@
 TextBreakIterator* cursorMovementIterator(StringView string)
 {
-    // FIXME: These rules need to be updated for additional gender-based emoji support.
 #if !PLATFORM(IOS)
-    // This rule set is based on character-break iterator rules of ICU 4.0
-    // <http://source.icu-project.org/repos/icu/icu/tags/release-4-0/source/data/brkitr/char.txt>.
+    // This rule set is based on character-break iterator rules of ICU 57
+    // <http://source.icu-project.org/repos/icu/icu/tags/release-57-1/source/data/brkitr/>.
     // The major differences from the original ones are listed below:
     // * Replaced '[\p{Grapheme_Cluster_Break = SpacingMark}]' with '[\p{General_Category = Spacing Mark} - $Extend]' for ICU 3.8 or earlier;
@@ -481 +480 @@
     "$PRcm = $PR $CM*;"
     "$QUcm = $QU $CM*;"
-    "$RIcm = $QU $CM*;"
+    "$RIcm = $RI $CM*;"
     "$SYcm = $SY $CM*;"
     "$WJcm = $WJ $CM*;";
@@ -523 +522 @@
     "$PR $CM+;"
     "$QU $CM+;"
+    "$RI $CM+;"
     "$SY $CM+;"
     "$WJ $CM+;"
@@ -573 +573 @@
     "$BBcm $LB20NonBreaks $CM*;"
     "$HLcm ($HYcm | $BAcm) [^$CB]?;"
+    "$SYcm $HLcm;"
     "($ALcm | $HLcm) $INcm;"
     "$CM+ $INcm;"
+    "$EXcm $INcm;"
     "$IDcm $INcm;"
     "$INcm $INcm;"
@@ -681 +683 @@
     "[^$CB] $CM* $BB;"
     "[^$CB] $CM* ($HY | $BA) $CM* $HL;"
+    "$CM* $HL $CM* $SY;"
     "$CM* $IN $CM* ($ALPlus | $HL);"
+    "$CM* $IN $CM* $EX;"
     "$CM* $IN $CM* $ID;"
     "$CM* $IN $CM* $IN;"