Changeset 268843 in webkit

Timestamp:

Oct 21, 2020, 4:09:27 PM (6 years ago)

Author:

commit-queue@webkit.org

Message:

Implement ISO-2022-JP encoder step 3
​https://bugs.webkit.org/show_bug.cgi?id=218046

Patch by Alex Christensen <​achristensen@webkit.org> on 2020-10-21
Reviewed by Darin Adler.

LayoutTests/imported/w3c:

• web-platform-tests/encoding/iso-2022-jp-encoder-expected.txt:
• web-platform-tests/encoding/iso-2022-jp-encoder.html:

Source/WebCore:

​https://encoding.spec.whatwg.org/#iso-2022-jp-encoder step 3 was put in a few years ago and recently implemented by Firefox.
Covered by updated and newly passing web platform tests.

• platform/text/TextCodecCJK.cpp:

(WebCore::iso2022JPEncode):

Location:

trunk

Files:

• LayoutTests/imported/w3c/ChangeLog (modified) (1 diff)
• LayoutTests/imported/w3c/web-platform-tests/encoding/iso-2022-jp-encoder-expected.txt (modified) (1 diff)
• LayoutTests/imported/w3c/web-platform-tests/encoding/iso-2022-jp-encoder.html (modified) (1 diff)
• Source/WebCore/ChangeLog (modified) (1 diff)
• Source/WebCore/platform/text/TextCodecCJK.cpp (modified) (1 diff)

trunk/LayoutTests/imported/w3c/ChangeLog

@@ +1 @@
+2020-10-21  Alex Christensen  <achristensen@webkit.org>
+
+        Implement ISO-2022-JP encoder step 3
+        https://bugs.webkit.org/show_bug.cgi?id=218046
+
+        Reviewed by Darin Adler.
+
+        * web-platform-tests/encoding/iso-2022-jp-encoder-expected.txt:
+        * web-platform-tests/encoding/iso-2022-jp-encoder.html:
+
 2020-10-21  Chris Dumez  <cdumez@apple.com>
 

trunk/LayoutTests/imported/w3c/web-platform-tests/encoding/iso-2022-jp-encoder-expected.txt

@@ -2 +2 @@
 PASS iso-2022-jp encoder: very basic
 PASS iso-2022-jp encoder: basics
+PASS iso-2022-jp encoder: Katakana
+PASS iso-2022-jp encoder: jis0208
 PASS iso-2022-jp encoder: SO/SI ESC
+PASS iso-2022-jp encoder: Roman SO/SI ESC
+PASS iso-2022-jp encoder: Katakana SO/SI ESC
+PASS iso-2022-jp encoder: jis0208 SO/SI ESC
 PASS iso-2022-jp encoder: U+FFFD
+PASS iso-2022-jp encoder: Roman U+FFFD
+PASS iso-2022-jp encoder: Katakana U+FFFD
+PASS iso-2022-jp encoder: jis0208 U+FFFD
 

trunk/LayoutTests/imported/w3c/web-platform-tests/encoding/iso-2022-jp-encoder.html

@@ -13 +13 @@
  }
 
- encode("s", "s", "very basic")
- encode("\u00A5\u203Es\\\uFF90\u4F69", "%1B(J\\~s%1B(B\\%1B$B%_PP%1B(B", "basics")
- encode("\x0E\x0F\x1Bx", "%0E%0F%1Bx", "SO/SI ESC")
+ encode("s", "s", "very basic");
+ encode("\u00A5\u203Es\\\uFF90\u4F69", "%1B(J\\~s%1B(B\\%1B$B%_PP%1B(B", "basics");
+ encode("\uFF61", "%1B$B!%23%1B(B", "Katakana");
+ encode("\u0393", "%1B$B&%23%1B(B", "jis0208");
+ encode("\x0E\x0F\x1Bx", "%26%2365533%3B%26%2365533%3B%26%2365533%3Bx", "SO/SI ESC");
+ encode("\u203E\x0E\x0F\x1Bx", "%1B(J~%26%2365533%3B%26%2365533%3B%26%2365533%3Bx%1B(B", "Roman SO/SI ESC");
+ encode("\uFF61\x0E\x0F\x1Bx", "%1B$B!%23%1B(B%26%2365533%3B%26%2365533%3B%26%2365533%3Bx", "Katakana SO/SI ESC");
+ encode("\u0393\x0E\x0F\x1Bx", "%1B$B&%23%1B(B%26%2365533%3B%26%2365533%3B%26%2365533%3Bx", "jis0208 SO/SI ESC");
  encode("\uFFFD", "%26%2365533%3B", "U+FFFD");
+ encode("\u203E\uFFFD", "%1B(J~%26%2365533%3B%1B(B", "Roman U+FFFD");
+ encode("\uFF61\uFFFD", "%1B$B!%23%1B(B%26%2365533%3B", "Katakana U+FFFD");
+ encode("\u0393\uFFFD", "%1B$B&%23%1B(B%26%2365533%3B", "jis0208 U+FFFD");
 </script>

trunk/Source/WebCore/ChangeLog

@@ +1 @@
+2020-10-21  Alex Christensen  <achristensen@webkit.org>
+
+        Implement ISO-2022-JP encoder step 3
+        https://bugs.webkit.org/show_bug.cgi?id=218046
+
+        Reviewed by Darin Adler.
+
+        https://encoding.spec.whatwg.org/#iso-2022-jp-encoder step 3 was put in a few years ago and recently implemented by Firefox.
+        Covered by updated and newly passing web platform tests.
+
+        * platform/text/TextCodecCJK.cpp:
+        (WebCore::iso2022JPEncode):
+
 2020-10-21  Chris Dumez  <cdumez@apple.com>
 

trunk/Source/WebCore/platform/text/TextCodecCJK.cpp

@@ -484 +484 @@
     Function<void(UChar32)> parseCodePoint;
     parseCodePoint = [&] (UChar32 codePoint) {
+        if ((state == State::ASCII || state == State::Roman) && (codePoint == 0x000E || codePoint == 0x000F || codePoint == 0x001B)) {
+            statefulUnencodableHandler(replacementCharacter, result);
+            return;
+        }
         if (state == State::ASCII && isASCII(codePoint)) {
             result.append(codePoint);