Changeset 208090 in webkit


Ignore:
Timestamp:
Oct 28, 2016, 6:06:13 PM (9 years ago)
Author:
Wenson Hsieh
Message:

iOS autocorrection does not trigger an input event of inputType "insertReplacementText"
​https://bugs.webkit.org/show_bug.cgi?id=164077
<rdar://problem/28987810>

Reviewed by Simon Fraser.

Source/WebCore:

Fixes candidate insertion on iOS, so that it fires input events of type "insertReplacementText" and adds two
iOS unit tests covering this change as well as the test infrastructure needed to support these tests. See
comments below for more details.

Tests: fast/events/ios/before-input-events-prevent-candidate-insertion.html

fast/events/ios/input-events-insert-replacement-text.html

  • dom/TextEvent.h:

Adds isAutocompletion() to TextEvent, as well as the TextEventInputAutocompletion text input type. When the
Editor handles this TextEvent, it will use this information when creating or modifying the corresponding typing
command.

  • dom/TextEventInputType.h:
  • editing/Editor.cpp:

(WebCore::Editor::insertText):
(WebCore::Editor::insertTextWithoutSendingTextEvent):

  • editing/Editor.h:
  • editing/TypingCommand.cpp:

(WebCore::editActionForTypingCommand):

Now takes whether the command is autocorrection into account. If so, the corresponding edit action should be
EditActionInsertReplacement rather than EditActionTypingInsertText.

(WebCore::TypingCommand::TypingCommand):
(WebCore::TypingCommand::deleteSelection):
(WebCore::TypingCommand::deleteKeyPressed):
(WebCore::TypingCommand::forwardDeleteKeyPressed):
(WebCore::TypingCommand::insertText):
(WebCore::TypingCommand::insertLineBreak):
(WebCore::TypingCommand::insertParagraphSeparatorInQuotedContent):
(WebCore::TypingCommand::insertParagraphSeparator):
(WebCore::TypingCommand::inputEventData):
(WebCore::TypingCommand::willAddTypingToOpenCommand):

  • editing/TypingCommand.h:

Adds a new TypingCommand option, IsAutocompletion.

Source/WebKit2:

Small tweak to mark text insertion when autocorrecting as such, as opposed to regular keyboard input.

  • WebProcess/WebPage/ios/WebPageIOS.mm:

(WebKit::WebPage::syncApplyAutocorrection):

Tools:

Adds test support for inserting text candidates on iOS in the form of
UIScriptController.selectTextCandidateAtIndex(index, callback), which selects the text candidate at a given
index (this needs to be a value between 0-2 on iOS) and fires the callback when done.

To implement this, we wait for the text prediction view to have predictions (we determine this by polling at a
given interval) and then tap the center of the text prediction view at the given index.

  • DumpRenderTree/ios/UIScriptControllerIOS.mm:

(WTR::UIScriptController::selectTextCandidateAtIndex):

  • TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
  • TestRunnerShared/UIScriptContext/UIScriptController.cpp:

(WTR::UIScriptController::selectTextCandidateAtIndex):
(WTR::UIScriptController::waitForTextPredictionsViewAndSelectCandidateAtIndex):

  • TestRunnerShared/UIScriptContext/UIScriptController.h:
  • WebKitTestRunner/ios/UIKitSPI.h:
  • WebKitTestRunner/ios/UIScriptControllerIOS.mm:

(WTR::UIScriptController::selectTextCandidateAtIndex):
(WTR::UIScriptController::waitForTextPredictionsViewAndSelectCandidateAtIndex):

LayoutTests:

Adds 2 new unit tests verifying that candidate text insertion can be prevented via beforeinput events, and that
beforeinput and input events of type "insertReplacementText" are fired when inserting candidate text on iOS.

  • fast/events/ios/before-input-events-prevent-candidate-insertion-expected.txt: Added.
  • fast/events/ios/before-input-events-prevent-candidate-insertion.html: Added.
  • fast/events/ios/input-events-insert-replacement-text-expected.txt: Added.
  • fast/events/ios/input-events-insert-replacement-text.html: Added.
Location:
trunk
Files:
4 added
17 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r208087 r208090  
     12016-10-28  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        iOS autocorrection does not trigger an input event of inputType "insertReplacementText"
     4        https://bugs.webkit.org/show_bug.cgi?id=164077
     5        <rdar://problem/28987810>
     6
     7        Reviewed by Simon Fraser.
     8
     9        Adds 2 new unit tests verifying that candidate text insertion can be prevented via beforeinput events, and that
     10        beforeinput and input events of type "insertReplacementText" are fired when inserting candidate text on iOS.
     11
     12        * fast/events/ios/before-input-events-prevent-candidate-insertion-expected.txt: Added.
     13        * fast/events/ios/before-input-events-prevent-candidate-insertion.html: Added.
     14        * fast/events/ios/input-events-insert-replacement-text-expected.txt: Added.
     15        * fast/events/ios/input-events-insert-replacement-text.html: Added.
     16
    1172016-10-28  Alex Christensen  <achristensen@webkit.org>
    218

  • trunk/Source/WebCore/ChangeLog

    r208089 r208090  
     12016-10-28  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        iOS autocorrection does not trigger an input event of inputType "insertReplacementText"
     4        https://bugs.webkit.org/show_bug.cgi?id=164077
     5        <rdar://problem/28987810>
     6
     7        Reviewed by Simon Fraser.
     8
     9        Fixes candidate insertion on iOS, so that it fires input events of type "insertReplacementText" and adds two
     10        iOS unit tests covering this change as well as the test infrastructure needed to support these tests. See
     11        comments below for more details.
     12
     13        Tests: fast/events/ios/before-input-events-prevent-candidate-insertion.html
     14               fast/events/ios/input-events-insert-replacement-text.html
     15
     16        * dom/TextEvent.h:
     17
     18        Adds isAutocompletion() to TextEvent, as well as the TextEventInputAutocompletion text input type. When the
     19        Editor handles this TextEvent, it will use this information when creating or modifying the corresponding typing
     20        command.
     21
     22        * dom/TextEventInputType.h:
     23        * editing/Editor.cpp:
     24        (WebCore::Editor::insertText):
     25        (WebCore::Editor::insertTextWithoutSendingTextEvent):
     26        * editing/Editor.h:
     27        * editing/TypingCommand.cpp:
     28        (WebCore::editActionForTypingCommand):
     29
     30        Now takes whether the command is autocorrection into account. If so, the corresponding edit action should be
     31        EditActionInsertReplacement rather than EditActionTypingInsertText.
     32
     33        (WebCore::TypingCommand::TypingCommand):
     34        (WebCore::TypingCommand::deleteSelection):
     35        (WebCore::TypingCommand::deleteKeyPressed):
     36        (WebCore::TypingCommand::forwardDeleteKeyPressed):
     37        (WebCore::TypingCommand::insertText):
     38        (WebCore::TypingCommand::insertLineBreak):
     39        (WebCore::TypingCommand::insertParagraphSeparatorInQuotedContent):
     40        (WebCore::TypingCommand::insertParagraphSeparator):
     41        (WebCore::TypingCommand::inputEventData):
     42        (WebCore::TypingCommand::willAddTypingToOpenCommand):
     43        * editing/TypingCommand.h:
     44
     45        Adds a new TypingCommand option, IsAutocompletion.
     46
    1472016-10-28  Commit Queue  <commit-queue@webkit.org>
    248

  • trunk/Source/WebCore/dom/TextEvent.h

    r204717 r208090  
    6060        bool isDrop() const { return m_inputType == TextEventInputDrop; }
    6161        bool isDictation() const { return m_inputType == TextEventInputDictation; }
     62        bool isAutocompletion() const { return m_inputType == TextEventInputAutocompletion; }
    6263
    6364        bool shouldSmartReplace() const { return m_shouldSmartReplace; }

  • trunk/Source/WebCore/dom/TextEventInputType.h

    r114220 r208090  
    3131enum TextEventInputType {
    3232    TextEventInputKeyboard, // any newline characters in the text are line breaks only, not paragraph separators.
     33    TextEventInputAutocompletion,
    3334    TextEventInputLineBreak, // any tab characters in the text are backtabs.
    3435    TextEventInputComposition,

  • trunk/Source/WebCore/editing/Editor.cpp

    r208014 r208090  
    12141214}
    12151215
    1216 bool Editor::insertText(const String& text, Event* triggeringEvent)
    1217 {
    1218     return m_frame.eventHandler().handleTextInputEvent(text, triggeringEvent);
     1216bool Editor::insertText(const String& text, Event* triggeringEvent, TextEventInputType inputType)
     1217{
     1218    return m_frame.eventHandler().handleTextInputEvent(text, triggeringEvent, inputType);
    12191219}
    12201220
    … …  
    12701270                if (autocorrectionWasApplied)
    12711271                    options |= TypingCommand::RetainAutocorrectionIndicator;
     1272                if (triggeringEvent && triggeringEvent->isAutocompletion())
     1273                    options |= TypingCommand::IsAutocompletion;
    12721274                TypingCommand::insertText(document, text, selection, options, triggeringEvent && triggeringEvent->isComposition() ? TypingCommand::TextCompositionFinal : TypingCommand::TextCompositionNone);
    12731275            }

  • trunk/Source/WebCore/editing/Editor.h

    r207797 r208090  
    3737#include "FrameSelection.h"
    3838#include "TextChecking.h"
     39#include "TextEventInputType.h"
    3940#include "TextIteratorBehavior.h"
    4041#include "VisibleSelection.h"
    … …  
    240241    WEBCORE_EXPORT static bool commandIsSupportedFromMenuOrKeyBinding(const String& commandName); // Works without a frame.
    241242
    242     WEBCORE_EXPORT bool insertText(const String&, Event* triggeringEvent);
     243    WEBCORE_EXPORT bool insertText(const String&, Event* triggeringEvent, TextEventInputType = TextEventInputKeyboard);
    243244    bool insertTextForConfirmedComposition(const String& text);
    244245    WEBCORE_EXPORT bool insertDictatedText(const String&, const Vector<DictationAlternative>& dictationAlternatives, Event* triggeringEvent);

  • trunk/Source/WebCore/editing/TypingCommand.cpp

    r207698 r208090  
    7878};
    7979
    80 static inline EditAction editActionForTypingCommand(TypingCommand::ETypingCommand command, TextGranularity granularity, TypingCommand::TextCompositionType compositionType)
     80static inline EditAction editActionForTypingCommand(TypingCommand::ETypingCommand command, TextGranularity granularity, TypingCommand::TextCompositionType compositionType, bool isAutocompletion)
    8181{
    8282    if (compositionType == TypingCommand::TextCompositionPending) {
    … …  
    113113        return EditActionTypingDeleteForward;
    114114    case TypingCommand::InsertText:
    115         return EditActionTypingInsertText;
     115        return isAutocompletion ? EditActionInsertReplacement : EditActionTypingInsertText;
    116116    case TypingCommand::InsertLineBreak:
    117117        return EditActionTypingInsertLineBreak;
    … …  
    141141
    142142TypingCommand::TypingCommand(Document& document, ETypingCommand commandType, const String &textToInsert, Options options, TextGranularity granularity, TextCompositionType compositionType)
    143     : TextInsertionBaseCommand(document, editActionForTypingCommand(commandType, granularity, compositionType))
     143    : TextInsertionBaseCommand(document, editActionForTypingCommand(commandType, granularity, compositionType, options & IsAutocompletion))
    144144    , m_commandType(commandType)
    145145    , m_textToInsert(textToInsert)
    … …  
    151151    , m_compositionType(compositionType)
    152152    , m_shouldAddToKillRing(options & AddsToKillRing)
     153    , m_isAutocompletion(options & IsAutocompletion)
    153154    , m_openedByBackwardDelete(false)
    154155    , m_shouldRetainAutocorrectionIndicator(options & RetainAutocorrectionIndicator)
    … …  
    168169
    169170    if (RefPtr<TypingCommand> lastTypingCommand = lastTypingCommandIfStillOpenForTyping(*frame)) {
     171        lastTypingCommand->setIsAutocompletion(options & IsAutocompletion);
    170172        lastTypingCommand->setCompositionType(compositionType);
    171173        lastTypingCommand->setShouldPreventSpellChecking(options & PreventSpellChecking);
    … …  
    182184        if (RefPtr<TypingCommand> lastTypingCommand = lastTypingCommandIfStillOpenForTyping(*document.frame())) {
    183185            updateSelectionIfDifferentFromCurrentSelection(lastTypingCommand.get(), document.frame());
     186            lastTypingCommand->setIsAutocompletion(options & IsAutocompletion);
    184187            lastTypingCommand->setCompositionType(TextCompositionNone);
    185188            lastTypingCommand->setShouldPreventSpellChecking(options & PreventSpellChecking);
    … …  
    199202        if (RefPtr<TypingCommand> lastTypingCommand = lastTypingCommandIfStillOpenForTyping(*frame)) {
    200203            updateSelectionIfDifferentFromCurrentSelection(lastTypingCommand.get(), frame);
     204            lastTypingCommand->setIsAutocompletion(options & IsAutocompletion);
    201205            lastTypingCommand->setCompositionType(TextCompositionNone);
    202206            lastTypingCommand->setShouldPreventSpellChecking(options & PreventSpellChecking);
    … …  
    252256        }
    253257
     258        lastTypingCommand->setIsAutocompletion(options & IsAutocompletion);
    254259        lastTypingCommand->setCompositionType(compositionType);
    255260        lastTypingCommand->setShouldRetainAutocorrectionIndicator(options & RetainAutocorrectionIndicator);
    … …  
    266271{
    267272    if (RefPtr<TypingCommand> lastTypingCommand = lastTypingCommandIfStillOpenForTyping(*document.frame())) {
     273        lastTypingCommand->setIsAutocompletion(options & IsAutocompletion);
    268274        lastTypingCommand->setCompositionType(TextCompositionNone);
    269275        lastTypingCommand->setShouldRetainAutocorrectionIndicator(options & RetainAutocorrectionIndicator);
    … …  
    278284{
    279285    if (RefPtr<TypingCommand> lastTypingCommand = lastTypingCommandIfStillOpenForTyping(*document.frame())) {
     286        lastTypingCommand->setIsAutocompletion(false);
    280287        lastTypingCommand->setCompositionType(TextCompositionNone);
    281288        lastTypingCommand->insertParagraphSeparatorInQuotedContentAndNotifyAccessibility();
    … …  
    289296{
    290297    if (RefPtr<TypingCommand> lastTypingCommand = lastTypingCommandIfStillOpenForTyping(*document.frame())) {
     298        lastTypingCommand->setIsAutocompletion(options & IsAutocompletion);
    291299        lastTypingCommand->setCompositionType(TextCompositionNone);
    292300        lastTypingCommand->setShouldRetainAutocorrectionIndicator(options & RetainAutocorrectionIndicator);
    … …  
    399407    switch (m_currentTypingEditAction) {
    400408    case EditActionTypingInsertText:
     409    case EditActionInsertReplacement:
    401410    case EditActionTypingInsertPendingComposition:
    402411    case EditActionTypingInsertFinalComposition:
    … …  
    470479{
    471480    m_currentTextToInsert = text;
    472     m_currentTypingEditAction = editActionForTypingCommand(commandType, granularity, m_compositionType);
     481    m_currentTypingEditAction = editActionForTypingCommand(commandType, granularity, m_compositionType, m_isAutocompletion);
    473482
    474483    if (!shouldDeferWillApplyCommandUntilAddingTypingCommand())

  • trunk/Source/WebCore/editing/TypingCommand.h

    r207698 r208090  
    5454        RetainAutocorrectionIndicator = 1 << 2,
    5555        PreventSpellChecking = 1 << 3,
    56         SmartDelete = 1 << 4
     56        SmartDelete = 1 << 4,
     57        IsAutocompletion = 1 << 5,
    5758    };
    5859    typedef unsigned Options;
    … …  
    8081    void deleteSelection(bool smartDelete);
    8182    void setCompositionType(TextCompositionType type) { m_compositionType = type; }
     83    void setIsAutocompletion(bool isAutocompletion) { m_isAutocompletion = isAutocompletion; }
    8284
    8385#if PLATFORM(IOS)
    … …  
    152154    bool m_shouldAddToKillRing;
    153155    bool m_preservesTypingStyle;
     156    bool m_isAutocompletion;
    154157   
    155158    // Undoing a series of backward deletes will restore a selection around all of the

  • trunk/Source/WebKit2/ChangeLog

    r208081 r208090  
     12016-10-28  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        iOS autocorrection does not trigger an input event of inputType "insertReplacementText"
     4        https://bugs.webkit.org/show_bug.cgi?id=164077
     5        <rdar://problem/28987810>
     6
     7        Reviewed by Simon Fraser.
     8
     9        Small tweak to mark text insertion when autocorrecting as such, as opposed to regular keyboard input.
     10
     11        * WebProcess/WebPage/ios/WebPageIOS.mm:
     12        (WebKit::WebPage::syncApplyAutocorrection):
     13
    1142016-10-28  Megan Gardner  <megan_gardner@apple.com>
    215

  • trunk/Source/WebKit2/WebProcess/WebPage/ios/WebPageIOS.mm

    r208020 r208090  
    21722172    frame.selection().setSelectedRange(range.get(), UPSTREAM, true);
    21732173    if (correction.length())
    2174         frame.editor().insertText(correction, 0);
     2174        frame.editor().insertText(correction, 0, originalText.isEmpty() ? TextEventInputKeyboard : TextEventInputAutocompletion);
    21752175    else
    21762176        frame.editor().deleteWithDirection(DirectionBackward, CharacterGranularity, false, true);

  • trunk/Tools/ChangeLog

    r208089 r208090  
     12016-10-28  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        iOS autocorrection does not trigger an input event of inputType "insertReplacementText"
     4        https://bugs.webkit.org/show_bug.cgi?id=164077
     5        <rdar://problem/28987810>
     6
     7        Reviewed by Simon Fraser.
     8
     9        Adds test support for inserting text candidates on iOS in the form of
     10        UIScriptController.selectTextCandidateAtIndex(index, callback), which selects the text candidate at a given
     11        index (this needs to be a value between 0-2 on iOS) and fires the callback when done.
     12
     13        To implement this, we wait for the text prediction view to have predictions (we determine this by polling at a
     14        given interval) and then tap the center of the text prediction view at the given index.
     15
     16        * DumpRenderTree/ios/UIScriptControllerIOS.mm:
     17        (WTR::UIScriptController::selectTextCandidateAtIndex):
     18        * TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
     19        * TestRunnerShared/UIScriptContext/UIScriptController.cpp:
     20        (WTR::UIScriptController::selectTextCandidateAtIndex):
     21        (WTR::UIScriptController::waitForTextPredictionsViewAndSelectCandidateAtIndex):
     22        * TestRunnerShared/UIScriptContext/UIScriptController.h:
     23        * WebKitTestRunner/ios/UIKitSPI.h:
     24        * WebKitTestRunner/ios/UIScriptControllerIOS.mm:
     25        (WTR::UIScriptController::selectTextCandidateAtIndex):
     26        (WTR::UIScriptController::waitForTextPredictionsViewAndSelectCandidateAtIndex):
     27
    1282016-10-28  Commit Queue  <commit-queue@webkit.org>
    229

  • trunk/Tools/DumpRenderTree/ios/UIScriptControllerIOS.mm

    r207447 r208090  
    117117}
    118118
     119void UIScriptController::selectTextCandidateAtIndex(long, JSValueRef)
     120{
     121}
     122
    119123void UIScriptController::keyDownUsingHardwareKeyboard(JSStringRef character, JSValueRef callback)
    120124{

  • trunk/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl

    r207447 r208090  
    4949    void keyDownUsingHardwareKeyboard(DOMString character, object callback);
    5050    void keyUpUsingHardwareKeyboard(DOMString character, object callback);
     51
     52    void selectTextCandidateAtIndex(long index, object callback);
    5153
    5254    // eventsJSON describes a series of user events in JSON form. For the keys, see HIDEventGenerator.mm.

  • trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.cpp

    r207447 r208090  
    215215}
    216216
     217void UIScriptController::selectTextCandidateAtIndex(long, JSValueRef)
     218{
     219}
     220
     221void UIScriptController::waitForTextPredictionsViewAndSelectCandidateAtIndex(long, unsigned, float)
     222{
     223}
     224
    217225void UIScriptController::keyDownUsingHardwareKeyboard(JSStringRef, JSValueRef)
    218226{

  • trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h

    r207447 r208090  
    7171    void keyUpUsingHardwareKeyboard(JSStringRef character, JSValueRef callback);
    7272
     73    void selectTextCandidateAtIndex(long index, JSValueRef callback);
     74
    7375    void keyboardAccessoryBarNext();
    7476    void keyboardAccessoryBarPrevious();
    … …  
    137139
    138140    JSObjectRef objectFromRect(const WebCore::FloatRect&) const;
     141    void waitForTextPredictionsViewAndSelectCandidateAtIndex(long index, unsigned callbackID, float interval);
    139142
    140143    UIScriptContext* m_context;

  • trunk/Tools/WebKitTestRunner/ios/UIKitSPI.h

    r190841 r208090  
    3838#import <UIKit/UIWindow_Private.h>
    3939
     40@interface UIKeyboardPredictionView : UIView
     41+ (UIKeyboardPredictionView *)activeInstance;
     42- (BOOL)hasPredictions;
     43@end
     44
    4045#else
    4146

  • trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm

    r207447 r208090  
    256256}
    257257
     258void UIScriptController::selectTextCandidateAtIndex(long index, JSValueRef callback)
     259{
     260#if USE(APPLE_INTERNAL_SDK)
     261    static const float textPredictionsPollingInterval = 0.1;
     262    unsigned callbackID = m_context->prepareForAsyncTask(callback, CallbackTypeNonPersistent);
     263    waitForTextPredictionsViewAndSelectCandidateAtIndex(index, callbackID, textPredictionsPollingInterval);
     264#else
     265    // FIXME: This is a no-op on non-internal builds due to UIKeyboardPredictionView being unavailable. Ideally, there should be a better way to
     266    // retrieve information and interact with the predictive text view that will be compatible with OpenSource.
     267    UNUSED_PARAM(index);
     268    UNUSED_PARAM(callback);
     269#endif
     270}
     271
     272void UIScriptController::waitForTextPredictionsViewAndSelectCandidateAtIndex(long index, unsigned callbackID, float interval)
     273{
     274#if USE(APPLE_INTERNAL_SDK)
     275    if (![UIKeyboardPredictionView activeInstance].hasPredictions) {
     276        dispatch_after(dispatch_time(DISPATCH_TIME_NOW, interval * NSEC_PER_SEC), dispatch_get_main_queue(), ^() {
     277            waitForTextPredictionsViewAndSelectCandidateAtIndex(index, callbackID, interval);
     278        });
     279        return;
     280    }
     281
     282    PlatformWKView webView = TestController::singleton().mainWebView()->platformView();
     283    CGRect predictionViewFrame = [UIKeyboardPredictionView activeInstance].frame;
     284    // This assumes there are 3 predicted text cells of equal width, which is the case on iOS.
     285    float offsetX = (index * 2 + 1) * CGRectGetWidth(predictionViewFrame) / 6;
     286    float offsetY = CGRectGetHeight(webView.window.frame) - CGRectGetHeight([UIKeyboardPredictionView activeInstance].superview.frame) + CGRectGetHeight(predictionViewFrame) / 2;
     287    [[HIDEventGenerator sharedHIDEventGenerator] tap:CGPointMake(offsetX, offsetY) completionBlock:^{
     288        if (m_context)
     289            m_context->asyncTaskComplete(callbackID);
     290    }];
     291#else
     292    UNUSED_PARAM(index);
     293    UNUSED_PARAM(callbackID);
     294    UNUSED_PARAM(interval);
     295#endif
     296}
     297
    258298void UIScriptController::dismissFormAccessoryView()
    259299{
Note: See TracChangeset for help on using the changeset viewer.