⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 242833 in webkit


Ignore:
Timestamp:
Mar 12, 2019, 4:51:57 PM (7 years ago)
Author:
Wenson Hsieh
Message:

[iOS] Input view sometimes flickers when blurring and refocusing an element
​https://bugs.webkit.org/show_bug.cgi?id=195639
<rdar://problem/48735337>

Reviewed by Tim Horton.

Source/WebKit:

On iOS, if a focused element is blurred and immediately refocused in the scope of user interaction, we will end
up reloading interaction state (input views, autocorrection contexts, etc.) in the UI process. On certain well-
trafficked websites, this results in the input view and input accessory view flickering (or more egregiously,
scrolling to re-reveal the focused element) when changing selection.

To fix the issue, this patch refactors some focus management logic to suppress sending focused element updates
to the UI process in the case where the same element is being blurred and immediately refocused. To do this, we
track the most recently blurred element and bail when the recently blurred element is identical to the newly
focused element. See below for more detail.

Test: fast/forms/ios/keyboard-stability-when-refocusing-element.html

  • UIProcess/WebPageProxy.h:
  • UIProcess/ios/WKContentViewInteraction.mm:

(-[WKContentView _elementDidFocus:userIsInteracting:blurPreviousNode:changingActivityState:userObject:]):
(-[WKContentView _elementDidBlur]):

Update the web process' notion of whether an input view is showing. Importantly, this accounts for decisions
made by _WKUIDelegate. See below for more details.

(isAssistableInputType): Deleted.

Removed this helper function; this was only used in one place as a sanity check that the focused element's type
is not none, right before attempting to show an input view. Instead, we can just check the focused element's
type directly against InputType::None in the if statement of the early return.

  • UIProcess/ios/WebPageProxyIOS.mm:

(WebKit::WebPageProxy::setIsShowingInputViewForFocusedElement):

Add a hook to notify the web process when an input view is showing or not (see below for more detail).

  • WebProcess/WebPage/WebPage.cpp:

(WebKit::WebPage::didStartPageTransition):
(WebKit::WebPage::elementDidRefocus):
(WebKit::WebPage::shouldDispatchUpdateAfterFocusingElement const):

Add a helper to determine whether we notify the UI process about a newly focused element. On macOS, this is true
only when the new focused element is neither the currently focused element, nor the focused element that was
just blurred. On iOS, we have an additional constraint that when the input view is not showing, we still need to
notify the UI process, since the UI process might want to begin showing the keyboard for an element that has
only been programmatically focused, for which we aren't currently showing the input view.

(WebKit::WebPage::elementDidFocus):
(WebKit::WebPage::elementDidBlur):

Replace a couple of existing member variables in WebPage used for focus management:

  • Replace m_hasPendingBlurNotification with m_recentlyBlurredElement, a RefPtr to the Element that is being

blurred. Behavior here is the same as before (i.e. having a pending blur notification is equivalent to
having recently blurred a focused element). However, this allows us to check newly focused elements against
the recently blurred element in WebPage::elementDidFocus().

  • Replace m_isFocusingElementDueToUserInteraction with m_isShowingInputViewForFocusedElement. The flag

m_isFocusingElementDueToUserInteraction was originally added to fix <webkit.org/b/146735>, by ensuring that
we don't send redundant ElementDidFocus (formerly, StartAssistingNode) messages to the UI process even when
the keyboard is already up. In these simpler times, user interaction when focusing an element was equivalent
to showing an input view for the focused element. However, in today's world, there are a variety of reasons
why we might or might not show an input view for a given element (including, but not limited to activity
state changes and decisions made by _WKInputDelegate). As such, it doesn't make sense to continue relying on
m_isFocusingElementDueToUserInteraction in this early return. Instead, have the UI process propagate a
message back to the web process, to let it know whether there is a keyboard showing, and use this flag
instead.

  • WebProcess/WebPage/WebPage.h:
  • WebProcess/WebPage/WebPage.messages.in:
  • WebProcess/WebPage/ios/WebPageIOS.mm:

(WebKit::WebPage::setIsShowingInputViewForFocusedElement):

LayoutTests:

Add a test to ensure that the form control interaction doesn't stop and start again when blurring and focusing
an editable element.

  • fast/forms/ios/keyboard-stability-when-refocusing-element-expected.txt: Added.
  • fast/forms/ios/keyboard-stability-when-refocusing-element.html: Added.
Location:
trunk
Files:
2 added
9 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r242826 r242833  
     12019-03-12  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Input view sometimes flickers when blurring and refocusing an element
     4        https://bugs.webkit.org/show_bug.cgi?id=195639
     5        <rdar://problem/48735337>
     6
     7        Reviewed by Tim Horton.
     8
     9        Add a test to ensure that the form control interaction doesn't stop and start again when blurring and focusing
     10        an editable element.
     11
     12        * fast/forms/ios/keyboard-stability-when-refocusing-element-expected.txt: Added.
     13        * fast/forms/ios/keyboard-stability-when-refocusing-element.html: Added.
     14
    1152019-03-12  Dean Jackson  <dino@apple.com>
    216

  • trunk/Source/WebKit/ChangeLog

    r242824 r242833  
     12019-03-12  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Input view sometimes flickers when blurring and refocusing an element
     4        https://bugs.webkit.org/show_bug.cgi?id=195639
     5        <rdar://problem/48735337>
     6
     7        Reviewed by Tim Horton.
     8
     9        On iOS, if a focused element is blurred and immediately refocused in the scope of user interaction, we will end
     10        up reloading interaction state (input views, autocorrection contexts, etc.) in the UI process. On certain well-
     11        trafficked websites, this results in the input view and input accessory view flickering (or more egregiously,
     12        scrolling to re-reveal the focused element) when changing selection.
     13
     14        To fix the issue, this patch refactors some focus management logic to suppress sending focused element updates
     15        to the UI process in the case where the same element is being blurred and immediately refocused. To do this, we
     16        track the most recently blurred element and bail when the recently blurred element is identical to the newly
     17        focused element. See below for more detail.
     18
     19        Test: fast/forms/ios/keyboard-stability-when-refocusing-element.html
     20
     21        * UIProcess/WebPageProxy.h:
     22        * UIProcess/ios/WKContentViewInteraction.mm:
     23        (-[WKContentView _elementDidFocus:userIsInteracting:blurPreviousNode:changingActivityState:userObject:]):
     24        (-[WKContentView _elementDidBlur]):
     25
     26        Update the web process' notion of whether an input view is showing. Importantly, this accounts for decisions
     27        made by _WKUIDelegate. See below for more details.
     28
     29        (isAssistableInputType): Deleted.
     30
     31        Removed this helper function; this was only used in one place as a sanity check that the focused element's type
     32        is not none, right before attempting to show an input view. Instead, we can just check the focused element's
     33        type directly against InputType::None in the if statement of the early return.
     34
     35        * UIProcess/ios/WebPageProxyIOS.mm:
     36        (WebKit::WebPageProxy::setIsShowingInputViewForFocusedElement):
     37
     38        Add a hook to notify the web process when an input view is showing or not (see below for more detail).
     39
     40        * WebProcess/WebPage/WebPage.cpp:
     41        (WebKit::WebPage::didStartPageTransition):
     42        (WebKit::WebPage::elementDidRefocus):
     43        (WebKit::WebPage::shouldDispatchUpdateAfterFocusingElement const):
     44
     45        Add a helper to determine whether we notify the UI process about a newly focused element. On macOS, this is true
     46        only when the new focused element is neither the currently focused element, nor the focused element that was
     47        just blurred. On iOS, we have an additional constraint that when the input view is not showing, we still need to
     48        notify the UI process, since the UI process might want to begin showing the keyboard for an element that has
     49        only been programmatically focused, for which we aren't currently showing the input view.
     50
     51        (WebKit::WebPage::elementDidFocus):
     52        (WebKit::WebPage::elementDidBlur):
     53
     54        Replace a couple of existing member variables in WebPage used for focus management:
     55        -   Replace m_hasPendingBlurNotification with m_recentlyBlurredElement, a RefPtr to the Element that is being
     56            blurred. Behavior here is the same as before (i.e. having a pending blur notification is equivalent to
     57            having recently blurred a focused element). However, this allows us to check newly focused elements against
     58            the recently blurred element in WebPage::elementDidFocus().
     59        -   Replace m_isFocusingElementDueToUserInteraction with m_isShowingInputViewForFocusedElement. The flag
     60            m_isFocusingElementDueToUserInteraction was originally added to fix <webkit.org/b/146735>, by ensuring that
     61            we don't send redundant ElementDidFocus (formerly, StartAssistingNode) messages to the UI process even when
     62            the keyboard is already up. In these simpler times, user interaction when focusing an element was equivalent
     63            to showing an input view for the focused element. However, in today's world, there are a variety of reasons
     64            why we might or might not show an input view for a given element (including, but not limited to activity
     65            state changes and decisions made by _WKInputDelegate). As such, it doesn't make sense to continue relying on
     66            m_isFocusingElementDueToUserInteraction in this early return. Instead, have the UI process propagate a
     67            message back to the web process, to let it know whether there is a keyboard showing, and use this flag
     68            instead.
     69
     70        * WebProcess/WebPage/WebPage.h:
     71        * WebProcess/WebPage/WebPage.messages.in:
     72        * WebProcess/WebPage/ios/WebPageIOS.mm:
     73        (WebKit::WebPage::setIsShowingInputViewForFocusedElement):
     74
    1752019-03-12  Tim Horton  <timothy_horton@apple.com>
    276

  • trunk/Source/WebKit/UIProcess/WebPageProxy.h

    r242796 r242833  
    11951195
    11961196    void blurFocusedElement();
     1197    void setIsShowingInputViewForFocusedElement(bool);
    11971198#endif
    11981199

  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm

    r242801 r242833  
    48184818}
    48194819
    4820 static bool isAssistableInputType(WebKit::InputType type)
    4821 {
    4822     switch (type) {
    4823     case WebKit::InputType::ContentEditable:
    4824     case WebKit::InputType::Text:
    4825     case WebKit::InputType::Password:
    4826     case WebKit::InputType::TextArea:
    4827     case WebKit::InputType::Search:
    4828     case WebKit::InputType::Email:
    4829     case WebKit::InputType::URL:
    4830     case WebKit::InputType::Phone:
    4831     case WebKit::InputType::Number:
    4832     case WebKit::InputType::NumberPad:
    4833     case WebKit::InputType::Date:
    4834     case WebKit::InputType::DateTime:
    4835     case WebKit::InputType::DateTimeLocal:
    4836     case WebKit::InputType::Month:
    4837     case WebKit::InputType::Week:
    4838     case WebKit::InputType::Time:
    4839     case WebKit::InputType::Select:
    4840     case WebKit::InputType::Drawing:
    4841 #if ENABLE(INPUT_TYPE_COLOR)
    4842     case WebKit::InputType::Color:
    4843 #endif
    4844         return true;
    4845 
    4846     case WebKit::InputType::None:
    4847         return false;
    4848     }
    4849 
    4850     ASSERT_NOT_REACHED();
    4851     return false;
    4852 }
    4853 
    48544820static const double minimumFocusedElementAreaForSuppressingSelectionAssistant = 4;
    48554821
    … …  
    49344900#endif
    49354901
    4936     if (!shouldShowInputView)
    4937         return;
    4938 
    4939     if (!isAssistableInputType(information.elementType))
    4940         return;
     4902    if (!shouldShowInputView || information.elementType == WebKit::InputType::None) {
     4903        _page->setIsShowingInputViewForFocusedElement(false);
     4904        return;
     4905    }
     4906
     4907    _page->setIsShowingInputViewForFocusedElement(true);
    49414908
    49424909    // FIXME: We should remove this check when we manage to send ElementDidFocus from the WebProcess
    … …  
    50575024
    50585025    [_webView didEndFormControlInteraction];
     5026    _page->setIsShowingInputViewForFocusedElement(false);
    50595027
    50605028    if (!_isChangingFocus) {

  • trunk/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm

    r242796 r242833  
    899899}
    900900
     901void WebPageProxy::setIsShowingInputViewForFocusedElement(bool showingInputView)
     902{
     903    process().send(Messages::WebPage::SetIsShowingInputViewForFocusedElement(showingInputView), m_pageID);
     904}
     905
    901906void WebPageProxy::elementDidFocus(const FocusedElementInformation& information, bool userIsInteracting, bool blurPreviousNode, bool changingActivityState, const UserData& userData)
    902907{

  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp

    r242762 r242833  
    31253125#endif
    31263126    m_hasEverFocusedElementDueToUserInteractionSincePageTransition = false;
    3127     m_isFocusingElementDueToUserInteraction = false;
    31283127    m_lastEditorStateWasContentEditable = EditorStateIsContentEditable::Unset;
    31293128#if PLATFORM(MAC)
    … …  
    31383137        send(Messages::WebPageProxy::SetNeedsPlainTextQuirk(m_needsPlainTextQuirk));
    31393138    }
     3139#endif
     3140#if PLATFORM(IOS_FAMILY)
     3141    m_isShowingInputViewForFocusedElement = false;
    31403142#endif
    31413143}
    … …  
    53045306    elementDidFocus(element);
    53055307
    5306     if (m_isFocusingElementDueToUserInteraction)
     5308    if (m_userIsInteracting)
    53075309        scheduleFullEditorStateUpdate();
    53085310}
    53095311
     5312bool WebPage::shouldDispatchUpdateAfterFocusingElement(const Element& element) const
     5313{
     5314    if (m_focusedElement == &element || m_recentlyBlurredElement == &element) {
     5315#if PLATFORM(IOS_FAMILY)
     5316        return !m_isShowingInputViewForFocusedElement;
     5317#else
     5318        return false;
     5319#endif
     5320    }
     5321    return true;
     5322}
     5323
    53105324void WebPage::elementDidFocus(WebCore::Element& element)
    53115325{
    5312     if (m_focusedElement == &element && m_isFocusingElementDueToUserInteraction)
    5313         return;
     5326    if (!shouldDispatchUpdateAfterFocusingElement(element)) {
     5327        m_focusedElement = &element;
     5328        m_recentlyBlurredElement = nullptr;
     5329        return;
     5330    }
    53145331
    53155332    if (element.hasTagName(WebCore::HTMLNames::selectTag) || element.hasTagName(WebCore::HTMLNames::inputTag) || element.hasTagName(WebCore::HTMLNames::textareaTag) || element.hasEditableStyle()) {
    53165333        m_focusedElement = &element;
    5317         m_isFocusingElementDueToUserInteraction |= m_userIsInteracting;
    53185334
    53195335#if PLATFORM(IOS_FAMILY)
    … …  
    53315347        m_formClient->willBeginInputSession(this, &element, WebFrame::fromCoreFrame(*element.document().frame()), m_userIsInteracting, userData);
    53325348
    5333         send(Messages::WebPageProxy::ElementDidFocus(information, m_userIsInteracting, m_hasPendingBlurNotification, m_changingActivityState, UserData(WebProcess::singleton().transformObjectsToHandles(userData.get()).get())));
     5349        send(Messages::WebPageProxy::ElementDidFocus(information, m_userIsInteracting, m_recentlyBlurredElement, m_changingActivityState, UserData(WebProcess::singleton().transformObjectsToHandles(userData.get()).get())));
    53345350#elif PLATFORM(MAC)
    5335         if (element.hasTagName(WebCore::HTMLNames::selectTag))
    5336             send(Messages::WebPageProxy::SetEditableElementIsFocused(false));
    5337         else
    5338             send(Messages::WebPageProxy::SetEditableElementIsFocused(true));
    5339 #endif
    5340         m_hasPendingBlurNotification = false;
     5351        // FIXME: This can be unified with the iOS code above by bringing ElementDidFocus to macOS.
     5352        // This also doesn't take other noneditable controls into account, such as input type color.
     5353        send(Messages::WebPageProxy::SetEditableElementIsFocused(!element.hasTagName(WebCore::HTMLNames::selectTag)));
     5354#endif
     5355        m_recentlyBlurredElement = nullptr;
    53415356    }
    53425357}
    … …  
    53455360{
    53465361    if (m_focusedElement == &element) {
    5347         m_hasPendingBlurNotification = true;
    5348         RefPtr<WebPage> protectedThis(this);
    5349         callOnMainThread([protectedThis] {
    5350             if (protectedThis->m_hasPendingBlurNotification) {
     5362        m_recentlyBlurredElement = WTFMove(m_focusedElement);
     5363        callOnMainThread([protectedThis = makeRefPtr(this)] {
     5364            if (protectedThis->m_recentlyBlurredElement) {
    53515365#if PLATFORM(IOS_FAMILY)
    53525366                protectedThis->send(Messages::WebPageProxy::ElementDidBlur());
    … …  
    53555369#endif
    53565370            }
    5357             protectedThis->m_hasPendingBlurNotification = false;
     5371            protectedThis->m_recentlyBlurredElement = nullptr;
    53585372        });
    5359 
    5360         m_isFocusingElementDueToUserInteraction = false;
    5361         m_focusedElement = nullptr;
    53625373    }
    53635374}

  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.h

    r242796 r242833  
    659659    void setFocusedElementValueAsNumber(double);
    660660    void setFocusedElementSelectedIndex(uint32_t index, bool allowMultipleSelection);
     661    void setIsShowingInputViewForFocusedElement(bool);
    661662    void updateSelectionAppearance();
    662663    void getSelectionContext(CallbackID);
    … …  
    15281529    void cancelGesturesBlockedOnSynchronousReplies();
    15291530
     1531    bool shouldDispatchUpdateAfterFocusingElement(const WebCore::Element&) const;
     1532
    15301533    uint64_t m_pageID;
    15311534
    … …  
    17251728
    17261729    bool m_userIsInteracting { false };
    1727     bool m_isFocusingElementDueToUserInteraction { false };
    17281730    bool m_hasEverFocusedElementDueToUserInteractionSincePageTransition { false };
    17291731    bool m_needsHiddenContentEditableQuirk { false };
    … …  
    17361738
    17371739    RefPtr<WebCore::Element> m_focusedElement;
    1738     bool m_hasPendingBlurNotification { false };
     1740    RefPtr<WebCore::Element> m_recentlyBlurredElement;
    17391741    bool m_hasPendingEditorStateUpdate { false };
    17401742
    … …  
    17471749    RefPtr<WebCore::Node> m_interactionNode;
    17481750    WebCore::IntPoint m_lastInteractionLocation;
     1751
     1752    bool m_isShowingInputViewForFocusedElement { false };
    17491753   
    17501754    enum SelectionAnchor { Start, End };

  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.messages.in

    r242796 r242833  
    110110    RequestFocusedElementInformation(WebKit::CallbackID callbackID)
    111111    HardwareKeyboardAvailabilityChanged(bool keyboardIsAttached)
     112    SetIsShowingInputViewForFocusedElement(bool showingInputView)
    112113#endif
    113114

  • trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm

    r242798 r242833  
    931931}
    932932
     933void WebPage::setIsShowingInputViewForFocusedElement(bool showingInputView)
     934{
     935    m_isShowingInputViewForFocusedElement = showingInputView;
     936}
     937
    933938void WebPage::setFocusedElementValue(const String& value)
    934939{
Note: See TracChangeset for help on using the changeset viewer.