Changeset 295299 in webkit

Timestamp:

Jun 6, 2022, 3:08:33 PM (4 years ago)

Author:

Wenson Hsieh

Message:

[iOS] Keyboard obscures file upload context menu when trying to attach a file in gmail.com
​https://bugs.webkit.org/show_bug.cgi?id=241320
rdar://84667910

Reviewed by Tim Horton.

When composing a message on gmail.com, if a user taps the "Attach file" button in the bottom
toolbar, Gmail's script programmatically clicks a hidden file input and then immediately focuses the
editable body field. The context menu presents at the interaction location near the bottom of the
viewport; however, if no hardware keyboard is attached, the software keyboard appears right
afterwards, obscuring the context menu completely.

To address this, teach WKFileUploadPanel to reposition its context menu when the keyboard is
shown, if it might overlap with the bounds of the input view.

• LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt: Added.
• LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html: Added.

Add a layout test to verify that focusing a text field with the software keyboard while presenting a
file picker doesn't result in the file picker's context menu being obscured behind the keyboard.

• LayoutTests/resources/ui-helper.js:

(window.UIHelper.dismissMenu):
(window.UIHelper.contextMenuRect):

Add a couple of new UIHelper methods.

• Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:

(-[WKContentView _keyboardDidShow]):

If the keyboard appears while the file upload panel is active, reposition the context menu if it
overlaps with the new keyboard bounds.

(-[WKContentView _zoomToFocusRectAfterShowingKeyboardIfNeeded]):

Factor this existing logic out into a helper method.

• Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h:
• Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm:

(-[WKFileUploadPanel contextMenuInteraction:willEndForConfiguration:animator:]):

If the _isRepositioningContextMenu flag is set, avoid removing and destroying the context menu
interaction.

(-[WKFileUploadPanel ensureContextMenuInteraction]):

Drive-by fix: adjust -ensureContextMenuInteraction so that it returns the either the existing or
newly created context menu interaction.

(-[WKFileUploadPanel repositionContextMenuIfNeeded]):

Add a helper method to reposition the context menu by removing and re-presenting the context menu
interaction without animation, only if it overlaps the input view bounds. While removing and
presenting the context menu again, set a _isRepositioningContextMenu flag, such that

(-[WKFileUploadPanel showDocumentPickerMenu]):

• Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
• Tools/TestRunnerShared/UIScriptContext/UIScriptController.h:

(WTR::UIScriptController::contextMenuRect const):

• Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm:

(-[TestRunnerWKWebView _dismissAllContextMenuInteractions]):

Adjust this helper to immediately cancel all context menu interactions without animation, so that
context menu removal delegate methods don't bleed into subsequent tests.

• Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h:
• Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm:

(WTR::UIScriptControllerIOS::menuRect const):

Use the new toObject helper method.

(WTR::UIScriptControllerIOS::contextMenuRect const):

Add support for a new script controller method to return the context menu container view's bounds
in window coordinates. Note that I'm using window coordinates here as opposed to root view
coordinates, since the new test that uses this method needs to compare the context menu's position
against the input view bounds in window coordinates.

(WTR::UIScriptControllerIOS::toObject const):

Add a helper method to convert the given CGRect into a JavaScript object reference.

Canonical link: ​https://commits.webkit.org/251344@main

Location:

trunk

Files:

• LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt (added)
• LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html (added)
• LayoutTests/resources/ui-helper.js (modified) (2 diffs)
• Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm (modified) (1 diff)
• Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h (modified) (1 diff)
• Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm (modified) (7 diffs)
• Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl (modified) (1 diff)
• Tools/TestRunnerShared/UIScriptContext/UIScriptController.h (modified) (1 diff)
• Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm (modified) (1 diff)
• Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h (modified) (3 diffs)
• Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm (modified) (1 diff)

trunk/LayoutTests/resources/ui-helper.js

@@ -779 +779 @@
     }
 
+    static dismissMenu()
+    {
+        if (!this.isWebKit2())
+            return Promise.resolve();
+
+        return new Promise(resolve => {
+            testRunner.runUIScript("uiController.dismissMenu()", resolve);
+        });
+    }
+
     static waitForKeyboardToHide()
     {
@@ -1353 +1363 @@
         return new Promise(resolve => {
             testRunner.runUIScript("JSON.stringify(uiController.menuRect)", result => resolve(JSON.parse(result)));
+        });
+    }
+
+    static contextMenuRect()
+    {
+        return new Promise(resolve => {
+            testRunner.runUIScript("JSON.stringify(uiController.contextMenuRect)", result => resolve(JSON.parse(result)));
         });
     }

trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm

@@ -2378 +2378 @@
 - (void)_keyboardDidShow
 {
+    [self _zoomToFocusRectAfterShowingKeyboardIfNeeded];
+
+#if USE(UICONTEXTMENU)
+    [_fileUploadPanel repositionContextMenuIfNeeded];
+#endif
+}
+
+- (void)_zoomToFocusRectAfterShowingKeyboardIfNeeded
+{
     if (!_shouldZoomToFocusRectAfterShowingKeyboard)
         return;

trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h

@@ -45 +45 @@
 - (void)dismiss;
 
+#if USE(UICONTEXTMENU)
+- (void)repositionContextMenuIfNeeded;
+#endif
+
 - (NSArray<NSString *> *)currentAvailableActionTitles;
 - (NSArray<NSString *> *)acceptedTypeIdentifiers;

trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm

@@ -40 +40 @@
 #import "WKStringCF.h"
 #import "WKURLCF.h"
+#import "WKWebViewInternal.h"
 #import "WebIconUtilities.h"
 #import "WebOpenPanelResultListenerProxy.h"
@@ -50 +51 @@
 #import <wtf/OptionSet.h>
 #import <wtf/RetainPtr.h>
+#import <wtf/SetForScope.h>
 #import <wtf/WeakObjCPtr.h>
 #import <wtf/text/StringView.h>
@@ -357 +359 @@
     ALLOW_DEPRECATED_DECLARATIONS_END
 #if USE(UICONTEXTMENU)
+    BOOL _isRepositioningContextMenu;
     RetainPtr<UIContextMenuInteraction> _documentContextMenuInteraction;
 #endif
@@ -683 +686 @@
 - (void)contextMenuInteraction:(UIContextMenuInteraction *)interaction willEndForConfiguration:(UIContextMenuConfiguration *)configuration animator:(id<UIContextMenuInteractionAnimating>)animator
 {
+    if (_isRepositioningContextMenu)
+        return;
+
     [animator addCompletion:^{
         [self removeContextMenuInteraction];
@@ -699 +705 @@
 }
 
-- (void)ensureContextMenuInteraction
+- (UIContextMenuInteraction *)ensureContextMenuInteraction
 {
     if (!_documentContextMenuInteraction) {
@@ -706 +712 @@
         self->_isPresentingSubMenu = NO;
     }
-}
-
-#endif
+    return _documentContextMenuInteraction.get();
+}
+
+- (void)repositionContextMenuIfNeeded
+{
+    if (!_documentContextMenuInteraction)
+        return;
+
+    auto *webView = [_view webView];
+    if (!webView)
+        return;
+
+    auto inputViewBoundsInWindow = webView->_inputViewBoundsInWindow;
+    if (CGRectIsEmpty(inputViewBoundsInWindow))
+        return;
+
+    // The exact bounds of the context menu container itself isn't exposed through any UIKit API or SPI,
+    // and would require traversing the view hierarchy in search of internal UIKit views. For now, just
+    // reposition the context menu if its presentation location is covered by the input view.
+    if (!CGRectContainsPoint(inputViewBoundsInWindow, [_view convertPoint:_interactionPoint toView:webView.window]))
+        return;
+
+    SetForScope repositioningContextMenuScope { _isRepositioningContextMenu, YES };
+    [UIView performWithoutAnimation:^{
+        [_documentContextMenuInteraction dismissMenu];
+        [_view presentContextMenu:_documentContextMenuInteraction.get() atLocation:_interactionPoint];
+    }];
+}
+
+#endif // USE(UICONTEXTMENU)
 
 - (void)showFilePickerMenu
@@ -728 +761 @@
     // FIXME 49961589: Support picking media with UIImagePickerController
 #if HAVE(UICONTEXTMENU_LOCATION)
-    if (_allowedImagePickerTypes.containsAny({ WKFileUploadPanelImagePickerType::Image, WKFileUploadPanelImagePickerType::Video })) {
-        [self ensureContextMenuInteraction];
-        [_view presentContextMenu:_documentContextMenuInteraction.get() atLocation:_interactionPoint];
-    } else // Image and Video types are not accepted so bypass the menu and open the file picker directly.
+    if (_allowedImagePickerTypes.containsAny({ WKFileUploadPanelImagePickerType::Image, WKFileUploadPanelImagePickerType::Video }))
+        [_view presentContextMenu:self.ensureContextMenuInteraction atLocation:_interactionPoint];
+    else // Image and Video types are not accepted so bypass the menu and open the file picker directly.
 #endif
         [self showFilePickerMenu];

trunk/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl

@@ -267 +267 @@
     readonly attribute boolean isShowingMenu;
     readonly attribute object menuRect;
+    readonly attribute object contextMenuRect;
     object rectForMenuAction(DOMString action);
     undefined chooseMenuAction(DOMString action, object callback);

trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h

@@ -286 +286 @@
     virtual JSObjectRef rectForMenuAction(JSStringRef) const { notImplemented(); return nullptr; }
     virtual JSObjectRef menuRect() const { notImplemented(); return nullptr; }
+    virtual JSObjectRef contextMenuRect() const { notImplemented(); return nullptr; }
     virtual bool isShowingContextMenu() const { notImplemented(); return false; }
 

trunk/Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm

@@ -262 +262 @@
 - (void)_dismissAllContextMenuInteractions
 {
-#if PLATFORM(IOS)
+#if USE(UICONTEXTMENU)
     for (id <UIInteraction> interaction in self.contentView.interactions) {
-        if ([interaction isKindOfClass:UIContextMenuInteraction.class])
-            [(UIContextMenuInteraction *)interaction dismissMenu];
+        if (auto contextMenuInteraction = dynamic_objc_cast<UIContextMenuInteraction>(interaction)) {
+            [UIView performWithoutAnimation:^{
+                [contextMenuInteraction dismissMenu];
+            }];
+        }
     }
 #endif

trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h

@@ -30 +30 @@
 #import "UIScriptControllerCocoa.h"
 #import <wtf/BlockPtr.h>
+
+typedef struct CGRect CGRect;
 
 namespace WebCore {
@@ -124 +126 @@
     bool isShowingPopover() const override;
     JSObjectRef rectForMenuAction(JSStringRef) const override;
+    JSObjectRef contextMenuRect() const override;
     JSObjectRef menuRect() const override;
     bool isDismissingMenu() const override;
@@ -171 +174 @@
     void singleTapAtPointWithModifiers(WebCore::FloatPoint location, Vector<String>&& modifierFlags, BlockPtr<void()>&&);
 
+    JSObjectRef toObject(CGRect) const;
+
     bool isWebContentFirstResponder() const override;
 };

trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm

@@ -1106 +1106 @@
         return nullptr;
 
-    CGRect rectInRootViewCoordinates = [calloutBar convertRect:calloutBar.bounds toView:platformContentView()];
-    return m_context->objectFromRect(WebCore::FloatRect(rectInRootViewCoordinates.origin.x, rectInRootViewCoordinates.origin.y, rectInRootViewCoordinates.size.width, rectInRootViewCoordinates.size.height));
+    return toObject([calloutBar convertRect:calloutBar.bounds toView:platformContentView()]);
+}
+
+JSObjectRef UIScriptControllerIOS::contextMenuRect() const
+{
+    auto *window = webView().window;
+    auto *contextMenuView = [findAllViewsInHierarchyOfType(window, NSClassFromString(@"_UIContextMenuView")) firstObject];
+    if (!contextMenuView)
+        return nullptr;
+
+    return toObject([contextMenuView convertRect:contextMenuView.bounds toView:nil]);
+}
+
+JSObjectRef UIScriptControllerIOS::toObject(CGRect rect) const
+{
+    return m_context->objectFromRect(WebCore::FloatRect(rect.origin.x, rect.origin.y, rect.size.width, rect.size.height));
 }