Changeset 275546 in webkit

Timestamp:

Apr 6, 2021 1:39:10 PM (3 years ago)

Author:

Wenson Hsieh

Message:

REGRESSION (r274610): Unable to drag images when image extraction is enabled
​https://bugs.webkit.org/show_bug.cgi?id=224211
<rdar://problem/76229563>

Reviewed by Tim Horton.

Source/WebKit:

r274610 introduced a new deferring gesture recognizer intended to prevent several text interaction gestures from
recognizing during pending image extraction. However, this also causes dragging on iOS to fail, since the
gesture used to initiate dragging is excluded by the new deferring gesture recognizer. To fix this, allow the
new deferring gesture to recognize simultaneously alongside all gestures with the exception of only the gestures
it is intended to defer (i.e. text interaction gestures).

Test: fast/events/ios/dragstart-on-image-by-long-pressing.html

• UIProcess/API/ios/WKWebViewPrivateForTestingIOS.h:
• UIProcess/API/ios/WKWebViewTestingIOS.mm:

(-[WKWebView _isAnimatingDragCancel]):

• UIProcess/ios/WKContentViewInteraction.h:
• UIProcess/ios/WKContentViewInteraction.mm:

(-[UIGestureRecognizer _wk_isTapAndAHalf]):

Add a WebKit category method that returns whether or not a gesture recognizer is a tap-and-a-half gesture.

(-[WKContentView gestureRecognizer:shouldRecognizeSimultaneouslyWithGestureRecognizer:]):

Allow the image extraction deferring gesture to recognize alongside all other gestures, with the exception of
the text interaction gestures determined by -shouldDeferGestureDueToImageExtraction:. This limits the impact
of this new deferring gesture, such that it only affects the text interaction gestures it is intended to defer.

(-[WKContentView shouldDeferGestureDueToImageExtraction:]):

Add a helper method to determine whether or not a gesture recognizer should be deferred, due to pending image
extraction. We pull this logic behind a helper method because it's now consulted from two call sites.

(-[WKContentView deferringGestureRecognizer:shouldDeferOtherGestureRecognizer:]):
(-[WKContentView dragInteraction:item:willAnimateCancelWithAnimator:]):
(-[WKContentView isAnimatingDragCancel]):

Add a testing hook to return whether or not the drag cancel animation is running. See Tools/ChangeLog and the
new layout test for more detail.

(tapAndAHalfRecognizerClass): Deleted.

Tools:

Add support for some new testing infrastructure; see below for more details.

• TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
• TestRunnerShared/UIScriptContext/UIScriptController.h:

(WTR::UIScriptController::isAnimatingDragCancel const):

Add a new testing hook to return whether or not the web view's drag interaction is currently animating a drag
cancel (i.e., the drag preview is animating back to its original frame).

• WebKitTestRunner/TestOptions.cpp:

(WTR::TestOptions::defaults):
(WTR::TestOptions::keyTypeMapping):

• WebKitTestRunner/TestOptions.h:

(WTR::TestOptions::dragInteractionPolicy const):

Add a test option that allows tests to override the drag interaction policy to "always-allow",
"always-disallow", and the default value. This option allows us to force drag and drop to be enabled when
testing on iPhone simulator.

• WebKitTestRunner/ios/TestControllerIOS.mm:

(WTR::dragInteractionPolicy):
(WTR::TestController::platformResetStateToConsistentValues):

• WebKitTestRunner/ios/UIScriptControllerIOS.h:
• WebKitTestRunner/ios/UIScriptControllerIOS.mm:

(WTR::UIScriptControllerIOS::isAnimatingDragCancel const):

LayoutTests:

Add a new layout test that initiates dragging on an image using a synthesized event stream, and verifies that
"dragstart" and "dragend" events are dispatched on the image.

• fast/events/ios/dragstart-on-image-by-long-pressing-expected.txt: Added.
• fast/events/ios/dragstart-on-image-by-long-pressing.html: Added.
• resources/ui-helper.js:

(window.UIHelper.isAnimatingDragCancel):

Add a UIHelper method that returns whether or not the dragging animation is being cancelled. The new test uses
this hook to wait for the drag cancel animation to end before proceeding to the next test.

Location:

trunk

Files:

• LayoutTests/ChangeLog (modified) (1 diff)
• LayoutTests/fast/events/ios/dragstart-on-image-by-long-pressing-expected.txt (added)
• LayoutTests/fast/events/ios/dragstart-on-image-by-long-pressing.html (added)
• LayoutTests/resources/ui-helper.js (modified) (1 diff)
• Source/WebKit/ChangeLog (modified) (1 diff)
• Source/WebKit/UIProcess/API/ios/WKWebViewPrivateForTestingIOS.h (modified) (1 diff)
• Source/WebKit/UIProcess/API/ios/WKWebViewTestingIOS.mm (modified) (1 diff)
• Source/WebKit/UIProcess/ios/WKContentViewInteraction.h (modified) (2 diffs)
• Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm (modified) (10 diffs)
• Tools/ChangeLog (modified) (1 diff)
• Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl (modified) (1 diff)
• Tools/TestRunnerShared/UIScriptContext/UIScriptController.h (modified) (1 diff)
• Tools/WebKitTestRunner/TestOptions.cpp (modified) (2 diffs)
• Tools/WebKitTestRunner/TestOptions.h (modified) (1 diff)
• Tools/WebKitTestRunner/ios/TestControllerIOS.mm (modified) (2 diffs)
• Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h (modified) (1 diff)
• Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm (modified) (1 diff)

trunk/LayoutTests/ChangeLog

@@ +1 @@
+2021-04-06  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        REGRESSION (r274610): Unable to drag images when image extraction is enabled
+        https://bugs.webkit.org/show_bug.cgi?id=224211
+        <rdar://problem/76229563>
+
+        Reviewed by Tim Horton.
+
+        Add a new layout test that initiates dragging on an image using a synthesized event stream, and verifies that
+        "dragstart" and "dragend" events are dispatched on the image.
+
+        * fast/events/ios/dragstart-on-image-by-long-pressing-expected.txt: Added.
+        * fast/events/ios/dragstart-on-image-by-long-pressing.html: Added.
+        * resources/ui-helper.js:
+        (window.UIHelper.isAnimatingDragCancel):
+
+        Add a `UIHelper` method that returns whether or not the dragging animation is being cancelled. The new test uses
+        this hook to wait for the drag cancel animation to end before proceeding to the next test.
+
 2021-04-06  Ryosuke Niwa  <rniwa@webkit.org>
 

trunk/LayoutTests/resources/ui-helper.js

@@ -368 +368 @@
         return new Promise(resolve => {
             testRunner.runUIScript(`uiController.keyboardIsAutomaticallyShifted`, result => resolve(result === "true"));
+        });
+    }
+
+    static isAnimatingDragCancel()
+    {
+        return new Promise(resolve => {
+            testRunner.runUIScript(`uiController.isAnimatingDragCancel`, result => resolve(result === "true"));
         });
     }

trunk/Source/WebKit/ChangeLog

@@ +1 @@
+2021-04-06  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        REGRESSION (r274610): Unable to drag images when image extraction is enabled
+        https://bugs.webkit.org/show_bug.cgi?id=224211
+        <rdar://problem/76229563>
+
+        Reviewed by Tim Horton.
+
+        r274610 introduced a new deferring gesture recognizer intended to prevent several text interaction gestures from
+        recognizing during pending image extraction. However, this also causes dragging on iOS to fail, since the
+        gesture used to initiate dragging is excluded by the new deferring gesture recognizer. To fix this, allow the
+        new deferring gesture to recognize simultaneously alongside all gestures with the exception of only the gestures
+        it is intended to defer (i.e. text interaction gestures).
+
+        Test: fast/events/ios/dragstart-on-image-by-long-pressing.html
+
+        * UIProcess/API/ios/WKWebViewPrivateForTestingIOS.h:
+        * UIProcess/API/ios/WKWebViewTestingIOS.mm:
+        (-[WKWebView _isAnimatingDragCancel]):
+        * UIProcess/ios/WKContentViewInteraction.h:
+        * UIProcess/ios/WKContentViewInteraction.mm:
+        (-[UIGestureRecognizer _wk_isTapAndAHalf]):
+
+        Add a WebKit category method that returns whether or not a gesture recognizer is a tap-and-a-half gesture.
+
+        (-[WKContentView gestureRecognizer:shouldRecognizeSimultaneouslyWithGestureRecognizer:]):
+
+        Allow the image extraction deferring gesture to recognize alongside all other gestures, with the exception of
+        the text interaction gestures determined by `-shouldDeferGestureDueToImageExtraction:`. This limits the impact
+        of this new deferring gesture, such that it only affects the text interaction gestures it is intended to defer.
+
+        (-[WKContentView shouldDeferGestureDueToImageExtraction:]):
+
+        Add a helper method to determine whether or not a gesture recognizer should be deferred, due to pending image
+        extraction. We pull this logic behind a helper method because it's now consulted from two call sites.
+
+        (-[WKContentView deferringGestureRecognizer:shouldDeferOtherGestureRecognizer:]):
+        (-[WKContentView dragInteraction:item:willAnimateCancelWithAnimator:]):
+        (-[WKContentView isAnimatingDragCancel]):
+
+        Add a testing hook to return whether or not the drag cancel animation is running. See Tools/ChangeLog and the
+        new layout test for more detail.
+
+        (tapAndAHalfRecognizerClass): Deleted.
+
 2021-04-06  Alex Christensen  <achristensen@webkit.org>
 

trunk/Source/WebKit/UIProcess/API/ios/WKWebViewPrivateForTestingIOS.h

@@ -43 +43 @@
 @property (nonatomic, readonly) NSNumber *_stableStateOverride;
 @property (nonatomic, readonly) CGRect _dragCaretRect;
+@property (nonatomic, readonly, getter=_isAnimatingDragCancel) BOOL _animatingDragCancel;
 
 - (void)keyboardAccessoryBarNext;

trunk/Source/WebKit/UIProcess/API/ios/WKWebViewTestingIOS.mm

@@ -314 +314 @@
 }
 
+- (BOOL)_isAnimatingDragCancel
+{
+#if ENABLE(DRAG_SUPPORT)
+    return [_contentView isAnimatingDragCancel];
+#else
+    return NO;
+#endif
+}
+
 - (void)_simulateElementAction:(_WKElementActionType)actionType atLocation:(CGPoint)location
 {

trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h

@@ -439 +439 @@
     RetainPtr<UIDragInteraction> _dragInteraction;
     RetainPtr<UIDropInteraction> _dropInteraction;
+    BOOL _isAnimatingDragCancel;
     BOOL _shouldRestoreCalloutBarAfterDrop;
     RetainPtr<UIView> _visibleContentViewSnapshot;
@@ -736 +737 @@
 @property (nonatomic, readonly) WKDateTimeInputControl *dateTimeInputControl;
 @property (nonatomic, readonly) WKFormSelectControl *selectControl;
+#if ENABLE(DRAG_SUPPORT)
+@property (nonatomic, readonly, getter=isAnimatingDragCancel) BOOL animatingDragCancel;
+#endif
 
 @end

trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm

@@ -682 +682 @@
 }
 
+- (BOOL)_wk_isTapAndAHalf
+{
+    static dispatch_once_t onceToken;
+    static Class tapAndAHalfGestureRecognizerClass;
+    dispatch_once(&onceToken, ^{
+        tapAndAHalfGestureRecognizerClass = NSClassFromString(@"UITapAndAHalfRecognizer");
+    });
+    return [self isKindOfClass:tapAndAHalfGestureRecognizerClass];
+}
+
 @end
 
@@ -2227 +2237 @@
 }
 
-static Class tapAndAHalfRecognizerClass()
-{
-    static dispatch_once_t onceToken;
-    static Class theClass;
-    dispatch_once(&onceToken, ^{
-        theClass = NSClassFromString(@"UITapAndAHalfRecognizer");
-    });
-    return theClass;
-}
-
 - (BOOL)gestureRecognizer:(UIGestureRecognizer *)gestureRecognizer shouldRecognizeSimultaneouslyWithGestureRecognizer:(UIGestureRecognizer*)otherGestureRecognizer
 {
@@ -2246 +2246 @@
     if ([gestureRecognizer isKindOfClass:WKDeferringGestureRecognizer.class] && [otherGestureRecognizer isKindOfClass:WKDeferringGestureRecognizer.class])
         return YES;
+
+#if ENABLE(IMAGE_EXTRACTION)
+    if (gestureRecognizer == _imageExtractionDeferringGestureRecognizer)
+        return ![self shouldDeferGestureDueToImageExtraction:otherGestureRecognizer];
+
+    if (otherGestureRecognizer == _imageExtractionDeferringGestureRecognizer)
+        return ![self shouldDeferGestureDueToImageExtraction:gestureRecognizer];
+#endif
 
     if (isSamePair(gestureRecognizer, otherGestureRecognizer, _highlightLongPressGestureRecognizer.get(), _longPressGestureRecognizer.get()))
@@ -2275 +2283 @@
             return YES;
 
-        if ([gestureRecognizer isKindOfClass:tapAndAHalfRecognizerClass()] || [otherGestureRecognizer isKindOfClass:tapAndAHalfRecognizerClass()])
+        if (gestureRecognizer._wk_isTapAndAHalf || otherGestureRecognizer._wk_isTapAndAHalf)
             return YES;
     }
@@ -7514 +7522 @@
 }
 
+#if ENABLE(IMAGE_EXTRACTION)
+
+- (BOOL)shouldDeferGestureDueToImageExtraction:(UIGestureRecognizer *)gesture
+{
+    return gesture == [_textInteractionAssistant loupeGesture] || gesture._wk_isTapAndAHalf || gesture == [_textInteractionAssistant forcePressGesture];
+}
+
+#endif // ENABLE(IMAGE_EXTRACTION)
+
 #if HAVE(PASTEBOARD_DATA_OWNER)
 
@@ -7660 +7677 @@
         return NO;
 
-    BOOL isLoupeGesture = gestureRecognizer == [_textInteractionAssistant loupeGesture];
-    BOOL isTapAndAHalfGesture = [gestureRecognizer isKindOfClass:tapAndAHalfRecognizerClass()];
-
 #if ENABLE(IMAGE_EXTRACTION)
     if (deferringGestureRecognizer == _imageExtractionDeferringGestureRecognizer)
-        return isLoupeGesture || isTapAndAHalfGesture || gestureRecognizer == [_textInteractionAssistant forcePressGesture];
+        return [self shouldDeferGestureDueToImageExtraction:gestureRecognizer];
 #endif
 
@@ -7679 +7693 @@
 #endif
 
-        if (isTapAndAHalfGesture)
+        if (gestureRecognizer._wk_isTapAndAHalf)
             return YES;
 
-        if (isLoupeGesture)
+        if (gestureRecognizer == [_textInteractionAssistant loupeGesture])
             return YES;
 
@@ -8686 +8700 @@
 - (void)dragInteraction:(UIDragInteraction *)interaction item:(UIDragItem *)item willAnimateCancelWithAnimator:(id <UIDragAnimating>)animator
 {
+    _isAnimatingDragCancel = YES;
     RELEASE_LOG(DragAndDrop, "Drag interaction willAnimateCancelWithAnimator");
     [animator addCompletion:[protectedSelf = retainPtr(self), page = _page] (UIViewAnimatingPosition finalPosition) {
@@ -8691 +8706 @@
         page->dragCancelled();
         if (auto completion = protectedSelf->_dragDropInteractionState.takeDragCancelSetDownBlock()) {
-            page->callAfterNextPresentationUpdate([completion] (WebKit::CallbackBase::Error) {
+            page->callAfterNextPresentationUpdate([completion, protectedSelf] (WebKit::CallbackBase::Error) {
                 completion();
+                protectedSelf->_isAnimatingDragCancel = NO;
             });
         }
@@ -9543 +9559 @@
     return nil;
 }
+
+#if ENABLE(DRAG_SUPPORT)
+
+- (BOOL)isAnimatingDragCancel
+{
+    return _isAnimatingDragCancel;
+}
+
+#endif // ENABLE(DRAG_SUPPORT)
 
 - (void)_simulateTextEntered:(NSString *)text

trunk/Tools/ChangeLog

@@ +1 @@
+2021-04-06  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        REGRESSION (r274610): Unable to drag images when image extraction is enabled
+        https://bugs.webkit.org/show_bug.cgi?id=224211
+        <rdar://problem/76229563>
+
+        Reviewed by Tim Horton.
+
+        Add support for some new testing infrastructure; see below for more details.
+
+        * TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
+        * TestRunnerShared/UIScriptContext/UIScriptController.h:
+        (WTR::UIScriptController::isAnimatingDragCancel const):
+
+        Add a new testing hook to return whether or not the web view's drag interaction is currently animating a drag
+        cancel (i.e., the drag preview is animating back to its original frame).
+
+        * WebKitTestRunner/TestOptions.cpp:
+        (WTR::TestOptions::defaults):
+        (WTR::TestOptions::keyTypeMapping):
+        * WebKitTestRunner/TestOptions.h:
+        (WTR::TestOptions::dragInteractionPolicy const):
+
+        Add a test option that allows tests to override the drag interaction policy to "always-allow",
+        "always-disallow", and the default value. This option allows us to force drag and drop to be enabled when
+        testing on iPhone simulator.
+
+        * WebKitTestRunner/ios/TestControllerIOS.mm:
+        (WTR::dragInteractionPolicy):
+        (WTR::TestController::platformResetStateToConsistentValues):
+        * WebKitTestRunner/ios/UIScriptControllerIOS.h:
+        * WebKitTestRunner/ios/UIScriptControllerIOS.mm:
+        (WTR::UIScriptControllerIOS::isAnimatingDragCancel const):
+
 2021-04-06  Yusuke Suzuki  <ysuzuki@apple.com>
 

trunk/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl

@@ -90 +90 @@
     undefined rawKeyDown(DOMString key);
     undefined rawKeyUp(DOMString key);
+
+    readonly attribute boolean isAnimatingDragCancel;
 
     // eventsJSON describes a series of user events in JSON form. For the keys, see HIDEventGenerator.mm.

trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h

@@ -177 +177 @@
     }
 
+    virtual bool isAnimatingDragCancel() const
+    {
+        notImplemented();
+        return false;
+    }
+
     virtual void rawKeyDown(JSStringRef) { notImplemented(); }
     virtual void rawKeyUp(JSStringRef) { notImplemented(); }

trunk/Tools/WebKitTestRunner/TestOptions.cpp

@@ -114 +114 @@
             { "applicationManifest", { } },
             { "contentMode", { } },
+            { "dragInteractionPolicy", { } },
             { "jscOptions", { } },
             { "standaloneWebApplicationURL", { } },
@@ -162 +163 @@
         { "applicationManifest", TestHeaderKeyType::StringRelativePathTestRunner },
         { "contentMode", TestHeaderKeyType::StringTestRunner },
+        { "dragInteractionPolicy", TestHeaderKeyType::StringTestRunner },
         { "jscOptions", TestHeaderKeyType::StringTestRunner },
         { "standaloneWebApplicationURL", TestHeaderKeyType::StringURLTestRunner },

trunk/Tools/WebKitTestRunner/TestOptions.h

@@ -79 +79 @@
     std::string applicationManifest() const { return stringTestRunnerFeatureValue("applicationManifest"); }
     std::string contentMode() const { return stringTestRunnerFeatureValue("contentMode"); }
+    std::string dragInteractionPolicy() const { return stringTestRunnerFeatureValue("dragInteractionPolicy"); }
     std::string jscOptions() const { return stringTestRunnerFeatureValue("jscOptions"); }
     std::string standaloneWebApplicationURL() const { return stringTestRunnerFeatureValue("standaloneWebApplicationURL"); }

trunk/Tools/WebKitTestRunner/ios/TestControllerIOS.mm

@@ -155 +155 @@
 }
 
+static _WKDragInteractionPolicy dragInteractionPolicy(const TestOptions& options)
+{
+    auto policy = options.dragInteractionPolicy();
+    if (policy == "always-enable")
+        return _WKDragInteractionPolicyAlwaysEnable;
+    if (policy == "always-disable")
+        return _WKDragInteractionPolicyAlwaysDisable;
+    return _WKDragInteractionPolicyDefault;
+}
+
 bool TestController::platformResetStateToConsistentValues(const TestOptions& options)
 {
@@ -213 +223 @@
         [webView resetCustomMenuAction];
         [webView setAllowedMenuActions:nil];
+        webView._dragInteractionPolicy = dragInteractionPolicy(options);
 
         UIScrollView *scrollView = webView.scrollView;

trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h

@@ -135 +135 @@
     void toggleCapsLock(JSValueRef) override;
     bool keyboardIsAutomaticallyShifted() const override;
+    bool isAnimatingDragCancel() const override;
     JSObjectRef attachmentInfo(JSStringRef) override;
     UIView *platformContentView() const override;

trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm

@@ -1171 +1171 @@
 }
 
+bool UIScriptControllerIOS::isAnimatingDragCancel() const
+{
+    return webView()._animatingDragCancel;
+}
+
 JSObjectRef UIScriptControllerIOS::attachmentInfo(JSStringRef jsAttachmentIdentifier)
 {