Changeset 245778 in webkit


Ignore:
Timestamp:
May 26, 2019 4:31:49 AM (5 years ago)
Author:
Wenson Hsieh
Message:

[iOS] Dropped text, attachments, and images should animate into place
​https://bugs.webkit.org/show_bug.cgi?id=198243
<rdar://problem/35205373>

Reviewed by Tim Horton.

Source/WebCore:

Add some hooks to notify the chrome client when an HTMLImageElement's image is finished loading. See WebKit
changelog for more detail.

Test: DragAndDropTests.DropPreviewForImageInEditableArea

  • loader/EmptyClients.h:
  • page/ChromeClient.h:
  • page/Page.cpp:

(WebCore::Page::didFinishLoadingImageForElement):

  • page/Page.h:
  • rendering/RenderImage.cpp:

(WebCore::RenderImage::notifyFinished):

Source/WebKit:

Adds support for targeted drop animations on iOS in modern WebKit by adopting UIKit SPI introduced in
<rdar://problem/31075005> to allow updating the drop preview mid-flight. To get the animation right, we refactor
and augment existing logic for taking snapshots after performing a drop in an editable content.

Currently, upon dropping in editable content, we first snapshot the web view and temporarily cover the real web
view with this snapshot. When the TextIndicator data arrives that contains (1) a snapshot of the visible part of
the web view ignoring the selection, and (2) a snapshot of just the selected contents after drop, we crossfade
from the web view snapshot to the snapshot in (1) using a hard-coded time delay (~500ms), and target the drop
preview to the drag caret rect. During this process, snapshot (2) is completely ignored.

This was effectively a halfway implemention of the desired effect of animating the dropped content into place
and crossfading to the final content; before UIKit implemented updateable drag previews, the full implementation
was not possible in modern WebKit (without using synchronous IPC).

Now that we're able to update the drag preview in the middle of the drop animation, we can now utilize snapshot
(2) above and clean up some parts of the drop animation in editable content. See below for more details.

  • UIProcess/API/Cocoa/WKWebView.mm:

(-[WKWebView _doAfterReceivingEditDragSnapshotForTesting:]):

Add a testing hook to perform the given block after any pending edit drag snapshot has been received. See
TestWebKitAPI changes for more detail.

  • UIProcess/API/Cocoa/WKWebViewPrivate.h:
  • UIProcess/PageClient.h:
  • UIProcess/WebPageProxy.h:
  • UIProcess/WebPageProxy.messages.in:

Split up the existing DidConcludeEditDrag IPC message into two messages: WillReceiveEditDragSnapshot and
DidReceiveEditDragSnapshot. This allows us to defer cleaning up the drag session state during an edit drop,
until after the final edit drag snapshot has been received.

  • UIProcess/ios/DragDropInteractionState.h:

Add some new methods to help manage the lifecycle of drop preview provider blocks.

  • UIProcess/ios/DragDropInteractionState.mm:

(WebKit::createTargetedDragPreview):

Drive-by fix: make this return a RetainPtr.

(WebKit::DragDropInteractionState::prepareForDelayedDropPreview):

Stores a drop preview provider, given to us by UIKit.

(WebKit::DragDropInteractionState::deliverDelayedDropPreview):

Invokes the stored drop preview providers with given text indicator data. This is invoked after snapshots are
taken following an edit drag (this is additionally after all images in the inserted fragment have finished
loading).

(WebKit::DragDropInteractionState::clearAllDelayedItemPreviewProviders):

Invokes all stored drop preview providers with a nil preview. This is invoked in any case where drag session
cleanup occurs earlier than normal (e.g., if the web process crashes during drop), and ensures that the handlers
are always invoked when cleaning up the drag session.

(WebKit::DragDropInteractionState::previewForDragItem const):
(WebKit::DragDropInteractionState::dragAndDropSessionsDidEnd):

Call clearAllDelayedItemPreviewProviders.

  • UIProcess/ios/PageClientImplIOS.h:
  • UIProcess/ios/PageClientImplIOS.mm:

(WebKit::PageClientImpl::willReceiveEditDragSnapshot):
(WebKit::PageClientImpl::didReceiveEditDragSnapshot):
(WebKit::PageClientImpl::didConcludeEditDrag): Deleted.

More plumbing (see changes to DidConcludeEditDrag above).

  • UIProcess/ios/WKContentViewInteraction.h:
  • UIProcess/ios/WKContentViewInteraction.mm:

(-[WKContentView cleanupInteraction]):
(-[WKContentView cleanUpDragSourceSessionState]):
(-[WKContentView _willReceiveEditDragSnapshot]):
(-[WKContentView _didReceiveEditDragSnapshot:]):

Set _waitingForEditDragSnapshot to YES in the gap between when -_willReceiveEditDragSnapshot is invoked, and
when -_didReceiveEditDragSnapshot is invoked. If _waitingForEditDragSnapshot is YES, we bail out of
-cleanUpDragSourceSessionState, and instead clean up drag session state after the edit drag snapshot is
received.

(-[WKContentView _deliverDelayedDropPreviewIfPossible:]):
(-[WKContentView _didPerformDragOperation:]):
(-[WKContentView textEffectsWindow]):

Drive-by fix to remove a workaround for a deprecation warning.

(-[WKContentView dropInteraction:item:willAnimateDropWithAnimator:]):
(-[WKContentView dropInteraction:concludeDrop:]):

Implement this hook to ensure that the unselected content snapshot view and visible content snapshot view are
guaranteed to be removed from the view after a drop in editable content, even if the drag edit snapshot arrives
after the drop is concluded.

(-[WKContentView dropInteraction:previewForDroppingItem:withDefault:]):
(-[WKContentView _dropInteraction:delayedPreviewProviderForDroppingItem:previewProvider:]):

Implement the new UIKit SPI here. UIKit hands us a preview provider here, which we can invoke at a later time
to update the drop preview. We do this in _didReceiveEditDragSnapshot.

(-[WKContentView _doAfterReceivingEditDragSnapshotForTesting:]):
(-[WKContentView _didConcludeEditDrag:]): Deleted.

  • UIProcess/ios/WebPageProxyIOS.mm:

(WebKit::WebPageProxy::willReceiveEditDragSnapshot):
(WebKit::WebPageProxy::didReceiveEditDragSnapshot):
(WebKit::WebPageProxy::didConcludeDrop):
(WebKit::WebPageProxy::didConcludeEditDrag): Deleted.

  • WebProcess/WebCoreSupport/WebChromeClient.cpp:

(WebKit::WebChromeClient::didFinishLoadingImageForElement):

  • WebProcess/WebCoreSupport/WebChromeClient.h:
  • WebProcess/WebPage/WebPage.cpp:

(WebKit::WebPage::didFinishLoadingImageForElement):

  • WebProcess/WebPage/WebPage.h:
  • WebProcess/WebPage/WebPage.messages.in:
  • WebProcess/WebPage/ios/WebPageIOS.mm:

(WebKit::WebPage::didConcludeDrop):

If an edit drag has been concluded, there's no need to hang on to pending dropped image elements anymore; clear
out the set here.

(WebKit::WebPage::didConcludeEditDrag):

After concluding an edit drag, we try to deliver two web content snapshots to the UI process, so that the UI
process can assemble a targeted drop preview for UIKit. One snapshot is of the visible content area, not
including any selected content. The other snapshot is of the selected content only. However, when dropping
images (or a text selection containing images), these images may not yet have been loaded. If that is the case,
these images will appear to be missing from these snapshots.

To ensure that we don't take this snapshot too early, defer it until all image elements in the dropped content
range have finished loading. We can tell that all dropped images have finished loading by using a new client
hook that is invoked when an image has finished loading.

(WebKit::WebPage::didFinishLoadingImageForElement):
(WebKit::WebPage::computeAndSendEditDragSnapshot):

Snapshot the selected content and send it to the UI process.

Source/WebKitLegacy/mac:

Add a new chrome client method. See other changelogs for more detail.

  • WebCoreSupport/WebChromeClient.h:
  • WebCoreSupport/WebChromeClient.mm:

(WebChromeClient::didFinishLoadingImageForElement):

Source/WebKitLegacy/win:

  • WebCoreSupport/WebChromeClient.cpp:

(WebChromeClient::didFinishLoadingImageForElement):

  • WebCoreSupport/WebChromeClient.h:

Tools:

Adjusts the iOS dragging simulator, and adds a new API test. See below for more detail.

  • TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm:

(TestWebKitAPI::isCompletelyWhite):
(TestWebKitAPI::TEST):

Add a test that drags and drops an image into a contenteditable element, and then observes the resulting
UITargetedDragPreviews upon drop.

  • TestWebKitAPI/cocoa/DragAndDropSimulator.h:
  • TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm:

(-[DragAndDropSimulator _resetSimulatedState]):
(-[DragAndDropSimulator runFrom:to:additionalItemRequestLocations:]):
(-[DragAndDropSimulator _concludeDropAndPerformOperationIfNecessary]):

Teach the iOS version of DragAndDropSimulator to invoke -dropInteraction:concludeDrop: on the drop interaction
delegate when the drop finishes. This additionally uses _doAfterReceivingEditDragSnapshotForTesting: to defer
the end of the simulated drag and drop until after drag previews have been received during an edit drag.

(-[DragAndDropSimulator dropPreviews]):
(-[DragAndDropSimulator delayedDropPreviews]):

Have the drag and drop simulator remember which previews were returned by the delegate on drop, as well as which
previews were provided asynchronously.

(-[DragAndDropSimulator _webView:dataInteractionOperationWasHandled:forSession:itemProviders:]):

  • TestWebKitAPI/ios/UIKitSPI.h:

Stage the new private drop interacton delegate method.

Location:
trunk
Files:
36 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r245776 r245778  
     12019-05-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Dropped text, attachments, and images should animate into place
     4        https://bugs.webkit.org/show_bug.cgi?id=198243
     5        <rdar://problem/35205373>
     6
     7        Reviewed by Tim Horton.
     8
     9        Add some hooks to notify the chrome client when an HTMLImageElement's image is finished loading. See WebKit
     10        changelog for more detail.
     11
     12        Test: DragAndDropTests.DropPreviewForImageInEditableArea
     13
     14        * loader/EmptyClients.h:
     15        * page/ChromeClient.h:
     16        * page/Page.cpp:
     17        (WebCore::Page::didFinishLoadingImageForElement):
     18        * page/Page.h:
     19        * rendering/RenderImage.cpp:
     20        (WebCore::RenderImage::notifyFinished):
     21
    1222019-05-25  Zalan Bujtas  <zalan@apple.com>
    223

  • trunk/Source/WebCore/loader/EmptyClients.h

    r245174 r245778  
    4141class DiagnosticLoggingClient;
    4242class EditorClient;
     43class HTMLImageElement;
    4344class PageConfiguration;
    4445
    … …  
    111112    IntPoint accessibilityScreenToRootView(const IntPoint& p) const final { return p; };
    112113    IntRect rootViewToAccessibilityScreen(const IntRect& r) const final { return r; };
     114
     115    void didFinishLoadingImageForElement(HTMLImageElement&) final { }
    113116
    114117    PlatformPageClient platformPageClient() const final { return 0; }

  • trunk/Source/WebCore/page/ChromeClient.h

    r245366 r245778  
    8888class GraphicsLayer;
    8989class GraphicsLayerFactory;
     90class HTMLImageElement;
    9091class HTMLInputElement;
    9192class HTMLMediaElement;
    … …  
    183184    virtual IntRect rootViewToAccessibilityScreen(const IntRect&) const = 0;
    184185
     186    virtual void didFinishLoadingImageForElement(HTMLImageElement&) = 0;
     187
    185188    virtual PlatformPageClient platformPageClient() const = 0;
    186189

  • trunk/Source/WebCore/page/Page.cpp

    r245543 r245778  
    29972997}
    29982998
     2999void Page::didFinishLoadingImageForElement(HTMLImageElement& element)
     3000{
     3001    chrome().client().didFinishLoadingImageForElement(element);
     3002}
     3003
    29993004} // namespace WebCore

  • trunk/Source/WebCore/page/Page.h

    r245072 r245778  
    528528    WEBCORE_EXPORT void resumeAnimatingImages();
    529529
     530    void didFinishLoadingImageForElement(HTMLImageElement&);
     531
    530532    WEBCORE_EXPORT void addLayoutMilestones(OptionSet<LayoutMilestone>);
    531533    WEBCORE_EXPORT void removeLayoutMilestones(OptionSet<LayoutMilestone>);

  • trunk/Source/WebCore/rendering/RenderImage.cpp

    r245543 r245778  
    379379        contentChanged(ImageChanged);
    380380    }
     381
     382    if (is<HTMLImageElement>(element()))
     383        page().didFinishLoadingImageForElement(downcast<HTMLImageElement>(*element()));
    381384}
    382385

  • trunk/Source/WebKit/ChangeLog

    r245767 r245778  
     12019-05-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Dropped text, attachments, and images should animate into place
     4        https://bugs.webkit.org/show_bug.cgi?id=198243
     5        <rdar://problem/35205373>
     6
     7        Reviewed by Tim Horton.
     8
     9        Adds support for targeted drop animations on iOS in modern WebKit by adopting UIKit SPI introduced in
     10        <rdar://problem/31075005> to allow updating the drop preview mid-flight. To get the animation right, we refactor
     11        and augment existing logic for taking snapshots after performing a drop in an editable content.
     12
     13        Currently, upon dropping in editable content, we first snapshot the web view and temporarily cover the real web
     14        view with this snapshot. When the TextIndicator data arrives that contains (1) a snapshot of the visible part of
     15        the web view ignoring the selection, and (2) a snapshot of just the selected contents after drop, we crossfade
     16        from the web view snapshot to the snapshot in (1) using a hard-coded time delay (~500ms), and target the drop
     17        preview to the drag caret rect. During this process, snapshot (2) is completely ignored.
     18
     19        This was effectively a halfway implemention of the desired effect of animating the dropped content into place
     20        and crossfading to the final content; before UIKit implemented updateable drag previews, the full implementation
     21        was not possible in modern WebKit (without using synchronous IPC).
     22
     23        Now that we're able to update the drag preview in the middle of the drop animation, we can now utilize snapshot
     24        (2) above and clean up some parts of the drop animation in editable content. See below for more details.
     25
     26        * UIProcess/API/Cocoa/WKWebView.mm:
     27        (-[WKWebView _doAfterReceivingEditDragSnapshotForTesting:]):
     28
     29        Add a testing hook to perform the given block after any pending edit drag snapshot has been received. See
     30        TestWebKitAPI changes for more detail.
     31
     32        * UIProcess/API/Cocoa/WKWebViewPrivate.h:
     33        * UIProcess/PageClient.h:
     34        * UIProcess/WebPageProxy.h:
     35        * UIProcess/WebPageProxy.messages.in:
     36
     37        Split up the existing DidConcludeEditDrag IPC message into two messages: WillReceiveEditDragSnapshot and
     38        DidReceiveEditDragSnapshot. This allows us to defer cleaning up the drag session state during an edit drop,
     39        until after the final edit drag snapshot has been received.
     40
     41        * UIProcess/ios/DragDropInteractionState.h:
     42
     43        Add some new methods to help manage the lifecycle of drop preview provider blocks.
     44
     45        * UIProcess/ios/DragDropInteractionState.mm:
     46        (WebKit::createTargetedDragPreview):
     47
     48        Drive-by fix: make this return a RetainPtr.
     49
     50        (WebKit::DragDropInteractionState::prepareForDelayedDropPreview):
     51
     52        Stores a drop preview provider, given to us by UIKit.
     53
     54        (WebKit::DragDropInteractionState::deliverDelayedDropPreview):
     55
     56        Invokes the stored drop preview providers with given text indicator data. This is invoked after snapshots are
     57        taken following an edit drag (this is additionally after all images in the inserted fragment have finished
     58        loading).
     59
     60        (WebKit::DragDropInteractionState::clearAllDelayedItemPreviewProviders):
     61
     62        Invokes all stored drop preview providers with a nil preview. This is invoked in any case where drag session
     63        cleanup occurs earlier than normal (e.g., if the web process crashes during drop), and ensures that the handlers
     64        are always invoked when cleaning up the drag session.
     65
     66        (WebKit::DragDropInteractionState::previewForDragItem const):
     67        (WebKit::DragDropInteractionState::dragAndDropSessionsDidEnd):
     68
     69        Call clearAllDelayedItemPreviewProviders.
     70
     71        * UIProcess/ios/PageClientImplIOS.h:
     72        * UIProcess/ios/PageClientImplIOS.mm:
     73        (WebKit::PageClientImpl::willReceiveEditDragSnapshot):
     74        (WebKit::PageClientImpl::didReceiveEditDragSnapshot):
     75        (WebKit::PageClientImpl::didConcludeEditDrag): Deleted.
     76
     77        More plumbing (see changes to DidConcludeEditDrag above).
     78
     79        * UIProcess/ios/WKContentViewInteraction.h:
     80        * UIProcess/ios/WKContentViewInteraction.mm:
     81        (-[WKContentView cleanupInteraction]):
     82        (-[WKContentView cleanUpDragSourceSessionState]):
     83        (-[WKContentView _willReceiveEditDragSnapshot]):
     84        (-[WKContentView _didReceiveEditDragSnapshot:]):
     85
     86        Set _waitingForEditDragSnapshot to YES in the gap between when -_willReceiveEditDragSnapshot is invoked, and
     87        when -_didReceiveEditDragSnapshot is invoked. If _waitingForEditDragSnapshot is YES, we bail out of
     88        -cleanUpDragSourceSessionState, and instead clean up drag session state after the edit drag snapshot is
     89        received.
     90
     91        (-[WKContentView _deliverDelayedDropPreviewIfPossible:]):
     92        (-[WKContentView _didPerformDragOperation:]):
     93        (-[WKContentView textEffectsWindow]):
     94
     95        Drive-by fix to remove a workaround for a deprecation warning.
     96
     97        (-[WKContentView dropInteraction:item:willAnimateDropWithAnimator:]):
     98        (-[WKContentView dropInteraction:concludeDrop:]):
     99
     100        Implement this hook to ensure that the unselected content snapshot view and visible content snapshot view are
     101        guaranteed to be removed from the view after a drop in editable content, even if the drag edit snapshot arrives
     102        after the drop is concluded.
     103
     104        (-[WKContentView dropInteraction:previewForDroppingItem:withDefault:]):
     105        (-[WKContentView _dropInteraction:delayedPreviewProviderForDroppingItem:previewProvider:]):
     106
     107        Implement the new UIKit SPI here. UIKit hands us a preview provider here, which we can invoke at a later time
     108        to update the drop preview. We do this in _didReceiveEditDragSnapshot.
     109
     110        (-[WKContentView _doAfterReceivingEditDragSnapshotForTesting:]):
     111        (-[WKContentView _didConcludeEditDrag:]): Deleted.
     112        * UIProcess/ios/WebPageProxyIOS.mm:
     113        (WebKit::WebPageProxy::willReceiveEditDragSnapshot):
     114        (WebKit::WebPageProxy::didReceiveEditDragSnapshot):
     115        (WebKit::WebPageProxy::didConcludeDrop):
     116        (WebKit::WebPageProxy::didConcludeEditDrag): Deleted.
     117        * WebProcess/WebCoreSupport/WebChromeClient.cpp:
     118        (WebKit::WebChromeClient::didFinishLoadingImageForElement):
     119        * WebProcess/WebCoreSupport/WebChromeClient.h:
     120        * WebProcess/WebPage/WebPage.cpp:
     121        (WebKit::WebPage::didFinishLoadingImageForElement):
     122        * WebProcess/WebPage/WebPage.h:
     123        * WebProcess/WebPage/WebPage.messages.in:
     124        * WebProcess/WebPage/ios/WebPageIOS.mm:
     125        (WebKit::WebPage::didConcludeDrop):
     126
     127        If an edit drag has been concluded, there's no need to hang on to pending dropped image elements anymore; clear
     128        out the set here.
     129
     130        (WebKit::WebPage::didConcludeEditDrag):
     131
     132        After concluding an edit drag, we try to deliver two web content snapshots to the UI process, so that the UI
     133        process can assemble a targeted drop preview for UIKit. One snapshot is of the visible content area, not
     134        including any selected content. The other snapshot is of the selected content only. However, when dropping
     135        images (or a text selection containing images), these images may not yet have been loaded. If that is the case,
     136        these images will appear to be missing from these snapshots.
     137
     138        To ensure that we don't take this snapshot too early, defer it until all image elements in the dropped content
     139        range have finished loading. We can tell that all dropped images have finished loading by using a new client
     140        hook that is invoked when an image has finished loading.
     141
     142        (WebKit::WebPage::didFinishLoadingImageForElement):
     143        (WebKit::WebPage::computeAndSendEditDragSnapshot):
     144
     145        Snapshot the selected content and send it to the UI process.
     146
    11472019-05-24  Youenn Fablet  <youenn@apple.com>
    2148

  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm

    r245006 r245778  
    68906890}
    68916891
     6892- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action
     6893{
     6894    [_contentView _doAfterReceivingEditDragSnapshotForTesting:action];
     6895}
     6896
    68926897#endif // PLATFORM(IOS_FAMILY)
    68936898

  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h

    r245294 r245778  
    489489
    490490@property (nonatomic, readonly) CGRect _dragCaretRect WK_API_AVAILABLE(ios(11.0));
     491
     492- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action WK_API_AVAILABLE(ios(WK_IOS_TBA));
    491493
    492494- (void)_requestActivatedElementAtPosition:(CGPoint)position completionBlock:(void (^)(_WKActivatedElementInfo *))block WK_API_AVAILABLE(ios(11.0));

  • trunk/Source/WebKit/UIProcess/PageClient.h

    r245112 r245778  
    480480    virtual void didHandleDragStartRequest(bool started) = 0;
    481481    virtual void didHandleAdditionalDragItemsRequest(bool added) = 0;
    482     virtual void didConcludeEditDrag(Optional<WebCore::TextIndicatorData>) = 0;
     482    virtual void willReceiveEditDragSnapshot() = 0;
     483    virtual void didReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData>) = 0;
    483484    virtual void didChangeDragCaretRect(const WebCore::IntRect& previousCaretRect, const WebCore::IntRect& caretRect) = 0;
    484485#endif

  • trunk/Source/WebKit/UIProcess/WebPageProxy.h

    r245679 r245778  
    727727    void requestDocumentEditingContext(WebKit::DocumentEditingContextRequest, CompletionHandler<void(WebKit::DocumentEditingContext)>&&);
    728728    void generateSyntheticEditingCommand(SyntheticEditingCommandType);
    729 #if ENABLE(DATA_INTERACTION)
     729#if ENABLE(DRAG_SUPPORT)
    730730    void didHandleDragStartRequest(bool started);
    731731    void didHandleAdditionalDragItemsRequest(bool added);
    732732    void requestDragStart(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, WebCore::DragSourceAction allowedActions);
    733733    void requestAdditionalItemsForDragSession(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, WebCore::DragSourceAction allowedActions);
    734     void didConcludeEditDrag(Optional<WebCore::TextIndicatorData>);
     734    void willReceiveEditDragSnapshot();
     735    void didReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData>);
     736    void didConcludeDrop();
    735737#endif
    736738#endif // PLATFORM(IOS_FAMILY)

  • trunk/Source/WebKit/UIProcess/WebPageProxy.messages.in

    r245565 r245778  
    334334    DidHandleDragStartRequest(bool started)
    335335    DidHandleAdditionalDragItemsRequest(bool added)
    336     DidConcludeEditDrag(Optional<WebCore::TextIndicatorData> textIndicator)
     336    WillReceiveEditDragSnapshot()
     337    DidReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData> textIndicator)
    337338#endif
    338339

  • trunk/Source/WebKit/UIProcess/ios/DragDropInteractionState.h

    r244955 r245778  
    4141namespace WebCore {
    4242struct DragItem;
     43struct TextIndicatorData;
    4344}
    4445
    … …  
    5758
    5859    NSInteger itemIdentifier { 0 };
     60};
     61
     62struct ItemAndPreviewProvider {
     63    RetainPtr<UIDragItem> item;
     64    BlockPtr<void(UITargetedDragPreview *)> provider;
    5965};
    6066
    … …  
    9399    BlockPtr<void(NSArray<UIDragItem *> *)> takeAddDragItemCompletionBlock() { return WTFMove(m_addDragItemCompletionBlock); }
    94100
     101    void prepareForDelayedDropPreview(UIDragItem *, void(^provider)(UITargetedDragPreview *preview));
     102    void deliverDelayedDropPreview(UIView *contentView, UIView *previewContainer, const WebCore::TextIndicatorData&);
     103    void clearAllDelayedItemPreviewProviders();
     104
    95105private:
    96106    void updatePreviewsForActiveDragSources();
    … …  
    109119    Optional<DragSourceState> m_stagedDragSource;
    110120    Vector<DragSourceState> m_activeDragSources;
     121    Vector<ItemAndPreviewProvider> m_delayedItemPreviewProviders;
    111122};
    112123

  • trunk/Source/WebKit/UIProcess/ios/DragDropInteractionState.mm

    r239427 r245778  
    4545}
    4646
    47 static UITargetedDragPreview *createTargetedDragPreview(UIImage *image, UIView *rootView, UIView *previewContainer, const FloatRect& frameInRootViewCoordinates, const Vector<FloatRect>& clippingRectsInFrameCoordinates, UIColor *backgroundColor, UIBezierPath *visiblePath)
     47static RetainPtr<UITargetedDragPreview> createTargetedDragPreview(UIImage *image, UIView *rootView, UIView *previewContainer, const FloatRect& frameInRootViewCoordinates, const Vector<FloatRect>& clippingRectsInFrameCoordinates, UIColor *backgroundColor, UIBezierPath *visiblePath)
    4848{
    4949    if (frameInRootViewCoordinates.isEmpty() || !image)
    … …  
    7979    CGPoint centerInContainerCoordinates = { CGRectGetMidX(frameInContainerCoordinates), CGRectGetMidY(frameInContainerCoordinates) };
    8080    auto target = adoptNS([[UIDragPreviewTarget alloc] initWithContainer:previewContainer center:centerInContainerCoordinates]);
    81     auto dragPreview = adoptNS([[UITargetedDragPreview alloc] initWithView:imageView.get() parameters:parameters.get() target:target.get()]);
    82     return dragPreview.autorelease();
     81    return adoptNS([[UITargetedDragPreview alloc] initWithView:imageView.get() parameters:parameters.get() target:target.get()]);
    8382}
    8483
    … …  
    185184    m_didBeginDragging = true;
    186185    updatePreviewsForActiveDragSources();
     186}
     187
     188void DragDropInteractionState::prepareForDelayedDropPreview(UIDragItem *item, void(^provider)(UITargetedDragPreview *preview))
     189{
     190    m_delayedItemPreviewProviders.append({ item, provider });
     191}
     192
     193void DragDropInteractionState::deliverDelayedDropPreview(UIView *contentView, UIView *previewContainer, const WebCore::TextIndicatorData& indicator)
     194{
     195    if (m_delayedItemPreviewProviders.isEmpty())
     196        return;
     197
     198    auto textIndicatorImage = uiImageForImage(indicator.contentImage.get());
     199    auto preview = createTargetedDragPreview(textIndicatorImage.get(), contentView, previewContainer, indicator.textBoundingRectInRootViewCoordinates, indicator.textRectsInBoundingRectCoordinates, [UIColor colorWithCGColor:cachedCGColor(indicator.estimatedBackgroundColor)], nil);
     200    for (auto& itemAndPreviewProvider : m_delayedItemPreviewProviders)
     201        itemAndPreviewProvider.provider(preview.get());
     202    m_delayedItemPreviewProviders.clear();
     203}
     204
     205void DragDropInteractionState::clearAllDelayedItemPreviewProviders()
     206{
     207    for (auto& itemAndPreviewProvider : m_delayedItemPreviewProviders)
     208        itemAndPreviewProvider.provider(nil);
     209    m_delayedItemPreviewProviders.clear();
    187210}
    188211
    … …  
    198221            auto path = source.visiblePath.value();
    199222            UIBezierPath *visiblePath = [UIBezierPath bezierPathWithCGPath:path.ensurePlatformPath()];
    200             return createTargetedDragPreview(source.image.get(), contentView, previewContainer, source.dragPreviewFrameInRootViewCoordinates, { }, nil, visiblePath);
     223            return createTargetedDragPreview(source.image.get(), contentView, previewContainer, source.dragPreviewFrameInRootViewCoordinates, { }, nil, visiblePath).autorelease();
    201224        }
    202         return createTargetedDragPreview(source.image.get(), contentView, previewContainer, source.dragPreviewFrameInRootViewCoordinates, { }, nil, nil);
     225        return createTargetedDragPreview(source.image.get(), contentView, previewContainer, source.dragPreviewFrameInRootViewCoordinates, { }, nil, nil).autorelease();
    203226    }
    204227
    … …  
    206229        auto indicator = source.indicatorData.value();
    207230        auto textIndicatorImage = uiImageForImage(indicator.contentImage.get());
    208         return createTargetedDragPreview(textIndicatorImage.get(), contentView, previewContainer, indicator.textBoundingRectInRootViewCoordinates, indicator.textRectsInBoundingRectCoordinates, [UIColor colorWithCGColor:cachedCGColor(indicator.estimatedBackgroundColor)], nil);
     231        return createTargetedDragPreview(textIndicatorImage.get(), contentView, previewContainer, indicator.textBoundingRectInRootViewCoordinates, indicator.textRectsInBoundingRectCoordinates, [UIColor colorWithCGColor:cachedCGColor(indicator.estimatedBackgroundColor)], nil).autorelease();
    209232    }
    210233
    … …  
    267290void DragDropInteractionState::dragAndDropSessionsDidEnd()
    268291{
     292    clearAllDelayedItemPreviewProviders();
     293
    269294    // If any of UIKit's completion blocks are still in-flight when the drag interaction ends, we need to ensure that they are still invoked
    270295    // to prevent UIKit from getting into an inconsistent state.

  • trunk/Source/WebKit/UIProcess/ios/PageClientImplIOS.h

    r245112 r245778  
    238238    void didHandleAdditionalDragItemsRequest(bool added) override;
    239239    void startDrag(const WebCore::DragItem&, const ShareableBitmap::Handle& image) override;
    240     void didConcludeEditDrag(Optional<WebCore::TextIndicatorData>) override;
     240    void willReceiveEditDragSnapshot() override;
     241    void didReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData>) override;
    241242    void didChangeDragCaretRect(const WebCore::IntRect& previousCaretRect, const WebCore::IntRect& caretRect) override;
    242243#endif

  • trunk/Source/WebKit/UIProcess/ios/PageClientImplIOS.mm

    r245268 r245778  
    821821}
    822822
    823 void PageClientImpl::didConcludeEditDrag(Optional<TextIndicatorData> data)
    824 {
    825     [m_contentView _didConcludeEditDrag:data];
     823void PageClientImpl::willReceiveEditDragSnapshot()
     824{
     825    [m_contentView _willReceiveEditDragSnapshot];
     826}
     827
     828void PageClientImpl::didReceiveEditDragSnapshot(Optional<TextIndicatorData> data)
     829{
     830    [m_contentView _didReceiveEditDragSnapshot:data];
    826831}
    827832

  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h

    r245268 r245778  
    335335
    336336    BOOL _focusRequiresStrongPasswordAssistance;
     337    BOOL _waitingForEditDragSnapshot;
    337338
    338339    BOOL _hasSetUpInteractions;
    … …  
    346347    RetainPtr<UIDropInteraction> _dropInteraction;
    347348    BOOL _shouldRestoreCalloutBarAfterDrop;
    348     BOOL _isAnimatingConcludeEditDrag;
    349349    RetainPtr<UIView> _visibleContentViewSnapshot;
     350    RetainPtr<UIView> _unselectedContentSnapshot;
    350351    RetainPtr<_UITextDragCaretView> _editDropCaretView;
     352    BlockPtr<void()> _actionToPerformAfterReceivingEditDragSnapshot;
    351353#endif
    352354
    … …  
    491493- (void)_didHandleAdditionalDragItemsRequest:(BOOL)added;
    492494- (void)_startDrag:(RetainPtr<CGImageRef>)image item:(const WebCore::DragItem&)item;
    493 - (void)_didConcludeEditDrag:(Optional<WebCore::TextIndicatorData>)data;
     495- (void)_willReceiveEditDragSnapshot;
     496- (void)_didReceiveEditDragSnapshot:(Optional<WebCore::TextIndicatorData>)data;
    494497- (void)_didChangeDragCaretRect:(CGRect)previousRect currentRect:(CGRect)rect;
    495498#endif
    … …  
    518521- (void)setTimePickerValueToHour:(NSInteger)hour minute:(NSInteger)minute;
    519522- (NSDictionary *)_contentsOfUserInterfaceItem:(NSString *)userInterfaceItem;
     523- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action;
    520524
    521525@property (nonatomic, readonly) NSString *textContentTypeForTesting;

  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm

    r245691 r245778  
    842842
    843843    _focusRequiresStrongPasswordAssistance = NO;
     844    _waitingForEditDragSnapshot = NO;
    844845
    845846#if USE(UIKIT_KEYBOARD_ADDITIONS)
    … …  
    62706271- (void)cleanUpDragSourceSessionState
    62716272{
     6273    if (_waitingForEditDragSnapshot)
     6274        return;
     6275
    62726276    if (_dragDropInteractionState.dragSession() || _dragDropInteractionState.isPerformingDrop())
    62736277        RELEASE_LOG(DragAndDrop, "Cleaning up dragging state (has pending operation: %d)", [[WebItemProviderPasteboard sharedInstance] hasPendingOperation]);
    … …  
    62826286    [self _restoreCalloutBarIfNeeded];
    62836287
    6284     [_visibleContentViewSnapshot removeFromSuperview];
    6285     _visibleContentViewSnapshot = nil;
     6288    [std::exchange(_visibleContentViewSnapshot, nil) removeFromSuperview];
    62866289    [_editDropCaretView remove];
    62876290    _editDropCaretView = nil;
    6288     _isAnimatingConcludeEditDrag = NO;
    62896291    _shouldRestoreCalloutBarAfterDrop = NO;
    62906292
    … …  
    63086310}
    63096311
    6310 - (void)_didConcludeEditDrag:(Optional<WebCore::TextIndicatorData>)data
    6311 {
     6312- (void)_willReceiveEditDragSnapshot
     6313{
     6314    _waitingForEditDragSnapshot = YES;
     6315}
     6316
     6317- (void)_didReceiveEditDragSnapshot:(Optional<WebCore::TextIndicatorData>)data
     6318{
     6319    _waitingForEditDragSnapshot = NO;
     6320
     6321    [self _deliverDelayedDropPreviewIfPossible:data];
     6322    [self cleanUpDragSourceSessionState];
     6323
     6324    if (auto action = WTFMove(_actionToPerformAfterReceivingEditDragSnapshot))
     6325        action();
     6326}
     6327
     6328- (void)_deliverDelayedDropPreviewIfPossible:(Optional<WebCore::TextIndicatorData>)data
     6329{
     6330    if (!_visibleContentViewSnapshot)
     6331        return;
     6332
    63126333    if (!data)
     6334        return;
     6335
     6336    if (!data->contentImage)
    63136337        return;
    63146338
    … …  
    63226346
    63236347    auto unselectedContentImageForEditDrag = adoptNS([[UIImage alloc] initWithCGImage:unselectedSnapshotImage.get() scale:_page->deviceScaleFactor() orientation:UIImageOrientationUp]);
    6324     auto unselectedContentSnapshot = adoptNS([[UIImageView alloc] initWithImage:unselectedContentImageForEditDrag.get()]);
    6325     [unselectedContentSnapshot setFrame:data->contentImageWithoutSelectionRectInRootViewCoordinates];
    6326 
    6327     auto protectedSelf = retainPtr(self);
    6328     auto visibleContentViewSnapshot = adoptNS(_visibleContentViewSnapshot.leakRef());
    6329 
    6330     _isAnimatingConcludeEditDrag = YES;
    6331     [self insertSubview:unselectedContentSnapshot.get() belowSubview:visibleContentViewSnapshot.get()];
    6332     [UIView animateWithDuration:0.25 animations:^() {
    6333         [visibleContentViewSnapshot setAlpha:0];
    6334     } completion:^(BOOL completed) {
    6335         [visibleContentViewSnapshot removeFromSuperview];
    6336         [UIView animateWithDuration:0.25 animations:^() {
    6337             [protectedSelf _stopSuppressingSelectionAssistantForReason:WebKit::DropAnimationIsRunning];
    6338             [unselectedContentSnapshot setAlpha:0];
    6339         } completion:^(BOOL completed) {
    6340             [unselectedContentSnapshot removeFromSuperview];
    6341         }];
    6342     }];
     6348    _unselectedContentSnapshot = adoptNS([[UIImageView alloc] initWithImage:unselectedContentImageForEditDrag.get()]);
     6349    [_unselectedContentSnapshot setFrame:data->contentImageWithoutSelectionRectInRootViewCoordinates];
     6350
     6351    [self insertSubview:_unselectedContentSnapshot.get() belowSubview:_visibleContentViewSnapshot.get()];
     6352    _dragDropInteractionState.deliverDelayedDropPreview(self, self.unscaledView, data.value());
    63436353}
    63446354
    … …  
    63506360    if ([self.webViewUIDelegate respondsToSelector:@selector(_webView:dataInteractionOperationWasHandled:forSession:itemProviders:)])
    63516361        [self.webViewUIDelegate _webView:_webView dataInteractionOperationWasHandled:handled forSession:dropSession itemProviders:[WebItemProviderPasteboard sharedInstance].itemProviders];
    6352 
    6353     if (!_isAnimatingConcludeEditDrag)
    6354         [self _stopSuppressingSelectionAssistantForReason:WebKit::DropAnimationIsRunning];
    63556362
    63566363    CGPoint global;
    … …  
    65106517}
    65116518
     6519- (UIView *)textEffectsWindow
     6520{
     6521#if HAVE(UISCENE)
     6522    return [UITextEffectsWindow sharedTextEffectsWindowForWindowScene:self.window.windowScene];
     6523#else
     6524    return [UITextEffectsWindow sharedTextEffectsWindow];
     6525#endif
     6526}
     6527
    65126528- (NSDictionary *)_autofillContext
    65136529{
    … …  
    68916907}
    68926908
     6909- (void)dropInteraction:(UIDropInteraction *)interaction item:(UIDragItem *)item willAnimateDropWithAnimator:(id <UIDragAnimating>)animator
     6910{
     6911    [animator addCompletion:[strongSelf = retainPtr(self)] (UIViewAnimatingPosition) {
     6912        [std::exchange(strongSelf->_unselectedContentSnapshot, nil) removeFromSuperview];
     6913    }];
     6914}
     6915
     6916- (void)dropInteraction:(UIDropInteraction *)interaction concludeDrop:(id <UIDropSession>)session
     6917{
     6918    [self _stopSuppressingSelectionAssistantForReason:WebKit::DropAnimationIsRunning];
     6919    [std::exchange(_visibleContentViewSnapshot, nil) removeFromSuperview];
     6920    [std::exchange(_unselectedContentSnapshot, nil) removeFromSuperview];
     6921    _dragDropInteractionState.clearAllDelayedItemPreviewProviders();
     6922    _page->didConcludeDrop();
     6923}
     6924
    68936925- (UITargetedDragPreview *)dropInteraction:(UIDropInteraction *)interaction previewForDroppingItem:(UIDragItem *)item withDefault:(UITargetedDragPreview *)defaultPreview
    68946926{
    … …  
    68976929        return nil;
    68986930
    6899 ALLOW_DEPRECATED_DECLARATIONS_BEGIN
    6900     // FIXME: <rdar://problem/31074376> [WK2] Performing an edit drag should transition from the initial drag preview to the final drop preview
    6901     // This is blocked on UIKit support, since we aren't able to update the text clipping rects of a UITargetedDragPreview mid-flight. For now,
    6902     // just zoom to the center of the caret rect while shrinking the drop preview.
    6903     auto caretRectInWindowCoordinates = [self convertRect:caretRect toView:[UITextEffectsWindow sharedTextEffectsWindow]];
     6931    UIView *textEffectsWindow = self.textEffectsWindow;
     6932    auto caretRectInWindowCoordinates = [self convertRect:caretRect toView:textEffectsWindow];
    69046933    auto caretCenterInWindowCoordinates = CGPointMake(CGRectGetMidX(caretRectInWindowCoordinates), CGRectGetMidY(caretRectInWindowCoordinates));
    6905     auto target = adoptNS([[UIDragPreviewTarget alloc] initWithContainer:[UITextEffectsWindow sharedTextEffectsWindow] center:caretCenterInWindowCoordinates transform:CGAffineTransformMakeScale(0, 0)]);
    6906 ALLOW_DEPRECATED_DECLARATIONS_END
     6934    auto targetPreviewCenterInWindowCoordinates = CGPointMake(caretCenterInWindowCoordinates.x + defaultPreview.size.width / 2, caretCenterInWindowCoordinates.y + defaultPreview.size.height / 2);
     6935    auto target = adoptNS([[UIDragPreviewTarget alloc] initWithContainer:textEffectsWindow center:targetPreviewCenterInWindowCoordinates transform:CGAffineTransformIdentity]);
    69076936    return [defaultPreview retargetedPreviewWithTarget:target.get()];
     6937}
     6938
     6939- (void)_dropInteraction:(UIDropInteraction *)interaction delayedPreviewProviderForDroppingItem:(UIDragItem *)item previewProvider:(void(^)(UITargetedDragPreview *preview))previewProvider
     6940{
     6941    // FIXME: This doesn't currently handle multiple items in a drop session.
     6942    _dragDropInteractionState.prepareForDelayedDropPreview(item, previewProvider);
    69086943}
    69096944
    … …  
    71667201
    71677202@implementation WKContentView (WKTesting)
     7203
     7204- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action
     7205{
     7206#if ENABLE(DRAG_SUPPORT)
     7207    ASSERT(!_actionToPerformAfterReceivingEditDragSnapshot);
     7208    if (_waitingForEditDragSnapshot) {
     7209        _actionToPerformAfterReceivingEditDragSnapshot = action;
     7210        return;
     7211    }
     7212#endif
     7213    action();
     7214}
    71687215
    71697216- (WKFormInputControl *)formInputControl

  • trunk/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm

    r245679 r245778  
    12201220}
    12211221
    1222 void WebPageProxy::didConcludeEditDrag(Optional<TextIndicatorData> data)
    1223 {
    1224     pageClient().didConcludeEditDrag(data);
     1222void WebPageProxy::willReceiveEditDragSnapshot()
     1223{
     1224    pageClient().willReceiveEditDragSnapshot();
     1225}
     1226
     1227void WebPageProxy::didReceiveEditDragSnapshot(Optional<TextIndicatorData> data)
     1228{
     1229    pageClient().didReceiveEditDragSnapshot(data);
     1230}
     1231
     1232void WebPageProxy::didConcludeDrop()
     1233{
     1234    m_process->send(Messages::WebPage::DidConcludeDrop(), m_pageID);
    12251235}
    12261236

  • trunk/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp

    r245540 r245778  
    588588}
    589589
     590void WebChromeClient::didFinishLoadingImageForElement(HTMLImageElement& element)
     591{
     592    m_page.didFinishLoadingImageForElement(element);
     593}
     594
    590595PlatformPageClient WebChromeClient::platformPageClient() const
    591596{

  • trunk/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h

    r245366 r245778  
    3030
    3131namespace WebCore {
     32class HTMLImageElement;
    3233class RegistrableDomain;
    3334enum class StorageAccessPromptWasShown : bool;
    … …  
    121122    WebCore::IntPoint accessibilityScreenToRootView(const WebCore::IntPoint&) const final;
    122123    WebCore::IntRect rootViewToAccessibilityScreen(const WebCore::IntRect&) const final;
     124
     125    void didFinishLoadingImageForElement(WebCore::HTMLImageElement&) final;
    123126
    124127    PlatformPageClient platformPageClient() const final;

  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp

    r245716 r245778  
    67416741#endif
    67426742
     6743#if !PLATFORM(IOS_FAMILY) || !ENABLE(DRAG_SUPPORT)
     6744
     6745void WebPage::didFinishLoadingImageForElement(WebCore::HTMLImageElement&)
     6746{
     6747}
     6748
     6749#endif
     6750
    67436751} // namespace WebKit
    67446752

  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.h

    r245679 r245778  
    155155class FrameView;
    156156class GraphicsContext;
     157class HTMLImageElement;
    157158class HTMLMenuElement;
    158159class HTMLMenuItemElement;
    … …  
    11141115    void setUseIconLoadingClient(bool);
    11151116
    1116 #if ENABLE(DATA_INTERACTION)
     1117#if PLATFORM(IOS_FAMILY) && ENABLE(DRAG_SUPPORT)
    11171118    void didConcludeEditDrag();
    1118 #endif
     1119    void didConcludeDrop();
     1120#endif
     1121
     1122    void didFinishLoadingImageForElement(WebCore::HTMLImageElement&);
    11191123
    11201124    WebURLSchemeHandlerProxy* urlSchemeHandlerForScheme(const String&);
    … …  
    12591263    void requestDragStart(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, uint64_t allowedActions);
    12601264    void requestAdditionalItemsForDragSession(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, uint64_t allowedActions);
     1265    void computeAndSendEditDragSnapshot();
    12611266#endif
    12621267
    … …  
    17691774    bool m_isStartingDrag { false };
    17701775    WebCore::DragSourceAction m_allowedDragSourceActions { WebCore::DragSourceActionAny };
     1776#endif
     1777
     1778#if ENABLE(DRAG_SUPPORT) && PLATFORM(IOS_FAMILY)
     1779    HashSet<RefPtr<WebCore::HTMLImageElement>> m_pendingImageElementsForDropSnapshot;
    17711780#endif
    17721781

  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.messages.in

    r245678 r245778  
    304304#endif
    305305
    306 #if ENABLE(DATA_INTERACTION)
     306#if PLATFORM(IOS_FAMILY) && ENABLE(DRAG_SUPPORT)
    307307    RequestDragStart(WebCore::IntPoint clientPosition, WebCore::IntPoint globalPosition, uint64_t allowedActions)
    308308    RequestAdditionalItemsForDragSession(WebCore::IntPoint clientPosition, WebCore::IntPoint globalPosition, uint64_t allowedActions)
     309    DidConcludeDrop()
    309310#endif
    310311

  • trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm

    r245716 r245778  
    826826}
    827827
     828void WebPage::didConcludeDrop()
     829{
     830    m_pendingImageElementsForDropSnapshot.clear();
     831}
     832
    828833void WebPage::didConcludeEditDrag()
    829834{
     835    send(Messages::WebPageProxy::WillReceiveEditDragSnapshot());
     836
     837    layoutIfNeeded();
     838
     839    m_pendingImageElementsForDropSnapshot.clear();
     840
     841    bool waitingForAnyImageToLoad = false;
     842    auto& frame = m_page->focusController().focusedOrMainFrame();
     843    if (auto range = frame.selection().selection().toNormalizedRange()) {
     844        for (TextIterator iterator(range.get()); !iterator.atEnd(); iterator.advance()) {
     845            auto* node = iterator.node();
     846            if (!is<HTMLImageElement>(node))
     847                continue;
     848
     849            auto& imageElement = downcast<HTMLImageElement>(*node);
     850            auto* cachedImage = imageElement.cachedImage();
     851            if (cachedImage && cachedImage->image() && cachedImage->image()->isNull()) {
     852                m_pendingImageElementsForDropSnapshot.add(&imageElement);
     853                waitingForAnyImageToLoad = true;
     854            }
     855        }
     856    }
     857
     858    if (!waitingForAnyImageToLoad)
     859        computeAndSendEditDragSnapshot();
     860}
     861
     862void WebPage::didFinishLoadingImageForElement(WebCore::HTMLImageElement& element)
     863{
     864    if (m_pendingImageElementsForDropSnapshot.isEmpty())
     865        return;
     866
     867    m_pendingImageElementsForDropSnapshot.remove(&element);
     868
     869    if (m_pendingImageElementsForDropSnapshot.isEmpty())
     870        computeAndSendEditDragSnapshot();
     871}
     872
     873void WebPage::computeAndSendEditDragSnapshot()
     874{
    830875    Optional<TextIndicatorData> textIndicatorData;
    831 
    832876    static auto defaultTextIndicatorOptionsForEditDrag = TextIndicatorOptionIncludeSnapshotOfAllVisibleContentWithoutSelection | TextIndicatorOptionExpandClipBeyondVisibleRect | TextIndicatorOptionPaintAllContent | TextIndicatorOptionIncludeMarginIfRangeMatchesSelection | TextIndicatorOptionPaintBackgrounds | TextIndicatorOptionComputeEstimatedBackgroundColor| TextIndicatorOptionUseSelectionRectForSizing | TextIndicatorOptionIncludeSnapshotWithSelectionHighlight;
    833877    auto& frame = m_page->focusController().focusedOrMainFrame();
    834878    if (auto range = frame.selection().selection().toNormalizedRange()) {
    835         if (auto textIndicator = TextIndicator::createWithRange(*range, defaultTextIndicatorOptionsForEditDrag, TextIndicatorPresentationTransition::None, FloatSize()))
     879        if (auto textIndicator = TextIndicator::createWithRange(*range, defaultTextIndicatorOptionsForEditDrag, TextIndicatorPresentationTransition::None, { }))
    836880            textIndicatorData = textIndicator->data();
    837881    }
    838 
    839     send(Messages::WebPageProxy::DidConcludeEditDrag(WTFMove(textIndicatorData)));
    840 }
     882    send(Messages::WebPageProxy::DidReceiveEditDragSnapshot(WTFMove(textIndicatorData)));
     883}
     884
    841885#endif
    842886

  • trunk/Source/WebKitLegacy/mac/ChangeLog

    r245596 r245778  
     12019-05-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Dropped text, attachments, and images should animate into place
     4        https://bugs.webkit.org/show_bug.cgi?id=198243
     5        <rdar://problem/35205373>
     6
     7        Reviewed by Tim Horton.
     8
     9        Add a new chrome client method. See other changelogs for more detail.
     10
     11        * WebCoreSupport/WebChromeClient.h:
     12        * WebCoreSupport/WebChromeClient.mm:
     13        (WebChromeClient::didFinishLoadingImageForElement):
     14
    1152019-05-21  Alex Christensen  <achristensen@webkit.org>
    216

  • trunk/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h

    r244633 r245778  
    3232#import <wtf/Forward.h>
    3333
     34namespace WebCore {
     35class HTMLImageElement;
     36}
     37
    3438@class WebView;
    3539
    … …  
    102106    WebCore::IntPoint accessibilityScreenToRootView(const WebCore::IntPoint&) const final;
    103107    WebCore::IntRect rootViewToAccessibilityScreen(const WebCore::IntRect&) const final;
     108
     109    void didFinishLoadingImageForElement(WebCore::HTMLImageElement&) final;
    104110
    105111    PlatformPageClient platformPageClient() const final;

  • trunk/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm

    r244440 r245778  
    611611}
    612612
     613void WebChromeClient::didFinishLoadingImageForElement(HTMLImageElement&)
     614{
     615}
     616
    613617PlatformPageClient WebChromeClient::platformPageClient() const
    614618{

  • trunk/Source/WebKitLegacy/win/ChangeLog

    r244932 r245778  
     12019-05-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Dropped text, attachments, and images should animate into place
     4        https://bugs.webkit.org/show_bug.cgi?id=198243
     5        <rdar://problem/35205373>
     6
     7        Reviewed by Tim Horton.
     8
     9        * WebCoreSupport/WebChromeClient.cpp:
     10        (WebChromeClient::didFinishLoadingImageForElement):
     11        * WebCoreSupport/WebChromeClient.h:
     12
    1132019-05-03  Daniel Bates  <dabates@apple.com>
    214

  • trunk/Source/WebKitLegacy/win/WebCoreSupport/WebChromeClient.cpp

    r244633 r245778  
    704704}
    705705
     706void WebChromeClient::didFinishLoadingImageForElement(WebCore::HTMLImageElement&)
     707{
     708}
     709
    706710void WebChromeClient::setCursor(const Cursor& cursor)
    707711{

  • trunk/Source/WebKitLegacy/win/WebCoreSupport/WebChromeClient.h

    r244633 r245778  
    173173    RefPtr<WebCore::Icon> createIconForFiles(const Vector<String>&) final;
    174174
     175    void didFinishLoadingImageForElement(WebCore::HTMLImageElement&) final;
     176
    175177private:
    176178    COMPtr<IWebUIDelegate> uiDelegate();

  • trunk/Tools/ChangeLog

    r245777 r245778  
     12019-05-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Dropped text, attachments, and images should animate into place
     4        https://bugs.webkit.org/show_bug.cgi?id=198243
     5        <rdar://problem/35205373>
     6
     7        Reviewed by Tim Horton.
     8
     9        Adjusts the iOS dragging simulator, and adds a new API test. See below for more detail.
     10
     11        * TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm:
     12        (TestWebKitAPI::isCompletelyWhite):
     13        (TestWebKitAPI::TEST):
     14
     15        Add a test that drags and drops an image into a contenteditable element, and then observes the resulting
     16        UITargetedDragPreviews upon drop.
     17
     18        * TestWebKitAPI/cocoa/DragAndDropSimulator.h:
     19        * TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm:
     20        (-[DragAndDropSimulator _resetSimulatedState]):
     21        (-[DragAndDropSimulator runFrom:to:additionalItemRequestLocations:]):
     22        (-[DragAndDropSimulator _concludeDropAndPerformOperationIfNecessary]):
     23
     24        Teach the iOS version of DragAndDropSimulator to invoke -dropInteraction:concludeDrop: on the drop interaction
     25        delegate when the drop finishes. This additionally uses _doAfterReceivingEditDragSnapshotForTesting: to defer
     26        the end of the simulated drag and drop until after drag previews have been received during an edit drag.
     27
     28        (-[DragAndDropSimulator dropPreviews]):
     29        (-[DragAndDropSimulator delayedDropPreviews]):
     30
     31        Have the drag and drop simulator remember which previews were returned by the delegate on drop, as well as which
     32        previews were provided asynchronously.
     33
     34        (-[DragAndDropSimulator _webView:dataInteractionOperationWasHandled:forSession:itemProviders:]):
     35        * TestWebKitAPI/ios/UIKitSPI.h:
     36
     37        Stage the new private drop interacton delegate method.
     38
    1392019-05-25  Simon Fraser  <simon.fraser@apple.com>
    240

  • trunk/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm

    r244955 r245778  
    21082108}
    21092109
     2110static BOOL isCompletelyWhite(UIImage *image)
     2111{
     2112    auto data = adoptCF(CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage)));
     2113    auto* dataPtr = CFDataGetBytePtr(data.get());
     2114    int imageWidth = image.size.width;
     2115    for (int row = 0; row < image.size.height; ++row) {
     2116        for (int column = 0; column < imageWidth; ++column) {
     2117            int pixelOffset = ((imageWidth * row) + column) * 4;
     2118            if (dataPtr[pixelOffset] != 0xFF || dataPtr[pixelOffset + 1] != 0xFF || dataPtr[pixelOffset + 2] != 0xFF)
     2119                return NO;
     2120        }
     2121    }
     2122    return YES;
     2123}
     2124
     2125TEST(DragAndDropTests, DropPreviewForImageInEditableArea)
     2126{
     2127    auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]);
     2128    [webView synchronouslyLoadTestPageNamed:@"image-and-contenteditable"];
     2129
     2130    // Ensure that the resulting snapshot on drop contains only the dragged image.
     2131    [webView stringByEvaluatingJavaScript:@"editor.style.border = 'none'; editor.style.outline = 'none'"];
     2132
     2133    auto simulator = adoptNS([[DragAndDropSimulator alloc] initWithWebView:webView.get()]);
     2134    [simulator runFrom:CGPointMake(100, 50) to:CGPointMake(100, 300)];
     2135
     2136    NSArray *dropPreviews = [simulator dropPreviews];
     2137    NSArray *delayedDropPreviews = [simulator delayedDropPreviews];
     2138    EXPECT_EQ(1U, dropPreviews.count);
     2139    EXPECT_EQ(1U, delayedDropPreviews.count);
     2140    EXPECT_EQ(UITargetedDragPreview.class, [dropPreviews.firstObject class]);
     2141    EXPECT_EQ(UITargetedDragPreview.class, [delayedDropPreviews.firstObject class]);
     2142
     2143    UITargetedDragPreview *finalPreview = (UITargetedDragPreview *)delayedDropPreviews.firstObject;
     2144    EXPECT_EQ(UIImageView.class, finalPreview.view.class);
     2145    EXPECT_FALSE(isCompletelyWhite([(UIImageView *)finalPreview.view image]));
     2146}
     2147
    21102148} // namespace TestWebKitAPI
    21112149

  • trunk/Tools/TestWebKitAPI/cocoa/DragAndDropSimulator.h

    r244955 r245778  
    112112@property (nonatomic, readonly) CGRect lastKnownDragCaretRect;
    113113@property (nonatomic, readonly) NSArray<UITargetedDragPreview *> *liftPreviews;
     114@property (nonatomic, readonly) NSArray *dropPreviews;
     115@property (nonatomic, readonly) NSArray *delayedDropPreviews;
    114116@property (nonatomic, readonly) BOOL suppressedSelectionCommandsDuringDrop;
    115117

  • trunk/Tools/TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm

    r244955 r245778  
    3535#import <UIKit/UIDragInteraction.h>
    3636#import <UIKit/UIDragItem.h>
     37#import <UIKit/UIDropInteraction.h>
    3738#import <UIKit/UIInteraction.h>
    3839#import <WebKit/WKWebViewPrivate.h>
    … …  
    311312    RetainPtr<NSMutableArray<NSValue *>>_queuedAdditionalItemRequestLocations;
    312313    RetainPtr<NSMutableArray<UITargetedDragPreview *>> _liftPreviews;
     314    RetainPtr<NSMutableArray> _dropPreviews;
     315    RetainPtr<NSMutableArray> _delayedDropPreviews;
    313316
    314317    RetainPtr<NSMutableArray<_WKAttachment *>> _insertedAttachments;
    … …  
    318321    double _currentProgress;
    319322    bool _isDoneWithCurrentRun;
     323    bool _isDoneWaitingForDelayedDropPreviews;
    320324    DragAndDropPhase _phase;
    321325
    … …  
    374378    _currentProgress = 0;
    375379    _isDoneWithCurrentRun = false;
     380    _isDoneWaitingForDelayedDropPreviews = true;
    376381    _observedEventNames = adoptNS([[NSMutableArray alloc] init]);
    377382    _insertedAttachments = adoptNS([[NSMutableArray alloc] init]);
    … …  
    385390    _queuedAdditionalItemRequestLocations = adoptNS([[NSMutableArray alloc] init]);
    386391    _liftPreviews = adoptNS([[NSMutableArray alloc] init]);
     392    _dropPreviews = adoptNS([[NSMutableArray alloc] init]);
     393    _delayedDropPreviews = adoptNS([[NSMutableArray alloc] init]);
    387394    _hasStartedInputSession = false;
    388395}
    … …  
    455462
    456463    Util::run(&_isDoneWithCurrentRun);
     464    Util::run(&_isDoneWaitingForDelayedDropPreviews);
    457465    [_webView clearMessageHandlers:dragAndDropEventNames()];
    458466    _finalSelectionRects = [_webView selectionRectsAfterPresentationUpdate];
    … …  
    471479    auto operation = [_lastKnownDropProposal operation];
    472480    if (operation != UIDropOperationCancel && operation != UIDropOperationForbidden) {
     481        NSInteger dropPreviewIndex = 0;
     482        __block NSUInteger numberOfPendingPreviews = [_dropSession items].count;
     483        _isDoneWaitingForDelayedDropPreviews = !numberOfPendingPreviews;
     484        for (UIDragItem *item in [_dropSession items]) {
     485            auto defaultPreview = adoptNS([[UITargetedDragPreview alloc] initWithView:_webView.get()]);
     486            id <UIDropInteractionDelegate_Staging_31075005> delegate = (id <UIDropInteractionDelegate_Staging_31075005>)[_webView dropInteractionDelegate];
     487            UIDropInteraction *interaction = [_webView dropInteraction];
     488            [_dropPreviews addObject:[delegate dropInteraction:interaction previewForDroppingItem:item withDefault:defaultPreview.get()] ?: NSNull.null];
     489            [_delayedDropPreviews addObject:NSNull.null];
     490            [delegate _dropInteraction:interaction delayedPreviewProviderForDroppingItem:item previewProvider:^(UITargetedDragPreview *preview) {
     491                if (preview)
     492                    [_delayedDropPreviews setObject:preview atIndexedSubscript:dropPreviewIndex];
     493
     494                if (!--numberOfPendingPreviews)
     495                    _isDoneWaitingForDelayedDropPreviews = true;
     496            }];
     497            ++dropPreviewIndex;
     498        }
    473499        [[_webView dropInteractionDelegate] dropInteraction:[_webView dropInteraction] performDrop:_dropSession.get()];
    474500        _phase = DragAndDropPhasePerformingDrop;
    475501    } else {
    476         _isDoneWithCurrentRun = YES;
     502        _isDoneWithCurrentRun = true;
    477503        _phase = DragAndDropPhaseCancelled;
     504        [[_webView dropInteractionDelegate] dropInteraction:[_webView dropInteraction] concludeDrop:_dropSession.get()];
    478505    }
    479506
    … …  
    645672}
    646673
     674- (NSArray<UITargetedDragPreview *> *)dropPreviews
     675{
     676    return _dropPreviews.get();
     677}
     678
     679- (NSArray<UITargetedDragPreview *> *)delayedDropPreviews
     680{
     681    return _delayedDropPreviews.get();
     682}
     683
    647684- (CGRect)lastKnownDragCaretRect
    648685{
    … …  
    730767{
    731768    _suppressedSelectionCommandsDuringDrop = [_webView textInputContentView]._shouldSuppressSelectionCommands;
    732     _isDoneWithCurrentRun = true;
    733769
    734770    if (self.dropCompletionBlock)
    735771        self.dropCompletionBlock(handled, itemProviders);
     772
     773    [_webView _doAfterReceivingEditDragSnapshotForTesting:^{
     774        [[_webView dropInteractionDelegate] dropInteraction:[_webView dropInteraction] concludeDrop:_dropSession.get()];
     775        _isDoneWithCurrentRun = true;
     776    }];
    736777}
    737778

  • trunk/Tools/TestWebKitAPI/ios/UIKitSPI.h

    r244955 r245778  
    184184@end
    185185
     186#if PLATFORM(IOS)
     187
     188@protocol UIDropInteractionDelegate_Staging_31075005 <UIDropInteractionDelegate>
     189- (void)_dropInteraction:(UIDropInteraction *)interaction delayedPreviewProviderForDroppingItem:(UIDragItem *)item previewProvider:(void(^)(UITargetedDragPreview *preview))previewProvider;
     190@end
     191
     192#endif // PLATFORM(IOS)
     193
    186194#endif // PLATFORM(IOS_FAMILY)
Note: See TracChangeset for help on using the changeset viewer.