Changeset 245803 in webkit


Ignore:
Timestamp:
May 27, 2019 6:03:10 PM (5 years ago)
Author:
Wenson Hsieh
Message:

[iOS] Dropping in an editable element should result in a ranged selection
https://bugs.webkit.org/show_bug.cgi?id=198267
<rdar://problem/51145977>

Reviewed by Tim Horton.

Source/WebKit:

When drag and drop was first implemented for iOS in iOS 11, selection behavior when dropping into editable
elements matched that of macOS, by leaving the inserted content selected after performing the drop. However, in
other parts of the platform (e.g. Notes), both the keyboard and selection views are not shown after a drop.

Instead of matching macOS behavior, WebKit on iOS should match the rest of the platform. This is a little
tricky, since we use the selection range after a drop to create a TextIndicator snapshot when creating a drag
preview. To resolve this, we refactor some of the logic introduced in r245778 to remember the DOM range to
snapshot before collapsing the range to the end of the inserted content.

Tested by adjusting some existing API tests.

  • UIProcess/ios/WKContentViewInteraction.mm:

(-[WKContentView _elementDidFocus:userIsInteracting:blurPreviousNode:activityStateChanges:userObject:]):

Remove some logic that currently presents the keyboard while the user is performing a drop that focuses an
editable element.

  • WebProcess/WebPage/WebPage.h:

Add a member variable to keep track of which range should be snapshotted when generating a drop preview.

  • WebProcess/WebPage/ios/WebPageIOS.mm:

(WebKit::WebPage::didConcludeDrop):
(WebKit::WebPage::didConcludeEditDrag):

Collapse the selection range to the end after an edit drag (i.e., a drop in an editable area that inserted
content).

(WebKit::WebPage::computeAndSendEditDragSnapshot):

Tools:

Adjust some existing API tests that currently check for selection rects after a drop. Instead of checking for
visible selection rects, simply check for the start caret rect, as determined by WKContentView's
-selectionRange.

  • TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm:

(TestWebKitAPI::TEST):
(makeCGRectValue): Deleted.
(checkSelectionRectsWithLogging): Deleted.

  • TestWebKitAPI/cocoa/DragAndDropSimulator.h:

Replace finalSelectionRects with finalSelectionStartRect.

  • TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm:

(-[DragAndDropSimulator _resetSimulatedState]):
(-[DragAndDropSimulator runFrom:to:additionalItemRequestLocations:]):
(-[DragAndDropSimulator finalSelectionRects]): Deleted.

Location:
trunk
Files:
8 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebKit/ChangeLog

    r245801 r245803  
     12019-05-27  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Dropping in an editable element should result in a ranged selection
     4        https://bugs.webkit.org/show_bug.cgi?id=198267
     5        <rdar://problem/51145977>
     6
     7        Reviewed by Tim Horton.
     8
     9        When drag and drop was first implemented for iOS in iOS 11, selection behavior when dropping into editable
     10        elements matched that of macOS, by leaving the inserted content selected after performing the drop. However, in
     11        other parts of the platform (e.g. Notes), both the keyboard and selection views are not shown after a drop.
     12
     13        Instead of matching macOS behavior, WebKit on iOS should match the rest of the platform. This is a little
     14        tricky, since we use the selection range after a drop to create a TextIndicator snapshot when creating a drag
     15        preview. To resolve this, we refactor some of the logic introduced in r245778 to remember the DOM range to
     16        snapshot before collapsing the range to the end of the inserted content.
     17
     18        Tested by adjusting some existing API tests.
     19
     20        * UIProcess/ios/WKContentViewInteraction.mm:
     21        (-[WKContentView _elementDidFocus:userIsInteracting:blurPreviousNode:activityStateChanges:userObject:]):
     22
     23        Remove some logic that currently presents the keyboard while the user is performing a drop that focuses an
     24        editable element.
     25
     26        * WebProcess/WebPage/WebPage.h:
     27
     28        Add a member variable to keep track of which range should be snapshotted when generating a drop preview.
     29
     30        * WebProcess/WebPage/ios/WebPageIOS.mm:
     31        (WebKit::WebPage::didConcludeDrop):
     32        (WebKit::WebPage::didConcludeEditDrag):
     33
     34        Collapse the selection range to the end after an edit drag (i.e., a drop in an editable area that inserted
     35        content).
     36
     37        (WebKit::WebPage::computeAndSendEditDragSnapshot):
     38
    1392019-05-27  Chris Dumez  <cdumez@apple.com>
    240
  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm

    r245778 r245803  
    50755075                return YES;
    50765076
    5077 #if ENABLE(DRAG_SUPPORT)
    5078             if (_dragDropInteractionState.isPerformingDrop())
    5079                 return YES;
    5080 #endif
    5081 
    50825077            if (self.isFirstResponder || _becomingFirstResponder) {
    50835078                // When the software keyboard is being used to enter an url, only the focus activity state is changing.
  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.h

    r245796 r245803  
    17811781#if ENABLE(DRAG_SUPPORT) && PLATFORM(IOS_FAMILY)
    17821782    HashSet<RefPtr<WebCore::HTMLImageElement>> m_pendingImageElementsForDropSnapshot;
     1783    RefPtr<WebCore::Range> m_rangeForDropSnapshot;
    17831784#endif
    17841785
  • trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm

    r245787 r245803  
    828828void WebPage::didConcludeDrop()
    829829{
     830    m_rangeForDropSnapshot = nullptr;
    830831    m_pendingImageElementsForDropSnapshot.clear();
    831832}
     
    840841
    841842    bool waitingForAnyImageToLoad = false;
    842     auto& frame = m_page->focusController().focusedOrMainFrame();
    843     if (auto range = frame.selection().selection().toNormalizedRange()) {
    844         for (TextIterator iterator(range.get()); !iterator.atEnd(); iterator.advance()) {
     843    auto frame = makeRef(m_page->focusController().focusedOrMainFrame());
     844    if (auto selectionRange = frame->selection().selection().toNormalizedRange()) {
     845        for (TextIterator iterator(selectionRange.get()); !iterator.atEnd(); iterator.advance()) {
    845846            auto* node = iterator.node();
    846847            if (!is<HTMLImageElement>(node))
     
    854855            }
    855856        }
     857        auto collapsedRange = Range::create(selectionRange->ownerDocument(), selectionRange->endPosition(), selectionRange->endPosition());
     858        frame->selection().setSelectedRange(collapsedRange.ptr(), DOWNSTREAM, FrameSelection::ShouldCloseTyping::Yes, UserTriggered);
     859
     860        m_rangeForDropSnapshot = WTFMove(selectionRange);
    856861    }
    857862
     
    875880    Optional<TextIndicatorData> textIndicatorData;
    876881    static auto defaultTextIndicatorOptionsForEditDrag = TextIndicatorOptionIncludeSnapshotOfAllVisibleContentWithoutSelection | TextIndicatorOptionExpandClipBeyondVisibleRect | TextIndicatorOptionPaintAllContent | TextIndicatorOptionIncludeMarginIfRangeMatchesSelection | TextIndicatorOptionPaintBackgrounds | TextIndicatorOptionComputeEstimatedBackgroundColor| TextIndicatorOptionUseSelectionRectForSizing | TextIndicatorOptionIncludeSnapshotWithSelectionHighlight;
    877     auto& frame = m_page->focusController().focusedOrMainFrame();
    878     if (auto range = frame.selection().selection().toNormalizedRange()) {
     882    if (auto range = std::exchange(m_rangeForDropSnapshot, nullptr)) {
    879883        if (auto textIndicator = TextIndicator::createWithRange(*range, defaultTextIndicatorOptionsForEditDrag, TextIndicatorPresentationTransition::None, { }))
    880884            textIndicatorData = textIndicator->data();
  • trunk/Tools/ChangeLog

    r245798 r245803  
     12019-05-27  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [iOS] Dropping in an editable element should result in a ranged selection
     4        https://bugs.webkit.org/show_bug.cgi?id=198267
     5        <rdar://problem/51145977>
     6
     7        Reviewed by Tim Horton.
     8
     9        Adjust some existing API tests that currently check for selection rects after a drop. Instead of checking for
     10        visible selection rects, simply check for the start caret rect, as determined by WKContentView's
     11        -selectionRange.
     12
     13        * TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm:
     14        (TestWebKitAPI::TEST):
     15        (makeCGRectValue): Deleted.
     16        (checkSelectionRectsWithLogging): Deleted.
     17        * TestWebKitAPI/cocoa/DragAndDropSimulator.h:
     18
     19        Replace finalSelectionRects with finalSelectionStartRect.
     20
     21        * TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm:
     22        (-[DragAndDropSimulator _resetSimulatedState]):
     23        (-[DragAndDropSimulator runFrom:to:additionalItemRequestLocations:]):
     24        (-[DragAndDropSimulator finalSelectionRects]): Deleted.
     25
    1262019-05-27  Oriol Brufau  <obrufau@igalia.com>
    227
  • trunk/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm

    r245778 r245803  
    118118}
    119119
    120 static NSValue *makeCGRectValue(CGFloat x, CGFloat y, CGFloat width, CGFloat height)
    121 {
    122     return [NSValue valueWithCGRect:CGRectMake(x, y, width, height)];
    123 }
    124 
    125120static void checkCGRectIsEqualToCGRectWithLogging(CGRect expected, CGRect observed)
    126121{
     
    129124    if (!isEqual)
    130125        NSLog(@"Expected: %@ but observed: %@", NSStringFromCGRect(expected), NSStringFromCGRect(observed));
    131 }
    132 
    133 static void checkSelectionRectsWithLogging(NSArray *expected, NSArray *observed)
    134 {
    135     if (![expected isEqualToArray:observed])
    136         NSLog(@"Expected selection rects: %@ but observed: %@", expected, observed);
    137     EXPECT_TRUE([expected isEqualToArray:observed]);
    138126}
    139127
     
    276264    EXPECT_TRUE([observedEventNames containsObject:@"dragover"]);
    277265    EXPECT_TRUE([observedEventNames containsObject:@"drop"]);
    278     checkSelectionRectsWithLogging(@[ makeCGRectValue(1, 201, 215, 174) ], [simulator finalSelectionRects]);
     266    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(214, 201, 2, 174), [simulator finalSelectionStartRect]);
    279267    checkFirstTypeIsPresentAndSecondTypeIsMissing(simulator.get(), kUTTypePNG, kUTTypeFileURL);
    280268    checkEstimatedSize(simulator.get(), { 215, 174 });
     
    321309
    322310    EXPECT_WK_STREQ("https://www.apple.com/", [webView editorValue].UTF8String);
    323     checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 241, 2057, 232) ], [simulator finalSelectionRects]);
     311    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(2156, 241, 2, 232), [simulator finalSelectionStartRect]);
    324312    checkSuggestedNameAndEstimatedSize(simulator.get(), @"icon.png", { 215, 174 });
    325313    checkTypeIdentifierIsRegisteredAtIndex(simulator.get(), (__bridge NSString *)kUTTypePNG, 0);
     
    391379    EXPECT_TRUE([observedEventNames containsObject:@"dragover"]);
    392380    EXPECT_TRUE([observedEventNames containsObject:@"drop"]);
    393     checkSelectionRectsWithLogging(@[ makeCGRectValue(1, 201, 961, 227) ], [simulator finalSelectionRects]);
     381    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(960, 201, 2, 227), [simulator finalSelectionStartRect]);
    394382    checkRichTextTypePrecedesPlainTextType(simulator.get());
    395383    EXPECT_TRUE([simulator lastKnownDropProposal].precise);
     
    415403    EXPECT_TRUE([observedEventNames containsObject:@"dragover"]);
    416404    EXPECT_TRUE([observedEventNames containsObject:@"drop"]);
    417     checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 203, 990, 232) ], [simulator finalSelectionRects]);
     405    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(1089, 203, 2, 232), [simulator finalSelectionStartRect]);
    418406    checkRichTextTypePrecedesPlainTextType(simulator.get());
    419407    EXPECT_TRUE([simulator lastKnownDropProposal].precise);
     
    472460    EXPECT_FALSE(secondParagraphOffset == NSNotFound);
    473461    EXPECT_GT(firstParagraphOffset, secondParagraphOffset);
    474     checkSelectionRectsWithLogging(@[ makeCGRectValue(190, 100, 130, 20), makeCGRectValue(0, 120, 320, 100), makeCGRectValue(0, 220, 252, 20) ], [simulator finalSelectionRects]);
     462    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(251, 220, 2, 20), [simulator finalSelectionStartRect]);
    475463    EXPECT_TRUE([simulator lastKnownDropProposal].precise);
    476464}
     
    498486    EXPECT_EQ([webView stringByEvaluatingJavaScript:@"source.value"].length, 0UL);
    499487    EXPECT_WK_STREQ("Hello world", [webView editorValue].UTF8String);
    500     checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 241, 990, 232) ], [simulator finalSelectionRects]);
     488    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(1089, 241, 2, 232), [simulator finalSelectionStartRect]);
    501489}
    502490
     
    569557    EXPECT_TRUE([observedEventNames containsObject:@"dragover"]);
    570558    EXPECT_TRUE([observedEventNames containsObject:@"drop"]);
    571     checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 273, 2057, 232) ], [simulator finalSelectionRects]);
     559    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(2156, 273, 2, 232), [simulator finalSelectionStartRect]);
    572560    checkTypeIdentifierIsRegisteredAtIndex(simulator.get(), (__bridge NSString *)kUTTypeURL, 0);
    573561}
     
    587575    EXPECT_TRUE([observedEventNames containsObject:@"dragover"]);
    588576    EXPECT_TRUE([observedEventNames containsObject:@"drop"]);
    589     checkSelectionRectsWithLogging(@[ makeCGRectValue(101, 241, 2057, 232) ], [simulator finalSelectionRects]);
     577    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(2156, 241, 2, 232), [simulator finalSelectionStartRect]);
    590578    checkTypeIdentifierIsRegisteredAtIndex(simulator.get(), (__bridge NSString *)kUTTypeURL, 0);
    591579}
     
    605593    EXPECT_FALSE([observedEventNames containsObject:@"dragenter"]);
    606594    EXPECT_FALSE([observedEventNames containsObject:@"dragover"]);
    607     checkSelectionRectsWithLogging(@[ ], [simulator finalSelectionRects]);
     595    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(0, 0, 0, 0), [simulator finalSelectionStartRect]);
    608596}
    609597
     
    621609    EXPECT_TRUE([observedEventNames containsObject:@"dragenter"]);
    622610    EXPECT_TRUE([observedEventNames containsObject:@"dragover"]);
    623     checkSelectionRectsWithLogging(@[ ], [simulator finalSelectionRects]);
     611    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(0, 0, 0, 0), [simulator finalSelectionStartRect]);
    624612}
    625613
     
    639627    EXPECT_TRUE([observedEventNames containsObject:@"dragleave"]);
    640628    EXPECT_FALSE([observedEventNames containsObject:@"drop"]);
    641     checkSelectionRectsWithLogging(@[ ], [simulator finalSelectionRects]);
     629    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(0, 0, 0, 0), [simulator finalSelectionStartRect]);
    642630}
    643631
     
    10261014    [simulator runFrom:CGPointMake(300, 400) to:CGPointMake(100, 300)];
    10271015    EXPECT_WK_STREQ(textPayload.UTF8String, [webView stringByEvaluatingJavaScript:@"editor.textContent"].UTF8String);
    1028     checkSelectionRectsWithLogging(@[ makeCGRectValue(1, 201, 1936, 227) ], [simulator finalSelectionRects]);
     1016    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(1935, 201, 2, 227), [simulator finalSelectionStartRect]);
    10291017}
    10301018
     
    10461034    [simulator runFrom:CGPointMake(300, 400) to:CGPointMake(100, 300)];
    10471035    EXPECT_TRUE([webView editorContainsImageElement]);
    1048     checkSelectionRectsWithLogging(@[ makeCGRectValue(1, 201, 215, 174) ], [simulator finalSelectionRects]);
     1036    checkCGRectIsEqualToCGRectWithLogging(CGRectMake(214, 201, 2, 223), [simulator finalSelectionStartRect]);
    10491037}
    10501038
  • trunk/Tools/TestWebKitAPI/cocoa/DragAndDropSimulator.h

    r245778 r245803  
    109109@property (nonatomic, readonly) NSArray *sourceItemProviders;
    110110@property (nonatomic, readonly) NSArray *observedEventNames;
    111 @property (nonatomic, readonly) NSArray *finalSelectionRects;
     111@property (nonatomic, readonly) CGRect finalSelectionStartRect;
    112112@property (nonatomic, readonly) CGRect lastKnownDragCaretRect;
    113113@property (nonatomic, readonly) NSArray<UITargetedDragPreview *> *liftPreviews;
  • trunk/Tools/TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm

    r245778 r245803  
    304304    RetainPtr<NSArray> _externalItemProviders;
    305305    RetainPtr<NSArray> _sourceItemProviders;
    306     RetainPtr<NSArray> _finalSelectionRects;
     306    CGRect _finalSelectionStartRect;
    307307    CGPoint _startLocation;
    308308    CGPoint _endLocation;
     
    382382    _insertedAttachments = adoptNS([[NSMutableArray alloc] init]);
    383383    _removedAttachments = adoptNS([[NSMutableArray alloc] init]);
    384     _finalSelectionRects = @[ ];
     384    _finalSelectionStartRect = CGRectNull;
    385385    _dragSession = nil;
    386386    _dropSession = nil;
     
    464464    Util::run(&_isDoneWaitingForDelayedDropPreviews);
    465465    [_webView clearMessageHandlers:dragAndDropEventNames()];
    466     _finalSelectionRects = [_webView selectionRectsAfterPresentationUpdate];
     466    [_webView waitForNextPresentationUpdate];
     467
     468    auto contentView = [_webView textInputContentView];
     469    _finalSelectionStartRect = [contentView caretRectForPosition:contentView.selectedTextRange.start];
    467470
    468471    [defaultCenter removeObserver:self];
    469 }
    470 
    471 - (NSArray *)finalSelectionRects
    472 {
    473     return _finalSelectionRects.get();
    474472}
    475473
Note: See TracChangeset for help on using the changeset viewer.