Changeset 223726 in webkit


Ignore:
Timestamp:
Oct 19, 2017 4:34:16 PM (6 years ago)
Author:
n_wang@apple.com
Message:

AX: Provide a way for Accessibility to cache the selection while retrieving rects for speak selection
https://bugs.webkit.org/show_bug.cgi?id=176247
<rdar://problem/34217143>

Reviewed by Ryosuke Niwa.

Source/WebKit:

When getting the rects for highlighting the spoken text within a selection range on iOS, we can get a
list of totally wrong rects if the user changed the selection to some other text. This is because the
calculation is based on the current selection range. Therefore, we need to provide a way for accessibility
codepath to store the selection during a speaking session.

  • UIProcess/API/Cocoa/WKWebView.mm:

(-[WKWebView _accessibilityStoreSelection]):
(-[WKWebView _accessibilityClearSelection]):

  • UIProcess/API/Cocoa/WKWebViewPrivate.h:
  • UIProcess/WebPageProxy.h:
  • UIProcess/ios/WKContentViewInteraction.h:
  • UIProcess/ios/WKContentViewInteraction.mm:

(-[WKContentView _accessibilityStoreSelection]):
(-[WKContentView _accessibilityClearSelection]):

  • UIProcess/ios/WebPageProxyIOS.mm:

(WebKit::WebPageProxy::storeSelectionForAccessibility):

  • WebProcess/WebPage/WebPage.h:
  • WebProcess/WebPage/WebPage.messages.in:
  • WebProcess/WebPage/ios/WebPageIOS.mm:

(WebKit::WebPage::getRectsForGranularityWithSelectionOffset):
(WebKit::WebPage::storeSelectionForAccessibility):
(WebKit::WebPage::getRectsAtSelectionOffsetWithText):

Tools:

  • TestWebKitAPI/Tests/ios/AccessibilityTestsIOS.mm:

(TestWebKitAPI::TEST):

Location:
trunk
Files:
12 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebKit/ChangeLog

    r223720 r223726  
     12017-10-19  Nan Wang  <n_wang@apple.com>
     2
     3        AX: Provide a way for Accessibility to cache the selection while retrieving rects for speak selection
     4        https://bugs.webkit.org/show_bug.cgi?id=176247
     5        <rdar://problem/34217143>
     6
     7        Reviewed by Ryosuke Niwa.
     8
     9        When getting the rects for highlighting the spoken text within a selection range on iOS, we can get a
     10        list of totally wrong rects if the user changed the selection to some other text. This is because the
     11        calculation is based on the current selection range. Therefore, we need to provide a way for accessibility
     12        codepath to store the selection during a speaking session.
     13
     14        * UIProcess/API/Cocoa/WKWebView.mm:
     15        (-[WKWebView _accessibilityStoreSelection]):
     16        (-[WKWebView _accessibilityClearSelection]):
     17        * UIProcess/API/Cocoa/WKWebViewPrivate.h:
     18        * UIProcess/WebPageProxy.h:
     19        * UIProcess/ios/WKContentViewInteraction.h:
     20        * UIProcess/ios/WKContentViewInteraction.mm:
     21        (-[WKContentView _accessibilityStoreSelection]):
     22        (-[WKContentView _accessibilityClearSelection]):
     23        * UIProcess/ios/WebPageProxyIOS.mm:
     24        (WebKit::WebPageProxy::storeSelectionForAccessibility):
     25        * WebProcess/WebPage/WebPage.h:
     26        * WebProcess/WebPage/WebPage.messages.in:
     27        * WebProcess/WebPage/ios/WebPageIOS.mm:
     28        (WebKit::WebPage::getRectsForGranularityWithSelectionOffset):
     29        (WebKit::WebPage::storeSelectionForAccessibility):
     30        (WebKit::WebPage::getRectsAtSelectionOffsetWithText):
     31
    1322017-10-19  Sam Weinig  <sam@webkit.org>
    233
  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm

    r223439 r223726  
    55325532}
    55335533
     5534- (void)_accessibilityStoreSelection
     5535{
     5536    [_contentView _accessibilityStoreSelection];
     5537}
     5538
     5539- (void)_accessibilityClearSelection
     5540{
     5541    [_contentView _accessibilityClearSelection];
     5542}
     5543
    55345544- (CGRect)_contentVisibleRect
    55355545{
  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h

    r223439 r223726  
    393393- (void)_requestActivatedElementAtPosition:(CGPoint)position completionBlock:(void (^)(_WKActivatedElementInfo *))block WK_API_AVAILABLE(ios(11.0));
    394394- (void)_accessibilityRetrieveRectsAtSelectionOffset:(NSInteger)offset withText:(NSString *)text completionHandler:(void (^)(NSArray<NSValue *> *rects))completionHandler WK_API_AVAILABLE(ios(WK_IOS_TBA));
     395- (void)_accessibilityStoreSelection WK_API_AVAILABLE(ios(WK_IOS_TBA));
     396- (void)_accessibilityClearSelection WK_API_AVAILABLE(ios(WK_IOS_TBA));
    395397
    396398#else
  • trunk/Source/WebKit/UIProcess/WebPageProxy.h

    r223436 r223726  
    563563    void requestRectsAtSelectionOffsetWithText(int32_t offset, const String&, WTF::Function<void(const Vector<WebCore::SelectionRect>&, CallbackBase::Error)>&&);
    564564    void autofillLoginCredentials(const String& username, const String& password);
     565    void storeSelectionForAccessibility(bool);
    565566#if ENABLE(DATA_INTERACTION)
    566567    void didPerformDataInteractionControllerOperation(bool handled);
  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h

    r221870 r223726  
    313313- (void)_accessibilityRetrieveRectsAtSelectionOffset:(NSInteger)offset withText:(NSString *)text completionHandler:(void (^)(const Vector<WebCore::SelectionRect>& rects))completionHandler;
    314314- (void)_accessibilityRetrieveRectsAtSelectionOffset:(NSInteger)offset withText:(NSString *)text;
     315- (void)_accessibilityStoreSelection;
     316- (void)_accessibilityClearSelection;
    315317
    316318@property (nonatomic, readonly) WebKit::InteractionInformationAtPosition currentPositionInformation;
  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm

    r223647 r223726  
    23062306            [view _accessibilityDidGetSelectionRects:[view webSelectionRectsForSelectionRects:selectionRects] withGranularity:UITextGranularityWord atOffset:offset];
    23072307    });
     2308}
     2309
     2310- (void)_accessibilityStoreSelection
     2311{
     2312    _page->storeSelectionForAccessibility(true);
     2313}
     2314
     2315- (void)_accessibilityClearSelection
     2316{
     2317    _page->storeSelectionForAccessibility(false);
    23082318}
    23092319
  • trunk/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm

    r222991 r223726  
    724724}
    725725
     726void WebPageProxy::storeSelectionForAccessibility(bool shouldStore)
     727{
     728    m_process->send(Messages::WebPage::StoreSelectionForAccessibility(shouldStore), m_pageID);
     729}
     730
    726731void WebPageProxy::moveSelectionByOffset(int32_t offset, WTF::Function<void (CallbackBase::Error)>&& callbackFunction)
    727732{
  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.h

    r223634 r223726  
    9797
    9898#if PLATFORM(COCOA)
     99#include <WebCore/VisibleSelection.h>
    99100#include <wtf/RetainPtr.h>
    100101OBJC_CLASS CALayer;
     
    138139class URL;
    139140class VisiblePosition;
    140 class VisibleSelection;
    141141enum class TextIndicatorPresentationTransition : uint8_t;
    142142enum SyntheticClickType : int8_t;
     
    587587    void getRectsForGranularityWithSelectionOffset(uint32_t, int32_t, CallbackID);
    588588    void getRectsAtSelectionOffsetWithText(int32_t, const String&, CallbackID);
     589    void storeSelectionForAccessibility(bool);
    589590#if ENABLE(IOS_TOUCH_EVENTS)
    590591    void dispatchAsynchronousTouchEvents(const Vector<WebTouchEvent, 1>& queue);
     
    15281529
    15291530    RefPtr<WebCore::Range> m_initialSelection;
     1531    WebCore::VisibleSelection m_storedSelectionForAccessibility { WebCore::VisibleSelection() };
    15301532    WebCore::IntSize m_blockSelectionDesiredSize;
    15311533    WebCore::FloatSize m_maximumUnobscuredSize;
  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.messages.in

    r222941 r223726  
    9898    HandleTwoFingerTapAtPoint(WebCore::IntPoint point, uint64_t requestID)
    9999    SetForceAlwaysUserScalable(bool userScalable)
    100     GetRectsForGranularityWithSelectionOffset(uint32_t granularity, int32_t offset, WebKit::CallbackID callbackID);
    101     GetRectsAtSelectionOffsetWithText(int32_t offset, String text, WebKit::CallbackID callbackID);
     100    GetRectsForGranularityWithSelectionOffset(uint32_t granularity, int32_t offset, WebKit::CallbackID callbackID)
     101    GetRectsAtSelectionOffsetWithText(int32_t offset, String text, WebKit::CallbackID callbackID)
     102    StoreSelectionForAccessibility(bool shouldStore)
    102103#endif
    103104
  • trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm

    r223047 r223726  
    18681868{
    18691869    Frame& frame = m_page->focusController().focusedOrMainFrame();
    1870     VisibleSelection selection = frame.selection().selection();
     1870    VisibleSelection selection = m_storedSelectionForAccessibility.isNone() ? frame.selection().selection() : m_storedSelectionForAccessibility;
    18711871    VisiblePosition selectionStart = selection.visibleStart();
    18721872
     
    18911891}
    18921892
     1893void WebPage::storeSelectionForAccessibility(bool shouldStore)
     1894{
     1895    if (!shouldStore)
     1896        m_storedSelectionForAccessibility = VisibleSelection();
     1897    else {
     1898        Frame& frame = m_page->focusController().focusedOrMainFrame();
     1899        m_storedSelectionForAccessibility = frame.selection().selection();
     1900    }
     1901}
     1902
    18931903static RefPtr<Range> rangeNearPositionMatchesText(const VisiblePosition& position, RefPtr<Range> originalRange, const String& matchText, RefPtr<Range> selectionRange)
    18941904{
     
    19021912    Frame& frame = m_page->focusController().focusedOrMainFrame();
    19031913    uint32_t length = text.length();
    1904     VisibleSelection selection = frame.selection().selection();
     1914    VisibleSelection selection = m_storedSelectionForAccessibility.isNone() ? frame.selection().selection() : m_storedSelectionForAccessibility;
    19051915    VisiblePosition selectionStart = selection.visibleStart();
    19061916    VisiblePosition selectionEnd = selection.visibleEnd();
  • trunk/Tools/ChangeLog

    r223720 r223726  
     12017-10-19  Nan Wang  <n_wang@apple.com>
     2
     3        AX: Provide a way for Accessibility to cache the selection while retrieving rects for speak selection
     4        https://bugs.webkit.org/show_bug.cgi?id=176247
     5        <rdar://problem/34217143>
     6
     7        Reviewed by Ryosuke Niwa.
     8
     9        * TestWebKitAPI/Tests/ios/AccessibilityTestsIOS.mm:
     10        (TestWebKitAPI::TEST):
     11
    1122017-10-19  Sam Weinig  <sam@webkit.org>
    213
  • trunk/Tools/TestWebKitAPI/Tests/ios/AccessibilityTestsIOS.mm

    r221233 r223726  
    9797}
    9898
     99TEST(AccessibilityTests, StoreSelection)
     100{
     101    auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]);
     102    [webView synchronouslyLoadHTMLString:@"<meta name='viewport' content='width=device-width,initial-scale=1'><span id='first'>first</span><br><span id='second'>first</span>"];
     103   
     104    // Select first node and store the selection
     105    [webView stringByEvaluatingJavaScript:@"getSelection().setBaseAndExtent(first, 0, first, 1)"];
     106    [webView _accessibilityStoreSelection];
     107    checkCGRectValueAtIndex([webView rectsAtSelectionOffset:0 withText:@"first"], CGRectMake(8, 8, 26, 19), 0);
     108    // Now select the second node, we should use the stored selection to retrieve rects
     109    [webView stringByEvaluatingJavaScript:@"getSelection().setBaseAndExtent(second, 0, second, 1)"];
     110    checkCGRectValueAtIndex([webView rectsAtSelectionOffset:0 withText:@"first"], CGRectMake(8, 8, 26, 19), 0);
     111   
     112    // Clear the stored selection, we should use the current selection to retrieve rects
     113    [webView _accessibilityClearSelection];
     114    checkCGRectValueAtIndex([webView rectsAtSelectionOffset:0 withText:@"first"], CGRectMake(8, 27, 26, 20), 0);
     115}
     116
    99117} // namespace TestWebKitAPI
    100118
Note: See TracChangeset for help on using the changeset viewer.