⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 279164 in webkit


Ignore:
Timestamp:
Jun 22, 2021, 6:32:54 PM (5 years ago)
Author:
Wenson Hsieh
Message:

[Live Text] [macOS] Add an internal option to disable inline text selection in images
​https://bugs.webkit.org/show_bug.cgi?id=227265
rdar://79460142

Reviewed by Tim Horton.

Source/WebCore:

Add support for the new feature flag. See WebKit/ChangeLog for more details.

  • en.lproj/Localizable.strings:

Add additional localizable strings to represent each of the possible titles for the "Quick Look"/"Look Up"
context menu item. When inline text selection is enabled, the item shows up as "Look Up", and is only added if
we actually discover visual search results.

However, when inline text selection is disabled, this item will always be added to the context menu under the
title "Quick Look", and will be retitled to either "Look Up in Quick Look" or "Select Text in Quick Look" once
either visual search results or text is recognized, respectively.

  • page/ContextMenuController.cpp:

(WebCore::ContextMenuController::contextMenuItemSelected):
(WebCore::ContextMenuController::populate):

In the case where inline text selection is disabled, add the Quick Look item to the end of the context menu,
after a separator. Note that in the case where inline text selection is enabled, it doesn't actually matter
where we add this item, since it is automatically removed and only reinserted if we find visual search results
for the image.

(WebCore::ContextMenuController::checkOrEnableIfNeeded const):

Rename the existing ContextMenuItemTagLookUpImage to ContextMenuItemTagQuickLookImage, since it is no longer
limited to the visual look up feature.

  • page/EventHandler.cpp:

(WebCore::EventHandler::updateMouseEventTargetNode):

Consult the feature flag and avoid kicking off the text recognition timer if inline text selection is disabled.

  • platform/ContextMenuItem.cpp:

(WebCore::isValidContextMenuAction):

  • platform/ContextMenuItem.h:
  • platform/LocalizedStrings.h:
  • platform/cocoa/LocalizedStringsCocoa.mm:

(WebCore::contextMenuItemTagQuickLookImage):
(WebCore::contextMenuItemTagQuickLookImageForTextSelection):
(WebCore::contextMenuItemTagQuickLookImageForVisualSearch):

Source/WebKit:

Refactor Live Text and Visual Look Up logic, such that the ability to select text inside of images is behind an
internal feature flag that can be toggled at runtime. When Live Text is disabled, we'll unconditionally show the
"Look Up" context menu action using the more generic title "Quick Look". This "Quick Look" action is then
retitled to either "Look Up in Quick Look" or "Select Text in Quick Look" once image analysis completes.

  • Shared/API/c/WKSharedAPICast.h:

(WebKit::toAPI):
(WebKit::toImpl):

Rename ContextMenuItemTagLookUpImage to ContextMenuItemTagQuickLookImage.

  • UIProcess/Cocoa/QuickLookPreviewActivity.h: Copied from Source/WebKit/UIProcess/mac/WKQuickLookPreviewController.h.

Move this enum out of WKQuickLookPreviewController.h, such that we can use and import it in non-Cocoa headers
and implementation files.

  • UIProcess/Cocoa/WebViewImpl.h:
  • UIProcess/Cocoa/WebViewImpl.mm:

(WebKit::WebViewImpl::computeHasImageAnalysisResults):

Rename computeHasVisualSearchResults to computeHasImageAnalysisResults, and allow it to take a flag to
indicate whether or not it should request text recognition results or visual search results. We use this for the
"Quick Look" context menu action when inline text selection is disabled, in order to retitle the item to "Select
Text in Quick Look".

(WebKit::WebViewImpl::computeHasVisualSearchResults): Deleted.

  • UIProcess/PageClient.h:

(WebKit::PageClient::computeHasImageAnalysisResults):
(WebKit::PageClient::computeHasVisualSearchResults): Deleted.

  • UIProcess/WebContextMenuProxy.h:

(WebKit::WebContextMenuProxy::quickLookPreviewActivity const):

Add a flag to WebContextMenuProxy to determine what initial activity to use when presenting the QuickLook
preview panel after selecting the ContextMenuItemTagQuickLookImage item.

  • UIProcess/WebPageProxy.cpp:

(WebKit::WebPageProxy::contextMenuItemSelected):
(WebKit::WebPageProxy::computeHasImageAnalysisResults):
(WebKit::WebPageProxy::computeHasVisualSearchResults): Deleted.

  • UIProcess/WebPageProxy.h:
  • UIProcess/mac/PageClientImplMac.h:
  • UIProcess/mac/PageClientImplMac.mm:

(WebKit::PageClientImpl::computeHasImageAnalysisResults):
(WebKit::PageClientImpl::computeHasVisualSearchResults): Deleted.

  • UIProcess/mac/WKQuickLookPreviewController.h:
  • UIProcess/mac/WKQuickLookPreviewController.mm:
  • UIProcess/mac/WebContextMenuProxyMac.h:
  • UIProcess/mac/WebContextMenuProxyMac.mm:

(WebKit::menuItemIdentifier):
(WebKit::WebContextMenuProxyMac::getContextMenuFromItems):

Add logic to either remove and reinsert the "Look Up" context menu action (if inline text selection in images is
enabled), or to promote the "Quick Look" context menu action to either "Select Text in Quick Look" or "Look Up
in Quick Look" in the case where inline text selection in images is disabled.

Note that when inline text selection in images is disabled, we'll first check for visual search results before
checking for text. This is because visual search results will take precedence over text selection (i.e., we'll
show "Look Up in Quick Look" even if both visual search results and text is present), so we can avoid performing
text analysis in the case where we already found visual search results.

(WebKit::WebContextMenuProxyMac::insertOrUpdateQuickLookImageItem):
(WebKit::WebContextMenuProxyMac::updateQuickLookContextMenuItemTitle):

Add a helper method to find and update the title of the NSMenuItem corresponding to
ContextMenuItemTagQuickLookImage. See getContextMenuFromItems above for more details.

  • UIProcess/mac/WebPageProxyMac.mm:

(WebKit::WebPageProxy::handleContextMenuQuickLookImage):
(WebKit::WebPageProxy::handleContextMenuLookUpImage): Deleted.

Rename this to handleContextMenuQuickLookImage, and have it take an argument to indicate whether the QuickLook
panel should start out in Visual Look Up mode.

  • WebKit.xcodeproj/project.pbxproj:

Source/WebKitLegacy/mac:

Rename ContextMenuItemTagLookUpImage to ContextMenuItemTagQuickLookImage.

  • WebView/WebHTMLView.mm:

(toTag):

Source/WTF:

Add a new internal feature flag. See WebKit ChangeLog for more details.

  • Scripts/Preferences/WebPreferencesInternal.yaml:
Location:
trunk/Source
Files:
28 edited
1 copied

Legend:

Unmodified
Added
Removed

  • trunk/Source/WTF/ChangeLog

    r279137 r279164  
     12021-06-22  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Live Text] [macOS] Add an internal option to disable inline text selection in images
     4        https://bugs.webkit.org/show_bug.cgi?id=227265
     5        rdar://79460142
     6
     7        Reviewed by Tim Horton.
     8
     9        Add a new internal feature flag. See WebKit ChangeLog for more details.
     10
     11        * Scripts/Preferences/WebPreferencesInternal.yaml:
     12
    1132021-06-22  Fujii Hironori  <Hironori.Fujii@sony.com>
    214

  • trunk/Source/WTF/Scripts/Preferences/WebPreferencesInternal.yaml

    r277564 r279164  
    585585      default: false
    586586
     587PreferInlineTextSelectionInImages:
     588  type: bool
     589  humanReadableName: "Inline Text Selection in Images"
     590  humanReadableDescription: "Prefer inline text selection in images over using the context menu"
     591  condition: ENABLE(IMAGE_ANALYSIS)
     592  defaultValue:
     593    WebCore:
     594      default: true
     595    WebKitLegacy:
     596      default: true
     597    WebKit:
     598      default: true
     599
    587600PreferPageRenderingUpdatesNear60FPSEnabled:
    588601  type: bool

  • trunk/Source/WebCore/ChangeLog

    r279148 r279164  
     12021-06-22  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Live Text] [macOS] Add an internal option to disable inline text selection in images
     4        https://bugs.webkit.org/show_bug.cgi?id=227265
     5        rdar://79460142
     6
     7        Reviewed by Tim Horton.
     8
     9        Add support for the new feature flag. See WebKit/ChangeLog for more details.
     10
     11        * en.lproj/Localizable.strings:
     12
     13        Add additional localizable strings to represent each of the possible titles for the "Quick Look"/"Look Up"
     14        context menu item. When inline text selection is enabled, the item shows up as "Look Up", and is only added if
     15        we actually discover visual search results.
     16
     17        However, when inline text selection is disabled, this item will always be added to the context menu under the
     18        title "Quick Look", and will be retitled to either "Look Up in Quick Look" or "Select Text in Quick Look" once
     19        either visual search results or text is recognized, respectively.
     20
     21        * page/ContextMenuController.cpp:
     22        (WebCore::ContextMenuController::contextMenuItemSelected):
     23        (WebCore::ContextMenuController::populate):
     24
     25        In the case where inline text selection is disabled, add the Quick Look item to the end of the context menu,
     26        after a separator. Note that in the case where inline text selection is enabled, it doesn't actually matter
     27        where we add this item, since it is automatically removed and only reinserted if we find visual search results
     28        for the image.
     29
     30        (WebCore::ContextMenuController::checkOrEnableIfNeeded const):
     31
     32        Rename the existing ContextMenuItemTagLookUpImage to ContextMenuItemTagQuickLookImage, since it is no longer
     33        limited to the visual look up feature.
     34
     35        * page/EventHandler.cpp:
     36        (WebCore::EventHandler::updateMouseEventTargetNode):
     37
     38        Consult the feature flag and avoid kicking off the text recognition timer if inline text selection is disabled.
     39
     40        * platform/ContextMenuItem.cpp:
     41        (WebCore::isValidContextMenuAction):
     42        * platform/ContextMenuItem.h:
     43        * platform/LocalizedStrings.h:
     44        * platform/cocoa/LocalizedStringsCocoa.mm:
     45        (WebCore::contextMenuItemTagQuickLookImage):
     46        (WebCore::contextMenuItemTagQuickLookImageForTextSelection):
     47        (WebCore::contextMenuItemTagQuickLookImageForVisualSearch):
     48
    1492021-06-22  Devin Rousso  <drousso@apple.com>
    250

  • trunk/Source/WebCore/en.lproj/Localizable.strings

    r278767 r279164  
    578578"Look Up in Dictionary" = "Look Up in Dictionary";
    579579
     580/* Title for Quick Look action button for images with visual search results */
     581"Look Up in Quick Look" = "Look Up in Quick Look";
     582
    580583/* Look Up context menu item with selected word */
    581584"Look Up “%@”" = "Look Up “%@”";
    … …  
    752755"PostScript" = "PostScript";
    753756
     757/* Title for Quick Look action button */
     758"Quick Look" = "Quick Look";
     759
    754760/* Undo action name */
    755761"Raise Baseline (Undo action name)" = "Raise Baseline";
    … …  
    794800"Search with %@" = "Search with %@";
    795801
     802/* Title for Quick Look action button for images with text */
     803"Select Text in Quick Look" = "Select Text in Quick Look";
     804
    796805/* Validation message for required file inputs that have no value */
    797806"Select a file" = "Select a file";

  • trunk/Source/WebCore/page/ContextMenuController.cpp

    r279098 r279164  
    516516        frame->editor().applyDictationAlternative(title);
    517517        break;
    518     case ContextMenuItemTagLookUpImage:
     518    case ContextMenuItemTagQuickLookImage:
    519519        // This should be handled at the client layer.
    520520        ASSERT_NOT_REACHED();
    … …  
    825825    ContextMenuItem InsertEmojiItem(ActionType, ContextMenuItemTagInsertEmoji, contextMenuItemTagInsertEmoji());
    826826#endif
    827 #if ENABLE(IMAGE_ANALYSIS)
    828     ContextMenuItem LookUpImageItem(ActionType, ContextMenuItemTagLookUpImage, contextMenuItemTagLookUpImage());
    829 #endif
    830827
    831828#if PLATFORM(GTK) || PLATFORM(WIN)
    … …  
    850847    if (m_context.controlledImage())
    851848        return;
     849#endif
     850
     851#if ENABLE(IMAGE_ANALYSIS)
     852    bool shouldAppendQuickLookImageItem = false;
     853    auto quickLookItemTitle = frame->settings().preferInlineTextSelectionInImages() ? contextMenuItemTagLookUpImage() : contextMenuItemTagQuickLookImage();
     854    ContextMenuItem QuickLookImageItem { ActionType, ContextMenuItemTagQuickLookImage, quickLookItemTitle };
    852855#endif
    853856
    … …  
    901904#if ENABLE(IMAGE_ANALYSIS)
    902905                if (m_client.supportsLookUpInImages() && image && !image->isAnimated())
    903                     appendItem(LookUpImageItem, m_contextMenu.get());
     906                    shouldAppendQuickLookImageItem = true;
    904907#endif
    905908            }
    … …  
    11651168        }
    11661169    }
     1170
     1171#if ENABLE(IMAGE_ANALYSIS)
     1172    if (shouldAppendQuickLookImageItem) {
     1173        if (!frame->settings().preferInlineTextSelectionInImages()) {
     1174            // In the case where inline text selection is enabled, the Look Up item is only added if
     1175            // we discover visual look up results after image analysis. In that scenario, a separator
     1176            // is only added before the Look Up item once we're certain that we want to show it.
     1177            appendItem(*separatorItem(), m_contextMenu.get());
     1178        }
     1179        appendItem(QuickLookImageItem, m_contextMenu.get());
     1180    }
     1181#endif // ENABLE(IMAGE_ANALYSIS)
    11671182}
    11681183
    … …  
    14901505            shouldCheck = shouldEnable &&  m_context.hitTestResult().mediaMuted();
    14911506            break;
    1492         case ContextMenuItemTagLookUpImage:
     1507        case ContextMenuItemTagQuickLookImage:
    14931508        case ContextMenuItemTagTranslate:
    14941509            break;

  • trunk/Source/WebCore/page/EventHandler.cpp

    r279066 r279164  
    25382538
    25392539#if ENABLE(IMAGE_ANALYSIS)
    2540     if (!m_elementUnderMouse || !is<RenderImage>(m_elementUnderMouse->renderer()))
    2541         m_textRecognitionHoverTimer.stop();
    2542     else if (!platformMouseEvent.movementDelta().isZero())
    2543         m_textRecognitionHoverTimer.restart();
     2540    if (m_frame.settings().preferInlineTextSelectionInImages()) {
     2541        if (!m_elementUnderMouse || !is<RenderImage>(m_elementUnderMouse->renderer()))
     2542            m_textRecognitionHoverTimer.stop();
     2543        else if (!platformMouseEvent.movementDelta().isZero())
     2544            m_textRecognitionHoverTimer.restart();
     2545    }
    25442546#endif // ENABLE(IMAGE_ANALYSIS)
    25452547

  • trunk/Source/WebCore/platform/ContextMenuItem.cpp

    r278767 r279164  
    259259    case ContextMenuAction::ContextMenuItemTagShareMenu:
    260260    case ContextMenuAction::ContextMenuItemTagToggleVideoEnhancedFullscreen:
    261     case ContextMenuAction::ContextMenuItemTagLookUpImage:
     261    case ContextMenuAction::ContextMenuItemTagQuickLookImage:
    262262    case ContextMenuAction::ContextMenuItemTagTranslate:
    263263    case ContextMenuAction::ContextMenuItemBaseCustomTag:

  • trunk/Source/WebCore/platform/ContextMenuItem.h

    r278767 r279164  
    148148    ContextMenuItemTagAddHighlightToCurrentQuickNote,
    149149    ContextMenuItemTagAddHighlightToNewQuickNote,
    150     ContextMenuItemTagLookUpImage,
     150    ContextMenuItemTagQuickLookImage,
    151151    ContextMenuItemTagTranslate,
    152152    ContextMenuItemBaseCustomTag = 5000,

  • trunk/Source/WebCore/platform/LocalizedStrings.h

    r278767 r279164  
    369369#if ENABLE(IMAGE_ANALYSIS)
    370370    WEBCORE_EXPORT String contextMenuItemTagLookUpImage();
    371 #endif
     371    WEBCORE_EXPORT String contextMenuItemTagQuickLookImage();
     372    WEBCORE_EXPORT String contextMenuItemTagQuickLookImageForTextSelection();
     373    WEBCORE_EXPORT String contextMenuItemTagQuickLookImageForVisualSearch();
     374#endif // ENABLE(IMAGE_ANALYSIS)
    372375
    373376#if HAVE(TRANSLATION_UI_SERVICES)

  • trunk/Source/WebCore/platform/cocoa/LocalizedStringsCocoa.mm

    r278767 r279164  
    320320}
    321321
    322 #endif
     322String contextMenuItemTagQuickLookImage()
     323{
     324    return WEB_UI_STRING("Quick Look", "Title for Quick Look action button");
     325}
     326
     327String contextMenuItemTagQuickLookImageForTextSelection()
     328{
     329    return WEB_UI_STRING("Select Text in Quick Look", "Title for Quick Look action button for images with text");
     330}
     331
     332String contextMenuItemTagQuickLookImageForVisualSearch()
     333{
     334    return WEB_UI_STRING("Look Up in Quick Look", "Title for Quick Look action button for images with visual search results");
     335}
     336
     337#endif // ENABLE(IMAGE_ANALYSIS)
    323338
    324339} // namespace WebCore

  • trunk/Source/WebKit/ChangeLog

    r279155 r279164  
     12021-06-22  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Live Text] [macOS] Add an internal option to disable inline text selection in images
     4        https://bugs.webkit.org/show_bug.cgi?id=227265
     5        rdar://79460142
     6
     7        Reviewed by Tim Horton.
     8
     9        Refactor Live Text and Visual Look Up logic, such that the ability to select text inside of images is behind an
     10        internal feature flag that can be toggled at runtime. When Live Text is disabled, we'll unconditionally show the
     11        "Look Up" context menu action using the more generic title "Quick Look". This "Quick Look" action is then
     12        retitled to either "Look Up in Quick Look" or "Select Text in Quick Look" once image analysis completes.
     13
     14        * Shared/API/c/WKSharedAPICast.h:
     15        (WebKit::toAPI):
     16        (WebKit::toImpl):
     17
     18        Rename ContextMenuItemTagLookUpImage to ContextMenuItemTagQuickLookImage.
     19
     20        * UIProcess/Cocoa/QuickLookPreviewActivity.h: Copied from Source/WebKit/UIProcess/mac/WKQuickLookPreviewController.h.
     21
     22        Move this enum out of WKQuickLookPreviewController.h, such that we can use and import it in non-Cocoa headers
     23        and implementation files.
     24
     25        * UIProcess/Cocoa/WebViewImpl.h:
     26        * UIProcess/Cocoa/WebViewImpl.mm:
     27        (WebKit::WebViewImpl::computeHasImageAnalysisResults):
     28
     29        Rename `computeHasVisualSearchResults` to `computeHasImageAnalysisResults`, and allow it to take a flag to
     30        indicate whether or not it should request text recognition results or visual search results. We use this for the
     31        "Quick Look" context menu action when inline text selection is disabled, in order to retitle the item to "Select
     32        Text in Quick Look".
     33
     34        (WebKit::WebViewImpl::computeHasVisualSearchResults): Deleted.
     35        * UIProcess/PageClient.h:
     36        (WebKit::PageClient::computeHasImageAnalysisResults):
     37        (WebKit::PageClient::computeHasVisualSearchResults): Deleted.
     38        * UIProcess/WebContextMenuProxy.h:
     39        (WebKit::WebContextMenuProxy::quickLookPreviewActivity const):
     40
     41        Add a flag to WebContextMenuProxy to determine what initial activity to use when presenting the QuickLook
     42        preview panel after selecting the ContextMenuItemTagQuickLookImage item.
     43
     44        * UIProcess/WebPageProxy.cpp:
     45        (WebKit::WebPageProxy::contextMenuItemSelected):
     46        (WebKit::WebPageProxy::computeHasImageAnalysisResults):
     47        (WebKit::WebPageProxy::computeHasVisualSearchResults): Deleted.
     48        * UIProcess/WebPageProxy.h:
     49        * UIProcess/mac/PageClientImplMac.h:
     50        * UIProcess/mac/PageClientImplMac.mm:
     51        (WebKit::PageClientImpl::computeHasImageAnalysisResults):
     52        (WebKit::PageClientImpl::computeHasVisualSearchResults): Deleted.
     53        * UIProcess/mac/WKQuickLookPreviewController.h:
     54        * UIProcess/mac/WKQuickLookPreviewController.mm:
     55        * UIProcess/mac/WebContextMenuProxyMac.h:
     56        * UIProcess/mac/WebContextMenuProxyMac.mm:
     57        (WebKit::menuItemIdentifier):
     58        (WebKit::WebContextMenuProxyMac::getContextMenuFromItems):
     59
     60        Add logic to either remove and reinsert the "Look Up" context menu action (if inline text selection in images is
     61        enabled), or to promote the "Quick Look" context menu action to either "Select Text in Quick Look" or "Look Up
     62        in Quick Look" in the case where inline text selection in images is disabled.
     63
     64        Note that when inline text selection in images is disabled, we'll first check for visual search results before
     65        checking for text. This is because visual search results will take precedence over text selection (i.e., we'll
     66        show "Look Up in Quick Look" even if both visual search results and text is present), so we can avoid performing
     67        text analysis in the case where we already found visual search results.
     68
     69        (WebKit::WebContextMenuProxyMac::insertOrUpdateQuickLookImageItem):
     70        (WebKit::WebContextMenuProxyMac::updateQuickLookContextMenuItemTitle):
     71
     72        Add a helper method to find and update the title of the NSMenuItem corresponding to
     73        ContextMenuItemTagQuickLookImage. See `getContextMenuFromItems` above for more details.
     74
     75        * UIProcess/mac/WebPageProxyMac.mm:
     76        (WebKit::WebPageProxy::handleContextMenuQuickLookImage):
     77        (WebKit::WebPageProxy::handleContextMenuLookUpImage): Deleted.
     78
     79        Rename this to `handleContextMenuQuickLookImage`, and have it take an argument to indicate whether the QuickLook
     80        panel should start out in Visual Look Up mode.
     81
     82        * WebKit.xcodeproj/project.pbxproj:
     83
    1842021-06-22  Chris Dumez  <cdumez@apple.com>
    285

  • trunk/Source/WebKit/Shared/API/c/WKSharedAPICast.h

    r278767 r279164  
    544544    case WebCore::ContextMenuItemTagShareMenu:
    545545        return kWKContextMenuItemTagShareMenu;
    546     case WebCore::ContextMenuItemTagLookUpImage:
     546    case WebCore::ContextMenuItemTagQuickLookImage:
    547547        return kWKContextMenuItemTagRevealImage;
    548548    case WebCore::ContextMenuItemTagTranslate:
    … …  
    747747#endif
    748748    case kWKContextMenuItemTagRevealImage:
    749         return WebCore::ContextMenuItemTagLookUpImage;
     749        return WebCore::ContextMenuItemTagQuickLookImage;
    750750    case kWKContextMenuItemTagTranslate:
    751751        return WebCore::ContextMenuItemTagTranslate;

  • trunk/Source/WebKit/UIProcess/Cocoa/QuickLookPreviewActivity.h

    r279158 r279164  
    2626#pragma once
    2727
    28 #if HAVE(QUICKLOOK_PREVIEW_ITEM_DATA_PROVIDER)
    29 
    30 #import <Foundation/Foundation.h>
     28#if PLATFORM(COCOA)
    3129
    3230namespace WebKit {
    33 class WebPageProxy;
    34 enum class QuickLookPreviewActivity : uint8_t { None, VisualSearch };
    35 }
    3631
    37 @class QLPreviewPanel;
     32enum class QuickLookPreviewActivity : uint8_t {
     33    None,
     34    VisualSearch,
     35};
    3836
    39 @interface WKQuickLookPreviewController : NSObject
    40 - (instancetype)initWithPage:(WebKit::WebPageProxy&)page imageData:(NSData *)data title:(NSString *)title imageURL:(NSURL *)imageURL activity:(WebKit::QuickLookPreviewActivity)activity;
    41 - (void)beginControl:(QLPreviewPanel *)panel;
    42 - (BOOL)isControlling:(QLPreviewPanel *)panel;
    43 - (void)endControl:(QLPreviewPanel *)panel;
    44 - (void)closePanelIfNecessary;
    45 @end
     37} // namespace WebKit
    4638
    47 #endif // HAVE(QUICKLOOK_PREVIEW_ITEM_DATA_PROVIDER)
     39#endif // PLATFORM(COCOA)

  • trunk/Source/WebKit/UIProcess/Cocoa/WebViewImpl.h

    r278575 r279164  
    169169
    170170enum class ContinueUnsafeLoad : bool;
     171enum class ImageAnalysisType : uint8_t;
    171172enum class UndoOrRedo : bool;
    172173
    … …  
    599600#if ENABLE(IMAGE_ANALYSIS)
    600601    void requestTextRecognition(const URL& imageURL, const ShareableBitmap::Handle& imageData, CompletionHandler<void(WebCore::TextRecognitionResult&&)>&&);
    601     void computeHasVisualSearchResults(const URL& imageURL, ShareableBitmap& imageBitmap, CompletionHandler<void(bool)>&&);
     602    void computeHasImageAnalysisResults(const URL& imageURL, ShareableBitmap& imageBitmap, ImageAnalysisType, CompletionHandler<void(bool)>&&);
    602603#endif
    603604

  • trunk/Source/WebKit/UIProcess/Cocoa/WebViewImpl.mm

    r279060 r279164  
    231231}
    232232
    233 void WebViewImpl::computeHasVisualSearchResults(const URL& imageURL, ShareableBitmap& imageBitmap, CompletionHandler<void(bool)>&& completion)
     233void WebViewImpl::computeHasImageAnalysisResults(const URL& imageURL, ShareableBitmap& imageBitmap, ImageAnalysisType type, CompletionHandler<void(bool)>&& completion)
    234234{
    235235    if (!isLiveTextAvailableAndEnabled()) {
    … …  
    239239
    240240    auto cgImage = imageBitmap.makeCGImage();
    241     auto request = createImageAnalysisRequest(cgImage.get(), imageURL, [NSURL _web_URLWithWTFString:m_page->currentURL()], VKAnalysisTypeVisualSearch);
     241    auto analysisType = type == ImageAnalysisType::VisualSearch ? VKAnalysisTypeVisualSearch : VKAnalysisTypeText;
     242    auto request = createImageAnalysisRequest(cgImage.get(), imageURL, [NSURL _web_URLWithWTFString:m_page->currentURL()], analysisType);
    242243    auto startTime = MonotonicTime::now();
    243     [ensureImageAnalyzer() processRequest:request.get() progressHandler:nil completionHandler:makeBlockPtr([completion = WTFMove(completion), startTime] (VKImageAnalysis *analysis, NSError *) mutable {
    244         BOOL result = [analysis hasResultsForAnalysisTypes:VKAnalysisTypeVisualSearch];
    245         CFRunLoopPerformBlock(CFRunLoopGetMain(), (__bridge CFStringRef)NSEventTrackingRunLoopMode, makeBlockPtr([completion = WTFMove(completion), result, startTime] () mutable {
    246             RELEASE_LOG(Images, "Image analysis completed in %.0f ms (found visual search results? %d)", (MonotonicTime::now() - startTime).milliseconds(), result);
     244    [ensureImageAnalyzer() processRequest:request.get() progressHandler:nil completionHandler:makeBlockPtr([completion = WTFMove(completion), startTime, analysisType] (VKImageAnalysis *analysis, NSError *) mutable {
     245        BOOL result = [analysis hasResultsForAnalysisTypes:analysisType];
     246        CFRunLoopPerformBlock(CFRunLoopGetMain(), (__bridge CFStringRef)NSEventTrackingRunLoopMode, makeBlockPtr([completion = WTFMove(completion), result, analysisType, startTime] () mutable {
     247            RELEASE_LOG(Images, "Image analysis completed in %.0f ms (found %s? %d)", (MonotonicTime::now() - startTime).milliseconds(), analysisType == VKAnalysisTypeVisualSearch ? "visual search results" : "text", result);
    247248            completion(result);
    248249        }).get());

  • trunk/Source/WebKit/UIProcess/PageClient.h

    r278575 r279164  
    161161enum class ContinueUnsafeLoad : bool { No, Yes };
    162162
     163#if ENABLE(IMAGE_ANALYSIS)
     164enum class ImageAnalysisType : uint8_t { Text, VisualSearch };
     165#endif
     166
    163167struct FocusedElementInformation;
    164168struct FrameInfoData;
    … …  
    541545#if ENABLE(IMAGE_ANALYSIS)
    542546    virtual void requestTextRecognition(const URL& imageURL, const ShareableBitmap::Handle& imageData, CompletionHandler<void(WebCore::TextRecognitionResult&&)>&& completion) { completion({ }); }
    543     virtual void computeHasVisualSearchResults(const URL&, ShareableBitmap&, CompletionHandler<void(bool)>&& completion) { completion(false); }
     547    virtual void computeHasImageAnalysisResults(const URL&, ShareableBitmap&, ImageAnalysisType, CompletionHandler<void(bool)>&& completion) { completion(false); }
    544548#endif
    545549

  • trunk/Source/WebKit/UIProcess/WebContextMenuProxy.h

    r274521 r279164  
    3434#include <wtf/WeakPtr.h>
    3535
     36#if PLATFORM(COCOA)
     37#include "QuickLookPreviewActivity.h"
     38#endif
     39
    3640OBJC_CLASS NSMenu;
    3741
    … …  
    5256    virtual NSMenu *platformMenu() const = 0;
    5357    virtual NSArray *platformData() const = 0;
     58    virtual QuickLookPreviewActivity quickLookPreviewActivity() const { return QuickLookPreviewActivity::None; };
    5459#endif // PLATFORM(COCOA)
    5560

  • trunk/Source/WebKit/UIProcess/WebPageProxy.cpp

    r279151 r279164  
    213213#if PLATFORM(COCOA)
    214214#include "InsertTextOptions.h"
     215#include "QuickLookPreviewActivity.h"
    215216#include "RemoteLayerTreeDrawingAreaProxy.h"
    216217#include "RemoteLayerTreeScrollingPerformanceData.h"
    … …  
    69066907        break;
    69076908
    6908     case ContextMenuItemTagLookUpImage:
     6909    case ContextMenuItemTagQuickLookImage:
    69096910#if ENABLE(IMAGE_ANALYSIS)
    6910         handleContextMenuLookUpImage();
     6911        if (m_activeContextMenu)
     6912            handleContextMenuQuickLookImage(m_activeContextMenu->quickLookPreviewActivity());
    69116913#endif
    69126914        return;
    … …  
    85008502
    85018503#if ENABLE(IMAGE_ANALYSIS)
     8504
    85028505void WebPageProxy::requestTextRecognition(const URL& imageURL, const ShareableBitmap::Handle& imageData, CompletionHandler<void(WebCore::TextRecognitionResult&&)>&& completionHandler)
    85038506{
    … …  
    85058508}
    85068509
    8507 void WebPageProxy::computeHasVisualSearchResults(const URL& imageURL, ShareableBitmap& imageBitmap, CompletionHandler<void(bool)>&& completion)
    8508 {
    8509     pageClient().computeHasVisualSearchResults(imageURL, imageBitmap, WTFMove(completion));
     8510void WebPageProxy::computeHasImageAnalysisResults(const URL& imageURL, ShareableBitmap& imageBitmap, ImageAnalysisType type, CompletionHandler<void(bool)>&& completion)
     8511{
     8512    pageClient().computeHasImageAnalysisResults(imageURL, imageBitmap, type, WTFMove(completion));
    85108513}
    85118514
    … …  
    85198522    sendWithAsyncReply(Messages::WebPage::UpdateWithTextRecognitionResult(WTFMove(results), context, location), WTFMove(completionHandler));
    85208523}
    8521 #endif
     8524
     8525#endif // ENABLE(IMAGE_ANALYSIS)
    85228526
    85238527#if ENABLE(ENCRYPTED_MEDIA)

  • trunk/Source/WebKit/UIProcess/WebPageProxy.h

    r279060 r279164  
    403403struct WebSpeechSynthesisVoice;
    404404
     405enum class ImageAnalysisType : uint8_t;
    405406enum class TextRecognitionUpdateResult : uint8_t;
    406407enum class NegotiatedLegacyTLS : bool;
    … …  
    16601661    void requestTextRecognition(const URL& imageURL, const ShareableBitmap::Handle& imageData, CompletionHandler<void(WebCore::TextRecognitionResult&&)>&&);
    16611662    void updateWithTextRecognitionResult(WebCore::TextRecognitionResult&&, const WebCore::ElementContext&, const WebCore::FloatPoint& location, CompletionHandler<void(TextRecognitionUpdateResult)>&&);
    1662     void computeHasVisualSearchResults(const URL& imageURL, ShareableBitmap& imageBitmap, CompletionHandler<void(bool)>&&);
     1663    void computeHasImageAnalysisResults(const URL& imageURL, ShareableBitmap& imageBitmap, ImageAnalysisType, CompletionHandler<void(bool)>&&);
    16631664#endif
    16641665
    … …  
    19071908#if ENABLE(IMAGE_ANALYSIS) && ENABLE(CONTEXT_MENUS)
    19081909    void handleContextMenuLookUpImage();
     1910    void handleContextMenuQuickLookImage(QuickLookPreviewActivity);
    19091911#endif
    19101912

  • trunk/Source/WebKit/UIProcess/mac/PageClientImplMac.h

    r278575 r279164  
    133133#if ENABLE(IMAGE_ANALYSIS)
    134134    void requestTextRecognition(const URL& imageURL, const ShareableBitmap::Handle& imageData, CompletionHandler<void(WebCore::TextRecognitionResult&&)>&&) override;
    135     void computeHasVisualSearchResults(const URL&, ShareableBitmap&, CompletionHandler<void(bool)>&&) override;
     135    void computeHasImageAnalysisResults(const URL&, ShareableBitmap&, ImageAnalysisType, CompletionHandler<void(bool)>&&) override;
    136136#endif
    137137

  • trunk/Source/WebKit/UIProcess/mac/PageClientImplMac.mm

    r278575 r279164  
    486486}
    487487
    488 void PageClientImpl::computeHasVisualSearchResults(const URL& imageURL, ShareableBitmap& imageBitmap, CompletionHandler<void(bool)>&& completion)
    489 {
    490     m_impl->computeHasVisualSearchResults(imageURL, imageBitmap, WTFMove(completion));
    491 }
    492 
    493 #endif
     488void PageClientImpl::computeHasImageAnalysisResults(const URL& imageURL, ShareableBitmap& imageBitmap, ImageAnalysisType type, CompletionHandler<void(bool)>&& completion)
     489{
     490    m_impl->computeHasImageAnalysisResults(imageURL, imageBitmap, type, WTFMove(completion));
     491}
     492
     493#endif // ENABLE(IMAGE_ANALYSIS)
    494494
    495495RefPtr<WebPopupMenuProxy> PageClientImpl::createPopupMenuProxy(WebPageProxy& page)

  • trunk/Source/WebKit/UIProcess/mac/WKQuickLookPreviewController.h

    r279060 r279164  
    3232namespace WebKit {
    3333class WebPageProxy;
    34 enum class QuickLookPreviewActivity : uint8_t { None, VisualSearch };
     34enum class QuickLookPreviewActivity : uint8_t;
    3535}
    3636

  • trunk/Source/WebKit/UIProcess/mac/WKQuickLookPreviewController.mm

    r279060 r279164  
    2929#if HAVE(QUICKLOOK_PREVIEW_ITEM_DATA_PROVIDER)
    3030
     31#import "QuickLookPreviewActivity.h"
    3132#import "WebPageProxy.h"
    3233#import <wtf/RetainPtr.h>

  • trunk/Source/WebKit/UIProcess/mac/WebContextMenuProxyMac.h

    r274521 r279164  
    6161    WebContextMenuProxyMac(NSView *, WebPageProxy&, ContextMenuContextData&&, const UserData&);
    6262
     63    QuickLookPreviewActivity quickLookPreviewActivity() const final { return m_quickLookPreviewActivity; }
     64
     65#if ENABLE(IMAGE_ANALYSIS)
     66    void insertOrUpdateQuickLookImageItem(const URL& imageURL, Ref<ShareableBitmap>&& imageBitmap, std::optional<WebContextMenuItemData>&&, bool);
     67    void updateQuickLookContextMenuItemTitle(const String&);
     68#endif
     69
    6370    void show() override;
    6471    void showContextMenuWithItems(Vector<Ref<WebContextMenuItem>>&&) override;
    … …  
    8087    RetainPtr<WKMenuDelegate> m_menuDelegate;
    8188    WeakObjCPtr<NSView> m_webView;
     89    QuickLookPreviewActivity m_quickLookPreviewActivity { QuickLookPreviewActivity::None };
    8290};
    8391

  • trunk/Source/WebKit/UIProcess/mac/WebContextMenuProxyMac.mm

    r278767 r279164  
    3939#import "WebContextMenuItemData.h"
    4040#import "WebPageProxy.h"
     41#import "WebPreferences.h"
    4142#import <WebCore/GraphicsContext.h>
    4243#import <WebCore/IntRect.h>
    … …  
    437438        return _WKMenuItemIdentifierReload;
    438439
    439     case ContextMenuItemTagLookUpImage:
     440    case ContextMenuItemTagQuickLookImage:
    440441        return _WKMenuItemIdentifierRevealImage;
    441442
    … …  
    508509    }
    509510
    510     std::optional<WebContextMenuItemData> lookUpImageItem;
     511    bool shouldUpdateQuickLookItemTitle = false;
     512    std::optional<WebContextMenuItemData> quickLookItemToInsertIfNeeded;
    511513
    512514#if ENABLE(IMAGE_ANALYSIS)
    513     filteredItems.removeFirstMatching([&] (auto& item) {
    514         if (item.action() != WebCore::ContextMenuItemTagLookUpImage)
    515             return false;
    516 
    517         lookUpImageItem = { item };
    518         return true;
     515    auto indexOfQuickLookItem = filteredItems.findMatching([&] (auto& item) {
     516        return item.action() == WebCore::ContextMenuItemTagQuickLookImage;
    519517    });
    520 #endif
     518
     519    if (indexOfQuickLookItem != notFound) {
     520        if (auto page = this->page(); page && page->preferences().preferInlineTextSelectionInImages()) {
     521            quickLookItemToInsertIfNeeded = filteredItems[indexOfQuickLookItem];
     522            filteredItems.remove(indexOfQuickLookItem);
     523        } else
     524            shouldUpdateQuickLookItemTitle = true;
     525    }
     526#endif // ENABLE(IMAGE_ANALYSIS)
    521527
    522528#if HAVE(TRANSLATION_UI_SERVICES)
    … …  
    532538
    533539    auto sparseMenuItems = retainPtr([NSPointerArray strongObjectsPointerArray]);
    534     auto insertMenuItem = makeBlockPtr([protectedThis = makeRef(*this), weakPage = makeWeakPtr(page()), imageURL = WTFMove(imageURL), imageBitmap = WTFMove(imageBitmap), lookUpImageItem = WTFMove(lookUpImageItem), completionHandler = WTFMove(completionHandler), itemsRemaining = filteredItems.size(), menu = WTFMove(menu), sparseMenuItems](NSMenuItem *item, NSUInteger index) mutable {
     540    auto insertMenuItem = makeBlockPtr([protectedThis = makeRef(*this), weakPage = makeWeakPtr(page()), imageURL = WTFMove(imageURL), imageBitmap = WTFMove(imageBitmap), shouldUpdateQuickLookItemTitle, quickLookItemToInsertIfNeeded = WTFMove(quickLookItemToInsertIfNeeded), completionHandler = WTFMove(completionHandler), itemsRemaining = filteredItems.size(), menu = WTFMove(menu), sparseMenuItems](NSMenuItem *item, NSUInteger index) mutable {
    535541        ASSERT(index < [sparseMenuItems count]);
    536542        ASSERT(![sparseMenuItems pointerAtIndex:index]);
    … …  
    542548
    543549        auto page = makeRefPtr(weakPage.get());
    544         if (lookUpImageItem && page && imageBitmap) {
     550        if (page && imageBitmap) {
    545551#if ENABLE(IMAGE_ANALYSIS)
    546             page->computeHasVisualSearchResults(imageURL, *imageBitmap, [protectedThis = WTFMove(protectedThis), lookUpImageItem = WTFMove(*lookUpImageItem)] (bool hasVisualSearchResults) mutable {
    547                 if (hasVisualSearchResults)
    548                     [protectedThis->m_menu addItem:createMenuActionItem(lookUpImageItem).get()];
    549             });
     552            protectedThis->insertOrUpdateQuickLookImageItem(imageURL, imageBitmap.releaseNonNull(), WTFMove(quickLookItemToInsertIfNeeded), shouldUpdateQuickLookItemTitle);
    550553#else
     554            UNUSED_PARAM(quickLookItemToInsertIfNeeded);
     555            UNUSED_PARAM(shouldUpdateQuickLookItemTitle);
    551556            UNUSED_PARAM(imageURL);
    552557#endif
    … …  
    563568    }
    564569}
     570
     571#if ENABLE(IMAGE_ANALYSIS)
     572
     573void WebContextMenuProxyMac::insertOrUpdateQuickLookImageItem(const URL& imageURL, Ref<ShareableBitmap>&& imageBitmap, std::optional<WebContextMenuItemData>&& quickLookItemToInsertIfNeeded, bool shouldUpdateQuickLookItemTitle)
     574{
     575    auto page = makeRef(*this->page());
     576    if (quickLookItemToInsertIfNeeded) {
     577        page->computeHasImageAnalysisResults(imageURL, imageBitmap.get(), ImageAnalysisType::VisualSearch, [weakThis = makeWeakPtr(*this), quickLookItemToInsertIfNeeded = WTFMove(*quickLookItemToInsertIfNeeded)] (bool hasVisualSearchResults) mutable {
     578            if (auto protectedThis = makeRefPtr(weakThis.get()); protectedThis && hasVisualSearchResults) {
     579                protectedThis->m_quickLookPreviewActivity = QuickLookPreviewActivity::VisualSearch;
     580                [protectedThis->m_menu addItem:NSMenuItem.separatorItem];
     581                [protectedThis->m_menu addItem:createMenuActionItem(quickLookItemToInsertIfNeeded).get()];
     582            }
     583        });
     584        return;
     585    }
     586
     587    if (shouldUpdateQuickLookItemTitle) {
     588        page->computeHasImageAnalysisResults(imageURL, imageBitmap.get(), ImageAnalysisType::VisualSearch, [weakThis = makeWeakPtr(*this), weakPage = makeWeakPtr(page.get()), imageURL, imageBitmap = WTFMove(imageBitmap)] (bool hasVisualSearchResults) mutable {
     589            auto protectedThis = makeRefPtr(weakThis.get());
     590            if (!protectedThis)
     591                return;
     592
     593            auto page = makeRefPtr(weakPage.get());
     594            if (!page)
     595                return;
     596
     597            if (hasVisualSearchResults) {
     598                protectedThis->m_quickLookPreviewActivity = QuickLookPreviewActivity::VisualSearch;
     599                protectedThis->updateQuickLookContextMenuItemTitle(contextMenuItemTagQuickLookImageForVisualSearch());
     600                return;
     601            }
     602
     603            page->computeHasImageAnalysisResults(imageURL, imageBitmap.get(), ImageAnalysisType::Text, [weakThis = WTFMove(weakThis), weakPage] (bool hasText) mutable {
     604                auto protectedThis = makeRefPtr(weakThis.get());
     605                if (!protectedThis)
     606                    return;
     607
     608                if (auto page = makeRefPtr(weakPage.get()); page && hasText)
     609                    protectedThis->updateQuickLookContextMenuItemTitle(contextMenuItemTagQuickLookImageForTextSelection());
     610            });
     611        });
     612    }
     613}
     614
     615void WebContextMenuProxyMac::updateQuickLookContextMenuItemTitle(const String& newTitle)
     616{
     617    for (NSInteger itemIndex = 0; itemIndex < [m_menu numberOfItems]; ++itemIndex) {
     618        auto item = [m_menu itemAtIndex:itemIndex];
     619        if (static_cast<ContextMenuAction>(item.tag) == ContextMenuItemTagQuickLookImage) {
     620            item.title = newTitle;
     621            break;
     622        }
     623    }
     624}
     625
     626#endif // ENABLE(IMAGE_ANALYSIS)
    565627
    566628void WebContextMenuProxyMac::getContextMenuItem(const WebContextMenuItemData& item, CompletionHandler<void(NSMenuItem *)>&& completionHandler)

  • trunk/Source/WebKit/UIProcess/mac/WebPageProxyMac.mm

    r279060 r279164  
    747747#if ENABLE(IMAGE_ANALYSIS)
    748748
    749 void WebPageProxy::handleContextMenuLookUpImage()
     749void WebPageProxy::handleContextMenuQuickLookImage(QuickLookPreviewActivity activity)
    750750{
    751751    auto& result = m_activeContextMenuContextData.webHitTestResultData();
    … …  
    753753        return;
    754754
    755     showImageInQuickLookPreviewPanel(*result.imageBitmap, result.toolTipText, URL { URL { }, result.absoluteImageURL }, QuickLookPreviewActivity::VisualSearch);
     755    showImageInQuickLookPreviewPanel(*result.imageBitmap, result.toolTipText, URL { URL { }, result.absoluteImageURL }, activity);
    756756}
    757757

  • trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj

    r279133 r279164  
    59965996                F44DFEB11E9E752F0038D196 /* WebIconUtilities.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; name = WebIconUtilities.mm; path = ios/WebIconUtilities.mm; sourceTree = "<group>"; };
    59975997                F4660BC125DEF08100E86598 /* PasteboardAccessIntent.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = PasteboardAccessIntent.h; sourceTree = "<group>"; };
     5998                F47A051626827A09007E5CF2 /* QuickLookPreviewActivity.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = QuickLookPreviewActivity.h; sourceTree = "<group>"; };
    59985999                F48570A22644BEC400C05F71 /* Timeout.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Timeout.h; sourceTree = "<group>"; };
    59996000                F48D2A8421583A0200C6752B /* AppKitSPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = AppKitSPI.h; sourceTree = "<group>"; };
    … …  
    70587059                                C145CC0E23DCA427003A5EEB /* PreferenceObserver.h */,
    70597060                                C15CBB3323F34C3800300CC7 /* PreferenceObserver.mm */,
     7061                                F47A051626827A09007E5CF2 /* QuickLookPreviewActivity.h */,
    70607062                                5CB7AFDD23C5273D00E49CF3 /* ResourceLoadDelegate.h */,
    70617063                                5CB7AFDE23C5273D00E49CF3 /* ResourceLoadDelegate.mm */,

  • trunk/Source/WebKitLegacy/mac/ChangeLog

    r279098 r279164  
     12021-06-22  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Live Text] [macOS] Add an internal option to disable inline text selection in images
     4        https://bugs.webkit.org/show_bug.cgi?id=227265
     5        rdar://79460142
     6
     7        Reviewed by Tim Horton.
     8
     9        Rename ContextMenuItemTagLookUpImage to ContextMenuItemTagQuickLookImage.
     10
     11        * WebView/WebHTMLView.mm:
     12        (toTag):
     13
    1142021-06-21  Wenson Hsieh  <wenson_hsieh@apple.com>
    215

  • trunk/Source/WebKitLegacy/mac/WebView/WebHTMLView.mm

    r278993 r279164  
    616616    case ContextMenuItemTagTranslate:
    617617        return WebMenuItemTagTranslate;
    618     case ContextMenuItemTagLookUpImage:
     618    case ContextMenuItemTagQuickLookImage:
    619619        return std::nullopt;
    620620
Note: See TracChangeset for help on using the changeset viewer.