Changeset 214231 in webkit


Ignore:
Timestamp:
Mar 21, 2017, 2:25:47 PM (8 years ago)
Author:
Wenson Hsieh
Message:

Teach TextIndicator to estimate the background color of the given Range
​https://bugs.webkit.org/show_bug.cgi?id=169869
<rdar://problem/31127272>

Reviewed by Anders Carlsson and Simon Fraser.

Source/WebCore:

Introduces a simple heuristic to compute a background color that is appropriate to use as a border surrounding
the snapshot of the given Range. This work is only carried out if TextIndicatorOptionComputeEstimatedBackgroundColor
is specified. The details of how this background color is estimated (as well as when to fall back) can be improved
in several ways (one idea is to sample colors along the edge of the snapshot). For the time being, this patch
naively walks up the render tree in search of enclosing parent renderers that have background colors. If any
renderers have a style that is deemed too complex to capture in a single background color, then fall back to the
base document background color; otherwise, the estimated background color is the result of blending these
background colors together.

  • page/TextIndicator.cpp:

(WebCore::styleContainsComplexBackground):

Bail out of the background color codepath if the renderer has backdrop filters, has a background image, or uses
a non-normal blend mode.

(WebCore::fallbackBackgroundColorForTextSelection):
(WebCore::estimatedBackgroundColorForRange):
(WebCore::initializeIndicator):

  • page/TextIndicator.h:

Source/WebKit/mac:

Plumb the estimated background color for WebKit1 clients through a new property in WebUITextIndicatorData.

  • WebView/WebView.mm:

(-[WebUITextIndicatorData initWithImage:textIndicatorData:scale:]):
(-[WebUITextIndicatorData dealloc]):

  • WebView/WebViewPrivate.h:

Source/WebKit2:

Send the estimated background color over XPC, and adopt TextIndicatorOptionComputeEstimatedBackgroundColor when
snapshotting after performing an edit data interaction.

  • Shared/WebCoreArgumentCoders.cpp:

(IPC::ArgumentCoder<TextIndicatorData>::encode):
(IPC::ArgumentCoder<TextIndicatorData>::decode):

  • WebProcess/WebPage/ios/WebPageIOS.mm:

(WebKit::WebPage::didConcludeEditDataInteraction):

Location:
trunk/Source
Files:
9 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r214227 r214231  
     12017-03-19  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Teach TextIndicator to estimate the background color of the given Range
     4        https://bugs.webkit.org/show_bug.cgi?id=169869
     5        <rdar://problem/31127272>
     6
     7        Reviewed by Anders Carlsson and Simon Fraser.
     8
     9        Introduces a simple heuristic to compute a background color that is appropriate to use as a border surrounding
     10        the snapshot of the given Range. This work is only carried out if TextIndicatorOptionComputeEstimatedBackgroundColor
     11        is specified. The details of how this background color is estimated (as well as when to fall back) can be improved
     12        in several ways (one idea is to sample colors along the edge of the snapshot). For the time being, this patch
     13        naively walks up the render tree in search of enclosing parent renderers that have background colors. If any
     14        renderers have a style that is deemed too complex to capture in a single background color, then fall back to the
     15        base document background color; otherwise, the estimated background color is the result of blending these
     16        background colors together.
     17
     18        * page/TextIndicator.cpp:
     19        (WebCore::styleContainsComplexBackground):
     20
     21        Bail out of the background color codepath if the renderer has backdrop filters, has a background image, or uses
     22        a non-normal blend mode.
     23
     24        (WebCore::fallbackBackgroundColorForTextSelection):
     25        (WebCore::estimatedBackgroundColorForRange):
     26        (WebCore::initializeIndicator):
     27        * page/TextIndicator.h:
     28
    1292017-03-20  Jer Noble  <jer.noble@apple.com>
    230

  • trunk/Source/WebCore/page/TextIndicator.cpp

    r214174 r214231  
    4040#include "NodeTraversal.h"
    4141#include "Range.h"
     42#include "RenderElement.h"
    4243#include "RenderObject.h"
    4344
    … …  
    201202#endif
    202203
     204static bool styleContainsComplexBackground(const RenderStyle& style)
     205{
     206    if (style.hasBlendMode())
     207        return true;
     208
     209    if (style.hasBackgroundImage())
     210        return true;
     211
     212    if (style.hasBackdropFilter())
     213        return true;
     214
     215    return false;
     216}
     217
     218static Color estimatedBackgroundColorForRange(const Range& range, const Frame& frame)
     219{
     220    auto estimatedBackgroundColor = frame.view() ? frame.view()->documentBackgroundColor() : Color::transparent;
     221
     222    RenderElement* renderer = nullptr;
     223    auto commonAncestor = range.commonAncestorContainer();
     224    while (commonAncestor) {
     225        if (is<RenderElement>(commonAncestor->renderer())) {
     226            renderer = downcast<RenderElement>(commonAncestor->renderer());
     227            break;
     228        }
     229        commonAncestor = commonAncestor->parentOrShadowHostElement();
     230    }
     231
     232    auto boundingRectForRange = enclosingIntRect(range.absoluteBoundingRect());
     233    Vector<Color> parentRendererBackgroundColors;
     234    for (; !!renderer; renderer = renderer->parent()) {
     235        auto absoluteBoundingBox = renderer->absoluteBoundingBoxRect();
     236        auto& style = renderer->style();
     237        if (!absoluteBoundingBox.contains(boundingRectForRange) || !style.hasBackground())
     238            continue;
     239
     240        if (styleContainsComplexBackground(style))
     241            return estimatedBackgroundColor;
     242
     243        auto visitedDependentBackgroundColor = style.visitedDependentColor(CSSPropertyBackgroundColor);
     244        if (visitedDependentBackgroundColor != Color::transparent)
     245            parentRendererBackgroundColors.append(visitedDependentBackgroundColor);
     246    }
     247    parentRendererBackgroundColors.reverse();
     248    for (auto backgroundColor : parentRendererBackgroundColors)
     249        estimatedBackgroundColor = estimatedBackgroundColor.blend(backgroundColor);
     250
     251    return estimatedBackgroundColor;
     252}
     253
    203254static bool initializeIndicator(TextIndicatorData& data, Frame& frame, const Range& range, FloatSize margin, bool indicatesCurrentSelection)
    204255{
     256    if (data.options & TextIndicatorOptionComputeEstimatedBackgroundColor)
     257        data.estimatedBackgroundColor = estimatedBackgroundColorForRange(range, frame);
     258
    205259    Vector<FloatRect> textRects;
    206260

  • trunk/Source/WebCore/page/TextIndicator.h

    r214174 r214231  
    8989    // Currently, this is only supported on iOS.
    9090    TextIndicatorOptionUseSelectionRectForSizing = 1 << 9,
     91
     92    // Compute a background color to use when rendering a platter around the content image, falling back to a default if the
     93    // content's background is too complex to be captured by a single color.
     94    TextIndicatorOptionComputeEstimatedBackgroundColor = 1 << 10,
    9195};
    9296typedef uint16_t TextIndicatorOptions;
    … …  
    101105    RefPtr<Image> contentImageWithoutSelection;
    102106    RefPtr<Image> contentImage;
     107    Color estimatedBackgroundColor;
    103108    TextIndicatorPresentationTransition presentationTransition;
    104109    TextIndicatorOptions options;

  • trunk/Source/WebKit/mac/ChangeLog

    r214177 r214231  
     12017-03-19  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Teach TextIndicator to estimate the background color of the given Range
     4        https://bugs.webkit.org/show_bug.cgi?id=169869
     5        <rdar://problem/31127272>
     6
     7        Reviewed by Anders Carlsson and Simon Fraser.
     8
     9        Plumb the estimated background color for WebKit1 clients through a new property in WebUITextIndicatorData.
     10
     11        * WebView/WebView.mm:
     12        (-[WebUITextIndicatorData initWithImage:textIndicatorData:scale:]):
     13        (-[WebUITextIndicatorData dealloc]):
     14        * WebView/WebViewPrivate.h:
     15
    1162017-03-20  Dan Bernstein  <mitz@apple.com>
    217

  • trunk/Source/WebKit/mac/WebView/WebView.mm

    r214111 r214231  
    639639@synthesize contentImageWithoutSelectionRectInRootViewCoordinates=_contentImageWithoutSelectionRectInRootViewCoordinates;
    640640@synthesize contentImage=_contentImage;
     641@synthesize estimatedBackgroundColor=_estimatedBackgroundColor;
    641642
    642643@end
    … …  
    671672    }
    672673
     674    if (indicatorData.options & TextIndicatorOptionComputeEstimatedBackgroundColor)
     675        _estimatedBackgroundColor = [[[getUIColorClass() alloc] initWithCGColor:cachedCGColor(indicatorData.estimatedBackgroundColor)] retain];
     676
    673677    return self;
    674678}
    … …  
    691695    [_contentImageWithoutSelection release];
    692696    [_contentImage release];
     697    [_estimatedBackgroundColor release];
    693698   
    694699    [super dealloc];

  • trunk/Source/WebKit/mac/WebView/WebViewPrivate.h

    r214111 r214231  
    5858#define WEBKIT_DI_BLOCK 1
    5959
     60@class UIColor;
    6061@class UIImage;
    6162@class NSError;
    … …  
    194195@property (nonatomic, retain) UIImage *contentImageWithoutSelection;
    195196@property (nonatomic, assign) CGRect contentImageWithoutSelectionRectInRootViewCoordinates;
     197@property (nonatomic, retain) UIColor *estimatedBackgroundColor;
    196198@end
    197199

  • trunk/Source/WebKit2/ChangeLog

    r214230 r214231  
     12017-03-19  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Teach TextIndicator to estimate the background color of the given Range
     4        https://bugs.webkit.org/show_bug.cgi?id=169869
     5        <rdar://problem/31127272>
     6
     7        Reviewed by Anders Carlsson and Simon Fraser.
     8
     9        Send the estimated background color over XPC, and adopt TextIndicatorOptionComputeEstimatedBackgroundColor when
     10        snapshotting after performing an edit data interaction.
     11
     12        * Shared/WebCoreArgumentCoders.cpp:
     13        (IPC::ArgumentCoder<TextIndicatorData>::encode):
     14        (IPC::ArgumentCoder<TextIndicatorData>::decode):
     15        * WebProcess/WebPage/ios/WebPageIOS.mm:
     16        (WebKit::WebPage::didConcludeEditDataInteraction):
     17
    1182017-03-21  Chris Dumez  <cdumez@apple.com>
    219

  • trunk/Source/WebKit2/Shared/WebCoreArgumentCoders.cpp

    r213936 r214231  
    20492049    encoder << textIndicatorData.contentImageWithoutSelectionRectInRootViewCoordinates;
    20502050    encoder << textIndicatorData.contentImageScaleFactor;
     2051    encoder << textIndicatorData.estimatedBackgroundColor;
    20512052    encoder.encodeEnum(textIndicatorData.presentationTransition);
    20522053    encoder << static_cast<uint64_t>(textIndicatorData.options);
    … …  
    20722073
    20732074    if (!decoder.decode(textIndicatorData.contentImageScaleFactor))
     2075        return false;
     2076
     2077    if (!decoder.decode(textIndicatorData.estimatedBackgroundColor))
    20742078        return false;
    20752079

  • trunk/Source/WebKit2/WebProcess/WebPage/ios/WebPageIOS.mm

    r214174 r214231  
    632632    std::optional<TextIndicatorData> textIndicatorData;
    633633
    634     static auto defaultEditDragTextIndicatorOptions = TextIndicatorOptionIncludeSnapshotOfAllVisibleContentWithoutSelection | TextIndicatorOptionDoNotClipToVisibleRect | TextIndicatorOptionPaintAllContent | TextIndicatorOptionIncludeMarginIfRangeMatchesSelection | TextIndicatorOptionPaintBackgrounds | TextIndicatorOptionUseSelectionRectForSizing | TextIndicatorOptionIncludeSnapshotWithSelectionHighlight;
     634    static auto defaultEditDataInteractionTextIndicatorOptions = TextIndicatorOptionIncludeSnapshotOfAllVisibleContentWithoutSelection | TextIndicatorOptionDoNotClipToVisibleRect | TextIndicatorOptionPaintAllContent | TextIndicatorOptionIncludeMarginIfRangeMatchesSelection | TextIndicatorOptionPaintBackgrounds | TextIndicatorOptionComputeEstimatedBackgroundColor| TextIndicatorOptionUseSelectionRectForSizing | TextIndicatorOptionIncludeSnapshotWithSelectionHighlight;
    635635    auto& frame = m_page->focusController().focusedOrMainFrame();
    636636    if (auto range = frame.selection().selection().toNormalizedRange()) {
    637         if (auto textIndicator = TextIndicator::createWithRange(*range, defaultEditDragTextIndicatorOptions, TextIndicatorPresentationTransition::None))
     637        if (auto textIndicator = TextIndicator::createWithRange(*range, defaultEditDataInteractionTextIndicatorOptions, TextIndicatorPresentationTransition::None))
    638638            textIndicatorData = textIndicator->data();
    639639    }
Note: See TracChangeset for help on using the changeset viewer.