Changeset 246781 in webkit


Ignore:
Timestamp:
Jun 24, 2019 9:06:51 PM (5 years ago)
Author:
Wenson Hsieh
Message:

[Text autosizing] [iPadOS] Revise our heuristics to determine idempotent text autosizing candidates
​https://bugs.webkit.org/show_bug.cgi?id=198763
<rdar://problem/51826266>

Reviewed by Simon Fraser.

Source/WebCore:

This patch adjusts existing text autosizing heuristics, based on a survey of text on websites in the Alexa top
500 that shrink down to fit the viewport when requesting the desktop version of the site. The new heuristic is
derived from training decision trees against the dataset obtained from this survey, and balances false positives
(cases where layout is broken due to autosizing) against overall accuracy (measured using cross-validation).

See below for more details. Additionally, please refer to the link in the radar for more details, as well as
resources used to generate, validate, and analyze these decision trees.

Test: fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-candidates.html

  • css/StyleResolver.cpp:

(WebCore::StyleResolver::adjustRenderStyleForTextAutosizing):

  • rendering/style/RenderStyle.cpp:

(WebCore::RenderStyle::isIdempotentTextAutosizingCandidate const):

Rename AutosizeStatus::shouldSkipSubtree to RenderStyle::isIdempotentTextAutosizingCandidate. We relocate this
logic to RenderStyle, such that we're able to ask the element's RenderStyle questions when determining whether
the element should be autosized.

Of course, this patch additionally revamps the heuristic used to determine whether it is safe to autosize an
element. Our current heuristic in trunk simply checks for the presence of inline block display, out of flow
positioning and a fixed height ancestor; if any of these conditions are satisfied, we opt the element out of
text autosizing. This is an excellent strategy for boosting some runs of text while avoiding autosizing in the
vast majority of cases where increasing font size may lead to layout breakage (e.g. overlapping or clipped text,
content unexpectedly flowing to the next line, etc.). However, it also avoids boosting font sizes in many
scenarios where boosting font sizes is desired; for concrete examples, see the (currently 24) radars about small
font sizes that are duped to <rdar://problem/51826266>.

To help analyze and identify trends in autosizable and non-autosizable text, we assembled a dataset of elements
with text from the Alexa top 500 that either: (1) were too small and could be boosted safely, or (2) would break
layout if boosted. With this labeled dataset, we then trained binary decision trees to classify the data. Each
decision tree was trained with a number of hyperparameters: namely, maximum depth, minimum leaf size, and the
amount of bias towards negative samples (i.e. the ratio of the weight of a non-autosizable sample relative to
the weight of an autosizable sample).

For each 3-tuple of these hyperparameters (800 in total: max depth between 3 and 10, min leaf size between 1 and
10 and bias between 1 and 10), for 5000 iterations each, we split the full dataset into a training dataset and
a cross-validation dataset, trained a decision tree using the training set, and tested against the cross-
validation set to compute average precision, recall, and overall accuracy for each tuple of hyperparameters.

The decision tree introduced in this patch was generated using a hand-picked set of hyperparameters (max depth
10, min leaf size 4, and negative bias 2) to provide a balance between precision scores (limiting layout
breakage) and recall score (ensuring that small text is mostly autosized), while optimizing for overall
accuracy. Cross-validation scores predict that the overall accuracy of this classifier is approximately 70%, up
from the current accuracy in trunk (~53%).

  • rendering/style/RenderStyle.h:

Grow the width of autosizeStatus from 4 to 8 (notably, this does not increase the size of RenderStyle).

  • rendering/style/TextSizeAdjustment.cpp:

(WebCore::AutosizeStatus::updateStatus):
(WebCore::AutosizeStatus::shouldSkipSubtree const): Deleted.

  • rendering/style/TextSizeAdjustment.h:

Introduce new text autosizing state flags, and remove some existing ones.

LayoutTests:

Rebaseline an existing text autosizing test, and introduce some new test cases that correspond to several common
patterns of autosizable (or non-autosizable) text on websites that were surveyed.

  • fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-after-changing-initial-scale.html:
  • fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-candidates-expected.txt: Added.
  • fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-candidates.html: Renamed from LayoutTests/fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-skip.html.

Rename this existing layout test too, to avoid using the term "skip" in the name of a layout test.

  • fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-skip-expected.txt: Removed.
Location:
trunk
Files:
1 added
1 deleted
8 edited
1 moved

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r246766 r246781  
     12019-06-24  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Text autosizing] [iPadOS] Revise our heuristics to determine idempotent text autosizing candidates
     4        https://bugs.webkit.org/show_bug.cgi?id=198763
     5        <rdar://problem/51826266>
     6
     7        Reviewed by Simon Fraser.
     8
     9        Rebaseline an existing text autosizing test, and introduce some new test cases that correspond to several common
     10        patterns of autosizable (or non-autosizable) text on websites that were surveyed.
     11
     12        * fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-after-changing-initial-scale.html:
     13        * fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-candidates-expected.txt: Added.
     14        * fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-candidates.html: Renamed from LayoutTests/fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-skip.html.
     15
     16        Rename this existing layout test too, to avoid using the term "skip" in the name of a layout test.
     17
     18        * fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-skip-expected.txt: Removed.
     19
    1202019-06-24  Simon Fraser  <simon.fraser@apple.com>
    221

  • trunk/LayoutTests/fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-after-changing-initial-scale.html

    r246348 r246781  
    2121    border: 1px solid black;
    2222    padding: 8px 12px 12px 12px;
    23     max-width: 320px;
    2423}
    2524

  • trunk/LayoutTests/fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-candidates.html

    r246780 r246781  
    1212</head>
    1313<body>
    14 <div style="background: green;"><span id="target" style="font-size: 12px;">Test</span></div>
     14<div style="background: green;"><span id="target1" style="font-size: 12px;">Test</span></div>
    1515<div style="background: green; overflow: auto;"><span id="target2" style="float: right; font-size: 12px;">Test</span></div>
    16 <div style="background: green;"><span id="target3" style="display: inline-block; font-size: 12px;">Test</span></div>
    17 <div style="background: green;"><span style="display: inline-block; font-size: 12px;"><span id="target4">Test</span></span></div>
     16<div style="background: green;"><span id="target3" style="display: inline-block; font-size: 12px; width: 20px;">Test</span></div>
     17<div style="background: green;"><span style="display: inline-block; height: 12px; font-size: 12px;"><span id="target4">Test</span></span></div>
    1818<div style="background: green;"><span id="target5" style="position: absolute; left: 0px; top: 0px; font-size: 12px;">Test</span></div>
    1919<div style="background: green;"><span id="target6" style="display: none; font-size: 12px;">Test</span></div>
    2020<div style="background: green;"><span id="comparison" style="font-size: 12px;">Test<span>Test<span>Test<span id="target7">Test</span></span></span></span></div>
    2121<div style="background: green;"><span id="target8" style="font-size: 12px; -webkit-text-size-adjust: 100%">Test</span></div>
     22<div style="background: green;"><span id="target9" style="font-size: 12px; display: inline-block;">Test</span></div>
     23<div style="background: green;"><span id="target10" style="font-size: 12px; width: 20px; height: 12px;">Test</span></div>
     24<div style="background: green;"><span id="target11" style="font-size: 12px; height: 20px; position: fixed; white-space: nowrap;">Test</span></div>
     25<div style="background: green;"><span id="target12" style="font-size: 12px; height: 20px; position: fixed; float: right;">Test</span></div>
     26<div style="background: green;"><span id="target13" style="font-size: 12px; height: 20px; position: fixed; float: right; overflow-x: hidden; width: 100px;">Test</span></div>
     27<div style="background: green;"><span id="target14" style="font-size: 12px; height: 20px; width: 100px; float: right;">Test</span></div>
     28<div style="background: green;"><span id="target15" style="overflow-y: hidden; float: right;">Test</span></div>
     29<div style="background: green;"><span id="target16" style="float: right;">Test</span></div>
    2230<script>
    2331let result;
    … …  
    2634    target.offsetWidth;
    2735    result = Number.parseInt(window.getComputedStyle(target).getPropertyValue("font-size"));
     36    debug(`Checking ${name}:`);
    2837    if (shouldGetAutosized)
    2938        shouldBeGreaterThanOrEqual("result", "13");
    … …  
    3140        shouldBe("result", "12");
    3241}
    33 check("target", true);
     42check("target1", true);
    3443check("target2", true);
    3544check("target3", false);
    36 check("target4", false);
    37 check("target5", false);
     45check("target4", true);
     46check("target5", true);
    3847check("target6", false);
    3948
     49debug(`Checking target7:`);
    4050let target = document.getElementById("target7");
    4151target.offsetWidth;
    … …  
    4757
    4858check("target8", true);
     59check("target9", true);
     60check("target10", true);
     61
     62// Below are some common scenarios where we prefer (or prefer to not) adjust text size. These examples are inspired by
     63// common patterns in real webpages.
     64check("target11", true);
     65check("target12", false);
     66check("target13", true);
     67check("target14", false);
     68check("target15", true);
     69check("target16", true);
    4970</script>
    5071<script src="../../../../resources/js-test-post.js"></script>

  • trunk/Source/WebCore/ChangeLog

    r246780 r246781  
     12019-06-24  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Text autosizing] [iPadOS] Revise our heuristics to determine idempotent text autosizing candidates
     4        https://bugs.webkit.org/show_bug.cgi?id=198763
     5        <rdar://problem/51826266>
     6
     7        Reviewed by Simon Fraser.
     8
     9        This patch adjusts existing text autosizing heuristics, based on a survey of text on websites in the Alexa top
     10        500 that shrink down to fit the viewport when requesting the desktop version of the site. The new heuristic is
     11        derived from training decision trees against the dataset obtained from this survey, and balances false positives
     12        (cases where layout is broken due to autosizing) against overall accuracy (measured using cross-validation).
     13
     14        See below for more details. Additionally, please refer to the link in the radar for more details, as well as
     15        resources used to generate, validate, and analyze these decision trees.
     16
     17        Test: fast/text-autosizing/ios/idempotentmode/idempotent-autosizing-candidates.html
     18
     19        * css/StyleResolver.cpp:
     20        (WebCore::StyleResolver::adjustRenderStyleForTextAutosizing):
     21        * rendering/style/RenderStyle.cpp:
     22        (WebCore::RenderStyle::isIdempotentTextAutosizingCandidate const):
     23
     24        Rename AutosizeStatus::shouldSkipSubtree to RenderStyle::isIdempotentTextAutosizingCandidate. We relocate this
     25        logic to RenderStyle, such that we're able to ask the element's RenderStyle questions when determining whether
     26        the element should be autosized.
     27
     28        Of course, this patch additionally revamps the heuristic used to determine whether it is safe to autosize an
     29        element. Our current heuristic in trunk simply checks for the presence of inline block display, out of flow
     30        positioning and a fixed height ancestor; if any of these conditions are satisfied, we opt the element out of
     31        text autosizing. This is an excellent strategy for boosting some runs of text while avoiding autosizing in the
     32        vast majority of cases where increasing font size may lead to layout breakage (e.g. overlapping or clipped text,
     33        content unexpectedly flowing to the next line, etc.). However, it also avoids boosting font sizes in many
     34        scenarios where boosting font sizes is desired; for concrete examples, see the (currently 24) radars about small
     35        font sizes that are duped to <rdar://problem/51826266>.
     36
     37        To help analyze and identify trends in autosizable and non-autosizable text, we assembled a dataset of elements
     38        with text from the Alexa top 500 that either: (1) were too small and could be boosted safely, or (2) would break
     39        layout if boosted. With this labeled dataset, we then trained binary decision trees to classify the data. Each
     40        decision tree was trained with a number of hyperparameters: namely, maximum depth, minimum leaf size, and the
     41        amount of bias towards negative samples (i.e. the ratio of the weight of a non-autosizable sample relative to
     42        the weight of an autosizable sample).
     43
     44        For each 3-tuple of these hyperparameters (800 in total: max depth between 3 and 10, min leaf size between 1 and
     45        10 and bias between 1 and 10), for 5000 iterations each, we split the full dataset into a training dataset and
     46        a cross-validation dataset, trained a decision tree using the training set, and tested against the cross-
     47        validation set to compute average precision, recall, and overall accuracy for each tuple of hyperparameters.
     48
     49        The decision tree introduced in this patch was generated using a hand-picked set of hyperparameters (max depth
     50        10, min leaf size 4, and negative bias 2) to provide a balance between precision scores (limiting layout
     51        breakage) and recall score (ensuring that small text is mostly autosized), while optimizing for overall
     52        accuracy. Cross-validation scores predict that the overall accuracy of this classifier is approximately 70%, up
     53        from the current accuracy in trunk (~53%).
     54
     55        * rendering/style/RenderStyle.h:
     56
     57        Grow the width of `autosizeStatus` from 4 to 8 (notably, this does not increase the size of RenderStyle).
     58
     59        * rendering/style/TextSizeAdjustment.cpp:
     60        (WebCore::AutosizeStatus::updateStatus):
     61        (WebCore::AutosizeStatus::shouldSkipSubtree const): Deleted.
     62        * rendering/style/TextSizeAdjustment.h:
     63
     64        Introduce new text autosizing state flags, and remove some existing ones.
     65
    1662019-06-24  Commit Queue  <commit-queue@webkit.org>
    267

  • trunk/Source/WebCore/css/StyleResolver.cpp

    r246755 r246781  
    880880void StyleResolver::adjustRenderStyleForTextAutosizing(RenderStyle& style, const Element& element)
    881881{
    882     auto newAutosizeStatus = AutosizeStatus::updateStatus(style);
    883882    if (!settings().textAutosizingEnabled() || !settings().textAutosizingUsesIdempotentMode())
    884883        return;
    885884
     885    AutosizeStatus::updateStatus(style);
    886886    if (!hasTextChildren(element))
    887887        return;
    … …  
    890890        return;
    891891
    892     if (newAutosizeStatus.shouldSkipSubtree())
     892    if (!style.isIdempotentTextAutosizingCandidate())
    893893        return;
    894894

  • trunk/Source/WebCore/rendering/style/RenderStyle.cpp

    r246490 r246781  
    491491        && m_nonInheritedFlags.floating == other.m_nonInheritedFlags.floating
    492492        && m_rareNonInheritedData->textOverflow == other.m_rareNonInheritedData->textOverflow;
     493}
     494
     495bool RenderStyle::isIdempotentTextAutosizingCandidate() const
     496{
     497    // Refer to <rdar://problem/51826266> for more information regarding how this function was generated.
     498    auto fields = OptionSet<AutosizeStatus::Fields>::fromRaw(m_inheritedFlags.autosizeStatus);
     499    if (fields.contains(AutosizeStatus::Fields::DisplayNone))
     500        return false;
     501
     502    if (fields.contains(AutosizeStatus::Fields::FixedHeight)) {
     503        if (whiteSpace() == WhiteSpace::NoWrap)
     504            return true;
     505
     506        if (fields.contains(AutosizeStatus::Fields::Floating))
     507            return fields.contains(AutosizeStatus::Fields::OutOfFlowPosition) && fields.contains(AutosizeStatus::Fields::OverflowXHidden);
     508
     509        if (fields.contains(AutosizeStatus::Fields::FixedWidth))
     510            return !fields.contains(AutosizeStatus::Fields::OutOfFlowPosition);
     511    }
     512
     513    if (fields.contains(AutosizeStatus::Fields::Floating))
     514        return true;
     515
     516    if (fields.contains(AutosizeStatus::Fields::FixedWidth))
     517        return fields.contains(AutosizeStatus::Fields::OverflowYHidden);
     518
     519    return !fields.contains(AutosizeStatus::Fields::OverflowYHidden) && !fields.contains(AutosizeStatus::Fields::FixedMaxWidth);
    493520}
    494521

  • trunk/Source/WebCore/rendering/style/RenderStyle.h

    r246490 r246781  
    742742    TextSizeAdjustment textSizeAdjust() const { return m_rareInheritedData->textSizeAdjust; }
    743743    AutosizeStatus autosizeStatus() const;
     744    bool isIdempotentTextAutosizingCandidate() const;
    744745#endif
    745746
    … …  
    18351836
    18361837#if ENABLE(TEXT_AUTOSIZING)
    1837         unsigned autosizeStatus : 4;
    1838 #endif
    1839         // 52 bits
     1838        unsigned autosizeStatus : 8;
     1839#endif
     1840        // 56 bits
    18401841    };
    18411842

  • trunk/Source/WebCore/rendering/style/TextSizeAdjustment.cpp

    r246348 r246781  
    4343}
    4444
    45 AutosizeStatus AutosizeStatus::updateStatus(RenderStyle& style)
     45void AutosizeStatus::updateStatus(RenderStyle& style)
    4646{
    47     OptionSet<Fields> result = style.autosizeStatus().fields();
     47    auto result = style.autosizeStatus().fields();
     48
     49    if (style.display() == DisplayType::None)
     50        result.add(Fields::DisplayNone);
     51
    4852    if (style.hasOutOfFlowPosition())
    49         result.add(Fields::FoundOutOfFlowPosition);
    50     switch (style.display()) {
    51     case DisplayType::InlineBlock:
    52         result.add(Fields::FoundInlineBlock);
    53         break;
    54     case DisplayType::None:
    55         result.add(Fields::FoundDisplayNone);
    56         break;
    57     default: // FIXME: Add more cases.
    58         break;
    59     }
     53        result.add(Fields::OutOfFlowPosition);
     54
    6055    if (style.height().isFixed())
    61         result.add(Fields::FoundFixedHeight);
     56        result.add(Fields::FixedHeight);
     57
     58    if (style.width().isFixed())
     59        result.add(Fields::FixedWidth);
     60
     61    if (style.maxWidth().isFixed())
     62        result.add(Fields::FixedMaxWidth);
     63
     64    if (style.overflowX() == Overflow::Hidden)
     65        result.add(Fields::OverflowXHidden);
     66
     67    if (style.overflowY() == Overflow::Hidden)
     68        result.add(Fields::OverflowYHidden);
     69
     70    if (style.isFloating())
     71        result.add(Fields::Floating);
     72
    6273    style.setAutosizeStatus(result);
    63     return result;
    64 }
    65 
    66 bool AutosizeStatus::shouldSkipSubtree() const
    67 {
    68     return m_fields.containsAny({ Fields::FoundOutOfFlowPosition, Fields::FoundInlineBlock, Fields::FoundFixedHeight, Fields::FoundDisplayNone });
    6974}
    7075

  • trunk/Source/WebCore/rendering/style/TextSizeAdjustment.h

    r245838 r246781  
    5353public:
    5454    enum class Fields : uint8_t {
    55         FoundOutOfFlowPosition = 1 << 0,
    56         FoundInlineBlock = 1 << 1,
    57         FoundFixedHeight = 1 << 2,
    58         FoundDisplayNone = 1 << 3
     55        DisplayNone = 1 << 0,
     56        FixedHeight = 1 << 1,
     57        FixedWidth = 1 << 2,
     58        Floating = 1 << 3,
     59        OverflowXHidden = 1 << 4,
     60        OverflowYHidden = 1 << 5,
     61        OutOfFlowPosition = 1 << 6,
     62        FixedMaxWidth = 1 << 7
    5963        // Adding new values requires giving RenderStyle::InheritedFlags::autosizeStatus additional bits.
    6064    };
    … …  
    6468
    6569    bool contains(Fields) const;
    66     bool shouldSkipSubtree() const;
    6770
    6871    static float idempotentTextSize(float specifiedSize, float pageScale);
    69     static AutosizeStatus updateStatus(RenderStyle&);
     72    static void updateStatus(RenderStyle&);
    7073
    7174private:
Note: See TracChangeset for help on using the changeset viewer.