Changeset 219319 in webkit


Ignore:
Timestamp:
Jul 10, 2017 7:52:07 PM (7 years ago)
Author:
Brent Fulgham
Message:

Resource Load Statistics: Prune statistics in orders of importance
​https://bugs.webkit.org/show_bug.cgi?id=174215
<rdar://problem/33164403>

Patch by John Wilander <​wilander@apple.com> on 2017-07-10
Reviewed by Chris Dumez.

Source/WebCore:

Test: http/tests/loading/resourceLoadStatistics/prune-statistics.html

  • loader/ResourceLoadObserver.cpp:

(WebCore::reduceTimeResolution):
(WebCore::ResourceLoadObserver::logFrameNavigation):
(WebCore::ResourceLoadObserver::logSubresourceLoading):
(WebCore::ResourceLoadObserver::logWebSocketLoading):
(WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution):

Now all set the new statistics field lastSeen.

  • loader/ResourceLoadStatistics.cpp:

(WebCore::ResourceLoadStatistics::encode):
(WebCore::ResourceLoadStatistics::decode):
(WebCore::ResourceLoadStatistics::toString):
(WebCore::ResourceLoadStatistics::merge):

Handling of the new statistics field lastSeen.

  • loader/ResourceLoadStatistics.h:

Source/WebKit2:

New functionality. Prunes statistics in this order:

  1. Non-prevalent resources without user interaction.
  2. Prevalent resources without user interaction.
  3. Non-prevalent resources with user interaction.
  4. Prevalent resources with user interaction.
  • Shared/WebCoreArgumentCoders.cpp:

(IPC::ArgumentCoder<ResourceLoadStatistics>::encode):
(IPC::ArgumentCoder<ResourceLoadStatistics>::decode):

Added timestamp field lastSeen.

  • UIProcess/API/Cocoa/WKWebsiteDataStore.mm:

(-[WKWebsiteDataStore _resourceLoadStatisticsSetLastSeen:forHost:]):
(-[WKWebsiteDataStore _resourceLoadStatisticsSetMaxStatisticsEntries:]):
(-[WKWebsiteDataStore _resourceLoadStatisticsSetPruneEntriesDownTo:]):
(-[WKWebsiteDataStore _resourceLoadStatisticsResetToConsistentState]):

Test infrastructure.

  • UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
  • UIProcess/Storage/ResourceLoadStatisticsStore.cpp:

(WebKit::ResourceLoadStatisticsStore::setMaxStatisticsEntries):

Test infrastructure.

(WebKit::ResourceLoadStatisticsStore::setPruneEntriesDownTo):

Test infrastructure.

(WebKit::sortAndPrune):

Convenience function.

(WebKit::ResourceLoadStatisticsStore::pruneStatisticsIfNeeded):

The new pruning function.

  • UIProcess/Storage/ResourceLoadStatisticsStore.h:
  • UIProcess/WebResourceLoadStatisticsStore.cpp:

(WebKit::WebResourceLoadStatisticsStore::processStatisticsAndDataRecords):

Now calls ResourceLoadStatisticsStore::pruneStatisticsIfNeeded().

(WebKit::WebResourceLoadStatisticsStore::setLastSeen):

Test infrastructure.

(WebKit::WebResourceLoadStatisticsStore::setMaxStatisticsEntries):

Test infrastructure.

(WebKit::WebResourceLoadStatisticsStore::setPruneEntriesDownTo):

Test infrastructure.

  • UIProcess/WebResourceLoadStatisticsStore.h:

Tools:

Nest infrastructure. Adds these functions:

  1. testRunner.setStatisticsLastSeen()
  2. setStatisticsMaxStatisticsEntries()
  3. setStatisticsPruneEntriesDownTo()
  • WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
  • WebKitTestRunner/InjectedBundle/TestRunner.cpp:

(WTR::TestRunner::setStatisticsLastSeen):
(WTR::TestRunner::setStatisticsMaxStatisticsEntries):
(WTR::TestRunner::setStatisticsPruneEntriesDownTo):

  • WebKitTestRunner/InjectedBundle/TestRunner.h:
  • WebKitTestRunner/TestController.cpp:

(WTR::TestController::setStatisticsLastSeen):
(WTR::TestController::setMaxStatisticsEntries):
(WTR::TestController::setPruneEntriesDownTo):

  • WebKitTestRunner/TestController.h:
  • WebKitTestRunner/TestInvocation.cpp:

(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

  • WebKitTestRunner/cocoa/TestControllerCocoa.mm:

(WTR::TestController::setStatisticsLastSeen):
(WTR::TestController::setStatisticsMaxStatisticsEntries):
(WTR::TestController::setStatisticsPruneEntriesDownTo):

LayoutTests:

  • http/tests/loading/resourceLoadStatistics/prune-statistics-expected.txt: Added.
  • http/tests/loading/resourceLoadStatistics/prune-statistics.html: Added.
  • platform/wk2/TestExpectations:

Added http/tests/loading/resourceLoadStatistics/prune-statistics.html as
[ Pass ] since Resource Load Statistics is WK2-only.

Location:
trunk
Files:
2 added
20 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r219316 r219319  
     12017-07-10  John Wilander  <wilander@apple.com>
     2
     3        Resource Load Statistics: Prune statistics in orders of importance
     4        https://bugs.webkit.org/show_bug.cgi?id=174215
     5        <rdar://problem/33164403>
     6
     7        Reviewed by Chris Dumez.
     8
     9        * http/tests/loading/resourceLoadStatistics/prune-statistics-expected.txt: Added.
     10        * http/tests/loading/resourceLoadStatistics/prune-statistics.html: Added.
     11        * platform/wk2/TestExpectations:
     12            Added http/tests/loading/resourceLoadStatistics/prune-statistics.html as
     13            [ Pass ] since Resource Load Statistics is WK2-only.
     14
    1152017-07-10  Devin Rousso  <drousso@apple.com>
    216

  • trunk/LayoutTests/platform/wk2/TestExpectations

    r219311 r219319  
    707707webkit.org/b/172452 http/tests/loading/resourceLoadStatistics/grandfathering.html [ Pass Failure Timeout ]
    708708webkit.org/b/173499 http/tests/loading/resourceLoadStatistics/telemetry-generation.html [ Pass Failure ]
     709http/tests/loading/resourceLoadStatistics/prune-statistics.html [ Pass ]
    709710
    710711### END OF (5) Progressions, expected successes that are expected failures in WebKit1.

  • trunk/Source/WebCore/ChangeLog

    r219316 r219319  
     12017-07-10  John Wilander  <wilander@apple.com>
     2
     3        Resource Load Statistics: Prune statistics in orders of importance
     4        https://bugs.webkit.org/show_bug.cgi?id=174215
     5        <rdar://problem/33164403>
     6
     7        Reviewed by Chris Dumez.
     8
     9        Test: http/tests/loading/resourceLoadStatistics/prune-statistics.html
     10
     11        * loader/ResourceLoadObserver.cpp:
     12        (WebCore::reduceTimeResolution):
     13        (WebCore::ResourceLoadObserver::logFrameNavigation):
     14        (WebCore::ResourceLoadObserver::logSubresourceLoading):
     15        (WebCore::ResourceLoadObserver::logWebSocketLoading):
     16        (WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution):
     17            Now all set the new statistics field lastSeen.
     18        * loader/ResourceLoadStatistics.cpp:
     19        (WebCore::ResourceLoadStatistics::encode):
     20        (WebCore::ResourceLoadStatistics::decode):
     21        (WebCore::ResourceLoadStatistics::toString):
     22        (WebCore::ResourceLoadStatistics::merge):
     23            Handling of the new statistics field lastSeen.
     24        * loader/ResourceLoadStatistics.h:
     25
    1262017-07-10  Devin Rousso  <drousso@apple.com>
    227

  • trunk/Source/WebCore/loader/ResourceLoadObserver.cpp

    r219284 r219319  
    9393}
    9494
     95static WallTime reduceToHourlyTimeResolution(WallTime time)
     96{
     97    return WallTime::fromRawSeconds(std::floor(time.secondsSinceEpoch() / timestampResolution) * timestampResolution.seconds());
     98}
     99
    95100void ResourceLoadObserver::logFrameNavigation(const Frame& frame, const Frame& topFrame, const ResourceRequest& newRequest)
    96101{
    … …  
    126131
    127132    auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
     133    targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
    128134    auto subframeUnderTopFrameOriginsResult = targetStatistics.subframeUnderTopFrameOrigins.add(mainFramePrimaryDomain);
    129135    if (subframeUnderTopFrameOriginsResult.isNewEntry)
    … …  
    159165    {
    160166        auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
     167        targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
    161168        if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
    162169            shouldCallNotificationCallback = true;
    … …  
    198205
    199206    auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
     207    targetStatistics.lastSeen = reduceToHourlyTimeResolution(WallTime::now());
    200208    if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
    201209        scheduleNotificationIfNeeded();
    202210}
    203211
    204 static WallTime reduceTimeResolution(WallTime time)
    205 {
    206     return WallTime::fromRawSeconds(std::floor(time.secondsSinceEpoch() / timestampResolution) * timestampResolution.seconds());
    207 }
    208 
    209212void ResourceLoadObserver::logUserInteractionWithReducedTimeResolution(const Document& document)
    210213{
    … …  
    219222
    220223    auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
    221     auto newTime = reduceTimeResolution(WallTime::now());
     224    auto newTime = reduceToHourlyTimeResolution(WallTime::now());
    222225    if (newTime == statistics.mostRecentUserInteractionTime)
    223226        return;
    224227
    225228    statistics.hadUserInteraction = true;
     229    statistics.lastSeen = newTime;
    226230    statistics.mostRecentUserInteractionTime = newTime;
    227231

  • trunk/Source/WebCore/loader/ResourceLoadStatistics.cpp

    r219231 r219319  
    5151    encoder.encodeString("PrevalentResourceOrigin", highLevelDomain);
    5252   
     53    encoder.encodeDouble("lastSeen", lastSeen.secondsSinceEpoch().value());
     54   
    5355    // User interaction
    5456    encoder.encodeBool("hadUserInteraction", hadUserInteraction);
    … …  
    115117        return false;
    116118
     119    double lastSeenTimeAsDouble;
     120    if (!decoder.decodeDouble("lastSeen", lastSeenTimeAsDouble))
     121        return false;
     122    lastSeen = WallTime::fromRawSeconds(lastSeenTimeAsDouble);
     123   
    117124    return true;
    118125}
    … …  
    148155    StringBuilder builder;
    149156   
     157    builder.appendLiteral("lastSeen");
     158    builder.appendNumber(lastSeen.secondsSinceEpoch().value());
     159    builder.append('\n');
     160   
    150161    // User interaction
    151162    appendBoolean(builder, "hadUserInteraction", hadUserInteraction);
    … …  
    190201    ASSERT(other.highLevelDomain == highLevelDomain);
    191202
     203    if (lastSeen < other.lastSeen)
     204        lastSeen = other.lastSeen;
     205   
    192206    if (!other.hadUserInteraction) {
    193207        // If user interaction has been reset do so here too.

  • trunk/Source/WebCore/loader/ResourceLoadStatistics.h

    r219231 r219319  
    6262    String highLevelDomain;
    6363
     64    WallTime lastSeen;
     65   
    6466    // User interaction
    6567    bool hadUserInteraction { false };

  • trunk/Source/WebKit2/ChangeLog

    r219318 r219319  
     12017-07-10  John Wilander  <wilander@apple.com>
     2
     3        Resource Load Statistics: Prune statistics in orders of importance
     4        https://bugs.webkit.org/show_bug.cgi?id=174215
     5        <rdar://problem/33164403>
     6
     7        Reviewed by Chris Dumez.
     8
     9        New functionality. Prunes statistics in this order:
     10        1. Non-prevalent resources without user interaction.
     11        2. Prevalent resources without user interaction.
     12        3. Non-prevalent resources with user interaction.
     13        4. Prevalent resources with user interaction.
     14
     15        * Shared/WebCoreArgumentCoders.cpp:
     16        (IPC::ArgumentCoder<ResourceLoadStatistics>::encode):
     17        (IPC::ArgumentCoder<ResourceLoadStatistics>::decode):
     18            Added timestamp field lastSeen.
     19        * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
     20        (-[WKWebsiteDataStore _resourceLoadStatisticsSetLastSeen:forHost:]):
     21        (-[WKWebsiteDataStore _resourceLoadStatisticsSetMaxStatisticsEntries:]):
     22        (-[WKWebsiteDataStore _resourceLoadStatisticsSetPruneEntriesDownTo:]):
     23        (-[WKWebsiteDataStore _resourceLoadStatisticsResetToConsistentState]):
     24            Test infrastructure.
     25        * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
     26        * UIProcess/Storage/ResourceLoadStatisticsStore.cpp:
     27        (WebKit::ResourceLoadStatisticsStore::setMaxStatisticsEntries):
     28            Test infrastructure.
     29        (WebKit::ResourceLoadStatisticsStore::setPruneEntriesDownTo):
     30            Test infrastructure.
     31        (WebKit::sortAndPrune):
     32            Convenience function.
     33        (WebKit::ResourceLoadStatisticsStore::pruneStatisticsIfNeeded):
     34            The new pruning function.
     35        * UIProcess/Storage/ResourceLoadStatisticsStore.h:
     36        * UIProcess/WebResourceLoadStatisticsStore.cpp:
     37        (WebKit::WebResourceLoadStatisticsStore::processStatisticsAndDataRecords):
     38            Now calls ResourceLoadStatisticsStore::pruneStatisticsIfNeeded().
     39        (WebKit::WebResourceLoadStatisticsStore::setLastSeen):
     40            Test infrastructure.
     41        (WebKit::WebResourceLoadStatisticsStore::setMaxStatisticsEntries):
     42            Test infrastructure.
     43        (WebKit::WebResourceLoadStatisticsStore::setPruneEntriesDownTo):
     44            Test infrastructure.
     45        * UIProcess/WebResourceLoadStatisticsStore.h:
     46
    1472017-07-10  Dean Jackson  <dino@apple.com>
    248

  • trunk/Source/WebKit2/Shared/WebCoreArgumentCoders.cpp

    r219231 r219319  
    22432243    encoder << statistics.highLevelDomain;
    22442244   
     2245    encoder << statistics.lastSeen.secondsSinceEpoch().value();
     2246   
    22452247    // User interaction
    22462248    encoder << statistics.hadUserInteraction;
    … …  
    22642266    if (!decoder.decode(statistics.highLevelDomain))
    22652267        return false;
     2268   
     2269    double lastSeenTimeAsDouble;
     2270    if (!decoder.decode(lastSeenTimeAsDouble))
     2271        return false;
     2272    statistics.lastSeen = WallTime::fromRawSeconds(lastSeenTimeAsDouble);
    22662273   
    22672274    // User interaction

  • trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebsiteDataStore.mm

    r219312 r219319  
    208208}
    209209
     210- (void)_resourceLoadStatisticsSetLastSeen:(double)seconds forHost:(NSString *)host
     211{
     212    auto* store = _websiteDataStore->websiteDataStore().resourceLoadStatistics();
     213    if (!store)
     214        return;
     215   
     216    store->setLastSeen(URL(URL(), host), Seconds { seconds });
     217}
     218
    210219- (void)_resourceLoadStatisticsSetIsPrevalentResource:(BOOL)value forHost:(NSString *)host
    211220{
    … …  
    346355}
    347356
     357- (void)_resourceLoadStatisticsSetMaxStatisticsEntries:(size_t)entries
     358{
     359    auto* store = _websiteDataStore->websiteDataStore().resourceLoadStatistics();
     360    if (!store)
     361        return;
     362
     363    store->setMaxStatisticsEntries(entries);
     364}
     365
     366- (void)_resourceLoadStatisticsSetPruneEntriesDownTo:(size_t)entries
     367{
     368    auto* store = _websiteDataStore->websiteDataStore().resourceLoadStatistics();
     369    if (!store)
     370        return;
     371
     372    store->setPruneEntriesDownTo(entries);
     373}
     374
    348375- (void)_resourceLoadStatisticsProcessStatisticsAndDataRecords
    349376{
    … …  
    442469
    443470    // FIXME: These needs to match the default data member values in ResourceLoadStatistics, which is fragile.
     471    store->setMaxStatisticsEntries(1000);
     472    store->setPruneEntriesDownTo(800);
    444473    store->setTimeToLiveUserInteraction(std::nullopt);
    445474    store->setTimeToLiveCookiePartitionFree(24_h);

  • trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h

    r219297 r219319  
    4545
    4646// ResourceLoadStatistics SPI for testing.
     47- (void)_resourceLoadStatisticsSetLastSeen:(double)seconds forHost:(NSString *)host WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
    4748- (void)_resourceLoadStatisticsSetIsPrevalentResource:(BOOL)value forHost:(NSString *)host WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
    4849- (void)_resourceLoadStatisticsIsPrevalentResource:(NSString *)host completionHandler:(void (^)(BOOL))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
    … …  
    5859- (void)_resourceLoadStatisticsSetMinimumTimeBetweenDataRecordsRemoval:(double)seconds WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
    5960- (void)_resourceLoadStatisticsSetGrandfatheringTime:(double)seconds WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
     61- (void)_resourceLoadStatisticsSetMaxStatisticsEntries:(size_t)entries WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
     62- (void)_resourceLoadStatisticsSetPruneEntriesDownTo:(size_t)entries WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
    6063- (void)_resourceLoadStatisticsProcessStatisticsAndDataRecords WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
    6164- (void)_resourceLoadStatisticsUpdateCookiePartitioning WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));

  • trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.cpp

    r219312 r219319  
    5151constexpr Seconds minimumStatisticsFileWriteInterval { 5_min };
    5252constexpr unsigned operatingDatesWindow { 30 };
    53 constexpr unsigned statisticsModelVersion { 6 };
     53constexpr unsigned statisticsModelVersion { 7 };
     54constexpr unsigned maxImportance { 3 };
    5455
    5556template<typename T> static inline String primaryDomain(const T& value)
    … …  
    147148        removeDataRecords();
    148149       
     150        pruneStatisticsIfNeeded();
     151
    149152        if (m_shouldNotifyPagesWhenDataRecordsWereScanned) {
    150153            RunLoop::main().dispatch([] {
    … …  
    501504}
    502505
     506void WebResourceLoadStatisticsStore::setLastSeen(const URL& url, Seconds seconds)
     507{
     508    if (url.isBlankURL() || url.isEmpty())
     509        return;
     510   
     511    m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), url = url.isolatedCopy(), seconds] {
     512        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
     513        statistics.lastSeen = WallTime::fromRawSeconds(seconds.seconds());
     514    });
     515}
     516   
    503517void WebResourceLoadStatisticsStore::setPrevalentResource(const URL& url)
    504518{
    … …  
    834848        ensureResourceStatisticsForPrimaryDomain(domain).isMarkedForCookiePartitioning = true;
    835849}
    836 
     850   
    837851void WebResourceLoadStatisticsStore::processStatistics(const WTF::Function<void (const ResourceLoadStatistics&)>& processFunction) const
    838852{
    … …  
    905919}
    906920   
     921void WebResourceLoadStatisticsStore::setMaxStatisticsEntries(size_t maximumEntryCount)
     922{
     923    m_maxStatisticsEntries = maximumEntryCount;
     924}
     925   
     926void WebResourceLoadStatisticsStore::setPruneEntriesDownTo(size_t pruneTargetCount)
     927{
     928    m_pruneEntriesDownTo = pruneTargetCount;
     929}
     930   
     931struct StatisticsLastSeen {
     932    String topPrivatelyOwnedDomain;
     933    WallTime lastSeen;
     934};
     935   
     936static void pruneResources(HashMap<String, WebCore::ResourceLoadStatistics>& statisticsMap, Vector<StatisticsLastSeen>& statisticsToPrune, size_t& numberOfEntriesToPrune)
     937{
     938    if (statisticsToPrune.size() > numberOfEntriesToPrune) {
     939        std::sort(statisticsToPrune.begin(), statisticsToPrune.end(), [](const StatisticsLastSeen& a, const StatisticsLastSeen& b) {
     940            return a.lastSeen < b.lastSeen;
     941        });
     942    }
     943   
     944    for (size_t i = 0, end = std::min(numberOfEntriesToPrune, statisticsToPrune.size()); i != end; ++i)
     945        statisticsMap.remove(statisticsToPrune[i].topPrivatelyOwnedDomain);
     946}
     947   
     948static unsigned computeImportance(const ResourceLoadStatistics& resourceStatistic)
     949{
     950    unsigned importance = maxImportance;
     951    if (!resourceStatistic.isPrevalentResource)
     952        importance -= 1;
     953    if (!resourceStatistic.hadUserInteraction)
     954        importance -= 2;
     955    return importance;
     956}
     957   
     958void WebResourceLoadStatisticsStore::pruneStatisticsIfNeeded()
     959{
     960    ASSERT(!RunLoop::isMain());
     961    if (m_resourceStatisticsMap.size() <= m_maxStatisticsEntries)
     962        return;
     963
     964    ASSERT(m_pruneEntriesDownTo <= m_maxStatisticsEntries);
     965
     966    size_t numberOfEntriesLeftToPrune = m_resourceStatisticsMap.size() - m_pruneEntriesDownTo;
     967    ASSERT(numberOfEntriesLeftToPrune);
     968   
     969    Vector<StatisticsLastSeen> resourcesToPrunePerImportance[maxImportance + 1];
     970    for (auto& resourceStatistic : m_resourceStatisticsMap.values())
     971        resourcesToPrunePerImportance[computeImportance(resourceStatistic)].append({ resourceStatistic.highLevelDomain, resourceStatistic.lastSeen });
     972   
     973    for (unsigned importance = 0; numberOfEntriesLeftToPrune && importance <= maxImportance; ++importance)
     974        pruneResources(m_resourceStatisticsMap, resourcesToPrunePerImportance[importance], numberOfEntriesLeftToPrune);
     975
     976    ASSERT(!numberOfEntriesLeftToPrune);
     977}
     978   
    907979} // namespace WebKit

  • trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.h

    r219312 r219319  
    8080    void clearUserInteraction(const WebCore::URL&);
    8181    void hasHadUserInteraction(const WebCore::URL&, WTF::Function<void (bool)>&&);
     82    void setLastSeen(const WebCore::URL&, Seconds);
    8283    void setPrevalentResource(const WebCore::URL&);
    8384    void isPrevalentResource(const WebCore::URL&, WTF::Function<void (bool)>&&);
    … …  
    101102    void setMinimumTimeBetweenDataRecordsRemoval(Seconds);
    102103    void setGrandfatheringTime(Seconds);
    103 
     104    void setMaxStatisticsEntries(size_t);
     105    void setPruneEntriesDownTo(size_t);
     106   
    104107    void processStatistics(const WTF::Function<void (const WebCore::ResourceLoadStatistics&)>&) const;
    105 
     108    void pruneStatisticsIfNeeded();
     109   
    106110private:
    107111    WebResourceLoadStatisticsStore(const String&, UpdateCookiePartitioningForDomainsHandler&&);
    … …  
    173177    Seconds m_timeToLiveCookiePartitionFree { 24_h };
    174178    Seconds m_grandfatheringTime { 1_h };
     179    size_t m_maxStatisticsEntries { 1000 };
     180    size_t m_pruneEntriesDownTo { 800 };
    175181    bool m_dataRecordsBeingRemoved { false };
    176182    bool m_didScheduleWrite { false };

  • trunk/Tools/ChangeLog

    r219301 r219319  
     12017-07-10  John Wilander  <wilander@apple.com>
     2
     3        Resource Load Statistics: Prune statistics in orders of importance
     4        https://bugs.webkit.org/show_bug.cgi?id=174215
     5        <rdar://problem/33164403>
     6
     7        Reviewed by Chris Dumez.
     8
     9        Nest infrastructure. Adds these functions:
     10        1. testRunner.setStatisticsLastSeen()
     11        2. setStatisticsMaxStatisticsEntries()
     12        3. setStatisticsPruneEntriesDownTo()
     13
     14        * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
     15        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
     16        (WTR::TestRunner::setStatisticsLastSeen):
     17        (WTR::TestRunner::setStatisticsMaxStatisticsEntries):
     18        (WTR::TestRunner::setStatisticsPruneEntriesDownTo):
     19        * WebKitTestRunner/InjectedBundle/TestRunner.h:
     20        * WebKitTestRunner/TestController.cpp:
     21        (WTR::TestController::setStatisticsLastSeen):
     22        (WTR::TestController::setMaxStatisticsEntries):
     23        (WTR::TestController::setPruneEntriesDownTo):
     24        * WebKitTestRunner/TestController.h:
     25        * WebKitTestRunner/TestInvocation.cpp:
     26        (WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
     27        * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
     28        (WTR::TestController::setStatisticsLastSeen):
     29        (WTR::TestController::setStatisticsMaxStatisticsEntries):
     30        (WTR::TestController::setStatisticsPruneEntriesDownTo):
     31
    1322017-07-03  Brian Burg  <bburg@apple.com>
    233

  • trunk/Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl

    r219297 r219319  
    255255    void installStatisticsDidScanDataRecordsCallback(object callback);
    256256    void installStatisticsDidRunTelemetryCallback(object callback);
     257    void setStatisticsLastSeen(DOMString hostName, double seconds);
    257258    void setStatisticsPrevalentResource(DOMString hostName, boolean value);
    258259    boolean isStatisticsPrevalentResource(DOMString hostName);
    … …  
    275276    void setStatisticsMinimumTimeBetweenDataRecordsRemoval(double seconds);
    276277    void setStatisticsGrandfatheringTime(double seconds);
     278    void setStatisticsMaxStatisticsEntries(unsigned long entries);
     279    void setStatisticsPruneEntriesDownTo(unsigned long entries);
    277280    void statisticsClearInMemoryAndPersistentStore();
    278281    void statisticsClearInMemoryAndPersistentStoreModifiedSinceHours(unsigned long hours);

  • trunk/Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp

    r219297 r219319  
    11761176}
    11771177
     1178void TestRunner::setStatisticsLastSeen(JSStringRef hostName, double seconds)
     1179{
     1180    Vector<WKRetainPtr<WKStringRef>> keys;
     1181    Vector<WKRetainPtr<WKTypeRef>> values;
     1182   
     1183    keys.append({ AdoptWK, WKStringCreateWithUTF8CString("HostName") });
     1184    values.append({ AdoptWK, WKStringCreateWithJSString(hostName) });
     1185   
     1186    keys.append({ AdoptWK, WKStringCreateWithUTF8CString("Value") });
     1187    values.append({ AdoptWK, WKDoubleCreate(seconds) });
     1188   
     1189    Vector<WKStringRef> rawKeys(keys.size());
     1190    Vector<WKTypeRef> rawValues(values.size());
     1191   
     1192    for (size_t i = 0; i < keys.size(); ++i) {
     1193        rawKeys[i] = keys[i].get();
     1194        rawValues[i] = values[i].get();
     1195    }
     1196   
     1197    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("SetStatisticsLastSeen"));
     1198    WKRetainPtr<WKDictionaryRef> messageBody(AdoptWK, WKDictionaryCreate(rawKeys.data(), rawValues.data(), rawKeys.size()));
     1199   
     1200    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
     1201}
     1202   
    11781203void TestRunner::setStatisticsPrevalentResource(JSStringRef hostName, bool value)
    11791204{
    … …  
    14921517}
    14931518
     1519void TestRunner::setStatisticsMaxStatisticsEntries(unsigned entries)
     1520{
     1521    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("SetMaxStatisticsEntries"));
     1522    WKRetainPtr<WKTypeRef> messageBody(AdoptWK, WKUInt64Create(entries));
     1523    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
     1524}
     1525   
     1526void TestRunner::setStatisticsPruneEntriesDownTo(unsigned entries)
     1527{
     1528    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("SetPruneEntriesDownTo"));
     1529    WKRetainPtr<WKTypeRef> messageBody(AdoptWK, WKUInt64Create(entries));
     1530    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
     1531}
     1532   
    14941533void TestRunner::statisticsClearInMemoryAndPersistentStore()
    14951534{

  • trunk/Tools/WebKitTestRunner/InjectedBundle/TestRunner.h

    r219297 r219319  
    358358    void statisticsSetShouldPartitionCookiesForHost(JSStringRef hostName, bool value);
    359359    void statisticsSubmitTelemetry();
     360    void setStatisticsLastSeen(JSStringRef hostName, double seconds);
    360361    void setStatisticsPrevalentResource(JSStringRef hostName, bool value);
    361362    bool isStatisticsPrevalentResource(JSStringRef hostName);
    … …  
    374375    void setStatisticsMinimumTimeBetweenDataRecordsRemoval(double);
    375376    void setStatisticsGrandfatheringTime(double seconds);
     377    void setStatisticsMaxStatisticsEntries(unsigned);
     378    void setStatisticsPruneEntriesDownTo(unsigned);
    376379    void statisticsClearInMemoryAndPersistentStore();
    377380    void statisticsClearInMemoryAndPersistentStoreModifiedSinceHours(unsigned hours);

  • trunk/Tools/WebKitTestRunner/TestController.cpp

    r219297 r219319  
    22502250#if !PLATFORM(COCOA) || !WK_API_ENABLED
    22512251
     2252void TestController::setStatisticsLastSeen(WKStringRef, double)
     2253{
     2254}
     2255   
    22522256void TestController::setStatisticsPrevalentResource(WKStringRef, bool)
    22532257{
    … …  
    23332337}
    23342338
     2339void TestController::setStatisticsMaxStatisticsEntries(unsigned)
     2340{
     2341}
     2342   
     2343void TestController::setStatisticsPruneEntriesDownTo(unsigned)
     2344{
     2345}
     2346   
    23352347void TestController::statisticsClearInMemoryAndPersistentStore()
    23362348{

  • trunk/Tools/WebKitTestRunner/TestController.h

    r219297 r219319  
    151151    void setShouldDownloadUndisplayableMIMETypes(bool value) { m_shouldDownloadUndisplayableMIMETypes = value; }
    152152
     153    void setStatisticsLastSeen(WKStringRef hostName, double seconds);
    153154    void setStatisticsPrevalentResource(WKStringRef hostName, bool value);
    154155    bool isStatisticsPrevalentResource(WKStringRef hostName);
    … …  
    171172    void setStatisticsMinimumTimeBetweenDataRecordsRemoval(double);
    172173    void setStatisticsGrandfatheringTime(double seconds);
     174    void setStatisticsMaxStatisticsEntries(unsigned);
     175    void setStatisticsPruneEntriesDownTo(unsigned);
    173176    void statisticsClearInMemoryAndPersistentStore();
    174177    void statisticsClearInMemoryAndPersistentStoreModifiedSinceHours(unsigned);

  • trunk/Tools/WebKitTestRunner/TestInvocation.cpp

    r219297 r219319  
    913913    }
    914914
     915    if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsLastSeen")) {
     916        ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
     917       
     918        WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
     919        WKRetainPtr<WKStringRef> hostNameKey(AdoptWK, WKStringCreateWithUTF8CString("HostName"));
     920        WKRetainPtr<WKStringRef> valueKey(AdoptWK, WKStringCreateWithUTF8CString("Value"));
     921       
     922        WKStringRef hostName = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hostNameKey.get()));
     923        WKDoubleRef value = static_cast<WKDoubleRef>(WKDictionaryGetItemForKey(messageBodyDictionary, valueKey.get()));
     924       
     925        TestController::singleton().setStatisticsLastSeen(hostName, WKDoubleGetValue(value));
     926       
     927        return nullptr;
     928    }
     929   
    915930    if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsPrevalentResource")) {
    916931        ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
    … …  
    11021117    }
    11031118   
     1119    if (WKStringIsEqualToUTF8CString(messageName, "SetMaxStatisticsEntries")) {
     1120        ASSERT(WKGetTypeID(messageBody) == WKUInt64GetTypeID());
     1121        WKUInt64Ref entries = static_cast<WKUInt64Ref>(messageBody);
     1122        TestController::singleton().setStatisticsMaxStatisticsEntries(WKUInt64GetValue(entries));
     1123        return nullptr;
     1124    }
     1125   
     1126    if (WKStringIsEqualToUTF8CString(messageName, "SetPruneEntriesDownTo")) {
     1127        ASSERT(WKGetTypeID(messageBody) == WKUInt64GetTypeID());
     1128        WKUInt64Ref entries = static_cast<WKUInt64Ref>(messageBody);
     1129        TestController::singleton().setStatisticsPruneEntriesDownTo(WKUInt64GetValue(entries));
     1130        return nullptr;
     1131    }
     1132   
    11041133    if (WKStringIsEqualToUTF8CString(messageName, "StatisticsClearInMemoryAndPersistentStore")) {
    11051134        TestController::singleton().statisticsClearInMemoryAndPersistentStore();

  • trunk/Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm

    r219297 r219319  
    223223
    224224#if WK_API_ENABLED
     225void TestController::setStatisticsLastSeen(WKStringRef hostName, double seconds)
     226{
     227    [globalWebViewConfiguration.websiteDataStore _resourceLoadStatisticsSetLastSeen:seconds forHost:toNSString(hostName)];
     228}
     229   
    225230void TestController::setStatisticsPrevalentResource(WKStringRef hostName, bool value)
    226231{
    … …  
    347352}
    348353
     354void TestController::setStatisticsMaxStatisticsEntries(unsigned entries)
     355{
     356    [globalWebViewConfiguration.websiteDataStore _resourceLoadStatisticsSetMaxStatisticsEntries:entries];
     357}
     358   
     359void TestController::setStatisticsPruneEntriesDownTo(unsigned entries)
     360{
     361    [globalWebViewConfiguration.websiteDataStore _resourceLoadStatisticsSetPruneEntriesDownTo:entries];
     362}
     363   
    349364void TestController::statisticsClearInMemoryAndPersistentStore()
    350365{
Note: See TracChangeset for help on using the changeset viewer.