Changeset 240498 in webkit


Ignore:
Timestamp:
Jan 25, 2019 12:52:11 PM (5 years ago)
Author:
Brent Fulgham
Message:

Activate the WebResourceLoadStatisticsStore in the NetworkProcess and deactivate it in the UIProcess.
​https://bugs.webkit.org/show_bug.cgi?id=193297
<rdar://problem/47158841>

Reviewed by Alex Christensen.

Source/WebCore:

Trigger logging to the UIProcess when the ResourceLoadObserver is used in the NetworkProcess.

  • Modules/websockets/WebSocket.cpp:

(WebCore::WebSocket::connect): Notify NetworkProcess a connection was made to a resource.

  • loader/ResourceLoadObserver.cpp:

(WebCore::ResourceLoadObserver::setLogWebSocketLoadingNotificationCallback): Added.
(WebCore::ResourceLoadObserver::setLogSubresourceLoadingNotificationCallback): Added.
(WebCore::ResourceLoadObserver::setLogSubresourceRedirectNotificationCallback): Added.
(WebCore::ResourceLoadObserver::logSubresourceLoading): Notify NetworkProcess of the load.
(WebCore::ResourceLoadObserver::logWebSocketLoading): Ditto.
(WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution): Ditto.

Source/WebKit:

This patch activates the ResourceLoadStatistics code in the NetworkProcess, and turns
it off in the UIProcess. It also updates test infrastructure to work with this change
in architecture.

  • NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:

(WebKit::ResourceLoadStatisticsMemoryStore::logSubresourceLoading): Added.
(WebKit::ResourceLoadStatisticsMemoryStore::logSubresourceRedirect): Added.
(WebKit::ResourceLoadStatisticsMemoryStore::logWebSocketLoading): Added.

  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:

(WebKit::WebResourceLoadStatisticsStore::logSubresourceLoading): Added.
(WebKit::WebResourceLoadStatisticsStore::logSubresourceRedirect): Added.
(WebKit::WebResourceLoadStatisticsStore::logWebSocketLoading): Added.
(WebKit::WebResourceLoadStatisticsStore::removeAllStorageAccess):
(WebKit::WebResourceLoadStatisticsStore::setCacheMaxAgeCap):
(WebKit::WebResourceLoadStatisticsStore::setCacheMaxAgeCapForPrevalentResources): Deleted.

  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
  • NetworkProcess/NetworkConnectionToWebProcess.cpp:

(WebKit::NetworkConnectionToWebProcess::logSubresourceLoading): Added.
(WebKit::NetworkConnectionToWebProcess::logSubresourceRedirect): Added.
(WebKit::NetworkConnectionToWebProcess::logWebSocketLoading): Added.

  • NetworkProcess/NetworkConnectionToWebProcess.messages.in:
  • NetworkProcess/NetworkSession.cpp:

(WebKit::NetworkSession::setResourceLoadStatisticsEnabled):

  • UIProcess/WebProcessPool.cpp:

(WebKit::WebProcessPool::ensureNetworkProcess):

  • UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:

(WebKit::WebsiteDataStore::parameters):

  • UIProcess/WebsiteData/WebsiteDataStore.cpp:

(WebKit::WebsiteDataStore::removeData):
(WebKit::WebsiteDataStore::setResourceLoadStatisticsEnabled):
(WebKit::WebsiteDataStore::enableResourceLoadStatisticsAndSetTestingCallback):

  • Webprocess/WebProcess.cpp:

(WebKit::WebProcess::initializeWebProcess): Register new ResourceLoadObserver callbacks.

Tools:

  • TestWebKitAPI/Tests/WebKit Cocoa/ResourceLoadStatistics.mm:

Revise tests to ensure a running NetworkProcess before attempting ResourceLoadStatistics operations.

  • WebKitTestRunner/InjectedBundle/TestRunner.cpp:

(WTR::TestRunner::installStatisticsDidScanDataRecordsCallback): Simplify test configuration by
activating the message used to trigger the callback when it is set.

Location:
trunk
Files:
25 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r240495 r240498  
     12019-01-25  Brent Fulgham  <bfulgham@apple.com>
     2
     3        Activate the WebResourceLoadStatisticsStore in the NetworkProcess and deactivate it in the UIProcess.
     4        https://bugs.webkit.org/show_bug.cgi?id=193297
     5        <rdar://problem/47158841>
     6
     7        Reviewed by Alex Christensen.
     8
     9        Trigger logging to the UIProcess when the ResourceLoadObserver is used in the NetworkProcess.
     10
     11        * Modules/websockets/WebSocket.cpp:
     12        (WebCore::WebSocket::connect): Notify NetworkProcess a connection was made to a resource.
     13        * loader/ResourceLoadObserver.cpp:
     14        (WebCore::ResourceLoadObserver::setLogWebSocketLoadingNotificationCallback): Added.
     15        (WebCore::ResourceLoadObserver::setLogSubresourceLoadingNotificationCallback): Added.
     16        (WebCore::ResourceLoadObserver::setLogSubresourceRedirectNotificationCallback): Added.
     17        (WebCore::ResourceLoadObserver::logSubresourceLoading): Notify NetworkProcess of the load.
     18        (WebCore::ResourceLoadObserver::logWebSocketLoading): Ditto.
     19        (WebCore::ResourceLoadObserver::logUserInteractionWithReducedTimeResolution): Ditto.
     20
    1212019-01-25  Zalan Bujtas  <zalan@apple.com>
    222

  • trunk/Source/WebCore/Modules/websockets/WebSocket.cpp

    r240453 r240498  
    280280    }
    281281
    282     RunLoop::main().dispatch([targetURL = m_url.isolatedCopy(), mainFrameURL = context.url().isolatedCopy(), usesEphemeralSession = context.sessionID().isEphemeral()]() {
    283         ResourceLoadObserver::shared().logWebSocketLoading(targetURL, mainFrameURL, usesEphemeralSession);
     282    RunLoop::main().dispatch([targetURL = m_url.isolatedCopy(), mainFrameURL = context.url().isolatedCopy(), sessionID = context.sessionID()]() {
     283        ResourceLoadObserver::shared().logWebSocketLoading(targetURL, mainFrameURL, sessionID);
    284284    });
    285285

  • trunk/Source/WebCore/loader/ResourceLoadObserver.cpp

    r240453 r240498  
    11/*
    2  * Copyright (C) 2016-2018 Apple Inc. All rights reserved.
     2 * Copyright (C) 2016-2019 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    7676}
    7777
     78void ResourceLoadObserver::setLogWebSocketLoadingNotificationCallback(Function<void(PAL::SessionID, const String&, const String&, WallTime)>&& callback)
     79{
     80    ASSERT(!m_logWebSocketLoadingNotificationCallback);
     81    m_logWebSocketLoadingNotificationCallback = WTFMove(callback);
     82}
     83
     84void ResourceLoadObserver::setLogSubresourceLoadingNotificationCallback(Function<void(PAL::SessionID, const String&, const String&, WallTime)>&& callback)
     85{
     86    ASSERT(!m_logSubresourceLoadingNotificationCallback);
     87    m_logSubresourceLoadingNotificationCallback = WTFMove(callback);
     88}
     89
     90void ResourceLoadObserver::setLogSubresourceRedirectNotificationCallback(Function<void(PAL::SessionID, const String&, const String&)>&& callback)
     91{
     92    ASSERT(!m_logSubresourceRedirectNotificationCallback);
     93    m_logSubresourceRedirectNotificationCallback = WTFMove(callback);
     94}
     95   
    7896ResourceLoadObserver::ResourceLoadObserver()
    7997    : m_notificationTimer(*this, &ResourceLoadObserver::notifyObserver)
    … …  
    123141    {
    124142        auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
    125         targetStatistics.lastSeen = ResourceLoadStatistics::reduceTimeResolution(WallTime::now());
     143        auto lastSeen = ResourceLoadStatistics::reduceTimeResolution(WallTime::now());
     144        targetStatistics.lastSeen = lastSeen;
    126145        if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
    127146            shouldCallNotificationCallback = true;
     147
     148        m_logSubresourceLoadingNotificationCallback(page->sessionID(), targetPrimaryDomain, mainFramePrimaryDomain, lastSeen);
    128149    }
    129150
    … …  
    136157        if (isNewRedirectToEntry || isNewRedirectFromEntry)
    137158            shouldCallNotificationCallback = true;
     159
     160        m_logSubresourceRedirectNotificationCallback(page->sessionID(), sourcePrimaryDomain, targetPrimaryDomain);
    138161    }
    139162
    … …  
    142165}
    143166
    144 void ResourceLoadObserver::logWebSocketLoading(const URL& targetURL, const URL& mainFrameURL, bool usesEphemeralSession)
    145 {
    146     if (!shouldLog(usesEphemeralSession))
     167void ResourceLoadObserver::logWebSocketLoading(const URL& targetURL, const URL& mainFrameURL, PAL::SessionID sessionID)
     168{
     169    if (!shouldLog(sessionID.isEphemeral()))
    147170        return;
    148171
    … …  
    159182        return;
    160183
     184    auto lastSeen = ResourceLoadStatistics::reduceTimeResolution(WallTime::now());
     185
    161186    auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
    162     targetStatistics.lastSeen = ResourceLoadStatistics::reduceTimeResolution(WallTime::now());
     187    targetStatistics.lastSeen = lastSeen;
    163188    if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
    164189        scheduleNotificationIfNeeded();
     190
     191    m_logWebSocketLoadingNotificationCallback(sessionID, targetPrimaryDomain, mainFramePrimaryDomain, lastSeen);
    165192}
    166193
    … …  
    198225    }
    199226
    200     // FIXME(193297): Uncomment this line when ResourceLoadStatistics are no longer gathered in the UI Process.
    201     // m_logUserInteractionNotificationCallback(document.sessionID(), domain);
     227    m_logUserInteractionNotificationCallback(document.sessionID(), domain);
    202228#endif
    203229

  • trunk/Source/WebCore/loader/ResourceLoadObserver.h

    r240453 r240498  
    11/*
    2  * Copyright (C) 2016-2017 Apple Inc.  All rights reserved.
     2 * Copyright (C) 2016-2019 Apple Inc.  All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    6161
    6262    void logSubresourceLoading(const Frame*, const ResourceRequest& newRequest, const ResourceResponse& redirectResponse);
    63     void logWebSocketLoading(const URL& targetURL, const URL& mainFrameURL, bool usesEphemeralSession);
     63    void logWebSocketLoading(const URL& targetURL, const URL& mainFrameURL, PAL::SessionID);
    6464    void logUserInteractionWithReducedTimeResolution(const Document&);
    6565   
    … …  
    7373
    7474    WEBCORE_EXPORT void setNotificationCallback(WTF::Function<void (Vector<ResourceLoadStatistics>&&)>&&);
    75     WEBCORE_EXPORT void setRequestStorageAccessUnderOpenerCallback(WTF::Function<void(const String&, uint64_t, const String&)>&&);
    76     WEBCORE_EXPORT void setLogUserInteractionNotificationCallback(WTF::Function<void(PAL::SessionID, const String&)>&&);
     75    WEBCORE_EXPORT void setRequestStorageAccessUnderOpenerCallback(Function<void(const String&, uint64_t, const String&)>&&);
     76    WEBCORE_EXPORT void setLogUserInteractionNotificationCallback(Function<void(PAL::SessionID, const String&)>&&);
     77    WEBCORE_EXPORT void setLogWebSocketLoadingNotificationCallback(Function<void(PAL::SessionID, const String&, const String&, WallTime)>&&);
     78    WEBCORE_EXPORT void setLogSubresourceLoadingNotificationCallback(Function<void(PAL::SessionID, const String&, const String&, WallTime)>&&);
     79    WEBCORE_EXPORT void setLogSubresourceRedirectNotificationCallback(Function<void(PAL::SessionID, const String&, const String&)>&&);
    7780
    7881    WEBCORE_EXPORT void notifyObserver();
    … …  
    99102    HashMap<String, ResourceLoadStatistics> m_resourceStatisticsMap;
    100103    HashMap<String, WTF::WallTime> m_lastReportedUserInteractionMap;
    101     WTF::Function<void (Vector<ResourceLoadStatistics>&&)> m_notificationCallback;
    102     WTF::Function<void(const String&, uint64_t, const String&)> m_requestStorageAccessUnderOpenerCallback;
    103     WTF::Function<void(PAL::SessionID, const String&)> m_logUserInteractionNotificationCallback;
     104    Function<void(Vector<ResourceLoadStatistics>&&)> m_notificationCallback;
     105    Function<void(const String&, uint64_t, const String&)> m_requestStorageAccessUnderOpenerCallback;
     106    Function<void(PAL::SessionID, const String&)> m_logUserInteractionNotificationCallback;
     107    Function<void(PAL::SessionID, const String&, const String&, WallTime)> m_logWebSocketLoadingNotificationCallback;
     108    Function<void(PAL::SessionID, const String&, const String&, WallTime)> m_logSubresourceLoadingNotificationCallback;
     109    Function<void(PAL::SessionID, const String&, const String&)> m_logSubresourceRedirectNotificationCallback;
     110
    104111    Timer m_notificationTimer;
    105112#if ENABLE(RESOURCE_LOAD_STATISTICS) && !RELEASE_LOG_DISABLED

  • trunk/Source/WebKit/ChangeLog

    r240497 r240498  
     12019-01-25  Brent Fulgham  <bfulgham@apple.com>
     2
     3        Activate the WebResourceLoadStatisticsStore in the NetworkProcess and deactivate it in the UIProcess.
     4        https://bugs.webkit.org/show_bug.cgi?id=193297
     5        <rdar://problem/47158841>
     6
     7        Reviewed by Alex Christensen.
     8
     9        This patch activates the ResourceLoadStatistics code in the NetworkProcess, and turns
     10        it off in the UIProcess. It also updates test infrastructure to work with this change
     11        in architecture.
     12
     13        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
     14        (WebKit::ResourceLoadStatisticsMemoryStore::logSubresourceLoading): Added.
     15        (WebKit::ResourceLoadStatisticsMemoryStore::logSubresourceRedirect): Added.
     16        (WebKit::ResourceLoadStatisticsMemoryStore::logWebSocketLoading): Added.
     17        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
     18        (WebKit::WebResourceLoadStatisticsStore::logSubresourceLoading): Added.
     19        (WebKit::WebResourceLoadStatisticsStore::logSubresourceRedirect): Added.
     20        (WebKit::WebResourceLoadStatisticsStore::logWebSocketLoading): Added.
     21        (WebKit::WebResourceLoadStatisticsStore::removeAllStorageAccess):
     22        (WebKit::WebResourceLoadStatisticsStore::setCacheMaxAgeCap):
     23        (WebKit::WebResourceLoadStatisticsStore::setCacheMaxAgeCapForPrevalentResources): Deleted.
     24        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
     25        * NetworkProcess/NetworkConnectionToWebProcess.cpp:
     26        (WebKit::NetworkConnectionToWebProcess::logSubresourceLoading): Added.
     27        (WebKit::NetworkConnectionToWebProcess::logSubresourceRedirect): Added.
     28        (WebKit::NetworkConnectionToWebProcess::logWebSocketLoading): Added.
     29        * NetworkProcess/NetworkConnectionToWebProcess.messages.in:
     30        * NetworkProcess/NetworkSession.cpp:
     31        (WebKit::NetworkSession::setResourceLoadStatisticsEnabled):
     32        * UIProcess/WebProcessPool.cpp:
     33        (WebKit::WebProcessPool::ensureNetworkProcess):
     34        * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
     35        (WebKit::WebsiteDataStore::parameters):
     36        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
     37        (WebKit::WebsiteDataStore::removeData):
     38        (WebKit::WebsiteDataStore::setResourceLoadStatisticsEnabled):
     39        (WebKit::WebsiteDataStore::enableResourceLoadStatisticsAndSetTestingCallback):
     40        * Webprocess/WebProcess.cpp:
     41        (WebKit::WebProcess::initializeWebProcess): Register new ResourceLoadObserver callbacks.
     42
    1432019-01-25  Ryosuke Niwa  <rniwa@webkit.org>
    244

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp

    r240453 r240498  
    639639}
    640640
     641void ResourceLoadStatisticsMemoryStore::logSubresourceLoading(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen)
     642{
     643    ASSERT(!RunLoop::isMain());
     644
     645    auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
     646    targetStatistics.lastSeen = lastSeen;
     647    if (targetStatistics.subresourceUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry)
     648        scheduleStatisticsProcessingRequestIfNecessary();
     649}
     650
     651void ResourceLoadStatisticsMemoryStore::logSubresourceRedirect(const String& sourcePrimaryDomain, const String& targetPrimaryDomain)
     652{
     653    ASSERT(!RunLoop::isMain());
     654
     655    auto& redirectingOriginStatistics = ensureResourceStatisticsForPrimaryDomain(sourcePrimaryDomain);
     656    bool isNewRedirectToEntry = redirectingOriginStatistics.subresourceUniqueRedirectsTo.add(targetPrimaryDomain).isNewEntry;
     657    auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain);
     658    bool isNewRedirectFromEntry = targetStatistics.subresourceUniqueRedirectsFrom.add(sourcePrimaryDomain).isNewEntry;
     659   
     660    if (isNewRedirectToEntry || isNewRedirectFromEntry)
     661        scheduleStatisticsProcessingRequestIfNecessary();
     662}
     663
    641664void ResourceLoadStatisticsMemoryStore::logUserInteraction(const String& primaryDomain)
    642665{

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h

    r240453 r240498  
    125125    void logFrameNavigation(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, const String& sourcePrimaryDomain, const String& targetHost, const String& mainFrameHost, bool isRedirect, bool isMainFrame);
    126126    void logUserInteraction(const String& primaryDomain);
     127    void logSubresourceLoading(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen);
     128    void logSubresourceRedirect(const String& sourcePrimaryDomain, const String& targetPrimaryDomain);
    127129
    128130    void clearUserInteraction(const String& primaryDomain);

  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp

    r240453 r240498  
    253253}
    254254
     255void WebResourceLoadStatisticsStore::requestUpdate()
     256{
     257    resourceLoadStatisticsUpdated({ });
     258}
     259
    255260void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(Vector<WebCore::ResourceLoadStatistics>&& origins)
    256261{
    … …  
    418423}
    419424
    420 void WebResourceLoadStatisticsStore::removeAllStorageAccess()
    421 {
     425void WebResourceLoadStatisticsStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler)
     426{
     427    ASSERT(RunLoop::isMain());
     428
     429    if (m_websiteDataStore) {
     430        m_websiteDataStore->removeAllStorageAccessHandler(WTFMove(completionHandler));
     431        return;
     432    }
     433
    422434    if (m_networkSession)
    423435        m_networkSession->networkStorageSession().removeAllStorageAccess();
    424 }
    425 
    426 void WebResourceLoadStatisticsStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler)
    427 {
    428     ASSERT(RunLoop::isMain());
    429 
    430     if (m_websiteDataStore) {
    431         m_websiteDataStore->removeAllStorageAccessHandler(WTFMove(completionHandler));
    432         return;
    433     }
    434     removeAllStorageAccess();
     436
    435437    completionHandler();
    436438}
    … …  
    502504        if (m_memoryStore)
    503505            m_memoryStore->logFrameNavigation(targetPrimaryDomain, mainFramePrimaryDomain, sourcePrimaryDomain, targetHost, mainFrameHost, isRedirect, isMainFrame);
     506    });
     507}
     508
     509void WebResourceLoadStatisticsStore::logWebSocketLoading(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen, CompletionHandler<void()>&& completionHandler)
     510{
     511    postTask([this, targetPrimaryDomain = targetPrimaryDomain.isolatedCopy(), mainFramePrimaryDomain = mainFramePrimaryDomain.isolatedCopy(), lastSeen, completionHandler = WTFMove(completionHandler)]() mutable {
     512        if (m_memoryStore)
     513            m_memoryStore->logSubresourceLoading(targetPrimaryDomain, mainFramePrimaryDomain, lastSeen);
     514
     515        postTaskReply(WTFMove(completionHandler));
     516    });
     517}
     518
     519void WebResourceLoadStatisticsStore::logSubresourceLoading(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen, CompletionHandler<void()>&& completionHandler)
     520{
     521    postTask([this, targetPrimaryDomain = targetPrimaryDomain.isolatedCopy(), mainFramePrimaryDomain = mainFramePrimaryDomain.isolatedCopy(), lastSeen, completionHandler = WTFMove(completionHandler)]() mutable {
     522        if (m_memoryStore)
     523            m_memoryStore->logSubresourceLoading(targetPrimaryDomain, mainFramePrimaryDomain, lastSeen);
     524       
     525        postTaskReply(WTFMove(completionHandler));
     526    });
     527}
     528
     529void WebResourceLoadStatisticsStore::logSubresourceRedirect(const String& sourcePrimaryDomain, const String& targetPrimaryDomain, CompletionHandler<void()>&& completionHandler)
     530{
     531    postTask([this, sourcePrimaryDomain = sourcePrimaryDomain.isolatedCopy(), targetPrimaryDomain = targetPrimaryDomain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
     532        if (m_memoryStore)
     533            m_memoryStore->logSubresourceRedirect(sourcePrimaryDomain, targetPrimaryDomain);
     534       
     535        postTaskReply(WTFMove(completionHandler));
    504536    });
    505537}
    … …  
    10811113}
    10821114
    1083 void WebResourceLoadStatisticsStore::setCacheMaxAgeCapForPrevalentResources(Seconds seconds)
    1084 {
     1115void WebResourceLoadStatisticsStore::setCacheMaxAgeCap(Seconds seconds, CompletionHandler<void()>&& completionHandler)
     1116{
     1117    ASSERT(RunLoop::isMain());
     1118    ASSERT(seconds >= 0_s);
     1119   
     1120    if (m_websiteDataStore) {
     1121        m_websiteDataStore->setCacheMaxAgeCapForPrevalentResources(seconds, WTFMove(completionHandler));
     1122        return;
     1123    }
     1124
    10851125    if (m_networkSession)
    10861126        m_networkSession->networkStorageSession().setCacheMaxAgeCapForPrevalentResources(seconds);
    1087 }
    1088 
    1089 void WebResourceLoadStatisticsStore::setCacheMaxAgeCap(Seconds seconds, CompletionHandler<void()>&& completionHandler)
    1090 {
    1091     ASSERT(RunLoop::isMain());
    1092     ASSERT(seconds >= 0_s);
    1093    
     1127
     1128    completionHandler();
     1129}
     1130
     1131void WebResourceLoadStatisticsStore::callUpdatePrevalentDomainsToBlockCookiesForHandler(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler)
     1132{
     1133    ASSERT(RunLoop::isMain());
     1134
    10941135    if (m_websiteDataStore) {
    1095         m_websiteDataStore->setCacheMaxAgeCapForPrevalentResources(seconds, WTFMove(completionHandler));
    1096         return;
    1097     }
    1098     setCacheMaxAgeCapForPrevalentResources(seconds);
    1099     completionHandler();
    1100 }
    1101 
    1102 void WebResourceLoadStatisticsStore::updatePrevalentDomainsToBlockCookiesFor(const Vector<String>& domainsToBlock)
    1103 {
     1136        m_websiteDataStore->updatePrevalentDomainsToBlockCookiesFor(domainsToBlock, WTFMove(completionHandler));
     1137        return;
     1138    }
     1139
    11041140    if (m_networkSession)
    11051141        m_networkSession->networkStorageSession().setPrevalentDomainsToBlockCookiesFor(domainsToBlock);
    1106 }
    1107 
    1108 void WebResourceLoadStatisticsStore::callUpdatePrevalentDomainsToBlockCookiesForHandler(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler)
    1109 {
    1110     ASSERT(RunLoop::isMain());
    1111 
    1112     if (m_websiteDataStore) {
    1113         m_websiteDataStore->updatePrevalentDomainsToBlockCookiesFor(domainsToBlock, WTFMove(completionHandler));
    1114         return;
    1115     }
    1116     updatePrevalentDomainsToBlockCookiesFor(domainsToBlock);
     1142
    11171143    completionHandler();
    11181144}
    … …  
    11761202        return;
    11771203    }
    1178     // FIXME(193297): Send message to UIProcess
     1204
     1205    if (m_networkSession)
     1206        m_networkSession->networkProcess().parentProcessConnection()->send(Messages::NetworkProcessProxy::LogTestingEvent(m_networkSession->sessionID(), event), 0);
    11791207}
    11801208

  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h

    r240453 r240498  
    9292    void logUserInteraction(const URL&, CompletionHandler<void()>&&);
    9393    void logUserInteraction(const String& targetPrimaryDomain, CompletionHandler<void()>&&);
     94    void logWebSocketLoading(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen, CompletionHandler<void()>&&);
     95    void logSubresourceLoading(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen, CompletionHandler<void()>&&);
     96    void logSubresourceRedirect(const String& sourcePrimaryDomain, const String& targetPrimaryDomain, CompletionHandler<void()>&&);
    9497    void clearUserInteraction(const URL&, CompletionHandler<void()>&&);
    9598    void clearUserInteraction(const String& targetPrimaryDomain, CompletionHandler<void()>&&);
    … …  
    102105    bool hasStorageAccessForFrame(const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID);
    103106    void requestStorageAccess(const String& resourceDomain, const String& firstPartyDomain, Optional<uint64_t> frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&);
     107    void requestUpdate();
    104108    void setLastSeen(const URL&, Seconds, CompletionHandler<void()>&&);
    105109    void setLastSeen(const String& resourceDomain, Seconds, CompletionHandler<void()>&&);
    … …  
    125129    void isGrandfathered(const URL&, CompletionHandler<void(bool)>&&);
    126130    void isGrandfathered(const String&, CompletionHandler<void(bool)>&&);
    127     void removeAllStorageAccess();
    128131    void removePrevalentDomains(const Vector<String>& domainsToBlock);
    129     void setCacheMaxAgeCapForPrevalentResources(Seconds);
    130132    void setNotifyPagesWhenDataRecordsWereScanned(bool, CompletionHandler<void()>&&);
    131133    void setSubframeUnderTopFrameOrigin(const URL& subframe, const URL& topFrame, CompletionHandler<void()>&&);
    … …  
    146148    void scheduleStatisticsAndDataRecordsProcessing(CompletionHandler<void()>&&);
    147149    void submitTelemetry(CompletionHandler<void()>&&);
    148     void updatePrevalentDomainsToBlockCookiesFor(const Vector<String>& domainsToBlock);
    149150    void scheduleClearInMemoryAndPersistent(ShouldGrandfatherStatistics, CompletionHandler<void()>&&);
    150151    void scheduleClearInMemoryAndPersistent(WallTime modifiedSince, ShouldGrandfatherStatistics, CompletionHandler<void()>&&);

  • trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.cpp

    r240453 r240498  
    596596}
    597597
     598void NetworkConnectionToWebProcess::logWebSocketLoading(PAL::SessionID sessionID, const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen)
     599{
     600#if ENABLE(RESOURCE_LOAD_STATISTICS)
     601    if (auto networkSession = networkProcess().networkSession(sessionID)) {
     602        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
     603            resourceLoadStatistics->logWebSocketLoading(targetPrimaryDomain, mainFramePrimaryDomain, lastSeen, [] { });
     604    }
     605#else
     606    UNUSED_PARAM(sessionID);
     607    UNUSED_PARAM(targetPrimaryDomain);
     608    UNUSED_PARAM(mainFramePrimaryDomain);
     609    UNUSED_PARAM(lastSeen);
     610#endif
     611}
     612
     613void NetworkConnectionToWebProcess::logSubresourceLoading(PAL::SessionID sessionID, const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen)
     614{
     615#if ENABLE(RESOURCE_LOAD_STATISTICS)
     616    if (auto networkSession = networkProcess().networkSession(sessionID)) {
     617        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
     618            resourceLoadStatistics->logSubresourceLoading(targetPrimaryDomain, mainFramePrimaryDomain, lastSeen, [] { });
     619    }
     620#else
     621    UNUSED_PARAM(sessionID);
     622    UNUSED_PARAM(targetPrimaryDomain);
     623    UNUSED_PARAM(mainFramePrimaryDomain);
     624    UNUSED_PARAM(lastSeen);
     625#endif
     626}
     627
     628void NetworkConnectionToWebProcess::logSubresourceRedirect(PAL::SessionID sessionID, const String& sourcePrimaryDomain, const String& targetPrimaryDomain)
     629{
     630#if ENABLE(RESOURCE_LOAD_STATISTICS)
     631    if (auto networkSession = networkProcess().networkSession(sessionID)) {
     632        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
     633            resourceLoadStatistics->logSubresourceRedirect(sourcePrimaryDomain, targetPrimaryDomain, [] { });
     634    }
     635#else
     636    UNUSED_PARAM(sessionID);
     637    UNUSED_PARAM(sourcePrimaryDomain);
     638    UNUSED_PARAM(targetPrimaryDomain);
     639#endif
     640}
     641
     642void NetworkConnectionToWebProcess::requestResourceLoadStatisticsUpdate()
     643{
     644#if ENABLE(RESOURCE_LOAD_STATISTICS)
     645    for (auto& networkSession : networkProcess().networkSessions().values()) {
     646        if (networkSession->sessionID().isEphemeral())
     647            continue;
     648
     649        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
     650            resourceLoadStatistics->requestUpdate();
     651    }
     652#else
     653    UNUSED_PARAM(sessionID);
     654#endif
     655}
     656
    598657void NetworkConnectionToWebProcess::addOriginAccessWhitelistEntry(const String& sourceOrigin, const String& destinationProtocol, const String& destinationHost, bool allowDestinationSubdomains)
    599658{

  • trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.h

    r240453 r240498  
    196196    void removeStorageAccessForAllFramesOnPage(PAL::SessionID, uint64_t pageID);
    197197    void logUserInteraction(PAL::SessionID, const String& topLevelOrigin);
     198    void logWebSocketLoading(PAL::SessionID, const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen);
     199    void logSubresourceLoading(PAL::SessionID, const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen);
     200    void logSubresourceRedirect(PAL::SessionID, const String& sourcePrimaryDomain, const String& targetPrimaryDomain);
     201    void requestResourceLoadStatisticsUpdate();
    198202
    199203    void addOriginAccessWhitelistEntry(const String& sourceOrigin, const String& destinationProtocol, const String& destinationHost, bool allowDestinationSubdomains);

  • trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.messages.in

    r240453 r240498  
    6161#if ENABLE(RESOURCE_LOAD_STATISTICS)
    6262    LogUserInteraction(PAL::SessionID sessionID, String topLevelOrigin)
     63    LogWebSocketLoading(PAL::SessionID sessionID, String targetPrimaryDomain, String mainFramePrimaryDomain, WallTime lastSeen)
     64    LogSubresourceLoading(PAL::SessionID sessionID, String targetPrimaryDomain, String mainFramePrimaryDomain, WallTime lastSeen)
     65    LogSubresourceRedirect(PAL::SessionID sessionID, String sourcePrimaryDomain, String targetPrimaryDomain)
     66    RequestResourceLoadStatisticsUpdate();
    6367#endif
    6468

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp

    r240437 r240498  
    13281328#endif
    13291329
    1330     if (websiteDataTypes.contains(WebsiteDataType::DiskCache) && !sessionID.isEphemeral())
    1331         clearDiskCache(modifiedSince, [clearTasksHandler = WTFMove(clearTasksHandler)] { });
    1332 
    13331330#if ENABLE(RESOURCE_LOAD_STATISTICS)
    13341331    if (websiteDataTypes.contains(WebsiteDataType::ResourceLoadStatistics)) {
    … …  
    13371334                auto deletedTypesRaw = websiteDataTypes.toRaw();
    13381335                auto monitoredTypesRaw = WebResourceLoadStatisticsStore::monitoredDataTypes().toRaw();
    1339 
     1336               
    13401337                // If we are deleting all of the data types that the resource load statistics store monitors
    13411338                // we do not need to re-grandfather old data.
    13421339                auto shouldGrandfather = ((monitoredTypesRaw & deletedTypesRaw) == monitoredTypesRaw) ? ShouldGrandfatherStatistics::No : ShouldGrandfatherStatistics::Yes;
    1343 
     1340               
    13441341                resourceLoadStatistics->scheduleClearInMemoryAndPersistent(modifiedSince, shouldGrandfather, [clearTasksHandler = clearTasksHandler.copyRef()] { });
    13451342            }
    … …  
    13471344    }
    13481345#endif
     1346
     1347    if (websiteDataTypes.contains(WebsiteDataType::DiskCache) && !sessionID.isEphemeral())
     1348        clearDiskCache(modifiedSince, [clearTasksHandler = WTFMove(clearTasksHandler)] { });
    13491349}
    13501350

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.h

    r240366 r240498  
    138138    NetworkSession* networkSession(const PAL::SessionID&) const override;
    139139    void destroySession(const PAL::SessionID&);
    140    
     140
     141    // Needed for test infrastructure
     142    HashMap<PAL::SessionID, Ref<NetworkSession>>& networkSessions() { return m_networkSessions; }
     143
    141144    void forEachNetworkStorageSession(const Function<void(WebCore::NetworkStorageSession&)>&);
    142145    WebCore::NetworkStorageSession* storageSession(const PAL::SessionID&) const;

  • trunk/Source/WebKit/NetworkProcess/NetworkSession.cpp

    r240453 r240498  
    9393        return;
    9494
     95    // FIXME(193728): Support ResourceLoadStatistics for ephemeral sessions, too.
     96    if (m_sessionID.isEphemeral())
     97        return;
     98   
    9599    m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(*this, m_resourceLoadStatisticsDirectory);
    96100}

  • trunk/Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp

    r240360 r240498  
    878878}
    879879
     880void NetworkProcessProxy::logTestingEvent(PAL::SessionID sessionID, const String& event)
     881{
     882    if (auto* websiteDataStore = websiteDataStoreFromSessionID(sessionID))
     883        websiteDataStore->logTestingEvent(event);
     884}
     885
    880886void NetworkProcessProxy::notifyResourceLoadStatisticsProcessed()
    881887{

  • trunk/Source/WebKit/UIProcess/Network/NetworkProcessProxy.h

    r240360 r240498  
    186186    void logGlobalDiagnosticMessageWithValue(const String& message, const String& description, double value, unsigned significantFigures, WebCore::ShouldSample);
    187187#if ENABLE(RESOURCE_LOAD_STATISTICS)
     188    void logTestingEvent(PAL::SessionID, const String& event);
    188189    void notifyResourceLoadStatisticsProcessed();
    189190    void notifyWebsiteDataDeletionForTopPrivatelyOwnedDomainsFinished();

  • trunk/Source/WebKit/UIProcess/Network/NetworkProcessProxy.messages.in

    r240360 r240498  
    4242
    4343#if ENABLE(RESOURCE_LOAD_STATISTICS)
     44    LogTestingEvent(PAL::SessionID sessionID, String event)
    4445    NotifyResourceLoadStatisticsProcessed()
    4546    NotifyWebsiteDataDeletionForTopPrivatelyOwnedDomainsFinished()

  • trunk/Source/WebKit/UIProcess/WebProcessPool.cpp

    r240453 r240498  
    556556    SandboxExtension::createHandleForReadWriteDirectory(parameters.defaultDataStoreParameters.networkSessionParameters.resourceLoadStatisticsDirectory, parameters.defaultDataStoreParameters.networkSessionParameters.resourceLoadStatisticsDirectoryExtensionHandle);
    557557
    558     parameters.defaultDataStoreParameters.networkSessionParameters.enableResourceLoadStatistics = false; // FIXME(193297): Turn on when the feature is on. (m_configuration->resourceLoadStatisticsEnabled()?)
     558    bool enableResourceLoadStatistics = false;
     559    if (withWebsiteDataStore)
     560        enableResourceLoadStatistics = withWebsiteDataStore->resourceLoadStatisticsEnabled();
     561    else if (m_websiteDataStore)
     562        enableResourceLoadStatistics = m_websiteDataStore->resourceLoadStatisticsEnabled();
     563
     564    parameters.defaultDataStoreParameters.networkSessionParameters.enableResourceLoadStatistics = enableResourceLoadStatistics;
    559565
    560566    // Add any platform specific parameters

  • trunk/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm

    r240453 r240498  
    103103        WTFMove(resourceLoadStatisticsDirectory),
    104104        WTFMove(resourceLoadStatisticsDirectoryHandle),
    105         false // FIXME(193297): Switch to m_configuration->resourceLoadStatisticsEnabled()
     105        false
    106106    };
    107107

  • trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp

    r240453 r240498  
    12531253        // If we are deleting all of the data types that the resource load statistics store monitors
    12541254        // we do not need to re-grandfather old data.
     1255        auto shouldGrandfather = ((monitoredTypesRaw & deletedTypesRaw) == monitoredTypesRaw) ? ShouldGrandfatherStatistics::No : ShouldGrandfatherStatistics::Yes;
     1256
    12551257        callbackAggregator->addPendingCallback();
    1256         if ((monitoredTypesRaw & deletedTypesRaw) == monitoredTypesRaw)
    1257             m_resourceLoadStatistics->scheduleClearInMemoryAndPersistent(ShouldGrandfatherStatistics::No, [callbackAggregator] {
    1258                 callbackAggregator->removePendingCallback();
    1259             });
    1260         else
    1261             m_resourceLoadStatistics->scheduleClearInMemoryAndPersistent(ShouldGrandfatherStatistics::Yes, [callbackAggregator] {
    1262                 callbackAggregator->removePendingCallback();
    1263             });
     1258        m_resourceLoadStatistics->scheduleClearInMemoryAndPersistent(shouldGrandfather, [callbackAggregator] {
     1259            callbackAggregator->removePendingCallback();
     1260        });
    12641261
    12651262        callbackAggregator->addPendingCallback();
    … …  
    23262323
    23272324    if (enabled) {
    2328         // FIXME(193297): Remove this assert
    2329         ASSERT(!m_resourceLoadStatistics);
    23302325        enableResourceLoadStatisticsAndSetTestingCallback(nullptr);
    23312326        return;
    23322327    }
    2333 
    2334     // FIXME(193297): Remove these two lines
    2335     unregisterWebResourceLoadStatisticsStoreAsMessageReceiver();
    2336     m_resourceLoadStatistics = nullptr;
    23372328
    23382329    for (auto& processPool : processPools(std::numeric_limits<size_t>::max(), false)) {
    … …  
    23962387    setStatisticsTestingCallback(WTFMove(callback));
    23972388
    2398     // FIXME(193297): Remove this check
    2399     if (m_resourceLoadStatistics)
    2400         return;
    2401 
    24022389    resolveDirectoriesIfNecessary();
    2403 
    2404     // FIXME(193297): Remove these two lines
    2405     m_resourceLoadStatistics = WebResourceLoadStatisticsStore::create(*this);
    2406     registerWebResourceLoadStatisticsStoreAsMessageReceiver();
    24072390
    24082391    for (auto& processPool : processPools(std::numeric_limits<size_t>::max(), false)) {

  • trunk/Source/WebKit/WebProcess/WebProcess.cpp

    r240453 r240498  
    208208    ResourceLoadObserver::shared().setNotificationCallback([this] (Vector<ResourceLoadStatistics>&& statistics) {
    209209        parentProcessConnection()->send(Messages::WebResourceLoadStatisticsStore::ResourceLoadStatisticsUpdated(WTFMove(statistics)), 0);
     210
     211        m_networkProcessConnection->connection().send(Messages::NetworkConnectionToWebProcess::RequestResourceLoadStatisticsUpdate(), 0);
    210212    });
    211213
    … …  
    389391        m_networkProcessConnection->connection().send(Messages::NetworkConnectionToWebProcess::LogUserInteraction(sessionID, topLevelOrigin), 0);
    390392    });
     393
     394    ResourceLoadObserver::shared().setLogWebSocketLoadingNotificationCallback([this] (PAL::SessionID sessionID, const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen) {
     395        m_networkProcessConnection->connection().send(Messages::NetworkConnectionToWebProcess::LogWebSocketLoading(sessionID, targetPrimaryDomain, mainFramePrimaryDomain, lastSeen), 0);
     396    });
     397   
     398    ResourceLoadObserver::shared().setLogSubresourceLoadingNotificationCallback([this] (PAL::SessionID sessionID, const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, WallTime lastSeen) {
     399        m_networkProcessConnection->connection().send(Messages::NetworkConnectionToWebProcess::LogSubresourceLoading(sessionID, targetPrimaryDomain, mainFramePrimaryDomain, lastSeen), 0);
     400    });
     401
     402    ResourceLoadObserver::shared().setLogSubresourceRedirectNotificationCallback([this] (PAL::SessionID sessionID, const String& sourcePrimaryDomain, const String& targetPrimaryDomain) {
     403        m_networkProcessConnection->connection().send(Messages::NetworkConnectionToWebProcess::LogSubresourceRedirect(sessionID, sourcePrimaryDomain, targetPrimaryDomain), 0);
     404    });
    391405#endif
    392406

  • trunk/Tools/ChangeLog

    r240490 r240498  
     12019-01-25  Brent Fulgham  <bfulgham@apple.com>
     2
     3        Activate the WebResourceLoadStatisticsStore in the NetworkProcess and deactivate it in the UIProcess.
     4        https://bugs.webkit.org/show_bug.cgi?id=193297
     5        <rdar://problem/47158841>
     6
     7        Reviewed by Alex Christensen.
     8
     9        * TestWebKitAPI/Tests/WebKit Cocoa/ResourceLoadStatistics.mm:
     10            Revise tests to ensure a running NetworkProcess before attempting ResourceLoadStatistics operations.
     11        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
     12        (WTR::TestRunner::installStatisticsDidScanDataRecordsCallback): Simplify test configuration by
     13        activating the message used to trigger the callback when it is set.
     14
    1152019-01-25  Alex Christensen  <achristensen@webkit.org>
    216

  • trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm

    r236690 r240498  
    5858TEST(ResourceLoadStatistics, GrandfatherCallback)
    5959{
     60    // We need an active NetworkProcess to perform ResourceLoadStatistics operations.
     61    auto webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600)]);
     62
    6063    auto *dataStore = [WKWebsiteDataStore defaultDataStore];
    6164    [dataStore _setResourceLoadStatisticsEnabled:NO];
    … …  
    119122TEST(ResourceLoadStatistics, ShouldNotGrandfatherOnStartup)
    120123{
     124    // We need an active NetworkProcess to perform ResourceLoadStatistics operations.
     125    auto webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600)]);
     126
    121127    auto *dataStore = [WKWebsiteDataStore defaultDataStore];
    122128    [dataStore _setResourceLoadStatisticsEnabled:NO];
    … …  
    147153    EXPECT_EQ((size_t)0, [sharedProcessPool _pluginProcessCount]);
    148154
     155    // We need an active NetworkProcess to perform ResourceLoadStatistics operations.
     156    auto webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600)]);
     157
    149158    auto *dataStore = [WKWebsiteDataStore defaultDataStore];
    150159    [dataStore _setResourceLoadStatisticsEnabled:NO];
    … …  
    168177
    169178    EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:targetURL.path]);
     179
     180    webView.clear();
    170181
    171182    EXPECT_EQ((size_t)0, [sharedProcessPool _pluginProcessCount]);

  • trunk/Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp

    r240453 r240498  
    18611861{
    18621862    cacheTestRunnerCallback(StatisticsDidScanDataRecordsCallbackID, callback);
     1863
     1864    bool notifyPagesWhenDataRecordsWereScanned = !!callback;
     1865
     1866    // Setting a callback implies we expect to receive callbacks. So register for them.
     1867    WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("StatisticsNotifyPagesWhenDataRecordsWereScanned"));
     1868    WKRetainPtr<WKBooleanRef> messageBody(AdoptWK, WKBooleanCreate(notifyPagesWhenDataRecordsWereScanned));
     1869    WKBundlePostSynchronousMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get(), nullptr);
    18631870}
    18641871
Note: See TracChangeset for help on using the changeset viewer.