Changeset 254296 in webkit


Ignore:
Timestamp:
Jan 9, 2020, 2:01:46 PM (6 years ago)
Author:
wilander@apple.com
Message:

Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
​https://bugs.webkit.org/show_bug.cgi?id=205522
<rdar://problem/58125759>

Reviewed by Chris Dumez.

Source/WebKit:

The purpose of this patch is to capture navigations that happen programmatically
after the document has loaded. These are delayed redirects and should be counted
as redirects in ITP.

To achieve this, a timestamp is captured in
WebPageProxy::didFinishDocumentLoadForFrame() to be able to calculate how much time
has passed since that timestamp in the IPC sent from
WebPageProxy::logFrameNavigation(). The IPC also gets information on whether
userInitiatedActivity exists so that ITP can decide whether to treat the navigation
as triggered by the user or not.

A new test was added and existing test expectations were updated.

  • NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:

(WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):

Now takes the extra parameters delayAfterMainFrameDocumentLoad
and wasPotentiallyInitiatedByUser.

  • NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
  • NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:

(WebKit::ResourceLoadStatisticsMemoryStore::logFrameNavigation):

Now takes the extra parameters delayAfterMainFrameDocumentLoad
and wasPotentiallyInitiatedByUser.

  • NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
  • NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:

(WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):

Now takes the extra parameters delayAfterMainFrameDocumentLoad
and wasPotentiallyInitiatedByUser.
The deleted WebResourceLoadStatisticsStore::logFrameNavigation() was dead code.

  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
  • NetworkProcess/NetworkProcess.cpp:

(WebKit::NetworkProcess::logFrameNavigation):

Now takes the extra parameters delayAfterMainFrameDocumentLoad
and wasPotentiallyInitiatedByUser.

  • NetworkProcess/NetworkProcess.h:
  • NetworkProcess/NetworkProcess.messages.in:
  • UIProcess/API/Cocoa/WKWebsiteDataStore.mm:

(-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:]): Deleted.

Dead code.

  • UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
  • UIProcess/WebPageProxy.cpp:

(WebKit::WebPageProxy::didFinishDocumentLoadForFrame):

Now captures a timestamp.

(WebKit::WebPageProxy::decidePolicyForNavigationAction):

Removed dead code.

(WebKit::WebPageProxy::logFrameNavigation):

Now sends the diff between now and the timestamp captured in
WebPageProxy::didFinishDocumentLoadForFrame().

  • UIProcess/WebPageProxy.h:
  • UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:

(WebKit::WebsiteDataStore::platformInitialize):

Removed dead code.

(WebKit::WebsiteDataStore::platformDestroy):

Removed dead code.

  • UIProcess/WebsiteData/WebsiteDataStore.h:

(WebKit::WebsiteDataStore::resourceLoadStatistics const): Deleted.

Dead code.

Tools:

  • WebKitTestRunner/cocoa/TestControllerCocoa.mm:

(WTR::initializeWebViewConfiguration):

Removed dead code.

LayoutTests:

Results updated with additional data now that delayed redirects are captured.

  • http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
  • http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
  • http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt: Added.
  • http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html: Added.
  • http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt: Added.
  • http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html: Added.
  • http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
  • http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:
Location:
trunk
Files:
2 added
24 edited
2 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r254285 r254296  
     12020-01-09  John Wilander  <wilander@apple.com>
     2
     3        Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
     4        https://bugs.webkit.org/show_bug.cgi?id=205522
     5        <rdar://problem/58125759>
     6
     7        Reviewed by Chris Dumez.
     8
     9        Results updated with additional data now that delayed redirects are captured.
     10
     11        * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
     12        * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
     13        * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt: Added.
     14        * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html: Added.
     15        * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt: Added.
     16        * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html: Added.
     17        * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
     18        * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:
     19
    1202020-01-09  Pablo Saavedra  <psaavedra@igalia.com>
    221

  • trunk/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt

    r252641 r254296  
    77    mostRecentUserInteraction: -1
    88    grandfathered: No
     9    TopFrameUniqueRedirectsTo:
     10        localhost
    911    IsScheduledForAllButCookieDataRemoval: No
    1012    isPrevalentResource: Yes
    … …  
    2325    mostRecentUserInteraction: -1
    2426    grandfathered: No
     27    TopFrameUniqueRedirectsFrom:
     28        127.0.0.1
    2529    TopFrameLinkDecorationsFrom:
    2630        127.0.0.1

  • trunk/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt

    r252641 r254296  
    77    mostRecentUserInteraction: -1
    88    grandfathered: No
     9    topFrameUniqueRedirectsFrom:
     10        127.0.0.1
    911    topFrameLinkDecorationsFrom:
    1012        127.0.0.1
    … …  
    1719    mostRecentUserInteraction: -1
    1820    grandfathered: No
     21    topFrameUniqueRedirectsTo:
     22        localhost
    1923    gotLinkDecorationFromPrevalentResource: No
    2024    isPrevalentResource: Yes

  • trunk/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt

    r254295 r254296  
    1 Test logging of link decorated cross-site navigations from a prevalent resource.
     1Non-user initiated, delayed cross-site navigations done client-side should be detected as top frame redirects by ITP.
    22
    33Resource load statistics:
    … …  
    77    mostRecentUserInteraction: -1
    88    grandfathered: No
    9     IsScheduledForAllButCookieDataRemoval: No
    10     isPrevalentResource: Yes
    11     isVeryPrevalentResource: No
    12     dataRecordsRemoved: 0
    13 Registrable domain: 127.0.0.2
    14     hadUserInteraction: No
    15     mostRecentUserInteraction: -1
    16     grandfathered: No
     9    TopFrameUniqueRedirectsTo:
     10        localhost
    1711    IsScheduledForAllButCookieDataRemoval: No
    1812    isPrevalentResource: No
    … …  
    2317    mostRecentUserInteraction: -1
    2418    grandfathered: No
    25     TopFrameLinkDecorationsFrom:
     19    TopFrameUniqueRedirectsFrom:
    2620        127.0.0.1
    27         127.0.0.2
    28     IsScheduledForAllButCookieDataRemoval: Yes
     21    IsScheduledForAllButCookieDataRemoval: No
    2922    isPrevalentResource: No
    3023    isVeryPrevalentResource: No

  • trunk/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt

    r254295 r254296  
    1 Test logging of link decorated cross-site navigations from a prevalent resource.
     1Non-user initiated, delayed cross-site navigations done client-side should be detected as top frame redirects by ITP.
    22
    33Resource load statistics:
    … …  
    77    mostRecentUserInteraction: -1
    88    grandfathered: No
    9     topFrameLinkDecorationsFrom:
     9    topFrameUniqueRedirectsFrom:
    1010        127.0.0.1
    11     gotLinkDecorationFromPrevalentResource: Yes
     11    gotLinkDecorationFromPrevalentResource: No
    1212    isPrevalentResource: No
    1313    isVeryPrevalentResource: No
    … …  
    1717    mostRecentUserInteraction: -1
    1818    grandfathered: No
     19    topFrameUniqueRedirectsTo:
     20        localhost
    1921    gotLinkDecorationFromPrevalentResource: No
    20     isPrevalentResource: Yes
     22    isPrevalentResource: No
    2123    isVeryPrevalentResource: No
    2224    dataRecordsRemoved: 0

  • trunk/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt

    r253863 r254296  
    1515    mostRecentUserInteraction: within 24 hours
    1616    grandfathered: No
     17    TopFrameUniqueRedirectsTo:
     18        localhost
     19    TopFrameUniqueRedirectsFrom:
     20        localhost
    1721    IsScheduledForAllButCookieDataRemoval: No
    18     isPrevalentResource: No
     22    isPrevalentResource: Yes
    1923    isVeryPrevalentResource: No
    2024    dataRecordsRemoved: 0
    … …  
    2428    grandfathered: No
    2529    StorageAccessUnderTopFrameDomains:
     30        127.0.0.1
     31    TopFrameUniqueRedirectsTo:
     32        127.0.0.1
     33    TopFrameUniqueRedirectsFrom:
    2634        127.0.0.1
    2735    IsScheduledForAllButCookieDataRemoval: No

  • trunk/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt

    r253863 r254296  
    1717    storageAccessUnderTopFrameDomains:
    1818        127.0.0.1
     19    topFrameUniqueRedirectsTo:
     20        127.0.0.1
     21    topFrameUniqueRedirectsFrom:
     22        127.0.0.1
    1923    gotLinkDecorationFromPrevalentResource: No
    2024    subframeUnderTopFrameDomains:
    … …  
    2731    mostRecentUserInteraction: within 24 hours
    2832    grandfathered: No
     33    topFrameUniqueRedirectsTo:
     34        localhost
     35    topFrameUniqueRedirectsFrom:
     36        localhost
    2937    gotLinkDecorationFromPrevalentResource: No
    30     isPrevalentResource: No
     38    isPrevalentResource: Yes
    3139    isVeryPrevalentResource: No
    3240    dataRecordsRemoved: 0

  • trunk/Source/WebKit/ChangeLog

    r254294 r254296  
     12020-01-09  John Wilander  <wilander@apple.com>
     2
     3        Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
     4        https://bugs.webkit.org/show_bug.cgi?id=205522
     5        <rdar://problem/58125759>
     6
     7        Reviewed by Chris Dumez.
     8
     9        The purpose of this patch is to capture navigations that happen programmatically
     10        after the document has loaded. These are delayed redirects and should be counted
     11        as redirects in ITP.
     12
     13        To achieve this, a timestamp is captured in
     14        WebPageProxy::didFinishDocumentLoadForFrame() to be able to calculate how much time
     15        has passed since that timestamp in the IPC sent from
     16        WebPageProxy::logFrameNavigation(). The IPC also gets information on whether
     17        userInitiatedActivity exists so that ITP can decide whether to treat the navigation
     18        as triggered by the user or not.
     19
     20        A new test was added and existing test expectations were updated.
     21
     22        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
     23        (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
     24            Now takes the extra parameters delayAfterMainFrameDocumentLoad
     25            and wasPotentiallyInitiatedByUser.
     26        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
     27        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
     28        (WebKit::ResourceLoadStatisticsMemoryStore::logFrameNavigation):
     29            Now takes the extra parameters delayAfterMainFrameDocumentLoad
     30            and wasPotentiallyInitiatedByUser.
     31        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
     32        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
     33        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
     34        (WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):
     35            Now takes the extra parameters delayAfterMainFrameDocumentLoad
     36            and wasPotentiallyInitiatedByUser.
     37            The deleted WebResourceLoadStatisticsStore::logFrameNavigation() was dead code.
     38        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
     39        * NetworkProcess/NetworkProcess.cpp:
     40        (WebKit::NetworkProcess::logFrameNavigation):
     41            Now takes the extra parameters delayAfterMainFrameDocumentLoad
     42            and wasPotentiallyInitiatedByUser.
     43        * NetworkProcess/NetworkProcess.h:
     44        * NetworkProcess/NetworkProcess.messages.in:
     45        * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
     46        (-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:]): Deleted.
     47            Dead code.
     48        * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
     49        * UIProcess/WebPageProxy.cpp:
     50        (WebKit::WebPageProxy::didFinishDocumentLoadForFrame):
     51            Now captures a timestamp.
     52        (WebKit::WebPageProxy::decidePolicyForNavigationAction):
     53            Removed dead code.
     54        (WebKit::WebPageProxy::logFrameNavigation):
     55            Now sends the diff between now and the timestamp captured in
     56            WebPageProxy::didFinishDocumentLoadForFrame().
     57        * UIProcess/WebPageProxy.h:
     58        * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
     59        (WebKit::WebsiteDataStore::platformInitialize):
     60            Removed dead code.
     61        (WebKit::WebsiteDataStore::platformDestroy):
     62            Removed dead code.
     63        * UIProcess/WebsiteData/WebsiteDataStore.h:
     64        (WebKit::WebsiteDataStore::resourceLoadStatistics const): Deleted.
     65            Dead code.
     66
    1672020-01-09  Andres Gonzalez  <andresg_22@apple.com>
    268

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp

    r253863 r254296  
    15091509}
    15101510
    1511 void ResourceLoadStatisticsDatabaseStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
     1511void ResourceLoadStatisticsDatabaseStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
    15121512{
    15131513    ASSERT(!RunLoop::isMain());
    … …  
    15241524    }
    15251525
    1526     if (isRedirect && !areTargetAndSourceDomainsSameSite) {
     1526    if (!areTargetAndSourceDomainsSameSite) {
    15271527        if (isMainFrame) {
    1528             auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
    1529             auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
    1530             insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
    1531             insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
    1532         } else {
     1528            bool wasNavigatedAfterShortDelayWithoutUserInteraction = !wasPotentiallyInitiatedByUser && delayAfterMainFrameDocumentLoad < parameters().minDelayAfterMainFrameDocumentLoadToNotBeARedirect;
     1529            if (isRedirect || wasNavigatedAfterShortDelayWithoutUserInteraction) {
     1530                auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
     1531                auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
     1532                insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
     1533                insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
     1534                statisticsWereUpdated = true;
     1535            }
     1536        } else if (isRedirect) {
    15331537            auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
    15341538            auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
    15351539            insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
    15361540            insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
     1541            statisticsWereUpdated = true;
    15371542        }
    1538         statisticsWereUpdated = true;
    15391543    }
    15401544

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h

    r253863 r254296  
    125125    void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebCore::StorageAccessPromptWasShown, CompletionHandler<void(WebCore::StorageAccessWasGranted)>&&) override;
    126126
    127     void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) override;
     127    void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser) override;
    128128    void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&) override;
    129129    void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&) override;

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp

    r253863 r254296  
    429429}
    430430
    431 void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
     431void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
    432432{
    433433    ASSERT(!RunLoop::isMain());
    … …  
    444444    }
    445445
    446     if (isRedirect && !areTargetAndSourceDomainsSameSite) {
     446    if (!areTargetAndSourceDomainsSameSite) {
    447447        if (isMainFrame) {
    448448            auto& redirectingDomainStatistics = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
    449             if (redirectingDomainStatistics.topFrameUniqueRedirectsTo.add(targetDomain).isNewEntry)
    450                 statisticsWereUpdated = true;
    451             auto& targetStatistics = ensureResourceStatisticsForRegistrableDomain(targetDomain);
    452             if (targetStatistics.topFrameUniqueRedirectsFrom.add(sourceDomain).isNewEntry)
    453                 statisticsWereUpdated = true;
    454         } else {
     449            bool wasNavigatedAfterShortDelayWithoutUserInteraction = !wasPotentiallyInitiatedByUser && delayAfterMainFrameDocumentLoad < parameters().minDelayAfterMainFrameDocumentLoadToNotBeARedirect;
     450            if (isRedirect || wasNavigatedAfterShortDelayWithoutUserInteraction) {
     451                if (redirectingDomainStatistics.topFrameUniqueRedirectsTo.add(targetDomain).isNewEntry)
     452                    statisticsWereUpdated = true;
     453                auto& targetStatistics = ensureResourceStatisticsForRegistrableDomain(targetDomain);
     454                if (targetStatistics.topFrameUniqueRedirectsFrom.add(sourceDomain).isNewEntry)
     455                    statisticsWereUpdated = true;
     456            }
     457        } else if (isRedirect) {
    455458            auto& redirectingDomainStatistics = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
    456459            if (redirectingDomainStatistics.subresourceUniqueRedirectsTo.add(targetDomain).isNewEntry)

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h

    r253484 r254296  
    105105    void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebCore::StorageAccessPromptWasShown, CompletionHandler<void(WebCore::StorageAccessWasGranted)>&&) override;
    106106
    107     void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) override;
     107    void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser) override;
    108108    void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&) override;
    109109    void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&) override;

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h

    r254293 r254296  
    172172    virtual void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebCore::StorageAccessPromptWasShown, CompletionHandler<void(WebCore::StorageAccessWasGranted)>&&) = 0;
    173173
    174     virtual void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) = 0;
     174    virtual void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser) = 0;
    175175    virtual void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&) = 0;
    176176    virtual void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&) = 0;
    … …  
    223223        Seconds cacheMaxAgeCapTime { 24_h * 7 };
    224224        Seconds clientSideCookiesAgeCapTime { 24_h * 7 };
     225        Seconds minDelayAfterMainFrameDocumentLoadToNotBeARedirect { 5_s };
    225226        bool shouldNotifyPagesWhenDataRecordsWereScanned { false };
    226227        bool shouldClassifyResourcesBeforeDataRecordsRemoval { true };

  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp

    r253484 r254296  
    555555}
    556556
    557 void WebResourceLoadStatisticsStore::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL)
    558 {
    559     ASSERT(RunLoop::isMain());
    560 
    561     auto sourceURL = redirectURL;
    562     bool isRedirect = !redirectURL.isNull();
    563     if (!isRedirect) {
    564         sourceURL = frame.url();
    565         if (sourceURL.isNull())
    566             sourceURL = pageURL;
    567     }
    568 
    569     auto& targetURL = request.url();
    570 
    571     if (!targetURL.isValid() || !pageURL.isValid())
    572         return;
    573 
    574     auto targetHost = targetURL.host();
    575     auto mainFrameHost = pageURL.host();
    576 
    577     if (targetHost.isEmpty() || mainFrameHost.isEmpty() || targetHost == sourceURL.host())
    578         return;
    579 
    580     RegistrableDomain targetDomain { targetURL };
    581     RegistrableDomain topFrameDomain { pageURL };
    582     RegistrableDomain sourceDomain { sourceURL };
    583 
    584     logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, frame.isMainFrame());
    585 }
    586 
    587 void WebResourceLoadStatisticsStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
    588 {
    589     ASSERT(RunLoop::isMain());
    590 
    591     postTask([this, targetDomain = targetDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), sourceDomain = sourceDomain.isolatedCopy(), isRedirect, isMainFrame] {
    592         if (m_statisticsStore)
    593             m_statisticsStore->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame);
     557void WebResourceLoadStatisticsStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
     558{
     559    ASSERT(RunLoop::isMain());
     560
     561    postTask([this, targetDomain = targetDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), sourceDomain = sourceDomain.isolatedCopy(), isRedirect, isMainFrame, delayAfterMainFrameDocumentLoad, wasPotentiallyInitiatedByUser] {
     562        if (m_statisticsStore)
     563            m_statisticsStore->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame, delayAfterMainFrameDocumentLoad, wasPotentiallyInitiatedByUser);
    594564    });
    595565}

  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h

    r253863 r254296  
    211211    void applicationWillTerminate();
    212212
    213     void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL);
    214     void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame);
     213    void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser);
    215214    void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&);
    216215    void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&, CompletionHandler<void()>&&);

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp

    r253740 r254296  
    985985}
    986986
    987 void NetworkProcess::logFrameNavigation(PAL::SessionID sessionID, const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
    988 {
    989     if (auto* networkSession = this->networkSession(sessionID)) {
    990         if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
    991             resourceLoadStatistics->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame);
     987void NetworkProcess::logFrameNavigation(PAL::SessionID sessionID, const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
     988{
     989    if (auto* networkSession = this->networkSession(sessionID)) {
     990        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
     991            resourceLoadStatistics->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame, delayAfterMainFrameDocumentLoad, wasPotentiallyInitiatedByUser);
    992992    } else
    993993        ASSERT_NOT_REACHED();

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.h

    r254081 r254296  
    225225    void hasLocalStorage(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
    226226    void getAllStorageAccessEntries(PAL::SessionID, CompletionHandler<void(Vector<String> domains)>&&);
    227     void logFrameNavigation(PAL::SessionID, const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame);
     227    void logFrameNavigation(PAL::SessionID, const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser);
    228228    void logUserInteraction(PAL::SessionID, const TopFrameDomain&, CompletionHandler<void()>&&);
    229229    void removePrevalentDomains(PAL::SessionID, const Vector<RegistrableDomain>&);

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.messages.in

    r253484 r254296  
    103103    IsRegisteredAsSubFrameUnder(PAL::SessionID sessionID, WebCore::RegistrableDomain subFrameDomain, WebCore::RegistrableDomain topFrameDomain) -> (bool isSubframeUnder) Async
    104104    IsRegisteredAsSubresourceUnder(PAL::SessionID sessionID, WebCore::RegistrableDomain subresourceDomain, WebCore::RegistrableDomain topFrameDomain) -> (bool isSubresourceUnder) Async
    105     LogFrameNavigation(PAL::SessionID sessionID, WebCore::RegistrableDomain targetDomain, WebCore::RegistrableDomain topFrameDomain, WebCore::RegistrableDomain sourceDomain, bool isRedirect, bool isMainFrame)
     105    LogFrameNavigation(PAL::SessionID sessionID, WebCore::RegistrableDomain targetDomain, WebCore::RegistrableDomain topFrameDomain, WebCore::RegistrableDomain sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
    106106    LogUserInteraction(PAL::SessionID sessionID, WebCore::RegistrableDomain topFrameDomain) -> () Async
    107107    RemovePrevalentDomains(PAL::SessionID sessionID, Vector<WebCore::RegistrableDomain> domainsWithInteraction)

  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm

    r254241 r254296  
    396396}
    397397
    398 - (void)_resourceLoadStatisticsSetShouldSubmitTelemetry:(BOOL)value
    399 {
    400 #if ENABLE(RESOURCE_LOAD_STATISTICS)
    401     auto* store = _websiteDataStore->resourceLoadStatistics();
    402     if (!store)
    403         return;
    404 
    405     store->setShouldSubmitTelemetry(value);
    406 #endif
    407 }
    408 
    409398- (void)_setResourceLoadStatisticsTestingCallback:(void (^)(WKWebsiteDataStore *, NSString *))callback
    410399{

  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h

    r254085 r254296  
    6262@property (nonatomic, readonly) NSURL *_indexedDBDatabaseDirectory;
    6363
    64 - (void)_resourceLoadStatisticsSetShouldSubmitTelemetry:(BOOL)value WK_API_AVAILABLE(macos(10.13), ios(11.0));
    6564- (void)_setResourceLoadStatisticsTestingCallback:(nullable void (^)(WKWebsiteDataStore *, NSString *))callback WK_API_AVAILABLE(macos(10.13), ios(11.0));
    6665- (void)_getAllStorageAccessEntriesFor:(WKWebView *)webView completionHandler:(void (^)(NSArray<NSString *> *domains))completionHandler WK_API_AVAILABLE(macos(10.14), ios(12.0));

  • trunk/Source/WebKit/UIProcess/WebPageProxy.cpp

    r254183 r254296  
    45734573        navigation = navigationState().navigation(navigationID);
    45744574
    4575     if (frame->isMainFrame())
     4575    if (frame->isMainFrame()) {
    45764576        m_navigationClient->didFinishDocumentLoad(*this, navigation.get(), m_process->transformHandlesToObjects(userData.object()).get());
     4577#if ENABLE(RESOURCE_LOAD_STATISTICS)
     4578        m_didFinishDocumentLoadForMainFrameTimestamp = MonotonicTime::now();
     4579#endif
     4580    }
    45774581}
    45784582
    … …  
    49874991    WebFrameProxy* originatingFrame = originatingFrameInfoData.frameID ? process->webFrame(*originatingFrameInfoData.frameID) : nullptr;
    49884992
     4993    auto userInitiatedActivity = process->userInitiatedActivity(navigationActionData.userGestureTokenIdentifier);
    49894994#if ENABLE(RESOURCE_LOAD_STATISTICS)
    4990     if (auto* resourceLoadStatisticsStore = websiteDataStore().resourceLoadStatistics())
    4991         resourceLoadStatisticsStore->logFrameNavigation(frame, URL(URL(), m_pageLoadState.url()), request, redirectResponse.url());
    4992     logFrameNavigation(frame, URL(URL(), m_pageLoadState.url()), request, redirectResponse.url());
     4995    logFrameNavigation(frame, URL(URL(), m_pageLoadState.url()), request, redirectResponse.url(), !!userInitiatedActivity);
    49934996#endif
    49944997
    … …  
    50055008        }
    50065009
    5007         auto userInitiatedActivity = process->userInitiatedActivity(navigationActionData.userGestureTokenIdentifier);
    50085010        bool shouldOpenAppLinks = !m_shouldSuppressAppLinksInNextNavigationPolicyDecision
    50095011            && destinationFrameInfo->isMainFrame()
    … …  
    50495051
    50505052#if ENABLE(RESOURCE_LOAD_STATISTICS)
    5051 void WebPageProxy::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL)
     5053void WebPageProxy::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL, bool wasPotentiallyInitiatedByUser)
    50525054{
    50535055    ASSERT(RunLoop::isMain());
    … …  
    50725074        return;
    50735075
    5074     m_process->processPool().sendToNetworkingProcess(Messages::NetworkProcess::LogFrameNavigation(m_websiteDataStore->sessionID(), RegistrableDomain { targetURL }, RegistrableDomain { pageURL }, RegistrableDomain { sourceURL }, isRedirect, frame.isMainFrame()));
     5076    m_process->processPool().sendToNetworkingProcess(Messages::NetworkProcess::LogFrameNavigation(m_websiteDataStore->sessionID(), RegistrableDomain { targetURL }, RegistrableDomain { pageURL }, RegistrableDomain { sourceURL }, isRedirect, frame.isMainFrame(), MonotonicTime::now() - m_didFinishDocumentLoadForMainFrameTimestamp, wasPotentiallyInitiatedByUser));
    50755077}
    50765078#endif

  • trunk/Source/WebKit/UIProcess/WebPageProxy.h

    r254183 r254296  
    21572157
    21582158#if ENABLE(RESOURCE_LOAD_STATISTICS)
    2159     void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL);
     2159    void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL, bool wasPotentiallyInitiatedByUser);
    21602160#endif
    21612161
    … …  
    26612661    std::unique_ptr<WebDeviceOrientationUpdateProviderProxy> m_webDeviceOrientationUpdateProviderProxy;
    26622662#endif
     2663
     2664#if ENABLE(RESOURCE_LOAD_STATISTICS)
     2665    MonotonicTime m_didFinishDocumentLoadForMainFrameTimestamp;
     2666#endif
    26632667};
    26642668

  • trunk/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm

    r254293 r254296  
    5252
    5353namespace WebKit {
    54 
    55 static id terminationObserver;
    5654
    5755static HashSet<WebsiteDataStore*>& dataStores()
    … …  
    230228void WebsiteDataStore::platformInitialize()
    231229{
    232     if (!terminationObserver) {
    233         ASSERT(dataStores().isEmpty());
    234 
    235 #if PLATFORM(MAC)
    236         NSString *notificationName = NSApplicationWillTerminateNotification;
    237 #else
    238         NSString *notificationName = UIApplicationWillTerminateNotification;
    239 #endif
    240         terminationObserver = [[NSNotificationCenter defaultCenter] addObserverForName:notificationName object:nil queue:nil usingBlock:^(NSNotification *note) {
    241             for (auto& dataStore : dataStores()) {
    242 #if ENABLE(RESOURCE_LOAD_STATISTICS)
    243                 if (dataStore->m_resourceLoadStatistics)
    244                     dataStore->m_resourceLoadStatistics->applicationWillTerminate();
    245 #endif
    246             }
    247         }];
    248     }
    249 
    250230    ASSERT(!dataStores().contains(this));
    251231    dataStores().add(this);
    … …  
    254234void WebsiteDataStore::platformDestroy()
    255235{
    256 #if ENABLE(RESOURCE_LOAD_STATISTICS)
    257     if (m_resourceLoadStatistics)
    258         m_resourceLoadStatistics->applicationWillTerminate();
    259 #endif
    260 
    261236    ASSERT(dataStores().contains(this));
    262237    dataStores().remove(this);
    263 
    264     if (dataStores().isEmpty()) {
    265         [[NSNotificationCenter defaultCenter] removeObserver:terminationObserver];
    266         terminationObserver = nil;
    267     }
    268238}
    269239

  • trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h

    r254085 r254296  
    130130
    131131#if ENABLE(RESOURCE_LOAD_STATISTICS)
    132     WebResourceLoadStatisticsStore* resourceLoadStatistics() const { return m_resourceLoadStatistics.get(); }
    133132    void clearResourceLoadStatisticsInWebProcesses(CompletionHandler<void()>&&);
    134133#endif
    … …  
    335334
    336335#if ENABLE(RESOURCE_LOAD_STATISTICS)
    337     RefPtr<WebResourceLoadStatisticsStore> m_resourceLoadStatistics;
    338336    bool m_resourceLoadStatisticsDebugMode { false };
    339337    bool m_resourceLoadStatisticsEnabled { false };

  • trunk/Tools/ChangeLog

    r254290 r254296  
     12020-01-09  John Wilander  <wilander@apple.com>
     2
     3        Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
     4        https://bugs.webkit.org/show_bug.cgi?id=205522
     5        <rdar://problem/58125759>
     6
     7        Reviewed by Chris Dumez.
     8
     9        * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
     10        (WTR::initializeWebViewConfiguration):
     11            Removed dead code.
     12
    1132020-01-09  Paulo Matos  <pmatos@igalia.com>
    214

  • trunk/Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm

    r254085 r254296  
    7979
    8080    [globalWebViewConfiguration.websiteDataStore _setResourceLoadStatisticsEnabled:YES];
    81     [globalWebViewConfiguration.websiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:NO];
    8281
    8382    [globalWebsiteDataStoreDelegateClient release];
Note: See TracChangeset for help on using the changeset viewer.