Changeset 250804 in webkit


Ignore:
Timestamp:
Oct 7, 2019 4:17:28 PM (4 years ago)
Author:
commit-queue@webkit.org
Message:

Domain relationships in the ITP Database should be inserted in a single query and ignore repeat insert attempts. (202604)
https://bugs.webkit.org/show_bug.cgi?id=202604
<rdar://problem/55995831>

Patch by Kate Cheney <Kate Cheney> on 2019-10-07
Source/WebKit:

Reviewed by Chris Dumez.

This patch addresses two clean-ups for the ITP SQLite Database Store.
First, by using INSERT OR IGNORE as opposed to INSERT, it eliminates
the need to check if a relationship already exists in the database
before inserting it. Second, instead of looping through domain lists
and inserting each relationship as a separate query, this patch now
converts lists to a string which SQLite can use to insert multiple
rows into a database using a single query.

Some Exists queries could not be deleted because they were being
used for testing.

  • NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:

(WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
(WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
Removed old insert queries and replaced them with queries able to
insert string-lists of domains in a single query. Also reorganized
the queries by functionality to make them easier to find and edit.

(WebKit::ResourceLoadStatisticsDatabaseStore::createUniqueIndices):
(WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
In order to properly take advantage of the INSERT OR IGNORE
functionality, unique indices must be established so the SQLite table
is aware of what patterns to look for when ignoring a new insert.

(WebKit::ResourceLoadStatisticsDatabaseStore::ensureAndmakeDomainList):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList):
(WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
Looping through the domain list is no longer needed with the new query
changes. Additionally, ensuring a domain is in the Observed Domains
table of the database must be done before utilizing any inserting of
relationships because the relationship queries rely on fetching the
topFrame domainID from the ObservedDomains table.

(WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccess):
(WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
(WebKit::ResourceLoadStatisticsDatabaseStore::logCrossSiteLoadWithLinkDecoration):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameDomain):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo):
(WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom):
(WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo):
(WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom):
With the ensure check being done once in the ensureAndmakeDomainList
function, these functions no longer have to make that check before
inserting.

  • NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:

(WebKit::WebResourceLoadStatisticsStore::mergeStatisticForTesting):
(WebKit::WebResourceLoadStatisticsStore::isRelationshipOnlyInDatabaseOnce):

  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:

Changes in WeResourceLoadStatisticsStore were for testing only. This
patch updated the merge statistic testing to also test this change by
having a topFrameDomain list with more than one domain.

  • NetworkProcess/NetworkProcess.cpp:

(WebKit::NetworkProcess::mergeStatisticForTesting):
(WebKit::NetworkProcess::isRelationshipOnlyInDatabaseOnce):

  • NetworkProcess/NetworkProcess.h:
  • NetworkProcess/NetworkProcess.messages.in:
  • UIProcess/API/C/WKWebsiteDataStoreRef.cpp:

(WKWebsiteDataStoreSetStatisticsMergeStatistic):
(WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce):

  • UIProcess/API/C/WKWebsiteDataStoreRef.h:
  • UIProcess/Network/NetworkProcessProxy.cpp:

(WebKit::NetworkProcessProxy::mergeStatisticForTesting):
(WebKit::NetworkProcessProxy::isRelationshipOnlyInDatabaseOnce):

  • UIProcess/Network/NetworkProcessProxy.h:
  • UIProcess/WebsiteData/WebsiteDataStore.cpp:

(WebKit::WebsiteDataStore::mergeStatisticForTesting):
(WebKit::WebsiteDataStore::isRelationshipOnlyInDatabaseOnce):

  • UIProcess/WebsiteData/WebsiteDataStore.h:

Added a new function for testing that there are no repeat inserts
into the database. Updated mergeStatistics to test the list-insert
functionality.

Tools:

Reviewed by Chris Dumez.

Updates to testing infrastructure to test successful list-merging of
top domains into the ITP SQLite database and test against repeat
inserts.

  • WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
  • WebKitTestRunner/InjectedBundle/TestRunner.cpp:

(WTR::TestRunner::setStatisticsMergeStatistic):
(WTR::TestRunner::isStatisticsHasHadUserInteraction):
(WTR::TestRunner::isStatisticsOnlyInDatabaseOnce):

  • WebKitTestRunner/InjectedBundle/TestRunner.h:
  • WebKitTestRunner/TestController.cpp:

(WTR::TestController::setStatisticsMergeStatistic):
(WTR::TestController::isStatisticsOnlyInDatabaseOnce):

  • WebKitTestRunner/TestController.h:
  • WebKitTestRunner/TestInvocation.cpp:

(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

Reviewed by Chris Dumez.

Edited the merge-statistics tests to also test the list-insertion
change made in this patch. Also added a new test to ensure repeat
inserts only result in one entry in the database.

  • http/tests/resourceLoadStatistics/many-inserts-only-insert-once-expected.txt: Added.
  • http/tests/resourceLoadStatistics/many-inserts-only-insert-once.html: Added.
  • http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html:
  • http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt:
  • http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html:
  • http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html:
Location:
trunk
Files:
2 added
26 edited

Legend:

Unmodified
Added
Removed
  • trunk/LayoutTests/ChangeLog

    r250800 r250804  
     12019-10-07  Kate Cheney  <katherine_cheney@apple.com>
     2
     3        Domain relationships in the ITP Database should be inserted in a single query and ignore repeat insert attempts. (202604)
     4        https://bugs.webkit.org/show_bug.cgi?id=202604
     5        <rdar://problem/55995831>
     6
     7        Reviewed by Chris Dumez.
     8
     9        Edited the merge-statistics tests to also test the list-insertion
     10        change made in this patch. Also added a new test to ensure repeat
     11        inserts only result in one entry in the database.
     12
     13        * http/tests/resourceLoadStatistics/many-inserts-only-insert-once-expected.txt: Added.
     14        * http/tests/resourceLoadStatistics/many-inserts-only-insert-once.html: Added.
     15        * http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html:
     16        * http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt:
     17        * http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html:
     18        * http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html:
     19
    1202019-10-07  Sihui Liu  <sihui_liu@apple.com>
    221
  • trunk/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-not-overwrite-database.html

    r250621 r250804  
    1717                                     
    1818    function insertSecondStatistic() {
    19       testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 99, false, false, false, 0, function() {
     19      testRunner.setStatisticsMergeStatistic(url, "", "", olderTimestamp, false, 99, false, false, false, 0, function() {
    2020          if (testRunner.isStatisticsPrevalentResource(url))
    2121              testPassed("Host did not overwrite old prevalent resource value.");
     
    4545   
    4646    function runTestRunnerTest() {
    47         testRunner.setStatisticsMergeStatistic(url, "", newerTimestamp, true, 100, true, true, true, 1, function() {
     47        testRunner.setStatisticsMergeStatistic(url, "", "", newerTimestamp, true, 100, true, true, true, 1, function() {
    4848            if (!testRunner.isStatisticsPrevalentResource(url))
    4949                testFailed("Host got set as prevalent resource.");
  • trunk/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database-expected.txt

    r250621 r250804  
    88PASS Host overwrote old user interaction value.
    99PASS Host overwrote old grandfathered value.
    10 PASS Host set as subframe under top frame.
     10PASS Host set as subframe under top frame1.
     11PASS Host set as subframe under top frame2.
    1112PASS successfullyParsed is true
    1213
  • trunk/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-overwrite-database.html

    r250621 r250804  
    1212
    1313    const url = "http://127.0.0.1:8000";
    14     const sampleTopFrameURL = "http://topFrameDomain:8000";
     14    const sampleTopFrameURL1 = "http://topFrameDomain1:8000";
     15    const sampleTopFrameURL2 = "http://topFrameDomain2:8000";
    1516    const olderTimestamp = Math.round((new Date()).getTime() / 1000);
    1617    const newerTimestamp = olderTimestamp + 10;
     
    1920    function insertSecondStatistic() {
    2021   
    21         // set this resource as prevalent so it is in the statistics store
    22         testRunner.setStatisticsPrevalentResource(sampleTopFrameURL, true, function() {
    23        
    24               testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, newerTimestamp, true, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
    25                   if (testRunner.isStatisticsPrevalentResource(url))
    26                       testPassed("Host overwrote old prevalent resource value.");
    27                   else
    28                       testFailed("Host not set as prevalent resource.");
     22        // set url1 as prevalent so it is in the statistics store
     23        // set url2 having UI so it is in the statistics store
     24        testRunner.setStatisticsPrevalentResource(sampleTopFrameURL1, true, function() {
     25            testRunner.setStatisticsHasHadUserInteraction(sampleTopFrameURL2, true, function () {
     26                  testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL1, sampleTopFrameURL2, newerTimestamp, true, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
     27                      if (testRunner.isStatisticsPrevalentResource(url))
     28                          testPassed("Host overwrote old prevalent resource value.");
     29                      else
     30                          testFailed("Host not set as prevalent resource.");
    2931
    30                   if (testRunner.isStatisticsVeryPrevalentResource(url))
    31                       testPassed("Host overwrote old very prevalent resource value.");
    32                   else
    33                       testFailed("Host not set as very prevalent resource.");
    34                                                      
    35                   if (testRunner.isStatisticsHasHadUserInteraction(url))
    36                       testPassed("Host overwrote old user interaction value.");
    37                   else
    38                       testFailed("Host not logged for user interaction.");
    39                    
    40                   if (testRunner.isStatisticsGrandfathered(url))
    41                       testPassed("Host overwrote old grandfathered value.");
    42                   else
    43                       testFailed("Host not set as grandfathered.");
     32                      if (testRunner.isStatisticsVeryPrevalentResource(url))
     33                          testPassed("Host overwrote old very prevalent resource value.");
     34                      else
     35                          testFailed("Host not set as very prevalent resource.");
     36                                                        
     37                      if (testRunner.isStatisticsHasHadUserInteraction(url))
     38                          testPassed("Host overwrote old user interaction value.");
     39                      else
     40                          testFailed("Host not logged for user interaction.");
     41                      
     42                      if (testRunner.isStatisticsGrandfathered(url))
     43                          testPassed("Host overwrote old grandfathered value.");
     44                      else
     45                          testFailed("Host not set as grandfathered.");
    4446
    45                   if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL))
    46                       testPassed("Host set as subframe under top frame.");
    47                   else
    48                       testFailed("Host not set as subframe under top frame.");
     47                      if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL1))
     48                          testPassed("Host set as subframe under top frame1.");
     49                      else
     50                          testFailed("Host not set as subframe under top frame1.");
     51                      if (testRunner.isStatisticsRegisteredAsSubFrameUnder(url, sampleTopFrameURL2))
     52                          testPassed("Host set as subframe under top frame2.");
     53                      else
     54                          testFailed("Host not set as subframe under top frame2.");
    4955
    50                   testRunner.statisticsResetToConsistentState(function() {
    51                       finishJSTest();
     56                      testRunner.statisticsResetToConsistentState(function() {
     57                          finishJSTest();
     58                      });
    5259                  });
    53               });
    54          });
     60             });
     61        });
    5562    }
    5663   
    5764    function runTestRunnerTest() {
    58         testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 0, false, false, false, 0, function() {
     65        testRunner.setStatisticsMergeStatistic(url, "", "", olderTimestamp, false, 0, false, false, false, 0, function() {
    5966            if (testRunner.isStatisticsPrevalentResource(url))
    6067                testFailed("Host got set as prevalent resource.");
  • trunk/LayoutTests/http/tests/resourceLoadStatistics/merge-statistic-does-partially-overwrite-database.html

    r250621 r250804  
    2222        testRunner.setStatisticsPrevalentResource(sampleTopFrameURL, true, function() {
    2323       
    24               testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, newerTimestamp, false, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
     24              testRunner.setStatisticsMergeStatistic(url, sampleTopFrameURL, "", newerTimestamp, false, mostRecentUIToTriggerFirstPartyInteractionCount, true, true, true, 1, function() {
    2525                  if (testRunner.isStatisticsPrevalentResource(url))
    2626                      testPassed("Host overwrote old prevalent resource value.");
     
    5656   
    5757    function runTestRunnerTest() {
    58         testRunner.setStatisticsMergeStatistic(url, "", olderTimestamp, false, 0, false, false, false, 0, function() {
     58        testRunner.setStatisticsMergeStatistic(url, "", "", olderTimestamp, false, 0, false, false, false, 0, function() {
    5959            if (testRunner.isStatisticsPrevalentResource(url))
    6060                testFailed("Host got set as prevalent resource.");
  • trunk/Source/WebKit/ChangeLog

    r250803 r250804  
     12019-10-07  Kate Cheney  <katherine_cheney@apple.com>
     2
     3        Domain relationships in the ITP Database should be inserted in a single query and ignore repeat insert attempts. (202604)
     4        https://bugs.webkit.org/show_bug.cgi?id=202604
     5        <rdar://problem/55995831>
     6
     7        Reviewed by Chris Dumez.
     8       
     9        This patch addresses two clean-ups for the ITP SQLite Database Store.
     10        First, by using INSERT OR IGNORE as opposed to INSERT, it eliminates
     11        the need to check if a relationship already exists in the database
     12        before inserting it. Second, instead of looping through domain lists
     13        and inserting each relationship as a separate query, this patch now
     14        converts lists to a string which SQLite can use to insert multiple
     15        rows into a database using a single query.
     16
     17        Some Exists queries could not be deleted because they were being
     18        used for testing.
     19
     20        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
     21        (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore):
     22        (WebKit::ResourceLoadStatisticsDatabaseStore::prepareStatements):
     23        Removed old insert queries and replaced them with queries able to
     24        insert string-lists of domains in a single query. Also reorganized
     25        the queries by functionality to make them easier to find and edit.
     26
     27        (WebKit::ResourceLoadStatisticsDatabaseStore::createUniqueIndices):
     28        (WebKit::ResourceLoadStatisticsDatabaseStore::createSchema):
     29        In order to properly take advantage of the INSERT OR IGNORE
     30        functionality, unique indices must be established so the SQLite table
     31        is aware of what patterns to look for when ignoring a new insert.
     32
     33        (WebKit::ResourceLoadStatisticsDatabaseStore::ensureAndmakeDomainList):
     34        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList):
     35        (WebKit::ResourceLoadStatisticsDatabaseStore::insertDomainRelationships):
     36        Looping through the domain list is no longer needed with the new query
     37        changes. Additionally, ensuring a domain is in the Observed Domains
     38        table of the database must be done before utilizing any inserting of
     39        relationships because the relationship queries rely on fetching the
     40        topFrame domainID from the ObservedDomains table.
     41
     42        (WebKit::ResourceLoadStatisticsDatabaseStore::grantStorageAccess):
     43        (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
     44        (WebKit::ResourceLoadStatisticsDatabaseStore::logCrossSiteLoadWithLinkDecoration):
     45        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubframeUnderTopFrameDomain):
     46        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUnderTopFrameDomain):
     47        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectTo):
     48        (WebKit::ResourceLoadStatisticsDatabaseStore::setSubresourceUniqueRedirectFrom):
     49        (WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectTo):
     50        (WebKit::ResourceLoadStatisticsDatabaseStore::setTopFrameUniqueRedirectFrom):
     51        With the ensure check being done once in the ensureAndmakeDomainList
     52        function, these functions no longer have to make that check before
     53        inserting.
     54        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
     55
     56        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
     57        (WebKit::WebResourceLoadStatisticsStore::mergeStatisticForTesting):
     58        (WebKit::WebResourceLoadStatisticsStore::isRelationshipOnlyInDatabaseOnce):
     59        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
     60        Changes in WeResourceLoadStatisticsStore were for testing only. This
     61        patch updated the merge statistic testing to also test this change by
     62        having a topFrameDomain list with more than one domain.
     63
     64        * NetworkProcess/NetworkProcess.cpp:
     65        (WebKit::NetworkProcess::mergeStatisticForTesting):
     66        (WebKit::NetworkProcess::isRelationshipOnlyInDatabaseOnce):
     67        * NetworkProcess/NetworkProcess.h:
     68        * NetworkProcess/NetworkProcess.messages.in:
     69        * UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
     70        (WKWebsiteDataStoreSetStatisticsMergeStatistic):
     71        (WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce):
     72        * UIProcess/API/C/WKWebsiteDataStoreRef.h:
     73        * UIProcess/Network/NetworkProcessProxy.cpp:
     74        (WebKit::NetworkProcessProxy::mergeStatisticForTesting):
     75        (WebKit::NetworkProcessProxy::isRelationshipOnlyInDatabaseOnce):
     76        * UIProcess/Network/NetworkProcessProxy.h:
     77        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
     78        (WebKit::WebsiteDataStore::mergeStatisticForTesting):
     79        (WebKit::WebsiteDataStore::isRelationshipOnlyInDatabaseOnce):
     80        * UIProcess/WebsiteData/WebsiteDataStore.h:
     81        Added a new function for testing that there are no repeat inserts
     82        into the database. Updated mergeStatistics to test the list-insert
     83        functionality.
     84
    1852019-10-07  Yusuke Suzuki  <ysuzuki@apple.com>
    286
  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp

    r250621 r250804  
    6363#endif
    6464
     65// COUNT Queries
    6566constexpr auto observedDomainCountQuery = "SELECT COUNT(*) FROM ObservedDomains"_s;
     67constexpr auto countSubframeUnderTopFrameQuery = "SELECT COUNT(*) FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ? AND topFrameDomainID = ?;"_s;
     68constexpr auto countSubresourceUnderTopFrameQuery = "SELECT COUNT(*) FROM SubresourceUnderTopFrameDomains WHERE subresourceDomainID = ? AND topFrameDomainID = ?;"_s;
     69constexpr auto countSubresourceUniqueRedirectsToQuery = "SELECT COUNT(*) FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID = ? AND toDomainID = ?;"_s;
     70
     71// INSERT Queries
    6672constexpr auto insertObservedDomainQuery = "INSERT INTO ObservedDomains (registrableDomain, lastSeen, hadUserInteraction,"
    6773    "mostRecentUserInteractionTime, grandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, timesAccessedAsFirstPartyDueToUserInteraction,"
    6874    "timesAccessedAsFirstPartyDueToStorageAccessAPI) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"_s;
    6975constexpr auto insertTopLevelDomainQuery = "INSERT INTO TopLevelDomains VALUES (?)"_s;
    70 constexpr auto domainIDFromStringQuery = "SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
    71 constexpr auto storageAccessUnderTopFrameDomainsQuery = "INSERT INTO StorageAccessUnderTopFrameDomains (domainID, topLevelDomainID) "
    72     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
    73 constexpr auto storageAccessUnderTopFrameDomainsExistsQuery = "SELECT EXISTS (SELECT 1 FROM StorageAccessUnderTopFrameDomains WHERE domainID = ? "
    74 "AND topLevelDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
    75 constexpr auto topFrameUniqueRedirectsToQuery = "INSERT INTO TopFrameUniqueRedirectsTo (sourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
    76 constexpr auto topFrameUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameUniqueRedirectsTo WHERE sourceDomainID = ? "
    77     "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
    78 constexpr auto topFrameUniqueRedirectsFromQuery = "INSERT INTO TopFrameUniqueRedirectsFrom (targetDomainID, fromDomainID) "
    79     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
    80 constexpr auto topFrameUniqueRedirectsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameUniqueRedirectsFrom WHERE targetDomainID = ? "
    81     "AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
    82 constexpr auto topFrameLinkDecorationsFromQuery = "INSERT INTO TopFrameLinkDecorationsFrom (fromDomainID, toDomainID) "
    83     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
    84 constexpr auto topFrameLinkDecorationsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameLinkDecorationsFrom WHERE fromDomainID = ? "
    85     "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
    86 constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT INTO SubframeUnderTopFrameDomains (subFrameDomainID, topFrameDomainID) "
    87     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
     76constexpr auto storageAccessUnderTopFrameDomainsQuery = "INSERT OR IGNORE INTO StorageAccessUnderTopFrameDomains (domainID, topLevelDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
     77constexpr auto topFrameUniqueRedirectsToQuery = "INSERT OR IGNORE into TopFrameUniqueRedirectsTo (sourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
     78constexpr auto subframeUnderTopFrameDomainsQuery = "INSERT OR IGNORE into SubframeUnderTopFrameDomains (subFrameDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains where registrableDomain in ( "_s;
     79constexpr auto topFrameUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO TopFrameUniqueRedirectsFrom (targetDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
     80constexpr auto topFrameLinkDecorationsFromQuery = "INSERT OR IGNORE INTO TopFrameLinkDecorationsFrom (fromDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
     81constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT OR IGNORE INTO SubresourceUnderTopFrameDomains (subresourceDomainID, topFrameDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
     82constexpr auto subresourceUniqueRedirectsToQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsTo (subresourceDomainID, toDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
     83constexpr auto subresourceUniqueRedirectsFromQuery = "INSERT OR IGNORE INTO SubresourceUniqueRedirectsFrom (subresourceDomainID, fromDomainID) SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain in ( "_s;
     84
     85// EXISTS Queries
    8886constexpr auto subframeUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubframeUnderTopFrameDomains WHERE subFrameDomainID = ? "
    8987    "AND topFrameDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
    90 constexpr auto subresourceUnderTopFrameDomainsQuery = "INSERT INTO SubresourceUnderTopFrameDomains (subresourceDomainID, topFrameDomainID) "
    91     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
    9288constexpr auto subresourceUnderTopFrameDomainExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUnderTopFrameDomains "
    9389    "WHERE subresourceDomainID = ? AND topFrameDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
    94 constexpr auto subresourceUniqueRedirectsToQuery = "INSERT INTO SubresourceUniqueRedirectsTo (subresourceDomainID, toDomainID) "
    95     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
    9690constexpr auto subresourceUniqueRedirectsToExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUniqueRedirectsTo WHERE subresourceDomainID = ? "
    9791    "AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
    98 constexpr auto subresourceUniqueRedirectsFromQuery = "INSERT INTO SubresourceUniqueRedirectsFrom (subresourceDomainID, fromDomainID) "
    99     "SELECT ?, domainID FROM ObservedDomains WHERE registrableDomain == ?"_s;
    100 constexpr auto subresourceUniqueRedirectsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM SubresourceUniqueRedirectsFrom WHERE subresourceDomainID = ? "
    101     "AND fromDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
     92constexpr auto topFrameLinkDecorationsFromExistsQuery = "SELECT EXISTS (SELECT 1 FROM TopFrameLinkDecorationsFrom WHERE fromDomainID = ? "
     93"AND toDomainID = (SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?))"_s;
     94
     95// UPDATE Queries
    10296constexpr auto mostRecentUserInteractionQuery = "UPDATE ObservedDomains SET hadUserInteraction = ?, mostRecentUserInteractionTime = ? "
    10397    "WHERE registrableDomain = ?"_s;
     
    10599constexpr auto updateDataRecordsRemovedQuery = "UPDATE ObservedDomains SET dataRecordsRemoved = ? WHERE registrableDomain = ?"_s;
    106100constexpr auto updatePrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = ? WHERE registrableDomain = ?"_s;
     101constexpr auto updateVeryPrevalentResourceQuery = "UPDATE ObservedDomains SET isVeryPrevalent = ? WHERE registrableDomain = ?"_s;
     102constexpr auto clearPrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = 0, isVeryPrevalent = 0 WHERE registrableDomain = ?"_s;
     103constexpr auto updateGrandfatheredQuery = "UPDATE ObservedDomains SET grandfathered = ? WHERE registrableDomain = ?"_s;
     104
     105
     106// SELECT Queries
     107constexpr auto domainIDFromStringQuery = "SELECT domainID FROM ObservedDomains WHERE registrableDomain = ?"_s;
    107108constexpr auto isPrevalentResourceQuery = "SELECT isPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
    108 constexpr auto updateVeryPrevalentResourceQuery = "UPDATE ObservedDomains SET isVeryPrevalent = ? WHERE registrableDomain = ?"_s;
    109109constexpr auto isVeryPrevalentResourceQuery = "SELECT isVeryPrevalent FROM ObservedDomains WHERE registrableDomain = ?"_s;
    110 constexpr auto clearPrevalentResourceQuery = "UPDATE ObservedDomains SET isPrevalent = 0, isVeryPrevalent = 0 WHERE registrableDomain = ?"_s;
    111110constexpr auto hadUserInteractionQuery = "SELECT hadUserInteraction, mostRecentUserInteractionTime FROM ObservedDomains WHERE registrableDomain = ?"_s;
    112 constexpr auto updateGrandfatheredQuery = "UPDATE ObservedDomains SET grandfathered = ? WHERE registrableDomain = ?"_s;
    113111constexpr auto isGrandfatheredQuery = "SELECT grandfathered FROM ObservedDomains WHERE registrableDomain = ?"_s;
    114112constexpr auto findExpiredUserInteractionQuery = "SELECT domainID FROM ObservedDomains WHERE hadUserInteraction = 1 AND mostRecentUserInteractionTime < ?"_s;
    115113
     114// CREATE TABLE Queries
    116115constexpr auto createObservedDomain = "CREATE TABLE ObservedDomains ("
    117116    "domainID INTEGER PRIMARY KEY, registrableDomain TEXT NOT NULL UNIQUE ON CONFLICT FAIL, lastSeen REAL NOT NULL, "
     
    177176    "FOREIGN KEY(subresourceDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE, "
    178177    "FOREIGN KEY(fromDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s;
     178
     179// CREATE UNIQUE INDEX Queries
     180constexpr auto createUniqueIndexStorageAccessUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS StorageAccessUnderTopFrameDomains_domainID_topLevelDomainID on StorageAccessUnderTopFrameDomains ( domainID, topLevelDomainID );"_s;
     181constexpr auto createUniqueIndexTopFrameUniqueRedirectsTo = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameUniqueRedirectsTo_sourceDomainID_toDomainID on TopFrameUniqueRedirectsTo ( sourceDomainID, toDomainID );"_s;
     182constexpr auto createUniqueIndexTopFrameUniqueRedirectsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameUniqueRedirectsFrom_targetDomainID_fromDomainID on TopFrameUniqueRedirectsFrom ( targetDomainID, fromDomainID );"_s;
     183constexpr auto createUniqueIndexTopFrameLinkDecorationsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS TopFrameLinkDecorationsFrom_fromDomainID_toDomainID on TopFrameLinkDecorationsFrom ( fromDomainID, toDomainID );"_s;
     184constexpr auto createUniqueIndexSubframeUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS SubframeUnderTopFrameDomains_subFrameDomainID_topFrameDomainID on SubframeUnderTopFrameDomains ( subFrameDomainID, topFrameDomainID );"_s;
     185constexpr auto createUniqueIndexSubresourceUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUnderTopFrameDomains_subresourceDomainID_topFrameDomainID on SubresourceUnderTopFrameDomains ( subresourceDomainID, topFrameDomainID );"_s;
     186constexpr auto createUniqueIndexSubresourceUniqueRedirectsTo = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUniqueRedirectsTo_subresourceDomainID_toDomainID on SubresourceUniqueRedirectsTo ( subresourceDomainID, toDomainID );"_s;
     187constexpr auto createUniqueIndexSubresourceUniqueRedirectsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUniqueRedirectsFrom_subresourceDomainID_fromDomainID on SubresourceUnderTopFrameDomains ( subresourceDomainID, fromDomainID );"_s;
     188
    179189   
    180190ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue, ShouldIncludeLocalhost shouldIncludeLocalhost, const String& storageDirectoryPath, PAL::SessionID sessionID)
     
    185195    , m_insertTopLevelDomainStatement(m_database, insertTopLevelDomainQuery)
    186196    , m_domainIDFromStringStatement(m_database, domainIDFromStringQuery)
    187     , m_storageAccessUnderTopFrameDomainsStatement(m_database, storageAccessUnderTopFrameDomainsQuery)
    188     , m_storageAccessUnderTopFrameDomainsExistsStatement(m_database, storageAccessUnderTopFrameDomainsExistsQuery)
    189     , m_topFrameUniqueRedirectsTo(m_database, topFrameUniqueRedirectsToQuery)
    190     , m_topFrameUniqueRedirectsToExists(m_database, topFrameUniqueRedirectsToExistsQuery)
    191     , m_topFrameUniqueRedirectsFrom(m_database, topFrameUniqueRedirectsFromQuery)
    192     , m_topFrameUniqueRedirectsFromExists(m_database, topFrameUniqueRedirectsFromExistsQuery)
    193     , m_topFrameLinkDecorationsFrom(m_database, topFrameLinkDecorationsFromQuery)
    194197    , m_topFrameLinkDecorationsFromExists(m_database, topFrameLinkDecorationsFromExistsQuery)
    195     , m_subframeUnderTopFrameDomains(m_database, subframeUnderTopFrameDomainsQuery)
    196198    , m_subframeUnderTopFrameDomainExists(m_database, subframeUnderTopFrameDomainExistsQuery)
    197     , m_subresourceUnderTopFrameDomains(m_database, subresourceUnderTopFrameDomainsQuery)
    198199    , m_subresourceUnderTopFrameDomainExists(m_database, subresourceUnderTopFrameDomainExistsQuery)
    199     , m_subresourceUniqueRedirectsTo(m_database, subresourceUniqueRedirectsToQuery)
    200200    , m_subresourceUniqueRedirectsToExists(m_database, subresourceUniqueRedirectsToExistsQuery)
    201     , m_subresourceUniqueRedirectsFrom(m_database, subresourceUniqueRedirectsFromQuery)
    202     , m_subresourceUniqueRedirectsFromExists(m_database, subresourceUniqueRedirectsFromExistsQuery)
    203201    , m_mostRecentUserInteractionStatement(m_database, mostRecentUserInteractionQuery)
    204202    , m_updateLastSeenStatement(m_database, updateLastSeenQuery)
     
    263261}
    264262
     263bool ResourceLoadStatisticsDatabaseStore::createUniqueIndices()
     264{
     265    if (!m_database.executeCommand(createUniqueIndexStorageAccessUnderTopFrameDomains)
     266        || !m_database.executeCommand(createUniqueIndexTopFrameUniqueRedirectsTo)
     267        || !m_database.executeCommand(createUniqueIndexTopFrameUniqueRedirectsFrom)
     268        || !m_database.executeCommand(createUniqueIndexTopFrameLinkDecorationsFrom)
     269        || !m_database.executeCommand(createUniqueIndexSubframeUnderTopFrameDomains)
     270        || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains)
     271        || !m_database.executeCommand(createUniqueIndexSubresourceUniqueRedirectsTo)
     272        || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains)) {
     273        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::createUniqueIndices failed to execute, error message: %{public}s", this, m_database.lastErrorMsg());
     274        return false;
     275    }
     276    return true;
     277}
     278
    265279bool ResourceLoadStatisticsDatabaseStore::createSchema()
    266280{
     
    316330        return false;
    317331    }
     332   
     333    if (!createUniqueIndices())
     334        return false;
    318335
    319336    return true;
     
    328345        || m_insertTopLevelDomainStatement.prepare() != SQLITE_OK
    329346        || m_domainIDFromStringStatement.prepare() != SQLITE_OK
    330         || m_storageAccessUnderTopFrameDomainsStatement.prepare() != SQLITE_OK
    331         || m_storageAccessUnderTopFrameDomainsExistsStatement.prepare() != SQLITE_OK
    332         || m_topFrameUniqueRedirectsTo.prepare() != SQLITE_OK
    333         || m_topFrameUniqueRedirectsToExists.prepare() != SQLITE_OK
    334         || m_topFrameUniqueRedirectsFrom.prepare() != SQLITE_OK
    335         || m_topFrameUniqueRedirectsFromExists.prepare() != SQLITE_OK
    336         || m_subframeUnderTopFrameDomains.prepare() != SQLITE_OK
    337347        || m_subframeUnderTopFrameDomainExists.prepare() != SQLITE_OK
    338         || m_subresourceUnderTopFrameDomains.prepare() != SQLITE_OK
    339348        || m_subresourceUnderTopFrameDomainExists.prepare() != SQLITE_OK
    340         || m_subresourceUniqueRedirectsTo.prepare() != SQLITE_OK
    341349        || m_subresourceUniqueRedirectsToExists.prepare() != SQLITE_OK
    342         || m_subresourceUniqueRedirectsFrom.prepare() != SQLITE_OK
    343         || m_subresourceUniqueRedirectsFromExists.prepare() != SQLITE_OK
    344350        || m_updateLastSeenStatement.prepare() != SQLITE_OK
    345351        || m_updateDataRecordsRemovedStatement.prepare() != SQLITE_OK
     
    354360        || m_isGrandfatheredStatement.prepare() != SQLITE_OK
    355361        || m_findExpiredUserInteractionStatement.prepare() != SQLITE_OK
    356         || m_topFrameLinkDecorationsFrom.prepare() != SQLITE_OK
    357362        || m_topFrameLinkDecorationsFromExists.prepare() != SQLITE_OK) {
    358363        RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::prepareStatements failed to prepare, error message: %{public}s", this, m_database.lastErrorMsg());
     
    466471}
    467472
     473String ResourceLoadStatisticsDatabaseStore::ensureAndMakeDomainList(const HashSet<RegistrableDomain>& subframeUnderTopFrameDomains)
     474{
     475    StringBuilder builder;
     476   
     477    for (auto& topFrameResource : subframeUnderTopFrameDomains) {
     478       
     479        // Insert query will fail if top frame domain is not already in the database
     480        ensureResourceStatisticsForRegistrableDomain(topFrameResource);
     481       
     482        if (!builder.isEmpty())
     483            builder.appendLiteral(", ");
     484        builder.append('"');
     485        builder.append(topFrameResource.string());
     486        builder.append('"');
     487    }
     488
     489    return builder.toString();
     490}
     491
     492void ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList(const String& statement, const HashSet<RegistrableDomain>& subframeUnderTopFrameDomains, unsigned domainID)
     493{
     494    SQLiteStatement insertRelationshipStatement(m_database, makeString(statement, ensureAndMakeDomainList(subframeUnderTopFrameDomains), " );"));
     495   
     496    if (insertRelationshipStatement.prepare() != SQLITE_OK
     497        || insertRelationshipStatement.bindInt(1, domainID) != SQLITE_OK
     498        || insertRelationshipStatement.step() != SQLITE_DONE) {
     499        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertDomainRelationshipList failed, error message: %{private}s", this, m_database.lastErrorMsg());
     500        ASSERT_NOT_REACHED();
     501    }
     502}
     503
    468504void ResourceLoadStatisticsDatabaseStore::insertDomainRelationships(const ResourceLoadStatistics& loadStatistics)
    469505{
     
    473509    if (!registrableDomainID)
    474510        return;
    475    
    476     for (auto& topFrameDomain : loadStatistics.storageAccessUnderTopFrameDomains) {
    477         if (!relationshipExists(m_storageAccessUnderTopFrameDomainsExistsStatement, registrableDomainID.value(), topFrameDomain))
    478             insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, registrableDomainID.value(), topFrameDomain);
    479     }
    480 
    481     for (auto& toDomain : loadStatistics.topFrameUniqueRedirectsTo) {
    482         if (!relationshipExists(m_topFrameUniqueRedirectsToExists, registrableDomainID.value(), toDomain))
    483             insertDomainRelationship(m_topFrameUniqueRedirectsTo, registrableDomainID.value(), toDomain);
    484     }
    485    
    486     for (auto& fromDomain : loadStatistics.topFrameUniqueRedirectsFrom) {
    487         if (!relationshipExists(m_topFrameUniqueRedirectsFromExists, registrableDomainID.value(), fromDomain))
    488             insertDomainRelationship(m_topFrameUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
    489     }
    490    
    491     for (auto& topFrameDomain : loadStatistics.subframeUnderTopFrameDomains) {
    492         if (!relationshipExists(m_subframeUnderTopFrameDomainExists, registrableDomainID.value(), topFrameDomain))
    493             insertDomainRelationship(m_subframeUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
    494     }
    495    
    496     for (auto& topFrameDomain : loadStatistics.subresourceUnderTopFrameDomains) {
    497         if (!relationshipExists(m_subresourceUnderTopFrameDomainExists, registrableDomainID.value(), topFrameDomain))
    498             insertDomainRelationship(m_subresourceUnderTopFrameDomains, registrableDomainID.value(), topFrameDomain);
    499     }
    500    
    501     for (auto& toDomain : loadStatistics.subresourceUniqueRedirectsTo) {
    502         if (!relationshipExists(m_subresourceUniqueRedirectsToExists, registrableDomainID.value(), toDomain))
    503             insertDomainRelationship(m_subresourceUniqueRedirectsTo, registrableDomainID.value(), toDomain);
    504     }
    505    
    506     for (auto& fromDomain : loadStatistics.subresourceUniqueRedirectsFrom) {
    507         if (!relationshipExists(m_subresourceUniqueRedirectsFromExists, registrableDomainID.value(), fromDomain))
    508             insertDomainRelationship(m_subresourceUniqueRedirectsFrom, registrableDomainID.value(), fromDomain);
    509     }
     511
     512    insertDomainRelationshipList(storageAccessUnderTopFrameDomainsQuery, loadStatistics.storageAccessUnderTopFrameDomains, registrableDomainID.value());
     513    insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, loadStatistics.topFrameUniqueRedirectsTo, registrableDomainID.value());
     514    insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, loadStatistics.topFrameUniqueRedirectsFrom, registrableDomainID.value());
     515    insertDomainRelationshipList(subframeUnderTopFrameDomainsQuery, loadStatistics.subframeUnderTopFrameDomains, registrableDomainID.value());
     516    insertDomainRelationshipList(subresourceUnderTopFrameDomainsQuery, loadStatistics.subresourceUnderTopFrameDomains, registrableDomainID.value());
     517    insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, loadStatistics.subresourceUniqueRedirectsTo, registrableDomainID.value());
     518    insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, loadStatistics.subresourceUniqueRedirectsFrom, registrableDomainID.value());
    510519}
    511520
     
    918927        ASSERT(subFrameStatus.first == AddedRecord::No);
    919928        ASSERT(hasHadUserInteraction(subFrameDomain, OperatingDatesWindow::Long));
    920         insertDomainRelationship(m_storageAccessUnderTopFrameDomainsStatement, subFrameStatus.second, topFrameDomain);
     929        insertDomainRelationshipList(storageAccessUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), subFrameStatus.second);
    921930    }
    922931
     
    10031012        auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
    10041013        updateLastSeen(targetDomain, ResourceLoadStatistics::reduceTimeResolution(WallTime::now()));
    1005         ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
    1006         if (!relationshipExists(m_subframeUnderTopFrameDomainExists, targetResult.second, topFrameDomain)) {
    1007             insertDomainRelationship(m_subframeUnderTopFrameDomains, targetResult.second, topFrameDomain);
    1008             statisticsWereUpdated = true;
    1009         }
     1014        insertDomainRelationshipList(subframeUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), targetResult.second);
     1015        statisticsWereUpdated = true;
    10101016    }
    10111017
     
    10141020            auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
    10151021            auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
    1016 
    1017             if (!relationshipExists(m_topFrameUniqueRedirectsToExists, redirectingDomainResult.second, targetDomain)) {
    1018                 insertDomainRelationship(m_topFrameUniqueRedirectsTo, redirectingDomainResult.second, targetDomain);
    1019                 statisticsWereUpdated = true;
    1020             }
    1021 
    1022             if (!relationshipExists(m_topFrameUniqueRedirectsFromExists, targetResult.second, sourceDomain)) {
    1023                 insertDomainRelationship(m_topFrameUniqueRedirectsFrom, targetResult.second, sourceDomain);
    1024                 statisticsWereUpdated = true;
    1025             }
     1022            insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
     1023            insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
    10261024        } else {
    10271025            auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
    10281026            auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
    1029             if (!relationshipExists(m_subresourceUniqueRedirectsToExists, redirectingDomainResult.second, targetDomain)) {
    1030                 insertDomainRelationship(m_subresourceUniqueRedirectsTo, redirectingDomainResult.second, targetDomain);
    1031                 statisticsWereUpdated = true;
    1032             }
    1033 
    1034             if (!relationshipExists(m_subresourceUniqueRedirectsFromExists, targetResult.second, sourceDomain)) {
    1035                 insertDomainRelationship(m_subresourceUniqueRedirectsFrom, targetResult.second, sourceDomain);
    1036                 statisticsWereUpdated = true;
    1037             }
     1027            insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
     1028            insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
    10381029        }
     1030        statisticsWereUpdated = true;
    10391031    }
    10401032
     
    10491041
    10501042    auto fromDomainResult = ensureResourceStatisticsForRegistrableDomain(fromDomain);
    1051 
    1052     if (!relationshipExists(m_topFrameLinkDecorationsFromExists, fromDomainResult.second, toDomain)) {
    1053         insertDomainRelationship(m_topFrameLinkDecorationsFrom, fromDomainResult.second, toDomain);
    1054         scheduleStatisticsProcessingRequestIfNecessary();
    1055     }
     1043    insertDomainRelationshipList(topFrameLinkDecorationsFromQuery, HashSet<RegistrableDomain>({ toDomain }), fromDomainResult.second);
     1044    scheduleStatisticsProcessingRequestIfNecessary();
    10561045}
    10571046
     
    13001289
    13011290    // For consistency, make sure we also have a statistics entry for the top frame domain.
    1302     ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
    1303 
    1304     insertDomainRelationship(m_subframeUnderTopFrameDomains, result.second, topFrameDomain);
     1291    insertDomainRelationshipList(subframeUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), result.second);
    13051292}
    13061293
     
    13121299
    13131300    // For consistency, make sure we also have a statistics entry for the top frame domain.
    1314     ensureResourceStatisticsForRegistrableDomain(topFrameDomain);
    1315 
    1316     insertDomainRelationship(m_subresourceUnderTopFrameDomains, result.second, topFrameDomain);
     1301    insertDomainRelationshipList(subresourceUnderTopFrameDomainsQuery, HashSet<RegistrableDomain>({ topFrameDomain }), result.second);
    13171302}
    13181303
     
    13241309
    13251310    // For consistency, make sure we also have a statistics entry for the redirect domain.
    1326     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
    1327 
    1328     insertDomainRelationship(m_subresourceUniqueRedirectsTo, result.second, redirectDomain);
     1311    insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
    13291312}
    13301313
     
    13361319
    13371320    // For consistency, make sure we also have a statistics entry for the redirect domain.
    1338     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
    1339 
    1340     insertDomainRelationship(m_subresourceUniqueRedirectsFrom, result.second, redirectDomain);
     1321    insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
    13411322}
    13421323
     
    13481329
    13491330    // For consistency, make sure we also have a statistics entry for the redirect domain.
    1350     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
    1351 
    1352     insertDomainRelationship(m_topFrameUniqueRedirectsTo, result.second, redirectDomain);
     1331    insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
    13531332}
    13541333
     
    13601339
    13611340    // For consistency, make sure we also have a statistics entry for the redirect domain.
    1362     ensureResourceStatisticsForRegistrableDomain(redirectDomain);
    1363 
    1364     insertDomainRelationship(m_topFrameUniqueRedirectsFrom, result.second, redirectDomain);
     1341    insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ redirectDomain }), result.second);
    13651342}
    13661343
     
    17591736}
    17601737
     1738bool ResourceLoadStatisticsDatabaseStore::isCorrectSubStatisticsCount(const RegistrableDomain& subframeDomain, const TopFrameDomain& topFrameDomain)
     1739{
     1740    SQLiteStatement subFrameUnderTopFrameCount(m_database, countSubframeUnderTopFrameQuery);
     1741    SQLiteStatement subresourceUnderTopFrameCount(m_database, countSubresourceUnderTopFrameQuery);
     1742    SQLiteStatement subresourceUniqueRedirectsTo(m_database, countSubresourceUniqueRedirectsToQuery);
     1743   
     1744    if (subFrameUnderTopFrameCount.prepare() != SQLITE_OK
     1745        || subresourceUnderTopFrameCount.prepare() != SQLITE_OK
     1746        || subresourceUniqueRedirectsTo.prepare() != SQLITE_OK) {
     1747        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::countSubStatisticsTesting failed to prepare, error message: %{private}s", this, m_database.lastErrorMsg());
     1748        ASSERT_NOT_REACHED();
     1749        return false;
     1750    }
     1751   
     1752    if (subFrameUnderTopFrameCount.bindInt(1, domainID(subframeDomain).value()) != SQLITE_OK
     1753        || subFrameUnderTopFrameCount.bindInt(2, domainID(topFrameDomain).value()) != SQLITE_OK
     1754        || subresourceUnderTopFrameCount.bindInt(1, domainID(subframeDomain).value()) != SQLITE_OK
     1755        || subresourceUnderTopFrameCount.bindInt(2, domainID(topFrameDomain).value()) != SQLITE_OK
     1756        || subresourceUniqueRedirectsTo.bindInt(1, domainID(subframeDomain).value()) != SQLITE_OK
     1757        || subresourceUniqueRedirectsTo.bindInt(2, domainID(topFrameDomain).value()) != SQLITE_OK) {
     1758        RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::countSubStatisticsTesting failed to bind, error message: %{private}s", this, m_database.lastErrorMsg());
     1759        ASSERT_NOT_REACHED();
     1760        return false;
     1761    }
     1762   
     1763    if (subFrameUnderTopFrameCount.step() != SQLITE_ROW
     1764        || subresourceUnderTopFrameCount.step() != SQLITE_ROW
     1765        || subresourceUniqueRedirectsTo.step() != SQLITE_ROW)
     1766        return false;
     1767   
     1768    return (subFrameUnderTopFrameCount.getColumnInt(0) == 1 && subresourceUnderTopFrameCount.getColumnInt(0) == 1 && subresourceUniqueRedirectsTo.getColumnInt(0) == 1);
     1769}
     1770
     1771
    17611772} // namespace WebKit
    17621773
  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h

    r250621 r250804  
    104104
    105105    void setLastSeen(const RegistrableDomain&, Seconds) override;
     106    bool isCorrectSubStatisticsCount(const RegistrableDomain&, const TopFrameDomain&);
    106107
    107108private:
     
    111112    bool insertObservedDomain(const ResourceLoadStatistics&);
    112113    void insertDomainRelationships(const ResourceLoadStatistics&);
     114    void insertDomainRelationshipList(const String&, const HashSet<RegistrableDomain>&, unsigned);
    113115    bool insertDomainRelationship(WebCore::SQLiteStatement&, unsigned domainID, const RegistrableDomain& topFrameDomain);
    114116    bool relationshipExists(WebCore::SQLiteStatement&, Optional<unsigned> firstDomainID, const RegistrableDomain& secondDomain) const;
     
    167169    bool isDatabaseStore() const final { return true; }
    168170
     171    bool createUniqueIndices();
    169172    bool createSchema();
    170173    bool prepareStatements();
     174    String ensureAndMakeDomainList(const HashSet<RegistrableDomain>&);
     175
    171176   
    172177    const String m_storageDirectoryPath;
     
    176181    WebCore::SQLiteStatement m_insertTopLevelDomainStatement;
    177182    mutable WebCore::SQLiteStatement m_domainIDFromStringStatement;
    178     WebCore::SQLiteStatement m_storageAccessUnderTopFrameDomainsStatement;
    179     WebCore::SQLiteStatement m_storageAccessUnderTopFrameDomainsExistsStatement;
    180     WebCore::SQLiteStatement m_topFrameUniqueRedirectsTo;
    181     mutable WebCore::SQLiteStatement m_topFrameUniqueRedirectsToExists;
    182     WebCore::SQLiteStatement m_topFrameUniqueRedirectsFrom;
    183     mutable WebCore::SQLiteStatement m_topFrameUniqueRedirectsFromExists;
    184     WebCore::SQLiteStatement m_topFrameLinkDecorationsFrom;
    185183    mutable WebCore::SQLiteStatement m_topFrameLinkDecorationsFromExists;
    186     WebCore::SQLiteStatement m_subframeUnderTopFrameDomains;
    187184    mutable WebCore::SQLiteStatement m_subframeUnderTopFrameDomainExists;
    188     WebCore::SQLiteStatement m_subresourceUnderTopFrameDomains;
    189185    mutable WebCore::SQLiteStatement m_subresourceUnderTopFrameDomainExists;
    190     WebCore::SQLiteStatement m_subresourceUniqueRedirectsTo;
    191186    mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsToExists;
    192     WebCore::SQLiteStatement m_subresourceUniqueRedirectsFrom;
    193     mutable WebCore::SQLiteStatement m_subresourceUniqueRedirectsFromExists;
    194187    WebCore::SQLiteStatement m_mostRecentUserInteractionStatement;
    195188    WebCore::SQLiteStatement m_updateLastSeenStatement;
  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp

    r250621 r250804  
    597597}
    598598
    599 void WebResourceLoadStatisticsStore::mergeStatisticForTesting(const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
    600 {
    601     ASSERT(RunLoop::isMain());
    602 
    603     postTask([this, domain = domain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, completionHandler = WTFMove(completionHandler)]() mutable {
     599void WebResourceLoadStatisticsStore::mergeStatisticForTesting(const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain1, const RegistrableDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
     600{
     601    ASSERT(RunLoop::isMain());
     602
     603    postTask([this, domain = domain.isolatedCopy(), topFrameDomain1 = topFrameDomain1.isolatedCopy(), topFrameDomain2 = topFrameDomain2.isolatedCopy(), lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, completionHandler = WTFMove(completionHandler)]() mutable {
    604604        if (m_statisticsStore) {
    605605            ResourceLoadStatistics statistic(domain);
     
    612612            statistic.dataRecordsRemoved = dataRecordsRemoved;
    613613           
    614             if (!topFrameDomain.isEmpty()) {
    615                 HashSet<RegistrableDomain> topFrameDomains;
    616                 topFrameDomains.add(topFrameDomain);
    617                 statistic.subframeUnderTopFrameDomains = WTFMove(topFrameDomains);
    618             }
     614            HashSet<RegistrableDomain> topFrameDomains;
     615           
     616            if (!topFrameDomain1.isEmpty())
     617                topFrameDomains.add(topFrameDomain1);
     618           
     619            if (!topFrameDomain2.isEmpty())
     620                topFrameDomains.add(topFrameDomain2);
     621
     622            statistic.subframeUnderTopFrameDomains = WTFMove(topFrameDomains);
    619623
    620624            Vector<ResourceLoadStatistics> statistics;
     
    623627        }
    624628        postTaskReply(WTFMove(completionHandler));
     629    });
     630}
     631
     632void WebResourceLoadStatisticsStore::isRelationshipOnlyInDatabaseOnce(const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&& completionHandler)
     633{
     634    ASSERT(RunLoop::isMain());
     635
     636    postTask([this, subDomain = subDomain.isolatedCopy(), topDomain = topDomain.isolatedCopy(), completionHandler = WTFMove(completionHandler)]() mutable {
     637        if (!m_statisticsStore || !is<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore)) {
     638            completionHandler(false);
     639            return;
     640        }
     641       
     642        bool isRelationshipOnlyInDatabaseOnce = downcast<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore).isCorrectSubStatisticsCount(subDomain, topDomain);
     643       
     644        postTaskReply([isRelationshipOnlyInDatabaseOnce, completionHandler = WTFMove(completionHandler)]() mutable {
     645            completionHandler(isRelationshipOnlyInDatabaseOnce);
     646        });
    625647    });
    626648}
  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h

    r250621 r250804  
    130130    void requestStorageAccess(const SubFrameDomain&, const TopFrameDomain&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebPageProxyIdentifier, CompletionHandler<void(StorageAccessWasGranted, StorageAccessPromptWasShown)>&&);
    131131    void setLastSeen(const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
    132     void mergeStatisticForTesting(const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     132    void mergeStatisticForTesting(const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     133    void isRelationshipOnlyInDatabaseOnce(const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&&);
    133134    void setPrevalentResource(const RegistrableDomain&, CompletionHandler<void()>&&);
    134135    void setVeryPrevalentResource(const RegistrableDomain&, CompletionHandler<void()>&&);
  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp

    r250739 r250804  
    969969}
    970970
    971 void NetworkProcess::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
    972 {
    973     if (auto* networkSession = this->networkSession(sessionID)) {
    974         if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
    975             resourceLoadStatistics->mergeStatisticForTesting(domain, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, WTFMove(completionHandler));
     971void NetworkProcess::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& domain, const RegistrableDomain& topFrameDomain1, const RegistrableDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
     972{
     973    if (auto* networkSession = this->networkSession(sessionID)) {
     974        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
     975            resourceLoadStatistics->mergeStatisticForTesting(domain, topFrameDomain1, topFrameDomain2, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, WTFMove(completionHandler));
    976976        else
    977977            completionHandler();
     
    10191019        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
    10201020            resourceLoadStatistics->hasHadUserInteraction(domain, WTFMove(completionHandler));
     1021        else
     1022            completionHandler(false);
     1023    } else {
     1024        ASSERT_NOT_REACHED();
     1025        completionHandler(false);
     1026    }
     1027}
     1028
     1029void NetworkProcess::isRelationshipOnlyInDatabaseOnce(PAL::SessionID sessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&& completionHandler)
     1030{
     1031    if (auto* networkSession = this->networkSession(sessionID)) {
     1032        if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
     1033            resourceLoadStatistics->isRelationshipOnlyInDatabaseOnce(subDomain, topDomain, WTFMove(completionHandler));
    10211034        else
    10221035            completionHandler(false);
  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.h

    r250739 r250804  
    229229    void setPruneEntriesDownTo(PAL::SessionID, uint64_t pruneTargetCount, CompletionHandler<void()>&&);
    230230    void hadUserInteraction(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
     231    void isRelationshipOnlyInDatabaseOnce(PAL::SessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&&);
    231232    void hasLocalStorage(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
    232233    void getAllStorageAccessEntries(PAL::SessionID, CompletionHandler<void(Vector<String> domains)>&&);
     
    243244    void setGrandfatheringTime(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
    244245    void setLastSeen(PAL::SessionID, const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
    245     void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     246    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
    246247    void setMinimumTimeBetweenDataRecordsRemoval(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
    247248    void setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID, bool value, CompletionHandler<void()>&&);
  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.messages.in

    r250739 r250804  
    9595    SetAgeCapForClientSideCookies(PAL::SessionID sessionID, Optional<Seconds> seconds) -> () Async
    9696    SetLastSeen(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, Seconds seconds) -> () Async
    97     MergeStatisticForTesting(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, WebCore::RegistrableDomain topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, uint64_t dataRecordsRemoved) -> () Async
     97    MergeStatisticForTesting(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, WebCore::RegistrableDomain topFrameDomain1, WebCore::RegistrableDomain topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, uint64_t dataRecordsRemoved) -> () Async
    9898    SetPrevalentResource(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async
    9999    SetPrevalentResourceForDebugMode(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async
    100100    HadUserInteraction(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> (bool hadUserInteraction) Async
     101    IsRelationshipOnlyInDatabaseOnce(PAL::SessionID sessionID, WebCore::RegistrableDomain subDomain, WebCore::RegistrableDomain topDomain) -> (bool hadUserInteraction) Async
    101102    HasLocalStorage(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> (bool hadUserInteraction) Async
    102103    GetAllStorageAccessEntries(PAL::SessionID sessionID) -> (Vector<String> domains) Async
  • trunk/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp

    r250621 r250804  
    120120}
    121121
    122 void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler)
    123 {
    124 #if ENABLE(RESOURCE_LOAD_STATISTICS)
    125     WebKit::toImpl(dataStoreRef)->mergeStatisticForTesting(URL(URL(), WebKit::toImpl(host)->string()), URL(URL(), WebKit::toImpl(topFrameDomain)->string()), Seconds { lastSeen }, hadUserInteraction, Seconds { mostRecentUserInteraction }, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [context, completionHandler] {
     122void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler)
     123{
     124#if ENABLE(RESOURCE_LOAD_STATISTICS)
     125    WebKit::toImpl(dataStoreRef)->mergeStatisticForTesting(URL(URL(), WebKit::toImpl(host)->string()), URL(URL(), WebKit::toImpl(topFrameDomain1)->string()), URL(URL(), WebKit::toImpl(topFrameDomain2)->string()), Seconds { lastSeen }, hadUserInteraction, Seconds { mostRecentUserInteraction }, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [context, completionHandler] {
    126126        completionHandler(context);
    127127    });
     
    256256    WebKit::toImpl(dataStoreRef)->hasHadUserInteraction(URL(URL(), WebKit::toImpl(host)->string()), [context, callback](bool hasHadUserInteraction) {
    257257        callback(hasHadUserInteraction, context);
     258    });
     259#else
     260    callback(false, context);
     261#endif
     262}
     263
     264void WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce(WKWebsiteDataStoreRef dataStoreRef, WKStringRef subHost, WKStringRef topHost, void* context, WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnceFunction callback)
     265{
     266#if ENABLE(RESOURCE_LOAD_STATISTICS)
     267    WebKit::toImpl(dataStoreRef)->isRelationshipOnlyInDatabaseOnce(URL(URL(), WebKit::toImpl(subHost)->string()), URL(URL(), WebKit::toImpl(topHost)->string()), [context, callback](bool onlyInDatabaseOnce) {
     268        callback(onlyInDatabaseOnce, context);
    258269    });
    259270#else
  • trunk/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.h

    r250621 r250804  
    5151WK_EXPORT void WKWebsiteDataStoreSetStatisticsLastSeen(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, double seconds, void* context, WKWebsiteDataStoreStatisticsLastSeenFunction completionHandler);
    5252typedef void (*WKWebsiteDataStoreStatisticsMergeStatisticFunction)(void* functionContext);
    53 WK_EXPORT void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler);
     53WK_EXPORT void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler);
    5454typedef void (*WKWebsiteDataStoreStatisticsPrevalentResourceFunction)(void* functionContext);
    5555WK_EXPORT void WKWebsiteDataStoreSetStatisticsPrevalentResource(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value, void* context, WKWebsiteDataStoreStatisticsPrevalentResourceFunction completionHandler);
     
    7171typedef void (*WKWebsiteDataStoreIsStatisticsHasHadUserInteractionFunction)(bool hasHadUserInteraction, void* functionContext);
    7272WK_EXPORT void WKWebsiteDataStoreIsStatisticsHasHadUserInteraction(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, void* context, WKWebsiteDataStoreIsStatisticsHasHadUserInteractionFunction callback);
     73typedef void (*WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnceFunction)(bool onlyInDatabaseOnce, void* functionContext);
     74WK_EXPORT void WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce(WKWebsiteDataStoreRef dataStoreRef, WKStringRef subHost, WKStringRef topHost, void* context, WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnceFunction callback);
    7375WK_EXPORT void WKWebsiteDataStoreSetStatisticsGrandfathered(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value);
    7476typedef void (*WKWebsiteDataStoreIsStatisticsGrandfatheredFunction)(bool isGrandfathered, void* functionContext);
  • trunk/Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp

    r250673 r250804  
    537537}
    538538
    539 void NetworkProcessProxy::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, const RegistrableDomain& topFrameDomain, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
    540 {
    541     sendWithAsyncReply(Messages::NetworkProcess::MergeStatisticForTesting(sessionID, resourceDomain, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved), WTFMove(completionHandler));
     539void NetworkProcessProxy::mergeStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, const RegistrableDomain& topFrameDomain1, const RegistrableDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
     540{
     541    sendWithAsyncReply(Messages::NetworkProcess::MergeStatisticForTesting(sessionID, resourceDomain, topFrameDomain1, topFrameDomain2, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved), WTFMove(completionHandler));
    542542}
    543543
     
    600600   
    601601    sendWithAsyncReply(Messages::NetworkProcess::HadUserInteraction(sessionID, resourceDomain), WTFMove(completionHandler));
     602}
     603
     604void NetworkProcessProxy::isRelationshipOnlyInDatabaseOnce(PAL::SessionID sessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&& completionHandler)
     605{
     606    sendWithAsyncReply(Messages::NetworkProcess::IsRelationshipOnlyInDatabaseOnce(sessionID, subDomain, topDomain), WTFMove(completionHandler));
    602607}
    603608
  • trunk/Source/WebKit/UIProcess/Network/NetworkProcessProxy.h

    r250621 r250804  
    104104    void updatePrevalentDomainsToBlockCookiesFor(PAL::SessionID, const Vector<RegistrableDomain>&, CompletionHandler<void()>&&);
    105105    void hasHadUserInteraction(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
     106    void isRelationshipOnlyInDatabaseOnce(PAL::SessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&&);
    106107    void hasLocalStorage(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
    107108    void isGrandfathered(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
     
    114115    void scheduleStatisticsAndDataRecordsProcessing(PAL::SessionID, CompletionHandler<void()>&&);
    115116    void setLastSeen(PAL::SessionID, const RegistrableDomain&, Seconds, CompletionHandler<void()>&&);
    116     void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain&, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     117    void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
    117118    void setAgeCapForClientSideCookies(PAL::SessionID, Optional<Seconds>, CompletionHandler<void()>&&);
    118119    void setCacheMaxAgeCap(PAL::SessionID, Seconds, CompletionHandler<void()>&&);
  • trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp

    r250621 r250804  
    15521552}
    15531553
    1554 void WebsiteDataStore::mergeStatisticForTesting(const URL& url , const URL& topFrameUrl, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
     1554void WebsiteDataStore::mergeStatisticForTesting(const URL& url , const URL& topFrameUrl1, const URL& topFrameUrl2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler)
    15551555{
    15561556    if (url.protocolIsAbout() || url.isEmpty()) {
     
    15631563    for (auto& processPool : processPools()) {
    15641564        if (auto* process = processPool->networkProcess())
    1565             process->mergeStatisticForTesting(m_sessionID, WebCore::RegistrableDomain { url }, WebCore::RegistrableDomain { topFrameUrl }, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [processPool, callbackAggregator = callbackAggregator.copyRef()] { });
     1565            process->mergeStatisticForTesting(m_sessionID, WebCore::RegistrableDomain { url }, WebCore::RegistrableDomain { topFrameUrl1 }, WebCore::RegistrableDomain { topFrameUrl2 }, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [processPool, callbackAggregator = callbackAggregator.copyRef()] { });
    15661566    }
    15671567}
     
    16411641        if (auto* process = processPool->networkProcess()) {
    16421642            process->hasHadUserInteraction(m_sessionID, WebCore::RegistrableDomain { url }, WTFMove(completionHandler));
     1643            ASSERT(processPools().size() == 1);
     1644            break;
     1645        }
     1646    }
     1647}
     1648
     1649void WebsiteDataStore::isRelationshipOnlyInDatabaseOnce(const URL& subUrl, const URL& topUrl, CompletionHandler<void(bool)>&& completionHandler)
     1650{
     1651    ASSERT(RunLoop::isMain());
     1652   
     1653    if (subUrl.protocolIsAbout() || subUrl.isEmpty() || topUrl.protocolIsAbout() || topUrl.isEmpty()) {
     1654        completionHandler(false);
     1655        return;
     1656    }
     1657   
     1658    for (auto& processPool : processPools()) {
     1659        if (auto* process = processPool->networkProcess()) {
     1660            process->isRelationshipOnlyInDatabaseOnce(m_sessionID, WebCore::RegistrableDomain { subUrl }, WebCore::RegistrableDomain { topUrl }, WTFMove(completionHandler));
    16431661            ASSERT(processPools().size() == 1);
    16441662            break;
  • trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h

    r250621 r250804  
    148148    void getAllStorageAccessEntries(WebPageProxyIdentifier, CompletionHandler<void(Vector<String>&& domains)>&&);
    149149    void hasHadUserInteraction(const URL&, CompletionHandler<void(bool)>&&);
     150    void isRelationshipOnlyInDatabaseOnce(const URL& subUrl, const URL& topUrl, CompletionHandler<void(bool)>&&);
    150151    void isPrevalentResource(const URL&, CompletionHandler<void(bool)>&&);
    151152    void isRegisteredAsRedirectingTo(const URL& hostRedirectedFrom, const URL& hostRedirectedTo, CompletionHandler<void(bool)>&&);
     
    164165    void setGrandfatheringTime(Seconds, CompletionHandler<void()>&&);
    165166    void setLastSeen(const URL&, Seconds, CompletionHandler<void()>&&);
    166     void mergeStatisticForTesting(const URL&, const URL& topFrameUrl, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
     167    void mergeStatisticForTesting(const URL&, const URL& topFrameUrl1, const URL& topFrameUrl2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&);
    167168    void setNotifyPagesWhenDataRecordsWereScanned(bool, CompletionHandler<void()>&&);
    168169    void setIsRunningResourceLoadStatisticsTest(bool, CompletionHandler<void()>&&);
  • trunk/Tools/ChangeLog

    r250794 r250804  
     12019-10-07  Kate Cheney  <katherine_cheney@apple.com>
     2
     3        Domain relationships in the ITP Database should be inserted in a single query and ignore repeat insert attempts. (202604)
     4        https://bugs.webkit.org/show_bug.cgi?id=202604
     5        <rdar://problem/55995831>
     6
     7        Reviewed by Chris Dumez.
     8
     9        Updates to testing infrastructure to test successful list-merging of
     10        top domains into the ITP SQLite database and test against repeat
     11        inserts.
     12        * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
     13        * WebKitTestRunner/InjectedBundle/TestRunner.cpp:
     14        (WTR::TestRunner::setStatisticsMergeStatistic):
     15        (WTR::TestRunner::isStatisticsHasHadUserInteraction):
     16        (WTR::TestRunner::isStatisticsOnlyInDatabaseOnce):
     17        * WebKitTestRunner/InjectedBundle/TestRunner.h:
     18        * WebKitTestRunner/TestController.cpp:
     19        (WTR::TestController::setStatisticsMergeStatistic):
     20        (WTR::TestController::isStatisticsOnlyInDatabaseOnce):
     21        * WebKitTestRunner/TestController.h:
     22        * WebKitTestRunner/TestInvocation.cpp:
     23        (WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):
     24
    1252019-10-07  Matt Lewis  <jlewis3@apple.com>
    226
  • trunk/Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl

    r250621 r250804  
    299299    void setStatisticsPrevalentResourceForDebugMode(DOMString hostName, object completionHandler);
    300300    void setStatisticsLastSeen(DOMString hostName, double seconds, object completionHandler);
    301     void setStatisticsMergeStatistic(DOMString hostName, DOMString topFrameDomain, double lastSeen, boolean hadUserInteraction, double mostRecentUserInteraction, boolean isGrandfathered, boolean isPrevalent, boolean isVeryPrevalent, unsigned long dataRecordsRemoved, object completionHandler);
     301    void setStatisticsMergeStatistic(DOMString hostName, DOMString topFrameDomain1, DOMString topFrameDomain2, double lastSeen, boolean hadUserInteraction, double mostRecentUserInteraction, boolean isGrandfathered, boolean isPrevalent, boolean isVeryPrevalent, unsigned long dataRecordsRemoved, object completionHandler);
    302302    void setStatisticsPrevalentResource(DOMString hostName, boolean value, object completionHandler);
    303303    void setStatisticsVeryPrevalentResource(DOMString hostName, boolean value, object completionHandler);
     
    309309    void setStatisticsHasHadUserInteraction(DOMString hostName, boolean value, object completionHandler);
    310310    boolean isStatisticsHasHadUserInteraction(DOMString hostName);
     311    boolean isStatisticsOnlyInDatabaseOnce(DOMString subHost, DOMString topHost);
    311312    void setStatisticsGrandfathered(DOMString hostName, boolean value);
    312313    boolean isStatisticsGrandfathered(DOMString hostName);
  • trunk/Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp

    r250621 r250804  
    14551455}
    14561456
    1457 void TestRunner::setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler)
     1457void TestRunner::setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain1, JSStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler)
    14581458{
    14591459    cacheTestRunnerCallback(SetStatisticsMergeStatisticCallbackID, completionHandler);
     
    14651465    values.append(adoptWK(WKStringCreateWithJSString(hostName)));
    14661466   
    1467     keys.append(adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain")));
    1468     values.append(adoptWK(WKStringCreateWithJSString(topFrameDomain)));
     1467    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain1")));
     1468    values.append(adoptWK(WKStringCreateWithJSString(topFrameDomain1)));
     1469   
     1470    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain2")));
     1471    values.append(adoptWK(WKStringCreateWithJSString(topFrameDomain2)));
    14691472
    14701473    keys.append(adoptWK(WKStringCreateWithUTF8CString("LastSeen")));
     
    17191722    WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("IsStatisticsHasHadUserInteraction"));
    17201723    WKRetainPtr<WKStringRef> messageBody = adoptWK(WKStringCreateWithJSString(hostName));
     1724    WKTypeRef returnData = nullptr;
     1725    WKBundlePagePostSynchronousMessageForTesting(InjectedBundle::singleton().page()->page(), messageName.get(), messageBody.get(), &returnData);
     1726    ASSERT(WKGetTypeID(returnData) == WKBooleanGetTypeID());
     1727    return WKBooleanGetValue(adoptWK(static_cast<WKBooleanRef>(returnData)).get());
     1728}
     1729
     1730bool TestRunner::isStatisticsOnlyInDatabaseOnce(JSStringRef subHost, JSStringRef topHost)
     1731{
     1732   
     1733    Vector<WKRetainPtr<WKStringRef>> keys;
     1734    Vector<WKRetainPtr<WKTypeRef>> values;
     1735
     1736    keys.append(adoptWK(WKStringCreateWithUTF8CString("SubHost")));
     1737    values.append(adoptWK(WKStringCreateWithJSString(subHost)));
     1738   
     1739    keys.append(adoptWK(WKStringCreateWithUTF8CString("TopHost")));
     1740    values.append(adoptWK(WKStringCreateWithJSString(topHost)));
     1741   
     1742    Vector<WKStringRef> rawKeys(keys.size());
     1743    Vector<WKTypeRef> rawValues(values.size());
     1744
     1745    for (size_t i = 0; i < keys.size(); ++i) {
     1746        rawKeys[i] = keys[i].get();
     1747        rawValues[i] = values[i].get();
     1748    }
     1749
     1750    auto messageName = adoptWK(WKStringCreateWithUTF8CString("IsStatisticsOnlyInDatabaseOnce"));
     1751    auto messageBody = adoptWK(WKDictionaryCreate(rawKeys.data(), rawValues.data(), rawKeys.size()));
    17211752    WKTypeRef returnData = nullptr;
    17221753    WKBundlePagePostSynchronousMessageForTesting(InjectedBundle::singleton().page()->page(), messageName.get(), messageBody.get(), &returnData);
  • trunk/Tools/WebKitTestRunner/InjectedBundle/TestRunner.h

    r250621 r250804  
    397397    void setStatisticsLastSeen(JSStringRef hostName, double seconds, JSValueRef completionHandler);
    398398    void statisticsCallDidSetLastSeenCallback();
    399     void setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler);
     399    void setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain1, JSStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler);
    400400    void statisticsCallDidSetMergeStatisticCallback();
    401401    void setStatisticsPrevalentResource(JSStringRef hostName, bool value, JSValueRef completionHandler);
     
    411411    void statisticsCallDidSetHasHadUserInteractionCallback();
    412412    bool isStatisticsHasHadUserInteraction(JSStringRef hostName);
     413    bool isStatisticsOnlyInDatabaseOnce(JSStringRef subHost, JSStringRef topHost);
    413414    void setStatisticsGrandfathered(JSStringRef hostName, bool value);
    414415    bool isStatisticsGrandfathered(JSStringRef hostName);
  • trunk/Tools/WebKitTestRunner/TestController.cpp

    r250729 r250804  
    32683268}
    32693269
    3270 void TestController::setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved)
     3270void TestController::setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved)
    32713271{
    32723272    ResourceStatisticsCallbackContext context(*this);
    3273     WKWebsiteDataStoreSetStatisticsMergeStatistic(TestController::websiteDataStore(), host, topFrameDomain, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, &context, resourceStatisticsVoidResultCallback);
     3273    WKWebsiteDataStoreSetStatisticsMergeStatistic(TestController::websiteDataStore(), host, topFrameDomain1, topFrameDomain2, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, &context, resourceStatisticsVoidResultCallback);
    32743274    runUntil(context.done, noTimeout);
    32753275    m_currentInvocation->didMergeStatistic();
     
    33523352    ResourceStatisticsCallbackContext context(*this);
    33533353    WKWebsiteDataStoreIsStatisticsHasHadUserInteraction(TestController::websiteDataStore(), host, &context, resourceStatisticsBooleanResultCallback);
     3354    runUntil(context.done, noTimeout);
     3355    return context.result;
     3356}
     3357
     3358bool TestController::isStatisticsOnlyInDatabaseOnce(WKStringRef subHost, WKStringRef topHost)
     3359{
     3360    ResourceStatisticsCallbackContext context(*this);
     3361    WKWebsiteDataStoreIsStatisticsOnlyInDatabaseOnce(TestController::websiteDataStore(), subHost, topHost, &context, resourceStatisticsBooleanResultCallback);
    33543362    runUntil(context.done, noTimeout);
    33553363    return context.result;
  • trunk/Tools/WebKitTestRunner/TestController.h

    r250621 r250804  
    211211    void setStatisticsPrevalentResourceForDebugMode(WKStringRef hostName);
    212212    void setStatisticsLastSeen(WKStringRef hostName, double seconds);
    213     void setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved);
     213    void setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved);
    214214    void setStatisticsPrevalentResource(WKStringRef hostName, bool value);
    215215    void setStatisticsVeryPrevalentResource(WKStringRef hostName, bool value);
     
    222222    void setStatisticsHasHadUserInteraction(WKStringRef hostName, bool value);
    223223    bool isStatisticsHasHadUserInteraction(WKStringRef hostName);
     224    bool isStatisticsOnlyInDatabaseOnce(WKStringRef subHost, WKStringRef topHost);
    224225    void setStatisticsGrandfathered(WKStringRef hostName, bool value);
    225226    bool isStatisticsGrandfathered(WKStringRef hostName);
  • trunk/Tools/WebKitTestRunner/TestInvocation.cpp

    r250621 r250804  
    11061106        WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
    11071107        WKRetainPtr<WKStringRef> hostNameKey = adoptWK(WKStringCreateWithUTF8CString("HostName"));
    1108         WKRetainPtr<WKStringRef> topFrameDomainKey = adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain"));
     1108        WKRetainPtr<WKStringRef> topFrameDomain1Key = adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain1"));
     1109        WKRetainPtr<WKStringRef> topFrameDomain2Key = adoptWK(WKStringCreateWithUTF8CString("TopFrameDomain2"));
    11091110        WKRetainPtr<WKStringRef> lastSeenKey = adoptWK(WKStringCreateWithUTF8CString("LastSeen"));
    11101111        WKRetainPtr<WKStringRef> hadUserInteractionKey = adoptWK(WKStringCreateWithUTF8CString("HadUserInteraction"));
     
    11171118
    11181119        WKStringRef hostName = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hostNameKey.get()));
    1119         WKStringRef topFrameDomain = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topFrameDomainKey.get()));
     1120        WKStringRef topFrameDomain1 = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topFrameDomain1Key.get()));
     1121        WKStringRef topFrameDomain2 = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topFrameDomain2Key.get()));
    11201122        WKDoubleRef lastSeen = static_cast<WKDoubleRef>(WKDictionaryGetItemForKey(messageBodyDictionary, lastSeenKey.get()));
    11211123        WKBooleanRef hadUserInteraction = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hadUserInteractionKey.get()));
     
    11261128        WKUInt64Ref dataRecordsRemoved = static_cast<WKUInt64Ref>(WKDictionaryGetItemForKey(messageBodyDictionary, dataRecordsRemovedKey.get()));
    11271129       
    1128         TestController::singleton().setStatisticsMergeStatistic(hostName, topFrameDomain, WKDoubleGetValue(lastSeen), WKBooleanGetValue(hadUserInteraction), WKDoubleGetValue(mostRecentUserInteraction), WKBooleanGetValue(isGrandfathered), WKBooleanGetValue(isPrevalent), WKBooleanGetValue(isVeryPrevalent), WKUInt64GetValue(dataRecordsRemoved));
     1130        TestController::singleton().setStatisticsMergeStatistic(hostName, topFrameDomain1, topFrameDomain2, WKDoubleGetValue(lastSeen), WKBooleanGetValue(hadUserInteraction), WKDoubleGetValue(mostRecentUserInteraction), WKBooleanGetValue(isGrandfathered), WKBooleanGetValue(isPrevalent), WKBooleanGetValue(isVeryPrevalent), WKUInt64GetValue(dataRecordsRemoved));
    11291131
    11301132        return nullptr;
     
    12481250        WKRetainPtr<WKTypeRef> result = adoptWK(WKBooleanCreate(hasHadUserInteraction));
    12491251        return result;
     1252    }
     1253   
     1254    if (WKStringIsEqualToUTF8CString(messageName, "IsStatisticsOnlyInDatabaseOnce")) {
     1255        ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID());
     1256       
     1257        WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody);
     1258        auto subHostKey = adoptWK(WKStringCreateWithUTF8CString("SubHost"));
     1259        auto topHostKey = adoptWK(WKStringCreateWithUTF8CString("TopHost"));
     1260       
     1261        WKStringRef subHost = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, subHostKey.get()));
     1262        WKStringRef topHost = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, topHostKey.get()));
     1263
     1264        bool statisticInDatabaseOnce = TestController::singleton().isStatisticsOnlyInDatabaseOnce(subHost, topHost);
     1265        return adoptWK(WKBooleanCreate(statisticInDatabaseOnce));
    12501266    }
    12511267   
Note: See TracChangeset for help on using the changeset viewer.