⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 263258 in webkit


Ignore:
Timestamp:
Jun 19, 2020, 12:49:23 AM (6 years ago)
Author:
Carlos Garcia Campos
Message:

Add support for fetching registrable domains with resource load statistics
​https://bugs.webkit.org/show_bug.cgi?id=213291

Source/WebKit:

Reviewed by Adrian Perez de Castro and Youenn Fablet.

WebsiteDataStore::fetchData() doesn't return anything for resource load statistics because
NetworkProcess::fetchWebsiteData() doesn't handle WebsiteDataType::ResourceLoadStatistics.

  • NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:

(WebKit::ResourceLoadStatisticsDatabaseStore::allDomains const): Query all registrable domains from database.

  • NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
  • NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:

(WebKit::ResourceLoadStatisticsMemoryStore::allDomains const): Return all registrable domains in memory map.

  • NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
  • NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:

(WebKit::WebResourceLoadStatisticsStore::registrableDomains): Get the list of registrable domains.

  • NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
  • NetworkProcess/NetworkProcess.cpp:

(WebKit::NetworkProcess::fetchWebsiteData): Handle WebsiteDataType::ResourceLoadStatistics.

  • Shared/WebsiteData/WebsiteData.cpp:

(WebKit::WebsiteData::encode const): Encode registrableDomainsWithResourceLoadStatistics.
(WebKit::WebsiteData::decode): Decode registrableDomainsWithResourceLoadStatistics.

  • Shared/WebsiteData/WebsiteData.h:
  • UIProcess/API/C/WKWebsiteDataStoreRef.cpp:

(WKWebsiteDataStoreRemoveITPDataForDomain): Use WebsiteDataRecord::addResourceLoadStatisticsRegistrableDomain()
instead of the display name.

  • UIProcess/WebsiteData/WebsiteDataRecord.cpp:

(WebKit::WebsiteDataRecord::addResourceLoadStatisticsRegistrableDomain): Add the given registrable domain to the list.

  • UIProcess/WebsiteData/WebsiteDataRecord.h:
  • UIProcess/WebsiteData/WebsiteDataStore.cpp:

(WebKit::WebsiteDataStore::fetchDataAndApply): Handle registrable domains with resource load statistics.
(WebKit::WebsiteDataStore::removeData): Use resourceLoadStatisticsRegistrableDomains instead of the display name.

Tools:

Reviewed by Adrian Perez de Castro.

Update GLib ITP unit test to check also fetch and remove.

  • TestWebKitAPI/Tests/WebKitGLib/TestWebsiteData.cpp:

(testWebsiteDataITP):

Location:
trunk
Files:
17 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebKit/ChangeLog

    r263251 r263258  
     12020-06-19  Carlos Garcia Campos  <cgarcia@igalia.com>
     2
     3        Add support for fetching registrable domains with resource load statistics
     4        https://bugs.webkit.org/show_bug.cgi?id=213291
     5
     6        Reviewed by Adrian Perez de Castro and Youenn Fablet.
     7
     8        WebsiteDataStore::fetchData() doesn't return anything for resource load statistics because
     9        NetworkProcess::fetchWebsiteData() doesn't handle WebsiteDataType::ResourceLoadStatistics.
     10
     11        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
     12        (WebKit::ResourceLoadStatisticsDatabaseStore::allDomains const): Query all registrable domains from database.
     13        * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
     14        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
     15        (WebKit::ResourceLoadStatisticsMemoryStore::allDomains const): Return all registrable domains in memory map.
     16        * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
     17        * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
     18        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
     19        (WebKit::WebResourceLoadStatisticsStore::registrableDomains): Get the list of registrable domains.
     20        * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
     21        * NetworkProcess/NetworkProcess.cpp:
     22        (WebKit::NetworkProcess::fetchWebsiteData): Handle WebsiteDataType::ResourceLoadStatistics.
     23        * Shared/WebsiteData/WebsiteData.cpp:
     24        (WebKit::WebsiteData::encode const): Encode registrableDomainsWithResourceLoadStatistics.
     25        (WebKit::WebsiteData::decode): Decode registrableDomainsWithResourceLoadStatistics.
     26        * Shared/WebsiteData/WebsiteData.h:
     27        * UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
     28        (WKWebsiteDataStoreRemoveITPDataForDomain): Use WebsiteDataRecord::addResourceLoadStatisticsRegistrableDomain()
     29        instead of the display name.
     30        * UIProcess/WebsiteData/WebsiteDataRecord.cpp:
     31        (WebKit::WebsiteDataRecord::addResourceLoadStatisticsRegistrableDomain): Add the given registrable domain to the list.
     32        * UIProcess/WebsiteData/WebsiteDataRecord.h:
     33        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
     34        (WebKit::WebsiteDataStore::fetchDataAndApply): Handle registrable domains with resource load statistics.
     35        (WebKit::WebsiteDataStore::removeData): Use resourceLoadStatisticsRegistrableDomains instead of the display name.
     36
    1372020-06-18  David Kilzer  <ddkilzer@apple.com>
    238

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp

    r262752 r263258  
    22622262}
    22632263
     2264Vector<RegistrableDomain> ResourceLoadStatisticsDatabaseStore::allDomains() const
     2265{
     2266    ASSERT(!RunLoop::isMain());
     2267
     2268    auto scopedStatement = this->scopedStatement(m_getAllDomainsStatement, getAllDomainsQuery, "allDomains"_s);
     2269    if (!scopedStatement)
     2270        return { };
     2271
     2272    Vector<RegistrableDomain> domains;
     2273    while (scopedStatement->step() == SQLITE_ROW)
     2274        domains.append(RegistrableDomain::uncheckedCreateFromRegistrableDomainString(scopedStatement->getColumnText(0)));
     2275    return domains;
     2276}
     2277
    22642278void ResourceLoadStatisticsDatabaseStore::clear(CompletionHandler<void()>&& completionHandler)
    22652279{

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h

    r262659 r263258  
    145145    void setIsNewResourceLoadStatisticsDatabaseFile(bool isNewResourceLoadStatisticsDatabaseFile) { m_isNewResourceLoadStatisticsDatabaseFile = isNewResourceLoadStatisticsDatabaseFile; }
    146146    void removeDataForDomain(const RegistrableDomain&) override;
     147    Vector<RegistrableDomain> allDomains() const final;
    147148    bool domainIDExistsInDatabase(int);
    148149    Optional<Vector<String>> checkForMissingTablesInSchema();

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp

    r262265 r263258  
    10711071}
    10721072
     1073Vector<RegistrableDomain> ResourceLoadStatisticsMemoryStore::allDomains() const
     1074{
     1075    ASSERT(!RunLoop::isMain());
     1076
     1077    return copyToVector(m_resourceStatisticsMap.keys());
     1078}
     1079
    10731080void ResourceLoadStatisticsMemoryStore::setPrevalentResource(const RegistrableDomain& domain)
    10741081{

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h

    r262265 r263258  
    114114    void setLastSeen(const RegistrableDomain&, Seconds) override;
    115115    void removeDataForDomain(const RegistrableDomain&) override;
     116    Vector<RegistrableDomain> allDomains() const final;
    116117    void insertExpiredStatisticForTesting(const RegistrableDomain&, bool hasUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent) override;
    117118

  • trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h

    r262024 r263258  
    193193
    194194    virtual void removeDataForDomain(const RegistrableDomain&) = 0;
    195    
     195
     196    virtual Vector<RegistrableDomain> allDomains() const = 0;
     197
    196198    void didCreateNetworkProcess();
    197199

  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp

    r262846 r263258  
    13621362}
    13631363
     1364void WebResourceLoadStatisticsStore::registrableDomains(CompletionHandler<void(Vector<RegistrableDomain>&&)>&& completionHandler)
     1365{
     1366    ASSERT(RunLoop::isMain());
     1367    postTask([this, completionHandler = WTFMove(completionHandler)]() mutable {
     1368        auto domains = m_statisticsStore ? m_statisticsStore->allDomains() : Vector<RegistrableDomain>();
     1369        postTaskReply([domains = crossThreadCopy(WTFMove(domains)), completionHandler = WTFMove(completionHandler)]() mutable {
     1370            completionHandler(WTFMove(domains));
     1371        });
     1372    });
     1373}
     1374
    13641375void WebResourceLoadStatisticsStore::deleteAndRestrictWebsiteDataForRegistrableDomains(OptionSet<WebsiteDataType> dataTypes, RegistrableDomainsToDeleteOrRestrictWebsiteDataFor&& domainsToDeleteAndRestrictWebsiteDataFor, bool shouldNotifyPage, CompletionHandler<void(const HashSet<RegistrableDomain>&)>&& completionHandler)
    13651376{

  • trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h

    r262024 r263258  
    220220    void removeDataForDomain(const RegistrableDomain, CompletionHandler<void()>&&);
    221221    void deleteAndRestrictWebsiteDataForRegistrableDomains(OptionSet<WebsiteDataType>, RegistrableDomainsToDeleteOrRestrictWebsiteDataFor&&, bool shouldNotifyPage, CompletionHandler<void(const HashSet<RegistrableDomain>&)>&&);
     222    void registrableDomains(CompletionHandler<void(Vector<RegistrableDomain>&&)>&&);
    222223    void registrableDomainsWithWebsiteData(OptionSet<WebsiteDataType>, bool shouldNotifyPage, CompletionHandler<void(HashSet<RegistrableDomain>&&)>&&);
    223224    StorageAccessWasGranted grantStorageAccessInStorageSession(const SubFrameDomain&, const TopFrameDomain&, Optional<WebCore::FrameIdentifier>, WebCore::PageIdentifier, StorageAccessScope);

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp

    r263038 r263258  
    15791579    }
    15801580#endif
     1581
     1582#if ENABLE(RESOURCE_LOAD_STATISTICS)
     1583    if (websiteDataTypes.contains(WebsiteDataType::ResourceLoadStatistics)) {
     1584        if (auto* session = networkSession(sessionID)) {
     1585            if (auto* resourceLoadStatistics = session->resourceLoadStatistics()) {
     1586                resourceLoadStatistics->registrableDomains([callbackAggregator = callbackAggregator.copyRef()](auto&& domains) mutable {
     1587                    while (!domains.isEmpty())
     1588                        callbackAggregator->m_websiteData.registrableDomainsWithResourceLoadStatistics.add(domains.takeLast());
     1589                });
     1590            }
     1591        }
     1592    }
     1593#endif
    15811594}
    15821595

  • trunk/Source/WebKit/Shared/WebsiteData/WebsiteData.cpp

    r262994 r263258  
    2929#include "ArgumentCoders.h"
    3030#include "WebsiteDataType.h"
     31#include <WebCore/RegistrableDomain.h>
    3132#include <WebCore/SecurityOriginData.h>
    3233#include <wtf/text/StringHash.h>
    … …  
    6869#endif
    6970    encoder << hostNamesWithHSTSCache;
     71#if ENABLE(RESOURCE_LOAD_STATISTICS)
     72    encoder << registrableDomainsWithResourceLoadStatistics;
     73#endif
    7074}
    7175
    … …  
    8286    if (!decoder.decode(result.hostNamesWithHSTSCache))
    8387        return false;
     88#if ENABLE(RESOURCE_LOAD_STATISTICS)
     89    if (!decoder.decode(result.registrableDomainsWithResourceLoadStatistics))
     90        return false;
     91#endif
    8492    return true;
    8593}

  • trunk/Source/WebKit/Shared/WebsiteData/WebsiteData.h

    r263208 r263258  
    2626#pragma once
    2727
     28#include <WebCore/RegistrableDomain.h>
    2829#include <WebCore/SecurityOriginData.h>
    2930#include <wtf/HashMap.h>
    … …  
    6061#endif
    6162    HashSet<String> hostNamesWithHSTSCache;
     63#if ENABLE(RESOURCE_LOAD_STATISTICS)
     64    HashSet<WebCore::RegistrableDomain> registrableDomainsWithResourceLoadStatistics;
     65#endif
    6266
    6367    void encode(IPC::Encoder&) const;

  • trunk/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp

    r262971 r263258  
    7373    WebKit::WebsiteDataRecord dataRecord;
    7474    dataRecord.types.add(WebKit::WebsiteDataType::ResourceLoadStatistics);
    75     dataRecord.displayName = WebKit::toImpl(host)->string();
     75    dataRecord.addResourceLoadStatisticsRegistrableDomain(WebCore::RegistrableDomain::uncheckedCreateFromHost(WebKit::toImpl(host)->string()));
    7676    Vector<WebKit::WebsiteDataRecord> dataRecords = { WTFMove(dataRecord) };
    7777

  • trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataRecord.cpp

    r257785 r263258  
    2929#include <WebCore/LocalizedStrings.h>
    3030#include <WebCore/PublicSuffix.h>
    31 #include <WebCore/RegistrableDomain.h>
    3231#include <WebCore/SecurityOrigin.h>
    3332
    … …  
    115114}
    116115
     116#if ENABLE(RESOURCE_LOAD_STATISTICS)
     117void WebsiteDataRecord::addResourceLoadStatisticsRegistrableDomain(const WebCore::RegistrableDomain& domain)
     118{
     119    types.add(WebsiteDataType::ResourceLoadStatistics);
     120    resourceLoadStatisticsRegistrableDomains.add(domain);
     121}
     122#endif
     123
    117124static inline bool hostIsInDomain(StringView host, StringView domain)
    118125{

  • trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataRecord.h

    r257785 r263258  
    2727
    2828#include "WebsiteDataType.h"
     29#include <WebCore/RegistrableDomain.h>
    2930#include <WebCore/SecurityOriginData.h>
    3031#include <WebCore/SecurityOriginHash.h>
    … …  
    3738
    3839namespace WebCore {
    39 class RegistrableDomain;
    4040class SecurityOrigin;
    4141}
    … …  
    5656    void addHSTSCacheHostname(const String& hostName);
    5757    void addAlternativeServicesHostname(const String& hostName);
     58#if ENABLE(RESOURCE_LOAD_STATISTICS)
     59    void addResourceLoadStatisticsRegistrableDomain(const WebCore::RegistrableDomain&);
     60#endif
    5861
    5962    String displayName;
    … …  
    7376    HashSet<String> HSTSCacheHostNames;
    7477    HashSet<String> alternativeServicesHostNames;
     78#if ENABLE(RESOURCE_LOAD_STATISTICS)
     79    HashSet<WebCore::RegistrableDomain> resourceLoadStatisticsRegistrableDomains;
     80#endif
    7581
    7682    bool matches(const WebCore::RegistrableDomain&) const;

  • trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp

    r262971 r263258  
    367367            }
    368368
     369#if ENABLE(RESOURCE_LOAD_STATISTICS)
     370            for (const auto& domain : websiteData.registrableDomainsWithResourceLoadStatistics) {
     371                auto displayName = WebsiteDataRecord::displayNameForHostName(domain.string());
     372                if (!displayName)
     373                    continue;
     374
     375                auto& record = m_websiteDataRecords.add(displayName, WebsiteDataRecord { }).iterator->value;
     376                if (!record.displayName)
     377                    record.displayName = WTFMove(displayName);
     378
     379                record.addResourceLoadStatisticsRegistrableDomain(domain);
     380            }
     381#endif
     382
    369383            callIfNeeded();
    370384        }
    … …  
    9971011                for (auto& hostName : dataRecord.HSTSCacheHostNames)
    9981012                    HSTSCacheHostNames.append(hostName);
    999                 registrableDomains.append(WebCore::RegistrableDomain::uncheckedCreateFromHost(dataRecord.displayName));
     1013                for (auto& registrableDomain : dataRecord.resourceLoadStatisticsRegistrableDomains)
     1014                    registrableDomains.append(registrableDomain);
    10001015            }
    10011016

  • trunk/Tools/ChangeLog

    r263257 r263258  
     12020-06-19  Carlos Garcia Campos  <cgarcia@igalia.com>
     2
     3        Add support for fetching registrable domains with resource load statistics
     4        https://bugs.webkit.org/show_bug.cgi?id=213291
     5
     6        Reviewed by Adrian Perez de Castro.
     7
     8        Update GLib ITP unit test to check also fetch and remove.
     9
     10        * TestWebKitAPI/Tests/WebKitGLib/TestWebsiteData.cpp:
     11        (testWebsiteDataITP):
     12
    1132020-06-19  Tomoki Imai  <Tomoki.Imai@sony.com>
    214

  • trunk/Tools/TestWebKitAPI/Tests/WebKitGLib/TestWebsiteData.cpp

    r263204 r263258  
    671671
    672672    g_assert_false(webkit_website_data_manager_get_itp_enabled(test->m_manager));
    673     test->loadURI(kServer->getURIForPath("/").data());
     673    test->loadURI(kServer->getURIForPath("/empty").data());
    674674    test->waitUntilLoadFinished();
    675675
    … …  
    679679    g_assert_false(g_file_test(itpLogFile.get(), G_FILE_TEST_IS_REGULAR));
    680680
    681     test->loadURI(kServer->getURIForPath("/").data());
     681    test->loadURI(kServer->getURIForPath("/empty").data());
    682682    test->waitUntilLoadFinished();
    683683
    684684    test->waitUntilFileChanged(itpLogFile.get(), G_FILE_MONITOR_EVENT_CREATED);
    685 
    686685    g_assert_true(g_file_test(itpDirectory, G_FILE_TEST_IS_DIR));
    687686    g_assert_true(g_file_test(itpLogFile.get(), G_FILE_TEST_IS_REGULAR));
     687
     688    GList* dataList = test->fetch(WEBKIT_WEBSITE_DATA_ITP);
     689    g_assert_nonnull(dataList);
     690    g_assert_cmpuint(g_list_length(dataList), ==, 1);
     691    auto* data = static_cast<WebKitWebsiteData*>(dataList->data);
     692    g_assert_nonnull(data);
     693    WebKitSecurityOrigin* origin = webkit_security_origin_new_for_uri(kServer->getURIForPath("/").data());
     694    g_assert_cmpstr(webkit_website_data_get_name(data), ==, webkit_security_origin_get_host(origin));
     695    webkit_security_origin_unref(origin);
     696
     697    // Remove the registration.
     698    GList removeList = { data, nullptr, nullptr };
     699    test->remove(WEBKIT_WEBSITE_DATA_ITP, &removeList);
     700    dataList = test->fetch(WEBKIT_WEBSITE_DATA_ITP);
     701    g_assert_null(dataList);
    688702
    689703    // Clear all.
Note: See TracChangeset for help on using the changeset viewer.