Changeset 261963 in webkit
- Timestamp:
- May 20, 2020 3:14:13 PM (4 years ago)
- Location:
- trunk
- Files:
-
- 8 added
- 29 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/LayoutTests/ChangeLog
r261953 r261963 1 2020-05-20 Kate Cheney <katherine_cheney@apple.com> 2 3 Support operating dates in ResourceLoadStatisticsDatabaseStore 4 https://bugs.webkit.org/show_bug.cgi?id=212034 5 <rdar://problem/63349242> 6 7 Reviewed by Brent Fulgham. 8 9 Add tests for memory and database stores. 10 11 * http/tests/resourceLoadStatistics/operating-dates-all-but-cookies-removed-database-expected.txt: Added. 12 * http/tests/resourceLoadStatistics/operating-dates-all-but-cookies-removed-database.html: Added. 13 * http/tests/resourceLoadStatistics/operating-dates-all-but-cookies-removed-expected.txt: Added. 14 * http/tests/resourceLoadStatistics/operating-dates-all-but-cookies-removed.html: Added. 15 * http/tests/resourceLoadStatistics/operating-dates-all-website-data-removed-database-expected.txt: Added. 16 * http/tests/resourceLoadStatistics/operating-dates-all-website-data-removed-database.html: Added. 17 * http/tests/resourceLoadStatistics/operating-dates-all-website-data-removed-expected.txt: Added. 18 * http/tests/resourceLoadStatistics/operating-dates-all-website-data-removed.html: Added. 19 1 20 2020-05-20 Jacob Uphoff <jacob_uphoff@apple.com> 2 21 -
trunk/Source/WebKit/ChangeLog
r261960 r261963 1 2020-05-20 Kate Cheney <katherine_cheney@apple.com> 2 3 Support operating dates in ResourceLoadStatisticsDatabaseStore 4 https://bugs.webkit.org/show_bug.cgi?id=212034 5 <rdar://problem/63349242> 6 7 Reviewed by Brent Fulgham. 8 9 Tests: http/tests/resourceLoadStatistics/operating-dates-all-but-cookies-removed-database.html 10 http/tests/resourceLoadStatistics/operating-dates-all-but-cookies-removed.html 11 http/tests/resourceLoadStatistics/operating-dates-all-website-data-removed-database.html 12 http/tests/resourceLoadStatistics/operating-dates-all-website-data-removed.html 13 14 Patch to support Operating Dates in the ResourceLoadStatisticsDatabaseStore. 15 Moves functions out of ResourceLoadStatisticsStore now that each store 16 needs its own implementation. 17 18 Adds a new function for testing which inserts an expired statistic 19 and makes sure the proper behavior occurs depending on additional 20 parameters. 21 22 * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp: 23 (WebKit::createTableQueries): 24 (WebKit::ResourceLoadStatisticsDatabaseStore::ResourceLoadStatisticsDatabaseStore): 25 (WebKit::ResourceLoadStatisticsDatabaseStore::checkForMissingTablesInSchema): 26 (WebKit::ResourceLoadStatisticsDatabaseStore::createUniqueIndices): 27 (WebKit::ResourceLoadStatisticsDatabaseStore::createSchema): 28 (WebKit::ResourceLoadStatisticsDatabaseStore::getLeastRecentOperatingDate): 29 (WebKit::ResourceLoadStatisticsDatabaseStore::mergeOperatingDates): 30 (WebKit::ResourceLoadStatisticsDatabaseStore::includeTodayAsOperatingDateIfNecessary): 31 (WebKit::ResourceLoadStatisticsDatabaseStore::hasStatisticsExpired const): 32 (WebKit::ResourceLoadStatisticsDatabaseStore::insertExpiredStatisticForTesting): 33 Migrated Operating Dates functions from ResourceLoadStatisticsStore and implemented 34 them using SQLite instead of a vector. Functionality is 35 identical to the original in-memory implementation. 36 37 * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h: 38 * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp: 39 (WebKit::ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore): 40 (WebKit::ResourceLoadStatisticsMemoryStore::mergeOperatingDates): 41 (WebKit::ResourceLoadStatisticsMemoryStore::includeTodayAsOperatingDateIfNecessary): 42 (WebKit::ResourceLoadStatisticsMemoryStore::hasStatisticsExpired const): 43 (WebKit::ResourceLoadStatisticsMemoryStore::insertExpiredStatisticForTesting): 44 * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h: 45 Migrated Operating Dates functions from ResourceLoadStatisticsStore 46 now that the database store does not use them. 47 48 * NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp: 49 (WebKit::ResourceLoadStatisticsPersistentStorage::populateMemoryStoreFromDisk): 50 Spelling error. 51 52 * NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp: 53 (WebKit::ResourceLoadStatisticsStore::ResourceLoadStatisticsStore): 54 (WebKit::ResourceLoadStatisticsStore::updateCookieBlockingForDomains): 55 (WebKit::ResourceLoadStatisticsStore::statisticsEpirationTime const): Deleted. 56 (WebKit::ResourceLoadStatisticsStore::mergeOperatingDates): Deleted. 57 (WebKit::ResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary): Deleted. 58 (WebKit::ResourceLoadStatisticsStore::hasStatisticsExpired const): Deleted. 59 * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h: 60 (WebKit::OperatingDate::year const): 61 (WebKit::OperatingDate::month const): 62 (WebKit::OperatingDate::monthDay const): 63 These functions should be public now that they are needed outside 64 of this file after migrating OperatingDates functions. 65 66 (WebKit::ResourceLoadStatisticsStore::parameters const): 67 (WebKit::ResourceLoadStatisticsStore::operatingDates const): Deleted. 68 (WebKit::ResourceLoadStatisticsStore::clearOperatingDates): Deleted. 69 Moved OperatingDates functions to the memory store. Moved 70 operatingDatesWindow constexpr variables to the header file so that 71 the database and memory stores can both use them. 72 73 * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp: 74 (WebKit::WebResourceLoadStatisticsStore::insertExpiredStatisticForTesting): 75 * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h: 76 * NetworkProcess/NetworkProcess.cpp: 77 (WebKit::NetworkProcess::insertExpiredStatisticForTesting): 78 * NetworkProcess/NetworkProcess.h: 79 * NetworkProcess/NetworkProcess.messages.in: 80 * UIProcess/API/C/WKWebsiteDataStoreRef.cpp: 81 (WKWebsiteDataStoreSetStatisticsExpiredStatistic): 82 * UIProcess/API/C/WKWebsiteDataStoreRef.h: 83 * UIProcess/Network/NetworkProcessProxy.cpp: 84 (WebKit::NetworkProcessProxy::insertExpiredStatisticForTesting): 85 * UIProcess/Network/NetworkProcessProxy.h: 86 * UIProcess/WebsiteData/WebsiteDataStore.cpp: 87 (WebKit::WebsiteDataStore::mergeStatisticForTesting): 88 (WebKit::WebsiteDataStore::insertExpiredStatisticForTesting): 89 * UIProcess/WebsiteData/WebsiteDataStore.h: 90 Function to add expired statistic to the ITP store. 91 1 92 2020-05-20 Chris Dumez <cdumez@apple.com> 2 93 -
trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp
r261935 r261963 223 223 "FOREIGN KEY(fromDomainID) REFERENCES ObservedDomains(domainID) ON DELETE CASCADE);"_s; 224 224 225 constexpr auto createOperatingDates = "CREATE TABLE OperatingDates (" 226 "year INTEGER NOT NULL, month INTEGER NOT NULL, monthDay INTEGER NOT NULL);"_s; 227 228 225 229 // CREATE UNIQUE INDEX Queries 226 230 constexpr auto createUniqueIndexStorageAccessUnderTopFrameDomains = "CREATE UNIQUE INDEX IF NOT EXISTS StorageAccessUnderTopFrameDomains_domainID_topLevelDomainID on StorageAccessUnderTopFrameDomains ( domainID, topLevelDomainID );"_s; … … 234 238 constexpr auto createUniqueIndexSubresourceUniqueRedirectsTo = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUniqueRedirectsTo_subresourceDomainID_toDomainID on SubresourceUniqueRedirectsTo ( subresourceDomainID, toDomainID );"_s; 235 239 constexpr auto createUniqueIndexSubresourceUniqueRedirectsFrom = "CREATE UNIQUE INDEX IF NOT EXISTS SubresourceUniqueRedirectsFrom_subresourceDomainID_fromDomainID on SubresourceUnderTopFrameDomains ( subresourceDomainID, fromDomainID );"_s; 240 constexpr auto createUniqueIndexOperatingDates = "CREATE UNIQUE INDEX IF NOT EXISTS OperatingDates_year_month_monthDay on OperatingDates ( year, month, monthDay );"_s; 236 241 237 242 const unsigned minimumPrevalentResourcesForTelemetry = 3; … … 266 271 { "SubresourceUnderTopFrameDomains"_s, createSubresourceUnderTopFrameDomains}, 267 272 { "SubresourceUniqueRedirectsTo"_s, createSubresourceUniqueRedirectsTo}, 268 { "SubresourceUniqueRedirectsFrom"_s, createSubresourceUniqueRedirectsFrom} 273 { "SubresourceUniqueRedirectsFrom"_s, createSubresourceUniqueRedirectsFrom}, 274 { "OperatingDates"_s, createOperatingDates} 269 275 }); 270 276 … … 293 299 weakThis->calculateAndSubmitTelemetry(); 294 300 }); 301 302 includeTodayAsOperatingDateIfNecessary(); 295 303 } 296 304 … … 541 549 || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains) 542 550 || !m_database.executeCommand(createUniqueIndexSubresourceUniqueRedirectsTo) 543 || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains)) { 551 || !m_database.executeCommand(createUniqueIndexSubresourceUnderTopFrameDomains) 552 || !m_database.executeCommand(createUniqueIndexOperatingDates)) { 544 553 RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::createUniqueIndices failed to execute, error message: %" PUBLIC_LOG_STRING, this, m_database.lastErrorMsg()); 545 554 return false; … … 609 618 if (!m_database.executeCommand(createSubresourceUniqueRedirectsFrom)) { 610 619 LOG_ERROR("Could not create SubresourceUniqueRedirectsFrom table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg()); 620 return false; 621 } 622 623 if (!m_database.executeCommand(createOperatingDates)) { 624 LOG_ERROR("Could not create OperatingDates table in database (%i) - %s", m_database.lastError(), m_database.lastErrorMsg()); 611 625 return false; 612 626 } … … 2412 2426 2413 2427 Vector<unsigned> results; 2414 Optional<Seconds> expirationDateTime = statisticsE pirationTime();2428 Optional<Seconds> expirationDateTime = statisticsExpirationTime(); 2415 2429 if (!expirationDateTime) 2416 2430 return results; … … 2926 2940 } 2927 2941 2942 void ResourceLoadStatisticsDatabaseStore::updateOperatingDatesParameters() 2943 { 2944 SQLiteStatement countOperatingDatesStatement(m_database, "SELECT COUNT(*) FROM OperatingDates;"_s); 2945 SQLiteStatement getMostRecentOperatingDateStatement(m_database, "SELECT * FROM OperatingDates ORDER BY year DESC, month DESC, monthDay DESC LIMIT 1;"_s); 2946 SQLiteStatement getLeastRecentOperatingDateStatement(m_database, "SELECT * FROM OperatingDates ORDER BY year, month, monthDay LIMIT 1;"_s); 2947 2948 if (countOperatingDatesStatement.prepare() != SQLITE_OK 2949 || countOperatingDatesStatement.step() != SQLITE_ROW) { 2950 RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::updateOperatingDatesParameters countOperatingDatesStatement failed to step, error message: %{private}s", this, m_database.lastErrorMsg()); 2951 ASSERT_NOT_REACHED(); 2952 return; 2953 } 2954 2955 m_operatingDatesSize = countOperatingDatesStatement.getColumnInt(0); 2956 2957 if (getMostRecentOperatingDateStatement.prepare() != SQLITE_OK 2958 || getMostRecentOperatingDateStatement.step() != SQLITE_ROW) { 2959 RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::updateOperatingDatesParameters getFirstOperatingDateStatement failed to step, error message: %{private}s", this, m_database.lastErrorMsg()); 2960 ASSERT_NOT_REACHED(); 2961 return; 2962 } 2963 2964 m_mostRecentOperatingDate = OperatingDate(getMostRecentOperatingDateStatement.getColumnInt(0), getMostRecentOperatingDateStatement.getColumnInt(1), getMostRecentOperatingDateStatement.getColumnInt(2)); 2965 2966 if (getLeastRecentOperatingDateStatement.prepare() != SQLITE_OK 2967 || getLeastRecentOperatingDateStatement.step() != SQLITE_ROW) { 2968 RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::updateOperatingDatesParameters getLeastRecentOperatingDateStatement failed to step, error message: %{private}s", this, m_database.lastErrorMsg()); 2969 ASSERT_NOT_REACHED(); 2970 return; 2971 } 2972 2973 m_leastRecentOperatingDate = OperatingDate(getLeastRecentOperatingDateStatement.getColumnInt(0), getLeastRecentOperatingDateStatement.getColumnInt(1), getLeastRecentOperatingDateStatement.getColumnInt(2)); 2974 } 2975 2976 Optional<Seconds> ResourceLoadStatisticsDatabaseStore::statisticsExpirationTime() const 2977 { 2978 ASSERT(!RunLoop::isMain()); 2979 2980 if (this->parameters().timeToLiveUserInteraction) 2981 return WallTime::now().secondsSinceEpoch() - this->parameters().timeToLiveUserInteraction.value(); 2982 2983 if (m_operatingDatesSize >= operatingDatesWindowLong) 2984 return m_leastRecentOperatingDate.secondsSinceEpoch(); 2985 2986 return WTF::nullopt; 2987 } 2988 2989 void ResourceLoadStatisticsDatabaseStore::includeTodayAsOperatingDateIfNecessary() 2990 { 2991 ASSERT(!RunLoop::isMain()); 2992 2993 auto today = OperatingDate::today(); 2994 if (m_operatingDatesSize > 0) { 2995 if (today <= m_mostRecentOperatingDate) 2996 return; 2997 } 2998 2999 int rowsToPrune = m_operatingDatesSize - operatingDatesWindowLong + 1; 3000 if (rowsToPrune > 0) { 3001 SQLiteStatement deleteLeastRecentOperatingDateStatement(m_database, "DELETE FROM OperatingDates ORDER BY year, month, monthDay LIMIT ?;"_s); 3002 if (deleteLeastRecentOperatingDateStatement.prepare() != SQLITE_OK 3003 || deleteLeastRecentOperatingDateStatement.bindInt(1, rowsToPrune) != SQLITE_OK 3004 || deleteLeastRecentOperatingDateStatement.step() != SQLITE_DONE) { 3005 RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::includeTodayAsOperatingDateIfNecessary deleteLeastRecentOperatingDateStatement failed to step, error message: %{private}s", this, m_database.lastErrorMsg()); 3006 ASSERT_NOT_REACHED(); 3007 } 3008 } 3009 3010 SQLiteStatement insertOperatingDateStatement(m_database, "INSERT OR IGNORE INTO OperatingDates (year, month, monthDay) SELECT ?, ?, ?;"_s); 3011 if (insertOperatingDateStatement.prepare() != SQLITE_OK 3012 || insertOperatingDateStatement.bindInt(1, today.year()) != SQLITE_OK 3013 || insertOperatingDateStatement.bindInt(2, today.month()) != SQLITE_OK 3014 || insertOperatingDateStatement.bindInt(3, today.monthDay()) != SQLITE_OK 3015 || insertOperatingDateStatement.step() != SQLITE_DONE) { 3016 RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::includeTodayAsOperatingDateIfNecessary insertOperatingDateStatement failed to step, error message: %{private}s", this, m_database.lastErrorMsg()); 3017 ASSERT_NOT_REACHED(); 3018 } 3019 3020 updateOperatingDatesParameters(); 3021 } 3022 3023 bool ResourceLoadStatisticsDatabaseStore::hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow operatingDatesWindow) const 3024 { 3025 ASSERT(!RunLoop::isMain()); 3026 3027 unsigned operatingDatesWindowInDays = 0; 3028 switch (operatingDatesWindow) { 3029 case OperatingDatesWindow::Long: 3030 operatingDatesWindowInDays = operatingDatesWindowLong; 3031 break; 3032 case OperatingDatesWindow::Short: 3033 operatingDatesWindowInDays = operatingDatesWindowShort; 3034 break; 3035 case OperatingDatesWindow::ForLiveOnTesting: 3036 return WallTime::now() > mostRecentUserInteractionTime + operatingTimeWindowForLiveOnTesting; 3037 case OperatingDatesWindow::ForReproTesting: 3038 return true; 3039 } 3040 3041 if (m_operatingDatesSize >= operatingDatesWindowInDays) { 3042 if (OperatingDate::fromWallTime(mostRecentUserInteractionTime) < m_leastRecentOperatingDate) 3043 return true; 3044 } 3045 3046 // If we don't meet the real criteria for an expired statistic, check the user setting for a tighter restriction (mainly for testing). 3047 if (this->parameters().timeToLiveUserInteraction) { 3048 if (WallTime::now() > mostRecentUserInteractionTime + this->parameters().timeToLiveUserInteraction.value()) 3049 return true; 3050 } 3051 return false; 3052 } 3053 3054 void ResourceLoadStatisticsDatabaseStore::insertExpiredStatisticForTesting(const RegistrableDomain& domain, bool hasUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent) 3055 { 3056 // Populate the Operating Dates table with enough days to require pruning. 3057 double daysAgoInSeconds = 0; 3058 for (unsigned i = 1; i <= operatingDatesWindowLong; i++) { 3059 double daysToSubtract = Seconds::fromHours(24 * i).value(); 3060 daysAgoInSeconds = WallTime::now().secondsSinceEpoch().value() - daysToSubtract; 3061 auto dateToInsert = OperatingDate::fromWallTime(WallTime::fromRawSeconds(daysAgoInSeconds)); 3062 3063 SQLiteStatement insertOperatingDateStatement(m_database, "INSERT OR IGNORE INTO OperatingDates (year, month, monthDay) SELECT ?, ?, ?;"_s); 3064 if (insertOperatingDateStatement.prepare() != SQLITE_OK 3065 || insertOperatingDateStatement.bindInt(1, dateToInsert.year()) != SQLITE_OK 3066 || insertOperatingDateStatement.bindInt(2, dateToInsert.month()) != SQLITE_OK 3067 || insertOperatingDateStatement.bindInt(3, dateToInsert.monthDay()) != SQLITE_OK 3068 || insertOperatingDateStatement.step() != SQLITE_DONE) { 3069 RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertExpiredStatisticForTesting insertOperatingDateStatement failed to step, error message: %{private}s", this, m_database.lastErrorMsg()); 3070 ASSERT_NOT_REACHED(); 3071 } 3072 insertOperatingDateStatement.reset(); 3073 } 3074 3075 updateOperatingDatesParameters(); 3076 3077 // Make sure mostRecentUserInteractionTime is the least recent of all entries. 3078 daysAgoInSeconds -= Seconds::fromHours(24).value(); 3079 auto scopedInsertObservedDomainStatement = this->scopedStatement(m_insertObservedDomainStatement, insertObservedDomainQuery, "insertExpiredStatisticForTesting"_s); 3080 if (scopedInsertObservedDomainStatement->bindText(RegistrableDomainIndex, domain.string()) != SQLITE_OK 3081 || scopedInsertObservedDomainStatement->bindDouble(LastSeenIndex, daysAgoInSeconds) != SQLITE_OK 3082 || scopedInsertObservedDomainStatement->bindInt(HadUserInteractionIndex, hasUserInteraction) != SQLITE_OK 3083 || scopedInsertObservedDomainStatement->bindDouble(MostRecentUserInteractionTimeIndex, daysAgoInSeconds) != SQLITE_OK 3084 || scopedInsertObservedDomainStatement->bindInt(GrandfatheredIndex, false) != SQLITE_OK 3085 || scopedInsertObservedDomainStatement->bindInt(IsPrevalentIndex, isPrevalent) != SQLITE_OK 3086 || scopedInsertObservedDomainStatement->bindInt(IsVeryPrevalentIndex, false) != SQLITE_OK 3087 || scopedInsertObservedDomainStatement->bindInt(DataRecordsRemovedIndex, 0) != SQLITE_OK 3088 || scopedInsertObservedDomainStatement->bindInt(TimesAccessedAsFirstPartyDueToUserInteractionIndex, 0) != SQLITE_OK 3089 || scopedInsertObservedDomainStatement->bindInt(TimesAccessedAsFirstPartyDueToStorageAccessAPIIndex, 0) != SQLITE_OK 3090 || scopedInsertObservedDomainStatement->bindInt(IsScheduledForAllButCookieDataRemovalIndex, isScheduledForAllButCookieDataRemoval) != SQLITE_OK) { 3091 RELEASE_LOG_ERROR_IF_ALLOWED(m_sessionID, "%p - ResourceLoadStatisticsDatabaseStore::insertExpiredStatisticForTesting failed to bind, error message: %{private}s", this, m_database.lastErrorMsg()); 3092 ASSERT_NOT_REACHED(); 3093 return; 3094 } 3095 3096 if (scopedInsertObservedDomainStatement->step() != SQLITE_DONE) { 3097 RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::insertExpiredStatisticForTesting failed to commit, error message: %{private}s", this, m_database.lastErrorMsg()); 3098 ASSERT_NOT_REACHED(); 3099 return; 3100 } 3101 } 3102 2928 3103 } // namespace WebKit 2929 3104 -
trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h
r261892 r261963 144 144 bool domainIDExistsInDatabase(int); 145 145 Optional<Vector<String>> checkForMissingTablesInSchema(); 146 void insertExpiredStatisticForTesting(const RegistrableDomain&, bool hasUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent) override; 146 147 147 148 private: 149 void includeTodayAsOperatingDateIfNecessary() override; 150 void clearOperatingDates() override { } 151 bool hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow) const override; 152 Optional<Seconds> statisticsExpirationTime() const; 153 void updateOperatingDatesParameters(); 154 148 155 void openITPDatabase(); 149 156 void addMissingTablesIfNecessary(); … … 278 285 PAL::SessionID m_sessionID; 279 286 bool m_isNewResourceLoadStatisticsDatabaseFile { false }; 287 unsigned m_operatingDatesSize { 0 }; 288 OperatingDate m_leastRecentOperatingDate; 289 OperatingDate m_mostRecentOperatingDate; 280 290 }; 281 291 -
trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp
r261242 r261963 81 81 weakThis->calculateAndSubmitTelemetry(); 82 82 }); 83 84 includeTodayAsOperatingDateIfNecessary(); 83 85 } 84 86 … … 1082 1084 } 1083 1085 1086 Vector<OperatingDate> ResourceLoadStatisticsMemoryStore::mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates) 1087 { 1088 if (existingDates.isEmpty()) 1089 return WTFMove(newDates); 1090 1091 Vector<OperatingDate> mergedDates(existingDates.size() + newDates.size()); 1092 1093 // Merge the two sorted vectors of dates. 1094 std::merge(existingDates.begin(), existingDates.end(), newDates.begin(), newDates.end(), mergedDates.begin()); 1095 // Remove duplicate dates. 1096 removeRepeatedElements(mergedDates); 1097 1098 // Drop old dates until the Vector size reaches operatingDatesWindowLong. 1099 while (mergedDates.size() > operatingDatesWindowLong) 1100 mergedDates.remove(0); 1101 1102 return mergedDates; 1103 } 1104 1105 void ResourceLoadStatisticsMemoryStore::mergeOperatingDates(Vector<OperatingDate>&& newDates) 1106 { 1107 ASSERT(!RunLoop::isMain()); 1108 1109 m_operatingDates = mergeOperatingDates(m_operatingDates, WTFMove(newDates)); 1110 } 1111 1112 void ResourceLoadStatisticsMemoryStore::includeTodayAsOperatingDateIfNecessary() 1113 { 1114 ASSERT(!RunLoop::isMain()); 1115 1116 auto today = OperatingDate::today(); 1117 if (!m_operatingDates.isEmpty() && today <= m_operatingDates.last()) 1118 return; 1119 1120 while (m_operatingDates.size() >= operatingDatesWindowLong) 1121 m_operatingDates.remove(0); 1122 1123 m_operatingDates.append(today); 1124 } 1125 1126 bool ResourceLoadStatisticsMemoryStore::hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow operatingDatesWindow) const 1127 { 1128 ASSERT(!RunLoop::isMain()); 1129 1130 unsigned operatingDatesWindowInDays = 0; 1131 switch (operatingDatesWindow) { 1132 case OperatingDatesWindow::Long: 1133 operatingDatesWindowInDays = operatingDatesWindowLong; 1134 break; 1135 case OperatingDatesWindow::Short: 1136 operatingDatesWindowInDays = operatingDatesWindowShort; 1137 break; 1138 case OperatingDatesWindow::ForLiveOnTesting: 1139 return WallTime::now() > mostRecentUserInteractionTime + operatingTimeWindowForLiveOnTesting; 1140 case OperatingDatesWindow::ForReproTesting: 1141 return true; 1142 } 1143 1144 if (m_operatingDates.size() >= operatingDatesWindowInDays) { 1145 if (OperatingDate::fromWallTime(mostRecentUserInteractionTime) < m_operatingDates.first()) 1146 return true; 1147 } 1148 1149 // If we don't meet the real criteria for an expired statistic, check the user setting for a tighter restriction (mainly for testing). 1150 if (this->parameters().timeToLiveUserInteraction) { 1151 if (WallTime::now() > mostRecentUserInteractionTime + this->parameters().timeToLiveUserInteraction.value()) 1152 return true; 1153 } 1154 1155 return false; 1156 } 1157 1158 bool ResourceLoadStatisticsMemoryStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic, OperatingDatesWindow operatingDatesWindow) const 1159 { 1160 return hasStatisticsExpired(resourceStatistic.mostRecentUserInteractionTime, operatingDatesWindow); 1161 } 1162 1163 void ResourceLoadStatisticsMemoryStore::insertExpiredStatisticForTesting(const RegistrableDomain& domain, bool hasUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent) 1164 { 1165 // Populate the Operating Dates table with enough days to require pruning. 1166 double daysAgoInSeconds = 0; 1167 for (unsigned i = 1; i <= operatingDatesWindowLong; i++) { 1168 double daysToSubtract = Seconds::fromHours(24 * i).value(); 1169 daysAgoInSeconds = WallTime::now().secondsSinceEpoch().value() - daysToSubtract; 1170 auto dateToInsert = OperatingDate::fromWallTime(WallTime::fromRawSeconds(daysAgoInSeconds)); 1171 m_operatingDates.append(OperatingDate(dateToInsert.year(), dateToInsert.month(), dateToInsert.monthDay())); 1172 } 1173 1174 // Make sure mostRecentUserInteractionTime is the least recent of all entries. 1175 daysAgoInSeconds -= Seconds::fromHours(24).value(); 1176 1177 auto statistic = ResourceLoadStatistics(domain); 1178 statistic.lastSeen = WallTime::fromRawSeconds(daysAgoInSeconds); 1179 statistic.hadUserInteraction = hasUserInteraction; 1180 statistic.mostRecentUserInteractionTime = WallTime::fromRawSeconds(daysAgoInSeconds); 1181 statistic.isPrevalentResource = isPrevalent; 1182 statistic.gotLinkDecorationFromPrevalentResource = isScheduledForAllButCookieDataRemoval; 1183 1184 m_resourceStatisticsMap.ensure(statistic.registrableDomain, [&statistic] { 1185 return WTFMove(statistic); 1186 }); 1187 } 1188 1084 1189 } // namespace WebKit 1085 1190 -
trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h
r260668 r261963 114 114 void setLastSeen(const RegistrableDomain&, Seconds) override; 115 115 void removeDataForDomain(const RegistrableDomain&) override; 116 void insertExpiredStatisticForTesting(const RegistrableDomain&, bool hasUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent) override; 116 117 117 118 private: 119 void includeTodayAsOperatingDateIfNecessary() override; 120 const Vector<OperatingDate>& operatingDates() const { return m_operatingDates; } 121 void clearOperatingDates() override { m_operatingDates.clear(); } 122 void mergeOperatingDates(Vector<OperatingDate>&&); 123 bool hasStatisticsExpired(const ResourceLoadStatistics&, OperatingDatesWindow) const; 124 static Vector<OperatingDate> mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates); 125 bool hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow) const override; 126 118 127 static bool shouldBlockAndKeepCookies(const ResourceLoadStatistics&); 119 128 static bool shouldBlockAndPurgeCookies(const ResourceLoadStatistics&); … … 138 147 WeakPtr<ResourceLoadStatisticsPersistentStorage> m_persistentStorage; 139 148 HashMap<RegistrableDomain, ResourceLoadStatistics> m_resourceStatisticsMap; 149 Vector<OperatingDate> m_operatingDates; 140 150 }; 141 151 -
trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp
r252123 r261963 209 209 } 210 210 211 // Debug mode has a prepo ulated memory store.211 // Debug mode has a prepopulated memory store. 212 212 ASSERT_WITH_MESSAGE(m_memoryStore.isEmpty() || m_memoryStore.isDebugModeEnabled(), "This is the initial import so the store should be empty"); 213 213 m_memoryStore.mergeWithDataFromDecoder(*decoder); -
trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.cpp
r261242 r261963 53 53 54 54 constexpr Seconds minimumStatisticsProcessingInterval { 5_s }; 55 constexpr unsigned operatingDatesWindowLong { 30 };56 constexpr unsigned operatingDatesWindowShort { 7 };57 constexpr Seconds operatingTimeWindowForLiveOnTesting { 1_h };58 55 59 56 static String domainsToString(const Vector<RegistrableDomain>& domains) … … 134 131 { 135 132 ASSERT(!RunLoop::isMain()); 136 137 includeTodayAsOperatingDateIfNecessary();138 133 } 139 134 … … 452 447 }); 453 448 } 454 455 456 Optional<Seconds> ResourceLoadStatisticsStore::statisticsEpirationTime() const457 {458 ASSERT(!RunLoop::isMain());459 460 if (m_parameters.timeToLiveUserInteraction)461 return WallTime::now().secondsSinceEpoch() - m_parameters.timeToLiveUserInteraction.value();462 463 if (m_operatingDates.size() >= operatingDatesWindowLong)464 return m_operatingDates.first().secondsSinceEpoch();465 466 return WTF::nullopt;467 }468 469 Vector<OperatingDate> ResourceLoadStatisticsStore::mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates)470 {471 if (existingDates.isEmpty())472 return WTFMove(newDates);473 474 Vector<OperatingDate> mergedDates(existingDates.size() + newDates.size());475 476 // Merge the two sorted vectors of dates.477 std::merge(existingDates.begin(), existingDates.end(), newDates.begin(), newDates.end(), mergedDates.begin());478 // Remove duplicate dates.479 removeRepeatedElements(mergedDates);480 481 // Drop old dates until the Vector size reaches operatingDatesWindowLong.482 while (mergedDates.size() > operatingDatesWindowLong)483 mergedDates.remove(0);484 485 return mergedDates;486 }487 488 void ResourceLoadStatisticsStore::mergeOperatingDates(Vector<OperatingDate>&& newDates)489 {490 ASSERT(!RunLoop::isMain());491 492 m_operatingDates = mergeOperatingDates(m_operatingDates, WTFMove(newDates));493 }494 495 void ResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary()496 {497 ASSERT(!RunLoop::isMain());498 499 auto today = OperatingDate::today();500 if (!m_operatingDates.isEmpty() && today <= m_operatingDates.last())501 return;502 503 while (m_operatingDates.size() >= operatingDatesWindowLong)504 m_operatingDates.remove(0);505 506 m_operatingDates.append(today);507 }508 509 bool ResourceLoadStatisticsStore::hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow operatingDatesWindow) const510 {511 ASSERT(!RunLoop::isMain());512 513 unsigned operatingDatesWindowInDays = 0;514 switch (operatingDatesWindow) {515 case OperatingDatesWindow::Long:516 operatingDatesWindowInDays = operatingDatesWindowLong;517 break;518 case OperatingDatesWindow::Short:519 operatingDatesWindowInDays = operatingDatesWindowShort;520 break;521 case OperatingDatesWindow::ForLiveOnTesting:522 return WallTime::now() > mostRecentUserInteractionTime + operatingTimeWindowForLiveOnTesting;523 case OperatingDatesWindow::ForReproTesting:524 return true;525 }526 527 if (m_operatingDates.size() >= operatingDatesWindowInDays) {528 if (OperatingDate::fromWallTime(mostRecentUserInteractionTime) < m_operatingDates.first())529 return true;530 }531 532 // If we don't meet the real criteria for an expired statistic, check the user setting for a tighter restriction (mainly for testing).533 if (m_parameters.timeToLiveUserInteraction) {534 if (WallTime::now() > mostRecentUserInteractionTime + m_parameters.timeToLiveUserInteraction.value())535 return true;536 }537 538 return false;539 }540 541 bool ResourceLoadStatisticsStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic, OperatingDatesWindow operatingDatesWindow) const542 {543 return hasStatisticsExpired(resourceStatistic.mostRecentUserInteractionTime, operatingDatesWindow);544 }545 449 546 450 bool ResourceLoadStatisticsStore::shouldEnforceSameSiteStrictForSpecificDomain(const RegistrableDomain& domain) const -
trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h
r261242 r261963 60 60 static OperatingDate today(); 61 61 Seconds secondsSinceEpoch() const; 62 int year() const { return m_year; } 63 int month() const { return m_month; } 64 int monthDay() const { return m_monthDay; } 65 62 66 bool operator==(const OperatingDate& other) const; 63 67 bool operator<(const OperatingDate& other) const; 64 68 bool operator<=(const OperatingDate& other) const; 65 69 66 private:67 70 OperatingDate(int year, int month, int monthDay) 68 71 : m_year(year) … … 71 74 { } 72 75 76 private: 73 77 int m_year { 0 }; 74 78 int m_month { 0 }; // [0, 11]. 75 79 int m_monthDay { 0 }; // [1, 31]. 76 80 }; 81 82 constexpr unsigned operatingDatesWindowLong { 30 }; 83 constexpr unsigned operatingDatesWindowShort { 7 }; 84 constexpr Seconds operatingTimeWindowForLiveOnTesting { 1_h }; 77 85 78 86 enum class OperatingDatesWindow : uint8_t { Long, Short, ForLiveOnTesting, ForReproTesting }; … … 106 114 void clearBlockingStateForDomains(const Vector<RegistrableDomain>& domains, CompletionHandler<void()>&&); 107 115 108 void includeTodayAsOperatingDateIfNecessary();109 116 void processStatisticsAndDataRecords(); 110 117 … … 149 156 void setPruneEntriesDownTo(size_t pruneTargetCount); 150 157 void resetParametersToDefaultValues(); 151 Optional<Seconds> statisticsEpirationTime() const;152 158 153 159 virtual void calculateAndSubmitTelemetry() const = 0; … … 197 203 virtual bool isDatabaseStore()const { return false; } 198 204 205 virtual void includeTodayAsOperatingDateIfNecessary() = 0; 206 virtual void clearOperatingDates() = 0; 207 virtual bool hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow) const = 0; 208 virtual void insertExpiredStatisticForTesting(const RegistrableDomain&, bool hasUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool) = 0; 209 199 210 protected: 200 211 static unsigned computeImportance(const WebCore::ResourceLoadStatistics&); 201 static Vector<OperatingDate> mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates);202 212 203 213 ResourceLoadStatisticsStore(WebResourceLoadStatisticsStore&, WorkQueue&, ShouldIncludeLocalhost); … … 205 215 bool dataRecordsBeingRemoved() const { return m_dataRecordsBeingRemoved; } 206 216 207 bool hasStatisticsExpired(const ResourceLoadStatistics&, OperatingDatesWindow) const;208 bool hasStatisticsExpired(WallTime mostRecentUserInteractionTime, OperatingDatesWindow) const;209 217 void scheduleStatisticsProcessingRequestIfNecessary(); 210 void mergeOperatingDates(Vector<OperatingDate>&&);211 218 bool shouldEnforceSameSiteStrictForSpecificDomain(const RegistrableDomain&) const; 212 219 virtual Vector<RegistrableDomain> ensurePrevalentResourcesForDebugMode() = 0; … … 238 245 }; 239 246 const Parameters& parameters() const { return m_parameters; } 240 const Vector<OperatingDate>& operatingDates() const { return m_operatingDates; }241 void clearOperatingDates() { m_operatingDates.clear(); }242 247 WallTime& endOfGrandfatheringTimestamp() { return m_endOfGrandfatheringTimestamp; } 243 248 const WallTime& endOfGrandfatheringTimestamp() const { return m_endOfGrandfatheringTimestamp; } … … 278 283 #endif 279 284 Parameters m_parameters; 280 Vector<OperatingDate> m_operatingDates;281 285 WallTime m_endOfGrandfatheringTimestamp; 282 286 RegistrableDomain m_debugManualPrevalentResource; -
trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp
r261242 r261963 1452 1452 } 1453 1453 1454 void WebResourceLoadStatisticsStore::insertExpiredStatisticForTesting(const RegistrableDomain& domain, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&& completionHandler) 1455 { 1456 ASSERT(RunLoop::isMain()); 1457 1458 postTask([this, domain = domain.isolatedCopy(), hadUserInteraction, isScheduledForAllButCookieDataRemoval, isPrevalent, completionHandler = WTFMove(completionHandler)]() mutable { 1459 if (m_statisticsStore) 1460 m_statisticsStore->insertExpiredStatisticForTesting(WTFMove(domain), hadUserInteraction, isScheduledForAllButCookieDataRemoval, isPrevalent); 1461 postTaskReply(WTFMove(completionHandler)); 1462 }); 1463 } 1464 1454 1465 } // namespace WebKit 1455 1466 -
trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h
r261242 r261963 298 298 299 299 bool isEphemeral() const { return m_isEphemeral == WebCore::ResourceLoadStatistics::IsEphemeral::Yes; }; 300 void insertExpiredStatisticForTesting(const RegistrableDomain&, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&&); 300 301 301 302 private: -
trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp
r261305 r261963 1035 1035 } 1036 1036 1037 void NetworkProcess::insertExpiredStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& domain, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&& completionHandler) 1038 { 1039 if (auto* networkSession = this->networkSession(sessionID)) { 1040 if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics()) 1041 resourceLoadStatistics->insertExpiredStatisticForTesting(domain, hadUserInteraction, isScheduledForAllButCookieDataRemoval, isPrevalent, WTFMove(completionHandler)); 1042 else 1043 completionHandler(); 1044 } else { 1045 ASSERT_NOT_REACHED(); 1046 completionHandler(); 1047 } 1048 } 1049 1037 1050 void NetworkProcess::getAllStorageAccessEntries(PAL::SessionID sessionID, CompletionHandler<void(Vector<String> domains)>&& completionHandler) 1038 1051 { -
trunk/Source/WebKit/NetworkProcess/NetworkProcess.h
r261305 r261963 241 241 void domainIDExistsInDatabase(PAL::SessionID, int domainID, CompletionHandler<void(bool)>&&); 242 242 void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&); 243 void insertExpiredStatisticForTesting(PAL::SessionID, const RegistrableDomain&, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&&); 243 244 void setMinimumTimeBetweenDataRecordsRemoval(PAL::SessionID, Seconds, CompletionHandler<void()>&&); 244 245 void setNotifyPagesWhenDataRecordsWereScanned(PAL::SessionID, bool value, CompletionHandler<void()>&&); -
trunk/Source/WebKit/NetworkProcess/NetworkProcess.messages.in
r261242 r261963 96 96 SetLastSeen(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, Seconds seconds) -> () Async 97 97 MergeStatisticForTesting(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, WebCore::RegistrableDomain topFrameDomain1, WebCore::RegistrableDomain topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, uint64_t dataRecordsRemoved) -> () Async 98 InsertExpiredStatisticForTesting(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent) -> () Async 98 99 SetPrevalentResource(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async 99 100 SetPrevalentResourceForDebugMode(PAL::SessionID sessionID, WebCore::RegistrableDomain resourceDomain) -> () Async -
trunk/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp
r261881 r261963 170 170 } 171 171 172 void WKWebsiteDataStoreSetStatisticsExpiredStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler) 173 { 174 #if ENABLE(RESOURCE_LOAD_STATISTICS) 175 WebKit::toImpl(dataStoreRef)->insertExpiredStatisticForTesting(URL(URL(), WebKit::toImpl(host)->string()), hadUserInteraction, isScheduledForAllButCookieDataRemoval, isPrevalent, [context, completionHandler] { 176 completionHandler(context); 177 }); 178 #else 179 completionHandler(context); 180 #endif 181 } 182 172 183 void WKWebsiteDataStoreSetStatisticsPrevalentResource(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value, void* context, WKWebsiteDataStoreStatisticsPrevalentResourceFunction completionHandler) 173 184 { -
trunk/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.h
r261242 r261963 56 56 typedef void (*WKWebsiteDataStoreStatisticsMergeStatisticFunction)(void* functionContext); 57 57 WK_EXPORT void WKWebsiteDataStoreSetStatisticsMergeStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, void* context, WKWebsiteDataStoreStatisticsMergeStatisticFunction completionHandler); 58 typedef void (*WKWebsiteDataStoreStatisticsExpiredStatisticFunction)(void* functionContext); 59 WK_EXPORT void WKWebsiteDataStoreSetStatisticsExpiredStatistic(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, void* context, WKWebsiteDataStoreStatisticsExpiredStatisticFunction completionHandler); 58 60 typedef void (*WKWebsiteDataStoreStatisticsPrevalentResourceFunction)(void* functionContext); 59 61 WK_EXPORT void WKWebsiteDataStoreSetStatisticsPrevalentResource(WKWebsiteDataStoreRef dataStoreRef, WKStringRef host, bool value, void* context, WKWebsiteDataStoreStatisticsPrevalentResourceFunction completionHandler); -
trunk/Source/WebKit/UIProcess/Network/NetworkProcessProxy.cpp
r261288 r261963 594 594 } 595 595 596 void NetworkProcessProxy::insertExpiredStatisticForTesting(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&& completionHandler) 597 { 598 sendWithAsyncReply(Messages::NetworkProcess::InsertExpiredStatisticForTesting(sessionID, resourceDomain, hadUserInteraction, isScheduledForAllButCookieDataRemoval, isPrevalent), WTFMove(completionHandler)); 599 } 600 596 601 void NetworkProcessProxy::clearPrevalentResource(PAL::SessionID sessionID, const RegistrableDomain& resourceDomain, CompletionHandler<void()>&& completionHandler) 597 602 { -
trunk/Source/WebKit/UIProcess/Network/NetworkProcessProxy.h
r261288 r261963 135 135 void statisticsDatabaseHasAllTables(PAL::SessionID, CompletionHandler<void(bool)>&&); 136 136 void mergeStatisticForTesting(PAL::SessionID, const RegistrableDomain&, const TopFrameDomain& topFrameDomain1, const TopFrameDomain& topFrameDomain2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&); 137 void insertExpiredStatisticForTesting(PAL::SessionID, const RegistrableDomain&, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&&); 137 138 void setAgeCapForClientSideCookies(PAL::SessionID, Optional<Seconds>, CompletionHandler<void()>&&); 138 139 void setCacheMaxAgeCap(PAL::SessionID, Seconds, CompletionHandler<void()>&&); -
trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.cpp
r261717 r261963 1623 1623 } 1624 1624 1625 void WebsiteDataStore::mergeStatisticForTesting(const URL& url 1625 void WebsiteDataStore::mergeStatisticForTesting(const URL& url, const URL& topFrameUrl1, const URL& topFrameUrl2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&& completionHandler) 1626 1626 { 1627 1627 if (url.protocolIsAbout() || url.isEmpty()) { … … 1635 1635 if (auto* process = processPool->networkProcess()) 1636 1636 process->mergeStatisticForTesting(m_sessionID, WebCore::RegistrableDomain { url }, WebCore::RegistrableDomain { topFrameUrl1 }, WebCore::RegistrableDomain { topFrameUrl2 }, lastSeen, hadUserInteraction, mostRecentUserInteraction, isGrandfathered, isPrevalent, isVeryPrevalent, dataRecordsRemoved, [processPool, callbackAggregator = callbackAggregator.copyRef()] { }); 1637 } 1638 } 1639 1640 void WebsiteDataStore::insertExpiredStatisticForTesting(const URL& url, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&& completionHandler) 1641 { 1642 if (url.protocolIsAbout() || url.isEmpty()) { 1643 completionHandler(); 1644 return; 1645 } 1646 1647 auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler)); 1648 1649 for (auto& processPool : processPools()) { 1650 if (auto* process = processPool->networkProcess()) 1651 process->insertExpiredStatisticForTesting(m_sessionID, WebCore::RegistrableDomain { url }, hadUserInteraction, isScheduledForAllButCookieDataRemoval, isPrevalent, [processPool, callbackAggregator = callbackAggregator.copyRef()] { }); 1637 1652 } 1638 1653 } -
trunk/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h
r261242 r261963 173 173 void statisticsDatabaseHasAllTables(CompletionHandler<void(bool)>&&); 174 174 void mergeStatisticForTesting(const URL&, const URL& topFrameUrl1, const URL& topFrameUrl2, Seconds lastSeen, bool hadUserInteraction, Seconds mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, CompletionHandler<void()>&&); 175 void insertExpiredStatisticForTesting(const URL&, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, CompletionHandler<void()>&&); 175 176 void setNotifyPagesWhenDataRecordsWereScanned(bool, CompletionHandler<void()>&&); 176 177 void setIsRunningResourceLoadStatisticsTest(bool, CompletionHandler<void()>&&); -
trunk/Tools/ChangeLog
r261960 r261963 1 2020-05-20 Kate Cheney <katherine_cheney@apple.com> 2 3 Support operating dates in ResourceLoadStatisticsDatabaseStore 4 https://bugs.webkit.org/show_bug.cgi?id=212034 5 <rdar://problem/63349242> 6 7 Reviewed by Brent Fulgham. 8 9 Add a testing function to insert an expired statistic. 10 11 * WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl: 12 * WebKitTestRunner/InjectedBundle/InjectedBundle.cpp: 13 (WTR::InjectedBundle::didReceiveMessageToPage): 14 * WebKitTestRunner/InjectedBundle/TestRunner.cpp: 15 (WTR::TestRunner::setStatisticsExpiredStatistic): 16 (WTR::TestRunner::statisticsCallDidSetExpiredStatisticCallback): 17 * WebKitTestRunner/InjectedBundle/TestRunner.h: 18 * WebKitTestRunner/TestController.cpp: 19 (WTR::TestController::setStatisticsExpiredStatistic): 20 * WebKitTestRunner/TestController.h: 21 * WebKitTestRunner/TestInvocation.cpp: 22 (WTR::TestInvocation::didReceiveMessageFromInjectedBundle): 23 (WTR::TestInvocation::didSetExpiredStatistic): 24 * WebKitTestRunner/TestInvocation.h: 25 1 26 2020-05-20 Chris Dumez <cdumez@apple.com> 2 27 -
trunk/Tools/WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl
r261620 r261963 304 304 void setStatisticsLastSeen(DOMString hostName, double seconds, object completionHandler); 305 305 void setStatisticsMergeStatistic(DOMString hostName, DOMString topFrameDomain1, DOMString topFrameDomain2, double lastSeen, boolean hadUserInteraction, double mostRecentUserInteraction, boolean isGrandfathered, boolean isPrevalent, boolean isVeryPrevalent, unsigned long dataRecordsRemoved, object completionHandler); 306 void setStatisticsExpiredStatistic(DOMString hostName, boolean hadUserInteraction, boolean isScheduledForAllButCookieDataRemoval, boolean isPrevalent, object completionHandler); 306 307 void setStatisticsPrevalentResource(DOMString hostName, boolean value, object completionHandler); 307 308 void setStatisticsVeryPrevalentResource(DOMString hostName, boolean value, object completionHandler); -
trunk/Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.cpp
r261242 r261963 387 387 return; 388 388 } 389 390 if (WKStringIsEqualToUTF8CString(messageName, "CallDidSetExpiredStatistic")) { 391 m_testRunner->statisticsCallDidSetExpiredStatisticCallback(); 392 return; 393 } 389 394 390 395 if (WKStringIsEqualToUTF8CString(messageName, "CallDidSetPrevalentResource")) { -
trunk/Tools/WebKitTestRunner/InjectedBundle/TestRunner.cpp
r261620 r261963 742 742 SetStatisticsLastSeenCallbackID, 743 743 SetStatisticsMergeStatisticCallbackID, 744 SetStatisticsExpiredStatisticCallbackID, 744 745 SetStatisticsPrevalentResourceCallbackID, 745 746 SetStatisticsVeryPrevalentResourceCallbackID, … … 1576 1577 } 1577 1578 1579 void TestRunner::setStatisticsExpiredStatistic(JSStringRef hostName, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, JSValueRef completionHandler) 1580 { 1581 cacheTestRunnerCallback(SetStatisticsExpiredStatisticCallbackID, completionHandler); 1582 1583 Vector<WKRetainPtr<WKStringRef>> keys; 1584 Vector<WKRetainPtr<WKTypeRef>> values; 1585 1586 keys.append(adoptWK(WKStringCreateWithUTF8CString("HostName"))); 1587 values.append(adoptWK(WKStringCreateWithJSString(hostName))); 1588 1589 keys.append(adoptWK(WKStringCreateWithUTF8CString("HadUserInteraction"))); 1590 values.append(adoptWK(WKBooleanCreate(hadUserInteraction))); 1591 1592 keys.append(adoptWK(WKStringCreateWithUTF8CString("IsScheduledForAllButCookieDataRemoval"))); 1593 values.append(adoptWK(WKBooleanCreate(isScheduledForAllButCookieDataRemoval))); 1594 1595 keys.append(adoptWK(WKStringCreateWithUTF8CString("IsPrevalent"))); 1596 values.append(adoptWK(WKBooleanCreate(isPrevalent))); 1597 1598 Vector<WKStringRef> rawKeys(keys.size()); 1599 Vector<WKTypeRef> rawValues(values.size()); 1600 1601 for (size_t i = 0; i < keys.size(); ++i) { 1602 rawKeys[i] = keys[i].get(); 1603 rawValues[i] = values[i].get(); 1604 } 1605 WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("SetStatisticsExpiredStatistic")); 1606 WKRetainPtr<WKDictionaryRef> messageBody = adoptWK(WKDictionaryCreate(rawKeys.data(), rawValues.data(), rawKeys.size())); 1607 WKBundlePostMessage(InjectedBundle::singleton().bundle(), messageName.get(), messageBody.get()); 1608 } 1609 1610 void TestRunner::statisticsCallDidSetExpiredStatisticCallback() 1611 { 1612 callTestRunnerCallback(SetStatisticsExpiredStatisticCallbackID); 1613 } 1614 1578 1615 void TestRunner::setStatisticsPrevalentResource(JSStringRef hostName, bool value, JSValueRef completionHandler) 1579 1616 { -
trunk/Tools/WebKitTestRunner/InjectedBundle/TestRunner.h
r261620 r261963 404 404 void setStatisticsMergeStatistic(JSStringRef hostName, JSStringRef topFrameDomain1, JSStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, unsigned dataRecordsRemoved, JSValueRef completionHandler); 405 405 void statisticsCallDidSetMergeStatisticCallback(); 406 void setStatisticsExpiredStatistic(JSStringRef hostName, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent, JSValueRef completionHandler); 407 void statisticsCallDidSetExpiredStatisticCallback(); 406 408 void setStatisticsPrevalentResource(JSStringRef hostName, bool value, JSValueRef completionHandler); 407 409 void statisticsCallDidSetPrevalentResourceCallback(); -
trunk/Tools/WebKitTestRunner/TestController.cpp
r261882 r261963 3479 3479 } 3480 3480 3481 void TestController::setStatisticsExpiredStatistic(WKStringRef host, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent) 3482 { 3483 ResourceStatisticsCallbackContext context(*this); 3484 WKWebsiteDataStoreSetStatisticsExpiredStatistic(websiteDataStore(), host, hadUserInteraction, isScheduledForAllButCookieDataRemoval, isPrevalent, &context, resourceStatisticsVoidResultCallback); 3485 runUntil(context.done, noTimeout); 3486 m_currentInvocation->didSetExpiredStatistic(); 3487 } 3488 3481 3489 void TestController::setStatisticsPrevalentResource(WKStringRef host, bool value) 3482 3490 { -
trunk/Tools/WebKitTestRunner/TestController.h
r261758 r261963 223 223 void setStatisticsLastSeen(WKStringRef hostName, double seconds); 224 224 void setStatisticsMergeStatistic(WKStringRef host, WKStringRef topFrameDomain1, WKStringRef topFrameDomain2, double lastSeen, bool hadUserInteraction, double mostRecentUserInteraction, bool isGrandfathered, bool isPrevalent, bool isVeryPrevalent, int dataRecordsRemoved); 225 void setStatisticsExpiredStatistic(WKStringRef host, bool hadUserInteraction, bool isScheduledForAllButCookieDataRemoval, bool isPrevalent); 225 226 void setStatisticsPrevalentResource(WKStringRef hostName, bool value); 226 227 void setStatisticsVeryPrevalentResource(WKStringRef hostName, bool value); -
trunk/Tools/WebKitTestRunner/TestInvocation.cpp
r261758 r261963 872 872 return; 873 873 } 874 875 if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsExpiredStatistic")) { 876 ASSERT(WKGetTypeID(messageBody) == WKDictionaryGetTypeID()); 877 878 WKDictionaryRef messageBodyDictionary = static_cast<WKDictionaryRef>(messageBody); 879 WKRetainPtr<WKStringRef> hostNameKey = adoptWK(WKStringCreateWithUTF8CString("HostName")); 880 WKRetainPtr<WKStringRef> hadUserInteractionKey = adoptWK(WKStringCreateWithUTF8CString("HadUserInteraction")); 881 WKRetainPtr<WKStringRef> mostRecentUserInteractionKey = adoptWK(WKStringCreateWithUTF8CString("MostRecentUserInteraction")); 882 WKRetainPtr<WKStringRef> isScheduledForAllButCookieDataRemovalKey = adoptWK(WKStringCreateWithUTF8CString("IsScheduledForAllButCookieDataRemoval")); 883 WKRetainPtr<WKStringRef> isPrevalentKey = adoptWK(WKStringCreateWithUTF8CString("IsPrevalent")); 884 885 WKStringRef hostName = static_cast<WKStringRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hostNameKey.get())); 886 WKBooleanRef hadUserInteraction = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, hadUserInteractionKey.get())); 887 WKBooleanRef isScheduledForAllButCookieDataRemoval = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, isScheduledForAllButCookieDataRemovalKey.get())); 888 WKBooleanRef isPrevalent = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(messageBodyDictionary, isPrevalentKey.get())); 889 890 TestController::singleton().setStatisticsExpiredStatistic(hostName, WKBooleanGetValue(hadUserInteraction), WKBooleanGetValue(isScheduledForAllButCookieDataRemoval), WKBooleanGetValue(isPrevalent)); 891 return; 892 } 874 893 875 894 if (WKStringIsEqualToUTF8CString(messageName, "SetStatisticsPrevalentResource")) { … … 1978 1997 } 1979 1998 1999 void TestInvocation::didSetExpiredStatistic() 2000 { 2001 WKRetainPtr<WKStringRef> messageName = adoptWK(WKStringCreateWithUTF8CString("CallDidSetExpiredStatistic")); 2002 WKPagePostMessageToInjectedBundle(TestController::singleton().mainWebView()->page(), messageName.get(), 0); 2003 } 2004 1980 2005 void TestInvocation::didSetPrevalentResource() 1981 2006 { -
trunk/Tools/WebKitTestRunner/TestInvocation.h
r261242 r261963 84 84 void didSetLastSeen(); 85 85 void didMergeStatistic(); 86 void didSetExpiredStatistic(); 86 87 void didSetPrevalentResource(); 87 88 void didSetVeryPrevalentResource();
Note: See TracChangeset
for help on using the changeset viewer.