Changeset 243276 in webkit

Timestamp:

Mar 20, 2019, 9:38:29 PM (7 years ago)

Author:

youenn@apple.com

Message:

Compute quota after network process restart based on default quota and space used
​https://bugs.webkit.org/show_bug.cgi?id=195804

Reviewed by Chris Dumez.

Source/WebCore:

At creation of quota manager, a default quota will be assigned.
This value is the same for all origins.
Some origins may have been granted a bigger quota by the user.
In that case, the space used might be greater for these origins.
Update at initialization time the quota according the space used as follows:

• If space used is below default quota, stick with default quota.
• If space used is above, set quota to space used rounded by one tenth of the default quota.

The rounding ensures that quota requests will not happen too quickly after a page is loaded.

Test: http/wpt/cache-storage/cache-quota-after-restart.any.html

• Modules/cache/CacheStorageConnection.h:

(WebCore::CacheStorageConnection::setQuotaBasedOnSpaceUsage):

• storage/StorageQuotaManager.cpp:

(WebCore::StorageQuotaManager::setQuotaBasedOnSpaceUsage):
(WebCore::StorageQuotaManager::addUser):

• storage/StorageQuotaManager.h:
• testing/Internals.cpp:

(WebCore::Internals::updateQuotaBasedOnSpaceUsage):

• testing/Internals.h:
• testing/Internals.idl:

Source/WebKit:

Make sure that Cache Storage quota user waits to declare as initialized to its manager
until all data is loaded so that it can report a valid space used from the start.

Add test API to reset the quota to its default value and compute it according current space use.

• NetworkProcess/NetworkProcess.cpp:

(WebKit::NetworkProcess::updateQuotaBasedOnSpaceUsageForTesting):

• NetworkProcess/NetworkProcess.h:
• NetworkProcess/NetworkProcess.messages.in:
• NetworkProcess/cache/CacheStorageEngineCaches.cpp:

(WebKit::CacheStorage::Caches::create):
(WebKit::CacheStorage::Caches::Caches):
(WebKit::CacheStorage::Caches::whenInitialized):

• NetworkProcess/cache/CacheStorageEngineCaches.h:
• NetworkProcess/cache/CacheStorageEngineConnection.cpp:

(WebKit::CacheStorageEngineConnection::dereference):

• WebProcess/Cache/WebCacheStorageConnection.cpp:

(WebKit::WebCacheStorageConnection::setQuotaBasedOnSpaceUsage):

• WebProcess/Cache/WebCacheStorageConnection.h:

LayoutTests:

• http/wpt/cache-storage/cache-quota-after-restart.any-expected.txt: Added.
• http/wpt/cache-storage/cache-quota-after-restart.any.html: Added.
• http/wpt/cache-storage/cache-quota-after-restart.any.js: Added.

(promise_test.async):

Location:

trunk

Files:

• LayoutTests/ChangeLog (modified) (1 diff)
• LayoutTests/http/wpt/cache-storage/cache-quota-after-restart.any-expected.txt (added)
• LayoutTests/http/wpt/cache-storage/cache-quota-after-restart.any.html (added)
• LayoutTests/http/wpt/cache-storage/cache-quota-after-restart.any.js (added)
• Source/WebCore/ChangeLog (modified) (1 diff)
• Source/WebCore/Modules/cache/CacheStorageConnection.h (modified) (1 diff)
• Source/WebCore/storage/StorageQuotaManager.cpp (modified) (2 diffs)
• Source/WebCore/storage/StorageQuotaManager.h (modified) (1 diff)
• Source/WebCore/testing/Internals.cpp (modified) (1 diff)
• Source/WebCore/testing/Internals.h (modified) (1 diff)
• Source/WebCore/testing/Internals.idl (modified) (1 diff)
• Source/WebKit/ChangeLog (modified) (1 diff)
• Source/WebKit/NetworkProcess/NetworkProcess.cpp (modified) (1 diff)
• Source/WebKit/NetworkProcess/NetworkProcess.h (modified) (1 diff)
• Source/WebKit/NetworkProcess/NetworkProcess.messages.in (modified) (1 diff)
• Source/WebKit/NetworkProcess/cache/CacheStorageEngineCaches.cpp (modified) (3 diffs)
• Source/WebKit/NetworkProcess/cache/CacheStorageEngineCaches.h (modified) (2 diffs)
• Source/WebKit/NetworkProcess/cache/CacheStorageEngineConnection.cpp (modified) (1 diff)
• Source/WebKit/WebProcess/Cache/WebCacheStorageConnection.cpp (modified) (2 diffs)
• Source/WebKit/WebProcess/Cache/WebCacheStorageConnection.h (modified) (1 diff)

trunk/LayoutTests/ChangeLog

@@ +1 @@
+2019-03-20  Youenn Fablet  <youenn@apple.com>
+
+        Compute quota after network process restart based on default quota and space used
+        https://bugs.webkit.org/show_bug.cgi?id=195804
+
+        Reviewed by Chris Dumez.
+
+        * http/wpt/cache-storage/cache-quota-after-restart.any-expected.txt: Added.
+        * http/wpt/cache-storage/cache-quota-after-restart.any.html: Added.
+        * http/wpt/cache-storage/cache-quota-after-restart.any.js: Added.
+        (promise_test.async):
+
 2019-03-20  Youenn Fablet  <youenn@apple.com>
 

trunk/Source/WebCore/ChangeLog

@@ +1 @@
+2019-03-20  Youenn Fablet  <youenn@apple.com>
+
+        Compute quota after network process restart based on default quota and space used
+        https://bugs.webkit.org/show_bug.cgi?id=195804
+
+        Reviewed by Chris Dumez.
+
+        At creation of quota manager, a default quota will be assigned.
+        This value is the same for all origins.
+        Some origins may have been granted a bigger quota by the user.
+        In that case, the space used might be greater for these origins.
+        Update at initialization time the quota according the space used as follows:
+        - If space used is below default quota, stick with default quota.
+        - If space used is above, set quota to space used rounded by one tenth of the default quota.
+        The rounding ensures that quota requests will not happen too quickly after a page is loaded.
+
+        Test: http/wpt/cache-storage/cache-quota-after-restart.any.html
+
+        * Modules/cache/CacheStorageConnection.h:
+        (WebCore::CacheStorageConnection::setQuotaBasedOnSpaceUsage):
+        * storage/StorageQuotaManager.cpp:
+        (WebCore::StorageQuotaManager::setQuotaBasedOnSpaceUsage):
+        (WebCore::StorageQuotaManager::addUser):
+        * storage/StorageQuotaManager.h:
+        * testing/Internals.cpp:
+        (WebCore::Internals::updateQuotaBasedOnSpaceUsage):
+        * testing/Internals.h:
+        * testing/Internals.idl:
+
 2019-03-20  Simon Fraser  <simon.fraser@apple.com>
 

trunk/Source/WebCore/Modules/cache/CacheStorageConnection.h

@@ -56 +56 @@
     virtual void clearMemoryRepresentation(const ClientOrigin&, DOMCacheEngine::CompletionCallback&& callback) { callback(DOMCacheEngine::Error::NotImplemented); }
     virtual void engineRepresentation(WTF::Function<void(const String&)>&& callback) { callback(String { }); }
+    virtual void updateQuotaBasedOnSpaceUsage(const ClientOrigin&) { }
 
 protected:

trunk/Source/WebCore/storage/StorageQuotaManager.cpp

@@ -45 +45 @@
 }
 
+void StorageQuotaManager::updateQuotaBasedOnSpaceUsage()
+{
+    if (!m_quota)
+        return;
+
+    auto defaultQuotaStep = m_quota / 10;
+    m_quota = std::max(m_quota, defaultQuotaStep * ((spaceUsage() / defaultQuotaStep) + 1));
+}
+
 void StorageQuotaManager::addUser(StorageQuotaUser& user)
 {
@@ -53 +62 @@
         if (!weakThis)
             return;
-        m_pendingInitializationUsers.remove(&user);
-        m_users.add(&user);
+
+        if (m_pendingInitializationUsers.remove(&user))
+            m_users.add(&user);
+
+        if (!m_pendingInitializationUsers.isEmpty())
+            return;
+
+        updateQuotaBasedOnSpaceUsage();
         processPendingRequests({ });
     });

trunk/Source/WebCore/storage/StorageQuotaManager.h

@@ -63 +63 @@
     void resetQuota(uint64_t newQuota) { m_quota = newQuota; }
 
+    WEBCORE_EXPORT void updateQuotaBasedOnSpaceUsage();
+
 private:
     uint64_t spaceUsage() const;

trunk/Source/WebCore/testing/Internals.cpp

@@ -4758 +4758 @@
 }
 
+void Internals::updateQuotaBasedOnSpaceUsage()
+{
+    auto* document = contextDocument();
+    if (!document)
+        return;
+
+    if (!m_cacheStorageConnection) {
+        if (auto* page = contextDocument()->page())
+            m_cacheStorageConnection = page->cacheStorageProvider().createCacheStorageConnection(page->sessionID());
+        if (!m_cacheStorageConnection)
+            return;
+    }
+
+    m_cacheStorageConnection->updateQuotaBasedOnSpaceUsage(ClientOrigin { document->topOrigin().data(), document->securityOrigin().data() });
+}
+
 void Internals::setConsoleMessageListener(RefPtr<StringCallback>&& listener)
 {

trunk/Source/WebCore/testing/Internals.h

@@ -707 +707 @@
     uint64_t responseSizeWithPadding(FetchResponse&) const;
 
+    void updateQuotaBasedOnSpaceUsage();
+
     void setConsoleMessageListener(RefPtr<StringCallback>&&);
 

trunk/Source/WebCore/testing/Internals.idl

@@ -690 +690 @@
     unsigned long long responseSizeWithPadding(FetchResponse response);
 
+    void updateQuotaBasedOnSpaceUsage();
+
     void setConsoleMessageListener(StringCallback callback);
 

trunk/Source/WebKit/ChangeLog

@@ +1 @@
+2019-03-20  Youenn Fablet  <youenn@apple.com>
+
+        Compute quota after network process restart based on default quota and space used
+        https://bugs.webkit.org/show_bug.cgi?id=195804
+
+        Reviewed by Chris Dumez.
+
+        Make sure that Cache Storage quota user waits to declare as initialized to its manager
+        until all data is loaded so that it can report a valid space used from the start.
+
+        Add test API to reset the quota to its default value and compute it according current space use.
+
+        * NetworkProcess/NetworkProcess.cpp:
+        (WebKit::NetworkProcess::updateQuotaBasedOnSpaceUsageForTesting):
+        * NetworkProcess/NetworkProcess.h:
+        * NetworkProcess/NetworkProcess.messages.in:
+        * NetworkProcess/cache/CacheStorageEngineCaches.cpp:
+        (WebKit::CacheStorage::Caches::create):
+        (WebKit::CacheStorage::Caches::Caches):
+        (WebKit::CacheStorage::Caches::whenInitialized):
+        * NetworkProcess/cache/CacheStorageEngineCaches.h:
+        * NetworkProcess/cache/CacheStorageEngineConnection.cpp:
+        (WebKit::CacheStorageEngineConnection::dereference):
+        * WebProcess/Cache/WebCacheStorageConnection.cpp:
+        (WebKit::WebCacheStorageConnection::setQuotaBasedOnSpaceUsage):
+        * WebProcess/Cache/WebCacheStorageConnection.h:
+
 2019-03-20  Simon Fraser  <simon.fraser@apple.com>
 

trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp

@@ -2229 +2229 @@
 #endif // ENABLE(INDEXED_DATABASE)
 
+void NetworkProcess::updateQuotaBasedOnSpaceUsageForTesting(PAL::SessionID sessionID, const ClientOrigin& origin)
+{
+    auto& manager = storageQuotaManager(sessionID, origin);
+    manager.resetQuota(m_storageQuotaManagers.find(sessionID)->value.defaultQuota);
+    manager.updateQuotaBasedOnSpaceUsage();
+}
+
 #if ENABLE(SANDBOX_EXTENSIONS)
 void NetworkProcess::getSandboxExtensionsForBlobFiles(const Vector<String>& filenames, CompletionHandler<void(SandboxExtension::HandleArray&&)>&& completionHandler)

trunk/Source/WebKit/NetworkProcess/NetworkProcess.h

@@ -282 +282 @@
     void setIDBPerOriginQuota(uint64_t);
 #endif
+    void updateQuotaBasedOnSpaceUsageForTesting(PAL::SessionID, const WebCore::ClientOrigin&);
 
 #if ENABLE(SANDBOX_EXTENSIONS)

trunk/Source/WebKit/NetworkProcess/NetworkProcess.messages.in

@@ -165 +165 @@
     SetIDBPerOriginQuota(uint64_t quota)
 #endif
+    UpdateQuotaBasedOnSpaceUsageForTesting(PAL::SessionID sessionID, struct WebCore::ClientOrigin origin)
 
     StoreAdClickAttribution(PAL::SessionID sessionID, WebCore::AdClickAttribution adClickAttribution)

trunk/Source/WebKit/NetworkProcess/cache/CacheStorageEngineCaches.cpp

@@ -52 +52 @@
 }
 
+Ref<Caches> Caches::create(Engine& engine, WebCore::ClientOrigin&& origin, String&& rootPath, WebCore::StorageQuotaManager& quotaManager)
+{
+    auto caches = adoptRef(*new Caches { engine, WTFMove(origin), WTFMove(rootPath), quotaManager });
+    quotaManager.addUser(caches.get());
+    return caches;
+}
+
 Caches::Caches(Engine& engine, WebCore::ClientOrigin&& origin, String&& rootPath, WebCore::StorageQuotaManager& quotaManager)
     : m_engine(&engine)
@@ -58 +65 @@
     , m_quotaManager(makeWeakPtr(quotaManager))
 {
-    quotaManager.addUser(*this);
 }
 
@@ -67 +73 @@
     if (m_quotaManager)
         m_quotaManager->removeUser(*this);
+}
+
+void Caches::whenInitialized(CompletionHandler<void()>&& callback)
+{
+    initialize([callback = WTFMove(callback)](auto&& error) mutable {
+        if (error)
+            RELEASE_LOG_ERROR(CacheStorage, "Caches::initialize failed, reported space used will be zero");
+        callback();
+    });
 }
 

trunk/Source/WebKit/NetworkProcess/cache/CacheStorageEngineCaches.h

@@ -45 +45 @@
 class Caches final : public RefCounted<Caches>, private WebCore::StorageQuotaUser {
 public:
-    static Ref<Caches> create(Engine& engine, WebCore::ClientOrigin&& origin, String&& rootPath, WebCore::StorageQuotaManager& quotaManager) { return adoptRef(*new Caches { engine, WTFMove(origin), WTFMove(rootPath), quotaManager }); }
+    static Ref<Caches> create(Engine&, WebCore::ClientOrigin&&, String&& rootPath, WebCore::StorageQuotaManager&);
     ~Caches();
 
@@ -92 +92 @@
     void writeCachesToDisk(WebCore::DOMCacheEngine::CompletionCallback&&);
 
+    void whenInitialized(CompletionHandler<void()>&&) final;
+
     void storeOrigin(WebCore::DOMCacheEngine::CompletionCallback&&);
     static Optional<WebCore::ClientOrigin> readOrigin(const NetworkCache::Data&);

trunk/Source/WebKit/NetworkProcess/cache/CacheStorageEngineConnection.cpp

@@ -139 +139 @@
 
     auto referenceResult = references.find(cacheIdentifier);
-    ASSERT(referenceResult != references.end());
     if (referenceResult == references.end())
         return;

trunk/Source/WebKit/WebProcess/Cache/WebCacheStorageConnection.cpp

@@ -31 +31 @@
 #include "NetworkConnectionToWebProcessMessages.h"
 #include "NetworkProcessConnection.h"
+#include "NetworkProcessMessages.h"
 #include "WebCacheStorageProvider.h"
 #include "WebCoreArgumentCoders.h"
@@ -152 +153 @@
 }
 
+void WebCacheStorageConnection::updateQuotaBasedOnSpaceUsage(const ClientOrigin& origin)
+{
+    connection().send(Messages::NetworkProcess::UpdateQuotaBasedOnSpaceUsageForTesting(m_sessionID, origin), 0);
 }
+
+}

trunk/Source/WebKit/WebProcess/Cache/WebCacheStorageConnection.h

@@ -66 +66 @@
     void clearMemoryRepresentation(const WebCore::ClientOrigin&, WebCore::DOMCacheEngine::CompletionCallback&&) final;
     void engineRepresentation(WTF::Function<void(const String&)>&&) final;
+    void updateQuotaBasedOnSpaceUsage(const WebCore::ClientOrigin&) final;
 
     void openCompleted(uint64_t requestIdentifier, const WebCore::DOMCacheEngine::CacheIdentifierOrError&);