Changeset 242911 in webkit


Ignore:
Timestamp:
Mar 13, 2019 2:47:13 PM (5 years ago)
Author:
youenn@apple.com
Message:

Check IDB quota usage through QuotaManager
​https://bugs.webkit.org/show_bug.cgi?id=195302

Reviewed by Chris Dumez.

Source/WebCore:

For every write operation, compute an estimate size and check for quota before proceeding.
When proceeding, store the estimate size in a map.
If size of the database is to be computed when the task is not done,
the estimate size will be added to the current size of the databases.
At the end of the task, the estimate size is removed from the map,
and the databases size is refreshed.

This patch implements size estimation for write tasks.
Put/add operations might overestimate the size
when an old value will be replaced by a new value.
In that case, we do not substract the old value size since we do not know it.

This patch implements database opening by adding a fixed small cost,
as we do not know whether the database is new or not.

For the first IDB request, we have not computed the size of the database.
To do so, we need to go to a background thread and do that file size computation.
For that purpose, we add support for being-initialized quota user.
Quota manager is calling whenInitialized on its quota user and will
delay any quota check requests until its quota user is answering this callback.

For in process IDB, use the default storage quota per origin and do not increase it.
Future work should move it to NetworkProcess and implement some quota checking.

Cache API and IDB quota management are not yet fully unified.
If IDB is used on start-up, we should check for Cache API storage size.
Conversely, on Cache API first wite task, even if IDB is not being used,
we should compute the size of the IDB data for the given origin.

Test: http/tests/IndexedDB/storage-limit.https.html

  • Modules/indexeddb/server/IDBBackingStore.h:
  • Modules/indexeddb/server/IDBServer.cpp:

(WebCore::IDBServer::IDBServer::create):
(WebCore::IDBServer::IDBServer::IDBServer):
(WebCore::IDBServer::m_quotaManagerGetter):
(WebCore::IDBServer::IDBServer::QuotaUser::QuotaUser):
(WebCore::IDBServer::IDBServer::QuotaUser::~QuotaUser):
(WebCore::IDBServer::IDBServer::QuotaUser::clearSpaceUsed):
(WebCore::IDBServer::IDBServer::QuotaUser::whenInitialized):
(WebCore::IDBServer::IDBServer::QuotaUser::initializeSpaceUsed):
(WebCore::IDBServer::IDBServer::quotaUser):
(WebCore::IDBServer::IDBServer::startComputingSpaceUsedForOrigin):
(WebCore::IDBServer::IDBServer::computeSpaceUsedForOrigin):
(WebCore::IDBServer::IDBServer::finishComputingSpaceUsedForOrigin):
(WebCore::IDBServer::IDBServer::requestSpace):
(WebCore::IDBServer::IDBServer::clearSpaceUsed):
(WebCore::IDBServer::IDBServer::setSpaceUsed):
(WebCore::IDBServer::IDBServer::increasePotentialSpaceUsed):
(WebCore::IDBServer::IDBServer::decreasePotentialSpaceUsed):

  • Modules/indexeddb/server/IDBServer.h:

(WebCore::IDBServer::IDBServer::create):

  • Modules/indexeddb/server/MemoryIDBBackingStore.cpp:

(WebCore::IDBServer::MemoryIDBBackingStore::databasesSizeForOrigin const):

  • Modules/indexeddb/server/MemoryIDBBackingStore.h:
  • Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:

(WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForFolder):
(WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForOrigin const):
(WebCore::IDBServer::SQLiteIDBBackingStore::maximumSize const):

  • Modules/indexeddb/server/SQLiteIDBBackingStore.h:
  • Modules/indexeddb/server/UniqueIDBDatabase.cpp:

(WebCore::IDBServer::estimateSize):
(WebCore::IDBServer::UniqueIDBDatabase::UniqueIDBDatabase):
(WebCore::IDBServer::quotaErrorMessageName):
(WebCore::IDBServer::UniqueIDBDatabase::requestSpace):
(WebCore::IDBServer::UniqueIDBDatabase::performCurrentOpenOperation):
(WebCore::IDBServer::UniqueIDBDatabase::storeCallbackOrFireError):
(WebCore::IDBServer::UniqueIDBDatabase::createObjectStore):
(WebCore::IDBServer::UniqueIDBDatabase::createObjectStoreAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::renameObjectStore):
(WebCore::IDBServer::UniqueIDBDatabase::renameObjectStoreAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::createIndex):
(WebCore::IDBServer::UniqueIDBDatabase::createIndexAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::renameIndex):
(WebCore::IDBServer::UniqueIDBDatabase::renameIndexAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::putOrAdd):
(WebCore::IDBServer::UniqueIDBDatabase::putOrAddAfterQuotaCheck):
(WebCore::IDBServer::UniqueIDBDatabase::postDatabaseTaskReply):
(WebCore::IDBServer::UniqueIDBDatabase::immediateCloseForUserDelete):
(WebCore::IDBServer::UniqueIDBDatabase::updateSpaceUsedIfNeeded):
(WebCore::IDBServer::UniqueIDBDatabase::performErrorCallback):
(WebCore::IDBServer::UniqueIDBDatabase::performKeyDataCallback):

  • Modules/indexeddb/server/UniqueIDBDatabase.h:

(WebCore::IDBServer::UniqueIDBDatabase::server):

  • Modules/indexeddb/shared/InProcessIDBServer.cpp:

(WebCore::InProcessIDBServer::create):
(WebCore::InProcessIDBServer::quotaManager):
(WebCore::storageQuotaManagerGetter):
(WebCore::InProcessIDBServer::InProcessIDBServer):

  • Modules/indexeddb/shared/InProcessIDBServer.h:
  • loader/EmptyClients.cpp:
  • storage/StorageQuotaManager.cpp:

(WebCore::StorageQuotaManager::addUser):
(WebCore::StorageQuotaManager::requestSpace):

  • storage/StorageQuotaManager.h:

(WebCore::StorageQuotaManager::defaultQuota):
(WebCore::StorageQuotaManager::removeUser):

  • storage/StorageQuotaUser.h:

(WebCore::StorageQuotaUser::whenInitialized):

Source/WebKit:

Set the quota manager getter for IDBServer at creation time.

  • NetworkProcess/NetworkProcess.cpp:

(WebKit::NetworkProcess::createIDBServer):
(WebKit::NetworkProcess::idbServer):

  • NetworkProcess/NetworkProcess.h:
  • WebProcess/Databases/WebDatabaseProvider.cpp:

(WebKit::WebDatabaseProvider::idbConnectionToServerForSession):

Source/WebKitLegacy:

  • Storage/WebDatabaseProvider.cpp:

(WebDatabaseProvider::idbConnectionToServerForSession):

LayoutTests:

Update IDB quota test according quota limit of 400ko.
Update WK1 test expectations to skip quota check tests.

  • http/tests/IndexedDB/resources/shared.js: Added.
  • http/tests/IndexedDB/resources/storage-limit.js: Added.
  • http/tests/IndexedDB/storage-limit.https-expected.txt: Added.
  • http/tests/IndexedDB/storage-limit.https.html: Added.
  • platform/mac-wk1/TestExpectations:
  • platform/win/TestExpectations:
  • storage/indexeddb/resources/storage-limit.js:
  • storage/indexeddb/storage-limit-expected.txt:
Location:
trunk
Files:
2 added
27 edited
2 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r242900 r242911  
     12019-03-13  Youenn Fablet  <youenn@apple.com>
     2
     3        Check IDB quota usage through QuotaManager
     4        https://bugs.webkit.org/show_bug.cgi?id=195302
     5
     6        Reviewed by Chris Dumez.
     7
     8        Update IDB quota test according quota limit of 400ko.
     9        Update WK1 test expectations to skip quota check tests.
     10
     11        * http/tests/IndexedDB/resources/shared.js: Added.
     12        * http/tests/IndexedDB/resources/storage-limit.js: Added.
     13        * http/tests/IndexedDB/storage-limit.https-expected.txt: Added.
     14        * http/tests/IndexedDB/storage-limit.https.html: Added.
     15        * platform/mac-wk1/TestExpectations:
     16        * platform/win/TestExpectations:
     17        * storage/indexeddb/resources/storage-limit.js:
     18        * storage/indexeddb/storage-limit-expected.txt:
     19
    1202019-03-13  Truitt Savell  <tsavell@apple.com>
    221

  • trunk/LayoutTests/http/tests/IndexedDB/resources/storage-limit.js

    r242910 r242911  
    1 if (this.importScripts) {
    2     importScripts('../../../resources/js-test.js');
    3     importScripts('shared.js');
    4 }
     1if (window.testRunner)
     2    testRunner.setAllowStorageQuotaIncrease(false);
    53
    6 var quota = 1024 * 1024; // 1 MB
    74description("This test makes sure that storage of indexedDB does not grow unboundedly.");
    85
    9 if (window.testRunner)
    10     testRunner.setIDBPerOriginQuota(quota);
    11 
    12 indexedDBTest(prepareDatabase, onOpenSuccess);
     6window.caches.open("test").then(cache => {
     7    return cache.put(new Request("/test"), new Response(new Uint8Array(204800)));
     8}).then(() => {
     9    indexedDBTest(prepareDatabase, onOpenSuccess, {'suffix': '-1'});
     10}).catch(e => {
     11    testFailed("Cache API store operation failed: " + e);
     12    finishJSTest();
     13});
    1314
    1415function prepareDatabase(event)
    1516{
    16     preamble(event);
    1717    evalAndLog("db = event.target.result");
    1818    evalAndLog("store = db.createObjectStore('store')");
    1919}
    2020
    21 function onOpenSuccess(event)
     21// Quota for test is 400ko, but IDB is eating some of it when initializing files.
     22// Let's make sure that 200ko is fine but 200ko after 200ko is not fine.
     23async function onOpenSuccess(event)
    2224{
    23     preamble(event);
    2425    evalAndLog("db = event.target.result");
    2526    evalAndLog("store = db.transaction('store', 'readwrite').objectStore('store')");
    26     evalAndLog("request = store.add(new Uint8Array(" + (quota + 1) + "), 0)");
     27    evalAndLog("request = store.add(new Uint8Array(204800), 'key')");
    2728    request.onerror = function(event) {
    2829        shouldBeTrue("'error' in request");

  • trunk/LayoutTests/http/tests/IndexedDB/storage-limit.https-expected.txt

    r242910 r242911  
    88indexedDB.deleteDatabase(dbname)
    99indexedDB.open(dbname)
    10 
    11 prepareDatabase():
    1210db = event.target.result
    1311store = db.createObjectStore('store')
    14 
    15 onOpenSuccess():
    1612db = event.target.result
    1713store = db.transaction('store', 'readwrite').objectStore('store')
    18 request = store.add(new Uint8Array(1048577), 0)
     14request = store.add(new Uint8Array(204800), 'key')
    1915PASS 'error' in request is true
    2016PASS request.error.code is DOMException.QUOTA_EXCEEDED_ERR

  • trunk/LayoutTests/platform/mac-wk1/TestExpectations

    r242809 r242911  
    162162imported/w3c/web-platform-tests/fetch/api/request/destination [ Skip ]
    163163imported/w3c/web-platform-tests/fetch/cross-origin-resource-policy [ Skip ]
     164
     165# Quota check missing in WK1
     166http/tests/IndexedDB/storage-limit.https.html [ Skip ]
     167storage/indexeddb/storage-limit.html [ Skip ]
    164168
    165169# Skip WebRTC for now in WK1

  • trunk/LayoutTests/platform/win/TestExpectations

    r242642 r242911  
    23732373http/tests/xmlhttprequest/web-apps/012.html
    23742374http/tests/xmlhttprequest/web-apps/013.html
     2375
     2376storage/indexeddb/storage-limit.html [ Skip ]
     2377http/tests/IndexedDB/storage-limit.https.html [ Skip ]
    23752378
    23762379# Assertion failures: Not investigated

  • trunk/LayoutTests/storage/indexeddb/resources/storage-limit.js

    r237700 r242911  
    44}
    55
    6 var quota = 1024 * 1024; // 1 MB
     6if (window.testRunner)
     7    testRunner.setAllowStorageQuotaIncrease(false);
     8
     9var quota = 400 * 1024; // default quota for testing.
    710description("This test makes sure that storage of indexedDB does not grow unboundedly.");
    8 
    9 if (window.testRunner)
    10     testRunner.setIDBPerOriginQuota(quota);
    1111
    1212indexedDBTest(prepareDatabase, onOpenSuccess);

  • trunk/LayoutTests/storage/indexeddb/storage-limit-expected.txt

    r237700 r242911  
    1616db = event.target.result
    1717store = db.transaction('store', 'readwrite').objectStore('store')
    18 request = store.add(new Uint8Array(1048577), 0)
     18request = store.add(new Uint8Array(409601), 0)
    1919PASS 'error' in request is true
    2020PASS request.error.code is DOMException.QUOTA_EXCEEDED_ERR

  • trunk/Source/WebCore/ChangeLog

    r242909 r242911  
     12019-03-13  Youenn Fablet  <youenn@apple.com>
     2
     3        Check IDB quota usage through QuotaManager
     4        https://bugs.webkit.org/show_bug.cgi?id=195302
     5
     6        Reviewed by Chris Dumez.
     7
     8        For every write operation, compute an estimate size and check for quota before proceeding.
     9        When proceeding, store the estimate size in a map.
     10        If size of the database is to be computed when the task is not done,
     11        the estimate size will be added to the current size of the databases.
     12        At the end of the task, the estimate size is removed from the map,
     13        and the databases size is refreshed.
     14
     15        This patch implements size estimation for write tasks.
     16        Put/add operations might overestimate the size
     17        when an old value will be replaced by a new value.
     18        In that case, we do not substract the old value size since we do not know it.
     19
     20        This patch implements database opening by adding a fixed small cost,
     21        as we do not know whether the database is new or not.
     22
     23        For the first IDB request, we have not computed the size of the database.
     24        To do so, we need to go to a background thread and do that file size computation.
     25        For that purpose, we add support for being-initialized quota user.
     26        Quota manager is calling whenInitialized on its quota user and will
     27        delay any quota check requests until its quota user is answering this callback.
     28
     29        For in process IDB, use the default storage quota per origin and do not increase it.
     30        Future work should move it to NetworkProcess and implement some quota checking.
     31
     32        Cache API and IDB quota management are not yet fully unified.
     33        If IDB is used on start-up, we should check for Cache API storage size.
     34        Conversely, on Cache API first wite task, even if IDB is not being used,
     35        we should compute the size of the IDB data for the given origin.
     36
     37        Test: http/tests/IndexedDB/storage-limit.https.html
     38
     39        * Modules/indexeddb/server/IDBBackingStore.h:
     40        * Modules/indexeddb/server/IDBServer.cpp:
     41        (WebCore::IDBServer::IDBServer::create):
     42        (WebCore::IDBServer::IDBServer::IDBServer):
     43        (WebCore::IDBServer::m_quotaManagerGetter):
     44        (WebCore::IDBServer::IDBServer::QuotaUser::QuotaUser):
     45        (WebCore::IDBServer::IDBServer::QuotaUser::~QuotaUser):
     46        (WebCore::IDBServer::IDBServer::QuotaUser::clearSpaceUsed):
     47        (WebCore::IDBServer::IDBServer::QuotaUser::whenInitialized):
     48        (WebCore::IDBServer::IDBServer::QuotaUser::initializeSpaceUsed):
     49        (WebCore::IDBServer::IDBServer::quotaUser):
     50        (WebCore::IDBServer::IDBServer::startComputingSpaceUsedForOrigin):
     51        (WebCore::IDBServer::IDBServer::computeSpaceUsedForOrigin):
     52        (WebCore::IDBServer::IDBServer::finishComputingSpaceUsedForOrigin):
     53        (WebCore::IDBServer::IDBServer::requestSpace):
     54        (WebCore::IDBServer::IDBServer::clearSpaceUsed):
     55        (WebCore::IDBServer::IDBServer::setSpaceUsed):
     56        (WebCore::IDBServer::IDBServer::increasePotentialSpaceUsed):
     57        (WebCore::IDBServer::IDBServer::decreasePotentialSpaceUsed):
     58        * Modules/indexeddb/server/IDBServer.h:
     59        (WebCore::IDBServer::IDBServer::create):
     60        * Modules/indexeddb/server/MemoryIDBBackingStore.cpp:
     61        (WebCore::IDBServer::MemoryIDBBackingStore::databasesSizeForOrigin const):
     62        * Modules/indexeddb/server/MemoryIDBBackingStore.h:
     63        * Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:
     64        (WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForFolder):
     65        (WebCore::IDBServer::SQLiteIDBBackingStore::databasesSizeForOrigin const):
     66        (WebCore::IDBServer::SQLiteIDBBackingStore::maximumSize const):
     67        * Modules/indexeddb/server/SQLiteIDBBackingStore.h:
     68        * Modules/indexeddb/server/UniqueIDBDatabase.cpp:
     69        (WebCore::IDBServer::estimateSize):
     70        (WebCore::IDBServer::UniqueIDBDatabase::UniqueIDBDatabase):
     71        (WebCore::IDBServer::quotaErrorMessageName):
     72        (WebCore::IDBServer::UniqueIDBDatabase::requestSpace):
     73        (WebCore::IDBServer::UniqueIDBDatabase::performCurrentOpenOperation):
     74        (WebCore::IDBServer::UniqueIDBDatabase::storeCallbackOrFireError):
     75        (WebCore::IDBServer::UniqueIDBDatabase::createObjectStore):
     76        (WebCore::IDBServer::UniqueIDBDatabase::createObjectStoreAfterQuotaCheck):
     77        (WebCore::IDBServer::UniqueIDBDatabase::renameObjectStore):
     78        (WebCore::IDBServer::UniqueIDBDatabase::renameObjectStoreAfterQuotaCheck):
     79        (WebCore::IDBServer::UniqueIDBDatabase::createIndex):
     80        (WebCore::IDBServer::UniqueIDBDatabase::createIndexAfterQuotaCheck):
     81        (WebCore::IDBServer::UniqueIDBDatabase::renameIndex):
     82        (WebCore::IDBServer::UniqueIDBDatabase::renameIndexAfterQuotaCheck):
     83        (WebCore::IDBServer::UniqueIDBDatabase::putOrAdd):
     84        (WebCore::IDBServer::UniqueIDBDatabase::putOrAddAfterQuotaCheck):
     85        (WebCore::IDBServer::UniqueIDBDatabase::postDatabaseTaskReply):
     86        (WebCore::IDBServer::UniqueIDBDatabase::immediateCloseForUserDelete):
     87        (WebCore::IDBServer::UniqueIDBDatabase::updateSpaceUsedIfNeeded):
     88        (WebCore::IDBServer::UniqueIDBDatabase::performErrorCallback):
     89        (WebCore::IDBServer::UniqueIDBDatabase::performKeyDataCallback):
     90        * Modules/indexeddb/server/UniqueIDBDatabase.h:
     91        (WebCore::IDBServer::UniqueIDBDatabase::server):
     92        * Modules/indexeddb/shared/InProcessIDBServer.cpp:
     93        (WebCore::InProcessIDBServer::create):
     94        (WebCore::InProcessIDBServer::quotaManager):
     95        (WebCore::storageQuotaManagerGetter):
     96        (WebCore::InProcessIDBServer::InProcessIDBServer):
     97        * Modules/indexeddb/shared/InProcessIDBServer.h:
     98        * loader/EmptyClients.cpp:
     99        * storage/StorageQuotaManager.cpp:
     100        (WebCore::StorageQuotaManager::addUser):
     101        (WebCore::StorageQuotaManager::requestSpace):
     102        * storage/StorageQuotaManager.h:
     103        (WebCore::StorageQuotaManager::defaultQuota):
     104        (WebCore::StorageQuotaManager::removeUser):
     105        * storage/StorageQuotaUser.h:
     106        (WebCore::StorageQuotaUser::whenInitialized):
     107
    11082019-03-13  Chris Dumez  <cdumez@apple.com>
    2109

  • trunk/Source/WebCore/Modules/indexeddb/server/IDBBackingStore.h

    r238283 r242911  
    100100    virtual bool isEphemeral() = 0;
    101101
     102    virtual uint64_t databasesSizeForOrigin() const = 0;
    102103    virtual void setQuota(uint64_t) = 0;
    103104protected:

  • trunk/Source/WebCore/Modules/indexeddb/server/IDBServer.cpp

    r242136 r242911  
    3636#include "SQLiteIDBBackingStore.h"
    3737#include "SecurityOrigin.h"
     38#include "StorageQuotaManager.h"
    3839#include <wtf/CrossThreadCopier.h>
    3940#include <wtf/Locker.h>
    … …  
    4344namespace IDBServer {
    4445
    45 Ref<IDBServer> IDBServer::create(IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
    46 {
    47     return adoptRef(*new IDBServer(fileHandler, WTFMove(isClosingDatabaseCallback)));
    48 }
    49 
    50 Ref<IDBServer> IDBServer::create(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
    51 {
    52     return adoptRef(*new IDBServer(databaseDirectoryPath, fileHandler, WTFMove(isClosingDatabaseCallback)));
    53 }
    54 
    55 IDBServer::IDBServer(IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
     46Ref<IDBServer> IDBServer::create(PAL::SessionID sessionID, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
     47{
     48    return adoptRef(*new IDBServer(sessionID, fileHandler, WTFMove(quotaManagerGetter), WTFMove(isClosingDatabaseCallback)));
     49}
     50
     51Ref<IDBServer> IDBServer::create(PAL::SessionID sessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
     52{
     53    return adoptRef(*new IDBServer(sessionID, databaseDirectoryPath, fileHandler, WTFMove(quotaManagerGetter), WTFMove(isClosingDatabaseCallback)));
     54}
     55
     56IDBServer::IDBServer(PAL::SessionID sessionID, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
    5657    : CrossThreadTaskHandler("IndexedDatabase Server")
     58    , m_sessionID(sessionID)
    5759    , m_backingStoreTemporaryFileHandler(fileHandler)
    5860    , m_isClosingDatabaseCallback(WTFMove(isClosingDatabaseCallback))
    5961    , m_isClosingDatabaseHysteresis([&](PAL::HysteresisState state) { m_isClosingDatabaseCallback(state == PAL::HysteresisState::Started); })
    60 {
    61 }
    62 
    63 IDBServer::IDBServer(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
     62    , m_quotaManagerGetter(WTFMove(quotaManagerGetter))
     63{
     64}
     65
     66IDBServer::IDBServer(PAL::SessionID sessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler& fileHandler, QuotaManagerGetter&& quotaManagerGetter, WTF::Function<void(bool)>&& isClosingDatabaseCallback)
    6467    : CrossThreadTaskHandler("IndexedDatabase Server")
     68    , m_sessionID(sessionID)
    6569    , m_databaseDirectoryPath(databaseDirectoryPath)
    6670    , m_backingStoreTemporaryFileHandler(fileHandler)
    6771    , m_isClosingDatabaseCallback(WTFMove(isClosingDatabaseCallback))
    6872    , m_isClosingDatabaseHysteresis([&](PAL::HysteresisState state) { m_isClosingDatabaseCallback(state == PAL::HysteresisState::Started); })
     73    , m_quotaManagerGetter(WTFMove(quotaManagerGetter))
    6974{
    7075    LOG(IndexedDB, "IDBServer created at path %s", databaseDirectoryPath.utf8().data());
    … …  
    680685}
    681686
     687IDBServer::QuotaUser::QuotaUser(IDBServer& server, StorageQuotaManager* manager, ClientOrigin&& origin)
     688    : m_server(server)
     689    , m_manager(makeWeakPtr(manager))
     690    , m_origin(WTFMove(origin))
     691    , m_isInitialized(m_server.m_sessionID.isEphemeral())
     692{
     693    if (manager)
     694        manager->addUser(*this);
     695}
     696
     697IDBServer::QuotaUser::~QuotaUser()
     698{
     699    if (m_manager)
     700        m_manager->removeUser(*this);
     701}
     702
     703void IDBServer::QuotaUser::resetSpaceUsed()
     704{
     705    m_spaceUsed = 0;
     706    m_estimatedSpaceIncrease = 0;
     707
     708    if (!m_manager)
     709        return;
     710
     711    if (m_server.m_sessionID.isEphemeral())
     712        return;
     713
     714    if (!m_isInitialized)
     715        return;
     716
     717    ASSERT(!m_initializationCallback);
     718
     719    m_isInitialized = false;
     720
     721    // Do add/remove to trigger call to whenInitialized.
     722    m_manager->removeUser(*this);
     723    m_manager->addUser(*this);
     724}
     725
     726void IDBServer::QuotaUser::whenInitialized(CompletionHandler<void()>&& callback)
     727{
     728    if (m_isInitialized) {
     729        callback();
     730        return;
     731    }
     732    m_initializationCallback = WTFMove(callback);
     733    m_server.startComputingSpaceUsedForOrigin(m_origin);
     734}
     735
     736void IDBServer::QuotaUser::initializeSpaceUsed(uint64_t spaceUsed)
     737{
     738    ASSERT(m_isInitialized || !m_estimatedSpaceIncrease);
     739    m_spaceUsed = spaceUsed;
     740    m_isInitialized = true;
     741
     742    if (auto callback = WTFMove(m_initializationCallback))
     743        callback();
     744}
     745
     746IDBServer::QuotaUser& IDBServer::quotaUser(const ClientOrigin& origin)
     747{
     748    return *m_quotaUsers.ensure(origin, [this, &origin] {
     749        return std::make_unique<QuotaUser>(*this, m_quotaManagerGetter(m_sessionID, origin), ClientOrigin { origin });
     750    }).iterator->value;
     751}
     752
     753void IDBServer::startComputingSpaceUsedForOrigin(const ClientOrigin& origin)
     754{
     755    ASSERT(!m_sessionID.isEphemeral());
     756    postDatabaseTask(createCrossThreadTask(*this, &IDBServer::computeSpaceUsedForOrigin, origin));
     757}
     758
     759void IDBServer::computeSpaceUsedForOrigin(const ClientOrigin& origin)
     760{
     761    ASSERT(!isMainThread());
     762
     763    auto path = IDBDatabaseIdentifier::databaseDirectoryRelativeToRoot(origin.topOrigin, origin.clientOrigin, m_databaseDirectoryPath);
     764    auto size = SQLiteIDBBackingStore::databasesSizeForFolder(path);
     765
     766    postDatabaseTaskReply(createCrossThreadTask(*this, &IDBServer::finishComputingSpaceUsedForOrigin, origin, size));
     767}
     768
     769void IDBServer::finishComputingSpaceUsedForOrigin(const ClientOrigin& origin, uint64_t spaceUsed)
     770{
     771    quotaUser(origin).initializeSpaceUsed(spaceUsed);
     772}
     773
     774void IDBServer::requestSpace(const ClientOrigin& origin, uint64_t taskSize, CompletionHandler<void(StorageQuotaManager::Decision)>&& callback)
     775{
     776    auto* quotaManager = quotaUser(origin).manager();
     777    if (!quotaManager) {
     778        callback(StorageQuotaManager::Decision::Deny);
     779        return;
     780    }
     781
     782    quotaManager->requestSpace(taskSize, WTFMove(callback));
     783}
     784
     785void IDBServer::resetSpaceUsed(const ClientOrigin& origin)
     786{
     787    if (auto* user = m_quotaUsers.get(origin))
     788        user->resetSpaceUsed();
     789}
     790
     791void IDBServer::setSpaceUsed(const ClientOrigin& origin, uint64_t taskSize)
     792{
     793    quotaUser(origin).setSpaceUsed(taskSize);
     794}
     795
     796void IDBServer::increasePotentialSpaceUsed(const ClientOrigin& origin, uint64_t taskSize)
     797{
     798    quotaUser(origin).increasePotentialSpaceUsed(taskSize);
     799}
     800
     801void IDBServer::decreasePotentialSpaceUsed(const ClientOrigin& origin, uint64_t spaceUsed)
     802{
     803    quotaUser(origin).decreasePotentialSpaceUsed(spaceUsed);
     804}
     805
    682806} // namespace IDBServer
    683807} // namespace WebCore

  • trunk/Source/WebCore/Modules/indexeddb/server/IDBServer.h

    r242136 r242911  
    3030#include "IDBConnectionToClient.h"
    3131#include "IDBDatabaseIdentifier.h"
     32#include "StorageQuotaManager.h"
     33#include "StorageQuotaUser.h"
    3234#include "UniqueIDBDatabase.h"
    3335#include "UniqueIDBDatabaseConnection.h"
    3436#include <pal/HysteresisActivity.h>
     37#include <pal/SessionID.h>
    3538#include <wtf/CrossThreadTaskHandler.h>
    3639#include <wtf/HashMap.h>
    … …  
    4548class IDBRequestData;
    4649class IDBValue;
     50class StorageQuotaManager;
    4751
    4852struct IDBGetRecordData;
    … …  
    5660class IDBServer : public RefCounted<IDBServer>, public CrossThreadTaskHandler {
    5761public:
    58     static Ref<IDBServer> create(IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
    59     WEBCORE_EXPORT static Ref<IDBServer> create(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
     62    using QuotaManagerGetter = WTF::Function<StorageQuotaManager*(PAL::SessionID, const ClientOrigin&)>;
     63    static Ref<IDBServer> create(PAL::SessionID, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
     64    WEBCORE_EXPORT static Ref<IDBServer> create(PAL::SessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&& isClosingDatabaseCallback = [](bool) { });
    6065
    6166    WEBCORE_EXPORT void registerConnection(IDBConnectionToClient&);
    … …  
    115120    void hysteresisUpdated(PAL::HysteresisState);
    116121
     122    void requestSpace(const ClientOrigin&, uint64_t taskSize, CompletionHandler<void(StorageQuotaManager::Decision)>&&);
     123    void increasePotentialSpaceUsed(const ClientOrigin&, uint64_t taskSize);
     124    void decreasePotentialSpaceUsed(const ClientOrigin&, uint64_t taskSize);
     125    void setSpaceUsed(const ClientOrigin&, uint64_t spaceUsed);
     126    void resetSpaceUsed(const ClientOrigin&);
     127
    117128private:
    118     IDBServer(IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&&);
    119     IDBServer(const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, WTF::Function<void(bool)>&&);
     129    IDBServer(PAL::SessionID, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&&);
     130    IDBServer(PAL::SessionID, const String& databaseDirectoryPath, IDBBackingStoreTemporaryFileHandler&, QuotaManagerGetter&&, WTF::Function<void(bool)>&&);
    120131
    121132    UniqueIDBDatabase& getOrCreateUniqueIDBDatabase(const IDBDatabaseIdentifier&);
    … …  
    128139    void didPerformCloseAndDeleteDatabases(uint64_t callbackID);
    129140
     141    class QuotaUser final : public StorageQuotaUser {
     142        WTF_MAKE_FAST_ALLOCATED;
     143    public:
     144        QuotaUser(IDBServer&, StorageQuotaManager*, ClientOrigin&&);
     145        ~QuotaUser();
     146
     147        StorageQuotaManager* manager() { return m_manager.get(); }
     148
     149        void setSpaceUsed(uint64_t spaceUsed) { m_spaceUsed = spaceUsed; }
     150        void resetSpaceUsed();
     151
     152        void increasePotentialSpaceUsed(uint64_t increase) { m_estimatedSpaceIncrease += increase; }
     153        void decreasePotentialSpaceUsed(uint64_t decrease)
     154        {
     155            ASSERT(m_estimatedSpaceIncrease >= decrease);
     156            m_estimatedSpaceIncrease -= decrease;
     157        }
     158
     159        void initializeSpaceUsed(uint64_t spaceUsed);
     160
     161    private:
     162        uint64_t spaceUsed() const final { return m_spaceUsed + m_estimatedSpaceIncrease; }
     163        void whenInitialized(CompletionHandler<void()>&&) final;
     164
     165        IDBServer& m_server;
     166        WeakPtr<StorageQuotaManager> m_manager;
     167        ClientOrigin m_origin;
     168        bool m_isInitialized { false };
     169        uint64_t m_spaceUsed { 0 };
     170        uint64_t m_estimatedSpaceIncrease { 0 };
     171        CompletionHandler<void()> m_initializationCallback;
     172    };
     173
     174    QuotaUser& quotaUser(const ClientOrigin&);
     175    void startComputingSpaceUsedForOrigin(const ClientOrigin&);
     176    void computeSpaceUsedForOrigin(const ClientOrigin&);
     177    void finishComputingSpaceUsedForOrigin(const ClientOrigin&, uint64_t spaceUsed);
     178
     179    PAL::SessionID m_sessionID;
    130180    HashMap<uint64_t, RefPtr<IDBConnectionToClient>> m_connectionMap;
    131181    HashMap<IDBDatabaseIdentifier, std::unique_ptr<UniqueIDBDatabase>> m_uniqueIDBDatabaseMap;
    … …  
    144194    WTF::Function<void(bool)> m_isClosingDatabaseCallback;
    145195    PAL::HysteresisActivity m_isClosingDatabaseHysteresis;
     196
     197    HashMap<ClientOrigin, std::unique_ptr<QuotaUser>> m_quotaUsers;
     198    QuotaManagerGetter m_quotaManagerGetter;
    146199};
    147200

  • trunk/Source/WebCore/Modules/indexeddb/server/MemoryIDBBackingStore.cpp

    r238130 r242911  
    593593}
    594594
     595uint64_t MemoryIDBBackingStore::databasesSizeForOrigin() const
     596{
     597    // FIXME: Implement this.
     598    return 0;
     599}
     600
    595601} // namespace IDBServer
    596602} // namespace WebCore

  • trunk/Source/WebCore/Modules/indexeddb/server/MemoryIDBBackingStore.h

    r241183 r242911  
    8181
    8282    void setQuota(uint64_t quota) final { UNUSED_PARAM(quota); };
     83    uint64_t databasesSizeForOrigin() const final;
    8384
    8485    void removeObjectStoreForVersionChangeAbort(MemoryObjectStore&);

  • trunk/Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.cpp

    r240557 r242911  
    851851}
    852852
     853uint64_t SQLiteIDBBackingStore::databasesSizeForFolder(const String& folder)
     854{
     855    uint64_t diskUsage = 0;
     856    for (auto& directory : FileSystem::listDirectory(folder, "*")) {
     857        for (auto& file : FileSystem::listDirectory(directory, "*.sqlite3"_s))
     858            diskUsage += SQLiteFileSystem::getDatabaseFileSize(file);
     859    }
     860    return diskUsage;
     861}
     862
     863uint64_t SQLiteIDBBackingStore::databasesSizeForOrigin() const
     864{
     865    return databasesSizeForFolder(m_absoluteDatabaseDirectory);
     866}
     867
    853868uint64_t SQLiteIDBBackingStore::maximumSize() const
    854869{
    … …  
    860875    uint64_t quota = quotaForOrigin();
    861876
    862     uint64_t diskUsage = 0;
    863     for (auto& directory : FileSystem::listDirectory(m_absoluteDatabaseDirectory, "*")) {
    864         for (auto& file : FileSystem::listDirectory(directory, "*.sqlite3"_s))
    865             diskUsage += SQLiteFileSystem::getDatabaseFileSize(file);
    866     }
     877    uint64_t diskUsage = databasesSizeForOrigin();
    867878    ASSERT(diskUsage >= databaseFileSize);
    868879

  • trunk/Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.h

    r241183 r242911  
    8383
    8484    void setQuota(uint64_t quota) final { m_quota = quota; }
     85    uint64_t databasesSizeForOrigin() const final;
    8586
    8687    bool supportsSimultaneousTransactions() final { return false; }
    … …  
    9697
    9798    static String databaseNameFromEncodedFilename(const String&);
     99    static uint64_t databasesSizeForFolder(const String& folder);
    98100
    99101private:

  • trunk/Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp

    r242555 r242911  
    4242#include "Logging.h"
    4343#include "SerializedScriptValue.h"
     44#include "StorageQuotaManager.h"
    4445#include "UniqueIDBDatabaseConnection.h"
    4546#include <JavaScriptCore/AuxiliaryBarrierInlines.h>
    … …  
    5556namespace IDBServer {
    5657
     58static const uint64_t defaultWriteOperationCost = 4;
     59
     60static inline uint64_t estimateSize(const IDBKeyData& keyData)
     61{
     62    uint64_t size = 4;
     63    switch (keyData.type()) {
     64    case IndexedDB::KeyType::String:
     65        size += keyData.string().sizeInBytes();
     66        break;
     67    case IndexedDB::KeyType::Binary: {
     68        size += keyData.binary().size();
     69        break;
     70    }
     71    case IndexedDB::KeyType::Array:
     72        for (auto& data : keyData.array())
     73            size += estimateSize(data);
     74        break;
     75    default:
     76        break;
     77    }
     78    return size;
     79}
     80
     81static inline uint64_t estimateSize(const IDBValue& value)
     82{
     83    uint64_t size = 4;
     84    size += value.data().size();
     85    for (auto& url : value.blobURLs())
     86        size += url.sizeInBytes();
     87    for (auto& path : value.blobFilePaths())
     88        size += path.sizeInBytes();
     89    return size;
     90}
     91
     92static inline uint64_t estimateSize(const IDBIndexInfo& info)
     93{
     94    uint64_t size = 4;
     95    size += info.name().sizeInBytes();
     96    return size;
     97}
     98
     99static inline uint64_t estimateSize(const IDBObjectStoreInfo& info)
     100{
     101    uint64_t size = 4;
     102    size += info.name().sizeInBytes();
     103    // FIXME: estimate keyPath.
     104    for (auto& indexInfo : info.indexMap().values())
     105        size += estimateSize(indexInfo);
     106    return size;
     107}
     108
    57109UniqueIDBDatabase::UniqueIDBDatabase(IDBServer& server, const IDBDatabaseIdentifier& identifier)
    58     : m_server(&server)
     110    : m_server(server)
    59111    , m_identifier(identifier)
    60112    , m_operationAndTransactionTimer(*this, &UniqueIDBDatabase::operationAndTransactionTimerFired)
    … …  
    119171}
    120172
     173static inline String quotaErrorMessageName(const char* taskName)
     174{
     175    return makeString("Failed to ", taskName, " in database because not enough space for domain");
     176}
     177
     178void UniqueIDBDatabase::requestSpace(uint64_t taskSize, const char* taskName, CompletionHandler<void(Optional<IDBError>&&)>&& callback)
     179{
     180    m_server->requestSpace(m_identifier.origin(), taskSize, [weakThis = makeWeakPtr(this), taskName, callback = WTFMove(callback)](auto decision) mutable {
     181        if (!weakThis) {
     182            callback(IDBError { UnknownError });
     183            return;
     184        }
     185        switch (decision) {
     186        case StorageQuotaManager::Decision::Deny:
     187            callback(IDBError { QuotaExceededError, quotaErrorMessageName(taskName) });
     188            return;
     189        case StorageQuotaManager::Decision::Grant:
     190            callback({ });
     191        };
     192    });
     193}
     194
    121195void UniqueIDBDatabase::performCurrentOpenOperation()
    122196{
    … …  
    129203        if (!m_isOpeningBackingStore) {
    130204            m_isOpeningBackingStore = true;
    131             postDatabaseTask(createCrossThreadTask(*this, &UniqueIDBDatabase::openBackingStore, m_identifier));
    132         }
    133 
     205            // We do not know whether this is an existing or a new database.
     206            // We set a small cost so that it is not possible to open an infinite number of database.
     207            m_server->requestSpace(m_identifier.origin(), defaultWriteOperationCost, [this, weakThis = makeWeakPtr(this)](auto decision) mutable {
     208                if (!weakThis)
     209                    return;
     210
     211                switch (decision) {
     212                case StorageQuotaManager::Decision::Deny: {
     213                    auto result = IDBResultData::error(m_currentOpenDBRequest->requestData().requestIdentifier(), IDBError { QuotaExceededError, quotaErrorMessageName("openDatabase") });
     214                    m_currentOpenDBRequest->connection().didOpenDatabase(result);
     215                    m_currentOpenDBRequest = nullptr;
     216                    break;
     217                }
     218                case StorageQuotaManager::Decision::Grant:
     219                    this->postDatabaseTask(createCrossThreadTask(*this, &UniqueIDBDatabase::openBackingStore, m_identifier));
     220                };
     221            });
     222        }
    134223        return;
    135224    }
    … …  
    428517}
    429518
    430 uint64_t UniqueIDBDatabase::storeCallbackOrFireError(ErrorCallback&& callback)
     519uint64_t UniqueIDBDatabase::storeCallbackOrFireError(ErrorCallback&& callback, uint64_t taskSize)
    431520{
    432521    if (m_hardClosedForUserDelete) {
    … …  
    438527    ASSERT(!m_errorCallbacks.contains(identifier));
    439528    m_errorCallbacks.add(identifier, WTFMove(callback));
     529
     530    if (taskSize) {
     531        m_server->increasePotentialSpaceUsed(m_identifier.origin(), taskSize);
     532        m_pendingSpaceIncreasingTasks.add(identifier, taskSize);
     533    }
     534
    440535    m_callbackQueue.append(identifier);
    441536    return identifier;
    442537}
    443538
    444 uint64_t UniqueIDBDatabase::storeCallbackOrFireError(KeyDataCallback&& callback)
     539uint64_t UniqueIDBDatabase::storeCallbackOrFireError(KeyDataCallback&& callback, uint64_t taskSize)
    445540{
    446541    if (m_hardClosedForUserDelete) {
    … …  
    452547    ASSERT(!m_keyDataCallbacks.contains(identifier));
    453548    m_keyDataCallbacks.add(identifier, WTFMove(callback));
     549
     550    if (taskSize) {
     551        m_server->increasePotentialSpaceUsed(m_identifier.origin(), taskSize);
     552        m_pendingSpaceIncreasingTasks.add(identifier, taskSize);
     553    }
     554
    454555    m_callbackQueue.append(identifier);
    455556    return identifier;
    … …  
    669770    LOG(IndexedDB, "(main) UniqueIDBDatabase::createObjectStore");
    670771
    671     uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
     772    auto taskSize = defaultWriteOperationCost + estimateSize(info);
     773    requestSpace(taskSize, "createObjectStore", [this, taskSize, transaction = makeRef(transaction), info, callback = WTFMove(callback)](auto error) mutable {
     774        if (error) {
     775            callback(WTFMove(error.value()));
     776            return;
     777        }
     778        this->createObjectStoreAfterQuotaCheck(taskSize, transaction.get(), info, WTFMove(callback));
     779    });
     780}
     781
     782void UniqueIDBDatabase::createObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, const IDBObjectStoreInfo& info, ErrorCallback callback)
     783{
     784    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
    672785    if (!callbackID)
    673786        return;
    … …  
    745858    LOG(IndexedDB, "(main) UniqueIDBDatabase::renameObjectStore");
    746859
    747     uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
     860    auto taskSize = defaultWriteOperationCost + newName.sizeInBytes();
     861    requestSpace(taskSize, "renameObjectStore", [this, taskSize, transaction = makeRef(transaction), objectStoreIdentifier, newName, callback = WTFMove(callback)](auto error) mutable {
     862        if (error) {
     863            callback(WTFMove(error.value()));
     864            return;
     865        }
     866        this->renameObjectStoreAfterQuotaCheck(taskSize, transaction.get(), objectStoreIdentifier, newName, WTFMove(callback));
     867    });
     868}
     869
     870void UniqueIDBDatabase::renameObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, uint64_t objectStoreIdentifier, const String& newName, ErrorCallback callback)
     871{
     872    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
    748873    if (!callbackID)
    749874        return;
    … …  
    817942    LOG(IndexedDB, "(main) UniqueIDBDatabase::createIndex");
    818943
    819     uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
     944    auto taskSize = defaultWriteOperationCost + estimateSize(info);
     945    requestSpace(taskSize, "createIndex", [this, taskSize, transaction = makeRef(transaction), info, callback = WTFMove(callback)](auto error) mutable {
     946        if (error) {
     947            callback(WTFMove(error.value()));
     948            return;
     949        }
     950        this->createIndexAfterQuotaCheck(taskSize, transaction.get(), info, WTFMove(callback));
     951    });
     952}
     953
     954void UniqueIDBDatabase::createIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, const IDBIndexInfo& info, ErrorCallback callback)
     955{
     956    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
    820957    if (!callbackID)
    821958        return;
    … …  
    9041041    LOG(IndexedDB, "(main) UniqueIDBDatabase::renameIndex");
    9051042
    906     uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
     1043    auto taskSize = defaultWriteOperationCost + newName.sizeInBytes();
     1044    requestSpace(taskSize, "renameIndex", [this, taskSize, transaction = makeRef(transaction), objectStoreIdentifier, indexIdentifier, newName, callback = WTFMove(callback)](auto error) mutable {
     1045        if (error) {
     1046            callback(WTFMove(error.value()));
     1047            return;
     1048        }
     1049        this->renameIndexAfterQuotaCheck(taskSize, transaction.get(), objectStoreIdentifier, indexIdentifier, newName, WTFMove(callback));
     1050    });
     1051}
     1052
     1053void UniqueIDBDatabase::renameIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction& transaction, uint64_t objectStoreIdentifier, uint64_t indexIdentifier, const String& newName, ErrorCallback callback)
     1054{
     1055    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
    9071056    if (!callbackID)
    9081057        return;
    … …  
    9581107    LOG(IndexedDB, "(main) UniqueIDBDatabase::putOrAdd");
    9591108
    960     uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback));
     1109    auto taskSize = defaultWriteOperationCost + estimateSize(keyData) + estimateSize(value);
     1110    requestSpace(taskSize, "putOrAdd", [this, taskSize, requestData, keyData, value, callback = WTFMove(callback), overwriteMode](auto error) mutable {
     1111        if (error) {
     1112            callback(WTFMove(error.value()), { });
     1113            return;
     1114        }
     1115        this->putOrAddAfterQuotaCheck(taskSize, requestData, keyData, value, overwriteMode, WTFMove(callback));
     1116    });
     1117}
     1118
     1119void UniqueIDBDatabase::putOrAddAfterQuotaCheck(uint64_t taskSize, const IDBRequestData& requestData, const IDBKeyData& keyData, const IDBValue& value, IndexedDB::ObjectStoreOverwriteMode overwriteMode, KeyDataCallback callback)
     1120{
     1121    uint64_t callbackID = storeCallbackOrFireError(WTFMove(callback), taskSize);
    9611122    if (!callbackID)
    9621123        return;
    … …  
    17691930void UniqueIDBDatabase::postDatabaseTaskReply(CrossThreadTask&& task)
    17701931{
     1932    // FIXME: We might want to compute total size only for modification operations.
     1933    if (m_backingStore)
     1934        m_databasesSizeForOrigin = m_backingStore->databasesSizeForOrigin();
    17711935    m_databaseReplyQueue.append(WTFMove(task));
    17721936    m_server->postDatabaseTaskReply(createCrossThreadTask(*this, &UniqueIDBDatabase::executeNextDatabaseTaskReply));
    … …  
    18321996
    18331997    ASSERT(isMainThread());
     1998
     1999    m_pendingSpaceIncreasingTasks.clear();
     2000    m_server->resetSpaceUsed(m_identifier.origin());
    18342001
    18352002    // Error out all transactions
    … …  
    19032070}
    19042071
     2072void UniqueIDBDatabase::updateSpaceUsedIfNeeded(uint64_t callbackIdentifier)
     2073{
     2074    auto iterator = m_pendingSpaceIncreasingTasks.find(callbackIdentifier);
     2075    if (iterator == m_pendingSpaceIncreasingTasks.end())
     2076        return;
     2077
     2078    m_server->decreasePotentialSpaceUsed(m_identifier.origin(), iterator->value);
     2079    m_server->setSpaceUsed(m_identifier.origin(), m_databasesSizeForOrigin);
     2080    m_pendingSpaceIncreasingTasks.remove(iterator);
     2081}
     2082
    19052083void UniqueIDBDatabase::performErrorCallback(uint64_t callbackIdentifier, const IDBError& error)
    19062084{
     2085    updateSpaceUsedIfNeeded(callbackIdentifier);
     2086
    19072087    auto callback = m_errorCallbacks.take(callbackIdentifier);
    19082088    ASSERT(callback || m_hardClosedForUserDelete);
    … …  
    19162096void UniqueIDBDatabase::performKeyDataCallback(uint64_t callbackIdentifier, const IDBError& error, const IDBKeyData& resultKey)
    19172097{
     2098    updateSpaceUsedIfNeeded(callbackIdentifier);
     2099
    19182100    auto callback = m_keyDataCallbacks.take(callbackIdentifier);
    19192101    ASSERT(callback || m_hardClosedForUserDelete);
    … …  
    19602142void UniqueIDBDatabase::forgetErrorCallback(uint64_t callbackIdentifier)
    19612143{
     2144    updateSpaceUsedIfNeeded(callbackIdentifier);
     2145
    19622146    ASSERT(m_errorCallbacks.contains(callbackIdentifier));
    19632147    ASSERT(m_callbackQueue.last() == callbackIdentifier);

  • trunk/Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.h

    r242555 r242911  
    5555class IDBRequestData;
    5656class IDBTransactionInfo;
     57class StorageQuotaManager;
    5758
    5859enum class IDBGetRecordDataType;
    … …  
    8384
    8485    const IDBDatabaseInfo& info() const;
    85     IDBServer& server() { return *m_server; }
     86    IDBServer& server() { return m_server.get(); }
    8687    const IDBDatabaseIdentifier& identifier() const { return m_identifier; }
    8788
    … …  
    120121    bool hardClosedForUserDelete() const { return m_hardClosedForUserDelete; }
    121122
     123    uint64_t spaceUsed() const;
     124
    122125    void setQuota(uint64_t);
     126
    123127private:
    124128    enum class CloseState { Start, Done };
    … …  
    143147
    144148    void scheduleShutdownForClose();
     149
     150    void createObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, const IDBObjectStoreInfo&, ErrorCallback);
     151    void renameObjectStoreAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, uint64_t objectStoreIdentifier, const String& newName, ErrorCallback);
     152    void createIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, const IDBIndexInfo&, ErrorCallback);
     153    void renameIndexAfterQuotaCheck(uint64_t taskSize, UniqueIDBDatabaseTransaction&, uint64_t objectStoreIdentifier, uint64_t indexIdentifier, const String& newName, ErrorCallback);
     154    void putOrAddAfterQuotaCheck(uint64_t taskSize, const IDBRequestData&, const IDBKeyData&, const IDBValue&, IndexedDB::ObjectStoreOverwriteMode, KeyDataCallback);
    145155
    146156    // Database thread operations
    … …  
    194204    void didShutdownForClose();
    195205
    196     uint64_t storeCallbackOrFireError(ErrorCallback&&);
    197     uint64_t storeCallbackOrFireError(KeyDataCallback&&);
     206    uint64_t storeCallbackOrFireError(ErrorCallback&&, uint64_t taskSize = 0);
     207    uint64_t storeCallbackOrFireError(KeyDataCallback&&, uint64_t taskSize = 0);
    198208    uint64_t storeCallbackOrFireError(GetAllResultsCallback&&);
    199209    uint64_t storeCallbackOrFireError(GetResultCallback&&);
    … …  
    230240    void notifyServerAboutClose(CloseState);
    231241
    232     RefPtr<IDBServer> m_server;
     242    void requestSpace(uint64_t taskSize, const char* errorMessage, CompletionHandler<void(Optional<IDBError>&&)>&&);
     243    void updateSpaceUsedIfNeeded(uint64_t callbackIdentifier);
     244
     245    Ref<IDBServer> m_server;
    233246    IDBDatabaseIdentifier m_identifier;
    234    
     247
    235248    ListHashSet<RefPtr<ServerOpenDBRequest>> m_pendingOpenDBRequests;
    236249    RefPtr<ServerOpenDBRequest> m_currentOpenDBRequest;
    … …  
    280293
    281294    HashSet<IDBResourceIdentifier> m_cursorPrefetches;
     295
     296    HashMap<uint64_t, uint64_t> m_pendingSpaceIncreasingTasks;
     297    uint64_t m_databasesSizeForOrigin { 0 };
    282298};
    283299

  • trunk/Source/WebCore/Modules/indexeddb/shared/InProcessIDBServer.cpp

    r240437 r242911  
    4545namespace WebCore {
    4646
    47 Ref<InProcessIDBServer> InProcessIDBServer::create()
    48 {
    49     Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer);
     47Ref<InProcessIDBServer> InProcessIDBServer::create(PAL::SessionID sessionID)
     48{
     49    Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer(sessionID));
    5050    server->m_server->registerConnection(server->connectionToClient());
    5151    return server;
    5252}
    5353
    54 Ref<InProcessIDBServer> InProcessIDBServer::create(const String& databaseDirectoryPath)
    55 {
    56     Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer(databaseDirectoryPath));
     54Ref<InProcessIDBServer> InProcessIDBServer::create(PAL::SessionID sessionID, const String& databaseDirectoryPath)
     55{
     56    Ref<InProcessIDBServer> server = adoptRef(*new InProcessIDBServer(sessionID, databaseDirectoryPath));
    5757    server->m_server->registerConnection(server->connectionToClient());
    5858    return server;
    5959}
    6060
    61 InProcessIDBServer::InProcessIDBServer()
    62     : m_server(IDBServer::IDBServer::create(*this))
     61StorageQuotaManager* InProcessIDBServer::quotaManager(const ClientOrigin& origin)
     62{
     63    return m_quotaManagers.ensure(origin, [] {
     64        return std::make_unique<StorageQuotaManager>(StorageQuotaManager::defaultQuota(), [](uint64_t quota, uint64_t currentSpace, uint64_t spaceIncrease, auto callback) {
     65            callback(quota + currentSpace + spaceIncrease);
     66        });
     67    }).iterator->value.get();
     68}
     69
     70static inline IDBServer::IDBServer::QuotaManagerGetter storageQuotaManagerGetter(InProcessIDBServer& server)
     71{
     72    return [&server, weakServer = makeWeakPtr(server)](PAL::SessionID, const auto& origin) {
     73        return weakServer ? server.quotaManager(origin) : nullptr;
     74    };
     75}
     76
     77InProcessIDBServer::InProcessIDBServer(PAL::SessionID sessionID)
     78    : m_server(IDBServer::IDBServer::create(sessionID, *this, storageQuotaManagerGetter(*this)))
    6379{
    6480    relaxAdoptionRequirement();
    … …  
    6783}
    6884
    69 InProcessIDBServer::InProcessIDBServer(const String& databaseDirectoryPath)
    70     : m_server(IDBServer::IDBServer::create(databaseDirectoryPath, *this))
     85InProcessIDBServer::InProcessIDBServer(PAL::SessionID sessionID, const String& databaseDirectoryPath)
     86    : m_server(IDBServer::IDBServer::create(sessionID, databaseDirectoryPath, *this, storageQuotaManagerGetter(*this)))
    7187{
    7288    relaxAdoptionRequirement();

  • trunk/Source/WebCore/Modules/indexeddb/shared/InProcessIDBServer.h

    r238283 r242911  
    3131#include "IDBConnectionToServer.h"
    3232#include "IDBServer.h"
     33#include "StorageQuotaManager.h"
    3334#include <wtf/RefCounted.h>
    3435#include <wtf/RefPtr.h>
     36#include <wtf/WeakPtr.h>
     37
     38namespace PAL {
     39class SessionID;
     40}
    3541
    3642namespace WebCore {
     43
     44struct ClientOrigin;
    3745
    3846namespace IDBClient {
    … …  
    4654class InProcessIDBServer final : public IDBClient::IDBConnectionToServerDelegate, public IDBServer::IDBConnectionToClientDelegate, public RefCounted<InProcessIDBServer>, public IDBServer::IDBBackingStoreTemporaryFileHandler {
    4755public:
    48     WEBCORE_EXPORT static Ref<InProcessIDBServer> create();
    49     WEBCORE_EXPORT static Ref<InProcessIDBServer> create(const String& databaseDirectoryPath);
     56    WEBCORE_EXPORT static Ref<InProcessIDBServer> create(PAL::SessionID);
     57    WEBCORE_EXPORT static Ref<InProcessIDBServer> create(PAL::SessionID, const String& databaseDirectoryPath);
    5058
    5159    WEBCORE_EXPORT IDBClient::IDBConnectionToServer& connectionToServer() const;
    … …  
    115123    void accessToTemporaryFileComplete(const String& path) override;
    116124
     125    StorageQuotaManager* quotaManager(const ClientOrigin&);
     126
     127    const WeakPtrFactory<IDBClient::IDBConnectionToServerDelegate>& weakPtrFactory() const { return IDBClient::IDBConnectionToServerDelegate::weakPtrFactory(); }
     128
    117129private:
    118     InProcessIDBServer();
    119     InProcessIDBServer(const String& databaseDirectoryPath);
     130    explicit InProcessIDBServer(PAL::SessionID);
     131    InProcessIDBServer(PAL::SessionID, const String& databaseDirectoryPath);
    120132
    121133    Ref<IDBServer::IDBServer> m_server;
    122134    RefPtr<IDBClient::IDBConnectionToServer> m_connectionToServer;
    123135    RefPtr<IDBServer::IDBConnectionToClient> m_connectionToClient;
     136
     137    HashMap<ClientOrigin, std::unique_ptr<StorageQuotaManager>> m_quotaManagers;
    124138};
    125139

  • trunk/Source/WebCore/loader/EmptyClients.cpp

    r242681 r242911  
    119119class EmptyDatabaseProvider final : public DatabaseProvider {
    120120#if ENABLE(INDEXED_DATABASE)
    121     IDBClient::IDBConnectionToServer& idbConnectionToServerForSession(const PAL::SessionID&) final
     121    IDBClient::IDBConnectionToServer& idbConnectionToServerForSession(const PAL::SessionID& sessionID) final
    122122    {
    123         static auto& sharedConnection = InProcessIDBServer::create().leakRef();
     123        static auto& sharedConnection = InProcessIDBServer::create(sessionID).leakRef();
    124124        return sharedConnection.connectionToServer();
    125125    }

  • trunk/Source/WebCore/storage/StorageQuotaManager.cpp

    r242599 r242911  
    4545}
    4646
     47void StorageQuotaManager::addUser(StorageQuotaUser& user)
     48{
     49    ASSERT(!m_pendingInitializationUsers.contains(&user));
     50    ASSERT(!m_users.contains(&user));
     51    m_pendingInitializationUsers.add(&user);
     52    user.whenInitialized([this, &user, weakThis = makeWeakPtr(this)]() {
     53        if (!weakThis)
     54            return;
     55        m_pendingInitializationUsers.remove(&user);
     56        m_users.add(&user);
     57        processPendingRequests({ });
     58    });
     59}
     60
    4761void StorageQuotaManager::requestSpace(uint64_t spaceIncrease, RequestCallback&& callback)
    4862{
    49     if (!m_pendingRequests.isEmpty()) {
     63    if (!m_pendingRequests.isEmpty() || !m_pendingInitializationUsers.isEmpty()) {
    5064        m_pendingRequests.append({ spaceIncrease, WTFMove(callback) });
    5165        return;

  • trunk/Source/WebCore/storage/StorageQuotaManager.h

    r242599 r242911  
    4747    WEBCORE_EXPORT ~StorageQuotaManager();
    4848
    49     void addUser(StorageQuotaUser& user)
    50     {
    51         ASSERT(!m_users.contains(&user));
    52         m_users.add(&user);
    53     }
     49    static constexpr uint64_t defaultQuota() { return 500 * MB; }
    5450
     51    WEBCORE_EXPORT void addUser(StorageQuotaUser&);
    5552    void removeUser(StorageQuotaUser& user)
    5653    {
    57         ASSERT(m_users.contains(&user));
     54        ASSERT(m_users.contains(&user) || m_pendingInitializationUsers.contains(&user));
     55        m_pendingInitializationUsers.remove(&user);
    5856        m_users.remove(&user);
    5957    }
    … …  
    7068    uint64_t m_quota { 0 };
    7169    SpaceIncreaseRequester m_spaceIncreaseRequester;
     70    HashSet<const StorageQuotaUser*> m_pendingInitializationUsers;
    7271    HashSet<const StorageQuotaUser*> m_users;
    7372

  • trunk/Source/WebCore/storage/StorageQuotaUser.h

    r242599 r242911  
    2626#pragma once
    2727
     28#include <wtf/CompletionHandler.h>
     29
    2830namespace WebCore {
    2931
    … …  
    3335
    3436    virtual uint64_t spaceUsed() const = 0;
     37    virtual void whenInitialized(CompletionHandler<void()>&& callback) { callback(); }
    3538};
    3639

  • trunk/Source/WebKit/ChangeLog

    r242908 r242911  
     12019-03-13  Youenn Fablet  <youenn@apple.com>
     2
     3        Check IDB quota usage through QuotaManager
     4        https://bugs.webkit.org/show_bug.cgi?id=195302
     5
     6        Reviewed by Chris Dumez.
     7
     8        Set the quota manager getter for IDBServer at creation time.
     9
     10        * NetworkProcess/NetworkProcess.cpp:
     11        (WebKit::NetworkProcess::createIDBServer):
     12        (WebKit::NetworkProcess::idbServer):
     13        * NetworkProcess/NetworkProcess.h:
     14        * WebProcess/Databases/WebDatabaseProvider.cpp:
     15        (WebKit::WebDatabaseProvider::idbConnectionToServerForSession):
     16
    1172019-03-13  Timothy Hatcher  <timothy@apple.com>
    218

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp

    r242905 r242911  
    20362036
    20372037#if ENABLE(INDEXED_DATABASE)
    2038 IDBServer::IDBServer& NetworkProcess::idbServer(PAL::SessionID sessionID)
    2039 {
    2040     auto addResult = m_idbServers.add(sessionID, nullptr);
    2041     if (!addResult.isNewEntry) {
    2042         ASSERT(addResult.iterator->value);
    2043         return *addResult.iterator->value;
    2044     }
    2045    
     2038Ref<IDBServer::IDBServer> NetworkProcess::createIDBServer(PAL::SessionID sessionID)
     2039{
    20462040    auto path = m_idbDatabasePaths.get(sessionID);
    20472041    // There should already be a registered path for this PAL::SessionID.
    20482042    // If there's not, then where did this PAL::SessionID come from?
    20492043    ASSERT(!path.isEmpty());
    2050    
    2051     addResult.iterator->value = IDBServer::IDBServer::create(path, *this, [this](bool isHoldingLockedFiles) {
    2052         notifyHoldingLockedFiles(isHoldingLockedFiles);
     2044
     2045    auto server = IDBServer::IDBServer::create(sessionID, path, *this, [this, weakThis = makeWeakPtr(this)](PAL::SessionID sessionID, const auto& origin) -> StorageQuotaManager* {
     2046        if (!weakThis)
     2047            return nullptr;
     2048        return &this->storageQuotaManager(sessionID, origin);
     2049    }, [this, weakThis = makeWeakPtr(this)](bool isHoldingLockedFiles) {
     2050        if (!weakThis)
     2051            return;
     2052        this->notifyHoldingLockedFiles(isHoldingLockedFiles);
    20532053    });
    2054     addResult.iterator->value->setPerOriginQuota(m_idbPerOriginQuota);
    2055     return *addResult.iterator->value;
     2054    server->setPerOriginQuota(m_idbPerOriginQuota);
     2055    return server;
     2056}
     2057
     2058IDBServer::IDBServer& NetworkProcess::idbServer(PAL::SessionID sessionID)
     2059{
     2060    return *m_idbServers.ensure(sessionID, [this, sessionID] {
     2061        return this->createIDBServer(sessionID);
     2062    }).iterator->value;
    20562063}
    20572064

  • trunk/Source/WebKit/NetworkProcess/NetworkProcess.h

    r242905 r242911  
    431431    void addIndexedDatabaseSession(PAL::SessionID, String&, SandboxExtension::Handle&);
    432432    HashSet<WebCore::SecurityOriginData> indexedDatabaseOrigins(const String& path);
     433    Ref<WebCore::IDBServer::IDBServer> createIDBServer(PAL::SessionID);
    433434#endif
    434435
    … …  
    525526
    526527    struct StorageQuotaManagers {
    527         uint64_t defaultQuota { 0 };
     528        uint64_t defaultQuota { WebCore::StorageQuotaManager::defaultQuota() };
    528529        HashMap<WebCore::ClientOrigin, std::unique_ptr<WebCore::StorageQuotaManager>> managersPerOrigin;
    529530    };

  • trunk/Source/WebKit/WebProcess/Databases/WebDatabaseProvider.cpp

    r236035 r242911  
    7373        auto result = m_idbEphemeralConnectionMap.add(sessionID.sessionID(), nullptr);
    7474        if (result.isNewEntry)
    75             result.iterator->value = WebCore::InProcessIDBServer::create();
     75            result.iterator->value = WebCore::InProcessIDBServer::create(sessionID);
    7676
    7777        return result.iterator->value->connectionToServer();

  • trunk/Source/WebKitLegacy/ChangeLog

    r242581 r242911  
     12019-03-13  Youenn Fablet  <youenn@apple.com>
     2
     3        Check IDB quota usage through QuotaManager
     4        https://bugs.webkit.org/show_bug.cgi?id=195302
     5
     6        Reviewed by Chris Dumez.
     7
     8        * Storage/WebDatabaseProvider.cpp:
     9        (WebDatabaseProvider::idbConnectionToServerForSession):
     10
    1112019-03-06  Sam Weinig  <sam@webkit.org>
    212

  • trunk/Source/WebKitLegacy/Storage/WebDatabaseProvider.cpp

    r237700 r242911  
    4949    if (result.isNewEntry) {
    5050        if (sessionID.isEphemeral())
    51             result.iterator->value = WebCore::InProcessIDBServer::create();
     51            result.iterator->value = WebCore::InProcessIDBServer::create(sessionID);
    5252        else
    53             result.iterator->value = WebCore::InProcessIDBServer::create(indexedDatabaseDirectoryPath());
     53            result.iterator->value = WebCore::InProcessIDBServer::create(sessionID, indexedDatabaseDirectoryPath());
    5454    }
    5555
Note: See TracChangeset for help on using the changeset viewer.