⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 276653 in webkit


Ignore:
Timestamp:
Apr 27, 2021, 12:02:53 PM (5 years ago)
Author:
Chris Dumez
Message:

Don't keep local storage data in memory in the NetworkProcess
​https://bugs.webkit.org/show_bug.cgi?id=225065

Reviewed by Alex Christensen.

Source/WebCore:

  • platform/sql/SQLiteDatabase.h:
  • platform/sql/SQLiteFileSystem.h:

Export a couple more symbols.

  • storage/StorageMap.cpp:

(WebCore::StorageMap::clear):

  • storage/StorageMap.h:

Add a clear() function to StorageMap so that StorageArea doesn't reconstruct a
new StorageMap object unnecessarily when:

  • The StorageMap is not shared
  • The StorageMap is empty (and clear() is a no-op)

Source/WebKit:

When a WebPage would start to use the local storage API, the WebProcess
would send a sync IPC to the NetworkProcess to retrieve all the local
storage entries for the origin. The NetworkProcess would read those
entries from a SQLite database and send them back to the WebProcess.
Both the NetworkProcess would keep the entries in memory, in a
StorageMap object (which is basically a HashMap). On some sites, the
strings in the local storage may be very large. It is useful for the
WebProcess to keep them in memory for performance reasons, especially
considering that the Web API is synchronous. However, there is no real
need to keep them in memory in the Network Process side, especially
given that the WebProcess already has its own copy. On the network
process side, we can get rid of the StorageMap entirely in the local
storage case and 100% rely on the SQLite database. This is what this
patch implements.

A/B testing shows this is a 2-3% progression on Membuster. There does
not appear to be a meaningful progression on PLUM sadly. This is also
neutral on PLT.

  • NetworkProcess/WebStorage/LocalStorageDatabase.cpp:

(WebKit::estimateEntrySize):
(WebKit::LocalStorageDatabase::create):
(WebKit::LocalStorageDatabase::LocalStorageDatabase):
(WebKit::LocalStorageDatabase::openDatabase):
(WebKit::LocalStorageDatabase::tryToOpenDatabase):
(WebKit::LocalStorageDatabase::items const):
(WebKit::LocalStorageDatabase::removeItem):
(WebKit::LocalStorageDatabase::item const):
(WebKit::LocalStorageDatabase::setItem):
(WebKit::LocalStorageDatabase::clear):
(WebKit::LocalStorageDatabase::close):
(WebKit::LocalStorageDatabase::databaseIsEmpty const):
(WebKit::LocalStorageDatabase::openIfExisting):
(WebKit::LocalStorageDatabase::scopedStatement const):

  • NetworkProcess/WebStorage/LocalStorageDatabase.h:
  • Update LocalStorageDatabase API to match more closely the API of StorageMap. This way StorageArea can rely on the LocalStorageDatabase entirely instead of the StorageMap with minimal changes.
  • Add quota support similarly to what is done in StorageArea so that LocalStorageDatabase::setItem() fails when the quota is reached.
  • Cache the SQLiteStatements for performance reasons
  • NetworkProcess/WebStorage/LocalStorageNamespace.cpp:

(WebKit::LocalStorageNamespace::flushAndClose):
We no longer need to call syncToDatabase() before closing since
LocalStorageDatabase no longer has a queue on pending operations.

  • NetworkProcess/WebStorage/StorageArea.cpp:

(WebKit::StorageArea::StorageArea):
(WebKit::StorageArea::addListener):
(WebKit::StorageArea::removeListener):
(WebKit::StorageArea::setItem):
(WebKit::StorageArea::removeItem):
(WebKit::StorageArea::clear):
(WebKit::StorageArea::items const):
(WebKit::StorageArea::ensureDatabase const):

  • NetworkProcess/WebStorage/StorageArea.h:

If the StorageArea is used for local storage, we now don't even initialize
a StorageMap and rely entirely on LocalStorageDatabase. If the StorageArea
is used for session storage, we keep using a StorageMap since there is no
backing database. This is fairly straightforward because I updated the
LocalStorageDatabase API to match fairly closely the StorageMap one.

  • NetworkProcess/WebStorage/StorageManagerSet.cpp:

(WebKit::StorageManagerSet::waitUntilSyncingLocalStorageFinished):

Source/WebKitLegacy:

  • Storage/StorageAreaImpl.cpp:

(WebKit::StorageAreaImpl::clear):
(WebKit::StorageAreaImpl::clearForOriginDeletion):
Use the new StorageMap::clear() to avoid constructing a new StorageMap unnecessarily
when:

  • The StorageMap is not shared
  • The StorageMap is empty
Location:
trunk/Source
Files:
14 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r276649 r276653  
     12021-04-27  Chris Dumez  <cdumez@apple.com>
     2
     3        Don't keep local storage data in memory in the NetworkProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=225065
     5
     6        Reviewed by Alex Christensen.
     7
     8        * platform/sql/SQLiteDatabase.h:
     9        * platform/sql/SQLiteFileSystem.h:
     10        Export a couple more symbols.
     11
     12        * storage/StorageMap.cpp:
     13        (WebCore::StorageMap::clear):
     14        * storage/StorageMap.h:
     15        Add a clear() function to StorageMap so that StorageArea doesn't reconstruct a
     16        new StorageMap object unnecessarily when:
     17        - The StorageMap is not shared
     18        - The StorageMap is empty (and clear() is a no-op)
     19
    1202021-04-27  Youenn Fablet  <youenn@apple.com>
    221

  • trunk/Source/WebCore/platform/sql/SQLiteDatabase.h

    r276648 r276653  
    5959    WEBCORE_EXPORT void close();
    6060
    61     void updateLastChangesCount();
     61    WEBCORE_EXPORT void updateLastChangesCount();
    6262
    6363    WEBCORE_EXPORT bool executeCommand(const String&);
    … …  
    7575
    7676    int64_t lastInsertRowID();
    77     int lastChanges();
     77    WEBCORE_EXPORT int lastChanges();
    7878
    7979    void setBusyTimeout(int ms);

  • trunk/Source/WebCore/platform/sql/SQLiteFileSystem.h

    r243019 r276653  
    9090#endif
    9191   
    92     static long long getDatabaseFileSize(const String& fileName);
     92    WEBCORE_EXPORT static long long getDatabaseFileSize(const String& fileName);
    9393    WEBCORE_EXPORT static Optional<WallTime> databaseCreationTime(const String& fileName);
    9494    WEBCORE_EXPORT static Optional<WallTime> databaseModificationTime(const String& fileName);

  • trunk/Source/WebCore/storage/StorageMap.cpp

    r254859 r276653  
    176176}
    177177
     178RefPtr<StorageMap> StorageMap::clear()
     179{
     180    // Implement copy-on-write semantics here. We're guaranteed that the only refs of StorageMaps belong to Storage objects
     181    // so if more than one Storage object refs this map, copy it before mutating it.
     182    if (refCount() > 1 && !m_map.isEmpty())
     183        return StorageMap::create(m_quotaSize);
     184
     185    m_map.clear();
     186    m_currentLength = 0;
     187    return nullptr;
     188}
     189
    178190bool StorageMap::contains(const String& key) const
    179191{

  • trunk/Source/WebCore/storage/StorageMap.h

    r254859 r276653  
    4444    WEBCORE_EXPORT RefPtr<StorageMap> setItemIgnoringQuota(const String& key, const String& value);
    4545    WEBCORE_EXPORT RefPtr<StorageMap> removeItem(const String&, String& oldValue);
     46    WEBCORE_EXPORT RefPtr<StorageMap> clear();
    4647
    4748    WEBCORE_EXPORT bool contains(const String& key) const;

  • trunk/Source/WebKit/ChangeLog

    r276651 r276653  
     12021-04-27  Chris Dumez  <cdumez@apple.com>
     2
     3        Don't keep local storage data in memory in the NetworkProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=225065
     5
     6        Reviewed by Alex Christensen.
     7
     8        When a WebPage would start to use the local storage API, the WebProcess
     9        would send a sync IPC to the NetworkProcess to retrieve all the local
     10        storage entries for the origin. The NetworkProcess would read those
     11        entries from a SQLite database and send them back to the WebProcess.
     12        Both the NetworkProcess would keep the entries in memory, in a
     13        StorageMap object (which is basically a HashMap). On some sites, the
     14        strings in the local storage may be very large. It is useful for the
     15        WebProcess to keep them in memory for performance reasons, especially
     16        considering that the Web API is synchronous. However, there is no real
     17        need to keep them in memory in the Network Process side, especially
     18        given that the WebProcess already has its own copy. On the network
     19        process side, we can get rid of the StorageMap entirely in the local
     20        storage case and 100% rely on the SQLite database. This is what this
     21        patch implements.
     22
     23        A/B testing shows this is a 2-3% progression on Membuster. There does
     24        not appear to be a meaningful progression on PLUM sadly. This is also
     25        neutral on PLT.
     26
     27        * NetworkProcess/WebStorage/LocalStorageDatabase.cpp:
     28        (WebKit::estimateEntrySize):
     29        (WebKit::LocalStorageDatabase::create):
     30        (WebKit::LocalStorageDatabase::LocalStorageDatabase):
     31        (WebKit::LocalStorageDatabase::openDatabase):
     32        (WebKit::LocalStorageDatabase::tryToOpenDatabase):
     33        (WebKit::LocalStorageDatabase::items const):
     34        (WebKit::LocalStorageDatabase::removeItem):
     35        (WebKit::LocalStorageDatabase::item const):
     36        (WebKit::LocalStorageDatabase::setItem):
     37        (WebKit::LocalStorageDatabase::clear):
     38        (WebKit::LocalStorageDatabase::close):
     39        (WebKit::LocalStorageDatabase::databaseIsEmpty const):
     40        (WebKit::LocalStorageDatabase::openIfExisting):
     41        (WebKit::LocalStorageDatabase::scopedStatement const):
     42        * NetworkProcess/WebStorage/LocalStorageDatabase.h:
     43        - Update LocalStorageDatabase API to match more closely the API of
     44          StorageMap. This way StorageArea can rely on the LocalStorageDatabase
     45          entirely instead of the StorageMap with minimal changes.
     46        - Add quota support similarly to what is done in StorageArea so that
     47          LocalStorageDatabase::setItem() fails when the quota is reached.
     48        - Cache the SQLiteStatements for performance reasons
     49
     50        * NetworkProcess/WebStorage/LocalStorageNamespace.cpp:
     51        (WebKit::LocalStorageNamespace::flushAndClose):
     52        We no longer need to call syncToDatabase() before closing since
     53        LocalStorageDatabase no longer has a queue on pending operations.
     54
     55        * NetworkProcess/WebStorage/StorageArea.cpp:
     56        (WebKit::StorageArea::StorageArea):
     57        (WebKit::StorageArea::addListener):
     58        (WebKit::StorageArea::removeListener):
     59        (WebKit::StorageArea::setItem):
     60        (WebKit::StorageArea::removeItem):
     61        (WebKit::StorageArea::clear):
     62        (WebKit::StorageArea::items const):
     63        (WebKit::StorageArea::ensureDatabase const):
     64        * NetworkProcess/WebStorage/StorageArea.h:
     65        If the StorageArea is used for local storage, we now don't even initialize
     66        a StorageMap and rely entirely on LocalStorageDatabase. If the StorageArea
     67        is used for session storage, we keep using a StorageMap since there is no
     68        backing database. This is fairly straightforward because I updated the
     69        LocalStorageDatabase API to match fairly closely the StorageMap one.
     70
     71        * NetworkProcess/WebStorage/StorageManagerSet.cpp:
     72        (WebKit::StorageManagerSet::waitUntilSyncingLocalStorageFinished):
     73
    1742021-04-27  Chris Dumez  <cdumez@apple.com>
    275

  • trunk/Source/WebKit/NetworkProcess/WebStorage/LocalStorageDatabase.cpp

    r276648 r276653  
    11/*
    2  * Copyright (C) 2008, 2009, 2010, 2013, 2019 Apple Inc. All rights reserved.
     2 * Copyright (C) 2008-2021 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2828
    2929#include "LocalStorageDatabaseTracker.h"
     30#include <WebCore/SQLiteFileSystem.h>
    3031#include <WebCore/SQLiteStatement.h>
     32#include <WebCore/SQLiteStatementAutoResetScope.h>
    3133#include <WebCore/SQLiteTransaction.h>
    32 #include <WebCore/SecurityOrigin.h>
     34#include <WebCore/SecurityOriginData.h>
    3335#include <WebCore/StorageMap.h>
    3436#include <WebCore/SuddenTermination.h>
    3537#include <wtf/FileSystem.h>
     38#include <wtf/HashMap.h>
    3639#include <wtf/RefPtr.h>
    3740#include <wtf/RunLoop.h>
    … …  
    4043#include <wtf/text/WTFString.h>
    4144
    42 static const auto databaseUpdateInterval = 1_s;
    43 
    44 static const int maximumItemsToUpdate = 100;
    45 
    4645namespace WebKit {
    4746using namespace WebCore;
    4847
    49 Ref<LocalStorageDatabase> LocalStorageDatabase::create(Ref<WorkQueue>&& queue, Ref<LocalStorageDatabaseTracker>&& tracker, const SecurityOriginData& securityOrigin)
    50 {
    51     return adoptRef(*new LocalStorageDatabase(WTFMove(queue), WTFMove(tracker), securityOrigin));
    52 }
    53 
    54 LocalStorageDatabase::LocalStorageDatabase(Ref<WorkQueue>&& queue, Ref<LocalStorageDatabaseTracker>&& tracker, const SecurityOriginData& securityOrigin)
     48static const char getItemsQueryString[] = "SELECT key, value FROM ItemTable";
     49
     50static CheckedUint64 estimateEntrySize(const String& key, const String& value)
     51{
     52    CheckedUint64 entrySize;
     53    entrySize += key.length() * sizeof(UChar);
     54    entrySize += value.length() * sizeof(UChar);
     55    return entrySize;
     56}
     57
     58Ref<LocalStorageDatabase> LocalStorageDatabase::create(Ref<WorkQueue>&& queue, Ref<LocalStorageDatabaseTracker>&& tracker, const SecurityOriginData& securityOrigin, unsigned quotaInBytes)
     59{
     60    return adoptRef(*new LocalStorageDatabase(WTFMove(queue), WTFMove(tracker), securityOrigin, quotaInBytes));
     61}
     62
     63LocalStorageDatabase::LocalStorageDatabase(Ref<WorkQueue>&& queue, Ref<LocalStorageDatabaseTracker>&& tracker, const SecurityOriginData& securityOrigin, unsigned quotaInBytes)
    5564    : m_queue(WTFMove(queue))
    5665    , m_tracker(WTFMove(tracker))
    5766    , m_securityOrigin(securityOrigin)
    5867    , m_databasePath(m_tracker->databasePath(m_securityOrigin))
     68    , m_quotaInBytes(quotaInBytes)
    5969{
    6070    ASSERT(!RunLoop::isMain());
    … …  
    6777}
    6878
    69 void LocalStorageDatabase::openDatabase(DatabaseOpeningStrategy openingStrategy)
    70 {
     79void LocalStorageDatabase::openDatabase(ShouldCreateDatabase shouldCreateDatabase)
     80{
     81    ASSERT(!RunLoop::isMain());
    7182    ASSERT(!m_database.isOpen());
    7283    ASSERT(!m_failedToOpenDatabase);
    7384
    74     if (!tryToOpenDatabase(openingStrategy)) {
     85    if (!tryToOpenDatabase(shouldCreateDatabase)) {
    7586        m_failedToOpenDatabase = true;
    7687        return;
    … …  
    8192}
    8293
    83 bool LocalStorageDatabase::tryToOpenDatabase(DatabaseOpeningStrategy openingStrategy)
    84 {
    85     ASSERT(!RunLoop::isMain());
    86     if (!FileSystem::fileExists(m_databasePath) && openingStrategy == SkipIfNonExistent)
     94bool LocalStorageDatabase::tryToOpenDatabase(ShouldCreateDatabase shouldCreateDatabase)
     95{
     96    ASSERT(!RunLoop::isMain());
     97    if (!FileSystem::fileExists(m_databasePath) && shouldCreateDatabase == ShouldCreateDatabase::No)
    8798        return true;
    8899
    … …  
    118129bool LocalStorageDatabase::migrateItemTableIfNeeded()
    119130{
     131    ASSERT(!RunLoop::isMain());
    120132    if (!m_database.tableExists("ItemTable"))
    121133        return true;
    … …  
    154166}
    155167
    156 void LocalStorageDatabase::importItems(StorageMap& storageMap)
    157 {
    158     if (m_didImportItems)
    159         return;
    160 
    161     // FIXME: If it can't import, then the default WebKit behavior should be that of private browsing,
    162     // not silently ignoring it. https://bugs.webkit.org/show_bug.cgi?id=25894
    163 
    164     // We set this to true even if we don't end up importing any items due to failure because
    165     // there's really no good way to recover other than not importing anything.
    166     m_didImportItems = true;
    167 
    168     openDatabase(SkipIfNonExistent);
    169     if (!m_database.isOpen())
    170         return;
    171 
    172     SQLiteStatement query(m_database, "SELECT key, value FROM ItemTable"_str);
    173     if (query.prepare() != SQLITE_OK) {
     168HashMap<String, String> LocalStorageDatabase::items() const
     169{
     170    ASSERT(!RunLoop::isMain());
     171    if (!m_database.isOpen())
     172        return { };
     173
     174    auto query = scopedStatement(m_getItemsStatement, getItemsQueryString);
     175    if (!query) {
    174176        LOG_ERROR("Unable to select items from ItemTable for local storage");
    175         return;
     177        return { };
    176178    }
    177179
    178180    HashMap<String, String> items;
    179 
    180     int result = query.step();
     181    int result = query->step();
    181182    while (result == SQLITE_ROW) {
    182         String key = query.getColumnText(0);
    183         String value = query.getColumnBlobAsString(1);
     183        String key = query->getColumnText(0);
     184        String value = query->getColumnBlobAsString(1);
    184185        if (!key.isNull() && !value.isNull())
    185186            items.add(WTFMove(key), WTFMove(value));
    186         result = query.step();
    187     }
    188 
     187        result = query->step();
     188    }
     189
     190    if (result != SQLITE_DONE)
     191        LOG_ERROR("Error reading items from ItemTable for local storage");
     192
     193    return items;
     194}
     195
     196void LocalStorageDatabase::removeItem(const String& key, String& oldValue)
     197{
     198    ASSERT(!RunLoop::isMain());
     199    if (!m_database.isOpen())
     200        return;
     201
     202    oldValue = item(key);
     203    if (oldValue.isNull())
     204        return;
     205
     206    auto deleteStatement = scopedStatement(m_deleteItemStatement, "DELETE FROM ItemTable WHERE key=?"_s);
     207    if (!deleteStatement) {
     208        LOG_ERROR("Failed to prepare delete statement - cannot write to local storage database");
     209        return;
     210    }
     211    deleteStatement->bindText(1, key);
     212
     213    int result = deleteStatement->step();
    189214    if (result != SQLITE_DONE) {
    190         LOG_ERROR("Error reading items from ItemTable for local storage");
    191         return;
    192     }
    193 
    194     storageMap.importItems(WTFMove(items));
    195 }
    196 
    197 void LocalStorageDatabase::setItem(const String& key, const String& value)
    198 {
    199     itemDidChange(key, value);
    200 }
    201 
    202 void LocalStorageDatabase::removeItem(const String& key)
    203 {
    204     itemDidChange(key, String());
    205 }
    206 
    207 void LocalStorageDatabase::clear()
    208 {
    209     m_changedItems.clear();
    210     m_shouldClearItems = true;
    211 
    212     scheduleDatabaseUpdate();
     215        LOG_ERROR("Failed to delete item in the local storage database - %i", result);
     216        return;
     217    }
     218
     219    if (m_databaseSize) {
     220        CheckedUint64 entrySize = estimateEntrySize(key, oldValue);
     221        if (entrySize.hasOverflowed() || entrySize.unsafeGet() >= *m_databaseSize)
     222            *m_databaseSize = 0;
     223        else
     224            *m_databaseSize -= entrySize.unsafeGet();
     225    }
     226}
     227
     228String LocalStorageDatabase::item(const String& key) const
     229{
     230    ASSERT(!RunLoop::isMain());
     231    if (!m_database.isOpen())
     232        return { };
     233
     234    auto query = scopedStatement(m_getItemStatement, "SELECT value FROM ItemTable WHERE key=?"_s);
     235    if (!query) {
     236        LOG_ERROR("Unable to get item from ItemTable for local storage");
     237        return { };
     238    }
     239    query->bindText(1, key);
     240
     241    int result = query->step();
     242    if (result == SQLITE_ROW)
     243        return query->getColumnBlobAsString(0);
     244    if (result != SQLITE_DONE)
     245        LOG_ERROR("Error get item from ItemTable for local storage");
     246    return { };
     247}
     248
     249void LocalStorageDatabase::setItem(const String& key, const String& value, String& oldValue, bool& quotaException)
     250{
     251    ASSERT(!RunLoop::isMain());
     252    if (!m_database.isOpen())
     253        openDatabase(ShouldCreateDatabase::Yes);
     254    if (!m_database.isOpen())
     255        return;
     256
     257    if (m_quotaInBytes != WebCore::StorageMap::noQuota) {
     258        if (!m_databaseSize)
     259            m_databaseSize = SQLiteFileSystem::getDatabaseFileSize(m_databasePath);
     260        if (*m_databaseSize >= m_quotaInBytes) {
     261            quotaException = true;
     262            return;
     263        }
     264        CheckedUint64 newDatabaseSize = *m_databaseSize;
     265        newDatabaseSize += estimateEntrySize(key, value);
     266        if (newDatabaseSize.hasOverflowed() || newDatabaseSize.unsafeGet() > m_quotaInBytes) {
     267            quotaException = true;
     268            return;
     269        }
     270        m_databaseSize = newDatabaseSize.unsafeGet();
     271    }
     272
     273    oldValue = item(key);
     274
     275    auto insertStatement = scopedStatement(m_insertStatement, "INSERT INTO ItemTable VALUES (?, ?)"_s);
     276    if (!insertStatement) {
     277        LOG_ERROR("Failed to prepare insert statement - cannot write to local storage database");
     278        return;
     279    }
     280
     281    insertStatement->bindText(1, key);
     282    insertStatement->bindBlob(2, value);
     283
     284    int result = insertStatement->step();
     285    if (result != SQLITE_DONE)
     286        LOG_ERROR("Failed to update item in the local storage database - %i", result);
     287}
     288
     289bool LocalStorageDatabase::clear()
     290{
     291    ASSERT(!RunLoop::isMain());
     292    if (!m_database.isOpen())
     293        return false;
     294
     295    auto clearStatement = scopedStatement(m_clearStatement, "DELETE FROM ItemTable"_s);
     296    if (!clearStatement) {
     297        LOG_ERROR("Failed to prepare clear statement - cannot write to local storage database");
     298        return false;
     299    }
     300
     301    int result = clearStatement->step();
     302    if (result != SQLITE_DONE) {
     303        LOG_ERROR("Failed to clear all items in the local storage database - %i", result);
     304        return false;
     305    }
     306
     307    m_databaseSize = 0;
     308
     309    return m_database.lastChanges() > 0;
    213310}
    214311
    215312void LocalStorageDatabase::close()
    216313{
     314    ASSERT(!RunLoop::isMain());
    217315    if (m_isClosed)
    218316        return;
    219317    m_isClosed = true;
    220318
    221     if (m_didScheduleDatabaseUpdate) {
    222         updateDatabaseWithChangedItems(m_changedItems);
    223         m_changedItems.clear();
    224     }
    225 
    226319    bool isEmpty = databaseIsEmpty();
     320
     321    m_clearStatement = nullptr;
     322    m_insertStatement = nullptr;
     323    m_getItemStatement = nullptr;
     324    m_getItemsStatement = nullptr;
     325    m_deleteItemStatement = nullptr;
    227326
    228327    if (m_database.isOpen())
    … …  
    233332}
    234333
    235 void LocalStorageDatabase::itemDidChange(const String& key, const String& value)
    236 {
    237     m_changedItems.set(key, value);
    238     scheduleDatabaseUpdate();
    239 }
    240 
    241 void LocalStorageDatabase::scheduleDatabaseUpdate()
    242 {
    243     if (m_didScheduleDatabaseUpdate)
    244         return;
    245 
    246     if (!m_disableSuddenTerminationWhileWritingToLocalStorage)
    247         m_disableSuddenTerminationWhileWritingToLocalStorage = makeUnique<SuddenTerminationDisabler>();
    248 
    249     m_didScheduleDatabaseUpdate = true;
    250 
    251     m_queue->dispatch([protectedThis = makeRef(*this)] {
    252         protectedThis->updateDatabase();
    253     });
    254 }
    255 
    256 void LocalStorageDatabase::updateDatabase()
    257 {
    258     if (m_isClosed)
    259         return;
    260 
    261     m_didScheduleDatabaseUpdate = false;
    262 
    263     HashMap<String, String> changedItems;
    264     if (m_changedItems.size() <= maximumItemsToUpdate) {
    265         // There are few enough changed items that we can just always write all of them.
    266         m_changedItems.swap(changedItems);
    267         updateDatabaseWithChangedItems(changedItems);
    268         m_disableSuddenTerminationWhileWritingToLocalStorage = nullptr;
    269     } else {
    270         for (int i = 0; i < maximumItemsToUpdate; ++i) {
    271             auto it = m_changedItems.begin();
    272             changedItems.add(it->key, it->value);
    273 
    274             m_changedItems.remove(it);
    275         }
    276 
    277         ASSERT(changedItems.size() <= maximumItemsToUpdate);
    278 
    279         // Reschedule the update for the remaining items.
    280         scheduleDatabaseUpdate();
    281         updateDatabaseWithChangedItems(changedItems);
    282     }
    283 }
    284 
    285 void LocalStorageDatabase::updateDatabaseWithChangedItems(const HashMap<String, String>& changedItems)
    286 {
    287     if (!m_database.isOpen())
    288         openDatabase(CreateIfNonExistent);
    289     if (!m_database.isOpen())
    290         return;
    291 
    292     if (m_shouldClearItems) {
    293         m_shouldClearItems = false;
    294 
    295         SQLiteStatement clearStatement(m_database, "DELETE FROM ItemTable");
    296         if (clearStatement.prepare() != SQLITE_OK) {
    297             LOG_ERROR("Failed to prepare clear statement - cannot write to local storage database");
    298             return;
    299         }
    300 
    301         int result = clearStatement.step();
    302         if (result != SQLITE_DONE) {
    303             LOG_ERROR("Failed to clear all items in the local storage database - %i", result);
    304             return;
    305         }
    306     }
    307 
    308     SQLiteStatement insertStatement(m_database, "INSERT INTO ItemTable VALUES (?, ?)");
    309     if (insertStatement.prepare() != SQLITE_OK) {
    310         LOG_ERROR("Failed to prepare insert statement - cannot write to local storage database");
    311         return;
    312     }
    313 
    314     SQLiteStatement deleteStatement(m_database, "DELETE FROM ItemTable WHERE key=?");
    315     if (deleteStatement.prepare() != SQLITE_OK) {
    316         LOG_ERROR("Failed to prepare delete statement - cannot write to local storage database");
    317         return;
    318     }
    319 
    320     SQLiteTransaction transaction(m_database);
    321     transaction.begin();
    322 
    323     for (auto it = changedItems.begin(), end = changedItems.end(); it != end; ++it) {
    324         // A null value means that the key/value pair should be deleted.
    325         SQLiteStatement& statement = it->value.isNull() ? deleteStatement : insertStatement;
    326 
    327         statement.bindText(1, it->key);
    328 
    329         // If we're inserting a key/value pair, bind the value as well.
    330         if (!it->value.isNull())
    331             statement.bindBlob(2, it->value);
    332 
    333         int result = statement.step();
    334         if (result != SQLITE_DONE) {
    335             LOG_ERROR("Failed to update item in the local storage database - %i", result);
    336             break;
    337         }
    338 
    339         statement.reset();
    340     }
    341 
    342     transaction.commit();
    343 }
    344 
    345 bool LocalStorageDatabase::databaseIsEmpty()
    346 {
     334bool LocalStorageDatabase::databaseIsEmpty() const
     335{
     336    ASSERT(!RunLoop::isMain());
    347337    if (!m_database.isOpen())
    348338        return false;
    … …  
    363353}
    364354
     355void LocalStorageDatabase::openIfExisting()
     356{
     357    ASSERT(!RunLoop::isMain());
     358    if (m_database.isOpen())
     359        return;
     360
     361    openDatabase(ShouldCreateDatabase::No);
     362
     363    // Prewarm the getItems statement for performance since the pages block synchronously on retrieving the items.
     364    if (m_database.isOpen())
     365        scopedStatement(m_getItemsStatement, getItemsQueryString);
     366}
     367
     368SQLiteStatementAutoResetScope LocalStorageDatabase::scopedStatement(std::unique_ptr<SQLiteStatement>& statement, const String& query) const
     369{
     370    ASSERT(!RunLoop::isMain());
     371    if (!statement) {
     372        statement = makeUnique<SQLiteStatement>(m_database, query);
     373        ASSERT(m_database.isOpen());
     374        if (statement->prepare() != SQLITE_OK)
     375            return SQLiteStatementAutoResetScope { };
     376    }
     377    return SQLiteStatementAutoResetScope { statement.get() };
     378}
     379
    365380void LocalStorageDatabase::handleLowMemoryWarning()
    366381{

  • trunk/Source/WebKit/NetworkProcess/WebStorage/LocalStorageDatabase.h

    r276648 r276653  
    11/*
    2  * Copyright (C) 2008, 2009, 2010, 2013, 2019 Apple Inc. All rights reserved.
     2 * Copyright (C) 2008-2021 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    2727
    2828#include <WebCore/SQLiteDatabase.h>
    29 #include <WebCore/SecurityOriginData.h>
    3029#include <wtf/Forward.h>
    31 #include <wtf/HashMap.h>
    3230#include <wtf/RefCounted.h>
    33 #include <wtf/WorkQueue.h>
    3431
    3532namespace WebCore {
    36 class SecurityOrigin;
    37 class StorageMap;
     33class SQLiteStatementAutoResetScope;
    3834class SuddenTerminationDisabler;
     35
     36struct SecurityOriginData;
    3937}
    4038
    … …  
    4543class LocalStorageDatabase : public RefCounted<LocalStorageDatabase> {
    4644public:
    47     static Ref<LocalStorageDatabase> create(Ref<WorkQueue>&&, Ref<LocalStorageDatabaseTracker>&&, const WebCore::SecurityOriginData&);
     45    static Ref<LocalStorageDatabase> create(Ref<WorkQueue>&&, Ref<LocalStorageDatabaseTracker>&&, const WebCore::SecurityOriginData&, unsigned quotaInBytes);
    4846    ~LocalStorageDatabase();
    4947
    50     // Will block until the import is complete.
    51     void importItems(WebCore::StorageMap&);
    52 
    53     void setItem(const String& key, const String& value);
    54     void removeItem(const String& key);
    55     void clear();
    56 
    57     void updateDatabase();
     48    HashMap<String, String> items() const;
     49    void openIfExisting();
     50    void removeItem(const String& key, String& oldValue);
     51    bool clear();
     52    String item(const String& key) const;
     53    void setItem(const String& key, const String& value, String& oldValue, bool& quotaException);
    5854
    5955    // Will block until all pending changes have been written to disk.
    … …  
    6359
    6460private:
    65     LocalStorageDatabase(Ref<WorkQueue>&&, Ref<LocalStorageDatabaseTracker>&&, const WebCore::SecurityOriginData&);
     61    LocalStorageDatabase(Ref<WorkQueue>&&, Ref<LocalStorageDatabaseTracker>&&, const WebCore::SecurityOriginData&, unsigned quotaInBytes);
    6662
    67     enum DatabaseOpeningStrategy {
    68         CreateIfNonExistent,
    69         SkipIfNonExistent
    70     };
    71     bool tryToOpenDatabase(DatabaseOpeningStrategy);
    72     void openDatabase(DatabaseOpeningStrategy);
     63    enum class ShouldCreateDatabase : bool { No, Yes };
     64    bool tryToOpenDatabase(ShouldCreateDatabase);
     65    void openDatabase(ShouldCreateDatabase);
    7366
    7467    bool migrateItemTableIfNeeded();
     68    bool databaseIsEmpty() const;
    7569
    76     void itemDidChange(const String& key, const String& value);
    77 
    78     void scheduleDatabaseUpdate();
    79     void updateDatabaseWithChangedItems(const HashMap<String, String>&);
    80 
    81     bool databaseIsEmpty();
     70    WebCore::SQLiteStatementAutoResetScope scopedStatement(std::unique_ptr<WebCore::SQLiteStatement>&, const String& query) const;
    8271
    8372    Ref<WorkQueue> m_queue;
    … …  
    8675
    8776    String m_databasePath;
    88     WebCore::SQLiteDatabase m_database;
     77    mutable WebCore::SQLiteDatabase m_database;
     78    unsigned m_quotaInBytes { 0 };
    8979    bool m_failedToOpenDatabase { false };
    90     bool m_didImportItems { false };
    9180    bool m_isClosed { false };
    92 
    93     bool m_didScheduleDatabaseUpdate { false };
    94     bool m_shouldClearItems { false };
    95     HashMap<String, String> m_changedItems;
     81    Optional<uint64_t> m_databaseSize;
    9682
    9783    std::unique_ptr<WebCore::SuddenTerminationDisabler> m_disableSuddenTerminationWhileWritingToLocalStorage;
     84    mutable std::unique_ptr<WebCore::SQLiteStatement> m_clearStatement;
     85    mutable std::unique_ptr<WebCore::SQLiteStatement> m_insertStatement;
     86    mutable std::unique_ptr<WebCore::SQLiteStatement> m_getItemStatement;
     87    mutable std::unique_ptr<WebCore::SQLiteStatement> m_getItemsStatement;
     88    mutable std::unique_ptr<WebCore::SQLiteStatement> m_deleteItemStatement;
    9889};
    9990

  • trunk/Source/WebKit/NetworkProcess/WebStorage/LocalStorageNamespace.cpp

    r275886 r276653  
    7474{
    7575    ASSERT(!RunLoop::isMain());
    76     if (auto* storageArea = m_storageAreaMap.get(origin)) {
    77         storageArea->syncToDatabase();
     76    if (auto* storageArea = m_storageAreaMap.get(origin))
    7877        storageArea->close();
    79     }
    8078}
    8179

  • trunk/Source/WebKit/NetworkProcess/WebStorage/StorageArea.cpp

    r276648 r276653  
    3232#include "StorageManager.h"
    3333#include <WebCore/StorageMap.h>
     34#include <wtf/Scope.h>
    3435
    3536namespace WebKit {
    … …  
    4142    , m_securityOrigin(securityOrigin)
    4243    , m_quotaInBytes(quotaInBytes)
    43     , m_storageMap(StorageMap::create(m_quotaInBytes))
    4444    , m_identifier(Identifier::generate())
    4545    , m_queue(WTFMove(queue))
    4646{
    4747    ASSERT(!RunLoop::isMain());
     48    if (isEphemeral())
     49        m_storageMap = StorageMap::create(m_quotaInBytes);
    4850}
    4951
    … …  
    6264
    6365    if (m_eventListeners.isEmpty() && !isEphemeral())
    64         openDatabaseAndImportItemsIfNeeded();
     66        ensureDatabase();
    6567
    6668    m_eventListeners.add(connectionID);
    … …  
    7577        return;
    7678
    77     if (!m_localStorageNamespace)
    78         return;
    79 
    80     syncToDatabase();
    81     m_localStorageNamespace->removeStorageArea(m_securityOrigin);
     79    if (m_localStorageNamespace)
     80        m_localStorageNamespace->removeStorageArea(m_securityOrigin);
    8281}
    8382
    … …  
    102101{
    103102    ASSERT(!RunLoop::isMain());
    104     openDatabaseAndImportItemsIfNeeded();
    105103
    106104    String oldValue;
    107 
    108     auto newStorageMap = m_storageMap->setItem(key, value, oldValue, quotaException);
    109     if (newStorageMap)
    110         m_storageMap = WTFMove(newStorageMap);
     105    if (isEphemeral()) {
     106        auto newStorageMap = m_storageMap->setItem(key, value, oldValue, quotaException);
     107        if (newStorageMap)
     108            m_storageMap = WTFMove(newStorageMap);
     109    } else
     110        ensureDatabase().setItem(key, value, oldValue, quotaException);
    111111
    112112    if (quotaException)
    113113        return;
    114114
    115     if (m_localStorageDatabase)
    116         m_localStorageDatabase->setItem(key, value);
    117 
    118115    dispatchEvents(sourceConnection, storageAreaImplID, key, oldValue, value, urlString);
    119116}
    … …  
    122119{
    123120    ASSERT(!RunLoop::isMain());
    124     openDatabaseAndImportItemsIfNeeded();
    125121
    126122    String oldValue;
    127     auto newStorageMap = m_storageMap->removeItem(key, oldValue);
    128     if (newStorageMap)
    129         m_storageMap = WTFMove(newStorageMap);
    130 
     123    if (isEphemeral()) {
     124        auto newStorageMap = m_storageMap->removeItem(key, oldValue);
     125        if (newStorageMap)
     126            m_storageMap = WTFMove(newStorageMap);
     127    } else
     128        ensureDatabase().removeItem(key, oldValue);
    131129    if (oldValue.isNull())
    132130        return;
    133131
    134     if (m_localStorageDatabase)
    135         m_localStorageDatabase->removeItem(key);
    136 
    137132    dispatchEvents(sourceConnection, storageAreaImplID, key, oldValue, String(), urlString);
    138133}
    … …  
    141136{
    142137    ASSERT(!RunLoop::isMain());
    143     openDatabaseAndImportItemsIfNeeded();
    144 
    145     if (!m_storageMap->length())
    146         return;
    147 
    148     m_storageMap = StorageMap::create(m_quotaInBytes);
    149 
    150     if (m_localStorageDatabase)
    151         m_localStorageDatabase->clear();
     138
     139    if (isEphemeral()) {
     140        if (!m_storageMap->length())
     141            return;
     142        if (auto newStorageMap = m_storageMap->clear())
     143            m_storageMap = WTFMove(newStorageMap);
     144    } else {
     145        if (!ensureDatabase().clear())
     146            return;
     147    }
    152148
    153149    dispatchEvents(sourceConnection, storageAreaImplID, String(), String(), String(), urlString);
    154150}
    155151
    156 const HashMap<String, String>& StorageArea::items() const
    157 {
    158     ASSERT(!RunLoop::isMain());
    159     openDatabaseAndImportItemsIfNeeded();
    160 
    161     return m_storageMap->items();
     152HashMap<String, String> StorageArea::items() const
     153{
     154    ASSERT(!RunLoop::isMain());
     155    if (isEphemeral())
     156        return m_storageMap->items();
     157
     158    return ensureDatabase().items();
    162159}
    163160
    … …  
    165162{
    166163    ASSERT(!RunLoop::isMain());
    167     m_storageMap = StorageMap::create(m_quotaInBytes);
    168 
    169     if (m_localStorageDatabase) {
    170         m_localStorageDatabase->close();
    171         m_localStorageDatabase = nullptr;
     164    if (isEphemeral()) {
     165        if (auto newStorageMap = m_storageMap->clear())
     166            m_storageMap = WTFMove(newStorageMap);
     167    } else {
     168        if (m_localStorageDatabase) {
     169            m_localStorageDatabase->close();
     170            m_localStorageDatabase = nullptr;
     171        }
    172172    }
    173173
    … …  
    180180}
    181181
    182 void StorageArea::openDatabaseAndImportItemsIfNeeded() const
    183 {
    184     ASSERT(!RunLoop::isMain());
    185     if (!m_localStorageNamespace)
    186         return;
     182LocalStorageDatabase& StorageArea::ensureDatabase() const
     183{
     184    ASSERT(!isEphemeral());
    187185
    188186    ASSERT(m_localStorageNamespace->storageManager()->localStorageDatabaseTracker());
    189187    // We open the database here even if we've already imported our items to ensure that the database is open if we need to write to it.
    190     if (!m_localStorageDatabase)
    191         m_localStorageDatabase = LocalStorageDatabase::create(m_queue.copyRef(), *m_localStorageNamespace->storageManager()->localStorageDatabaseTracker(), m_securityOrigin);
    192 
    193     if (m_didImportItemsFromDatabase)
    194         return;
    195 
    196     m_localStorageDatabase->importItems(*m_storageMap);
    197     m_didImportItemsFromDatabase = true;
     188    if (!m_localStorageDatabase) {
     189        m_localStorageDatabase = LocalStorageDatabase::create(m_queue.copyRef(), *m_localStorageNamespace->storageManager()->localStorageDatabaseTracker(), m_securityOrigin, m_quotaInBytes);
     190        m_localStorageDatabase->openIfExisting();
     191    }
     192    return *m_localStorageDatabase;
    198193}
    199194
    … …  
    212207}
    213208
    214 void StorageArea::syncToDatabase()
    215 {
    216     if (!m_localStorageDatabase)
    217         return;
    218 
    219     m_localStorageDatabase->updateDatabase();
    220 }
    221 
    222209void StorageArea::close()
    223210{

  • trunk/Source/WebKit/NetworkProcess/WebStorage/StorageArea.h

    r276648 r276653  
    6464    void clear(IPC::Connection::UniqueID sourceConnection, StorageAreaImplIdentifier, const String& urlString);
    6565
    66     const HashMap<String, String>& items() const;
     66    HashMap<String, String> items() const;
    6767    void clear();
    6868
    6969    bool isEphemeral() const { return !m_localStorageNamespace; }
    7070
    71     void openDatabaseAndImportItemsIfNeeded() const;
    72 
    73     void syncToDatabase();
    7471    void close();
    7572
    … …  
    7976    void dispatchEvents(IPC::Connection::UniqueID sourceConnection, StorageAreaImplIdentifier, const String& key, const String& oldValue, const String& newValue, const String& urlString) const;
    8077
     78    LocalStorageDatabase& ensureDatabase() const;
     79
    8180    // Will be null if the storage area belongs to a session storage namespace or the storage area is in an ephemeral session.
    8281    WeakPtr<LocalStorageNamespace> m_localStorageNamespace;
    8382    mutable RefPtr<LocalStorageDatabase> m_localStorageDatabase;
    84     mutable bool m_didImportItemsFromDatabase { false };
    8583
    8684    WebCore::SecurityOriginData m_securityOrigin;

  • trunk/Source/WebKit/NetworkProcess/WebStorage/StorageManagerSet.cpp

    r276648 r276653  
    156156    ASSERT(RunLoop::isMain());
    157157
    158     BinarySemaphore semaphore;
    159     m_queue->dispatch([this, &semaphore] {
    160         for (const auto& storageArea : m_storageAreas.values()) {
    161             ASSERT(storageArea);
    162             if (storageArea)
    163                 storageArea->syncToDatabase();
    164         }
    165         semaphore.signal();
    166     });
    167     semaphore.wait();
     158    m_queue->dispatchSync([] { });
    168159}
    169160

  • trunk/Source/WebKitLegacy/ChangeLog

    r276614 r276653  
     12021-04-27  Chris Dumez  <cdumez@apple.com>
     2
     3        Don't keep local storage data in memory in the NetworkProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=225065
     5
     6        Reviewed by Alex Christensen.
     7
     8        * Storage/StorageAreaImpl.cpp:
     9        (WebKit::StorageAreaImpl::clear):
     10        (WebKit::StorageAreaImpl::clearForOriginDeletion):
     11        Use the new StorageMap::clear() to avoid constructing a new StorageMap unnecessarily
     12        when:
     13        - The StorageMap is not shared
     14        - The StorageMap is empty
     15
    1162021-04-26  Alex Christensen  <achristensen@webkit.org>
    217

  • trunk/Source/WebKitLegacy/Storage/StorageAreaImpl.cpp

    r254169 r276653  
    175175        return;
    176176
    177     unsigned quota = m_storageMap->quota();
    178     m_storageMap = StorageMap::create(quota);
     177    if (auto newStorageMap = m_storageMap->clear())
     178        m_storageMap = WTFMove(newStorageMap);
    179179
    180180    if (m_storageAreaSync)
    … …  
    215215    blockUntilImportComplete();
    216216   
    217     if (m_storageMap->length()) {
    218         unsigned quota = m_storageMap->quota();
    219         m_storageMap = StorageMap::create(quota);
    220     }
     217    if (auto newStorageMap = m_storageMap->clear())
     218        m_storageMap = WTFMove(newStorageMap);
    221219
    222220    if (m_storageAreaSync) {
Note: See TracChangeset for help on using the changeset viewer.