Changeset 261141 in webkit
- Timestamp:
- May 4, 2020 8:19:56 PM (4 years ago)
- Location:
- trunk
- Files:
-
- 4 added
- 11 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/Source/WebCore/ChangeLog
r261139 r261141 1 2020-05-04 Ben Nham <nham@apple.com> 2 3 IndexedDB WAL file keeps growing while app is in use 4 https://bugs.webkit.org/show_bug.cgi?id=202137 5 6 Reviewed by Brady Eidson. 7 8 It's easy to get into a situation where the WAL file associated with a SQLite-backed 9 IndexedDB grows indefinitely while a site is in use for two reasons: 10 11 1. We don't promptly reset cached prepared statements in SQLiteIDBBackingStore. Many 12 statements are left hanging in the SQLITE_ROW state without being reset or fully stepped to 13 the SQLITE_DONE state. These hanging statements keep their associated transactions open and 14 prevent the WAL checkpointer from progressing past those active transactions. 15 16 To fix this, I added SQLiteStatementAutoResetScope. This is a scope guard that 17 SQLiteIDBBackingStore uses to ensure that cached statements are reset in a timely manner. 18 19 While going through the reset code I also noticed we aren't clearing bindings after 20 resetting statements. We should be doing this because sqlite3_reset does not clear bindings 21 (and their associated copies of blobs/strings); sqlite3_clear_bindings does that. 22 23 2. The default WAL hook for auto-checkpointing in upstream SQLite uses the 24 SQLITE_CHECKPOINT_PASSIVE mode, which doesn't truncate the WAL until the next write 25 transaction occurs. (It actually doesn't truncate at all when compiled with default 26 settings, but macOS's SQLite sets SQLITE_DEFAULT_JOURNAL_SIZE_LIMIT, which causes the 27 truncation to occur on the next write.) 28 29 We want the WAL to be truncated more promptly, because otherwise the quota check that 30 happens on each mutation won't be as accurate. To do this, I installed a WAL hook that 31 truncates the WAL with SQLITE_CHECKPOINT_TRUNCATE after the default threshold of 1000 WAL 32 pages. I didn't enable this for all SQLiteDatabases because this checkpoint call can block 33 on the busy handler. This isn't a problem for IDB since we don't use busy handlers in IDB. 34 35 * Headers.cmake: 36 * Modules/indexeddb/server/SQLiteIDBBackingStore.cpp: 37 (WebCore::IDBServer::SQLiteIDBBackingStore::getOrEstablishDatabaseInfo): 38 (WebCore::IDBServer::SQLiteIDBBackingStore::createObjectStore): 39 (WebCore::IDBServer::SQLiteIDBBackingStore::deleteObjectStore): 40 (WebCore::IDBServer::SQLiteIDBBackingStore::renameObjectStore): 41 (WebCore::IDBServer::SQLiteIDBBackingStore::clearObjectStore): 42 (WebCore::IDBServer::SQLiteIDBBackingStore::createIndex): 43 (WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedHasIndexRecord): 44 (WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedPutIndexRecord): 45 (WebCore::IDBServer::SQLiteIDBBackingStore::deleteIndex): 46 (WebCore::IDBServer::SQLiteIDBBackingStore::renameIndex): 47 (WebCore::IDBServer::SQLiteIDBBackingStore::keyExistsInObjectStore): 48 (WebCore::IDBServer::SQLiteIDBBackingStore::deleteUnusedBlobFileRecords): 49 (WebCore::IDBServer::SQLiteIDBBackingStore::deleteRecord): 50 (WebCore::IDBServer::SQLiteIDBBackingStore::updateAllIndexesForAddRecord): 51 (WebCore::IDBServer::SQLiteIDBBackingStore::addRecord): 52 (WebCore::IDBServer::SQLiteIDBBackingStore::getBlobRecordsForObjectStoreRecord): 53 (WebCore::IDBServer::SQLiteIDBBackingStore::getRecord): 54 (WebCore::IDBServer::SQLiteIDBBackingStore::cachedStatementForGetAllObjectStoreRecords): 55 (WebCore::IDBServer::SQLiteIDBBackingStore::getAllObjectStoreRecords): 56 (WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedGetIndexRecordForOneKey): 57 (WebCore::IDBServer::SQLiteIDBBackingStore::getCount): 58 (WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedGetKeyGeneratorValue): 59 (WebCore::IDBServer::SQLiteIDBBackingStore::uncheckedSetKeyGeneratorValue): 60 (WebCore::IDBServer::SQLiteIDBBackingStore::cachedStatement): 61 * Modules/indexeddb/server/SQLiteIDBBackingStore.h: 62 * Sources.txt: 63 * WebCore.xcodeproj/project.pbxproj: 64 * platform/sql/SQLiteDatabase.cpp: 65 (WebCore::walAutomaticTruncationHook): 66 (WebCore::SQLiteDatabase::enableAutomaticWALTruncation): 67 * platform/sql/SQLiteDatabase.h: 68 * platform/sql/SQLiteStatement.cpp: 69 (WebCore::SQLiteStatement::reset): 70 * platform/sql/SQLiteStatementAutoResetScope.cpp: Added. 71 (WebCore::SQLiteStatementAutoResetScope::SQLiteStatementAutoResetScope): 72 (WebCore::SQLiteStatementAutoResetScope::operator=): 73 (WebCore::SQLiteStatementAutoResetScope::~SQLiteStatementAutoResetScope): 74 * platform/sql/SQLiteStatementAutoResetScope.h: Added. 75 (WebCore::SQLiteStatementAutoResetScope::operator bool const): 76 (WebCore::SQLiteStatementAutoResetScope::operator! const): 77 (WebCore::SQLiteStatementAutoResetScope::get): 78 (WebCore::SQLiteStatementAutoResetScope::operator->): 79 1 80 2020-05-04 Wenson Hsieh <wenson_hsieh@apple.com> 2 81 -
trunk/Source/WebCore/Headers.cmake
r261132 r261141 1314 1314 platform/sql/SQLiteFileSystem.h 1315 1315 platform/sql/SQLiteStatement.h 1316 platform/sql/SQLiteStatementAutoResetScope.h 1316 1317 platform/sql/SQLiteTransaction.h 1317 1318 -
trunk/Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.cpp
r259519 r261141 970 970 return IDBError { UnknownError, "Unable to open database file on disk"_s }; 971 971 972 m_sqliteDB->enableAutomaticWALTruncation(); 973 972 974 m_sqliteDB->setCollationFunction("IDBKEY", [](int aLength, const void* a, int bLength, const void* b) { 973 975 return idbKeyCollate(aLength, a, bLength, b); … … 1127 1129 1128 1130 { 1129 auto *sql = cachedStatement(SQL::CreateObjectStoreInfo, "INSERT INTO ObjectStoreInfo VALUES (?, ?, ?, ?);"_s);1131 auto sql = cachedStatement(SQL::CreateObjectStoreInfo, "INSERT INTO ObjectStoreInfo VALUES (?, ?, ?, ?);"_s); 1130 1132 if (!sql 1131 1133 || sql->bindInt64(1, info.identifier()) != SQLITE_OK … … 1140 1142 1141 1143 { 1142 auto *sql = cachedStatement(SQL::CreateObjectStoreKeyGenerator, "INSERT INTO KeyGenerators VALUES (?, 0);"_s);1144 auto sql = cachedStatement(SQL::CreateObjectStoreKeyGenerator, "INSERT INTO KeyGenerators VALUES (?, 0);"_s); 1143 1145 if (!sql 1144 1146 || sql->bindInt64(1, info.identifier()) != SQLITE_OK … … 1172 1174 // Delete the ObjectStore record 1173 1175 { 1174 auto *sql = cachedStatement(SQL::DeleteObjectStoreInfo, "DELETE FROM ObjectStoreInfo WHERE id = ?;"_s);1176 auto sql = cachedStatement(SQL::DeleteObjectStoreInfo, "DELETE FROM ObjectStoreInfo WHERE id = ?;"_s); 1175 1177 if (!sql 1176 1178 || sql->bindInt64(1, objectStoreIdentifier) != SQLITE_OK … … 1183 1185 // Delete the ObjectStore's key generator record if there is one. 1184 1186 { 1185 auto *sql = cachedStatement(SQL::DeleteObjectStoreKeyGenerator, "DELETE FROM KeyGenerators WHERE objectStoreID = ?;"_s);1187 auto sql = cachedStatement(SQL::DeleteObjectStoreKeyGenerator, "DELETE FROM KeyGenerators WHERE objectStoreID = ?;"_s); 1186 1188 if (!sql 1187 1189 || sql->bindInt64(1, objectStoreIdentifier) != SQLITE_OK … … 1194 1196 // Delete all associated records 1195 1197 { 1196 auto *sql = cachedStatement(SQL::DeleteObjectStoreRecords, "DELETE FROM Records WHERE objectStoreID = ?;"_s);1198 auto sql = cachedStatement(SQL::DeleteObjectStoreRecords, "DELETE FROM Records WHERE objectStoreID = ?;"_s); 1197 1199 if (!sql 1198 1200 || sql->bindInt64(1, objectStoreIdentifier) != SQLITE_OK … … 1205 1207 // Delete all associated Indexes 1206 1208 { 1207 auto *sql = cachedStatement(SQL::DeleteObjectStoreIndexInfo, "DELETE FROM IndexInfo WHERE objectStoreID = ?;"_s);1209 auto sql = cachedStatement(SQL::DeleteObjectStoreIndexInfo, "DELETE FROM IndexInfo WHERE objectStoreID = ?;"_s); 1208 1210 if (!sql 1209 1211 || sql->bindInt64(1, objectStoreIdentifier) != SQLITE_OK … … 1216 1218 // Delete all associated Index records 1217 1219 { 1218 auto *sql = cachedStatement(SQL::DeleteObjectStoreIndexRecords, "DELETE FROM IndexRecords WHERE objectStoreID = ?;"_s);1220 auto sql = cachedStatement(SQL::DeleteObjectStoreIndexRecords, "DELETE FROM IndexRecords WHERE objectStoreID = ?;"_s); 1219 1221 if (!sql 1220 1222 || sql->bindInt64(1, objectStoreIdentifier) != SQLITE_OK … … 1227 1229 // Delete all unused Blob URL records. 1228 1230 { 1229 auto *sql = cachedStatement(SQL::DeleteObjectStoreBlobRecords, "DELETE FROM BlobRecords WHERE objectStoreRow NOT IN (SELECT recordID FROM Records)"_s);1231 auto sql = cachedStatement(SQL::DeleteObjectStoreBlobRecords, "DELETE FROM BlobRecords WHERE objectStoreRow NOT IN (SELECT recordID FROM Records)"_s); 1230 1232 if (!sql 1231 1233 || sql->step() != SQLITE_DONE) { … … 1262 1264 1263 1265 { 1264 auto *sql = cachedStatement(SQL::RenameObjectStore, "UPDATE ObjectStoreInfo SET name = ? WHERE id = ?;"_s);1266 auto sql = cachedStatement(SQL::RenameObjectStore, "UPDATE ObjectStoreInfo SET name = ? WHERE id = ?;"_s); 1265 1267 if (!sql 1266 1268 || sql->bindText(1, newName) != SQLITE_OK … … 1294 1296 1295 1297 { 1296 auto *sql = cachedStatement(SQL::ClearObjectStoreRecords, "DELETE FROM Records WHERE objectStoreID = ?;"_s);1298 auto sql = cachedStatement(SQL::ClearObjectStoreRecords, "DELETE FROM Records WHERE objectStoreID = ?;"_s); 1297 1299 if (!sql 1298 1300 || sql->bindInt64(1, objectStoreID) != SQLITE_OK … … 1304 1306 1305 1307 { 1306 auto *sql = cachedStatement(SQL::ClearObjectStoreIndexRecords, "DELETE FROM IndexRecords WHERE objectStoreID = ?;"_s);1308 auto sql = cachedStatement(SQL::ClearObjectStoreIndexRecords, "DELETE FROM IndexRecords WHERE objectStoreID = ?;"_s); 1307 1309 if (!sql 1308 1310 || sql->bindInt64(1, objectStoreID) != SQLITE_OK … … 1339 1341 } 1340 1342 1341 auto* sql = cachedStatement(SQL::CreateIndexInfo, "INSERT INTO IndexInfo VALUES (?, ?, ?, ?, ?, ?);"_s); 1342 if (!sql 1343 || sql->bindInt64(1, info.identifier()) != SQLITE_OK 1344 || sql->bindText(2, info.name()) != SQLITE_OK 1345 || sql->bindInt64(3, info.objectStoreIdentifier()) != SQLITE_OK 1346 || sql->bindBlob(4, keyPathBlob->data(), keyPathBlob->size()) != SQLITE_OK 1347 || sql->bindInt(5, info.unique()) != SQLITE_OK 1348 || sql->bindInt(6, info.multiEntry()) != SQLITE_OK 1349 || sql->step() != SQLITE_DONE) { 1350 LOG_ERROR("Could not add index '%s' to IndexInfo table (%i) - %s", info.name().utf8().data(), m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg()); 1351 return IDBError { UnknownError, "Unable to create index in database"_s }; 1343 { 1344 auto sql = cachedStatement(SQL::CreateIndexInfo, "INSERT INTO IndexInfo VALUES (?, ?, ?, ?, ?, ?);"_s); 1345 if (!sql 1346 || sql->bindInt64(1, info.identifier()) != SQLITE_OK 1347 || sql->bindText(2, info.name()) != SQLITE_OK 1348 || sql->bindInt64(3, info.objectStoreIdentifier()) != SQLITE_OK 1349 || sql->bindBlob(4, keyPathBlob->data(), keyPathBlob->size()) != SQLITE_OK 1350 || sql->bindInt(5, info.unique()) != SQLITE_OK 1351 || sql->bindInt(6, info.multiEntry()) != SQLITE_OK 1352 || sql->step() != SQLITE_DONE) { 1353 LOG_ERROR("Could not add index '%s' to IndexInfo table (%i) - %s", info.name().utf8().data(), m_sqliteDB->lastError(), m_sqliteDB->lastErrorMsg()); 1354 return IDBError { UnknownError, "Unable to create index in database"_s }; 1355 } 1352 1356 } 1353 1357 … … 1370 1374 IDBError error = updateOneIndexForAddRecord(info, key, valueBuffer, cursor->currentRecordRowID()); 1371 1375 if (!error.isNull()) { 1372 auto *sql = cachedStatement(SQL::DeleteIndexInfo, "DELETE FROM IndexInfo WHERE id = ? AND objectStoreID = ?;"_s);1376 auto sql = cachedStatement(SQL::DeleteIndexInfo, "DELETE FROM IndexInfo WHERE id = ? AND objectStoreID = ?;"_s); 1373 1377 if (!sql 1374 1378 || sql->bindInt64(1, info.identifier()) != SQLITE_OK … … 1412 1416 } 1413 1417 1414 auto *sql = cachedStatement(SQL::HasIndexRecord, "SELECT rowid FROM IndexRecords WHERE indexID = ? AND key = CAST(? AS TEXT);"_s);1418 auto sql = cachedStatement(SQL::HasIndexRecord, "SELECT rowid FROM IndexRecords WHERE indexID = ? AND key = CAST(? AS TEXT);"_s); 1415 1419 if (!sql 1416 1420 || sql->bindInt64(1, info.identifier()) != SQLITE_OK … … 1488 1492 1489 1493 { 1490 auto *sql = cachedStatement(SQL::PutIndexRecord, "INSERT INTO IndexRecords VALUES (?, ?, CAST(? AS TEXT), CAST(? AS TEXT), ?);"_s);1494 auto sql = cachedStatement(SQL::PutIndexRecord, "INSERT INTO IndexRecords VALUES (?, ?, CAST(? AS TEXT), CAST(? AS TEXT), ?);"_s); 1491 1495 if (!sql 1492 1496 || sql->bindInt64(1, indexID) != SQLITE_OK … … 1522 1526 1523 1527 { 1524 auto *sql = cachedStatement(SQL::DeleteIndexInfo, "DELETE FROM IndexInfo WHERE id = ? AND objectStoreID = ?;"_s);1528 auto sql = cachedStatement(SQL::DeleteIndexInfo, "DELETE FROM IndexInfo WHERE id = ? AND objectStoreID = ?;"_s); 1525 1529 if (!sql 1526 1530 || sql->bindInt64(1, indexIdentifier) != SQLITE_OK … … 1533 1537 1534 1538 { 1535 auto *sql = cachedStatement(SQL::DeleteIndexRecords, "DELETE FROM IndexRecords WHERE indexID = ?;"_s);1539 auto sql = cachedStatement(SQL::DeleteIndexRecords, "DELETE FROM IndexRecords WHERE indexID = ?;"_s); 1536 1540 if (!sql 1537 1541 || sql->bindInt64(1, indexIdentifier) != SQLITE_OK … … 1574 1578 1575 1579 { 1576 auto *sql = cachedStatement(SQL::RenameIndex, "UPDATE IndexInfo SET name = ? WHERE objectStoreID = ? AND id = ?;"_s);1580 auto sql = cachedStatement(SQL::RenameIndex, "UPDATE IndexInfo SET name = ? WHERE objectStoreID = ? AND id = ?;"_s); 1577 1581 if (!sql 1578 1582 || sql->bindText(1, newName) != SQLITE_OK … … 1608 1612 return IDBError { UnknownError, "Unable to serialize IDBKey to check for existence in object store"_s }; 1609 1613 } 1610 auto *sql = cachedStatement(SQL::KeyExistsInObjectStore, "SELECT key FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT) LIMIT 1;"_s);1614 auto sql = cachedStatement(SQL::KeyExistsInObjectStore, "SELECT key FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT) LIMIT 1;"_s); 1611 1615 if (!sql 1612 1616 || sql->bindInt64(1, objectStoreID) != SQLITE_OK … … 1637 1641 HashSet<String> removedBlobFilenames; 1638 1642 { 1639 auto *sql = cachedStatement(SQL::GetUnusedBlobFilenames, "SELECT fileName FROM BlobFiles WHERE blobURL NOT IN (SELECT blobURL FROM BlobRecords)"_s);1643 auto sql = cachedStatement(SQL::GetUnusedBlobFilenames, "SELECT fileName FROM BlobFiles WHERE blobURL NOT IN (SELECT blobURL FROM BlobRecords)"_s); 1640 1644 1641 1645 if (!sql) { … … 1658 1662 // Remove the blob records that are no longer in use. 1659 1663 if (!removedBlobFilenames.isEmpty()) { 1660 auto *sql = cachedStatement(SQL::DeleteUnusedBlobs, "DELETE FROM BlobFiles WHERE blobURL NOT IN (SELECT blobURL FROM BlobRecords)"_s);1664 auto sql = cachedStatement(SQL::DeleteUnusedBlobs, "DELETE FROM BlobFiles WHERE blobURL NOT IN (SELECT blobURL FROM BlobRecords)"_s); 1661 1665 1662 1666 if (!sql … … 1693 1697 ThreadSafeDataBuffer value; 1694 1698 { 1695 auto *sql = cachedStatement(SQL::GetObjectStoreRecord, "SELECT recordID, value FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT);"_s);1699 auto sql = cachedStatement(SQL::GetObjectStoreRecord, "SELECT recordID, value FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT);"_s); 1696 1700 1697 1701 if (!sql … … 1727 1731 // Delete the blob records for this object store record. 1728 1732 { 1729 auto *sql = cachedStatement(SQL::DeleteBlobRecord, "DELETE FROM BlobRecords WHERE objectStoreRow = ?;"_s);1733 auto sql = cachedStatement(SQL::DeleteBlobRecord, "DELETE FROM BlobRecords WHERE objectStoreRow = ?;"_s); 1730 1734 1731 1735 if (!sql … … 1743 1747 // Delete record from object store 1744 1748 { 1745 auto *sql = cachedStatement(SQL::DeleteObjectStoreRecord, "DELETE FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT);"_s);1749 auto sql = cachedStatement(SQL::DeleteObjectStoreRecord, "DELETE FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT);"_s); 1746 1750 1747 1751 if (!sql … … 1756 1760 // Delete record from indexes store 1757 1761 { 1758 auto *sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND objectStoreRecordID = ?;"_s);1762 auto sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND objectStoreRecordID = ?;"_s); 1759 1763 1760 1764 if (!sql … … 1874 1878 1875 1879 if (!error.isNull() && anyRecordsSucceeded) { 1876 auto *sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND objectStoreRecordID = ?;"_s);1880 auto sql = cachedStatement(SQL::DeleteObjectStoreIndexRecord, "DELETE FROM IndexRecords WHERE objectStoreID = ? AND objectStoreRecordID = ?;"_s); 1877 1881 1878 1882 if (!sql … … 1914 1918 int64_t recordID = 0; 1915 1919 { 1916 auto *sql = cachedStatement(SQL::AddObjectStoreRecord, "INSERT INTO Records VALUES (?, CAST(? AS TEXT), ?, NULL);"_s);1920 auto sql = cachedStatement(SQL::AddObjectStoreRecord, "INSERT INTO Records VALUES (?, CAST(? AS TEXT), ?, NULL);"_s); 1917 1921 if (!sql 1918 1922 || sql->bindInt64(1, objectStoreInfo.identifier()) != SQLITE_OK … … 1930 1934 1931 1935 if (!error.isNull()) { 1932 auto *sql = cachedStatement(SQL::DeleteObjectStoreRecord, "DELETE FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT);"_s);1936 auto sql = cachedStatement(SQL::DeleteObjectStoreRecord, "DELETE FROM Records WHERE objectStoreID = ? AND key = CAST(? AS TEXT);"_s); 1933 1937 if (!sql 1934 1938 || sql->bindInt64(1, objectStoreInfo.identifier()) != SQLITE_OK … … 1947 1951 auto& url = blobURLs[i]; 1948 1952 { 1949 auto *sql = cachedStatement(SQL::AddBlobRecord, "INSERT INTO BlobRecords VALUES (?, ?);"_s);1953 auto sql = cachedStatement(SQL::AddBlobRecord, "INSERT INTO BlobRecords VALUES (?, ?);"_s); 1950 1954 if (!sql 1951 1955 || sql->bindInt64(1, recordID) != SQLITE_OK … … 1960 1964 // If we already have a file for this blobURL, nothing left to do. 1961 1965 { 1962 auto *sql = cachedStatement(SQL::BlobFilenameForBlobURL, "SELECT fileName FROM BlobFiles WHERE blobURL = ?;"_s);1966 auto sql = cachedStatement(SQL::BlobFilenameForBlobURL, "SELECT fileName FROM BlobFiles WHERE blobURL = ?;"_s); 1963 1967 if (!sql 1964 1968 || sql->bindText(1, url) != SQLITE_OK) { … … 1980 1984 String storedFilename = makeString(potentialFileNameInteger, ".blob"); 1981 1985 { 1982 auto *sql = cachedStatement(SQL::AddBlobFilename, "INSERT INTO BlobFiles VALUES (?, ?);"_s);1986 auto sql = cachedStatement(SQL::AddBlobFilename, "INSERT INTO BlobFiles VALUES (?, ?);"_s); 1983 1987 if (!sql 1984 1988 || sql->bindText(1, url) != SQLITE_OK … … 2004 2008 HashSet<String> blobURLSet; 2005 2009 { 2006 auto *sql = cachedStatement(SQL::GetBlobURL, "SELECT blobURL FROM BlobRecords WHERE objectStoreRow = ?"_s);2010 auto sql = cachedStatement(SQL::GetBlobURL, "SELECT blobURL FROM BlobRecords WHERE objectStoreRow = ?"_s); 2007 2011 if (!sql 2008 2012 || sql->bindInt64(1, objectStoreRecord) != SQLITE_OK) { … … 2030 2034 ASSERT(!blobURLSet.isEmpty()); 2031 2035 for (auto& blobURL : blobURLSet) { 2032 auto *sql = cachedStatement(SQL::BlobFilenameForBlobURL, "SELECT fileName FROM BlobFiles WHERE blobURL = ?;"_s);2036 auto sql = cachedStatement(SQL::BlobFilenameForBlobURL, "SELECT fileName FROM BlobFiles WHERE blobURL = ?;"_s); 2033 2037 if (!sql 2034 2038 || sql->bindText(1, blobURL) != SQLITE_OK) { … … 2096 2100 static const char* const lowerClosedUpperClosedKeyOnly = "SELECT key FROM Records WHERE objectStoreID = ? AND key >= CAST(? AS TEXT) AND key <= CAST(? AS TEXT) ORDER BY key;"; 2097 2101 2098 SQLiteStatement * sql = nullptr;2102 SQLiteStatementAutoResetScope sql; 2099 2103 2100 2104 switch (type) { … … 2192 2196 } 2193 2197 2194 SQLiteStatement *SQLiteIDBBackingStore::cachedStatementForGetAllObjectStoreRecords(const IDBGetAllRecordsData& getAllRecordsData)2198 SQLiteStatementAutoResetScope SQLiteIDBBackingStore::cachedStatementForGetAllObjectStoreRecords(const IDBGetAllRecordsData& getAllRecordsData) 2195 2199 { 2196 2200 static const char* const lowerOpenUpperOpenKey ="SELECT key FROM Records WHERE objectStoreID = ? AND key > CAST(? AS TEXT) AND key < CAST(? AS TEXT) ORDER BY key;"; … … 2255 2259 } 2256 2260 2257 auto *sql = cachedStatementForGetAllObjectStoreRecords(getAllRecordsData);2261 auto sql = cachedStatementForGetAllObjectStoreRecords(getAllRecordsData); 2258 2262 if (!sql 2259 2263 || sql->bindInt64(1, getAllRecordsData.objectStoreIdentifier) != SQLITE_OK … … 2419 2423 } 2420 2424 2421 auto *sql = cachedStatement(SQL::GetIndexRecordForOneKey, "SELECT IndexRecords.value, Records.value, Records.recordID FROM Records INNER JOIN IndexRecords ON Records.objectStoreID = IndexRecords.objectStoreID AND Records.recordID = IndexRecords.objectStoreRecordID WHERE IndexRecords.indexID = ? AND IndexRecords.key = CAST(? AS TEXT) ORDER BY IndexRecords.key, IndexRecords.value"_s);2425 auto sql = cachedStatement(SQL::GetIndexRecordForOneKey, "SELECT IndexRecords.value, Records.value, Records.recordID FROM Records INNER JOIN IndexRecords ON Records.objectStoreID = IndexRecords.objectStoreID AND Records.recordID = IndexRecords.objectStoreRecordID WHERE IndexRecords.indexID = ? AND IndexRecords.key = CAST(? AS TEXT) ORDER BY IndexRecords.key, IndexRecords.value"_s); 2422 2426 2423 2427 if (!sql … … 2494 2498 } 2495 2499 2496 SQLiteStatement * statement = nullptr;2500 SQLiteStatementAutoResetScope statement; 2497 2501 2498 2502 if (!indexIdentifier) { … … 2551 2555 IDBError SQLiteIDBBackingStore::uncheckedGetKeyGeneratorValue(int64_t objectStoreID, uint64_t& outValue) 2552 2556 { 2553 auto *sql = cachedStatement(SQL::GetKeyGeneratorValue, "SELECT currentKey FROM KeyGenerators WHERE objectStoreID = ?;"_s);2557 auto sql = cachedStatement(SQL::GetKeyGeneratorValue, "SELECT currentKey FROM KeyGenerators WHERE objectStoreID = ?;"_s); 2554 2558 if (!sql 2555 2559 || sql->bindInt64(1, objectStoreID) != SQLITE_OK) { … … 2573 2577 IDBError SQLiteIDBBackingStore::uncheckedSetKeyGeneratorValue(int64_t objectStoreID, uint64_t value) 2574 2578 { 2575 auto *sql = cachedStatement(SQL::SetKeyGeneratorValue, "INSERT INTO KeyGenerators VALUES (?, ?);"_s);2579 auto sql = cachedStatement(SQL::SetKeyGeneratorValue, "INSERT INTO KeyGenerators VALUES (?, ?);"_s); 2576 2580 if (!sql 2577 2581 || sql->bindInt64(1, objectStoreID) != SQLITE_OK … … 2788 2792 } 2789 2793 2790 SQLiteStatement *SQLiteIDBBackingStore::cachedStatement(SQLiteIDBBackingStore::SQL sql, const char* statement)2794 SQLiteStatementAutoResetScope SQLiteIDBBackingStore::cachedStatement(SQLiteIDBBackingStore::SQL sql, const char* statement) 2791 2795 { 2792 2796 if (sql >= SQL::Invalid) { 2793 2797 LOG_ERROR("Invalid SQL statement ID passed to cachedStatement()"); 2794 return nullptr;2798 return SQLiteStatementAutoResetScope { }; 2795 2799 } 2796 2800 2797 2801 if (m_cachedStatements[static_cast<size_t>(sql)]) { 2798 if (m_cachedStatements[static_cast<size_t>(sql)]->reset() == SQLITE_OK) 2799 return m_cachedStatements[static_cast<size_t>(sql)].get(); 2800 m_cachedStatements[static_cast<size_t>(sql)] = nullptr; 2802 return SQLiteStatementAutoResetScope { m_cachedStatements[static_cast<size_t>(sql)].get() }; 2801 2803 } 2802 2804 … … 2807 2809 } 2808 2810 2809 return m_cachedStatements[static_cast<size_t>(sql)].get();2811 return SQLiteStatementAutoResetScope { m_cachedStatements[static_cast<size_t>(sql)].get() }; 2810 2812 } 2811 2813 -
trunk/Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.h
r256621 r261141 33 33 #include "IDBResourceIdentifier.h" 34 34 #include "SQLiteIDBTransaction.h" 35 #include "SQLiteStatementAutoResetScope.h" 35 36 #include <JavaScriptCore/Strong.h> 36 37 #include <pal/SessionID.h> … … 194 195 }; 195 196 196 SQLiteStatement *cachedStatement(SQL, const char*);197 SQLiteStatement *cachedStatementForGetAllObjectStoreRecords(const IDBGetAllRecordsData&);197 SQLiteStatementAutoResetScope cachedStatement(SQL, const char*); 198 SQLiteStatementAutoResetScope cachedStatementForGetAllObjectStoreRecords(const IDBGetAllRecordsData&); 198 199 199 200 std::unique_ptr<SQLiteStatement> m_cachedStatements[static_cast<int>(SQL::Invalid)]; -
trunk/Source/WebCore/Sources.txt
r261132 r261141 2093 2093 platform/sql/SQLiteFileSystem.cpp 2094 2094 platform/sql/SQLiteStatement.cpp 2095 platform/sql/SQLiteStatementAutoResetScope.cpp 2095 2096 platform/sql/SQLiteTransaction.cpp 2096 2097 -
trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj
r261132 r261141 5034 5034 E59DD4B821098287003C8B47 /* ListButtonArrow.png in Resources */ = {isa = PBXBuildFile; fileRef = E59DD4B721098285003C8B47 /* ListButtonArrow.png */; }; 5035 5035 E5BA7D63151437CA00FE1E3F /* LengthFunctions.h in Headers */ = {isa = PBXBuildFile; fileRef = E5BA7D62151437CA00FE1E3F /* LengthFunctions.h */; settings = {ATTRIBUTES = (Private, ); }; }; 5036 EBE5B226245A26EF003A5A88 /* SQLiteStatementAutoResetScope.h in Headers */ = {isa = PBXBuildFile; fileRef = EBE5B224245A26EE003A5A88 /* SQLiteStatementAutoResetScope.h */; settings = {ATTRIBUTES = (Private, ); }; }; 5036 5037 EBF5121C1696496C0056BD25 /* JSTypeConversions.cpp in Sources */ = {isa = PBXBuildFile; fileRef = EBF5121A1696496C0056BD25 /* JSTypeConversions.cpp */; }; 5037 5038 EBF5121D1696496C0056BD25 /* JSTypeConversions.h in Headers */ = {isa = PBXBuildFile; fileRef = EBF5121B1696496C0056BD25 /* JSTypeConversions.h */; }; … … 15684 15685 EB081CD81696084400553730 /* TypeConversions.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = TypeConversions.h; sourceTree = "<group>"; }; 15685 15686 EB081CD91696084400553730 /* TypeConversions.idl */ = {isa = PBXFileReference; lastKnownFileType = text; path = TypeConversions.idl; sourceTree = "<group>"; }; 15687 EBE5B224245A26EE003A5A88 /* SQLiteStatementAutoResetScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SQLiteStatementAutoResetScope.h; sourceTree = "<group>"; }; 15688 EBE5B227245A29CF003A5A88 /* SQLiteStatementAutoResetScope.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SQLiteStatementAutoResetScope.cpp; sourceTree = "<group>"; }; 15686 15689 EBF5121A1696496C0056BD25 /* JSTypeConversions.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSTypeConversions.cpp; sourceTree = "<group>"; }; 15687 15690 EBF5121B1696496C0056BD25 /* JSTypeConversions.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSTypeConversions.h; sourceTree = "<group>"; }; … … 17258 17261 1A2246450CC98DDB00C05240 /* SQLiteStatement.cpp */, 17259 17262 1A2246460CC98DDB00C05240 /* SQLiteStatement.h */, 17263 EBE5B227245A29CF003A5A88 /* SQLiteStatementAutoResetScope.cpp */, 17264 EBE5B224245A26EE003A5A88 /* SQLiteStatementAutoResetScope.h */, 17260 17265 1A2246470CC98DDB00C05240 /* SQLiteTransaction.cpp */, 17261 17266 1A2246480CC98DDB00C05240 /* SQLiteTransaction.h */, … … 32955 32960 511EC12C1C50ABBF0032F983 /* SQLiteIDBTransaction.h in Headers */, 32956 32961 1A22464C0CC98DDB00C05240 /* SQLiteStatement.h in Headers */, 32962 EBE5B226245A26EF003A5A88 /* SQLiteStatementAutoResetScope.h in Headers */, 32957 32963 1A22464E0CC98DDB00C05240 /* SQLiteTransaction.h in Headers */, 32958 32964 97BC6A421505F081001B74AC /* SQLResultSet.h in Headers */, -
trunk/Source/WebCore/platform/sql/SQLiteDatabase.cpp
r257688 r261141 159 159 160 160 return isOpen(); 161 } 162 163 static int walAutomaticTruncationHook(void* context, sqlite3* db, const char* dbName, int walPageCount) 164 { 165 UNUSED_PARAM(context); 166 167 static constexpr int checkpointThreshold = 1000; // matches SQLITE_DEFAULT_WAL_AUTOCHECKPOINT 168 169 if (walPageCount >= checkpointThreshold) { 170 int newWalPageCount = 0; 171 int result = sqlite3_wal_checkpoint_v2(db, dbName, SQLITE_CHECKPOINT_TRUNCATE, &newWalPageCount, nullptr); 172 173 #if LOG_DISABLED 174 UNUSED_VARIABLE(result); 175 #else 176 if (result != SQLITE_OK || newWalPageCount) { 177 LOG(SQLDatabase, "Can't fully checkpoint SQLite db %p", db); 178 179 sqlite3_stmt* stmt = nullptr; 180 while ((stmt = sqlite3_next_stmt(db, stmt))) { 181 if (sqlite3_stmt_busy(stmt)) 182 LOG(SQLDatabase, "SQLite db %p has busy stmt %p blocking checkpoint: %s", db, stmt, sqlite3_sql(stmt)); 183 } 184 } 185 #endif 186 } 187 188 return SQLITE_OK; 189 } 190 191 void SQLiteDatabase::enableAutomaticWALTruncation() 192 { 193 sqlite3_wal_hook(m_db, walAutomaticTruncationHook, nullptr); 161 194 } 162 195 -
trunk/Source/WebCore/platform/sql/SQLiteDatabase.h
r254087 r261141 82 82 void setFullsync(bool); 83 83 84 // This enables automatic WAL truncation via a commit hook that uses SQLITE_CHECKPOINT_TRUNCATE. 85 // However, it shouldn't be used if you use a custom busy handler or timeout. This is because 86 // SQLITE_CHECKPOINT_TRUNCATE will invoke the busy handler if it can't acquire the necessary 87 // locks, which can lead to unintended delays. 88 void enableAutomaticWALTruncation(); 89 84 90 // Gets/sets the maximum size in bytes 85 91 // Depending on per-database attributes, the size will only be settable in units that are the page size of the database, which is established at creation -
trunk/Source/WebCore/platform/sql/SQLiteStatement.cpp
r254087 r261141 123 123 return SQLITE_OK; 124 124 LOG(SQLDatabase, "SQL - reset - %s", m_query.ascii().data()); 125 return sqlite3_reset(m_statement); 125 int status = sqlite3_reset(m_statement); 126 sqlite3_clear_bindings(m_statement); 127 return status; 126 128 } 127 129 -
trunk/Tools/ChangeLog
r261138 r261141 1 2020-05-04 Ben Nham <nham@apple.com> 2 3 IndexedDB WAL file keeps growing while app is in use 4 https://bugs.webkit.org/show_bug.cgi?id=202137 5 6 Reviewed by Brady Eidson. 7 8 Add a test to make sure that WAL auto-checkpointing actually works. 9 10 * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: 11 * TestWebKitAPI/Tests/WebKitCocoa/IDBCheckpointWAL.html: Added. 12 * TestWebKitAPI/Tests/WebKitCocoa/IDBCheckpointWAL.mm: Added. 13 (-[IDBCheckpointWALMessageHandler userContentController:didReceiveScriptMessage:]): 14 (fileSizeAtPath): 15 (TEST): 16 1 17 2020-05-04 Darin Adler <darin@apple.com> 2 18 -
trunk/Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
r261044 r261141 1035 1035 E3F8AB92241AB9CE003E2A7E /* AccessibilityRemoteUIApp.mm in Sources */ = {isa = PBXBuildFile; fileRef = E3F8AB91241AB9CE003E2A7E /* AccessibilityRemoteUIApp.mm */; }; 1036 1036 E5036F78211BC25400BFDBE2 /* color-drop.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = E5036F77211BC22800BFDBE2 /* color-drop.html */; }; 1037 EB230D40245E727900C66AD1 /* IDBCheckpointWAL.mm in Sources */ = {isa = PBXBuildFile; fileRef = EB230D3E245E726300C66AD1 /* IDBCheckpointWAL.mm */; }; 1038 EB230D41245E813F00C66AD1 /* IDBCheckpointWAL.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = EB230D3D245E722E00C66AD1 /* IDBCheckpointWAL.html */; }; 1037 1039 ECA680CE1E68CC0900731D20 /* StringUtilities.mm in Sources */ = {isa = PBXBuildFile; fileRef = ECA680CD1E68CC0900731D20 /* StringUtilities.mm */; }; 1038 1040 F402F56C23ECC2FB00865549 /* UIWKInteractionViewProtocol.mm in Sources */ = {isa = PBXBuildFile; fileRef = F402F56B23ECC2FB00865549 /* UIWKInteractionViewProtocol.mm */; }; … … 1352 1354 9B26FCCA159D16DE00CC3765 /* HTMLFormCollectionNamedItem.html in Copy Resources */, 1353 1355 BCBD3737125ABBEB00D2C29F /* icon.png in Copy Resources */, 1356 EB230D41245E813F00C66AD1 /* IDBCheckpointWAL.html in Copy Resources */, 1354 1357 510477771D298E72009747EB /* IDBDeleteRecovery.html in Copy Resources */, 1355 1358 510477721D298DDD009747EB /* IDBDeleteRecovery.sqlite3 in Copy Resources */, … … 2669 2672 E4C9ABC71B3DB1710040A987 /* RunLoop.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RunLoop.cpp; sourceTree = "<group>"; }; 2670 2673 E5036F77211BC22800BFDBE2 /* color-drop.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "color-drop.html"; sourceTree = "<group>"; }; 2674 EB230D3D245E722E00C66AD1 /* IDBCheckpointWAL.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = IDBCheckpointWAL.html; sourceTree = "<group>"; }; 2675 EB230D3E245E726300C66AD1 /* IDBCheckpointWAL.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = IDBCheckpointWAL.mm; sourceTree = "<group>"; }; 2671 2676 EC79F168BE454E579E417B05 /* Markable.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = Markable.cpp; sourceTree = "<group>"; }; 2672 2677 ECA680CD1E68CC0900731D20 /* StringUtilities.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = StringUtilities.mm; sourceTree = "<group>"; }; … … 3080 3085 2DADF26221CB8F32003D3E3A /* GetResourceData.mm */, 3081 3086 51AF23DE1EF1A3720072F281 /* IconLoadingDelegate.mm */, 3087 EB230D3E245E726300C66AD1 /* IDBCheckpointWAL.mm */, 3082 3088 510477751D298E03009747EB /* IDBDeleteRecovery.mm */, 3083 3089 5110FCEF1E01CBAA006F8D0B /* IDBIndexUpgradeToV2.mm */, … … 3551 3557 467C565121B5ECDF0057516D /* GetSessionCookie.html */, 3552 3558 F47D30ED1ED28A6C000482E1 /* gif-and-file-input.html */, 3559 EB230D3D245E722E00C66AD1 /* IDBCheckpointWAL.html */, 3553 3560 510477761D298E57009747EB /* IDBDeleteRecovery.html */, 3554 3561 5104776F1D298D85009747EB /* IDBDeleteRecovery.sqlite3 */, … … 4935 4942 5C7C24FC237C975400599C91 /* HTTPServer.mm in Sources */, 4936 4943 51AF23DF1EF1A3730072F281 /* IconLoadingDelegate.mm in Sources */, 4944 EB230D40245E727900C66AD1 /* IDBCheckpointWAL.mm in Sources */, 4937 4945 510477781D29923B009747EB /* IDBDeleteRecovery.mm in Sources */, 4938 4946 5110FCFA1E01CDB8006F8D0B /* IDBIndexUpgradeToV2.mm in Sources */,
Note: See TracChangeset
for help on using the changeset viewer.