Changeset 25439 in webkit


Ignore:
Timestamp:
Sep 8, 2007, 1:33:08 PM (17 years ago)
Author:
beidson
Message:

WebCore:

Reviewed by Darin

<rdar://problem/5434431> - Asynchronous Icon Database

The IconDatabase API was originally designed to be fully asynchronous - if an icon wasn't read in from disk
when you asked for it, you would be notified when it was.

Safari 2 did writes on a background thread, but reads blocked the main thread.

The current WebCore implementation using SQLite attempted to get rid of the background thread by defering expensive
writes via timers, but falls short in moderate to extreme usage cases

Time to make the IconDatabase live up to it's fully asynchronous destiny.

This should -

  • Make the browser instantly usable while converting Safari 2 icons in the background occurs
  • Remedy any UI slowness/blocking when on slow network home directories
  • Remedy random UI slowness, pauses, and stutters do to random I/O occurring at the exact wrong time or under heavy disk usage from swapping or other apps on the system
  • Allow certain long-running procedures to be interruptible (Safari 2 import, reading icons in from disk when trying to quit, etc)

This will have a noticeable effect on current Safari 2 and Safari 3 beta browsers, including icons not appearing in bookmarks, history,
or the location field the first time they're asked for, as current released Safari's don't properly listen for these async notifations.
The second time such a menu or view is brought up, the icon should be there.

Additionally this includes a SQLite schema change which will be a lot more efficient but will result in the loss of current SQLite icons.
Converting from Safari 2 style icons will still work.

WebCore, welcome to multi-threadedness

  • WebCore.exp:
  • WebCore.xcodeproj/project.pbxproj:
  • WebCore.vcproj/WebCore.vcproj:
  • loader/DocumentLoader.cpp: (WebCore::DocumentLoader::iconLoadDecisionAvailable): Called when an Icon becomes available that was requested by this DocumentLoader (to support the webView:didReceiveIcon: delegate call in WebKit)
  • loader/DocumentLoader.h:
  • loader/FrameLoader.cpp: (WebCore::FrameLoader::iconLoadDecisionAvailable): Called from the DocumentLoaders who get notified - if the FrameLoader ends up not caring because the WebView has transitioned to a new page, nothing occurs. Otherwise, the FrameLoader possibly starts it Icon Loader and possibly sends the webView:didReceiveIcon: delegate call (WebCore::FrameLoader::startIconLoader): Instead of "Yes, load the icon now" or "No, don't load it" there is a third possibility - "You might be asked to load your icon later." Add supporting logic for receiving this state, and being called a second time when the load decision is finally available.
  • loader/FrameLoader.h:
  • loader/FrameLoaderClient.h: Added "registerForIconNotification" which is a way to tell WebViews "The icon you are interested in might become available via the generic WebIconDatabaseDidAddIconNotification instead of a targeted delegate call" A WebView can then receive the generic notification and pass on it's own targeted delegate call.
  • loader/icon/IconDataCache.cpp: Removed.
  • loader/icon/IconDataCache.h: Removed.
  • loader/icon/IconDatabase.cpp: (WebCore::urlForLogging): Cut a URL down in length for sane logging and debugging (WebCore::defaultClient): Return the default, empty IconDatabaseClient incase the API doesn't set one.

Following block of methods are for the Main thread's usage -
(WebCore::IconDatabase::setClient):
(WebCore::makeAllDirectories): Small optimization that checks to see if the entire path exists already, and doesn't try to loop

through each patch component if the full path is already present

(WebCore::IconDatabase::open): Makes all directories to the target path and kicks off the background thread - nothing more.
(WebCore::IconDatabase::close): Signals the thread to quit and waits for it to do so
(WebCore::IconDatabase::removeAllIcons): Purge the icon database
(WebCore::IconDatabase::iconForPageURL):
(WebCore::IconDatabase::readIconForPageURLFromDisk):
(WebCore::IconDatabase::iconURLForPageURL):
(WebCore::IconDatabase::defaultIcon):
(WebCore::IconDatabase::retainIconForPageURL):
(WebCore::IconDatabase::releaseIconForPageURL):
(WebCore::IconDatabase::setIconDataForIconURL):
(WebCore::IconDatabase::setIconURLForPageURL):
(WebCore::IconDatabase::loadDecisionForIconURL): Determine if an icon loader should load now. If the decision is "maybe later", then

mark the DocumentLoader to be notified later when the final decision is available.

(WebCore::IconDatabase::iconDataKnownForIconURL): Determine if the actual image data has been read from disk (or set from the loader) for

icon URL in question

(WebCore::IconDatabase::setEnabled):
(WebCore::IconDatabase::isEnabled):
(WebCore::IconDatabase::setPrivateBrowsingEnabled):
(WebCore::IconDatabase::isPrivateBrowsingEnabled):
(WebCore::IconDatabase::delayDatabaseCleanup): Restore this method from a year ago, as asynchronous pruning of icons can now occur on a

background thread.

(WebCore::IconDatabase::allowDatabaseCleanup):
(WebCore::IconDatabase::checkIntegrityBeforeOpening):
(WebCore::IconDatabase::pageURLMappingCount):
(WebCore::IconDatabase::retainedPageURLCount):
(WebCore::IconDatabase::iconRecordCount):
(WebCore::IconDatabase::iconRecordCountWithData):
(WebCore::IconDatabase::IconDatabase):
(WebCore::IconDatabase::~IconDatabase):
(WebCore::IconDatabase::notifyPendingLoadDecisions): Tell all the registered DocumentLoaders "Hey, we've read in all URL mappings from disk,

so check to see if you are interested in any of them"

(WebCore::IconDatabase::notifyPendingLoadDecisionsInternal):
(WebCore::IconDatabase::wakeSyncThread): Wake the sync thread, if it is idle
(WebCore::IconDatabase::scheduleOrDeferSyncTimer): Even though we're on a background thread, we still defer writing out to disk during

periods of high activity

(WebCore::IconDatabase::syncTimerFired): Call wakeSyncThread()

Following block of methods may be used by either thread -
(WebCore::IconDatabase::isOpen):
(WebCore::IconDatabase::databasePath):
(WebCore::IconDatabase::defaultDatabaseFilename):
(WebCore::IconDatabase::getOrCreateIconRecord):
(WebCore::IconDatabase::getOrCreatePageURLRecord):

Following block of methods are used by the secondary thread only -
(WebCore::IconDatabase::importIconURLForPageURL): For the Safari 2 import procedure - write a URL mapping directly out to disk
(WebCore::IconDatabase::importIconDataForIconURL): For the Safari 2 import procedure - write an Icon directly out to disk
(WebCore::IconDatabase::shouldStopThreadActivity): To check and see if the thread should stop what it is doing now to do something

more important (such as quit, or delete all icons)

(WebCore::IconDatabase::iconDatabaseSyncThreadStart):
(WebCore::IconDatabase::iconDatabaseSyncThread): Entry point for the background thread
(WebCore::databaseVersionNumber):
(WebCore::isValidDatabase):
(WebCore::createDatabaseTables):
(WebCore::IconDatabase::performOpenInitialization): Open and validate the SQLite database, making sure it's schema jives with what

is expected

(WebCore::IconDatabase::checkIntegrity):
(WebCore::IconDatabase::performURLImport): Import all the Page URL -> Icon URL mappings from the database. Done "1st thing" on startup,

this is necessary to be able to give the loader decisions about whether or not it should load icons from the network

(WebCore::IconDatabase::syncThreadMainLoop): Main loop - sleeps until woken up, then does a read cycle and a write cycle until both cycles

do no work - then it goes back to sleep.

(WebCore::IconDatabase::readFromDatabase): Reads icons from the database that clients are waiting on
(WebCore::IconDatabase::writeToDatabase): Writes any changes page -> icon url mappings to disk, as well as any new image data that has

been received from the loader

(WebCore::IconDatabase::pruneUnretainedIcons): Done only once, and only after the first write to the database, this procedure removes all

icons and page URLs from disk that haven't been retained by any client. Note that the prune can be delayed by utilizing delayDatabaseCleanup()

(WebCore::IconDatabase::checkForDanglingPageURLs): Usually part of the prune procedure, prunes any pages who point to icons that no longer exist

in the database

(WebCore::IconDatabase::removeAllIconsOnThread): Completely purge both the on-disk and in memory records of all icons
(WebCore::IconDatabase::deleteAllPreparedStatements): Part of removeAllIcons and the thread cleanup procedure
(WebCore::IconDatabase::cleanupSyncThread): Write out any last remaining writes to disk, close the database, and then end the thread
(WebCore::IconDatabase::imported): Checks the DB to see if the Safari 2 import has occured
(WebCore::IconDatabase::setImported): Sets the "Safari 2 imported" flag
(WebCore::readySQLStatement):
(WebCore::IconDatabase::setIconURLForPageURLInSQLDatabase): This and the following "SQLDatabase" suffixed methods are pretty self explanatory
(WebCore::IconDatabase::setIconIDForPageURLInSQLDatabase):
(WebCore::IconDatabase::removePageURLFromSQLDatabase):
(WebCore::IconDatabase::getIconIDForIconURLFromSQLDatabase):
(WebCore::IconDatabase::addIconURLToSQLDatabase):
(WebCore::IconDatabase::getImageDataForIconURLFromSQLDatabase):
(WebCore::IconDatabase::removeIconFromSQLDatabase):
(WebCore::IconDatabase::writeIconSnapshotToSQLDatabase):

  • loader/icon/IconDatabase.h:
  • loader/icon/IconDatabaseClient.h: Added. (WebCore::IconDatabaseClient::~IconDatabaseClient): (WebCore::IconDatabaseClient::performImport): Perform the Safari 2 import, implemented by WebKit (WebCore::IconDatabaseClient::dispatchDidRemoveAllIcons): Send the API notification (WebCore::IconDatabaseClient::dispatchDidAddIconForPageURL): Ditto
  • loader/icon/IconDatabaseNone.cpp: Best attempt to keep non icon-DB platforms building (WebCore::IconDatabase::defaultDatabaseFilename): (WebCore::IconDatabase::readIconForPageURLFromDisk): (WebCore::IconDatabase::loadDecisionForIconURL): (WebCore::IconDatabase::iconDataKnownForIconURL): (WebCore::IconDatabase::setIconURLForPageURL): (WebCore::IconDatabase::isEnabled): (WebCore::IconDatabase::delayDatabaseCleanup): (WebCore::IconDatabase::allowDatabaseCleanup): (WebCore::IconDatabase::setClient):
  • loader/icon/IconRecord.cpp: Added. (WebCore::IconRecord::IconRecord): IconRecord used to be "IconDataCache" - it is merely a container for the url, timestamp, and image for a site icon. It is Shared, and therefore ref counted - PageURLRecords are the owning containers. This is a tricky way to track how many page urls are retaining an IconRecord and therefore tracking when we should try to get rid of one. (WebCore::IconRecord::~IconRecord): (WebCore::IconRecord::image): (WebCore::IconRecord::setImageData): (WebCore::IconRecord::loadImageFromResource): (WebCore::IconRecord::imageDataStatus): Return whether the image data hasn't been read yet, exists in memory, or is absent (site with no icon) (WebCore::IconRecord::snapshot): Returns a snapshot of the icon's data - url, timestamp, and image data - to be written to disk
  • loader/icon/IconRecord.h: Added. (WebCore::IconSnapshot::IconSnapshot): (WebCore::IconRecord::getTimestamp): (WebCore::IconRecord::setTimestamp): (WebCore::IconRecord::iconURL): (WebCore::IconRecord::retainingPageURLs):
  • loader/icon/PageURLRecord.cpp: Added. (WebCore::PageURLRecord::PageURLRecord): PageURLRecord is fundamentally a pairing of a Page URL to an Icon. It has manual ref counting for the sake of "retainIconForPageURL" and "releaseIconForPageURL", and can provide a quick snapshot of it's Page URL -> Icon URL mapping for writing to the database (WebCore::PageURLRecord::setIconRecord): (WebCore::PageURLRecord::snapshot):
  • loader/icon/PageURLRecord.h: Added. (WebCore::PageURLSnapshot::PageURLSnapshot): (WebCore::PageURLRecord::url): (WebCore::PageURLRecord::PageURLRecord::iconRecord): (WebCore::PageURLRecord::retain): (WebCore::PageURLRecord::release): (WebCore::PageURLRecord::retainCount):
  • platform/SharedBuffer.cpp: (WebCore::SharedBuffer::copy): Added a deep copy method for the purposes of handing icon data across the thread boundary into the icon database
  • platform/SharedBuffer.h:
  • platform/graphics/svg/SVGImageEmptyClients.h: (WebCore::SVGEmptyFrameLoaderClient::registerForIconNotification):
  • platform/win/TemporaryLinkStubs.cpp: (WebCore::callOnMainThread): Only other IconDatabase utilizing platform - keep their build going

WebKit:

Reviewed by Darin

<rdar://problem/5434431> - Asynchronous Icon Database

WebKit side of things
Mainly, there are Notifications WebKit has to listen for now that tell it when to either call back into WebCore
for some purpose or to send the webView:didReceiveIcon: delegate call

Many smaller tweaks as well.

  • Misc/WebIconDatabase.h:
  • Misc/WebIconDatabase.mm: (defaultClient): (-[WebIconDatabase init]): (+[WebIconDatabase delayDatabaseCleanup]): Accessor so clients can prevent the thread from cleaning up the database before they've done all their necessary retaining of icons. (+[WebIconDatabase allowDatabaseCleanup]): (-[WebIconDatabase removeAllIcons]): (-[WebIconDatabase _isEnabled]): (-[WebIconDatabase _sendNotificationForURL:]): (-[WebIconDatabase _sendDidRemoveAllIconsNotification]): (-[WebIconDatabase _databaseDirectory]):

(-[ThreadEnabler threadEnablingSelector:]): Quick and dirty class to enabled Cocoa multithreading
(+[ThreadEnabler enableThreading]):
(importToWebCoreFormat):

  • Misc/WebIconDatabaseInternal.h: Expose the internal methods of WebIconDatabase that are required by WebIconDatabaseClient
  • Misc/WebNSNotificationCenterExtras.h: Added. - Great utility class whose design was borrowed from Colloquy that allows the posting of a Cocoa notification on the main thread from *any* thread
  • Misc/WebNSNotificationCenterExtras.m: Added. (-[NSNotificationCenter postNotificationOnMainThreadWithName:object:]): (-[NSNotificationCenter postNotificationOnMainThreadWithName:object:userInfo:]): (-[NSNotificationCenter postNotificationOnMainThreadWithName:object:userInfo:waitUntilDone:]): (+[NSNotificationCenter _postNotificationName:]):
  • WebCoreSupport/WebFrameLoaderClient.h:
  • WebCoreSupport/WebFrameLoaderClient.mm: (WebFrameLoaderClient::dispatchDidReceiveIcon): Send the webView:didReceiveIcon: delegate call (WebFrameLoaderClient::registerForIconNotification):
  • WebCoreSupport/WebIconDatabaseClient.h: Added.
  • WebCoreSupport/WebIconDatabaseClient.mm: Added. (WebIconDatabaseClient::performImport): Perform the Safari 2 icon import (WebIconDatabaseClient::dispatchDidRemoveAllIcons): Send the NSNotification (WebIconDatabaseClient::dispatchDidAddIconForPageURL): Ditto
  • WebView/WebView.mm: (-[WebView _receivedIconChangedNotification:]): Check and see if this notification is for this WebView's current URL by calling back into the IconDatabase (-[WebView _registerForIconNotification:]): Support for WebIconDatabaseClient (-[WebView _dispatchDidReceiveIconFromWebFrame:]): Dispatch this delegate call as well as unregister for the notification
  • WebView/WebViewInternal.h:
  • WebKit.xcodeproj/project.pbxproj:

win:

<rdar://problem/5434431> - Asynchronous Icon Database

  • WebFrame.cpp: (WebFrame::didPerformFirstNavigation): Empty impl for now (WebFrame::registerForIconNotification): Ditto
  • WebFrame.h:
Location:
trunk
Files:
9 added
2 deleted
28 edited

Legend:

Unmodified
Added
Removed

  • trunk/WebCore/ChangeLog

    r25435 r25439  
     12007-09-08  Brady Eidson  <beidson@apple.com>
     2
     3        Reviewed by Darin
     4
     5        <rdar://problem/5434431> - Asynchronous Icon Database
     6
     7        The IconDatabase API was originally designed to be fully asynchronous - if an icon wasn't read in from disk
     8        when you asked for it, you would be notified when it was.
     9
     10        Safari 2 did writes on a background thread, but reads blocked the main thread.
     11
     12        The current WebCore implementation using SQLite attempted to get rid of the background thread by defering expensive
     13        writes via timers, but falls short in moderate to extreme usage cases
     14
     15        Time to make the IconDatabase live up to it's fully asynchronous destiny.
     16
     17        This should -
     18        - Make the browser instantly usable while converting Safari 2 icons in the background occurs
     19        - Remedy any UI slowness/blocking when on slow network home directories
     20        - Remedy random UI slowness, pauses, and stutters do to random I/O occurring at the exact wrong time or under heavy
     21          disk usage from swapping or other apps on the system
     22        - Allow certain long-running procedures to be interruptible (Safari 2 import, reading icons in from disk when trying to quit, etc)
     23
     24        This will have a noticeable effect on current Safari 2 and Safari 3 beta browsers, including icons not appearing in bookmarks, history,
     25        or the location field the first time they're asked for, as current released Safari's don't properly listen for these async notifations. 
     26        The second time such a menu or view is brought up, the icon should be there.
     27
     28        Additionally this includes a SQLite schema change which will be a lot more efficient but will result in the loss of current SQLite icons.
     29        Converting from Safari 2 style icons will still work.
     30
     31        WebCore, welcome to multi-threadedness
     32
     33        * WebCore.exp:
     34        * WebCore.xcodeproj/project.pbxproj:
     35        * WebCore.vcproj/WebCore.vcproj:
     36
     37        * loader/DocumentLoader.cpp:
     38        (WebCore::DocumentLoader::iconLoadDecisionAvailable): Called when an Icon becomes available that was requested by this
     39          DocumentLoader (to support the webView:didReceiveIcon: delegate call in WebKit)
     40        * loader/DocumentLoader.h:
     41
     42        * loader/FrameLoader.cpp:
     43        (WebCore::FrameLoader::iconLoadDecisionAvailable): Called from the DocumentLoaders who get notified - if the FrameLoader
     44          ends up not caring because the WebView has transitioned to a new page, nothing occurs.  Otherwise, the FrameLoader possibly
     45          starts it Icon Loader and possibly sends the webView:didReceiveIcon: delegate call
     46        (WebCore::FrameLoader::startIconLoader): Instead of "Yes, load the icon now" or "No, don't load it" there is a third possibility -
     47          "You might be asked to load your icon later."  Add supporting logic for receiving this state, and being called a second time
     48          when the load decision is finally available.
     49        * loader/FrameLoader.h:
     50
     51        * loader/FrameLoaderClient.h: Added "registerForIconNotification" which is a way to tell WebViews "The icon you are interested in might
     52          become available via the generic WebIconDatabaseDidAddIconNotification instead of a targeted delegate call"
     53          A WebView can then receive the generic notification and pass on it's own targeted delegate call.
     54
     55        * loader/icon/IconDataCache.cpp: Removed.
     56        * loader/icon/IconDataCache.h: Removed.
     57
     58        * loader/icon/IconDatabase.cpp:
     59        (WebCore::urlForLogging): Cut a URL down in length for sane logging and debugging
     60        (WebCore::defaultClient): Return the default, empty IconDatabaseClient incase the API doesn't set one.
     61
     62        Following block of methods are for the Main thread's usage -
     63        (WebCore::IconDatabase::setClient):
     64        (WebCore::makeAllDirectories): Small optimization that checks to see if the entire path exists already, and doesn't try to loop
     65          through each patch component if the full path is already present
     66        (WebCore::IconDatabase::open): Makes all directories to the target path and kicks off the background thread - nothing more.
     67        (WebCore::IconDatabase::close): Signals the thread to quit and waits for it to do so
     68        (WebCore::IconDatabase::removeAllIcons): Purge the icon database
     69        (WebCore::IconDatabase::iconForPageURL):
     70        (WebCore::IconDatabase::readIconForPageURLFromDisk):
     71        (WebCore::IconDatabase::iconURLForPageURL):
     72        (WebCore::IconDatabase::defaultIcon):
     73        (WebCore::IconDatabase::retainIconForPageURL):
     74        (WebCore::IconDatabase::releaseIconForPageURL):
     75        (WebCore::IconDatabase::setIconDataForIconURL):
     76        (WebCore::IconDatabase::setIconURLForPageURL):
     77        (WebCore::IconDatabase::loadDecisionForIconURL): Determine if an icon loader should load now.  If the decision is "maybe later", then
     78          mark the DocumentLoader to be notified later when the final decision is available.
     79        (WebCore::IconDatabase::iconDataKnownForIconURL): Determine if the actual image data has been read from disk (or set from the loader) for
     80          icon URL in question
     81        (WebCore::IconDatabase::setEnabled):
     82        (WebCore::IconDatabase::isEnabled):
     83        (WebCore::IconDatabase::setPrivateBrowsingEnabled):
     84        (WebCore::IconDatabase::isPrivateBrowsingEnabled):
     85        (WebCore::IconDatabase::delayDatabaseCleanup): Restore this method from a year ago, as asynchronous pruning of icons can now occur on a
     86          background thread.
     87        (WebCore::IconDatabase::allowDatabaseCleanup):
     88        (WebCore::IconDatabase::checkIntegrityBeforeOpening):
     89        (WebCore::IconDatabase::pageURLMappingCount):
     90        (WebCore::IconDatabase::retainedPageURLCount):
     91        (WebCore::IconDatabase::iconRecordCount):
     92        (WebCore::IconDatabase::iconRecordCountWithData):
     93        (WebCore::IconDatabase::IconDatabase):
     94        (WebCore::IconDatabase::~IconDatabase):
     95        (WebCore::IconDatabase::notifyPendingLoadDecisions): Tell all the registered DocumentLoaders "Hey, we've read in all URL mappings from disk,
     96          so check to see if you are interested in any of them"
     97        (WebCore::IconDatabase::notifyPendingLoadDecisionsInternal):
     98        (WebCore::IconDatabase::wakeSyncThread): Wake the sync thread, if it is idle
     99        (WebCore::IconDatabase::scheduleOrDeferSyncTimer): Even though we're on a background thread, we still defer writing out to disk during
     100          periods of high activity
     101        (WebCore::IconDatabase::syncTimerFired): Call wakeSyncThread()
     102
     103        Following block of methods may be used by either thread -
     104        (WebCore::IconDatabase::isOpen):
     105        (WebCore::IconDatabase::databasePath):
     106        (WebCore::IconDatabase::defaultDatabaseFilename):
     107        (WebCore::IconDatabase::getOrCreateIconRecord):
     108        (WebCore::IconDatabase::getOrCreatePageURLRecord):
     109
     110        Following block of methods are used by the secondary thread only -
     111        (WebCore::IconDatabase::importIconURLForPageURL): For the Safari 2 import procedure - write a URL mapping directly out to disk
     112        (WebCore::IconDatabase::importIconDataForIconURL): For the Safari 2 import procedure - write an Icon directly out to disk
     113        (WebCore::IconDatabase::shouldStopThreadActivity): To check and see if the thread should stop what it is doing now to do something
     114          more important (such as quit, or delete all icons)
     115        (WebCore::IconDatabase::iconDatabaseSyncThreadStart):
     116        (WebCore::IconDatabase::iconDatabaseSyncThread): Entry point for the background thread
     117        (WebCore::databaseVersionNumber):
     118        (WebCore::isValidDatabase):
     119        (WebCore::createDatabaseTables):
     120        (WebCore::IconDatabase::performOpenInitialization): Open and validate the SQLite database, making sure it's schema jives with what
     121          is expected
     122        (WebCore::IconDatabase::checkIntegrity):
     123        (WebCore::IconDatabase::performURLImport): Import all the Page URL -> Icon URL mappings from the database.  Done "1st thing" on startup,
     124          this is necessary to be able to give the loader decisions about whether or not it should load icons from the network
     125        (WebCore::IconDatabase::syncThreadMainLoop): Main loop - sleeps until woken up, then does a read cycle and a write cycle until both cycles
     126          do no work - then it goes back to sleep.
     127        (WebCore::IconDatabase::readFromDatabase): Reads icons from the database that clients are waiting on
     128        (WebCore::IconDatabase::writeToDatabase): Writes any changes page -> icon url mappings to disk, as well as any new image data that has
     129          been received from the loader
     130        (WebCore::IconDatabase::pruneUnretainedIcons): Done only once, and only after the first write to the database, this procedure removes all
     131          icons and page URLs from disk that haven't been retained by any client.  Note that the prune can be delayed by utilizing delayDatabaseCleanup()
     132        (WebCore::IconDatabase::checkForDanglingPageURLs): Usually part of the prune procedure, prunes any pages who point to icons that no longer exist
     133          in the database
     134        (WebCore::IconDatabase::removeAllIconsOnThread): Completely purge both the on-disk and in memory records of all icons
     135        (WebCore::IconDatabase::deleteAllPreparedStatements): Part of removeAllIcons and the thread cleanup procedure
     136        (WebCore::IconDatabase::cleanupSyncThread): Write out any last remaining writes to disk, close the database, and then end the thread
     137        (WebCore::IconDatabase::imported): Checks the DB to see if the Safari 2 import has occured
     138        (WebCore::IconDatabase::setImported): Sets the "Safari 2 imported" flag
     139        (WebCore::readySQLStatement):
     140        (WebCore::IconDatabase::setIconURLForPageURLInSQLDatabase): This and the following "SQLDatabase" suffixed methods are pretty self explanatory
     141        (WebCore::IconDatabase::setIconIDForPageURLInSQLDatabase):
     142        (WebCore::IconDatabase::removePageURLFromSQLDatabase):
     143        (WebCore::IconDatabase::getIconIDForIconURLFromSQLDatabase):
     144        (WebCore::IconDatabase::addIconURLToSQLDatabase):
     145        (WebCore::IconDatabase::getImageDataForIconURLFromSQLDatabase):
     146        (WebCore::IconDatabase::removeIconFromSQLDatabase):
     147        (WebCore::IconDatabase::writeIconSnapshotToSQLDatabase):
     148        * loader/icon/IconDatabase.h:
     149
     150        * loader/icon/IconDatabaseClient.h: Added.
     151        (WebCore::IconDatabaseClient::~IconDatabaseClient):
     152        (WebCore::IconDatabaseClient::performImport): Perform the Safari 2 import, implemented by WebKit
     153        (WebCore::IconDatabaseClient::dispatchDidRemoveAllIcons): Send the API notification
     154        (WebCore::IconDatabaseClient::dispatchDidAddIconForPageURL): Ditto
     155
     156        * loader/icon/IconDatabaseNone.cpp: Best attempt to keep non icon-DB platforms building
     157        (WebCore::IconDatabase::defaultDatabaseFilename):
     158        (WebCore::IconDatabase::readIconForPageURLFromDisk):
     159        (WebCore::IconDatabase::loadDecisionForIconURL):
     160        (WebCore::IconDatabase::iconDataKnownForIconURL):
     161        (WebCore::IconDatabase::setIconURLForPageURL):
     162        (WebCore::IconDatabase::isEnabled):
     163        (WebCore::IconDatabase::delayDatabaseCleanup):
     164        (WebCore::IconDatabase::allowDatabaseCleanup):
     165        (WebCore::IconDatabase::setClient):
     166
     167        * loader/icon/IconRecord.cpp: Added.
     168        (WebCore::IconRecord::IconRecord): IconRecord used to be "IconDataCache" - it is merely a container for the url, timestamp, and image for a site icon.
     169          It is Shared, and therefore ref counted - PageURLRecords are the owning containers.  This is a tricky way to track how many page urls are retaining
     170          an IconRecord and therefore tracking when we should try to get rid of one.
     171        (WebCore::IconRecord::~IconRecord):
     172        (WebCore::IconRecord::image):
     173        (WebCore::IconRecord::setImageData):
     174        (WebCore::IconRecord::loadImageFromResource):
     175        (WebCore::IconRecord::imageDataStatus): Return whether the image data hasn't been read yet, exists in memory, or is absent (site with no icon)
     176        (WebCore::IconRecord::snapshot): Returns a snapshot of the icon's data - url, timestamp, and image data - to be written to disk
     177        * loader/icon/IconRecord.h: Added.
     178        (WebCore::IconSnapshot::IconSnapshot):
     179        (WebCore::IconRecord::getTimestamp):
     180        (WebCore::IconRecord::setTimestamp):
     181        (WebCore::IconRecord::iconURL):
     182        (WebCore::IconRecord::retainingPageURLs):
     183
     184        * loader/icon/PageURLRecord.cpp: Added.
     185        (WebCore::PageURLRecord::PageURLRecord): PageURLRecord is fundamentally a pairing of a Page URL to an Icon.  It has manual ref counting for the sake
     186          of "retainIconForPageURL" and "releaseIconForPageURL", and can provide a quick snapshot of it's Page URL -> Icon URL mapping for writing to
     187          the database
     188        (WebCore::PageURLRecord::setIconRecord):
     189        (WebCore::PageURLRecord::snapshot):
     190        * loader/icon/PageURLRecord.h: Added.
     191        (WebCore::PageURLSnapshot::PageURLSnapshot):
     192        (WebCore::PageURLRecord::url):
     193        (WebCore::PageURLRecord::PageURLRecord::iconRecord):
     194        (WebCore::PageURLRecord::retain):
     195        (WebCore::PageURLRecord::release):
     196        (WebCore::PageURLRecord::retainCount):
     197
     198        * platform/SharedBuffer.cpp:
     199        (WebCore::SharedBuffer::copy): Added a deep copy method for the purposes of handing icon data across the thread boundary into the icon database
     200        * platform/SharedBuffer.h:
     201
     202        * platform/graphics/svg/SVGImageEmptyClients.h:
     203        (WebCore::SVGEmptyFrameLoaderClient::registerForIconNotification):
     204
     205        * platform/win/TemporaryLinkStubs.cpp:
     206        (WebCore::callOnMainThread): Only other IconDatabase utilizing platform - keep their build going
     207
    12082007-09-07  David Kilzer  <ddkilzer@apple.com>
    2209

  • trunk/WebCore/WebCore.exp

    r25360 r25439  
    212212__ZN7WebCore12IconDatabase10setEnabledEb
    213213__ZN7WebCore12IconDatabase11defaultIconERKNS_7IntSizeE
    214 __ZN7WebCore12IconDatabase11setImportedEb
    215214__ZN7WebCore12IconDatabase14iconForPageURLERKNS_6StringERKNS_7IntSizeEb
    216215__ZN7WebCore12IconDatabase14removeAllIconsEv
     216__ZN7WebCore12IconDatabase15iconRecordCountEv
    217217__ZN7WebCore12IconDatabase17iconURLForPageURLERKNS_6StringE
    218 __ZN7WebCore12IconDatabase18hasEntryForIconURLERKNS_6StringE
     218__ZN7WebCore12IconDatabase19pageURLMappingCountEv
     219__ZN7WebCore12IconDatabase20allowDatabaseCleanupEv
     220__ZN7WebCore12IconDatabase20delayDatabaseCleanupEv
    219221__ZN7WebCore12IconDatabase20retainIconForPageURLERKNS_6StringE
     222__ZN7WebCore12IconDatabase20retainedPageURLCountEv
    220223__ZN7WebCore12IconDatabase20setIconURLForPageURLERKNS_6StringES3_
    221224__ZN7WebCore12IconDatabase21releaseIconForPageURLERKNS_6StringE
    222225__ZN7WebCore12IconDatabase21setIconDataForIconURLEN3WTF10PassRefPtrINS_12SharedBufferEEERKNS_6StringE
    223226__ZN7WebCore12IconDatabase23defaultDatabaseFilenameEv
    224 __ZN7WebCore12IconDatabase23isIconExpiredForIconURLERKNS_6StringE
    225 __ZN7WebCore12IconDatabase23setHaveNoIconForIconURLERKNS_6StringE
     227__ZN7WebCore12IconDatabase23iconRecordCountWithDataEv
     228__ZN7WebCore12IconDatabase23importIconURLForPageURLERKNS_6StringES3_
     229__ZN7WebCore12IconDatabase24importIconDataForIconURLEN3WTF10PassRefPtrINS_12SharedBufferEEERKNS_6StringE
    226230__ZN7WebCore12IconDatabase25setPrivateBrowsingEnabledEb
    227231__ZN7WebCore12IconDatabase27checkIntegrityBeforeOpeningEv
    228232__ZN7WebCore12IconDatabase4openERKNS_6StringE
    229233__ZN7WebCore12IconDatabase5closeEv
    230 __ZN7WebCore12IconDatabase7isEmptyEv
    231 __ZN7WebCore12IconDatabase8importedEv
     234__ZN7WebCore12IconDatabase9setClientEPNS_18IconDatabaseClientE
    232235__ZN7WebCore12SharedBuffer10wrapNSDataEP6NSData
    233236__ZN7WebCore12SharedBuffer12createNSDataEv
    … …  
    353356__ZN7WebCore4Page16setDefersLoadingEb
    354357__ZN7WebCore4Page23clearUndoRedoOperationsEv
     358__ZN7WebCore4Page37setInLowQualityImageInterpolationModeEb
    355359__ZN7WebCore4Page6goBackEv
    356360__ZN7WebCore4Page8goToItemEPNS_11HistoryItemENS_13FrameLoadTypeE
    357361__ZN7WebCore4Page9goForwardEv
    358 __ZN7WebCore4Page37setInLowQualityImageInterpolationModeEb
    359 __ZNK7WebCore4Page34inLowQualityImageInterpolationModeEv
    360362__ZN7WebCore4PageC1EPNS_12ChromeClientEPNS_17ContextMenuClientEPNS_12EditorClientEPNS_10DragClientEPNS_15InspectorClientE
    361363__ZN7WebCore4PageD1Ev
    … …  
    552554__ZNK7WebCore12EventHandler20currentKeyboardEventEv
    553555__ZNK7WebCore12IconDatabase12databasePathEv
    554 __ZNK7WebCore12IconDatabase15iconRecordCountEv
    555 __ZNK7WebCore12IconDatabase19pageURLMappingCountEv
    556 __ZNK7WebCore12IconDatabase20retainedPageURLCountEv
    557 __ZNK7WebCore12IconDatabase23iconRecordCountWithDataEv
    558 __ZNK7WebCore12IconDatabase7enabledEv
     556__ZNK7WebCore12IconDatabase24shouldStopThreadActivityEv
     557__ZNK7WebCore12IconDatabase9isEnabledEv
    559558__ZNK7WebCore12RenderObject25backslashAsCurrencySymbolEv
    560559__ZNK7WebCore12SharedBuffer4dataEv
    … …  
    619618__ZNK7WebCore4KURL4pathEv
    620619__ZNK7WebCore4KURL8getNSURLEv
     620__ZNK7WebCore4Page34inLowQualityImageInterpolationModeEv
    621621__ZNK7WebCore5Frame10isFrameSetEv
    622622__ZNK7WebCore5Frame12eventHandlerEv

  • trunk/WebCore/WebCore.vcproj/WebCore.vcproj

    r25088 r25439  
    5959                                DisableSpecificWarnings="4099;4100;4127;4138;4189;4244;4510;4512;4610;4706;4800;4996;4355;4291;4068;6011;6031;6211;6246;6255;6387"
    6060                                ForcedIncludeFiles="WebCorePrefix.h"
    61                                 EnablePREfast="$(EnablePREfast)"
    6261                        />
    6362                        <Tool
    … …  
    132131                                DisableSpecificWarnings="4099;4100;4127;4138;4189;4244;4510;4512;4610;4706;4800;4996;4355;4291;4068;6011;6031;6211;6246;6255;6387"
    133132                                ForcedIncludeFiles="WebCorePrefix.h"
    134                                 EnablePREfast="$(EnablePREfast)"
    135133                        />
    136134                        <Tool
    … …  
    207205                                DisableSpecificWarnings="4099;4100;4127;4138;4189;4244;4510;4512;4610;4706;4800;4996;4355;4291;4068;6011;6031;6211;6246;6255;6387"
    208206                                ForcedIncludeFiles="WebCorePrefix.h"
    209                                 EnablePREfast="$(EnablePREfast)"
    210207                        />
    211208                        <Tool
    … …  
    26782675                                >
    26792676                        </File>
    2680             <File
     2677                        <File
    26812678                                RelativePath="..\loader\FTPDirectoryDocument.cpp"
    26822679                                >
    2683             </File>
    2684             <File
     2680                        </File>
     2681                        <File
    26852682                                RelativePath="..\loader\FTPDirectoryDocument.h"
    26862683                                >
    2687             </File>
    2688             <File
     2684                        </File>
     2685                        <File
    26892686                                RelativePath="..\loader\FTPDirectoryParser.cpp"
    26902687                                >
    2691             </File>
    2692             <File
     2688                        </File>
     2689                        <File
    26932690                                RelativePath="..\loader\FTPDirectoryParser.h"
    26942691                                >
    2695             </File>
     2692                        </File>
    26962693                        <File
    26972694                                RelativePath="..\loader\ImageDocument.cpp"
    … …  
    28102807                                </File>
    28112808                                <File
    2812                                         RelativePath="..\loader\icon\IconDataCache.cpp"
    2813                                         >
    2814                                 </File>
    2815                                 <File
    2816                                         RelativePath="..\loader\icon\IconDataCache.h"
     2809                                        RelativePath="..\loader\icon\IconDatabaseClient.h"
    28172810                                        >
    28182811                                </File>
    … …  
    28232816                                <File
    28242817                                        RelativePath="..\loader\icon\IconLoader.h"
     2818                                        >
     2819                                </File>
     2820                                <File
     2821                                        RelativePath="..\loader\icon\IconRecord.cpp"
     2822                                        >
     2823                                </File>
     2824                                <File
     2825                                        RelativePath="..\loader\icon\IconRecord.h"
     2826                                        >
     2827                                </File>
     2828                                <File
     2829                                        RelativePath="..\loader\icon\PageURLRecord.cpp"
     2830                                        >
     2831                                </File>
     2832                                <File
     2833                                        RelativePath="..\loader\icon\PageURLRecord.h"
    28252834                                        >
    28262835                                </File>
    … …  
    28802889                        <File
    28812890                                RelativePath="..\platform\AtomicStringImpl.h"
     2891                                >
     2892                        </File>
     2893                        <File
     2894                                RelativePath="..\platform\AutodrainedPool.h"
    28822895                                >
    28832896                        </File>
    … …  
    33843397                        <File
    33853398                                RelativePath="..\platform\TextStyle.h"
     3399                                >
     3400                        </File>
     3401                        <File
     3402                                RelativePath="..\platform\Threading.h"
    33863403                                >
    33873404                        </File>
    … …  
    35293546                                        >
    35303547                                </File>
    3531         <File
     3548                                <File
    35323549                                        RelativePath="..\platform\win\Language.cpp"
    35333550                                        >
    3534         </File>
    3535         <File
     3551                                </File>
     3552                                <File
    35363553                                        RelativePath="..\platform\win\MIMETypeRegistryWin.cpp"
    35373554                                        >
    … …  
    35693586                                        >
    35703587                                </File>
    3571                 <File
     3588                                <File
    35723589                                        RelativePath="..\platform\win\SharedBufferWin.cpp"
    35733590                                        >
    3574                 </File>
    3575                 <File
     3591                                </File>
     3592                                <File
    35763593                                        RelativePath="..\platform\win\SharedTimerWin.cpp"
    35773594                                        >

  • trunk/WebCore/WebCore.xcodeproj/project.pbxproj

    r25421 r25439  
    342342                51741D110B07259A00ED442C /* HistoryItem.h in Headers */ = {isa = PBXBuildFile; fileRef = 51741D0D0B07259A00ED442C /* HistoryItem.h */; settings = {ATTRIBUTES = (Private, ); }; };
    343343                51741D120B07259A00ED442C /* HistoryItem.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 51741D0E0B07259A00ED442C /* HistoryItem.cpp */; };
    344                 5186C0560A9C21470034FE94 /* IconDataCache.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 5186C0550A9C21470034FE94 /* IconDataCache.cpp */; };
    345344                51AA3F6F0BD5AA9E00892971 /* ResourceLoaderMac.mm in Sources */ = {isa = PBXBuildFile; fileRef = 51AA3F6E0BD5AA9E00892971 /* ResourceLoaderMac.mm */; };
    346345                51C81B890C4422F70019ECE3 /* FTPDirectoryParser.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 51C81B870C4422F70019ECE3 /* FTPDirectoryParser.cpp */; };
    … …  
    355354                51E1ECB30C91C55600DC255B /* AutodrainedPool.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E1ECB10C91C55600DC255B /* AutodrainedPool.h */; };
    356355                51E1ECB40C91C55600DC255B /* Threading.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E1ECB20C91C55600DC255B /* Threading.h */; };
     356                51E1ECBE0C91C90400DC255B /* IconDatabaseClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E1ECB80C91C90400DC255B /* IconDatabaseClient.h */; };
     357                51E1ECC00C91C90400DC255B /* IconRecord.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 51E1ECBA0C91C90400DC255B /* IconRecord.cpp */; };
     358                51E1ECC10C91C90400DC255B /* IconRecord.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E1ECBB0C91C90400DC255B /* IconRecord.h */; };
     359                51E1ECC20C91C90400DC255B /* PageURLRecord.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 51E1ECBC0C91C90400DC255B /* PageURLRecord.cpp */; };
     360                51E1ECC30C91C90400DC255B /* PageURLRecord.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E1ECBD0C91C90400DC255B /* PageURLRecord.h */; };
    357361                51E4ADB60C42B4CF0042BC55 /* FTPDirectoryDocument.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 51E4ADB20C42B4CF0042BC55 /* FTPDirectoryDocument.cpp */; };
    358362                51E4ADB70C42B4CF0042BC55 /* FTPDirectoryDocument.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E4ADB30C42B4CF0042BC55 /* FTPDirectoryDocument.h */; };
    … …  
    415419                656D37450ADBA5DE00A4554D /* WebPlugInStreamLoaderDelegate.h in Headers */ = {isa = PBXBuildFile; fileRef = 656D372D0ADBA5DE00A4554D /* WebPlugInStreamLoaderDelegate.h */; settings = {ATTRIBUTES = (Private, ); }; };
    416420                656D37480ADBA5DE00A4554D /* SubresourceLoader.h in Headers */ = {isa = PBXBuildFile; fileRef = 656D37300ADBA5DE00A4554D /* SubresourceLoader.h */; settings = {ATTRIBUTES = (Private, ); }; };
    417                 657429170A9C2D0B00C52C97 /* IconDataCache.h in Headers */ = {isa = PBXBuildFile; fileRef = 657429140A9C2D0B00C52C97 /* IconDataCache.h */; };
    418421                657429180A9C2D0B00C52C97 /* SQLStatement.h in Headers */ = {isa = PBXBuildFile; fileRef = 657429150A9C2D0B00C52C97 /* SQLStatement.h */; };
    419422                657429190A9C2D0B00C52C97 /* SQLTransaction.h in Headers */ = {isa = PBXBuildFile; fileRef = 657429160A9C2D0B00C52C97 /* SQLTransaction.h */; };
    … …  
    35973600                51741D0D0B07259A00ED442C /* HistoryItem.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = HistoryItem.h; sourceTree = "<group>"; };
    35983601                51741D0E0B07259A00ED442C /* HistoryItem.cpp */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.cpp.cpp; path = HistoryItem.cpp; sourceTree = "<group>"; };
    3599                 5186C0550A9C21470034FE94 /* IconDataCache.cpp */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.cpp.cpp; path = IconDataCache.cpp; sourceTree = "<group>"; };
    36003602                51AA3F6E0BD5AA9E00892971 /* ResourceLoaderMac.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = ResourceLoaderMac.mm; sourceTree = "<group>"; };
    36013603                51C81B870C4422F70019ECE3 /* FTPDirectoryParser.cpp */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.cpp.cpp; path = FTPDirectoryParser.cpp; sourceTree = "<group>"; };
    … …  
    36103612                51E1ECB10C91C55600DC255B /* AutodrainedPool.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AutodrainedPool.h; sourceTree = "<group>"; };
    36113613                51E1ECB20C91C55600DC255B /* Threading.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Threading.h; sourceTree = "<group>"; };
     3614                51E1ECB80C91C90400DC255B /* IconDatabaseClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = IconDatabaseClient.h; sourceTree = "<group>"; };
     3615                51E1ECBA0C91C90400DC255B /* IconRecord.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = IconRecord.cpp; sourceTree = "<group>"; };
     3616                51E1ECBB0C91C90400DC255B /* IconRecord.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = IconRecord.h; sourceTree = "<group>"; };
     3617                51E1ECBC0C91C90400DC255B /* PageURLRecord.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PageURLRecord.cpp; sourceTree = "<group>"; };
     3618                51E1ECBD0C91C90400DC255B /* PageURLRecord.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PageURLRecord.h; sourceTree = "<group>"; };
    36123619                51E4ADB20C42B4CF0042BC55 /* FTPDirectoryDocument.cpp */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.cpp.cpp; path = FTPDirectoryDocument.cpp; sourceTree = "<group>"; };
    36133620                51E4ADB30C42B4CF0042BC55 /* FTPDirectoryDocument.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = FTPDirectoryDocument.h; sourceTree = "<group>"; };
    … …  
    36963703                656D372D0ADBA5DE00A4554D /* WebPlugInStreamLoaderDelegate.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = WebPlugInStreamLoaderDelegate.h; sourceTree = "<group>"; };
    36973704                656D37300ADBA5DE00A4554D /* SubresourceLoader.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = SubresourceLoader.h; sourceTree = "<group>"; };
    3698                 657429140A9C2D0B00C52C97 /* IconDataCache.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = IconDataCache.h; sourceTree = "<group>"; };
    36993705                657429150A9C2D0B00C52C97 /* SQLStatement.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = SQLStatement.h; sourceTree = "<group>"; };
    37003706                657429160A9C2D0B00C52C97 /* SQLTransaction.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = SQLTransaction.h; sourceTree = "<group>"; };
    … …  
    70857091                                5126E6B90A2E3B12005C29FA /* IconDatabase.cpp */,
    70867092                                5126E6BA0A2E3B12005C29FA /* IconDatabase.h */,
    7087                                 5186C0550A9C21470034FE94 /* IconDataCache.cpp */,
    7088                                 657429140A9C2D0B00C52C97 /* IconDataCache.h */,
     7093                                51E1ECB80C91C90400DC255B /* IconDatabaseClient.h */,
    70897094                                513F14510AB634C400094DDF /* IconLoader.cpp */,
    70907095                                513F14520AB634C400094DDF /* IconLoader.h */,
     7096                                51E1ECBA0C91C90400DC255B /* IconRecord.cpp */,
     7097                                51E1ECBB0C91C90400DC255B /* IconRecord.h */,
     7098                                51E1ECBC0C91C90400DC255B /* PageURLRecord.cpp */,
     7099                                51E1ECBD0C91C90400DC255B /* PageURLRecord.h */,
    70917100                                51D3EA130A3D24D300BADA35 /* SQLDatabase.cpp */,
    70927101                                51D3EA140A3D24D300BADA35 /* SQLDatabase.h */,
    … …  
    1085710866                                8518DCE90A9CC80D0091B7A6 /* DOMDOMImplementation.h in Headers */,
    1085810867                                8518DD780A9CF31B0091B7A6 /* DOMNamedNodeMap.h in Headers */,
    10859                                 657429170A9C2D0B00C52C97 /* IconDataCache.h in Headers */,
    1086010868                                657429180A9C2D0B00C52C97 /* SQLStatement.h in Headers */,
    1086110869                                657429190A9C2D0B00C52C97 /* SQLTransaction.h in Headers */,
    … …  
    1173111739                                51E1ECB30C91C55600DC255B /* AutodrainedPool.h in Headers */,
    1173211740                                51E1ECB40C91C55600DC255B /* Threading.h in Headers */,
     11741                                51E1ECBE0C91C90400DC255B /* IconDatabaseClient.h in Headers */,
     11742                                51E1ECC10C91C90400DC255B /* IconRecord.h in Headers */,
     11743                                51E1ECC30C91C90400DC255B /* PageURLRecord.h in Headers */,
    1173311744                        );
    1173411745                        runOnlyForDeploymentPostprocessing = 0;
    … …  
    1176911780                        isa = PBXProject;
    1177011781                        buildConfigurationList = 149C284308902B11008A9EFC /* Build configuration list for PBXProject "WebCore" */;
     11782                        compatibilityVersion = "Xcode 2.4";
    1177111783                        hasScannedForEncodings = 1;
    1177211784                        knownRegions = (
    … …  
    1252812540                                8518DCEA0A9CC80D0091B7A6 /* DOMDOMImplementation.mm in Sources */,
    1252912541                                8518DD790A9CF31B0091B7A6 /* DOMNamedNodeMap.mm in Sources */,
    12530                                 5186C0560A9C21470034FE94 /* IconDataCache.cpp in Sources */,
    1253112542                                AAC8DAB10AA1002000DC0907 /* SVGMetadataElement.cpp in Sources */,
    1253212543                                85DF2C5D0AA341F600AD64C5 /* DOMHTMLFormElement.mm in Sources */,
    … …  
    1319313204                                51E1ECAF0C91C54600DC255B /* AutodrainedPool.mm in Sources */,
    1319413205                                51E1ECB00C91C54600DC255B /* Threading.mm in Sources */,
     13206                                51E1ECC00C91C90400DC255B /* IconRecord.cpp in Sources */,
     13207                                51E1ECC20C91C90400DC255B /* PageURLRecord.cpp in Sources */,
    1319513208                        );
    1319613209                        runOnlyForDeploymentPostprocessing = 0;

  • trunk/WebCore/loader/DocumentLoader.cpp

    r23627 r25439  
    712712}
    713713
    714 }
     714void DocumentLoader::iconLoadDecisionAvailable()
     715{
     716    if (m_frame)
     717        m_frame->loader()->iconLoadDecisionAvailable();
     718}
     719
     720}

  • trunk/WebCore/loader/DocumentLoader.h

    r23677 r25439  
    3030#define DocumentLoader_h
    3131
     32#include "IconDatabase.h"
    3233#include "NavigationAction.h"
    3334#include "Shared.h"
    … …  
    137138        bool startLoadingMainResource(unsigned long identifier);
    138139        void cancelMainResourceLoad(const ResourceError&);
    139 
     140       
     141        void iconLoadDecisionAvailable();
     142       
    140143        bool isLoadingMainResource() const;
    141144        bool isLoadingSubresources() const;

  • trunk/WebCore/loader/FrameLoader.cpp

    r25430 r25439  
    10301030}
    10311031
     1032void FrameLoader::iconLoadDecisionAvailable()
     1033{
     1034    if (!m_mayLoadIconLater)
     1035        return;
     1036    LOG(IconDatabase, "FrameLoader %p was told a load decision is available for its icon", this);
     1037    startIconLoader();
     1038    m_mayLoadIconLater = false;
     1039}
     1040
    10321041void FrameLoader::startIconLoader()
    10331042{
    … …  
    10371046        return;
    10381047
    1039     if (!iconDatabase() || !iconDatabase()->enabled())
    1040         return;
    1041 
     1048    if (!iconDatabase() || !iconDatabase()->isEnabled())
     1049        return;
     1050   
    10421051    KURL url(iconURL());
    10431052    String urlString(url.url());
    … …  
    10451054        return;
    10461055
    1047     // If we already have an unexpired icon, we won't kick off a load but we *will* map the appropriate URLs to it
    1048     if (iconDatabase()->hasEntryForIconURL(urlString) && loadType() != FrameLoadTypeReload && !iconDatabase()->isIconExpiredForIconURL(urlString)) {
    1049         LOG(IconDatabase, "FrameLoader::startIconLoader() - Committing iconURL %s to database", urlString.ascii().data());
    1050         commitIconURLToIconDatabase(url);
    1051         return;
    1052     }
    1053 
     1056    // If we're not reloading and the icon database doesn't say to load now then bail before we actually start the load
     1057    if (loadType() != FrameLoadTypeReload) {
     1058        IconLoadDecision decision = iconDatabase()->loadDecisionForIconURL(urlString, m_documentLoader.get());
     1059        if (decision == IconLoadNo) {
     1060            LOG(IconDatabase, "FrameLoader::startIconLoader() - Told not to load this icon, committing iconURL %s to database for pageURL mapping", urlString.ascii().data());
     1061            commitIconURLToIconDatabase(url);
     1062           
     1063            // We were told not to load this icon - that means this icon is already known by the database
     1064            // If the icon data hasn't been read in from disk yet, kick off the read of the icon from the database to make sure someone
     1065            // has done it.  This is after registering for the notification so the WebView can call the appropriate delegate method.
     1066            // Otherwise if the icon data *is* available, and this icon load was previously deferred, we need to notifiy the delegate
     1067            if (!iconDatabase()->iconDataKnownForIconURL(urlString)) {
     1068                LOG(IconDatabase, "Told not to load icon %s but icon data is not yet available - registering for notification and requesting load from disk", urlString.ascii().data());
     1069                m_client->registerForIconNotification();
     1070                iconDatabase()->iconForPageURL(m_URL.url(), IntSize(0, 0));
     1071                iconDatabase()->iconForPageURL(originalRequestURL().url(), IntSize(0, 0));
     1072            } else if (m_mayLoadIconLater)
     1073                m_client->dispatchDidReceiveIcon();
     1074               
     1075            return;
     1076        }
     1077       
     1078        if (decision == IconLoadUnknown) {
     1079            // In this case, we may end up loading the icon later, but we still want to commit the icon url mapping to the database
     1080            // just in case we don't end up loading later - if we commit the mapping a second time after the load, that's no big deal
     1081            // We also tell the client to register for the notification that the icon is received now so it isn't missed in case the
     1082            // icon is later read in from disk
     1083            LOG(IconDatabase, "FrameLoader %p might load icon %s later", this, urlString.ascii().data());
     1084            m_mayLoadIconLater = true;   
     1085            m_client->registerForIconNotification();
     1086            commitIconURLToIconDatabase(url);
     1087            return;
     1088        }
     1089    }
     1090
     1091    // This is either a reload or the icon database said "yes, load the icon", so kick off the load!
    10541092    if (!m_iconLoader)
    10551093        m_iconLoader.set(IconLoader::create(m_frame).release());
     1094       
    10561095    m_iconLoader->startLoading();
    10571096}

  • trunk/WebCore/loader/FrameLoader.h

    r25430 r25439  
    438438        bool committingFirstRealLoad() const { return !m_creatingInitialEmptyDocument && !m_committedFirstRealDocumentLoad; }
    439439
     440        void iconLoadDecisionAvailable();
    440441    private:
    441442        PassRefPtr<HistoryItem> createHistoryItem(bool useOriginal);
    … …  
    609610
    610611        OwnPtr<IconLoader> m_iconLoader;
     612        bool m_mayLoadIconLater;
    611613
    612614        bool m_cancellingWithLoadInProgress;

  • trunk/WebCore/loader/FrameLoaderClient.h

    r25430 r25439  
    210210        virtual void didPerformFirstNavigation() const = 0; // "Navigation" here means a transition from one page to another that ends up in the back/forward list.
    211211       
     212        virtual void registerForIconNotification(bool listen = true) = 0;
     213       
    212214#if PLATFORM(MAC)
    213215        virtual NSCachedURLResponse* willCacheResponse(DocumentLoader*, unsigned long identifier, NSCachedURLResponse*) const = 0;

  • trunk/WebCore/loader/icon/IconDatabase.cpp

    r25348 r25439  
    2727#include "IconDatabase.h"
    2828
     29#include "AutodrainedPool.h"
    2930#include "CString.h"
     31#include "DocumentLoader.h"
    3032#include "FileSystem.h"
    31 #include "IconDataCache.h"
     33#include "IconDatabaseClient.h"
     34#include "IconRecord.h"
    3235#include "Image.h"
     36#include "IntSize.h"
     37#include "KURL.h"
    3338#include "Logging.h"
     39#include "PageURLRecord.h"
    3440#include "SQLStatement.h"
    3541#include "SQLTransaction.h"
    … …  
    4450#endif
    4551
     52// For methods that are meant to support API from the main thread - should not be called internally
     53#define ASSERT_NOT_SYNC_THREAD() ASSERT(!m_syncThreadRunning || !IS_ICON_SYNC_THREAD())
     54
     55// For methods that are meant to support the sync thread ONLY
     56#define IS_ICON_SYNC_THREAD() pthread_equal(pthread_self(), m_syncThread)
     57#define ASSERT_ICON_SYNC_THREAD() ASSERT(IS_ICON_SYNC_THREAD())
     58
    4659namespace WebCore {
    4760
    4861static IconDatabase* sharedIconDatabase = 0;
     62static int databaseCleanupCounter = 0;
    4963
    5064// This version number is in the DB and marks the current generation of the schema
    51 // Theoretically once the switch is flipped this should never change
    52 // Currently, an out-of-date schema causes the DB to be wiped and reset.  This isn't
     65// Currently, a mismatched schema causes the DB to be wiped and reset.  This isn't
    5366// so bad during development but in the future, we would need to write a conversion
    5467// function to advance older released schemas to "current"
    55 const int currentDatabaseVersion = 5;
    56 
    57 // Icons expire once a day
    58 const int iconExpirationTime = 60*60*24;
    59 // Absent icons are rechecked once a week
    60 const int missingIconExpirationTime = 60*60*24*7;
     68const int currentDatabaseVersion = 6;
     69
     70// Icons expire once every 4 days
     71const int iconExpirationTime = 60*60*24*4;
    6172
    6273const int updateTimerDelay = 5;
    … …  
    6475static bool checkIntegrityOnOpen = false;
    6576
    66 const String& IconDatabase::defaultDatabaseFilename()
    67 {
    68     static String defaultDatabaseFilename = "Icons.db";
    69     return defaultDatabaseFilename;
    70 }
    71 
    72 void IconDatabase::checkIntegrityBeforeOpening()
    73 {
    74     checkIntegrityOnOpen = true;
     77#ifndef NDEBUG
     78static String urlForLogging(const String& url)
     79{
     80    static unsigned urlTruncationLength = 120;
     81
     82    if (url.length() < urlTruncationLength)
     83        return url;
     84    return url.substring(0, urlTruncationLength) + "...";
     85}
     86#endif
     87
     88static IconDatabaseClient* defaultClient()
     89{
     90    static IconDatabaseClient* defaultClient = new IconDatabaseClient();
     91    return defaultClient;
    7592}
    7693
    … …  
    8299}
    83100
    84 IconDatabase::IconDatabase()
    85     : m_timeStampForIconURLStatement(0)
    86     , m_iconURLForPageURLStatement(0)
    87     , m_hasIconForIconURLStatement(0)
    88     , m_forgetPageURLStatement(0)
    89     , m_setIconIDForPageURLStatement(0)
    90     , m_getIconIDForIconURLStatement(0)
    91     , m_addIconForIconURLStatement(0)
    92     , m_imageDataForIconURLStatement(0)
    93     , m_importedStatement(0)
    94     , m_setImportedStatement(0)
    95     , m_currentDB(&m_mainDB)
    96     , m_defaultIconDataCache(0)
    97     , m_isEnabled(false)
    98     , m_privateBrowsingEnabled(false)
    99     , m_startupTimer(this, &IconDatabase::pruneUnretainedIconsOnStartup)
    100     , m_updateTimer(this, &IconDatabase::updateDatabase)
    101     , m_initialPruningComplete(false)
    102     , m_imported(false)
    103     , m_isImportedSet(false)
    104 {
    105    
     101// ************************
     102// *** Main Thread Only ***
     103// ************************
     104
     105void IconDatabase::setClient(IconDatabaseClient* client)
     106{
     107    // We don't allow a null client, because we never null check it anywhere in this code
     108    // Also don't allow a client change after the thread has already began
     109    // (setting the client should occur before the database is opened)
     110    ASSERT(client);
     111    ASSERT(!m_syncThreadRunning);
     112    if (!client || m_syncThreadRunning)
     113        return;
     114       
     115    m_client = client;
    106116}
    107117
    … …  
    113123        DWORD error = GetLastError();
    114124        if (error != ERROR_FILE_EXISTS && error != ERROR_ALREADY_EXISTS) {
    115             LOG_ERROR("Failed to create path %s", path.ascii().data());
     125            LOG_ERROR("Failed to create path %s", path.utf8().data());
    116126            return false;
    117127        }
    … …  
    119129#else
    120130    CString fullPath = path.utf8();
     131    if (!access(fullPath.data(), F_OK))
     132        return true;
     133       
    121134    char* p = fullPath.mutableData() + 1;
    122135    int length = fullPath.length();
    … …  
    141154bool IconDatabase::open(const String& databasePath)
    142155{
     156    ASSERT_NOT_SYNC_THREAD();
     157
    143158    if (!m_isEnabled)
    144159        return false;
    145        
     160
    146161    if (isOpen()) {
    147162        LOG_ERROR("Attempt to reopen the IconDatabase which is already open.  Must close it first.");
    148163        return false;
    149164    }
    150     
    151     // <rdar://problem/4730811> - Need to create the database path if it doesn't already exist
     165 
     166    // Need to create the database path if it doesn't already exist
    152167    makeAllDirectories(databasePath);
    153168   
    … …  
    166181#endif
    167182
    168     String journalFilename = dbFilename + "-journal";
     183    m_completeDatabasePath = dbFilename.copy();
     184   
     185    // Lock here as well as first thing in the thread so the tread doesn't actually commence until the pthread_create() call
     186    // completes and m_syncThreadRunning is properly set
     187    m_syncLock.lock();
     188    m_syncThreadRunning = !pthread_create(&m_syncThread, NULL, IconDatabase::iconDatabaseSyncThreadStart, this);
     189    m_syncLock.unlock();
     190
     191    return m_syncThreadRunning;
     192}
     193
     194void IconDatabase::close()
     195{
     196    ASSERT_NOT_SYNC_THREAD();
     197   
     198    if (m_syncThreadRunning) {
     199        // Set the flag to tell the sync thread to wrap it up
     200        m_threadTerminationRequested = true;
     201
     202        // Wake up the sync thread if it's waiting
     203        wakeSyncThread();
     204       
     205        // Wait for the sync thread to terminate
     206        if (pthread_join(m_syncThread, NULL) == EDEADLK)
     207            LOG_ERROR("m_syncThread was found to be deadlocked trying to quit");
     208    }
     209
     210    m_syncThreadRunning = false;   
     211    m_threadTerminationRequested = false;
     212    m_removeIconsRequested = false;
     213}
     214
     215void IconDatabase::removeAllIcons()
     216{
     217    ASSERT_NOT_SYNC_THREAD();
     218   
     219    if (!isOpen())
     220        return;
     221
     222    LOG(IconDatabase, "Requesting background thread to remove all icons");
     223 
     224    m_removeIconsRequested = true;
     225    wakeSyncThread();
     226}
     227
     228Image* IconDatabase::iconForPageURL(const String& pageURLOriginal, const IntSize& size, bool cache)
     229{   
     230    ASSERT_NOT_SYNC_THREAD();
     231   
     232    // pageURLOriginal can not be stored without being deep copied first. 
     233    // We should go our of our way to only copy it if we have to store it
     234   
     235    if (!isOpen())
     236        return defaultIcon(size);
     237               
     238    MutexLocker locker(m_urlAndIconLock);
     239   
     240    String pageURLCopy; // Creates a null string for easy testing
     241   
     242    PageURLRecord* pageRecord = m_pageURLToRecordMap.get(pageURLOriginal);
     243    if (!pageRecord) {
     244        pageURLCopy = pageURLOriginal.copy();
     245        pageRecord = getOrCreatePageURLRecord(pageURLCopy);
     246    }
     247   
     248    // If pageRecord is NULL, one of two things is true -
     249    // 1 - The initial url import is incomplete and this pageURL was marked to be notified once it is complete if an iconURL exists
     250    // 2 - The initial url import IS complete and this pageURL has no icon
     251    if (!pageRecord) {
     252        MutexLocker locker(m_pendingReadingLock);
     253       
     254        // Import is ongoing, there might be an icon.  In this case, register to be notified when the icon comes in
     255        // If we ever reach this condition, we know we've already made the pageURL copy
     256        if (!m_iconURLImportComplete)
     257            m_pageURLsInterestedInIcons.add(pageURLCopy);
     258       
     259        return 0;
     260    }
     261
     262    IconRecord* iconRecord = pageRecord->iconRecord();
     263   
     264    // If the initial URL import isn't complete, it's possible to have a PageURL record without an associated icon
     265    // In this case, the pageURL is already in the set to alert the client when the iconURL mapping is complete so
     266    // we can just bail now
     267    if (!m_iconURLImportComplete && !iconRecord)
     268        return 0;
     269   
     270    // The only way we should *not* have an icon record is if this pageURL is retained but has no icon yet - make sure of that
     271    ASSERT(iconRecord || m_retainedPageURLs.contains(pageURLOriginal));
     272   
     273    if (!iconRecord)
     274        return 0;
     275       
     276    // If it's a new IconRecord object that doesn't have its imageData set yet,
     277    // mark it to be read by the background thread
     278    if (iconRecord->imageDataStatus() == ImageDataStatusUnknown) {
     279        if (pageURLCopy.isNull())
     280            pageURLCopy = pageURLOriginal.copy();
     281   
     282        MutexLocker locker(m_pendingReadingLock);
     283        m_pageURLsInterestedInIcons.add(pageURLCopy);
     284        m_iconsPendingReading.add(iconRecord);
     285        wakeSyncThread();
     286        return 0;
     287    }
     288   
     289    // If the size parameter was (0, 0) that means the caller of this method just wanted the read from disk to be kicked off
     290    // and isn't actually interested in the image return value
     291    if (size == IntSize(0, 0))
     292        return 0;
     293       
     294    // PARANOID DISCUSSION: This method makes some assumptions.  It returns a WebCore::image which the icon database might dispose of at anytime in the future,
     295    // and Images aren't ref counted.  So there is no way for the client to guarantee continued existence of the image.
     296    // This has *always* been the case, but in practice clients would always create some other platform specific representation of the image
     297    // and drop the raw Image*.  On Mac an NSImage, and on windows drawing into an HBITMAP.
     298    // The async aspect adds a huge question - what if the image is deleted before the platform specific API has a chance to create its own
     299    // representation out of it?
     300    // If an image is read in from the icondatabase, we do *not* overwrite any image data that exists in the in-memory cache. 
     301    // This is because we make the assumption that anything in memory is newer than whatever is in the database.
     302    // So the only time the data will be set from the second thread is when it is INITIALLY being read in from the database, but we would never
     303    // delete the image on the secondary thread if the image already exists.
     304    return iconRecord->image(size);
     305}
     306
     307void IconDatabase::readIconForPageURLFromDisk(const String& pageURL)
     308{
     309    // The effect of asking for an Icon for a pageURL automatically queues it to be read from disk
     310    // if it hasn't already been set in memory.  The special IntSize (0, 0) is a special way of telling
     311    // that method "I don't care about the actual Image, i just want you to make sure you're getting it from disk.
     312    iconForPageURL(pageURL, IntSize(0,0));
     313}
     314
     315String IconDatabase::iconURLForPageURL(const String& pageURLOriginal)
     316{   
     317    ASSERT_NOT_SYNC_THREAD();
     318       
     319    // Cannot do anything with pageURLOriginal that would end up storing it without deep copying first
     320    // Also, in the case we have a real answer for the caller, we must deep copy that as well
     321   
     322    if (!isOpen() || pageURLOriginal.isEmpty())
     323        return String();
     324       
     325    MutexLocker locker(m_urlAndIconLock);
     326   
     327    PageURLRecord* pageRecord = m_pageURLToRecordMap.get(pageURLOriginal);
     328    if (!pageRecord)
     329        pageRecord = getOrCreatePageURLRecord(pageURLOriginal.copy());
     330   
     331    // If pageRecord is NULL, one of two things is true -
     332    // 1 - The initial url import is incomplete and this pageURL has already been marked to be notified once it is complete if an iconURL exists
     333    // 2 - The initial url import IS complete and this pageURL has no icon
     334    if (!pageRecord)
     335        return String();
     336   
     337    // Possible the pageRecord is around because it's a retained pageURL with no iconURL, so we have to check
     338    return pageRecord->iconRecord() ? pageRecord->iconRecord()->iconURL().copy() : String();
     339}
     340
     341Image* IconDatabase::defaultIcon(const IntSize& size)
     342{
     343    ASSERT_NOT_SYNC_THREAD();
     344
     345    if (!m_defaultIconRecord) {
     346        m_defaultIconRecord = new IconRecord("urlIcon");
     347        m_defaultIconRecord->loadImageFromResource("urlIcon");
     348    }
     349   
     350    return m_defaultIconRecord->image(size);
     351}
     352
     353
     354void IconDatabase::retainIconForPageURL(const String& pageURLOriginal)
     355{   
     356    ASSERT_NOT_SYNC_THREAD();
     357   
     358    // Cannot do anything with pageURLOriginal that would end up storing it without deep copying first
     359   
     360    if (!isOpen() || pageURLOriginal.isEmpty())
     361        return;
     362       
     363    MutexLocker locker(m_urlAndIconLock);
     364
     365    PageURLRecord* record = m_pageURLToRecordMap.get(pageURLOriginal);
     366   
     367    String pageURL;
     368   
     369    if (!record) {
     370        pageURL = pageURLOriginal.copy();
     371
     372        record = new PageURLRecord(pageURL);
     373        m_pageURLToRecordMap.set(pageURL, record);
     374        m_retainedPageURLs.add(pageURL);
     375    }
     376   
     377    if (record->retain()) {
     378        // If we read the iconURLs yet, we want to avoid any pageURL->iconURL lookups and the pageURLsPendingDeletion is moot,
     379        // so we bail here and skip those steps
     380        if (!m_iconURLImportComplete)
     381            return;
     382
     383        MutexLocker locker(m_pendingSyncLock);
     384        // If this pageURL waiting to be sync'ed, update the sync record
     385        // This saves us in the case where a page was ready to be deleted from the database but was just retained - so theres no need to delete it!
     386        if (!m_privateBrowsingEnabled && m_pageURLsPendingSync.contains(pageURLOriginal)) {
     387            if (pageURL.isNull())
     388                pageURL = pageURLOriginal.copy();
     389       
     390            LOG(IconDatabase, "Bringing %s back from the brink", pageURL.utf8().data());
     391            m_pageURLsPendingSync.set(pageURL, record->snapshot());
     392        }
     393    }
     394}
     395
     396void IconDatabase::releaseIconForPageURL(const String& pageURLOriginal)
     397{
     398    ASSERT_NOT_SYNC_THREAD();
     399       
     400    // Cannot do anything with pageURLOriginal that would end up storing it without deep copying first
     401   
     402    if (!isOpen() || pageURLOriginal.isEmpty())
     403        return;
     404   
     405    MutexLocker locker(m_urlAndIconLock);
     406
     407    // Check if this pageURL is actually retained
     408    if (!m_retainedPageURLs.contains(pageURLOriginal)) {
     409        LOG_ERROR("Attempting to release icon for URL %s which is not retained", urlForLogging(pageURLOriginal).utf8().data());
     410        return;
     411    }
     412   
     413    // Get its retain count - if it's retained, we'd better have a PageURLRecord for it
     414    PageURLRecord* pageRecord = m_pageURLToRecordMap.get(pageURLOriginal);
     415    ASSERT(pageRecord);
     416    LOG(IconDatabase, "Releasing pageURL %s to a retain count of %i", urlForLogging(pageURLOriginal).utf8().data(), pageRecord->retainCount() - 1);
     417    ASSERT(pageRecord->retainCount() > 0);
     418       
     419    // If it still has a positive retain count, store the new count and bail
     420    if (pageRecord->release())
     421        return;
     422       
     423    // This pageRecord has now been fully released.  Do the appropriate cleanup
     424    LOG(IconDatabase, "No more retainers for PageURL %s", urlForLogging(pageURLOriginal).utf8().data());
     425    m_pageURLToRecordMap.remove(pageURLOriginal);
     426    m_retainedPageURLs.remove(pageURLOriginal);       
     427   
     428    // Grab the iconRecord for later use (and do a sanity check on it for kicks)
     429    IconRecord* iconRecord = pageRecord->iconRecord();
     430   
     431    ASSERT(!iconRecord || (iconRecord && m_iconURLToRecordMap.get(iconRecord->iconURL()) == iconRecord));
     432
     433    {
     434        MutexLocker locker(m_pendingReadingLock);
     435       
     436        // Since this pageURL is going away, there's no reason anyone would ever be interested in its read results   
     437        if (!m_iconURLImportComplete)
     438            m_pageURLsPendingImport.remove(pageURLOriginal);
     439        m_pageURLsInterestedInIcons.remove(pageURLOriginal);
     440       
     441        // If this icon is down to it's last retainer, we don't care about reading it in from disk anymore
     442        if (iconRecord && iconRecord->hasOneRef()) {
     443            m_iconURLToRecordMap.remove(iconRecord->iconURL());
     444            m_iconsPendingReading.remove(iconRecord);
     445        }
     446    }
     447   
     448    // Mark stuff for deletion from the database only if we're not in private browsing
     449    if (!m_privateBrowsingEnabled) {
     450        MutexLocker locker(m_pendingSyncLock);
     451        m_pageURLsPendingSync.set(pageURLOriginal.copy(), pageRecord->snapshot(true));
     452   
     453        // If this page is the last page to refer to a particular IconRecord, that IconRecord needs to
     454        // be marked for deletion
     455        if (iconRecord && iconRecord->hasOneRef())
     456            m_iconsPendingSync.set(iconRecord->iconURL(), iconRecord->snapshot(true));
     457    }
     458   
     459    delete pageRecord;
     460
     461    scheduleOrDeferSyncTimer();
     462}
     463
     464void IconDatabase::setIconDataForIconURL(PassRefPtr<SharedBuffer> dataOriginal, const String& iconURLOriginal)
     465{   
     466    ASSERT_NOT_SYNC_THREAD();
     467   
     468    // Cannot do anything with dataOriginal or iconURLOriginal that would end up storing them without deep copying first
     469   
     470    if (!isOpen() || iconURLOriginal.isEmpty())
     471        return;
     472   
     473    RefPtr<SharedBuffer> data = dataOriginal ? dataOriginal->copy() : 0;
     474    String iconURL = iconURLOriginal.copy();
     475   
     476    Vector<String> pageURLs;
     477    {
     478        MutexLocker locker(m_urlAndIconLock);
     479   
     480        // If this icon was pending a read, remove it from that set because this new data should override what is on disk
     481        IconRecord* icon = m_iconURLToRecordMap.get(iconURL);
     482        if (icon) {
     483            MutexLocker locker(m_pendingReadingLock);
     484            m_iconsPendingReading.remove(icon);
     485        } else
     486            icon = getOrCreateIconRecord(iconURL);
     487   
     488        // Update the data and set the time stamp
     489        icon->setImageData(data);
     490        icon->setTimestamp((int)currentTime());
     491       
     492        // Copy the current retaining pageURLs - if any - to notify them of the change
     493        pageURLs.appendRange(icon->retainingPageURLs().begin(), icon->retainingPageURLs().end());
     494       
     495        // Mark the IconRecord as requiring an update to the database only if private browsing is disabled
     496        if (!m_privateBrowsingEnabled) {
     497            MutexLocker locker(m_pendingSyncLock);
     498            m_iconsPendingSync.set(iconURL, icon->snapshot());
     499        }
     500    }
     501
     502    // Send notification out regarding all PageURLs that retain this icon
     503    // But not if we're on the sync thread because that implies this mapping
     504    // comes from the initial import which we don't want notifications for
     505    if (!IS_ICON_SYNC_THREAD()) {
     506        // Start the timer to commit this change - or further delay the timer if it was already started
     507        scheduleOrDeferSyncTimer();
     508
     509        // Informal testing shows that draining the autorelease pool every 25 iterations is about as low as we can go
     510        // before performance starts to drop off, but we don't want to increase this number because then accumulated memory usage will go up       
     511        AutodrainedPool pool(25);
     512
     513        for (unsigned i = 0; i < pageURLs.size(); ++i) {
     514            LOG(IconDatabase, "Dispatching notification that retaining pageURL %s has a new icon", urlForLogging(pageURLs[i]).utf8().data());
     515            m_client->dispatchDidAddIconForPageURL(pageURLs[i]);
     516
     517            pool.cycle();
     518        }
     519    }
     520}
     521
     522void IconDatabase::setIconURLForPageURL(const String& iconURLOriginal, const String& pageURLOriginal)
     523{   
     524    ASSERT_NOT_SYNC_THREAD();
     525
     526    // Cannot do anything with iconURLOriginal or pageURLOriginal that would end up storing them without deep copying first
     527   
     528    ASSERT(!iconURLOriginal.isEmpty());
     529       
     530    if (!isOpen() || pageURLOriginal.isEmpty())
     531        return;
     532   
     533    String iconURL, pageURL;
     534   
     535    {
     536        MutexLocker locker(m_urlAndIconLock);
     537
     538        PageURLRecord* pageRecord = m_pageURLToRecordMap.get(pageURLOriginal);
     539       
     540        // If the urls already map to each other, bail.
     541        // This happens surprisingly often, and seems to cream iBench performance
     542        if (pageRecord && pageRecord->iconRecord() && pageRecord->iconRecord()->iconURL() == iconURLOriginal)
     543            return;
     544           
     545        pageURL = pageURLOriginal.copy();
     546        iconURL = iconURLOriginal.copy();
     547
     548        if (!pageRecord) {
     549            pageRecord = new PageURLRecord(pageURL);
     550            m_pageURLToRecordMap.set(pageURL, pageRecord);
     551        }
     552
     553        RefPtr<IconRecord> iconRecord = pageRecord->iconRecord();
     554
     555        // Otherwise, set the new icon record for this page
     556        IconRecord* newIconRecord = getOrCreateIconRecord(iconURL);
     557        pageRecord->setIconRecord(newIconRecord);
     558
     559        // If the current icon has only a single ref left, it is about to get wiped out.
     560        // Remove it from the in-memory records and don't bother reading it in from disk anymore
     561        if (iconRecord && iconRecord->hasOneRef()) {
     562            ASSERT(iconRecord->retainingPageURLs().size() == 0);
     563            LOG(IconDatabase, "Icon for icon url %s is about to be destroyed - removing mapping for it", urlForLogging(iconRecord->iconURL()).utf8().data());
     564            m_iconURLToRecordMap.remove(iconRecord->iconURL());
     565            MutexLocker locker(m_pendingReadingLock);
     566            m_iconsPendingReading.remove(iconRecord.get());
     567        }
     568       
     569        // And mark this mapping to be added to the database
     570        if (!m_privateBrowsingEnabled) {
     571            MutexLocker locker(m_pendingSyncLock);
     572            m_pageURLsPendingSync.set(pageURL, pageRecord->snapshot());
     573           
     574            // If the icon is on it's last ref, mark it for deletion
     575            if (iconRecord && iconRecord->hasOneRef())
     576                m_iconsPendingSync.set(iconRecord->iconURL(), iconRecord->snapshot(true));
     577        }
     578    }
     579
     580    // Since this mapping is new, send the notification out - but not if we're on the sync thread because that implies this mapping
     581    // comes from the initial import which we don't want notifications for
     582    if (!IS_ICON_SYNC_THREAD()) {
     583        // Start the timer to commit this change - or further delay the timer if it was already started
     584        scheduleOrDeferSyncTimer();
     585       
     586        LOG(IconDatabase, "Dispatching notification that we changed an icon mapping for url %s", urlForLogging(pageURL).utf8().data());
     587        AutodrainedPool pool;
     588        m_client->dispatchDidAddIconForPageURL(pageURL);
     589    }
     590}
     591
     592IconLoadDecision IconDatabase::loadDecisionForIconURL(const String& iconURL, DocumentLoader* notificationDocumentLoader)
     593{
     594    ASSERT_NOT_SYNC_THREAD();
     595
     596    if (!isOpen() || iconURL.isEmpty())
     597        return IconLoadNo;
     598   
     599    // If we have a IconRecord, it should also have its timeStamp marked because there is only two times when we create the IconRecord:
     600    // 1 - When we read the icon urls from disk, getting the timeStamp at the same time
     601    // 2 - When we get a new icon from the loader, in which case the timestamp is set at that time
     602    {
     603        MutexLocker locker(m_urlAndIconLock);
     604        if (IconRecord* icon = m_iconURLToRecordMap.get(iconURL)) {
     605            LOG(IconDatabase, "Found expiration time on a present icon based on existing IconRecord");
     606            return (int)currentTime() - icon->getTimestamp() > iconExpirationTime ? IconLoadYes : IconLoadNo;
     607        }
     608    }
     609   
     610    // If we don't have a record for it, but we *have* imported all iconURLs from disk, then we should load it now
     611    MutexLocker readingLocker(m_pendingReadingLock);
     612    if (m_iconURLImportComplete)
     613        return IconLoadYes;
     614       
     615    // Otherwise - since we refuse to perform I/O on the main thread to find out for sure - we return the answer that says
     616    // "You might be asked to load this later, so flag that"
     617    LOG(IconDatabase, "Don't know if we should load %s or not - adding %p to the set of document loaders waiting on a decision", iconURL.utf8().data(), notificationDocumentLoader);
     618    m_loadersPendingDecision.add(notificationDocumentLoader);   
     619
     620    return IconLoadUnknown;
     621}
     622
     623bool IconDatabase::iconDataKnownForIconURL(const String& iconURL)
     624{
     625    ASSERT_NOT_SYNC_THREAD();
     626   
     627    MutexLocker locker(m_urlAndIconLock);
     628    if (IconRecord* icon = m_iconURLToRecordMap.get(iconURL))
     629        return icon->imageDataStatus() != ImageDataStatusUnknown;
     630
     631    return false;
     632}
     633
     634void IconDatabase::setEnabled(bool enabled)
     635{
     636    ASSERT_NOT_SYNC_THREAD();
     637   
     638    if (!enabled && isOpen())
     639        close();
     640    m_isEnabled = enabled;
     641}
     642
     643bool IconDatabase::isEnabled() const
     644{
     645    ASSERT_NOT_SYNC_THREAD();
     646   
     647     return m_isEnabled;
     648}
     649
     650void IconDatabase::setPrivateBrowsingEnabled(bool flag)
     651{
     652    m_privateBrowsingEnabled = flag;
     653}
     654
     655bool IconDatabase::isPrivateBrowsingEnabled() const
     656{
     657    return m_privateBrowsingEnabled;
     658}
     659
     660void IconDatabase::delayDatabaseCleanup()
     661{
     662    ++databaseCleanupCounter;
     663    if (databaseCleanupCounter == 1)
     664        LOG(IconDatabase, "Database cleanup is now DISABLED");
     665}
     666
     667void IconDatabase::allowDatabaseCleanup()
     668{
     669    if (--databaseCleanupCounter < 0)
     670        databaseCleanupCounter = 0;
     671    if (databaseCleanupCounter == 0)
     672        LOG(IconDatabase, "Database cleanup is now ENABLED");
     673}
     674
     675void IconDatabase::checkIntegrityBeforeOpening()
     676{
     677    checkIntegrityOnOpen = true;
     678}
     679
     680size_t IconDatabase::pageURLMappingCount()
     681{
     682    MutexLocker locker(m_urlAndIconLock);
     683    return m_pageURLToRecordMap.size();
     684}
     685
     686size_t IconDatabase::retainedPageURLCount()
     687{
     688    MutexLocker locker(m_urlAndIconLock);
     689    return m_retainedPageURLs.size();
     690}
     691
     692size_t IconDatabase::iconRecordCount()
     693{
     694    MutexLocker locker(m_urlAndIconLock);
     695    return m_iconURLToRecordMap.size();
     696}
     697
     698size_t IconDatabase::iconRecordCountWithData()
     699{
     700    MutexLocker locker(m_urlAndIconLock);
     701    size_t result = 0;
     702   
     703    HashMap<String, IconRecord*>::iterator i = m_iconURLToRecordMap.begin();
     704    HashMap<String, IconRecord*>::iterator end = m_iconURLToRecordMap.end();
     705   
     706    for (; i != end; ++i)
     707        result += ((*i).second->imageDataStatus() == ImageDataStatusPresent);
     708           
     709    return result;
     710}
     711
     712IconDatabase::IconDatabase()
     713    : m_syncTimer(this, &IconDatabase::syncTimerFired)
     714    , m_syncThreadRunning(false)
     715    , m_defaultIconRecord(0)
     716    , m_isEnabled(false)
     717    , m_privateBrowsingEnabled(false)
     718    , m_threadTerminationRequested(false)
     719    , m_removeIconsRequested(false)
     720    , m_iconURLImportComplete(false)
     721    , m_initialPruningComplete(false)
     722    , m_client(defaultClient())
     723    , m_imported(false)
     724    , m_isImportedSet(false)
     725{
     726    ASSERT(pthread_main_np());
     727}
     728
     729IconDatabase::~IconDatabase()
     730{
     731    ASSERT_NOT_REACHED();
     732}
     733
     734void IconDatabase::notifyPendingLoadDecisions()
     735{   
     736    iconDatabase()->notifyPendingLoadDecisionsInternal();
     737}
     738
     739void IconDatabase::notifyPendingLoadDecisionsInternal()
     740{
     741    ASSERT_NOT_SYNC_THREAD();
     742   
     743    // This method should only be called upon completion of the initial url import from the database
     744    ASSERT(m_iconURLImportComplete);
     745    LOG(IconDatabase, "Notifying all DocumentLoaders that were waiting on a load decision for thier icons");
     746   
     747    HashSet<RefPtr<DocumentLoader> >::iterator i = m_loadersPendingDecision.begin();
     748    HashSet<RefPtr<DocumentLoader> >::iterator end = m_loadersPendingDecision.end();
     749   
     750    for (; i != end; ++i)
     751        if ((*i)->refCount() > 1)
     752            (*i)->iconLoadDecisionAvailable();
     753           
     754    m_loadersPendingDecision.clear();
     755}
     756
     757void IconDatabase::wakeSyncThread()
     758{
     759    MutexLocker locker(m_syncLock);
     760    m_syncCondition.signal();
     761}
     762
     763void IconDatabase::scheduleOrDeferSyncTimer()
     764{
     765    ASSERT_NOT_SYNC_THREAD();
     766   
     767    m_syncTimer.startOneShot(updateTimerDelay);
     768}
     769
     770void IconDatabase::syncTimerFired(Timer<IconDatabase>*)
     771{
     772    ASSERT_NOT_SYNC_THREAD();
     773    wakeSyncThread();
     774}
     775
     776// ******************
     777// *** Any Thread ***
     778// ******************
     779
     780bool IconDatabase::isOpen() const
     781{
     782    MutexLocker locker(m_syncLock);
     783    return m_syncDB.isOpen();
     784}
     785
     786String IconDatabase::databasePath() const
     787{
     788    MutexLocker locker(m_syncLock);
     789    return m_completeDatabasePath.copy();
     790}
     791
     792String IconDatabase::defaultDatabaseFilename()
     793{
     794    static String defaultDatabaseFilename = "Icons.db";
     795    return defaultDatabaseFilename.copy();
     796}
     797
     798// Unlike getOrCreatePageURLRecord(), getOrCreateIconRecord() does not mark the icon as "interested in import"
     799IconRecord* IconDatabase::getOrCreateIconRecord(const String& iconURL)
     800{
     801    // Clients of getOrCreateIconRecord() are required to acquire the m_urlAndIconLock before calling this method
     802    ASSERT(m_urlAndIconLock.tryLock() == EBUSY);
     803
     804    if (IconRecord* icon = m_iconURLToRecordMap.get(iconURL))
     805        return icon;
     806       
     807    IconRecord* newIcon = new IconRecord(iconURL);
     808    m_iconURLToRecordMap.set(iconURL, newIcon);
     809       
     810    return newIcon;   
     811}
     812
     813// This method retrieves the existing PageURLRecord, or creates a new one and marks it as "interested in the import" for later notification
     814PageURLRecord* IconDatabase::getOrCreatePageURLRecord(const String& pageURL)
     815{
     816    // Clients of getOrCreatePageURLRecord() are required to acquire the m_urlAndIconLock before calling this method
     817    ASSERT(m_urlAndIconLock.tryLock() == EBUSY);
     818
     819    PageURLRecord* pageRecord = m_pageURLToRecordMap.get(pageURL);
     820   
     821    MutexLocker locker(m_pendingReadingLock);
     822    if (!m_iconURLImportComplete) {
     823        // If the initial import of all URLs hasn't completed and we have no page record, we assume we *might* know about this later and create a record for it
     824        if (!pageRecord) {
     825            LOG(IconDatabase, "Creating new PageURLRecord for pageURL %s", urlForLogging(pageURL).utf8().data());
     826            pageRecord = new PageURLRecord(pageURL);
     827            m_pageURLToRecordMap.set(pageURL, pageRecord);
     828        }
     829
     830        // If the pageRecord for this page does not have an iconRecord attached to it, then it is a new pageRecord still awaiting the initial import
     831        // Mark the URL as "interested in the result of the import" then bail
     832        if (!pageRecord->iconRecord()) {
     833            m_pageURLsPendingImport.add(pageURL);
     834            return 0;
     835        }
     836    }
     837
     838    // We've done the initial import of all URLs known in the database.  If this record doesn't exist now, it never will   
     839     return pageRecord;
     840}
     841
     842
     843// ************************
     844// *** Sync Thread Only ***
     845// ************************
     846
     847void IconDatabase::importIconURLForPageURL(const String& iconURL, const String& pageURL)
     848{
     849    ASSERT_ICON_SYNC_THREAD();
     850   
     851    // This function is only for setting actual existing url mappings so assert that neither of these URLs are empty
     852    ASSERT(!iconURL.isEmpty() && !pageURL.isEmpty());
     853   
     854    setIconURLForPageURLInSQLDatabase(iconURL, pageURL);   
     855}
     856
     857void IconDatabase::importIconDataForIconURL(PassRefPtr<SharedBuffer> data, const String& iconURL)
     858{
     859    ASSERT_ICON_SYNC_THREAD();
     860   
     861    ASSERT(!iconURL.isEmpty());
     862
     863    writeIconSnapshotToSQLDatabase(IconSnapshot(iconURL, (int)currentTime(), data.get()));
     864}
     865
     866bool IconDatabase::shouldStopThreadActivity() const
     867{
     868    ASSERT_ICON_SYNC_THREAD();
     869   
     870    return m_threadTerminationRequested || m_removeIconsRequested;
     871}
     872
     873void* IconDatabase::iconDatabaseSyncThreadStart(void* vIconDatabase)
     874{   
     875    IconDatabase* iconDB = static_cast<IconDatabase*>(vIconDatabase);
     876   
     877    return iconDB->iconDatabaseSyncThread();
     878}
     879
     880void* IconDatabase::iconDatabaseSyncThread()
     881{
     882    // The call to create this thread might not complete before the thread actually starts, so we might fail this ASSERT_ICON_SYNC_THREAD() because the pointer
     883    // to our thread structure hasn't been filled in yet.
     884    // To fix this, the main thread acquires this lock before creating us, then releases the lock after creation is complete.  A quick lock/unlock cycle here will
     885    // prevent us from running before that call completes
     886    m_syncLock.lock();
     887    m_syncLock.unlock();
     888
     889    ASSERT_ICON_SYNC_THREAD();
     890   
     891    LOG(IconDatabase, "(THREAD) IconDatabase sync thread started");
     892
     893#ifndef NDEBUG
     894    double startTime = currentTime();
     895#endif
     896
     897    // Existence of a journal file is evidence of a previous crash/force quit and automatically qualifies
     898    // us to do an integrity check
     899    String journalFilename = m_completeDatabasePath + "-journal";
    169900    if (!checkIntegrityOnOpen)
    170901        checkIntegrityOnOpen = fileExists(journalFilename);
    171    
    172     // <rdar://problem/4707718> - If user's Icon directory is unwritable, Safari will crash at startup
    173     // Now, we'll see if we can open the on-disk database.  And, if we can't, we'll return false. 
    174     // WebKit will then ignore us and act as if the database is disabled
    175     if (!m_mainDB.open(dbFilename)) {
    176         LOG_ERROR("Unable to open icon database at path %s - %s", dbFilename.ascii().data(), m_mainDB.lastErrorMsg());
     902       
     903    {
     904        MutexLocker locker(m_syncLock);
     905        if (!m_syncDB.open(m_completeDatabasePath)) {
     906            LOG_ERROR("Unable to open icon database at path %s - %s", m_completeDatabasePath.utf8().data(), m_syncDB.lastErrorMsg());
     907            return 0;
     908        }
     909    }
     910   
     911    if (shouldStopThreadActivity())
     912        return syncThreadMainLoop();
     913       
     914#ifndef NDEBUG
     915    double timeStamp = currentTime();
     916    LOG(IconDatabase, "(THREAD) Open took %.4f seconds", timeStamp - startTime);
     917#endif   
     918
     919    performOpenInitialization();
     920    if (shouldStopThreadActivity())
     921        return syncThreadMainLoop();
     922       
     923#ifndef NDEBUG
     924    double newStamp = currentTime();
     925    LOG(IconDatabase, "(THREAD) performOpenInitialization() took %.4f seconds, now %.4f seconds from thread start", newStamp - timeStamp, newStamp - startTime);
     926    timeStamp = newStamp;
     927#endif
     928
     929    if (!imported()) {
     930        LOG(IconDatabase, "(THREAD) Performing Safari2 import procedure");
     931        SQLTransaction importTransaction(m_syncDB);
     932        importTransaction.begin();
     933       
     934        // Commit the transaction only if the import completes (the import should be atomic)
     935        if (m_client->performImport()) {
     936            setImported(true);
     937            importTransaction.commit();
     938        } else {
     939            LOG(IconDatabase, "(THREAD) Safari 2 import was cancelled");
     940            importTransaction.rollback();
     941        }
     942       
     943        if (shouldStopThreadActivity())
     944            return syncThreadMainLoop();
     945           
     946#ifndef NDEBUG
     947        newStamp = currentTime();
     948        LOG(IconDatabase, "(THREAD) performImport() took %.4f seconds, now %.4f seconds from thread start", newStamp - timeStamp, newStamp - startTime);
     949        timeStamp = newStamp;
     950#endif
     951    }
     952       
     953    // Uncomment the following line to simulate a long lasting URL import (*HUGE* icon databases, or network home directories)
     954    // while (currentTime() - timeStamp < 10);
     955
     956    // Read in URL mappings from the database         
     957    LOG(IconDatabase, "(THREAD) Starting iconURL import");
     958    performURLImport();
     959   
     960    if (shouldStopThreadActivity())
     961        return syncThreadMainLoop();
     962
     963#ifndef NDEBUG
     964    newStamp = currentTime();
     965    LOG(IconDatabase, "(THREAD) performURLImport() took %.4f seconds.  Entering main loop %.4f seconds from thread start", newStamp - timeStamp, newStamp - startTime);
     966#endif
     967
     968    LOG(IconDatabase, "(THREAD) Beginning sync");
     969    return syncThreadMainLoop();
     970}
     971
     972static int databaseVersionNumber(SQLDatabase& db)
     973{
     974    return SQLStatement(db, "SELECT value FROM IconDatabaseInfo WHERE key = 'Version';").getColumnInt(0);
     975}
     976
     977static bool isValidDatabase(SQLDatabase& db)
     978{
     979
     980    // These four tables should always exist in a valid db
     981    if (!db.tableExists("IconInfo") || !db.tableExists("IconData") || !db.tableExists("PageURL") || !db.tableExists("IconDatabaseInfo"))
    177982        return false;
    178     }
     983   
     984    if (databaseVersionNumber(db) != currentDatabaseVersion) {
     985        LOG(IconDatabase, "DB version is not found or below expected valid version");
     986        return false;
     987    }
     988   
     989    return true;
     990}
     991
     992static void createDatabaseTables(SQLDatabase& db)
     993{
     994    if (!db.executeCommand("CREATE TABLE PageURL (url TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT REPLACE,iconID INTEGER NOT NULL ON CONFLICT FAIL);")) {
     995        LOG_ERROR("Could not create PageURL table in database (%i) - %s", db.lastError(), db.lastErrorMsg());
     996        db.close();
     997        return;
     998    }
     999    if (!db.executeCommand("CREATE INDEX PageURLIndex ON PageURL (url);")) {
     1000        LOG_ERROR("Could not create PageURL index in database (%i) - %s", db.lastError(), db.lastErrorMsg());
     1001        db.close();
     1002        return;
     1003    }
     1004    if (!db.executeCommand("CREATE TABLE IconInfo (iconID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE ON CONFLICT REPLACE, url TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, stamp INTEGER);")) {
     1005        LOG_ERROR("Could not create IconInfo table in database (%i) - %s", db.lastError(), db.lastErrorMsg());
     1006        db.close();
     1007        return;
     1008    }
     1009    if (!db.executeCommand("CREATE INDEX IconInfoIndex ON IconInfo (url, iconID);")) {
     1010        LOG_ERROR("Could not create PageURL index in database (%i) - %s", db.lastError(), db.lastErrorMsg());
     1011        db.close();
     1012        return;
     1013    }
     1014    if (!db.executeCommand("CREATE TABLE IconData (iconID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE ON CONFLICT REPLACE, data BLOB);")) {
     1015        LOG_ERROR("Could not create IconData table in database (%i) - %s", db.lastError(), db.lastErrorMsg());
     1016        db.close();
     1017        return;
     1018    }
     1019    if (!db.executeCommand("CREATE INDEX IconDataIndex ON IconData (iconID);")) {
     1020        LOG_ERROR("Could not create PageURL index in database (%i) - %s", db.lastError(), db.lastErrorMsg());
     1021        db.close();
     1022        return;
     1023    }
     1024    if (!db.executeCommand("CREATE TABLE IconDatabaseInfo (key TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT REPLACE,value TEXT NOT NULL ON CONFLICT FAIL);")) {
     1025        LOG_ERROR("Could not create IconDatabaseInfo table in database (%i) - %s", db.lastError(), db.lastErrorMsg());
     1026        db.close();
     1027        return;
     1028    }
     1029    if (!db.executeCommand(String("INSERT INTO IconDatabaseInfo VALUES ('Version', ") + String::number(currentDatabaseVersion) + ");")) {
     1030        LOG_ERROR("Could not insert icon database version into IconDatabaseInfo table (%i) - %s", db.lastError(), db.lastErrorMsg());
     1031        db.close();
     1032        return;
     1033    }
     1034}   
     1035
     1036void IconDatabase::performOpenInitialization()
     1037{
     1038    ASSERT_ICON_SYNC_THREAD();
     1039   
     1040    if (!isOpen())
     1041        return;
    1791042   
    1801043    if (checkIntegrityOnOpen) {
    … …  
    1821045        if (!checkIntegrity()) {
    1831046            LOG(IconDatabase, "Integrity check was bad - dumping IconDatabase");
     1047
    1841048            close();
    185 
    186             // Should've been consumed by SQLite, delete just to make sure we don't see it again in the future;
    187             deleteFile(journalFilename);
    188             deleteFile(dbFilename);
    189 
    190             // Reopen the main database, creating it from scratch
    191             if (!m_mainDB.open(dbFilename)) {
    192                 LOG_ERROR("Unable to open icon database at path %s - %s", dbFilename.ascii().data(), m_mainDB.lastErrorMsg());
    193                 return false;
     1049           
     1050            {
     1051                MutexLocker locker(m_syncLock);
     1052                // Should've been consumed by SQLite, delete just to make sure we don't see it again in the future;
     1053                deleteFile(m_completeDatabasePath + "-journal");
     1054                deleteFile(m_completeDatabasePath);
    1941055            }
    195         }
    196     }
    197        
    198     if (!isValidDatabase(m_mainDB)) {
    199         LOG(IconDatabase, "%s is missing or in an invalid state - reconstructing", dbFilename.ascii().data());
    200         m_mainDB.clearAllTables();
    201         createDatabaseTables(m_mainDB);
    202     }
    203    
     1056           
     1057            // Reopen the write database, creating it from scratch
     1058            if (!m_syncDB.open(m_completeDatabasePath)) {
     1059                LOG_ERROR("Unable to open icon database at path %s - %s", m_completeDatabasePath.utf8().data(), m_syncDB.lastErrorMsg());
     1060                return;
     1061            }         
     1062        }
     1063    }
     1064       
     1065    if (!isValidDatabase(m_syncDB)) {
     1066        LOG(IconDatabase, "%s is missing or in an invalid state - reconstructing", m_syncDB.path().utf8().data());
     1067        m_syncDB.clearAllTables();
     1068        createDatabaseTables(m_syncDB);
     1069    }
     1070
    2041071    // Reduce sqlite RAM cache size from default 2000 pages (~1.5kB per page). 3MB of cache for icon database is overkill
    205     if (!SQLStatement(m_mainDB, "PRAGMA cache_size = 200;").executeCommand())         
     1072    if (!SQLStatement(m_syncDB, "PRAGMA cache_size = 200;").executeCommand())         
    2061073        LOG_ERROR("SQLite database could not set cache_size");
    207    
    208     // Open the in-memory table for private browsing
    209     if (!m_privateBrowsingDB.open(":memory:"))
    210         LOG_ERROR("Unable to open in-memory database for private browsing - %s", m_privateBrowsingDB.lastErrorMsg());
    211 
    212     // Only if we successfully remained open will we start our "initial purge timer"
    213     // rdar://4690949 - when we have deferred reads and writes all the way in, the prunetimer
    214     // will become "deferredTimer" or something along those lines, and will be set only when
    215     // a deferred read/write is queued
    216     if (isOpen())
    217         m_startupTimer.startOneShot(0);
    218    
    219     return isOpen();
    220 }
    221 
    222 bool IconDatabase::isOpen() const
    223 {
    224     return m_mainDB.isOpen() && m_privateBrowsingDB.isOpen();
    225 }
    226 
    227 void IconDatabase::close()
    228 {
    229     // This will close all the SQL statements and transactions we have open,
    230     // syncing the DB at the appropriate point
    231     deleteAllPreparedStatements(true);
    232  
    233     m_mainDB.close();
    234     m_privateBrowsingDB.close();
    235 }
    236 
    237 String IconDatabase::databasePath() const
    238 {
    239     return m_mainDB.isOpen() ? m_mainDB.path() : String();
    240 }
    241    
    242 void IconDatabase::removeAllIcons()
    243 {
     1074}
     1075
     1076bool IconDatabase::checkIntegrity()
     1077{
     1078    ASSERT_ICON_SYNC_THREAD();
     1079   
     1080    SQLStatement integrity(m_syncDB, "PRAGMA integrity_check;");
     1081    if (integrity.prepare() != SQLResultOk) {
     1082        LOG_ERROR("checkIntegrity failed to execute");
     1083        return false;
     1084    }
     1085   
     1086    int resultCode = integrity.step();
     1087    if (resultCode == SQLResultOk)
     1088        return true;
     1089       
     1090    if (resultCode != SQLResultRow)
     1091        return false;
     1092
     1093    int columns = integrity.columnCount();
     1094    if (columns != 1) {
     1095        LOG_ERROR("Received %i columns performing integrity check, should be 1", columns);
     1096        return false;
     1097    }
     1098       
     1099    String resultText = integrity.getColumnText16(0);
     1100       
     1101    // A successful, no-error integrity check will be "ok" - all other strings imply failure
     1102    if (resultText == "ok")
     1103        return true;
     1104   
     1105    LOG_ERROR("Icon database integrity check failed - \n%s", resultText.utf8().data());
     1106    return false;
     1107}
     1108
     1109void IconDatabase::performURLImport()
     1110{
     1111    ASSERT_ICON_SYNC_THREAD();
     1112
     1113    SQLStatement query(m_syncDB, "SELECT PageURL.url, IconInfo.url, IconInfo.stamp FROM PageURL INNER JOIN IconInfo ON PageURL.iconID=IconInfo.iconID;");
     1114   
     1115    if (query.prepare() != SQLResultOk) {
     1116        LOG_ERROR("Unable to prepare icon url import query");
     1117        return;
     1118    }
     1119   
     1120    // Informal testing shows that draining the autorelease pool every 25 iterations is about as low as we can go
     1121    // before performance starts to drop off, but we don't want to increase this number because then accumulated memory usage will go up
     1122    AutodrainedPool pool(25);
     1123       
     1124    int result = query.step();
     1125    while (result == SQLResultRow) {
     1126        String pageURL = query.getColumnText16(0);
     1127        String iconURL = query.getColumnText16(1);
     1128
     1129        {
     1130            MutexLocker locker(m_urlAndIconLock);
     1131           
     1132            PageURLRecord* pageRecord = m_pageURLToRecordMap.get(pageURL);
     1133           
     1134            // If the pageRecord doesn't exist in this map, then no one has retained this pageURL
     1135            // If the s_databaseCleanupCounter count is non-zero, then we're not supposed to be pruning the database in any manner,
     1136            // so go ahead and actually create a pageURLRecord for this url even though it's not retained.
     1137            // If database cleanup *is* allowed, we don't want to bother pulling in a page url from disk that noone is actually interested
     1138            // in - we'll prune it later instead!
     1139            if (!pageRecord && databaseCleanupCounter) {
     1140                pageRecord = new PageURLRecord(pageURL);
     1141                m_pageURLToRecordMap.set(pageURL, pageRecord);
     1142            }
     1143           
     1144            if (pageRecord) {
     1145                IconRecord* currentIcon = pageRecord->iconRecord();
     1146
     1147                if (!currentIcon || currentIcon->iconURL() != iconURL) {
     1148                    currentIcon = getOrCreateIconRecord(iconURL);
     1149                    pageRecord->setIconRecord(currentIcon);
     1150                }
     1151           
     1152                // Regardless, the time stamp from disk still takes precedence.  Until we read this icon from disk, we didn't think we'd seen it before
     1153                // so we marked the timestamp as "now", but it's really much older
     1154                currentIcon->setTimestamp(query.getColumnInt(2));
     1155            }           
     1156        }
     1157       
     1158        // FIXME: Currently the WebKit API supports 1 type of notification that is sent whenever we get an Icon URL for a Page URL.  We might want to re-purpose it to work for
     1159        // getting the actually icon itself also (so each pageurl would get this notification twice) or we might want to add a second type of notification -
     1160        // one for the URL and one for the Image itself
     1161        // Note that WebIconDatabase is not neccessarily API so we might be able to make this change
     1162        {
     1163            MutexLocker locker(m_pendingReadingLock);
     1164            if (m_pageURLsPendingImport.contains(pageURL)) {
     1165                m_client->dispatchDidAddIconForPageURL(pageURL);
     1166                m_pageURLsPendingImport.remove(pageURL);
     1167           
     1168                pool.cycle();
     1169            }
     1170        }
     1171       
     1172        // Stop the import at any time of the thread has been asked to shutdown
     1173        if (shouldStopThreadActivity()) {
     1174            LOG(IconDatabase, "IconDatabase asked to terminate during performURLImport()");
     1175            return;
     1176        }
     1177       
     1178        result = query.step();
     1179    }
     1180   
     1181    if (result != SQLResultDone)
     1182        LOG(IconDatabase, "Error reading page->icon url mappings from database");
     1183
     1184    // Clear the m_pageURLsPendingImport set - either the page URLs ended up with an iconURL (that we'll notify about) or not,
     1185    // but after m_iconURLImportComplete is set to true, we don't care about this set anymore
     1186    Vector<String> urls;
     1187    {
     1188        MutexLocker locker(m_pendingReadingLock);
     1189
     1190        urls.appendRange(m_pageURLsPendingImport.begin(), m_pageURLsPendingImport.end());
     1191        m_pageURLsPendingImport.clear();       
     1192        m_iconURLImportComplete = true;
     1193    }
     1194   
     1195    Vector<String> urlsToNotify;
     1196   
     1197    // Loop through the urls pending import
     1198    // Remove unretained ones if database cleanup is allowed
     1199    // Keep a set of ones that are retained and pending notification
     1200   
     1201    {
     1202        MutexLocker locker(m_urlAndIconLock);
     1203       
     1204        for (unsigned i = 0; i < urls.size(); ++i) {
     1205            if (!m_retainedPageURLs.contains(urls[i])) {
     1206                PageURLRecord* record = m_pageURLToRecordMap.get(urls[i]);
     1207                if (record && !databaseCleanupCounter) {
     1208                    m_pageURLToRecordMap.remove(urls[i]);
     1209                    IconRecord* iconRecord = record->iconRecord();
     1210                   
     1211                    // If this page is the only remaining retainer of its icon, mark that icon for deletion and don't bother
     1212                    // reading anything related to it
     1213                    if (iconRecord && iconRecord->hasOneRef()) {
     1214                        m_iconURLToRecordMap.remove(iconRecord->iconURL());
     1215                       
     1216                        {
     1217                            MutexLocker locker(m_pendingReadingLock);
     1218                            m_pageURLsInterestedInIcons.remove(urls[i]);
     1219                            m_iconsPendingReading.remove(iconRecord);
     1220                        }
     1221                        {
     1222                            MutexLocker locker(m_pendingSyncLock);
     1223                            m_iconsPendingSync.set(iconRecord->iconURL(), iconRecord->snapshot(true));                   
     1224                        }
     1225                    }
     1226                   
     1227                    delete record;
     1228                }
     1229            } else {
     1230                urlsToNotify.append(urls[i]);
     1231            }
     1232        }
     1233    }
     1234
     1235    LOG(IconDatabase, "Notifying %i interested page URLs that their icon URL is known due to the import", urlsToNotify.size());
     1236    // Now that we don't hold any locks, perform the actual notifications
     1237    for (unsigned i = 0; i < urlsToNotify.size(); ++i) {
     1238        LOG(IconDatabase, "Notifying icon info known for pageURL %s", urlsToNotify[i].utf8().data());
     1239        m_client->dispatchDidAddIconForPageURL(urlsToNotify[i]);
     1240        if (shouldStopThreadActivity())
     1241            return;
     1242
     1243        pool.cycle();
     1244    }
     1245   
     1246    // Notify all DocumentLoaders that were waiting for an icon load decision on the main thread
     1247    callOnMainThread(notifyPendingLoadDecisions);
     1248}
     1249
     1250void* IconDatabase::syncThreadMainLoop()
     1251{
     1252    ASSERT_ICON_SYNC_THREAD();
     1253    static bool prunedUnretainedIcons = false;
     1254
     1255    while (true) {
     1256        m_syncLock.unlock();
     1257
     1258#ifndef NDEBUG
     1259        double timeStamp = currentTime();
     1260#endif
     1261        LOG(IconDatabase, "(THREAD) Main work loop starting");
     1262
     1263        // If we should remove all icons, do it now.  This is an uninteruptible procedure that we will always do before quitting if it is requested
     1264        if (m_removeIconsRequested) {
     1265            removeAllIconsOnThread();
     1266            m_removeIconsRequested = false;
     1267        }
     1268       
     1269        // Then, if the thread should be quitting, quit now!
     1270        if (m_threadTerminationRequested)
     1271            break;
     1272       
     1273        bool didAnyWork = true;
     1274        while (didAnyWork) {
     1275            didAnyWork = readFromDatabase();
     1276            if (shouldStopThreadActivity())
     1277                break;
     1278               
     1279            bool didWrite = writeToDatabase();
     1280            if (shouldStopThreadActivity())
     1281                break;
     1282               
     1283            // Prune unretained icons after the first time we sync anything out to the database
     1284            // This way, pruning won't be the only operation we perform to the database by itself
     1285            // We also don't want to bother doing this if the thread should be terminating (the user is quitting)
     1286            // or if private browsing is enabled
     1287            // We also don't want to prune if the m_databaseCleanupCounter count is non-zero - that means someone
     1288            // has asked to delay pruning
     1289            if (didWrite && !m_privateBrowsingEnabled && !prunedUnretainedIcons && !databaseCleanupCounter) {
     1290#ifndef NDEBUG
     1291                double time = currentTime();
     1292#endif
     1293                LOG(IconDatabase, "(THREAD) Starting pruneUnretainedIcons()");
     1294               
     1295                pruneUnretainedIcons();
     1296               
     1297                LOG(IconDatabase, "(THREAD) pruneUnretainedIcons() took %.4f seconds", currentTime() - time);
     1298               
     1299                // If pruneUnretainedIcons() returned early due to requested thread termination, its still okay
     1300                // to mark prunedUnretainedIcons true because we're about to terminate anyway
     1301                prunedUnretainedIcons = true;
     1302            }
     1303           
     1304            didAnyWork = didAnyWork || didWrite;
     1305            if (shouldStopThreadActivity())
     1306                break;
     1307        }
     1308       
     1309#ifndef NDEBUG
     1310        double newstamp = currentTime();
     1311        LOG(IconDatabase, "(THREAD) Main work loop ran for %.4f seconds, %s requested to terminate", newstamp - timeStamp, shouldStopThreadActivity() ? "was" : "was not");
     1312#endif
     1313                   
     1314        m_syncLock.lock();
     1315       
     1316        // There is some condition that is asking us to stop what we're doing now and handle a special case
     1317        // This is either removing all icons, or shutting down the thread to quit the app
     1318        // We handle those at the top of this main loop so continue to jump back up there
     1319        if (shouldStopThreadActivity())
     1320            continue;
     1321           
     1322        m_syncCondition.wait(m_syncLock);
     1323    }
     1324    m_syncLock.unlock();
     1325   
     1326    // Thread is terminating at this point
     1327    return cleanupSyncThread();
     1328}
     1329
     1330bool IconDatabase::readFromDatabase()
     1331{
     1332    ASSERT_ICON_SYNC_THREAD();
     1333   
     1334#ifndef NDEBUG
     1335    double timeStamp = currentTime();
     1336#endif
     1337
     1338    bool didAnyWork = false;
     1339
     1340    // We'll make a copy of the sets of things that need to be read.  Then we'll verify at the time of updating the record that it still wants to be updated
     1341    // This way we won't hold the lock for a long period of time
     1342    Vector<IconRecord*> icons;
     1343    {
     1344        MutexLocker locker(m_pendingReadingLock);
     1345        icons.appendRange(m_iconsPendingReading.begin(), m_iconsPendingReading.end());
     1346    }
     1347   
     1348    // Keep track of icons we actually read to notify them of the new icon   
     1349    HashSet<String> urlsToNotify;
     1350   
     1351    for (unsigned i = 0; i < icons.size(); ++i) {
     1352        didAnyWork = true;
     1353        RefPtr<SharedBuffer> imageData = getImageDataForIconURLFromSQLDatabase(icons[i]->iconURL());
     1354
     1355        // Verify this icon still wants to be read from disk
     1356        {
     1357            MutexLocker urlLocker(m_urlAndIconLock);
     1358            {
     1359                MutexLocker readLocker(m_pendingReadingLock);
     1360               
     1361                if (m_iconsPendingReading.contains(icons[i])) {
     1362                    // Set the new data
     1363                    icons[i]->setImageData(imageData.get());
     1364                   
     1365                    // Remove this icon from the set that needs to be read
     1366                    m_iconsPendingReading.remove(icons[i]);
     1367                   
     1368                    // We have a set of all Page URLs that retain this icon as well as all PageURLs waiting for an icon
     1369                    // We want to find the intersection of these two sets to notify them
     1370                    // Check the sizes of these two sets to minimize the number of iterations
     1371                    const HashSet<String>* outerHash;
     1372                    const HashSet<String>* innerHash;
     1373                   
     1374                    if (icons[i]->retainingPageURLs().size() > m_pageURLsInterestedInIcons.size()) {
     1375                        outerHash = &m_pageURLsInterestedInIcons;
     1376                        innerHash = &(icons[i]->retainingPageURLs());
     1377                    } else {
     1378                        innerHash = &m_pageURLsInterestedInIcons;
     1379                        outerHash = &(icons[i]->retainingPageURLs());
     1380                    }
     1381                   
     1382                    HashSet<String>::const_iterator iter = outerHash->begin();
     1383                    HashSet<String>::const_iterator end = outerHash->end();
     1384                    for (; iter != end; ++iter) {
     1385                        if (innerHash->contains(*iter)) {
     1386                            LOG(IconDatabase, "%s is interesting in the icon we just read.  Adding it to the list and removing it from the interested set", urlForLogging(*iter).utf8().data());
     1387                            urlsToNotify.add(*iter);
     1388                        }
     1389                       
     1390                        // If we ever get to the point were we've seen every url interested in this icon, break early
     1391                        if (urlsToNotify.size() == m_pageURLsInterestedInIcons.size())
     1392                            break;
     1393                    }
     1394                   
     1395                    // We don't need to notify a PageURL twice, so all the ones we're about to notify can be removed from the interested set
     1396                    if (urlsToNotify.size() == m_pageURLsInterestedInIcons.size())
     1397                        m_pageURLsInterestedInIcons.clear();
     1398                    else {
     1399                        iter = urlsToNotify.begin();
     1400                        end = urlsToNotify.end();
     1401                        for (; iter != end; ++iter)
     1402                            m_pageURLsInterestedInIcons.remove(*iter);
     1403                    }
     1404                }
     1405            }
     1406        }
     1407   
     1408        if (shouldStopThreadActivity())
     1409            return didAnyWork;
     1410       
     1411        // Informal testing shows that draining the autorelease pool every 25 iterations is about as low as we can go
     1412        // before performance starts to drop off, but we don't want to increase this number because then accumulated memory usage will go up
     1413        AutodrainedPool pool(25);
     1414
     1415        // Now that we don't hold any locks, perform the actual notifications
     1416        HashSet<String>::iterator iter = urlsToNotify.begin();
     1417        HashSet<String>::iterator end = urlsToNotify.end();
     1418        for (unsigned iteration = 0; iter != end; ++iter, ++iteration) {
     1419            LOG(IconDatabase, "Notifying icon received for pageURL %s", urlForLogging(*iter).utf8().data());
     1420            m_client->dispatchDidAddIconForPageURL(*iter);
     1421            if (shouldStopThreadActivity())
     1422                return didAnyWork;
     1423           
     1424            pool.cycle();
     1425        }
     1426
     1427        LOG(IconDatabase, "Done notifying %i pageURLs who just received their icons", urlsToNotify.size());
     1428        urlsToNotify.clear();
     1429       
     1430        if (shouldStopThreadActivity())
     1431            return didAnyWork;
     1432    }
     1433
     1434    LOG(IconDatabase, "Reading from database took %.4f seconds", currentTime() - timeStamp);
     1435
     1436    return didAnyWork;
     1437}
     1438
     1439bool IconDatabase::writeToDatabase()
     1440{
     1441    ASSERT_ICON_SYNC_THREAD();
     1442
     1443#ifndef NDEBUG
     1444    double timeStamp = currentTime();
     1445#endif
     1446
     1447    bool didAnyWork = false;
     1448   
     1449    // We can copy the current work queue then clear it out - If any new work comes in while we're writing out,
     1450    // we'll pick it up on the next pass.  This greatly simplifies the locking strategy for this method and remains cohesive with changes
     1451    // asked for by the database on the main thread
     1452    Vector<IconSnapshot> iconSnapshots;
     1453    Vector<PageURLSnapshot> pageSnapshots;
     1454    {
     1455        MutexLocker locker(m_pendingSyncLock);
     1456       
     1457        iconSnapshots.appendRange(m_iconsPendingSync.begin().values(), m_iconsPendingSync.end().values());
     1458        m_iconsPendingSync.clear();
     1459       
     1460        pageSnapshots.appendRange(m_pageURLsPendingSync.begin().values(), m_pageURLsPendingSync.end().values());
     1461        m_pageURLsPendingSync.clear();
     1462    }
     1463   
     1464    if (iconSnapshots.size() || pageSnapshots.size())
     1465        didAnyWork = true;
     1466       
     1467    SQLTransaction syncTransaction(m_syncDB);
     1468    syncTransaction.begin();
     1469   
     1470    for (unsigned i = 0; i < iconSnapshots.size(); ++i) {
     1471        writeIconSnapshotToSQLDatabase(iconSnapshots[i]);
     1472        LOG(IconDatabase, "Wrote IconRecord for IconURL %s with timeStamp of %li to the DB", urlForLogging(iconSnapshots[i].iconURL).utf8().data(), iconSnapshots[i].timestamp);
     1473    }
     1474   
     1475    for (unsigned i = 0; i < pageSnapshots.size(); ++i) {
     1476        String iconURL = pageSnapshots[i].iconURL;
     1477
     1478        // If the icon URL is empty, this page is meant to be deleted
     1479        // ASSERTs are sanity checks to make sure the mappings exist if they should and don't if they shouldn't
     1480        if (pageSnapshots[i].iconURL.isEmpty())
     1481            removePageURLFromSQLDatabase(pageSnapshots[i].pageURL);
     1482        else {
     1483            setIconURLForPageURLInSQLDatabase(pageSnapshots[i].iconURL, pageSnapshots[i].pageURL);
     1484        }
     1485        LOG(IconDatabase, "Committed IconURL for PageURL %s to database", urlForLogging(pageSnapshots[i].pageURL).utf8().data());
     1486    }
     1487
     1488    syncTransaction.commit();
     1489   
     1490    // Check to make sure there are no dangling PageURLs - If there are, we want to output one log message but not spam the console potentially every few seconds
     1491    checkForDanglingPageURLs(false);
     1492
     1493    LOG(IconDatabase, "Updating the database took %.4f seconds", currentTime() - timeStamp);
     1494
     1495    return didAnyWork;
     1496}
     1497
     1498void IconDatabase::pruneUnretainedIcons()
     1499{
     1500    ASSERT_ICON_SYNC_THREAD();
     1501
    2441502    if (!isOpen())
    245         return;
    246        
    247     // We don't need to sync anything anymore since we're wiping everything. 
    248     // So we can kill the update timer, and clear all the hashes of "items that need syncing"
    249     m_updateTimer.stop();
    250     m_iconDataCachesPendingUpdate.clear();
    251     m_pageURLsPendingAddition.clear();
    252     m_pageURLsPendingDeletion.clear();
    253     m_iconURLsPendingDeletion.clear();
    254    
    255     //  Now clear all in-memory URLs and Icons
    256     m_pageURLToIconURLMap.clear();
    257     m_pageURLToRetainCount.clear();
    258     m_iconURLToRetainCount.clear();
    259    
    260     deleteAllValues(m_iconURLToIconDataCacheMap);
    261     m_iconURLToIconDataCacheMap.clear();
    262        
    263     // Wipe any pre-prepared statements, otherwise resetting the SQLDatabases themselves will fail
    264     deleteAllPreparedStatements(false);
    265    
    266     // The easiest way to wipe the in-memory database is by closing and reopening it
    267     m_privateBrowsingDB.close();
    268     if (!m_privateBrowsingDB.open(":memory:"))
    269         LOG_ERROR("Unable to open in-memory database for private browsing - %s", m_privateBrowsingDB.lastErrorMsg());
    270     createDatabaseTables(m_privateBrowsingDB);
    271        
    272     // To reset the on-disk database, we'll wipe all its tables then vacuum it
    273     // This is easier and safer than closing it, deleting the file, and recreating from scratch
    274     m_mainDB.clearAllTables();
    275     m_mainDB.runVacuumCommand();
    276     createDatabaseTables(m_mainDB);
    277 }
    278 
    279 // There are two instances where you'd want to deleteAllPreparedStatements - one with sync, and one without
    280 // A - Closing down the database on application exit - in this case, you *do* want to save the icons out
    281 // B - Resetting the DB via removeAllIcons() - in this case, you *don't* want to sync, because it would be a waste of time
    282 void IconDatabase::deleteAllPreparedStatements(bool withSync)
    283 {
    284     // Sync, if desired
    285     if (withSync)
    286         syncDatabase();
    287        
    288     // Order doesn't matter on these
    289     delete m_timeStampForIconURLStatement;
    290     m_timeStampForIconURLStatement = 0;
    291     delete m_iconURLForPageURLStatement;
    292     m_iconURLForPageURLStatement = 0;
    293     delete m_hasIconForIconURLStatement;
    294     m_hasIconForIconURLStatement = 0;
    295     delete m_forgetPageURLStatement;
    296     m_forgetPageURLStatement = 0;
    297     delete m_setIconIDForPageURLStatement;
    298     m_setIconIDForPageURLStatement = 0;
    299     delete m_getIconIDForIconURLStatement;
    300     m_getIconIDForIconURLStatement = 0;
    301     delete m_addIconForIconURLStatement;
    302     m_addIconForIconURLStatement = 0;
    303     delete m_imageDataForIconURLStatement;
    304     m_imageDataForIconURLStatement = 0;
    305     delete m_importedStatement;
    306     m_importedStatement = 0;
    307     delete m_setImportedStatement;
    308     m_setImportedStatement = 0;
    309 }
    310 
    311 bool IconDatabase::isEmpty()
    312 {
    313     if (m_privateBrowsingEnabled)
    314         if (!pageURLTableIsEmptyQuery(m_privateBrowsingDB))
    315             return false;
    316            
    317     return pageURLTableIsEmptyQuery(m_mainDB);
    318 }
    319 
    320 bool IconDatabase::isValidDatabase(SQLDatabase& db)
    321 {
    322     // These two tables should always exist in a valid db
    323     if (!db.tableExists("Icon") || !db.tableExists("PageURL") || !db.tableExists("IconDatabaseInfo"))
    324         return false;
    325    
    326     if (SQLStatement(db, "SELECT value FROM IconDatabaseInfo WHERE key = 'Version';").getColumnInt(0) < currentDatabaseVersion) {
    327         LOG(IconDatabase, "DB version is not found or below expected valid version");
    328         return false;
    329     }
    330    
    331     return true;
    332 }
    333 
    334 void IconDatabase::createDatabaseTables(SQLDatabase& db)
    335 {
    336     if (!db.executeCommand("CREATE TABLE PageURL (url TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT REPLACE,iconID INTEGER NOT NULL ON CONFLICT FAIL);")) {
    337         LOG_ERROR("Could not create PageURL table in database (%i) - %s", db.lastError(), db.lastErrorMsg());
    338         db.close();
    339         return;
    340     }
    341     if (!db.executeCommand("CREATE TABLE Icon (iconID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE ON CONFLICT REPLACE, url TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT FAIL, stamp INTEGER, data BLOB);")) {
    342         LOG_ERROR("Could not create Icon table in database (%i) - %s", db.lastError(), db.lastErrorMsg());
    343         db.close();
    344         return;
    345     }
    346     if (!db.executeCommand("CREATE TABLE IconDatabaseInfo (key TEXT NOT NULL ON CONFLICT FAIL UNIQUE ON CONFLICT REPLACE,value TEXT NOT NULL ON CONFLICT FAIL);")) {
    347         LOG_ERROR("Could not create IconDatabaseInfo table in database (%i) - %s", db.lastError(), db.lastErrorMsg());
    348         db.close();
    349         return;
    350     }
    351     if (!db.executeCommand(String("INSERT INTO IconDatabaseInfo VALUES ('Version', ") + String::number(currentDatabaseVersion) + ");")) {
    352         LOG_ERROR("Could not insert icon database version into IconDatabaseInfo table (%i) - %s", db.lastError(), db.lastErrorMsg());
    353         db.close();
    354         return;
    355     }
    356 }   
    357 
    358 PassRefPtr<SharedBuffer> IconDatabase::imageDataForIconURL(const String& iconURL)
    359 {     
    360     // If private browsing is enabled, we'll check there first as the most up-to-date data for an icon will be there
    361     if (m_privateBrowsingEnabled) {   
    362         RefPtr<SharedBuffer> result = imageDataForIconURLQuery(m_privateBrowsingDB, iconURL);
    363         if (result && !result->isEmpty())
    364             return result.release();
    365     }
    366    
    367     // It wasn't found there, so lets check the main tables
    368     return imageDataForIconURLQuery(m_mainDB, iconURL);
    369 }
    370 
    371 void IconDatabase::setPrivateBrowsingEnabled(bool flag)
    372 {
    373     if (!isOpen())
    374         return;
    375     if (m_privateBrowsingEnabled == flag)
    376         return;
    377    
    378     // Sync any deferred DB changes before we change the active DB
    379     syncDatabase();
    380    
    381     m_privateBrowsingEnabled = flag;
    382    
    383     if (m_privateBrowsingEnabled) {
    384         createDatabaseTables(m_privateBrowsingDB);
    385         m_currentDB = &m_privateBrowsingDB;
    386     } else {
    387         m_privateBrowsingDB.clearAllTables();
    388         m_currentDB = &m_mainDB;
    389     }
    390 }
    391 
    392 bool IconDatabase::isPrivateBrowsingEnabled() const
    393 {
    394     return m_privateBrowsingEnabled;
    395 }
    396 
    397 Image* IconDatabase::iconForPageURL(const String& pageURL, const IntSize& size, bool cache)
    398 {   
    399     if (!isOpen())
    400         return defaultIcon(size);
    401        
    402     // See if we even have an IconURL for this PageURL...
    403     String iconURL = iconURLForPageURL(pageURL);
    404     if (iconURL.isEmpty())
    405         return 0;
    406    
    407     // If we do, maybe we have a IconDataCache for this IconURL
    408     IconDataCache* icon = getOrCreateIconDataCache(iconURL);
    409    
    410     // If it's a new IconDataCache object that doesn't have its imageData set yet,
    411     // we'll read in that image data now
    412     if (icon->imageDataStatus() == ImageDataStatusUnknown) {
    413         RefPtr<SharedBuffer> data = imageDataForIconURL(iconURL);
    414         icon->setImageData(data.get());
    415     }
    416        
    417     return icon->getImage(size);
    418 }
    419 
    420 // FIXME 4667425 - this check needs to see if the icon's data is empty or not and apply
    421 // iconExpirationTime to present icons, and missingIconExpirationTime for missing icons
    422 bool IconDatabase::isIconExpiredForIconURL(const String& iconURL)
    423 {
    424     // If we're closed and someone is making this call, it is likely a return value of
    425     // false will discourage them to take any further action, which is our goal in this case
    426     // Same notion for an empty iconURL - which is now defined as "never expires"
    427     if (!isOpen() || iconURL.isEmpty())
    428         return false;
    429    
    430     // If we have a IconDataCache, then it definitely has the Timestamp in it
    431     IconDataCache* icon = m_iconURLToIconDataCacheMap.get(iconURL);
    432     if (icon)
    433         return (int)currentTime() - icon->getTimestamp() > iconExpirationTime;
    434            
    435     // Otherwise, we'll get the timestamp from the DB and use it
    436     int stamp;
    437     if (m_privateBrowsingEnabled) {
    438         stamp = timeStampForIconURLQuery(m_privateBrowsingDB, iconURL);
    439         if (stamp)
    440             return ((int)currentTime() - stamp) > iconExpirationTime;
    441     }
    442    
    443     stamp = timeStampForIconURLQuery(m_mainDB, iconURL);
    444     if (stamp)
    445         return ((int)currentTime() - stamp) > iconExpirationTime;
    446    
    447     return false;
    448 }
    449    
    450 String IconDatabase::iconURLForPageURL(const String& pageURL)
    451 {   
    452     if (!isOpen() || pageURL.isEmpty())
    453         return String();
    454        
    455     if (m_pageURLToIconURLMap.contains(pageURL))
    456         return m_pageURLToIconURLMap.get(pageURL);
    457    
    458     // Try the private browsing database because if any PageURL's IconURL was updated during privated browsing,
    459     // the most up-to-date iconURL would be there
    460     if (m_privateBrowsingEnabled) {
    461         String iconURL = iconURLForPageURLQuery(m_privateBrowsingDB, pageURL);
    462         if (!iconURL.isEmpty()) {
    463             m_pageURLToIconURLMap.set(pageURL, iconURL);
    464             return iconURL;
    465         }
    466     }
    467    
    468     String iconURL = iconURLForPageURLQuery(m_mainDB, pageURL);
    469     if (!iconURL.isEmpty())
    470         m_pageURLToIconURLMap.set(pageURL, iconURL);
    471     return iconURL;
    472 }
    473 
    474 Image* IconDatabase::defaultIcon(const IntSize& size)
    475 {
    476     if (!m_defaultIconDataCache) {
    477         m_defaultIconDataCache = new IconDataCache("urlIcon");
    478         m_defaultIconDataCache->loadImageFromResource("urlIcon");
    479     }
    480    
    481     return m_defaultIconDataCache->getImage(size);
    482 }
    483 
    484 void IconDatabase::retainIconForPageURL(const String& pageURL)
    485 {
    486     if (!isOpen() || pageURL.isEmpty())
    487         return;
    488    
    489     // If we don't have the retain count for this page, we need to setup records of its retain
    490     // Otherwise, get the count and increment it
    491     int retainCount;
    492     if (!(retainCount = m_pageURLToRetainCount.get(pageURL))) {
    493         m_pageURLToRetainCount.set(pageURL, 1);   
    494 
    495         // If we haven't done pruning yet, we want to avoid any pageURL->iconURL lookups and the pageURLsPendingDeletion is moot,
    496         // so we bail here and skip those steps
    497         if (!m_initialPruningComplete)
    498             return;
    499        
    500         // If this pageURL is marked for deletion, bring it back from the brink
    501         m_pageURLsPendingDeletion.remove(pageURL);
    502        
    503         // If we have an iconURL for this pageURL, we'll now retain the iconURL
    504         String iconURL = iconURLForPageURL(pageURL);
    505         if (!iconURL.isEmpty())
    506             retainIconURL(iconURL);
    507 
    508     } else
    509         m_pageURLToRetainCount.set(pageURL, retainCount + 1);   
    510 }
    511 
    512 void IconDatabase::releaseIconForPageURL(const String& pageURL)
    513 {
    514     if (!isOpen() || pageURL.isEmpty())
    515         return;
    516        
    517     // Check if this pageURL is actually retained
    518     if (!m_pageURLToRetainCount.contains(pageURL)) {
    519         LOG_ERROR("Attempting to release icon for URL %s which is not retained", pageURL.ascii().data());
    520         return;
    521     }
    522    
    523     // Get its retain count
    524     int retainCount = m_pageURLToRetainCount.get(pageURL);
    525     ASSERT(retainCount > 0);
    526    
    527     // If it still has a positive retain count, store the new count and bail
    528     if (--retainCount) {
    529         m_pageURLToRetainCount.set(pageURL, retainCount);
    530         return;
    531     }
    532    
    533     LOG(IconDatabase, "No more retainers for PageURL %s", pageURL.ascii().data());
    534    
    535     // Otherwise, remove all record of the retain count
    536     m_pageURLToRetainCount.remove(pageURL);   
    537    
    538     // If we haven't done pruning yet, we want to avoid any pageURL->iconURL lookups and the pageURLsPendingDeletion is moot,
    539     // so we bail here and skip those steps
    540     if (!m_initialPruningComplete)
    541         return;
    542 
    543    
    544     // Then mark this pageURL for deletion
    545     m_pageURLsPendingDeletion.add(pageURL);
    546    
    547     // Grab the iconURL and release it
    548     String iconURL = iconURLForPageURL(pageURL);
    549     if (!iconURL.isEmpty())
    550         releaseIconURL(iconURL);
    551 }
    552 
    553 void IconDatabase::retainIconURL(const String& iconURL)
    554 {
    555     ASSERT(!iconURL.isEmpty());
    556    
    557     if (int retainCount = m_iconURLToRetainCount.get(iconURL)) {
    558         ASSERT(retainCount > 0);
    559         m_iconURLToRetainCount.set(iconURL, retainCount + 1);
    560     } else {
    561         m_iconURLToRetainCount.set(iconURL, 1);
    562         if (m_iconURLsPendingDeletion.contains(iconURL))
    563             m_iconURLsPendingDeletion.remove(iconURL);
    564     }   
    565 }
    566 
    567 void IconDatabase::releaseIconURL(const String& iconURL)
    568 {
    569     ASSERT(!iconURL.isEmpty());
    570        
    571     // If the iconURL has no retain count, we can bail
    572     if (!m_iconURLToRetainCount.contains(iconURL))
    573         return;
    574    
    575     // Otherwise, decrement it
    576     int retainCount = m_iconURLToRetainCount.get(iconURL) - 1;
    577     ASSERT(retainCount > -1);
    578    
    579     // If the icon is still retained, store the count and bail
    580     if (retainCount) {
    581         m_iconURLToRetainCount.set(iconURL, retainCount);
    582         return;
    583     }
    584    
    585     LOG(IconDatabase, "No more retainers for IconURL %s", iconURL.ascii().data());
    586    
    587     // Otherwise, this icon is toast.  Remove all traces of its retain count...
    588     m_iconURLToRetainCount.remove(iconURL);
    589    
    590     // And since we delay the actual deletion of icons, so lets add it to that queue
    591     m_iconURLsPendingDeletion.add(iconURL);
    592 }
    593 
    594 void IconDatabase::forgetPageURL(const String& pageURL)
    595 {
    596     // Remove the PageURL->IconURL mapping
    597     m_pageURLToIconURLMap.remove(pageURL);
    598    
    599     // And remove this pageURL from the DB
    600     forgetPageURLQuery(*m_currentDB, pageURL);
    601 }
    602    
    603 bool IconDatabase::isIconURLRetained(const String& iconURL)
    604 {
    605     if (iconURL.isEmpty())
    606         return false;
    607        
    608     return m_iconURLToRetainCount.contains(iconURL);
    609 }
    610 
    611 void IconDatabase::forgetIconForIconURLFromDatabase(const String& iconURL)
    612 {
    613     if (iconURL.isEmpty())
    614         return;
    615 
    616     // For private browsing safety, since this alters the database, we only forget from the current database
    617     // If we're in private browsing and the Icon also exists in the main database, it will be pruned on the next startup
    618     int64_t iconID = establishIconIDForIconURL(*m_currentDB, iconURL, false);
    619    
    620     // If we didn't actually have an icon for this iconURL... well, thats a screwy condition we should track down, but also
    621     // something we could move on from
    622     ASSERT(iconID);
    623     if (!iconID) {
    624         LOG_ERROR("Attempting to forget icon for IconURL %s, though we don't have it in the database", iconURL.ascii().data());
    625         return;
    626     }
    627    
    628     if (!m_currentDB->executeCommand(String::format("DELETE FROM Icon WHERE Icon.iconID = %lli;", iconID)))
    629         LOG_ERROR("Unable to drop Icon for IconURL", iconURL.ascii().data());
    630     if (!m_currentDB->executeCommand(String::format("DELETE FROM PageURL WHERE PageURL.iconID = %lli", iconID)))
    631         LOG_ERROR("Unable to drop all PageURL for IconURL", iconURL.ascii().data());
    632 }
    633 
    634 IconDataCache* IconDatabase::getOrCreateIconDataCache(const String& iconURL)
    635 {
    636     IconDataCache* icon;
    637     if ((icon = m_iconURLToIconDataCacheMap.get(iconURL)))
    638         return icon;
    639        
    640     icon = new IconDataCache(iconURL);
    641     m_iconURLToIconDataCacheMap.set(iconURL, icon);
    642    
    643     // Get the most current time stamp for this IconURL
    644     int timestamp = 0;
    645     if (m_privateBrowsingEnabled)
    646         timestamp = timeStampForIconURLQuery(m_privateBrowsingDB, iconURL);
    647     if (!timestamp)
    648         timestamp = timeStampForIconURLQuery(m_mainDB, iconURL);
    649        
    650     // If we can't get a timestamp for this URL, then it is a new icon and we initialize its timestamp now
    651     if (!timestamp) {
    652         icon->setTimestamp((int)currentTime());
    653         m_iconDataCachesPendingUpdate.add(icon);
    654     } else
    655         icon->setTimestamp(timestamp);
    656        
    657     return icon;
    658 }
    659 
    660 void IconDatabase::setIconDataForIconURL(PassRefPtr<SharedBuffer> data, const String& iconURL)
    661 {
    662     if (!isOpen() || iconURL.isEmpty())
    663         return;
    664 
    665     // Get the IconDataCache for this IconURL (note, IconDataCacheForIconURL will create it if necessary)
    666     IconDataCache* icon = getOrCreateIconDataCache(iconURL);
    667    
    668     // Set the data in the IconDataCache
    669     icon->setImageData(data);
    670    
    671     // Update the timestamp in the IconDataCache to NOW
    672     icon->setTimestamp((int)currentTime());
    673 
    674     // Mark the IconDataCache as requiring an update to the database
    675     m_iconDataCachesPendingUpdate.add(icon);
    676 }
    677 
    678 void IconDatabase::setHaveNoIconForIconURL(const String& iconURL)
    679 {   
    680     setIconDataForIconURL(0, iconURL);
    681 }
    682 
    683 bool IconDatabase::setIconURLForPageURL(const String& iconURL, const String& pageURL)
    684 {
    685     ASSERT(!iconURL.isEmpty());
    686     if (!isOpen() || pageURL.isEmpty())
    687         return false;
    688    
    689     // If the urls already map to each other, bail.
    690     // This happens surprisingly often, and seems to cream iBench performance
    691     if (m_pageURLToIconURLMap.get(pageURL) == iconURL)
    692         return false;
    693 
    694     // If this pageURL is retained, we have some work to do on the IconURL retain counts
    695     if (m_pageURLToRetainCount.contains(pageURL)) {
    696         String oldIconURL = m_pageURLToIconURLMap.get(pageURL);
    697         if (!oldIconURL.isEmpty())
    698             releaseIconURL(oldIconURL);
    699         retainIconURL(iconURL);
    700     } else {
    701         // If this pageURL is *not* retained, then we may be marking it for deletion, as well!
    702         // As counterintuitive as it seems to mark it for addition and for deletion at the same time,
    703         // it's valid because when we do a new pageURL->iconURL mapping we *have* to mark it for addition,
    704         // no matter what, as there is no efficient was to determine if the mapping is in the DB already.
    705         // But, if the iconURL is marked for deletion, we'll also mark this pageURL for deletion - if a
    706         // client comes along and retains it before the timer fires, the "pendingDeletion" lists will
    707         // be manipulated appopriately and new pageURL will be brought back from the brink
    708         if (m_iconURLsPendingDeletion.contains(iconURL))
    709             m_pageURLsPendingDeletion.add(pageURL);
    710     }
    711    
    712     // Cache the pageURL->iconURL map
    713     m_pageURLToIconURLMap.set(pageURL, iconURL);
    714    
    715     // And mark this mapping to be added to the database
    716     m_pageURLsPendingAddition.add(pageURL);
    717    
    718     // Then start the timer to commit this change - or further delay the timer if it
    719     // was already started
    720     m_updateTimer.startOneShot(updateTimerDelay);
    721    
    722     return true;
    723 }
    724 
    725 void IconDatabase::setIconURLForPageURLInDatabase(const String& iconURL, const String& pageURL)
    726 {
    727     int64_t iconID = establishIconIDForIconURL(*m_currentDB, iconURL);
    728     if (!iconID) {
    729         LOG_ERROR("Failed to establish an ID for iconURL %s", iconURL.ascii().data());
    730         return;
    731     }
    732     setIconIDForPageURLQuery(*m_currentDB, iconID, pageURL);
    733 }
    734 
    735 int64_t IconDatabase::establishIconIDForIconURL(SQLDatabase& db, const String& iconURL, bool createIfNecessary)
    736 {   
    737     // Get the iconID thats already in this database and return it - or return 0 if we're read-only
    738     int64_t iconID = getIconIDForIconURLQuery(db, iconURL);
    739     if (iconID || !createIfNecessary)
    740         return iconID;
    741        
    742     // Create the icon table entry for the iconURL
    743     return addIconForIconURLQuery(db, iconURL);
    744 }
    745 
    746 void IconDatabase::pruneUnretainedIconsOnStartup(Timer<IconDatabase>*)
    747 {
    748     if (!isOpen())
    749         return;
    750        
    751     // This function should only be called once per run, and ideally only via the timer
    752     // on program startup
     1503        return;       
     1504   
     1505    // This method should only be called once per run
    7531506    ASSERT(!m_initialPruningComplete);
    7541507
    755 #ifndef NDEBUG
    756     double timestamp = currentTime();
    757 #endif
    758    
    759     // rdar://4690949 - Need to prune unretained iconURLs here, then prune out all pageURLs that reference
    760     // nonexistent icons
    761    
    762     SQLTransaction pruningTransaction(m_mainDB);
    763     pruningTransaction.begin();
    764    
    765     // Wipe all PageURLs that aren't retained
    766     // Temporary tables in sqlite seem to lose memory permanently so do this by hand instead. This is faster too.
    767 
    768     Vector<int64_t> pageURLIconIDsToDelete;
     1508    // This method relies on having read in all page URLs from the database earlier.
     1509    ASSERT(m_iconURLImportComplete);
    7691510
    7701511    // Get the known PageURLs from the db, and record the ID of any that are not in the retain count set.
    771     SQLStatement pageSQL(m_mainDB, "SELECT url, iconID FROM PageURL");
     1512    Vector<int64_t> pageIDsToDelete;
     1513
     1514    SQLStatement pageSQL(m_syncDB, "SELECT rowid, url FROM PageURL;");
    7721515    pageSQL.prepare();
     1516   
    7731517    int result;
    7741518    while ((result = pageSQL.step()) == SQLResultRow) {
    775         String pageURL = pageSQL.getColumnText16(0);
    776         if (pageURL.isEmpty() || !m_pageURLToRetainCount.contains(pageURL))
    777             pageURLIconIDsToDelete.append(pageSQL.getColumnInt64(1));
    778     }
    779     pageSQL.finalize();
     1519        MutexLocker locker(m_urlAndIconLock);
     1520        if (!m_pageURLToRecordMap.contains(pageSQL.getColumnText16(1)))
     1521            pageIDsToDelete.append(pageSQL.getColumnInt64(0));
     1522    }
     1523   
    7801524    if (result != SQLResultDone)
    7811525        LOG_ERROR("Error reading PageURL table from on-disk DB");
     1526    pageSQL.finalize();
    7821527   
    7831528    // Delete page URLs that were in the table, but not in our retain count set.
    784     size_t numToDelete = pageURLIconIDsToDelete.size();
     1529    size_t numToDelete = pageIDsToDelete.size();
    7851530    if (numToDelete) {
    786         SQLStatement pageDeleteSQL(m_mainDB, "DELETE FROM PageURL WHERE iconID = (?)");
     1531        SQLTransaction pruningTransaction(m_syncDB);
     1532        pruningTransaction.begin();
     1533       
     1534        SQLStatement pageDeleteSQL(m_syncDB, "DELETE FROM PageURL WHERE rowid = (?);");
    7871535        pageDeleteSQL.prepare();
    7881536        for (size_t i = 0; i < numToDelete; ++i) {
    789             pageDeleteSQL.bindInt64(1, pageURLIconIDsToDelete[i]);
    790             if (pageDeleteSQL.step() != SQLResultDone)
    791                 LOG_ERROR("Unable to delete icon ID %llu from PageURL table", static_cast<unsigned long long>(pageURLIconIDsToDelete[i]));
     1537            LOG(IconDatabase, "Pruning page with rowid %lli from disk", pageIDsToDelete[i]);
     1538            pageDeleteSQL.bindInt64(1, pageIDsToDelete[i]);
     1539            int result = pageDeleteSQL.step();
     1540            if (result != SQLResultDone)
     1541                LOG_ERROR("Unabled to delete page with id %lli from disk", pageIDsToDelete[i]);
    7921542            pageDeleteSQL.reset();
    793         }
     1543           
     1544            // If the thread was asked to terminate, we should commit what pruning we've done so far, figuring we can
     1545            // finish the rest later (hopefully)
     1546            if (shouldStopThreadActivity()) {
     1547                pruningTransaction.commit();
     1548                return;
     1549            }
     1550        }
     1551        pruningTransaction.commit();
    7941552        pageDeleteSQL.finalize();
    7951553    }
    7961554   
     1555    // Deleting unreferenced icons from the Icon tables has to be atomic -
     1556    // If the user quits while these are taking place, they might have to wait.  Thankfully this will rarely be an issue
     1557    // A user on a network home directory with a wildly inconsistent database might see quite a pause...
     1558
     1559    SQLTransaction pruningTransaction(m_syncDB);
     1560    pruningTransaction.begin();
     1561   
    7971562    // Wipe Icons that aren't retained
    798     if (!m_mainDB.executeCommand("DELETE FROM Icon WHERE Icon.iconID NOT IN (SELECT iconID FROM PageURL);"))
    799         LOG_ERROR("Failed to execute SQL to prune unretained icons from the on-disk tables");   
    800    
    801     // Since we lazily retained the pageURLs without getting the iconURLs or retaining the iconURLs,
    802     // we need to do that now
    803     SQLStatement sql(m_mainDB, "SELECT PageURL.url, Icon.url FROM PageURL INNER JOIN Icon ON PageURL.iconID=Icon.iconID");
    804     sql.prepare();
    805     while((result = sql.step()) == SQLResultRow) {
    806         String iconURL = sql.getColumnText16(1);
    807         retainIconURL(iconURL);
    808         LOG(IconDatabase, "Found a PageURL that mapped to %s", iconURL.ascii().data());
    809     }
    810     if (result != SQLResultDone)
    811         LOG_ERROR("Error reading PageURL->IconURL mappings from on-disk DB");
    812     sql.finalize();
     1563    if (!m_syncDB.executeCommand("DELETE FROM IconData WHERE iconID NOT IN (SELECT iconID FROM PageURL);"))
     1564        LOG_ERROR("Failed to execute SQL to prune unretained icons from the on-disk IconData table");   
     1565    if (!m_syncDB.executeCommand("DELETE FROM IconInfo WHERE iconID NOT IN (SELECT iconID FROM PageURL);"))
     1566        LOG_ERROR("Failed to execute SQL to prune unretained icons from the on-disk IconInfo table");   
    8131567   
    8141568    pruningTransaction.commit();
     1569       
     1570    checkForDanglingPageURLs(true);
     1571
    8151572    m_initialPruningComplete = true;
    816    
    817     // Handle dangling PageURLs, if any
    818     checkForDanglingPageURLs(true);
    819 
    820 #ifndef NDEBUG
    821     timestamp = currentTime() - timestamp;
    822     if (timestamp <= 1.0)
    823         LOG(IconDatabase, "Pruning unretained icons took %.4f seconds", timestamp);
    824     else
    825         LOG(IconDatabase, "Pruning unretained icons took %.4f seconds - this is much too long!", timestamp);
    826 
    827 #endif
    828 }
    829 
    830 void IconDatabase::updateDatabase(Timer<IconDatabase>*)
    831 {
    832     syncDatabase();
    833 }
    834 
    835 void IconDatabase::syncDatabase()
    836 {
    837 #ifndef NDEBUG
    838     double timestamp = currentTime();
    839 #endif
    840 
    841     SQLTransaction syncTransaction(*m_currentDB);
    842     syncTransaction.begin();
    843 
    844     // First we'll do the pending additions
    845     // Starting with the IconDataCaches that need updating/insertion
    846     for (HashSet<IconDataCache*>::iterator i = m_iconDataCachesPendingUpdate.begin(), end = m_iconDataCachesPendingUpdate.end(); i != end; ++i) {
    847         (*i)->writeToDatabase(*m_currentDB);
    848         LOG(IconDatabase, "Wrote IconDataCache for IconURL %s with timestamp of %li to the DB", (*i)->getIconURL().ascii().data(), (*i)->getTimestamp());
    849     }
    850     m_iconDataCachesPendingUpdate.clear();
    851    
    852     HashSet<String>::iterator i = m_pageURLsPendingAddition.begin(), end = m_pageURLsPendingAddition.end();
    853     for (; i != end; ++i) {
    854         setIconURLForPageURLInDatabase(m_pageURLToIconURLMap.get(*i), *i);
    855         LOG(IconDatabase, "Committed IconURL for PageURL %s to database", (*i).ascii().data());
    856     }
    857     m_pageURLsPendingAddition.clear();
    858    
    859     // Then we'll do the pending deletions
    860     // First lets wipe all the pageURLs
    861     for (i = m_pageURLsPendingDeletion.begin(), end = m_pageURLsPendingDeletion.end(); i != end; ++i) {   
    862         forgetPageURL(*i);
    863         LOG(IconDatabase, "Deleted PageURL %s", (*i).ascii().data());
    864     }
    865     m_pageURLsPendingDeletion.clear();
    866 
    867     // Then get rid of all traces of the icons and IconURLs
    868     IconDataCache* icon;   
    869     for (i = m_iconURLsPendingDeletion.begin(), end = m_iconURLsPendingDeletion.end(); i != end; ++i) {
    870         // Forget the IconDataCache
    871         icon = m_iconURLToIconDataCacheMap.get(*i);
    872         if (icon)
    873             m_iconURLToIconDataCacheMap.remove(*i);
    874         delete icon;
    875        
    876         // Forget the IconURL from the database
    877         forgetIconForIconURLFromDatabase(*i);
    878         LOG(IconDatabase, "Deleted icon %s", (*i).ascii().data());   
    879     }
    880     m_iconURLsPendingDeletion.clear();
    881     syncTransaction.commit();
    882 
    883     // If the timer was running to cause this update, we can kill the timer as its firing would be redundant
    884     m_updateTimer.stop();
    885    
    886 #ifndef NDEBUG
    887     timestamp = currentTime() - timestamp;
    888     if (timestamp <= 1.0)
    889         LOG(IconDatabase, "Updating the database took %.4f seconds", timestamp);
    890     else
    891         LOG(IconDatabase, "Updating the database took %.4f seconds - this is much too long!", timestamp);
    892    
    893     // Check to make sure there are no dangling PageURLs - If there are, we want to output one log message but not spam the console potentially every few seconds
    894     checkForDanglingPageURLs(false);
    895 #endif
    8961573}
    8971574
    8981575void IconDatabase::checkForDanglingPageURLs(bool pruneIfFound)
    8991576{
     1577    ASSERT_ICON_SYNC_THREAD();
     1578   
    9001579    // We don't want to keep performing this check and reporting this error if it has already found danglers so we keep track
    9011580    static bool danglersFound = false;
    … …  
    9051584        danglersFound = false;
    9061585       
    907     if (!danglersFound && SQLStatement(*m_currentDB, "SELECT url FROM PageURL WHERE PageURL.iconID NOT IN (SELECT iconID FROM Icon) LIMIT 1;").returnsAtLeastOneResult()) {
     1586    if (!danglersFound && SQLStatement(m_syncDB, "SELECT url FROM PageURL WHERE PageURL.iconID NOT IN (SELECT iconID FROM IconInfo) LIMIT 1;").returnsAtLeastOneResult()) {
    9081587        danglersFound = true;
    9091588        LOG_ERROR("Dangling PageURL entries found");
    910         if (pruneIfFound && !m_currentDB->executeCommand("DELETE FROM PageURL WHERE iconID NOT IN (SELECT iconID FROM Icon);"))
     1589        if (pruneIfFound && !m_syncDB.executeCommand("DELETE FROM PageURL WHERE iconID NOT IN (SELECT iconID FROM IconInfo);"))
    9111590            LOG_ERROR("Unable to prune dangling PageURLs");
    9121591    }
    9131592}
    9141593
    915 bool IconDatabase::hasEntryForIconURL(const String& iconURL)
    916 {
    917     if (!isOpen() || iconURL.isEmpty())
     1594void IconDatabase::removeAllIconsOnThread()
     1595{
     1596    ASSERT_ICON_SYNC_THREAD();
     1597
     1598    LOG(IconDatabase, "Removing all icons on the sync thread");
     1599
     1600    //Delete all the prepared statements so they can start over
     1601    deleteAllPreparedStatements();   
     1602       
     1603    {
     1604        MutexLocker locker(m_urlAndIconLock);
     1605       
     1606        // Clear all in-memory records of pages and icons
     1607        m_iconURLToRecordMap.clear();
     1608        // Deleting the PageURLRecords derefs the IconRecords automagically
     1609        deleteAllValues(m_pageURLToRecordMap);
     1610        m_pageURLToRecordMap.clear();
     1611       
     1612        // Clear all in-memory records of things that need to be synced out to disk
     1613        {
     1614            MutexLocker locker(m_pendingSyncLock);
     1615            m_pageURLsPendingSync.clear();
     1616            m_iconsPendingSync.clear();
     1617        }
     1618       
     1619        // Clear all in-memory records of things that need to be read in from disk
     1620        {
     1621            MutexLocker locker(m_pendingReadingLock);
     1622            m_pageURLsPendingImport.clear();
     1623            m_pageURLsInterestedInIcons.clear();
     1624            m_iconsPendingReading.clear();
     1625            m_loadersPendingDecision.clear();
     1626        }
     1627    }
     1628   
     1629    // To reset the on-disk database, we'll wipe all its tables then vacuum it
     1630    // This is easier and safer than closing it, deleting the file, and recreating from scratch
     1631    m_syncDB.clearAllTables();
     1632    m_syncDB.runVacuumCommand();
     1633    createDatabaseTables(m_syncDB);
     1634   
     1635    LOG(IconDatabase, "Dispatching notification that we removed all icons");
     1636    m_client->dispatchDidRemoveAllIcons();   
     1637}
     1638
     1639void IconDatabase::deleteAllPreparedStatements()
     1640{       
     1641    ASSERT_ICON_SYNC_THREAD();
     1642   
     1643    m_setIconIDForPageURLStatement.set(0);
     1644    m_removePageURLStatement.set(0);
     1645    m_getIconIDForIconURLStatement.set(0);
     1646    m_getImageDataForIconURLStatement.set(0);
     1647    m_addIconToIconInfoStatement.set(0);
     1648    m_addIconToIconDataStatement.set(0);
     1649    m_getImageDataStatement.set(0);
     1650    m_deletePageURLsForIconURLStatement.set(0);
     1651    m_deleteIconFromIconInfoStatement.set(0);
     1652    m_deleteIconFromIconDataStatement.set(0);
     1653    m_updateIconInfoStatement.set(0);
     1654    m_updateIconDataStatement.set(0);
     1655    m_setIconInfoStatement.set(0);
     1656    m_setIconDataStatement.set(0);
     1657}
     1658
     1659void* IconDatabase::cleanupSyncThread()
     1660{
     1661    ASSERT_ICON_SYNC_THREAD();
     1662   
     1663#ifndef NDEBUG
     1664    double timeStamp = currentTime();
     1665#endif
     1666
     1667    // If the removeIcons flag is set, remove all icons from the db.
     1668    // Otherwise, do a final write an sync out to disk
     1669    if (m_removeIconsRequested)
     1670        removeAllIconsOnThread();
     1671    else {
     1672        LOG(IconDatabase, "(THREAD) Doing final writeout and closure of sync thread");
     1673        writeToDatabase();
     1674    }
     1675   
     1676    // Close the database
     1677    MutexLocker locker(m_syncLock);
     1678   
     1679    m_completeDatabasePath = String();
     1680    deleteAllPreparedStatements();   
     1681    m_syncDB.close();
     1682   
     1683#ifndef NDEBUG
     1684    LOG(IconDatabase, "(THREAD) Final closure took %.4f seconds", currentTime() - timeStamp);
     1685#endif
     1686   
     1687    return 0;
     1688}
     1689
     1690bool IconDatabase::imported()
     1691{
     1692    ASSERT_ICON_SYNC_THREAD();
     1693   
     1694    if (m_isImportedSet)
     1695        return m_imported;
     1696       
     1697    SQLStatement query(m_syncDB, "SELECT IconDatabaseInfo.value FROM IconDatabaseInfo WHERE IconDatabaseInfo.key = \"ImportedSafari2Icons\";");
     1698    if (query.prepare() != SQLResultOk) {
     1699        LOG_ERROR("Unable to prepare imported statement");
    9181700        return false;
    919        
    920     // First check the in memory mapped icons...
    921     if (m_iconURLToIconDataCacheMap.contains(iconURL))
    922         return true;
    923 
    924     // Then we'll check the main database
    925     if (hasIconForIconURLQuery(m_mainDB, iconURL))
    926         return true;
    927        
    928     // Finally, the last resort - check the private browsing database
    929     if (m_privateBrowsingEnabled) 
    930         if (hasIconForIconURLQuery(m_privateBrowsingDB, iconURL))
    931             return true;   
    932 
    933     // We must not have this iconURL!
    934     return false;
    935 }
    936 
    937 void IconDatabase::setEnabled(bool enabled)
    938 {
    939     if (!enabled && isOpen())
    940         close();
    941     m_isEnabled = enabled;
    942 }
    943 
    944 bool IconDatabase::enabled() const
    945 {
    946      return m_isEnabled;
    947 }
    948 
    949 bool IconDatabase::imported()
    950 {
    951     if (!m_isImportedSet) {
    952         m_imported = importedQuery(m_mainDB);
    953         m_isImportedSet = true;
    954     }
    955     return m_imported;
     1701    }
     1702   
     1703    int result = query.step();
     1704    if (result == SQLResultRow)
     1705        result = query.getColumnInt(0);
     1706    else {
     1707        if (result != SQLResultDone)
     1708            LOG_ERROR("imported statement failed");
     1709        result = 0;
     1710    }
     1711   
     1712    m_isImportedSet = true;
     1713    return m_imported = result;
    9561714}
    9571715
    9581716void IconDatabase::setImported(bool import)
    9591717{
     1718    ASSERT_ICON_SYNC_THREAD();
     1719
    9601720    m_imported = import;
    9611721    m_isImportedSet = true;
    962     setImportedQuery(m_mainDB, import);
    963 }
    964 
    965 IconDatabase::~IconDatabase()
    966 {
    967     ASSERT_NOT_REACHED();
     1722   
     1723    String queryString = import ?
     1724        "INSERT INTO IconDatabaseInfo (key, value) VALUES (\"ImportedSafari2Icons\", 1);" :
     1725        "INSERT INTO IconDatabaseInfo (key, value) VALUES (\"ImportedSafari2Icons\", 0);";
     1726       
     1727    SQLStatement query(m_syncDB, queryString);
     1728   
     1729    if (query.prepare() != SQLResultOk) {
     1730        LOG_ERROR("Unable to prepare set imported statement");
     1731        return;
     1732    }   
     1733   
     1734    if (query.step() != SQLResultDone)
     1735        LOG_ERROR("set imported statement failed");
    9681736}
    9691737
    … …  
    9721740//     switches to and from private browsing
    9731741// 2 - Lazy construction of the Statement in the first place, in case we've never made this query before
    974 inline void readySQLStatement(SQLStatement*& statement, SQLDatabase& db, const String& str)
     1742inline void readySQLStatement(OwnPtr<SQLStatement>& statement, SQLDatabase& db, const String& str)
    9751743{
    9761744    if (statement && (statement->database() != &db || statement->isExpired())) {
    9771745        if (statement->isExpired())
    978             LOG(IconDatabase, "SQLStatement associated with %s is expired", str.ascii().data());
    979         delete statement;
    980         statement = 0;
     1746            LOG(IconDatabase, "SQLStatement associated with %s is expired", str.utf8().data());
     1747        statement.set(0);
    9811748    }
    9821749    if (!statement) {
    983         statement = new SQLStatement(db, str);
     1750        statement.set(new SQLStatement(db, str));
    9841751        int result;
    9851752        result = statement->prepare();
    … …  
    9881755}
    9891756
    990 // Any common IconDatabase query should be seperated into a fooQuery() and a *m_fooStatement. 
    991 // This way we can lazily construct the SQLStatment for a query on its first use, then reuse the Statement binding
    992 // the new parameter as needed
    993 // The statement must be deleted in IconDatabase::close() before the actual SQLDatabase::close() call
    994 // Also, m_fooStatement must be reset() before fooQuery() returns otherwise we will constantly get "database file locked"
    995 // errors in various combinations of queries
    996 
    997 bool IconDatabase::pageURLTableIsEmptyQuery(SQLDatabase& db)
    998 { 
    999     // We won't make this use a m_fooStatement because its not really a "common" query
    1000     return !SQLStatement(db, "SELECT iconID FROM PageURL LIMIT 1;").returnsAtLeastOneResult();
    1001 }
    1002 
    1003 PassRefPtr<SharedBuffer> IconDatabase::imageDataForIconURLQuery(SQLDatabase& db, const String& iconURL)
    1004 {
    1005     RefPtr<SharedBuffer> imageData;
    1006    
    1007     readySQLStatement(m_imageDataForIconURLStatement, db, "SELECT Icon.data FROM Icon WHERE Icon.url = (?);");
    1008     m_imageDataForIconURLStatement->bindText16(1, iconURL, false);
    1009    
    1010     int result = m_imageDataForIconURLStatement->step();
    1011     if (result == SQLResultRow) {
    1012         Vector<char> data;
    1013         m_imageDataForIconURLStatement->getColumnBlobAsVector(0, data);
    1014         imageData = new SharedBuffer;
    1015         imageData->append(data.data(), data.size());
    1016     } else if (result != SQLResultDone)
    1017         LOG_ERROR("imageDataForIconURLQuery failed");
    1018 
    1019     m_imageDataForIconURLStatement->reset();
    1020    
    1021     return imageData.release();
    1022 }
    1023 
    1024 int IconDatabase::timeStampForIconURLQuery(SQLDatabase& db, const String& iconURL)
    1025 {
    1026     readySQLStatement(m_timeStampForIconURLStatement, db, "SELECT Icon.stamp FROM Icon WHERE Icon.url = (?);");
    1027     m_timeStampForIconURLStatement->bindText16(1, iconURL, false);
    1028 
    1029     int result = m_timeStampForIconURLStatement->step();
    1030     if (result == SQLResultRow)
    1031         result = m_timeStampForIconURLStatement->getColumnInt(0);
    1032     else {
    1033         if (result != SQLResultDone)
    1034             LOG_ERROR("timeStampForIconURLQuery failed");
    1035         result = 0;
    1036     }
    1037 
    1038     m_timeStampForIconURLStatement->reset();
    1039     return result;
    1040 }
    1041 
    1042 String IconDatabase::iconURLForPageURLQuery(SQLDatabase& db, const String& pageURL)
    1043 {
    1044     readySQLStatement(m_iconURLForPageURLStatement, db, "SELECT Icon.url FROM Icon, PageURL WHERE PageURL.url = (?) AND Icon.iconID = PageURL.iconID;");
    1045     m_iconURLForPageURLStatement->bindText16(1, pageURL, false);
    1046    
    1047     int result = m_iconURLForPageURLStatement->step();
    1048     String iconURL;
    1049     if (result == SQLResultRow)
    1050         iconURL = m_iconURLForPageURLStatement->getColumnText16(0);
    1051     else if (result != SQLResultDone)
    1052         LOG_ERROR("iconURLForPageURLQuery failed");
    1053    
    1054     m_iconURLForPageURLStatement->reset();
    1055     return iconURL;
    1056 }
    1057 
    1058 void IconDatabase::forgetPageURLQuery(SQLDatabase& db, const String& pageURL)
    1059 {
    1060     readySQLStatement(m_forgetPageURLStatement, db, "DELETE FROM PageURL WHERE url = (?);");
    1061     m_forgetPageURLStatement->bindText16(1, pageURL, false);
    1062 
    1063     if (m_forgetPageURLStatement->step() != SQLResultDone)
    1064         LOG_ERROR("forgetPageURLQuery failed");
    1065    
    1066     m_forgetPageURLStatement->reset();
    1067 }
    1068 
    1069 void IconDatabase::setIconIDForPageURLQuery(SQLDatabase& db, int64_t iconID, const String& pageURL)
    1070 {
    1071     readySQLStatement(m_setIconIDForPageURLStatement, db, "INSERT INTO PageURL (url, iconID) VALUES ((?), ?);");
     1757void IconDatabase::setIconURLForPageURLInSQLDatabase(const String& iconURL, const String& pageURL)
     1758{
     1759    ASSERT_ICON_SYNC_THREAD();
     1760   
     1761    int64_t iconID = getIconIDForIconURLFromSQLDatabase(iconURL);
     1762
     1763    if (!iconID)
     1764        iconID = addIconURLToSQLDatabase(iconURL);
     1765   
     1766    if (!iconID) {
     1767        LOG_ERROR("Failed to establish an ID for iconURL %s", urlForLogging(iconURL).utf8().data());
     1768        ASSERT(false);
     1769        return;
     1770    }
     1771   
     1772    setIconIDForPageURLInSQLDatabase(iconID, pageURL);
     1773}
     1774
     1775void IconDatabase::setIconIDForPageURLInSQLDatabase(int64_t iconID, const String& pageURL)
     1776{
     1777    ASSERT_ICON_SYNC_THREAD();
     1778   
     1779    readySQLStatement(m_setIconIDForPageURLStatement, m_syncDB, "INSERT INTO PageURL (url, iconID) VALUES ((?), ?);");
    10721780    m_setIconIDForPageURLStatement->bindText16(1, pageURL, false);
    10731781    m_setIconIDForPageURLStatement->bindInt64(2, iconID);
    10741782
    1075     if (m_setIconIDForPageURLStatement->step() != SQLResultDone)
    1076         LOG_ERROR("setIconIDForPageURLQuery failed");
     1783    int result = m_setIconIDForPageURLStatement->step();
     1784    if (result != SQLResultDone) {
     1785        ASSERT(false);
     1786        LOG_ERROR("setIconIDForPageURLQuery failed for url %s", urlForLogging(pageURL).utf8().data());
     1787    }
    10771788
    10781789    m_setIconIDForPageURLStatement->reset();
    10791790}
    10801791
    1081 int64_t IconDatabase::getIconIDForIconURLQuery(SQLDatabase& db, const String& iconURL)
    1082 {
    1083     readySQLStatement(m_getIconIDForIconURLStatement, db, "SELECT Icon.iconID FROM Icon WHERE Icon.url = (?);");
     1792void IconDatabase::removePageURLFromSQLDatabase(const String& pageURL)
     1793{
     1794    ASSERT_ICON_SYNC_THREAD();
     1795   
     1796    readySQLStatement(m_removePageURLStatement, m_syncDB, "DELETE FROM PageURL WHERE url = (?);");
     1797    m_removePageURLStatement->bindText16(1, pageURL, false);
     1798
     1799    if (m_removePageURLStatement->step() != SQLResultDone)
     1800        LOG_ERROR("removePageURLFromSQLDatabase failed for url %s", urlForLogging(pageURL).utf8().data());
     1801   
     1802    m_removePageURLStatement->reset();
     1803}
     1804
     1805
     1806int64_t IconDatabase::getIconIDForIconURLFromSQLDatabase(const String& iconURL)
     1807{
     1808    ASSERT_ICON_SYNC_THREAD();
     1809   
     1810    readySQLStatement(m_getIconIDForIconURLStatement, m_syncDB, "SELECT IconInfo.iconID FROM IconInfo WHERE IconInfo.url = (?);");
    10841811    m_getIconIDForIconURLStatement->bindText16(1, iconURL, false);
    10851812   
    … …  
    10891816    else {
    10901817        if (result != SQLResultDone)
    1091             LOG_ERROR("getIconIDForIconURLQuery failed");
     1818            LOG_ERROR("getIconIDForIconURLFromSQLDatabase failed for url %s", urlForLogging(iconURL).utf8().data());
    10921819        result = 0;
    10931820    }
    … …  
    10971824}
    10981825
    1099 int64_t IconDatabase::addIconForIconURLQuery(SQLDatabase& db, const String& iconURL)
    1100 {
    1101     readySQLStatement(m_addIconForIconURLStatement, db, "INSERT INTO Icon (url) VALUES ((?));");
    1102     m_addIconForIconURLStatement->bindText16(1, iconURL, false);
    1103    
    1104     int64_t result = m_addIconForIconURLStatement->step();
    1105     if (result == SQLResultDone)
    1106         result = db.lastInsertRowID();
    1107     else {
    1108         LOG_ERROR("addIconForIconURLQuery failed");
    1109         result = 0;
    1110     }
    1111 
    1112     m_addIconForIconURLStatement->reset();
    1113     return result;
    1114 }
    1115 
    1116 bool IconDatabase::hasIconForIconURLQuery(SQLDatabase& db, const String& iconURL)
    1117 {
    1118     readySQLStatement(m_hasIconForIconURLStatement, db, "SELECT Icon.iconID FROM Icon WHERE Icon.url = (?);");
    1119     m_hasIconForIconURLStatement->bindText16(1, iconURL, false);
    1120 
    1121     int result = m_hasIconForIconURLStatement->step();
    1122 
    1123     if (result != SQLResultRow && result != SQLResultDone)
    1124         LOG_ERROR("hasIconForIconURLQuery failed");
    1125 
    1126     m_hasIconForIconURLStatement->reset();
    1127     return result == SQLResultRow;
    1128 }
    1129 
    1130 bool IconDatabase::importedQuery(SQLDatabase& db)
    1131 {
    1132     readySQLStatement(m_importedStatement, db, "SELECT IconDatabaseInfo.value FROM IconDatabaseInfo WHERE IconDatabaseInfo.key = \"ImportedSafari2Icons\";");
    1133    
    1134     int result = m_importedStatement->step();
    1135 
    1136     if (result == SQLResultRow)
    1137         result = m_importedStatement->getColumnInt(0);
    1138     else {
    1139         if (result != SQLResultDone)
    1140             LOG_ERROR("importedQuery failed");
    1141         result = 0;
    1142     }
    1143 
    1144     m_importedStatement->reset();
    1145     return result;
    1146 }
    1147 
    1148 void IconDatabase::setImportedQuery(SQLDatabase& db, bool imported)
    1149 {
    1150     if (imported)
    1151         readySQLStatement(m_setImportedStatement, db, "INSERT INTO IconDatabaseInfo (key, value) VALUES (\"ImportedSafari2Icons\", 1);");
    1152     else
    1153         readySQLStatement(m_setImportedStatement, db, "INSERT INTO IconDatabaseInfo (key, value) VALUES (\"ImportedSafari2Icons\", 0);");
    1154 
    1155     int result = m_setImportedStatement->step();
    1156 
    1157     if (result != SQLResultDone)
    1158         LOG_ERROR("setImportedQuery failed");
    1159 
    1160     m_setImportedStatement->reset();
    1161 }
    1162 
    1163 bool IconDatabase::checkIntegrity()
    1164 {
    1165     SQLStatement integrity(m_mainDB, "PRAGMA integrity_check;");
    1166     if (integrity.prepare() != SQLResultOk) {
    1167         LOG_ERROR("checkIntegrity failed to execute");
    1168         return false;
    1169     }
    1170    
    1171     int resultCode = integrity.step();
    1172     if (resultCode == SQLResultOk)
    1173         return true;
    1174        
    1175     if (resultCode != SQLResultRow)
    1176         return false;
    1177 
    1178     int columns = integrity.columnCount();
    1179     if (columns != 1) {
    1180         LOG_ERROR("Received %i columns performing integrity check, should be 1", columns);
    1181         return false;
    1182     }
    1183        
    1184     String resultText = integrity.getColumnText16(0);
    1185        
    1186     // A successful, no-error integrity check will be "ok" - all other strings imply failure
    1187     if (resultText == "ok")
    1188         return true;
    1189    
    1190     LOG_ERROR("Icon database integrity check failed - \n%s", resultText.ascii().data());
    1191     return false;
    1192 }
    1193 
    1194 size_t IconDatabase::pageURLMappingCount() const
    1195 {
    1196     return 0;
    1197 }
    1198 
    1199 size_t IconDatabase::retainedPageURLCount() const
    1200 {
    1201     return 0;
    1202 }
    1203 
    1204 size_t IconDatabase::iconRecordCount() const
    1205 {
    1206     return 0;
    1207 }
    1208 
    1209 size_t IconDatabase::iconRecordCountWithData() const
    1210 {
    1211     return 0;
     1826int64_t IconDatabase::addIconURLToSQLDatabase(const String& iconURL)
     1827{
     1828    ASSERT_ICON_SYNC_THREAD();
     1829   
     1830    // There would be a transaction here to make sure these two inserts are atomic
     1831    // In practice the only caller of this method is always wrapped in a transaction itself so placing another
     1832    // here is unnecessary
     1833   
     1834    readySQLStatement(m_addIconToIconInfoStatement, m_syncDB, "INSERT INTO IconInfo (url, stamp) VALUES (?, 0);");
     1835    m_addIconToIconInfoStatement->bindText16(1, iconURL);
     1836   
     1837    int result = m_addIconToIconInfoStatement->step();
     1838    m_addIconToIconInfoStatement->reset();
     1839    if (result != SQLResultDone) {
     1840        LOG_ERROR("addIconURLToSQLDatabase failed to insert %s into IconInfo", urlForLogging(iconURL).utf8().data());
     1841        return 0;
     1842    }
     1843    int64_t iconID = m_syncDB.lastInsertRowID();
     1844   
     1845    readySQLStatement(m_addIconToIconDataStatement, m_syncDB, "INSERT INTO IconData (iconID, data) VALUES (?, ?);");
     1846    m_addIconToIconDataStatement->bindInt64(1, iconID);
     1847   
     1848    result = m_addIconToIconDataStatement->step();
     1849    m_addIconToIconDataStatement->reset();
     1850    if (result != SQLResultDone) {
     1851        LOG_ERROR("addIconURLToSQLDatabase failed to insert %s into IconData", urlForLogging(iconURL).utf8().data());
     1852        return 0;
     1853    }
     1854   
     1855    return iconID;
     1856}
     1857
     1858PassRefPtr<SharedBuffer> IconDatabase::getImageDataForIconURLFromSQLDatabase(const String& iconURL)
     1859{
     1860    ASSERT_ICON_SYNC_THREAD();
     1861   
     1862    RefPtr<SharedBuffer> imageData;
     1863   
     1864    readySQLStatement(m_getImageDataForIconURLStatement, m_syncDB, "SELECT IconData.data FROM IconData WHERE IconData.iconID IN (SELECT iconID FROM IconInfo WHERE IconInfo.url = (?));");
     1865    m_getImageDataForIconURLStatement->bindText16(1, iconURL, false);
     1866   
     1867    int result = m_getImageDataForIconURLStatement->step();
     1868    if (result == SQLResultRow) {
     1869        Vector<char> data;
     1870        m_getImageDataForIconURLStatement->getColumnBlobAsVector(0, data);
     1871        imageData = new SharedBuffer;
     1872        imageData->append(data.data(), data.size());
     1873    } else if (result != SQLResultDone)
     1874        LOG_ERROR("getImageDataForIconURLFromSQLDatabase failed for url %s", urlForLogging(iconURL).utf8().data());
     1875
     1876    m_getImageDataForIconURLStatement->reset();
     1877   
     1878    return imageData.release();
     1879}
     1880
     1881void IconDatabase::removeIconFromSQLDatabase(const String& iconURL)
     1882{
     1883    ASSERT_ICON_SYNC_THREAD();
     1884   
     1885    if (iconURL.isEmpty())
     1886        return;
     1887
     1888    // There would be a transaction here to make sure these removals are atomic
     1889    // In practice the only caller of this method is always wrapped in a transaction itself so placing another here is unnecessary
     1890   
     1891    int64_t iconID = getIconIDForIconURLFromSQLDatabase(iconURL);
     1892    ASSERT(iconID);
     1893    if (!iconID) {
     1894        LOG_ERROR("Unable to get iconID for icon URL %s to delete it from the database", urlForLogging(iconURL).utf8().data());
     1895        return;
     1896    }
     1897   
     1898    readySQLStatement(m_deletePageURLsForIconURLStatement, m_syncDB, "DELETE FROM PageURL WHERE PageURL.iconID = (?);");
     1899    m_deletePageURLsForIconURLStatement->bindInt64(1, iconID);
     1900   
     1901    if (m_deletePageURLsForIconURLStatement->step() != SQLResultDone)
     1902        LOG_ERROR("m_deletePageURLsForIconURLStatement failed for url %s", urlForLogging(iconURL).utf8().data());
     1903   
     1904    readySQLStatement(m_deleteIconFromIconInfoStatement, m_syncDB, "DELETE FROM IconInfo WHERE IconInfo.iconID = (?);");
     1905    m_deleteIconFromIconInfoStatement->bindInt64(1, iconID);
     1906   
     1907    if (m_deleteIconFromIconInfoStatement->step() != SQLResultDone)
     1908        LOG_ERROR("m_deleteIconFromIconInfoStatement failed for url %s", urlForLogging(iconURL).utf8().data());
     1909       
     1910    readySQLStatement(m_deleteIconFromIconDataStatement, m_syncDB, "DELETE FROM IconData WHERE IconData.iconID = (?);");
     1911    m_deleteIconFromIconDataStatement->bindInt64(1, iconID);
     1912   
     1913    if (m_deleteIconFromIconDataStatement->step() != SQLResultDone)
     1914        LOG_ERROR("m_deleteIconFromIconDataStatement failed for url %s", urlForLogging(iconURL).utf8().data());
     1915       
     1916    m_deletePageURLsForIconURLStatement->reset();
     1917    m_deleteIconFromIconInfoStatement->reset();
     1918    m_deleteIconFromIconDataStatement->reset();
     1919}
     1920
     1921void IconDatabase::writeIconSnapshotToSQLDatabase(const IconSnapshot& snapshot)
     1922{
     1923    ASSERT_ICON_SYNC_THREAD();
     1924   
     1925    if (snapshot.iconURL.isEmpty())
     1926        return;
     1927       
     1928    // A nulled out timestamp and data means this icon is destined to be deleted - do that instead of writing it out
     1929    if (!snapshot.timestamp && !snapshot.data) {
     1930        LOG(IconDatabase, "Removing %s from on-disk database", urlForLogging(snapshot.iconURL).utf8().data());
     1931        removeIconFromSQLDatabase(snapshot.iconURL);
     1932        return;
     1933    }
     1934
     1935    // There would be a transaction here to make sure these removals are atomic
     1936    // In practice the only caller of this method is always wrapped in a transaction itself so placing another here is unnecessary
     1937       
     1938    // Get the iconID for this url
     1939    int64_t iconID = getIconIDForIconURLFromSQLDatabase(snapshot.iconURL);
     1940   
     1941    // If there is already an iconID in place, update the database. 
     1942    // Otherwise, insert new records
     1943    if (iconID) {   
     1944        readySQLStatement(m_updateIconInfoStatement, m_syncDB, "UPDATE IconInfo SET stamp = ?, url = ? WHERE iconID = ?;");
     1945        m_updateIconInfoStatement->bindInt64(1, snapshot.timestamp);
     1946        m_updateIconInfoStatement->bindText16(2, snapshot.iconURL);
     1947        m_updateIconInfoStatement->bindInt64(3, iconID);
     1948
     1949        if (m_updateIconInfoStatement->step() != SQLResultDone)
     1950            LOG_ERROR("Failed to update icon info for url %s", urlForLogging(snapshot.iconURL).utf8().data());
     1951       
     1952        m_updateIconInfoStatement->reset();
     1953       
     1954        readySQLStatement(m_updateIconDataStatement, m_syncDB, "UPDATE IconData SET data = ? WHERE iconID = ?;");
     1955        m_updateIconDataStatement->bindInt64(2, iconID);
     1956               
     1957        // If we *have* image data, bind it to this statement - Otherwise the DB will get "null" for the blob data,
     1958        // signifying that this icon doesn't have any data   
     1959        if (snapshot.data && snapshot.data->size())
     1960            m_updateIconDataStatement->bindBlob(1, snapshot.data->data(), snapshot.data->size());
     1961       
     1962        if (m_updateIconDataStatement->step() != SQLResultDone)
     1963            LOG_ERROR("Failed to update icon data for url %s", urlForLogging(snapshot.iconURL).utf8().data());
     1964
     1965        m_updateIconDataStatement->reset();
     1966    } else {   
     1967        readySQLStatement(m_setIconInfoStatement, m_syncDB, "INSERT INTO IconInfo (url,stamp) VALUES (?, ?);");
     1968        m_setIconInfoStatement->bindText16(1, snapshot.iconURL);
     1969        m_setIconInfoStatement->bindInt64(2, snapshot.timestamp);
     1970
     1971        if (m_setIconInfoStatement->step() != SQLResultDone)
     1972            LOG_ERROR("Failed to set icon info for url %s", urlForLogging(snapshot.iconURL).utf8().data());
     1973       
     1974        m_setIconInfoStatement->reset();
     1975       
     1976        int64_t iconID = m_syncDB.lastInsertRowID();
     1977
     1978        readySQLStatement(m_setIconDataStatement, m_syncDB, "INSERT INTO IconData (iconID, data) VALUES (?, ?);");
     1979        m_setIconDataStatement->bindInt64(1, iconID);
     1980
     1981        // If we *have* image data, bind it to this statement - Otherwise the DB will get "null" for the blob data,
     1982        // signifying that this icon doesn't have any data   
     1983        if (snapshot.data && snapshot.data->size())
     1984            m_setIconDataStatement->bindBlob(2, snapshot.data->data(), snapshot.data->size());
     1985       
     1986        if (m_setIconDataStatement->step() != SQLResultDone)
     1987            LOG_ERROR("Failed to set icon data for url %s", urlForLogging(snapshot.iconURL).utf8().data());
     1988
     1989        m_setIconDataStatement->reset();
     1990    }
    12121991}
    12131992

  • trunk/WebCore/loader/icon/IconDatabase.h

    r25348 r25439  
    3232
    3333#include "StringHash.h"
     34#include "Threading.h"
    3435#include "Timer.h"
     36#include <wtf/HashSet.h>
    3537#include <wtf/Noncopyable.h>
    36 #include <wtf/HashMap.h>
    37 #include <wtf/HashSet.h>
     38#include <wtf/OwnPtr.h>
    3839
    3940namespace WebCore {
    4041
     42class DocumentLoader;
    4143class Image;
    4244class IntSize;
    43 class IconDataCache;
     45class IconDatabaseClient;
     46class IconRecord;
     47class IconSnapshot;
     48class KURL;
     49class PageURLRecord;
     50class PageURLSnapshot;
    4451class SharedBuffer;
    4552
    … …  
    4855#endif
    4956
     57enum IconLoadDecision {
     58    IconLoadYes,
     59    IconLoadNo,
     60    IconLoadUnknown
     61};
     62
    5063class IconDatabase : Noncopyable {
     64
     65// *** Main Thread Only ***
    5166public:
     67    void setClient(IconDatabaseClient*);
     68
    5269    bool open(const String& path);
    53     bool isOpen() const;
    5470    void close();
    55    
    56     String databasePath() const;
    57    
     71           
    5872    void removeAllIcons();
    59    
    60     bool isEmpty();
    61  
     73
    6274    Image* iconForPageURL(const String&, const IntSize&, bool cache = true);
    63     Image* iconForIconURL(const String&, const IntSize&, bool cache = true);
     75    void readIconForPageURLFromDisk(const String&);
    6476    String iconURLForPageURL(const String&);
    6577    Image* defaultIcon(const IntSize&);
    … …  
    6779    void retainIconForPageURL(const String&);
    6880    void releaseIconForPageURL(const String&);
     81
     82    void setIconDataForIconURL(PassRefPtr<SharedBuffer> data, const String&);
     83    void setIconURLForPageURL(const String& iconURL, const String& pageURL);
     84
     85    IconLoadDecision loadDecisionForIconURL(const String&, DocumentLoader*);
     86    bool iconDataKnownForIconURL(const String&);
     87   
     88    void setEnabled(bool enabled);
     89    bool isEnabled() const;
    6990   
    7091    void setPrivateBrowsingEnabled(bool flag);
    7192    bool isPrivateBrowsingEnabled() const;
    72 
    73     bool hasEntryForIconURL(const String&);
    74 
    75     bool isIconExpiredForIconURL(const String&);
    76    
    77     void setIconDataForIconURL(PassRefPtr<SharedBuffer> data, const String&);
    78     void setHaveNoIconForIconURL(const String&);
    79    
    80     // Returns true if the set actually took place, false if the mapping already existed
    81     bool setIconURLForPageURL(const String& iconURL, const String& pageURL);
    82    
    83     void setEnabled(bool enabled);
    84     bool enabled() const;
    85 
    86     bool imported();
    87     void setImported(bool);
    88 
    89     static const String& defaultDatabaseFilename();
    90    
    91     // For clients to tell the IconDatabase that when it does open it should run the integrity check.
    92     // The flag is reset to false after the integrity check is run
     93   
     94    static void delayDatabaseCleanup();
     95    static void allowDatabaseCleanup();
    9396    static void checkIntegrityBeforeOpening();
    9497       
    95     bool checkIntegrity();
    96    
    9798    // Support for WebCoreStatistics in WebKit
    98     size_t pageURLMappingCount() const;
    99     size_t retainedPageURLCount() const;
    100     size_t iconRecordCount() const;
    101     size_t iconRecordCountWithData() const;
    102    
     99    size_t pageURLMappingCount();
     100    size_t retainedPageURLCount();
     101    size_t iconRecordCount();
     102    size_t iconRecordCountWithData();
     103
    103104private:
    104105    IconDatabase();
    … …  
    107108
    108109#if ENABLE(ICONDATABASE)
    109     // This tries to get the iconID for the IconURL and, if it doesn't exist and createIfNecessary is true,
    110     // it will create the entry and return the new iconID
    111     int64_t establishIconIDForIconURL(SQLDatabase&, const String&, bool createIfNecessary = true);
    112    
    113     // This method returns the SiteIcon for the given IconURL and, if it doesn't exist it creates it first
    114     IconDataCache* getOrCreateIconDataCache(const String& iconURL);
    115 
    116     // Remove traces of the given pageURL
    117     void forgetPageURL(const String& pageURL);
    118    
    119     // Remove the current database entry for this IconURL
    120     void forgetIconForIconURLFromDatabase(const String&);
    121    
    122     void setIconURLForPageURLInDatabase(const String&, const String&);
    123        
    124     // Called by the startup timer, this method removes all icons that are unretained
    125     // after initial retains are complete, and pageURLs that are dangling
    126     void pruneUnretainedIconsOnStartup(Timer<IconDatabase>*);
    127    
    128     // Called by the prune timer, this method periodically removes all the icons in the pending-deletion
    129     // queue
    130     void updateDatabase(Timer<IconDatabase>*);
    131    
    132     // This is called by updateDatabase and when private browsing shifts, and when the DB is closed down
    133     void syncDatabase();
    134 
    135     // Called to eliminate database inconsistency where pages point to non-existent iconIDs
    136     void checkForDanglingPageURLs(bool pruneIfFound);
    137    
    138     // Determine if an IconURL is still retained by anyone
    139     bool isIconURLRetained(const String&);
    140    
    141     // Do a quick check to make sure the database tables are in place and the db version is current
    142     bool isValidDatabase(SQLDatabase&);
    143        
    144     // Create the tables and triggers for the given database.
    145     void createDatabaseTables(SQLDatabase&);
    146    
    147     // Returns the image data for the given IconURL, checking both databases if necessary
    148     PassRefPtr<SharedBuffer> imageDataForIconURL(const String& iconURL);
    149    
    150     // Retains an iconURL, bringing it back from the brink if it was pending deletion
    151     void retainIconURL(const String& iconURL);
    152    
    153     // Releases an iconURL, putting it on the pending delete queue if it's totally released
    154     void releaseIconURL(const String& iconURL);
    155    
    156     // Query - Checks for at least 1 entry in the PageURL table
    157     bool pageURLTableIsEmptyQuery(SQLDatabase&);
    158    
    159     // Query - Returns the time stamp for an Icon entry
    160     int timeStampForIconURLQuery(SQLDatabase&, const String& iconURL);   
    161     SQLStatement* m_timeStampForIconURLStatement;
    162    
    163     // Query - Returns the IconURL for a PageURL
    164     String iconURLForPageURLQuery(SQLDatabase&, const String& pageURL);   
    165     SQLStatement* m_iconURLForPageURLStatement;
    166    
    167     // Query - Checks for the existence of the given IconURL in the Icon table
    168     bool hasIconForIconURLQuery(SQLDatabase& db, const String& iconURL);
    169     SQLStatement* m_hasIconForIconURLStatement;
    170    
    171     // Query - Deletes a PageURL from the PageURL table
    172     void forgetPageURLQuery(SQLDatabase& db, const String& pageURL);
    173     SQLStatement* m_forgetPageURLStatement;
    174    
    175     // Query - Sets the Icon.iconID for a PageURL in the PageURL table
    176     void setIconIDForPageURLQuery(SQLDatabase& db, int64_t, const String&);
    177     SQLStatement* m_setIconIDForPageURLStatement;
    178    
    179     // Query - Returns the iconID for the given IconURL
    180     int64_t getIconIDForIconURLQuery(SQLDatabase& db, const String& iconURL);
    181     SQLStatement* m_getIconIDForIconURLStatement;
    182    
    183     // Query - Creates the Icon entry for the given IconURL and returns the resulting iconID
    184     int64_t addIconForIconURLQuery(SQLDatabase& db, const String& iconURL);
    185     SQLStatement* m_addIconForIconURLStatement;
    186    
    187     // Query - Returns the image data from the given database for the given IconURL
    188     PassRefPtr<SharedBuffer> imageDataForIconURLQuery(SQLDatabase& db, const String& iconURL);
    189     SQLStatement* m_imageDataForIconURLStatement;
    190 
    191     // Query - Returns whether or not the "imported" flag is set
    192     bool importedQuery(SQLDatabase&);
    193     SQLStatement* m_importedStatement;
    194    
    195     // Query - Sets the "imported" flag
    196     void setImportedQuery(SQLDatabase&, bool);
    197     SQLStatement* m_setImportedStatement;
    198 
    199     void deleteAllPreparedStatements(bool withSync);
    200 
    201     SQLDatabase m_mainDB;
    202     SQLDatabase m_privateBrowsingDB;
    203     SQLDatabase* m_currentDB;
    204    
    205     IconDataCache* m_defaultIconDataCache;
     110    // This is called on the main thread via the callOnMainThread() function which currently
     111    // doesn't have any way to allow it to be an instance method, which it should be
     112    static void notifyPendingLoadDecisions();
     113   
     114    void notifyPendingLoadDecisionsInternal();
     115
     116    void wakeSyncThread();
     117    void scheduleOrDeferSyncTimer();
     118    Timer<IconDatabase> m_syncTimer;
     119    void syncTimerFired(Timer<IconDatabase>*);
     120   
     121    pthread_t m_syncThread;
     122    bool m_syncThreadRunning;
     123   
     124    HashSet<RefPtr<DocumentLoader> > m_loadersPendingDecision;
     125
     126    IconRecord* m_defaultIconRecord;
     127#endif // ENABLE(ICONDATABASE)
     128
     129// *** Any Thread ***
     130public:
     131    bool isOpen() const;
     132    String databasePath() const;
     133    static String defaultDatabaseFilename();
     134
     135private:
     136#if ENABLE(ICONDATABASE)
     137    IconRecord* getOrCreateIconRecord(const String& iconURL);
     138    PageURLRecord* getOrCreatePageURLRecord(const String& pageURL);
    206139   
    207140    bool m_isEnabled;
    208141    bool m_privateBrowsingEnabled;
    209142   
    210     Timer<IconDatabase> m_startupTimer;
    211     Timer<IconDatabase> m_updateTimer;
     143    mutable Mutex m_syncLock;
     144    ThreadCondition m_syncCondition;
     145    // Holding m_syncLock is required when accessing m_completeDatabasePath
     146    String m_completeDatabasePath;
     147
     148    Mutex m_removeLock;
     149    ThreadCondition m_removeCondition;
     150   
     151    bool m_threadTerminationRequested;
     152    bool m_removeIconsRequested;
     153    bool m_iconURLImportComplete;
     154   
     155    Mutex m_urlAndIconLock;
     156    // Holding m_urlAndIconLock is required when accessing any of the following data structures or the objects they contain
     157    HashMap<String, IconRecord*> m_iconURLToRecordMap;
     158    HashMap<String, PageURLRecord*> m_pageURLToRecordMap;
     159    HashSet<String> m_retainedPageURLs;
     160
     161    Mutex m_pendingSyncLock;
     162    // Holding m_pendingSyncLock is required when accessing any of the following data structures
     163    HashMap<String, PageURLSnapshot> m_pageURLsPendingSync;
     164    HashMap<String, IconSnapshot> m_iconsPendingSync;
     165   
     166    Mutex m_pendingReadingLock;   
     167    // Holding m_pendingSyncLock is required when accessing any of the following data structures - when dealing with IconRecord*s, holding m_urlAndIconLock is also required
     168    HashSet<String> m_pageURLsPendingImport;
     169    HashSet<String> m_pageURLsInterestedInIcons;
     170    HashSet<IconRecord*> m_iconsPendingReading;
     171
     172// *** Sync Thread Only ***
     173public:
     174    // Should be used only on the sync thread and only by the Safari 2 Icons import procedure
     175    void importIconURLForPageURL(const String& iconURL, const String& pageURL);
     176    void importIconDataForIconURL(PassRefPtr<SharedBuffer> data, const String& iconURL);
     177   
     178    bool shouldStopThreadActivity() const;
     179
     180private:   
     181    static void* iconDatabaseSyncThreadStart(void *);
     182    void* iconDatabaseSyncThread();
     183   
     184    // The following block of methods are called exclusively by the sync thread to manage i/o to and from the database
     185    // Each method should periodically monitor m_threadTerminationRequested when it makes sense to return early on shutdown
     186    void performOpenInitialization();
     187    bool checkIntegrity();
     188    void performURLImport();
     189    void* syncThreadMainLoop();
     190    bool readFromDatabase();
     191    bool writeToDatabase();
     192    void pruneUnretainedIcons();
     193    void checkForDanglingPageURLs(bool pruneIfFound);
     194    void removeAllIconsOnThread();
     195    void deleteAllPreparedStatements();
     196    void* cleanupSyncThread();
     197
     198    // Record (on disk) whether or not Safari 2-style icons were imported (once per dataabse)
     199    bool imported();
     200    void setImported(bool);
    212201   
    213202    bool m_initialPruningComplete;
    214    
     203       
     204    void setIconURLForPageURLInSQLDatabase(const String&, const String&);
     205    void setIconIDForPageURLInSQLDatabase(int64_t, const String&);
     206    void removePageURLFromSQLDatabase(const String& pageURL);
     207    int64_t getIconIDForIconURLFromSQLDatabase(const String& iconURL);
     208    int64_t addIconURLToSQLDatabase(const String&);
     209    PassRefPtr<SharedBuffer> getImageDataForIconURLFromSQLDatabase(const String& iconURL);
     210    void removeIconFromSQLDatabase(const String& iconURL);
     211    void writeIconSnapshotToSQLDatabase(const IconSnapshot&);   
     212   
     213    // The client is set by the main thread before the thread starts, and from then on is only used by the sync thread
     214    IconDatabaseClient* m_client;
     215   
     216    SQLDatabase m_syncDB;
     217   
     218    // Track whether the "Safari 2" import is complete and/or set in the database
    215219    bool m_imported;
    216     mutable bool m_isImportedSet;
    217    
    218     HashMap<String, IconDataCache*> m_iconURLToIconDataCacheMap;
    219     HashSet<IconDataCache*> m_iconDataCachesPendingUpdate;
    220    
    221     HashMap<String, String> m_pageURLToIconURLMap;
    222     HashSet<String> m_pageURLsPendingAddition;
    223 
    224     // This will keep track of the retaincount for each pageURL
    225     HashMap<String, int> m_pageURLToRetainCount;
    226     // This will keep track of the retaincount for each iconURL (ie - the number of pageURLs retaining this icon)
    227     HashMap<String, int> m_iconURLToRetainCount;
    228 
    229     HashSet<String> m_pageURLsPendingDeletion;
    230     HashSet<String> m_iconURLsPendingDeletion;
    231 #endif
     220    bool m_isImportedSet;
     221   
     222    OwnPtr<SQLStatement> m_setIconIDForPageURLStatement;
     223    OwnPtr<SQLStatement> m_removePageURLStatement;
     224    OwnPtr<SQLStatement> m_getIconIDForIconURLStatement;
     225    OwnPtr<SQLStatement> m_getImageDataForIconURLStatement;
     226    OwnPtr<SQLStatement> m_addIconToIconInfoStatement;
     227    OwnPtr<SQLStatement> m_addIconToIconDataStatement;
     228    OwnPtr<SQLStatement> m_getImageDataStatement;
     229    OwnPtr<SQLStatement> m_deletePageURLsForIconURLStatement;
     230    OwnPtr<SQLStatement> m_deleteIconFromIconInfoStatement;
     231    OwnPtr<SQLStatement> m_deleteIconFromIconDataStatement;
     232    OwnPtr<SQLStatement> m_updateIconInfoStatement;
     233    OwnPtr<SQLStatement> m_updateIconDataStatement;
     234    OwnPtr<SQLStatement> m_setIconInfoStatement;
     235    OwnPtr<SQLStatement> m_setIconDataStatement;
     236#endif // ENABLE(ICONDATABASE)
    232237};
    233238

  • trunk/WebCore/loader/icon/IconDatabaseNone.cpp

    r25081 r25439  
    4646const int updateTimerDelay = 5;
    4747
    48 const String& IconDatabase::defaultDatabaseFilename()
     48String IconDatabase::defaultDatabaseFilename()
    4949{
    5050    static String defaultDatabaseFilename = "Icons.db";
    … …  
    8686}
    8787
    88 bool IconDatabase::isEmpty()
    89 {
    90   return true;
    91 }
    92 
    9388void IconDatabase::setPrivateBrowsingEnabled(bool flag)
    9489{
    … …  
    10095}
    10196
     97void IconDatabase::readIconForPageURLFromDisk(const String&)
     98{
     99
     100}
     101
    102102Image* IconDatabase::iconForPageURL(const String& pageURL, const IntSize& size, bool cache)
    103103{
    … …  
    105105}
    106106
    107 bool IconDatabase::isIconExpiredForIconURL(const String& iconURL)
     107
     108IconLoadDecision IconDatabase::loadDecisionForIconURL(const String&, DocumentLoader*)
     109{
     110    return IconLoadNo;
     111}
     112
     113bool IconDatabase::iconDataKnownForIconURL(const String&)
    108114{
    109115    return false;
    … …  
    132138}
    133139
    134 void IconDatabase::setHaveNoIconForIconURL(const String& iconURL)
     140void IconDatabase::setIconURLForPageURL(const String& iconURL, const String& pageURL)
    135141{
    136 }
    137 
    138 bool IconDatabase::setIconURLForPageURL(const String& iconURL, const String& pageURL)
    139 {
    140     return false;
    141 }
    142 
    143 bool IconDatabase::hasEntryForIconURL(const String& iconURL)
    144 {
    145     return false;
    146142}
    147143
    … …  
    150146}
    151147
    152 bool IconDatabase::enabled() const
     148bool IconDatabase::isEnabled() const
    153149{
    154150    return false;
    … …  
    178174}
    179175
     176void IconDatabase::delayDatabaseCleanup()
     177{
     178}
     179
     180void IconDatabase::allowDatabaseCleanup()
     181{
     182}
     183
     184void IconDatabase::setClient(IconDatabaseClient*)
     185{
     186}
     187
    180188} // namespace WebCore

  • trunk/WebCore/platform/SharedBuffer.cpp

    r24395 r25439  
    6868}
    6969
     70PassRefPtr<SharedBuffer> SharedBuffer::copy() const
     71{
     72    return new SharedBuffer(data(), size());
     73}
     74
     75
    7076#if !PLATFORM(MAC)
    7177

  • trunk/WebCore/platform/SharedBuffer.h

    r24289 r25439  
    6868    unsigned platformDataSize() const;
    6969
     70    PassRefPtr<SharedBuffer> copy() const;
     71   
    7072private:
    7173    void clearPlatformData();

  • trunk/WebCore/platform/graphics/svg/SVGImageEmptyClients.h

    r25430 r25439  
    275275    virtual void didPerformFirstNavigation() const {}
    276276
     277    virtual void registerForIconNotification(bool listen) {}
     278
    277279#if PLATFORM(MAC)
    278280    virtual NSCachedURLResponse* willCacheResponse(DocumentLoader*, unsigned long identifier, NSCachedURLResponse* response) const { return response; }

  • trunk/WebCore/platform/win/TemporaryLinkStubs.cpp

    r25088 r25439  
    7272#include "SubresourceLoader.h"
    7373#include "SystemTime.h"
     74#include "Threading.h"
    7475#include "loader.h"
    7576#include <stdio.h>
    … …  
    114115Vector<String> WebCore::supportedKeySizes() { notImplemented(); return Vector<String>(); }
    115116String WebCore::signedPublicKeyAndChallengeString(unsigned, const String&, const KURL&) { notImplemented(); return String(); }
     117
     118void WebCore::callOnMainThread(void (*)()) { notImplemented(); }

  • trunk/WebKit/ChangeLog

    r25432 r25439  
     12007-09-08  Brady Eidson  <beidson@apple.com>
     2
     3        Reviewed by Darin
     4
     5        <rdar://problem/5434431> - Asynchronous Icon Database
     6
     7        WebKit side of things
     8        Mainly, there are Notifications WebKit has to listen for now that tell it when to either call back into WebCore
     9        for some purpose or to send the webView:didReceiveIcon: delegate call
     10
     11        Many smaller tweaks as well.
     12
     13        * Misc/WebIconDatabase.h:
     14        * Misc/WebIconDatabase.mm:
     15        (defaultClient):
     16        (-[WebIconDatabase init]):
     17        (+[WebIconDatabase delayDatabaseCleanup]): Accessor so clients can prevent the thread from cleaning up the database
     18          before they've done all their necessary retaining of icons.
     19        (+[WebIconDatabase allowDatabaseCleanup]):
     20        (-[WebIconDatabase removeAllIcons]):
     21        (-[WebIconDatabase _isEnabled]):
     22        (-[WebIconDatabase _sendNotificationForURL:]):
     23        (-[WebIconDatabase _sendDidRemoveAllIconsNotification]):
     24        (-[WebIconDatabase _databaseDirectory]):
     25
     26        (-[ThreadEnabler threadEnablingSelector:]): Quick and dirty class to enabled Cocoa multithreading
     27        (+[ThreadEnabler enableThreading]):
     28        (importToWebCoreFormat):
     29        * Misc/WebIconDatabaseInternal.h: Expose the internal methods of WebIconDatabase that are required by WebIconDatabaseClient
     30
     31        * Misc/WebNSNotificationCenterExtras.h: Added. - Great utility class whose design was borrowed from Colloquy
     32          that allows the posting of a Cocoa notification on the main thread from *any* thread
     33        * Misc/WebNSNotificationCenterExtras.m: Added.
     34        (-[NSNotificationCenter postNotificationOnMainThreadWithName:object:]):
     35        (-[NSNotificationCenter postNotificationOnMainThreadWithName:object:userInfo:]):
     36        (-[NSNotificationCenter postNotificationOnMainThreadWithName:object:userInfo:waitUntilDone:]):
     37        (+[NSNotificationCenter _postNotificationName:]):
     38
     39        * WebCoreSupport/WebFrameLoaderClient.h:
     40        * WebCoreSupport/WebFrameLoaderClient.mm:
     41        (WebFrameLoaderClient::dispatchDidReceiveIcon): Send the webView:didReceiveIcon: delegate call
     42        (WebFrameLoaderClient::registerForIconNotification):
     43
     44        * WebCoreSupport/WebIconDatabaseClient.h: Added.
     45        * WebCoreSupport/WebIconDatabaseClient.mm: Added.
     46        (WebIconDatabaseClient::performImport):  Perform the Safari 2 icon import
     47        (WebIconDatabaseClient::dispatchDidRemoveAllIcons): Send the NSNotification
     48        (WebIconDatabaseClient::dispatchDidAddIconForPageURL): Ditto
     49
     50        * WebView/WebView.mm:
     51        (-[WebView _receivedIconChangedNotification:]): Check and see if this notification is for this WebView's current URL by
     52          calling back into the IconDatabase
     53        (-[WebView _registerForIconNotification:]): Support for WebIconDatabaseClient
     54        (-[WebView _dispatchDidReceiveIconFromWebFrame:]): Dispatch this delegate call as well as unregister for the notification
     55        * WebView/WebViewInternal.h:
     56
     57        * WebKit.xcodeproj/project.pbxproj:
     58
    1592007-09-07  Geoffrey Garen  <ggaren@apple.com>
    260
    … …  
    103161        grow inverse-squaredly relative to available space.
    104162
     163>>>>>>> .r25438
    1051642007-09-05  Timothy Hatcher  <timothy@apple.com>
    106165
    … …  
    222281        * WebView/WebViewPrivate.h:
    223282
     283>>>>>>> .r25411
    2242842007-09-04  Timothy Hatcher  <timothy@apple.com>
    225285

  • trunk/WebKit/Misc/WebIconDatabase.h

    r16956 r25439  
    123123- (void)releaseIconForURL:(NSString *)URL;
    124124
     125/*!
     126    @method delayDatabaseCleanup:
     127    @discussion Only effective if called before the database begins removing icons.
     128    delayDatabaseCleanUp increments an internal counter that when 0 begins the database clean-up.
     129    The counter equals 0 at initialization.
     130*/
     131+ (void)delayDatabaseCleanup;
     132
     133/*!
     134    @method allowDatabaseCleanup:
     135    @discussion Informs the database that it now can begin removing icons.
     136    allowDatabaseCleanup decrements an internal counter that when 0 begins the database clean-up.
     137    The counter equals 0 at initialization.
     138*/
     139+ (void)allowDatabaseCleanup;
     140
    125141- (void)setDelegate:(id)delegate;
    126142- (id)delegate;

  • trunk/WebKit/Misc/WebIconDatabase.mm

    r25081 r25439  
    2929#import "WebIconDatabaseInternal.h"
    3030
     31#import "WebIconDatabaseClient.h"
    3132#import "WebIconDatabaseDelegate.h"
    3233#import "WebKitLogging.h"
    3334#import "WebKitNSStringExtras.h"
     35#import "WebNSNotificationCenterExtras.h"
    3436#import "WebNSURLExtras.h"
    3537#import "WebPreferences.h"
    3638#import "WebTypesInternal.h"
     39#import <WebCore/FoundationExtras.h>
    3740#import <WebCore/IconDatabase.h>
    3841#import <WebCore/Image.h>
    … …  
    6164#define UniqueFilePathSize (34)
    6265
    63 @interface WebIconDatabase (WebInternal)
     66static WebIconDatabaseClient* defaultClient()
     67{
     68    static WebIconDatabaseClient* defaultClient = new WebIconDatabaseClient();
     69    return defaultClient;
     70}
     71
     72@interface WebIconDatabase (WebReallyInternal)
    6473- (BOOL)_isEnabled;
    65 - (void)_setIconURL:(NSString *)iconURL forURL:(NSString *)URL;
    66 - (BOOL)_hasEntryForIconURL:(NSString *)iconURL;
    6774- (void)_sendNotificationForURL:(NSString *)URL;
     75- (void)_sendDidRemoveAllIconsNotification;
    6876- (NSImage *)_iconForFileURL:(NSString *)fileURL withSize:(NSSize)size;
    6977- (void)_resetCachedWebPreferences:(NSNotification *)notification;
    … …  
    7280- (NSImage *)_iconFromDictionary:(NSMutableDictionary *)icons forSize:(NSSize)size cache:(BOOL)cache;
    7381- (void)_scaleIcon:(NSImage *)icon toSize:(NSSize)size;
    74 - (void)_importToWebCoreFormat;
    7582- (NSString *)_databaseDirectory;
    7683@end
    … …  
    104111    if (!enabled)
    105112        return self;
     113    iconDatabase()->setClient(defaultClient());
    106114   
    107115    // Figure out the directory we should be using for the icon.db
    … …  
    118126    }
    119127   
    120     // Open the WebCore icon database and import the old WebKit icon database
     128    // Set the private browsing pref then open the WebCore icon database
     129    iconDatabase()->setPrivateBrowsingEnabled([[WebPreferences standardPreferences] privateBrowsingEnabled]);
    121130    if (!iconDatabase()->open(databaseDirectory))
    122131        LOG_ERROR("Unable to open icon database");
    123     else if ([self _isEnabled] && !iconDatabase()->imported()) {
    124         [self _importToWebCoreFormat];
    125        
    126 #ifndef BUILDING_ON_TIGER
    127         // Tell backup software (i.e., Time Machine) to never back up the icon database, because 
    128         // it's a large file that changes frequently, thus using a lot of backup disk space, and
    129         // it's unlikely that many users would be upset about it not being backed up. We do this
    130         // here because this code is only executed once for each icon database instance. We could
    131         // make this configurable on a per-client basis someday if that seemed useful.
    132         // See <rdar://problem/5320208>.
    133         CFStringRef databasePath = iconDatabase()->databasePath().createCFString();
    134         CFURLRef databasePathURL = CFURLCreateWithFileSystemPath(0, databasePath, kCFURLPOSIXPathStyle, FALSE);
    135         CFRelease(databasePath);
    136         CSBackupSetItemExcluded(databasePathURL, true, true);
    137         CFRelease(databasePathURL);
    138 #endif
    139     }
    140 
    141     iconDatabase()->setPrivateBrowsingEnabled([[WebPreferences standardPreferences] privateBrowsingEnabled]);
    142132   
    143133    // Register for important notifications
    … …  
    223213}
    224214
     215+ (void)delayDatabaseCleanup
     216{
     217    ASSERT_MAIN_THREAD();
     218
     219    IconDatabase::delayDatabaseCleanup();
     220}
     221
     222+ (void)allowDatabaseCleanup
     223{
     224    ASSERT_MAIN_THREAD();
     225
     226    IconDatabase::allowDatabaseCleanup();
     227}
     228
    225229- (void)setDelegate:(id)delegate
    226230{
    … …  
    244248    if (![self _isEnabled])
    245249        return;
     250
     251    // Via the IconDatabaseClient interface, removeAllIcons() will send the WebIconDatabaseDidRemoveAllIconsNotification
    246252    iconDatabase()->removeAllIcons();
    247     // FIXME: This notification won't get sent if WebCore calls removeAllIcons.
    248     [[NSNotificationCenter defaultCenter] postNotificationName:WebIconDatabaseDidRemoveAllIconsNotification
     253}
     254
     255@end
     256
     257@implementation WebIconDatabase (WebPrivate)
     258
     259+ (void)_checkIntegrityBeforeOpening
     260{
     261    iconDatabase()->checkIntegrityBeforeOpening();
     262}
     263
     264@end
     265
     266@implementation WebIconDatabase (WebInternal)
     267
     268- (BOOL)_isEnabled
     269{
     270    return iconDatabase()->isEnabled();
     271}
     272
     273- (void)_sendNotificationForURL:(NSString *)URL
     274{
     275    ASSERT(URL);
     276   
     277    NSDictionary *userInfo = [NSDictionary dictionaryWithObject:URL
     278                                                         forKey:WebIconNotificationUserInfoURLKey];
     279                                                         
     280    [[NSNotificationCenter defaultCenter] postNotificationOnMainThreadWithName:WebIconDatabaseDidAddIconNotification
     281                                                        object:self
     282                                                      userInfo:userInfo];
     283}
     284
     285- (void)_sendDidRemoveAllIconsNotification
     286{
     287    [[NSNotificationCenter defaultCenter] postNotificationOnMainThreadWithName:WebIconDatabaseDidRemoveAllIconsNotification
    249288                                                        object:self
    250289                                                      userInfo:nil];
    251 }
    252 
    253 @end
    254 
    255 @implementation WebIconDatabase (WebPrivate)
    256 
    257 + (void)_checkIntegrityBeforeOpening
    258 {
    259     iconDatabase()->checkIntegrityBeforeOpening();
    260 }
    261 
    262 @end
    263 
    264 @implementation WebIconDatabase (WebInternal)
    265 
    266 - (BOOL)_isEnabled
    267 {
    268     return iconDatabase()->enabled();
    269 }
    270 
    271 - (void)_setIconURL:(NSString *)iconURL forURL:(NSString *)URL
    272 {
    273     ASSERT(iconURL);
    274     ASSERT(URL);
    275     ASSERT([self _isEnabled]);
    276    
    277     // If this iconURL already maps to this pageURL, don't bother sending the notification
    278     // The WebCore::IconDatabase returns TRUE if we should send the notification, and false if we shouldn't.
    279     // This is a measurable win on the iBench - about 1% worth on average
    280     if (iconDatabase()->setIconURLForPageURL(iconURL, URL))
    281         // FIXME: This notification won't get set when WebCore sets an icon.
    282         [self _sendNotificationForURL:URL];
    283 }
    284 
    285 - (BOOL)_hasEntryForIconURL:(NSString *)iconURL;
    286 {
    287     ASSERT([self _isEnabled]);
    288 
    289     return iconDatabase()->hasEntryForIconURL(iconURL);
    290 }
    291 
    292 - (void)_sendNotificationForURL:(NSString *)URL
    293 {
    294     ASSERT(URL);
    295    
    296     NSDictionary *userInfo = [NSDictionary dictionaryWithObject:URL
    297                                                          forKey:WebIconNotificationUserInfoURLKey];
    298     [[NSNotificationCenter defaultCenter] postNotificationName:WebIconDatabaseDidAddIconNotification
    299                                                         object:self
    300                                                       userInfo:userInfo];
    301290}
    302291
    … …  
    513502}
    514503
    515 - (void)_importToWebCoreFormat
    516 {
    517     ASSERT(_private);
    518 
    519     // If we've already performed the import once we shouldn't be trying to do it again
    520     ASSERT(!iconDatabase()->imported());
     504- (NSString *)_databaseDirectory
     505{
     506    NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
     507
     508    // Figure out the directory we should be using for the icon.db
     509    NSString *databaseDirectory = [defaults objectForKey:WebIconDatabaseDirectoryDefaultsKey];
     510    if (!databaseDirectory) {
     511        databaseDirectory = WebIconDatabasePath;
     512        [defaults setObject:databaseDirectory forKey:WebIconDatabaseDirectoryDefaultsKey];
     513    }
     514   
     515    return [[databaseDirectory stringByExpandingTildeInPath] stringByStandardizingPath];
     516}
     517
     518@end
     519
     520@implementation WebIconDatabasePrivate
     521@end
     522
     523@interface ThreadEnabler : NSObject {
     524}
     525+ (void)enableThreading;
     526
     527- (void)threadEnablingSelector:(id)arg;
     528@end
     529
     530@implementation ThreadEnabler
     531
     532- (void)threadEnablingSelector:(id)arg
     533{
     534    return;
     535}
     536
     537+ (void)enableThreading
     538{
     539    ThreadEnabler *enabler = [[ThreadEnabler alloc] init];
     540    [NSThread detachNewThreadSelector:@selector(threadEnablingSelector:) toTarget:enabler withObject:nil];
     541    [enabler release];
     542}
     543
     544@end
     545
     546bool importToWebCoreFormat()
     547{
     548    // Since this is running on a secondary POSIX thread and Cocoa cannot be used multithreaded unless an NSThread has been detached,
     549    // make sure that happens here for all WebKit clients
     550    if (![NSThread isMultiThreaded])
     551        [ThreadEnabler enableThreading];
     552    ASSERT([NSThread isMultiThreaded]);   
     553   
     554#ifndef BUILDING_ON_TIGER
     555    // Tell backup software (i.e., Time Machine) to never back up the icon database, because 
     556    // it's a large file that changes frequently, thus using a lot of backup disk space, and
     557    // it's unlikely that many users would be upset about it not being backed up. We do this
     558    // here because this code is only executed once for each icon database instance. We could
     559    // make this configurable on a per-client basis someday if that seemed useful.
     560    // See <rdar://problem/5320208>.
     561    CFStringRef databasePath = iconDatabase()->databasePath().createCFString();
     562    if (databasePath) {
     563        CFURLRef databasePathURL = CFURLCreateWithFileSystemPath(0, databasePath, kCFURLPOSIXPathStyle, FALSE);
     564        CFRelease(databasePath);
     565        CSBackupSetItemExcluded(databasePathURL, true, true);
     566        CFRelease(databasePathURL);
     567    }
     568#endif
    521569
    522570    // Get the directory the old icon database *should* be in
    … …  
    549597        if (!iconURL)
    550598            continue;
    551         iconDatabase()->setIconURLForPageURL(iconURL, url);
     599        iconDatabase()->importIconURLForPageURL(iconURL, url);
     600        if (iconDatabase()->shouldStopThreadActivity())
     601            return false;
    552602    }   
    553603
    … …  
    561611        iconData = iconDataFromPathForIconURL(databaseDirectory, url);
    562612        if (iconData)
    563             iconDatabase()->setIconDataForIconURL(SharedBuffer::wrapNSData(iconData), url);
     613            iconDatabase()->importIconDataForIconURL(SharedBuffer::wrapNSData(iconData), url);
    564614        else {
    565615            // This really *shouldn't* happen, so it'd be good to track down why it might happen in a debug build
    566616            // however, we do know how to handle it gracefully in release
    567617            LOG_ERROR("%@ is marked as having an icon on disk, but we couldn't get the data for it", url);
    568             iconDatabase()->setHaveNoIconForIconURL(url);
     618            iconDatabase()->importIconDataForIconURL(0, url);
    569619        }
    570     }
    571 
    572     iconDatabase()->setImported(true);
     620        if (iconDatabase()->shouldStopThreadActivity())
     621            return false;
     622    }
    573623   
    574624    // After we're done importing old style icons over to webcore icons, we delete the entire directory hierarchy
    … …  
    594644    if (!foundIconDB)
    595645        rmdir([databaseDirectory fileSystemRepresentation]);
    596 }
    597 
    598 - (NSString *)_databaseDirectory
    599 {
    600     NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
    601 
    602     // Figure out the directory we should be using for the icon.db
    603     NSString *databaseDirectory = [defaults objectForKey:WebIconDatabaseDirectoryDefaultsKey];
    604     if (!databaseDirectory) {
    605         databaseDirectory = WebIconDatabasePath;
    606         [defaults setObject:databaseDirectory forKey:WebIconDatabaseDirectoryDefaultsKey];
    607     }
    608    
    609     return [[databaseDirectory stringByExpandingTildeInPath] stringByStandardizingPath];
    610 }
    611 
    612 @end
    613 
    614 @implementation WebIconDatabasePrivate
    615 @end
     646   
     647    return true;
     648}
    616649
    617650NSImage *webGetNSImage(Image* image, NSSize size)

  • trunk/WebKit/Misc/WebIconDatabaseInternal.h

    r17704 r25439  
    4141@end
    4242
     43@interface WebIconDatabase (WebInternal)
     44- (void)_sendNotificationForURL:(NSString *)URL;
     45- (void)_sendDidRemoveAllIconsNotification;
     46@end
     47
     48extern bool importToWebCoreFormat();
    4349NSImage *webGetNSImage(WebCore::Image*, NSSize);

  • trunk/WebKit/WebCoreSupport/WebFrameLoaderClient.h

    r25430 r25439  
    198198    virtual void didPerformFirstNavigation() const;
    199199
     200    virtual void registerForIconNotification(bool listen);
     201
    200202    void deliverArchivedResourcesAfterDelay() const;
    201203    bool canUseArchivedResource(NSURLRequest *) const;

  • trunk/WebKit/WebCoreSupport/WebFrameLoaderClient.mm

    r25430 r25439  
    501501    WebView *webView = getWebView(m_webFrame.get());   
    502502
    503     // FIXME: This willChangeValueForKey call is too late, because the icon has already changed by now.
    504     [webView _willChangeValueForKey:_WebMainFrameIconKey];
    505 
    506     WebFrameLoadDelegateImplementationCache implementations = WebViewGetFrameLoadDelegateImplementations(webView);
    507     if (implementations.didReceiveIconForFrameFunc) {
    508         Image* image = iconDatabase()->iconForPageURL(core(m_webFrame.get())->loader()->url().url(), IntSize(16, 16));
    509         if (NSImage *icon = webGetNSImage(image, NSMakeSize(16, 16)))
    510             CallFrameLoadDelegate(implementations.didReceiveIconForFrameFunc, webView, @selector(webView:didReceiveIcon:forFrame:), icon, m_webFrame.get());
    511     }
    512 
    513     [webView _didChangeValueForKey:_WebMainFrameIconKey];
     503    [webView _dispatchDidReceiveIconFromWebFrame:m_webFrame.get()];
    514504}
    515505
    … …  
    12421232}
    12431233
     1234void WebFrameLoaderClient::registerForIconNotification(bool listen)
     1235{
     1236    [[m_webFrame.get() webView] _registerForIconNotification:listen];
     1237}
     1238
    12441239void WebFrameLoaderClient::didPerformFirstNavigation() const
    12451240{

  • trunk/WebKit/WebKit.xcodeproj/project.pbxproj

    r25349 r25439  
    3939                4BF99F900AE050BC00815C2B /* WebEditorClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 4BF99F8E0AE050BC00815C2B /* WebEditorClient.h */; settings = {ATTRIBUTES = (); }; };
    4040                4BF99F910AE050BC00815C2B /* WebEditorClient.mm in Sources */ = {isa = PBXBuildFile; fileRef = 4BF99F8F0AE050BC00815C2B /* WebEditorClient.mm */; };
     41                51494CD60C7EBDE0004178C5 /* WebIconDatabaseClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 51494CD40C7EBDE0004178C5 /* WebIconDatabaseClient.h */; };
     42                51494CD70C7EBDE0004178C5 /* WebIconDatabaseClient.mm in Sources */ = {isa = PBXBuildFile; fileRef = 51494CD50C7EBDE0004178C5 /* WebIconDatabaseClient.mm */; };
     43                51494D240C7EC1B7004178C5 /* WebNSNotificationCenterExtras.h in Headers */ = {isa = PBXBuildFile; fileRef = 51494D220C7EC1B6004178C5 /* WebNSNotificationCenterExtras.h */; };
     44                51494D250C7EC1B7004178C5 /* WebNSNotificationCenterExtras.m in Sources */ = {isa = PBXBuildFile; fileRef = 51494D230C7EC1B7004178C5 /* WebNSNotificationCenterExtras.m */; };
    4145                51B2A1000ADB15D0002A9BEE /* WebIconDatabaseDelegate.h in Headers */ = {isa = PBXBuildFile; fileRef = 51B2A0FF0ADB15D0002A9BEE /* WebIconDatabaseDelegate.h */; };
    4246                51C714FB0B20F79F00E5E33C /* WebBackForwardListInternal.h in Headers */ = {isa = PBXBuildFile; fileRef = 51C714FA0B20F79F00E5E33C /* WebBackForwardListInternal.h */; };
    … …  
    367371                51443F9B0429392B00CA2D3A /* WebPolicyDelegate.mm */ = {isa = PBXFileReference; fileEncoding = 30; indentWidth = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebPolicyDelegate.mm; sourceTree = "<group>"; tabWidth = 8; usesTabs = 0; };
    368372                51443F9C0429392B00CA2D3A /* WebPolicyDelegatePrivate.h */ = {isa = PBXFileReference; fileEncoding = 30; indentWidth = 4; lastKnownFileType = sourcecode.c.h; path = WebPolicyDelegatePrivate.h; sourceTree = "<group>"; tabWidth = 8; usesTabs = 0; };
     373                51494CD40C7EBDE0004178C5 /* WebIconDatabaseClient.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = WebIconDatabaseClient.h; sourceTree = "<group>"; };
     374                51494CD50C7EBDE0004178C5 /* WebIconDatabaseClient.mm */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.cpp.objcpp; path = WebIconDatabaseClient.mm; sourceTree = "<group>"; };
     375                51494D220C7EC1B6004178C5 /* WebNSNotificationCenterExtras.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebNSNotificationCenterExtras.h; sourceTree = "<group>"; };
     376                51494D230C7EC1B7004178C5 /* WebNSNotificationCenterExtras.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = WebNSNotificationCenterExtras.m; sourceTree = "<group>"; };
    369377                5152FADD033FC50400CA2ACD /* WebDefaultContextMenuDelegate.h */ = {isa = PBXFileReference; fileEncoding = 4; indentWidth = 4; lastKnownFileType = sourcecode.c.h; path = WebDefaultContextMenuDelegate.h; sourceTree = "<group>"; tabWidth = 8; usesTabs = 0; };
    370378                5152FADE033FC50400CA2ACD /* WebDefaultContextMenuDelegate.mm */ = {isa = PBXFileReference; fileEncoding = 4; indentWidth = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebDefaultContextMenuDelegate.mm; sourceTree = "<group>"; tabWidth = 8; usesTabs = 0; };
    … …  
    763771                                8398847A03426FB000BC5F5E /* WebNSImageExtras.h */,
    764772                                8398847B03426FB000BC5F5E /* WebNSImageExtras.m */,
     773                                51494D220C7EC1B6004178C5 /* WebNSNotificationCenterExtras.h */,
     774                                51494D230C7EC1B7004178C5 /* WebNSNotificationCenterExtras.m */,
    765775                                93D1FE13067EB10B009CE68A /* WebNSObjectExtras.h */,
    766776                                ED2B2474033A2DA800C1A526 /* WebNSPasteboardExtras.h */,
    … …  
    10151025                                931633EA0AEDFF930062B92D /* WebFrameLoaderClient.h */,
    10161026                                931633EE0AEDFFAE0062B92D /* WebFrameLoaderClient.mm */,
     1027                                51494CD40C7EBDE0004178C5 /* WebIconDatabaseClient.h */,
     1028                                51494CD50C7EBDE0004178C5 /* WebIconDatabaseClient.mm */,
    10171029                                9CE1F8A302A5C6F30ECA2ACD /* WebImageRendererFactory.m */,
    10181030                                06693DDA0BFBA85200216072 /* WebInspectorClient.h */,
    … …  
    12741286                                BCDFA8F90C10B6F500D3A10C /* WebKitPluginContainerView.h in Headers */,
    12751287                                5D7BF8140C2A1D90008CE06D /* WebInspector.h in Headers */,
     1288                                51494CD60C7EBDE0004178C5 /* WebIconDatabaseClient.h in Headers */,
     1289                                51494D240C7EC1B7004178C5 /* WebNSNotificationCenterExtras.h in Headers */,
    12761290                        );
    12771291                        runOnlyForDeploymentPostprocessing = 0;
    … …  
    15521566                                BCDFA9130C10B93E00D3A10C /* WebKitPluginContainerView.mm in Sources */,
    15531567                                5D7BF8150C2A1D90008CE06D /* WebInspector.mm in Sources */,
     1568                                51494CD70C7EBDE0004178C5 /* WebIconDatabaseClient.mm in Sources */,
     1569                                51494D250C7EC1B7004178C5 /* WebNSNotificationCenterExtras.m in Sources */,
    15541570                        );
    15551571                        runOnlyForDeploymentPostprocessing = 0;

  • trunk/WebKit/WebView/WebView.mm

    r25431 r25439  
    5959#import "WebHistoryItemInternal.h"
    6060#import "WebIconDatabase.h"
     61#import "WebIconDatabaseInternal.h"
    6162#import "WebInspectorClient.h"
    6263#import "WebKitErrors.h"
    … …  
    40374038}
    40384039
     4040- (void)_receivedIconChangedNotification:(NSNotification *)notification
     4041{
     4042    // Get the URL for this notification
     4043    NSDictionary *userInfo = [notification userInfo];
     4044    ASSERT([userInfo isKindOfClass:[NSDictionary class]]);
     4045    NSString *urlString = [userInfo objectForKey:WebIconNotificationUserInfoURLKey];
     4046    ASSERT([urlString isKindOfClass:[NSString class]]);
     4047   
     4048    // If that URL matches the current main frame, dispatch the delegate call, which will also unregister
     4049    // us for this notification
     4050    if ([[self mainFrameURL] isEqualTo:urlString])
     4051        [self _dispatchDidReceiveIconFromWebFrame:[self mainFrame]];
     4052}
     4053
     4054- (void)_registerForIconNotification:(BOOL)listen
     4055{
     4056    if (listen)
     4057        [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(_receivedIconChangedNotification:) name:WebIconDatabaseDidAddIconNotification object:nil];       
     4058    else
     4059        [[NSNotificationCenter defaultCenter] removeObserver:self name:WebIconDatabaseDidAddIconNotification object:nil];
     4060}
     4061
     4062- (void)_dispatchDidReceiveIconFromWebFrame:(WebFrame *)webFrame
     4063{
     4064    // FIXME: This willChangeValueForKey call is too late, because the icon has already changed by now.
     4065    [self _willChangeValueForKey:_WebMainFrameIconKey];
     4066   
     4067    // Since we definitely have an icon and are about to send out the delegate call for that, this WebView doesn't need to listen for the general
     4068    // notification any longer
     4069    [self _registerForIconNotification:NO];
     4070
     4071    WebFrameLoadDelegateImplementationCache implementations = WebViewGetFrameLoadDelegateImplementations(self);
     4072    if (implementations.didReceiveIconForFrameFunc) {
     4073        Image* image = iconDatabase()->iconForPageURL(core(webFrame)->loader()->url().url(), IntSize(16, 16));
     4074        if (NSImage *icon = webGetNSImage(image, NSMakeSize(16, 16)))
     4075            CallFrameLoadDelegate(implementations.didReceiveIconForFrameFunc, self, @selector(webView:didReceiveIcon:forFrame:), icon, webFrame);
     4076    }
     4077
     4078    [self _didChangeValueForKey:_WebMainFrameIconKey];
     4079}
     4080
    40394081- (NSString *)_userVisibleBundleVersionFromFullVersion:(NSString *)fullVersion
    40404082{

  • trunk/WebKit/WebView/WebViewInternal.h

    r25430 r25439  
    112112- (BOOL)_becomingFirstResponderFromOutside;
    113113
     114- (void)_registerForIconNotification:(BOOL)listen;
     115- (void)_dispatchDidReceiveIconFromWebFrame:(WebFrame *)webFrame;
     116
    114117@end
    115118

  • trunk/WebKit/win/ChangeLog

    r25430 r25439  
     12007-09-08  Brady Eidson  <beidson@apple.com>
     2
     3        <rdar://problem/5434431> - Asynchronous Icon Database
     4
     5        * WebFrame.cpp:
     6        (WebFrame::didPerformFirstNavigation): Empty impl for now
     7        (WebFrame::registerForIconNotification): Ditto
     8        * WebFrame.h:
     9
    1102007-09-05  Geoffrey Garen  <ggaren@apple.com>
    211

  • trunk/WebKit/win/WebFrame.cpp

    r25430 r25439  
    23422342}
    23432343
    2344 void FrameLoaderClient::didPerformFirstNavigation() const
     2344void WebFrame::didPerformFirstNavigation() const
     2345{
     2346}
     2347
     2348void WebFrame::registerForIconNotification(bool /*listen*/)
    23452349{
    23462350}

  • trunk/WebKit/win/WebFrame.h

    r25430 r25439  
    315315    virtual void windowObjectCleared() const;
    316316    virtual void didPerformFirstNavigation() const;
     317   
     318    virtual void registerForIconNotification(bool listen);
    317319
    318320    // WebFrame
Note: See TracChangeset for help on using the changeset viewer.