⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 287013 in webkit


Ignore:
Timestamp:
Dec 13, 2021, 11:56:44 PM (5 years ago)
Author:
wilander@apple.com
Message:

PCM: Remove old DB update and migration code, and add a unit test for destination token DB columns
​https://bugs.webkit.org/show_bug.cgi?id=234281
<rdar://86347439>

Reviewed by Alex Christensen and Kate Cheney.

Source/WebKit:

WebKit::Database::needsUpdatedSchema() in PrivateClickMeasurement.cpp has a comment saying it
should be removed late 2021. WebKit::DatabaseUtilities::migrateDataToNewTablesIfNecessary()
returns directly if WebKit::Database::needsUpdatedSchema() returns false so always returning
false for WebKit::Database::needsUpdatedSchema() in PrivateClickMeasurement.cpp effectively
disables WebKit::DatabaseUtilities::migrateDataToNewTablesIfNecessary() for PCM.

New unit/API test PrivateClickMeasurement.MigrateWithDestinationToken added.

  • NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp:

(WebKit::PCM::Database::Database):

No longer calls the deleted DatabaseUtilities::migrateDataToNewTablesIfNecessary().

(WebKit::PCM::Database::attributionToStringForTesting const):

Now outputs destination token info.

(WebKit::PCM::Database::needsUpdatedSchema): Deleted.

  • NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h:

needsUpdatedSchema() now always returns false.

Tools:

  • TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm:

(addAttributedPCMv5):
(pollUntilPCMIsMigrated):

Now takes a parameter UsingDestinationToken and handles its expectation
accordingly.

(TEST):

New test PrivateClickMeasurement.MigrateWithDestinationToken.

LayoutTests:

These expect files are just updated with destination token dump output.

  • http/tests/privateClickMeasurement/attribution-conversion-through-fetch-keepalive-expected.txt:
  • http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-ephemeral-expected.txt:
  • http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-in-new-window-expected.txt:
  • http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-with-priority-expected.txt:
  • http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-without-priority-expected.txt:
  • http/tests/privateClickMeasurement/second-attribution-converted-with-higher-priority-expected.txt:
  • http/tests/privateClickMeasurement/second-attribution-converted-with-lower-priority-expected.txt:
  • http/tests/privateClickMeasurement/second-conversion-with-higher-priority-expected.txt:
  • http/tests/privateClickMeasurement/second-conversion-with-lower-priority-expected.txt:
  • http/tests/privateClickMeasurement/triggering-event-with-attribution-source-through-fetch-keepalive-expected.txt:
Location:
trunk
Files:
16 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r287010 r287013  
     12021-12-13  John Wilander  <wilander@apple.com>
     2
     3        PCM: Remove old DB update and migration code, and add a unit test for destination token DB columns
     4        https://bugs.webkit.org/show_bug.cgi?id=234281
     5        <rdar://86347439>
     6
     7        Reviewed by Alex Christensen and Kate Cheney.
     8
     9        These expect files are just updated with destination token dump output.
     10
     11        * http/tests/privateClickMeasurement/attribution-conversion-through-fetch-keepalive-expected.txt:
     12        * http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-ephemeral-expected.txt:
     13        * http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-in-new-window-expected.txt:
     14        * http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-with-priority-expected.txt:
     15        * http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-without-priority-expected.txt:
     16        * http/tests/privateClickMeasurement/second-attribution-converted-with-higher-priority-expected.txt:
     17        * http/tests/privateClickMeasurement/second-attribution-converted-with-lower-priority-expected.txt:
     18        * http/tests/privateClickMeasurement/second-conversion-with-higher-priority-expected.txt:
     19        * http/tests/privateClickMeasurement/second-conversion-with-lower-priority-expected.txt:
     20        * http/tests/privateClickMeasurement/triggering-event-with-attribution-source-through-fetch-keepalive-expected.txt:
     21
    1222021-12-13  Alex Christensen  <achristensen@webkit.org>
    223

  • trunk/LayoutTests/http/tests/privateClickMeasurement/attribution-conversion-through-fetch-keepalive-expected.txt

    r286519 r287013  
    1313Attribution priority: 0
    1414Attribution earliest time to send: Within 24-48 hours
     15Destination token: Not set
    1516Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-ephemeral-expected.txt

    r282884 r287013  
    1010Attribution priority: 0
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-in-new-window-expected.txt

    r282884 r287013  
    1010Attribution priority: 3
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-with-priority-expected.txt

    r282884 r287013  
    1010Attribution priority: 3
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/attribution-conversion-through-image-redirect-without-priority-expected.txt

    r282884 r287013  
    1010Attribution priority: 0
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/second-attribution-converted-with-higher-priority-expected.txt

    r282884 r287013  
    1010Attribution priority: 4
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/second-attribution-converted-with-lower-priority-expected.txt

    r282884 r287013  
    1010Attribution priority: 4
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/second-conversion-with-higher-priority-expected.txt

    r282884 r287013  
    1010Attribution priority: 4
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/second-conversion-with-lower-priority-expected.txt

    r282884 r287013  
    1010Attribution priority: 4
    1111Attribution earliest time to send: Within 24-48 hours
     12Destination token: Not set
    1213Application bundle identifier: com.apple.WebKit.WebKitTestRunner

  • trunk/LayoutTests/http/tests/privateClickMeasurement/triggering-event-with-attribution-source-through-fetch-keepalive-expected.txt

    r286519 r287013  
    1313Attribution priority: 0
    1414Attribution earliest time to send: Within 24-48 hours
     15Destination token: Not set
    1516Application bundle identifier: testBundleID

  • trunk/Source/WebKit/ChangeLog

    r287010 r287013  
     12021-12-13  John Wilander  <wilander@apple.com>
     2
     3        PCM: Remove old DB update and migration code, and add a unit test for destination token DB columns
     4        https://bugs.webkit.org/show_bug.cgi?id=234281
     5        <rdar://86347439>
     6
     7        Reviewed by Alex Christensen and Kate Cheney.
     8
     9        WebKit::Database::needsUpdatedSchema() in PrivateClickMeasurement.cpp has a comment saying it
     10        should be removed late 2021. WebKit::DatabaseUtilities::migrateDataToNewTablesIfNecessary()
     11        returns directly if WebKit::Database::needsUpdatedSchema() returns false so always returning
     12        false for WebKit::Database::needsUpdatedSchema() in PrivateClickMeasurement.cpp effectively
     13        disables WebKit::DatabaseUtilities::migrateDataToNewTablesIfNecessary() for PCM.
     14
     15        New unit/API test PrivateClickMeasurement.MigrateWithDestinationToken added.
     16
     17        * NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp:
     18        (WebKit::PCM::Database::Database):
     19            No longer calls the deleted DatabaseUtilities::migrateDataToNewTablesIfNecessary().
     20        (WebKit::PCM::Database::attributionToStringForTesting const):
     21            Now outputs destination token info.
     22        (WebKit::PCM::Database::needsUpdatedSchema): Deleted.
     23        * NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h:
     24            needsUpdatedSchema() now always returns false.
     25
    1262021-12-13  Alex Christensen  <achristensen@webkit.org>
    227

  • trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.cpp

    r286519 r287013  
    8989    enableForeignKeys();
    9090    addDestinationTokenColumnsIfNecessary();
    91     migrateDataToNewTablesIfNecessary();
    9291    allDatabases().add(this);
    9392}
    … …  
    450449            builder.append((secondsUntilSend >= 24_h && secondsUntilSend <= 48_h) ? "Within 24-48 hours" : "Outside 24-48 hours");
    451450        }
     451
     452        builder.append("\nDestination token: ");
     453        if (!triggerData->destinationSecretToken)
     454            builder.append("Not set");
     455        else
     456            builder.append("\ntoken: ", triggerData->destinationSecretToken->tokenBase64URL, "\nsignature: ", triggerData->destinationSecretToken->signatureBase64URL, "\nkey: ", triggerData->destinationSecretToken->keyIDBase64URL);
    452457    } else
    453458        builder.append("\nNo attribution trigger data.");
    … …  
    717722}
    718723
    719 bool Database::needsUpdatedSchema()
    720 {
    721     // FIXME: Remove this at the end of 2021. No public release was made with the schema missing sourceApplicationBundleID, so this is only needed to migrate internal users who updated in September 2021.
    722     for (auto& table : expectedTableAndIndexQueries().keys()) {
    723         if (currentTableAndIndexQueries(table) != expectedTableAndIndexQueries().get(table))
    724             return true;
    725     }
    726 
    727     return false;
    728 }
    729 
    730724Vector<String> Database::columnsForTable(const String& tableName)
    731725{

  • trunk/Source/WebKit/NetworkProcess/PrivateClickMeasurement/PrivateClickMeasurementDatabase.h

    r286519 r287013  
    7878
    7979    void addDestinationTokenColumnsIfNecessary();
    80     bool needsUpdatedSchema() final;
     80    bool needsUpdatedSchema() final { return false; };
    8181    bool createUniqueIndices() final;
    8282    const MemoryCompactLookupOnlyRobinHoodHashMap<String, TableAndIndexPair>& expectedTableAndIndexQueries() final;

  • trunk/Tools/ChangeLog

    r287010 r287013  
     12021-12-13  John Wilander  <wilander@apple.com>
     2
     3        PCM: Remove old DB update and migration code, and add a unit test for destination token DB columns
     4        https://bugs.webkit.org/show_bug.cgi?id=234281
     5        <rdar://86347439>
     6
     7        Reviewed by Alex Christensen and Kate Cheney.
     8
     9        * TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm:
     10        (addAttributedPCMv5):
     11        (pollUntilPCMIsMigrated):
     12            Now takes a parameter UsingDestinationToken and handles its expectation
     13            accordingly.
     14        (TEST):
     15            New test PrivateClickMeasurement.MigrateWithDestinationToken.
     16
    1172021-12-13  Alex Christensen  <achristensen@webkit.org>
    218

  • trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/PrivateClickMeasurement.mm

    r284474 r287013  
    210210}
    211211
     212static void addAttributedPCMv5(WebCore::SQLiteDatabase& database)
     213{
     214    constexpr auto createAttributedPrivateClickMeasurementV5 = "CREATE TABLE AttributedPrivateClickMeasurement ("
     215        "sourceSiteDomainID INTEGER NOT NULL, destinationSiteDomainID INTEGER NOT NULL, sourceID INTEGER NOT NULL, "
     216        "attributionTriggerData INTEGER NOT NULL, priority INTEGER NOT NULL, timeOfAdClick REAL NOT NULL, "
     217        "earliestTimeToSendToSource REAL, token TEXT, signature TEXT, keyID TEXT, earliestTimeToSendToDestination REAL, sourceApplicationBundleID TEXT, destinationToken, destinationSignature, destinationKeyID,"
     218        "FOREIGN KEY(sourceSiteDomainID) REFERENCES PCMObservedDomains(domainID) ON DELETE CASCADE, FOREIGN KEY(destinationSiteDomainID) REFERENCES "
     219        "PCMObservedDomains(domainID) ON DELETE CASCADE)"_s;
     220
     221    EXPECT_TRUE(database.executeCommand(createAttributedPrivateClickMeasurementV5));
     222    constexpr auto insertAttributedPrivateClickMeasurementQueryV5 = "INSERT OR REPLACE INTO AttributedPrivateClickMeasurement (sourceSiteDomainID, destinationSiteDomainID, "
     223        "sourceID, attributionTriggerData, priority, timeOfAdClick, earliestTimeToSendToSource, token, signature, keyID, earliestTimeToSendToDestination, sourceApplicationBundleID, destinationToken, destinationSignature, destinationKeyID) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"_s;
     224
     225#if PLATFORM(MAC)
     226    auto bundleID = "com.apple.Safari";
     227#else
     228    auto bundleID = "com.apple.mobilesafari";
     229#endif
     230    addValuesToTable<15>(database, insertAttributedPrivateClickMeasurementQueryV5, { 1, 2, 42, 14, 7, 1.0, earliestTimeToSend(), "test token", "test signature", "test key id", earliestTimeToSend(), bundleID, "test destination token", "test destination signature", "test destination key id" });
     231}
     232
    212233static RetainPtr<NSString> dumpedPCM(WKWebView *webView)
    213234{
    … …  
    223244
    224245enum class MigratingFromResourceLoadStatistics : bool { No, Yes };
    225 static void pollUntilPCMIsMigrated(WKWebView *webView, MigratingFromResourceLoadStatistics migratingFromResourceLoadStatistics)
     246enum class UsingDestinationToken : bool { No, Yes };
     247static void pollUntilPCMIsMigrated(WKWebView *webView, MigratingFromResourceLoadStatistics migratingFromResourceLoadStatistics, UsingDestinationToken usingDestinationToken)
    226248{
    227249    if (migratingFromResourceLoadStatistics == MigratingFromResourceLoadStatistics::Yes) {
    … …  
    252274        "Attribution priority: 7\n"
    253275        "Attribution earliest time to send: Outside 24-48 hours\n"
     276        "";
     277
     278    NSString *suffix = @"Destination token: ";
     279    suffix = [suffix stringByAppendingString:(usingDestinationToken == UsingDestinationToken::No ? @"Not set\n" : @"\ntoken: test destination token\nsignature: test destination signature\nkey: test destination key id\n")];
    254280#if PLATFORM(MAC)
    255         "Application bundle identifier: com.apple.Safari\n"
     281    suffix = [suffix stringByAppendingString:@"Application bundle identifier: com.apple.Safari\n"];
    256282#else
    257         "Application bundle identifier: com.apple.mobilesafari\n"
     283    suffix = [suffix stringByAppendingString:@"Application bundle identifier: com.apple.mobilesafari\n"];
    258284#endif
    259         "";
     285    expectedMigratedPCMDatabase = [expectedMigratedPCMDatabase stringByAppendingString:suffix];
    260286
    261287    while (![dumpedPCM(webView) isEqualToString:expectedMigratedPCMDatabase])
    262288        usleep(10000);
     289    EXPECT_WK_STREQ(dumpedPCM(webView).get(), expectedMigratedPCMDatabase);
    263290}
    264291
    … …  
    355382    setUpFromResourceLoadStatisticsDatabase(addUnattributedPCMv1, addAttributedPCMv1);
    356383    auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
    357     pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes);
     384    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes, UsingDestinationToken::No);
    358385    cleanUp();
    359386}
    … …  
    363390    setUpFromResourceLoadStatisticsDatabase(addUnattributedPCMv2, addAttributedPCMv2);
    364391    auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
    365     pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes);
     392    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes, UsingDestinationToken::No);
    366393    cleanUp();
    367394}
    … …  
    371398    setUpFromResourceLoadStatisticsDatabase(addUnattributedPCMv3, addAttributedPCMv3);
    372399    auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
    373     pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes);
     400    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::Yes, UsingDestinationToken::No);
    374401    cleanUp();
    375402}
    … …  
    379406    setUpFromPCMDatabase(addUnattributedPCMv4, addAttributedPCMv4);
    380407    auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
    381     pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::No);
     408    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::No, UsingDestinationToken::No);
    382409    cleanUp();
    383410}
     411
     412TEST(PrivateClickMeasurement, MigrateWithDestinationToken)
     413{
     414    setUpFromPCMDatabase(addUnattributedPCMv4, addAttributedPCMv5);
     415    auto webView = webViewWithResourceLoadStatisticsEnabledInNetworkProcess();
     416    pollUntilPCMIsMigrated(webView.get(), MigratingFromResourceLoadStatistics::No, UsingDestinationToken::Yes);
     417    cleanUp();
     418}
Note: See TracChangeset for help on using the changeset viewer.