Changeset 213669 in webkit


Ignore:
Timestamp:
Mar 9, 2017 2:17:13 PM (7 years ago)
Author:
achristensen@apple.com
Message:

[Content Extensions] Introduce if-top-url and unless-top-url
https://bugs.webkit.org/show_bug.cgi?id=169433

Reviewed by Brady Eidson.

Source/WebCore:

In r184116 I added if-domain and unless-domain to control whether a rule applies
based on the domain of the main document URL. I'm expanding this by adding if-top-url
and unless-top-url that run regular expressions on the entire main document URL so that
example.com/user1content can be distinguished from example.com/user2content.
To not add to the number of passes we make on the URLs for each load and to maintain JSON
backwards compatibility, I've made it so that if-top-url and unless-top-url can be used
instead of if-domain and unless-domain (which continue to work) but the two condition types
cannot be used together since running regular expressions on the entire main document URL
is strictly more powerful than checking the domain and subdomains.
As a minor detail, content extension regexes are by default ASCII-case-insensitive, so I've
done the same with top URL regexes, adding top-url-filter-is-case-sensitive to mirror the existing
url-filter-is-case-sensitive if the JSON author decides to make regexes case sensitive.

Covered by new API tests.

  • contentextensions/CompiledContentExtension.h:
  • contentextensions/ContentExtension.cpp:

(WebCore::ContentExtensions::ContentExtension::populateConditionCacheIfNeeded):
(WebCore::ContentExtensions::ContentExtension::topURLActions):
(WebCore::ContentExtensions::ContentExtension::cachedConditionedActions): Deleted.

  • contentextensions/ContentExtension.h:
  • contentextensions/ContentExtensionCompiler.cpp:

(WebCore::ContentExtensions::addUniversalActionsToDFA):
(WebCore::ContentExtensions::compileToBytecode):
We had three copies of compiling to bytecode that were almost the same and would've been made into three copies of the same code.
I moved them to one helper function that is called three times.
(WebCore::ContentExtensions::compileRuleList):

  • contentextensions/ContentExtensionCompiler.h:
  • contentextensions/ContentExtensionError.cpp:

(WebCore::ContentExtensions::contentExtensionErrorCategory):
Add the new error type for JSON that tries to use if-top-url and unless-top-url with if-domain and unless-domain.

  • contentextensions/ContentExtensionError.h:
  • contentextensions/ContentExtensionParser.cpp:

(WebCore::ContentExtensions::loadTrigger):
Parse the new values if-top-url, unless-top-url, and top-url-filter-is-case-sensitive.

  • contentextensions/ContentExtensionRule.h:

(WebCore::ContentExtensions::Trigger::~Trigger):

  • contentextensions/ContentExtensionsBackend.cpp:

(WebCore::ContentExtensions::ContentExtensionsBackend::actionsForResourceLoad):

  • contentextensions/DFABytecodeInterpreter.cpp:

(WebCore::ContentExtensions::DFABytecodeInterpreter::interpretAppendAction):
(WebCore::ContentExtensions::DFABytecodeInterpreter::interpretTestFlagsAndAppendAction):
(WebCore::ContentExtensions::DFABytecodeInterpreter::interpretWithConditions):

  • contentextensions/DFABytecodeInterpreter.h:

Source/WebKit2:

Rename conditionedFilters to topURLFilters to reflect the fact that they are the filters
that are run on the top URL, and possibly just the domain of the top url.
I was a bit too aggressive when renaming domain* to condition* in r213533.

  • Shared/WebCompiledContentExtension.cpp:

(WebKit::WebCompiledContentExtension::conditionsApplyOnlyToDomain):
(WebKit::WebCompiledContentExtension::topURLFiltersBytecode):
(WebKit::WebCompiledContentExtension::topURLFiltersBytecodeLength):
(WebKit::WebCompiledContentExtension::conditionedFiltersBytecode): Deleted.
(WebKit::WebCompiledContentExtension::conditionedFiltersBytecodeLength): Deleted.

  • Shared/WebCompiledContentExtension.h:
  • Shared/WebCompiledContentExtensionData.cpp:

(WebKit::WebCompiledContentExtensionData::encode):
(WebKit::WebCompiledContentExtensionData::decode):

  • Shared/WebCompiledContentExtensionData.h:

(WebKit::WebCompiledContentExtensionData::WebCompiledContentExtensionData):

  • UIProcess/API/APIUserContentExtensionStore.cpp:

(API::encodeContentExtensionMetaData):
(API::decodeContentExtensionMetaData):
(API::compiledToFile):
(API::createExtension):
(API::UserContentExtensionStore::invalidateContentExtensionVersion):
(API::userContentExtensionStoreErrorCategory):

  • UIProcess/API/APIUserContentExtensionStore.h:

Increment CurrentContentExtensionFileVersion because we have changed the format of the binary on disk.
We only added 4 bytes, but that's binary incompatible and requires re-compiling any existing content extensions.

Tools:

  • TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp:

(TestWebKitAPI::TEST_F):
Add tests for new functionality and new failure types.

Location:
trunk
Files:
22 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebCore/ChangeLog

    r213666 r213669  
     12017-03-09  Alex Christensen  <achristensen@webkit.org>
     2
     3        [Content Extensions] Introduce if-top-url and unless-top-url
     4        https://bugs.webkit.org/show_bug.cgi?id=169433
     5
     6        Reviewed by Brady Eidson.
     7
     8        In r184116 I added if-domain and unless-domain to control whether a rule applies
     9        based on the domain of the main document URL.  I'm expanding this by adding if-top-url
     10        and unless-top-url that run regular expressions on the entire main document URL so that
     11        example.com/user1content can be distinguished from example.com/user2content.
     12        To not add to the number of passes we make on the URLs for each load and to maintain JSON
     13        backwards compatibility, I've made it so that if-top-url and unless-top-url can be used
     14        instead of if-domain and unless-domain (which continue to work) but the two condition types
     15        cannot be used together since running regular expressions on the entire main document URL
     16        is strictly more powerful than checking the domain and subdomains.
     17        As a minor detail, content extension regexes are by default ASCII-case-insensitive, so I've
     18        done the same with top URL regexes, adding top-url-filter-is-case-sensitive to mirror the existing
     19        url-filter-is-case-sensitive if the JSON author decides to make regexes case sensitive.
     20
     21        Covered by new API tests.
     22
     23        * contentextensions/CompiledContentExtension.h:
     24        * contentextensions/ContentExtension.cpp:
     25        (WebCore::ContentExtensions::ContentExtension::populateConditionCacheIfNeeded):
     26        (WebCore::ContentExtensions::ContentExtension::topURLActions):
     27        (WebCore::ContentExtensions::ContentExtension::cachedConditionedActions): Deleted.
     28        * contentextensions/ContentExtension.h:
     29        * contentextensions/ContentExtensionCompiler.cpp:
     30        (WebCore::ContentExtensions::addUniversalActionsToDFA):
     31        (WebCore::ContentExtensions::compileToBytecode):
     32        We had three copies of compiling to bytecode that were almost the same and would've been made into three copies of the same code.
     33        I moved them to one helper function that is called three times.
     34        (WebCore::ContentExtensions::compileRuleList):
     35        * contentextensions/ContentExtensionCompiler.h:
     36        * contentextensions/ContentExtensionError.cpp:
     37        (WebCore::ContentExtensions::contentExtensionErrorCategory):
     38        Add the new error type for JSON that tries to use if-top-url and unless-top-url with if-domain and unless-domain.
     39        * contentextensions/ContentExtensionError.h:
     40        * contentextensions/ContentExtensionParser.cpp:
     41        (WebCore::ContentExtensions::loadTrigger):
     42        Parse the new values if-top-url, unless-top-url, and top-url-filter-is-case-sensitive.
     43        * contentextensions/ContentExtensionRule.h:
     44        (WebCore::ContentExtensions::Trigger::~Trigger):
     45        * contentextensions/ContentExtensionsBackend.cpp:
     46        (WebCore::ContentExtensions::ContentExtensionsBackend::actionsForResourceLoad):
     47        * contentextensions/DFABytecodeInterpreter.cpp:
     48        (WebCore::ContentExtensions::DFABytecodeInterpreter::interpretAppendAction):
     49        (WebCore::ContentExtensions::DFABytecodeInterpreter::interpretTestFlagsAndAppendAction):
     50        (WebCore::ContentExtensions::DFABytecodeInterpreter::interpretWithConditions):
     51        * contentextensions/DFABytecodeInterpreter.h:
     52
    1532017-03-09  Nikita Vasilyev  <nvasilyev@apple.com>
    254
  • trunk/Source/WebCore/contentextensions/CompiledContentExtension.h

    r213533 r213669  
    4343    virtual const DFABytecode* filtersWithConditionsBytecode() const = 0;
    4444    virtual unsigned filtersWithConditionsBytecodeLength() const = 0;
    45     virtual const DFABytecode* conditionedFiltersBytecode() const = 0;
    46     virtual unsigned conditionedFiltersBytecodeLength() const = 0;
     45    virtual const DFABytecode* topURLFiltersBytecode() const = 0;
     46    virtual unsigned topURLFiltersBytecodeLength() const = 0;
    4747    virtual const SerializedActionByte* actions() const = 0;
    4848    virtual unsigned actionsLength() const = 0;
     49    virtual bool conditionsApplyOnlyToDomain() const = 0;
    4950};
    5051
  • trunk/Source/WebCore/contentextensions/ContentExtension.cpp

    r213533 r213669  
    119119void ContentExtension::populateConditionCacheIfNeeded(const URL& topURL)
    120120{
    121     String domain = topURL.host();
    122     if (m_cachedDomain != domain) {
    123         DFABytecodeInterpreter interpreter(m_compiledExtension->conditionedFiltersBytecode(), m_compiledExtension->conditionedFiltersBytecodeLength());
     121    if (m_cachedTopURL != topURL) {
     122        DFABytecodeInterpreter interpreter(m_compiledExtension->topURLFiltersBytecode(), m_compiledExtension->topURLFiltersBytecodeLength());
    124123        const uint16_t allLoadTypesAndResourceTypes = LoadTypeMask | ResourceTypeMask;
    125         auto domainActions = interpreter.interpret(domain.utf8(), allLoadTypesAndResourceTypes);
     124        String string = m_compiledExtension->conditionsApplyOnlyToDomain() ? topURL.host() : topURL.string();
     125        auto topURLActions = interpreter.interpret(string.utf8(), allLoadTypesAndResourceTypes);
    126126       
    127         m_cachedConditionedActions.clear();
    128         for (uint64_t action : domainActions)
    129             m_cachedConditionedActions.add(action);
     127        m_cachedTopURLActions.clear();
     128        for (uint64_t action : topURLActions)
     129            m_cachedTopURLActions.add(action);
     130        for (uint64_t action : interpreter.actionsMatchingEverything())
     131            m_cachedTopURLActions.add(action);
    130132       
    131133        m_cachedUniversalConditionedActions.clear();
    132134        for (uint64_t action : m_universalActionsWithConditions) {
    133         ASSERT_WITH_MESSAGE((action & ~IfConditionFlag) == static_cast<uint32_t>(action), "Universal actions with domains should not have flags.");
    134             if (!!(action & IfConditionFlag) == m_cachedConditionedActions.contains(action))
     135        ASSERT_WITH_MESSAGE((action & ~IfConditionFlag) == static_cast<uint32_t>(action), "Universal actions with conditions should not have flags.");
     136            if (!!(action & IfConditionFlag) == m_cachedTopURLActions.contains(action))
    135137                m_cachedUniversalConditionedActions.append(static_cast<uint32_t>(action));
    136138        }
    137139        m_cachedUniversalConditionedActions.shrinkToFit();
    138         m_cachedDomain = domain;
     140        m_cachedTopURL = topURL;
    139141    }
    140142}
    141143
    142 const DFABytecodeInterpreter::Actions& ContentExtension::cachedConditionedActions(const URL& topURL)
     144const DFABytecodeInterpreter::Actions& ContentExtension::topURLActions(const URL& topURL)
    143145{
    144146    populateConditionCacheIfNeeded(topURL);
    145     return m_cachedConditionedActions;
     147    return m_cachedTopURLActions;
    146148}
    147149
  • trunk/Source/WebCore/contentextensions/ContentExtension.h

    r213533 r213669  
    4747    const CompiledContentExtension& compiledExtension() const { return m_compiledExtension.get(); }
    4848    StyleSheetContents* globalDisplayNoneStyleSheet();
    49     const DFABytecodeInterpreter::Actions& cachedConditionedActions(const URL& topURL);
     49    const DFABytecodeInterpreter::Actions& topURLActions(const URL& topURL);
    5050    const Vector<uint32_t>& universalActionsWithoutConditions() { return m_universalActionsWithoutConditions; }
    5151    const Vector<uint32_t>& universalActionsWithConditions(const URL& topURL);
     
    6161    void compileGlobalDisplayNoneStyleSheet();
    6262
    63     String m_cachedDomain;
     63    URL m_cachedTopURL;
    6464    void populateConditionCacheIfNeeded(const URL& topURL);
    65     DFABytecodeInterpreter::Actions m_cachedConditionedActions;
     65    DFABytecodeInterpreter::Actions m_cachedTopURLActions;
    6666    Vector<uint32_t> m_cachedUniversalConditionedActions;
    6767
  • trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp

    r213533 r213669  
    188188typedef HashSet<uint64_t, DefaultHash<uint64_t>::Hash, WTF::UnsignedWithZeroKeyHashTraits<uint64_t>> UniversalActionSet;
    189189
    190 static void addUniversalActionsToDFA(DFA& dfa, const UniversalActionSet& universalActions)
     190static void addUniversalActionsToDFA(DFA& dfa, UniversalActionSet&& universalActions)
    191191{
    192192    if (universalActions.isEmpty())
     
    206206}
    207207
     208template<typename Functor>
     209static void compileToBytecode(CombinedURLFilters&& filters, UniversalActionSet&& universalActions, Functor writeBytecodeToClient)
     210{
     211    // Smaller maxNFASizes risk high compiling and interpreting times from having too many DFAs,
     212    // larger maxNFASizes use too much memory when compiling.
     213    const unsigned maxNFASize = 75000;
     214
     215    bool firstNFASeen = false;
     216
     217    auto lowerDFAToBytecode = [&](DFA&& dfa)
     218    {
     219#if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
     220        dataLogF("DFA\n");
     221        dfa.debugPrintDot();
     222#endif
     223        ASSERT_WITH_MESSAGE(!dfa.nodes[dfa.root].hasActions(), "All actions on the DFA root should come from regular expressions that match everything.");
     224
     225        if (!firstNFASeen) {
     226            // Put all the universal actions on the first DFA.
     227            addUniversalActionsToDFA(dfa, WTFMove(universalActions));
     228        }
     229
     230        Vector<DFABytecode> bytecode;
     231        DFABytecodeCompiler compiler(dfa, bytecode);
     232        compiler.compile();
     233        LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
     234        writeBytecodeToClient(WTFMove(bytecode));
     235        firstNFASeen = true;
     236    };
     237
     238    const unsigned smallDFASize = 100;
     239    DFACombiner smallDFACombiner;
     240    filters.processNFAs(maxNFASize, [&](NFA&& nfa) {
     241#if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
     242        dataLogF("NFA\n");
     243        nfa.debugPrintDot();
     244#endif
     245        LOG_LARGE_STRUCTURES(nfa, nfa.memoryUsed());
     246        DFA dfa = NFAToDFA::convert(nfa);
     247        LOG_LARGE_STRUCTURES(dfa, dfa.memoryUsed());
     248
     249        if (dfa.graphSize() < smallDFASize)
     250            smallDFACombiner.addDFA(WTFMove(dfa));
     251        else {
     252            dfa.minimize();
     253            lowerDFAToBytecode(WTFMove(dfa));
     254        }
     255    });
     256
     257    smallDFACombiner.combineDFAs(smallDFASize, [&](DFA&& dfa) {
     258        LOG_LARGE_STRUCTURES(dfa, dfa.memoryUsed());
     259        lowerDFAToBytecode(WTFMove(dfa));
     260    });
     261
     262    ASSERT(filters.isEmpty());
     263
     264    if (!firstNFASeen) {
     265        // Our bytecode interpreter expects to have at least one DFA, so if we haven't seen any
     266        // create a dummy one and add any universal actions.
     267
     268        DFA dummyDFA = DFA::empty();
     269        addUniversalActionsToDFA(dummyDFA, WTFMove(universalActions));
     270
     271        Vector<DFABytecode> bytecode;
     272        DFABytecodeCompiler compiler(dummyDFA, bytecode);
     273        compiler.compile();
     274        LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
     275        writeBytecodeToClient(WTFMove(bytecode));
     276    }
     277    LOG_LARGE_STRUCTURES(universalActions, universalActions.capacity() * sizeof(unsigned));
     278}
     279
    208280std::error_code compileRuleList(ContentExtensionCompilationClient& client, String&& ruleJSON)
    209281{
     
    213285    Vector<ContentExtensionRule> parsedRuleList = WTFMove(ruleList.value());
    214286
     287    bool domainConditionSeen = false;
     288    bool topURLConditionSeen = false;
     289    for (const auto& rule : parsedRuleList) {
     290        switch (rule.trigger().conditionType) {
     291        case Trigger::ConditionType::None:
     292            break;
     293        case Trigger::ConditionType::IfDomain:
     294        case Trigger::ConditionType::UnlessDomain:
     295            domainConditionSeen = true;
     296            break;
     297        case Trigger::ConditionType::IfTopURL:
     298        case Trigger::ConditionType::UnlessTopURL:
     299            topURLConditionSeen = true;
     300            break;
     301        }
     302    }
     303    if (topURLConditionSeen && domainConditionSeen)
     304        return ContentExtensionError::JSONTopURLAndDomainConditions;
     305
    215306#if CONTENT_EXTENSIONS_PERFORMANCE_REPORTING
    216307    double patternPartitioningStart = monotonicallyIncreasingTime();
     
    219310    Vector<SerializedActionByte> actions;
    220311    Vector<unsigned> actionLocations = serializeActions(parsedRuleList, actions);
    221     client.writeActions(WTFMove(actions));
    222312    LOG_LARGE_STRUCTURES(actions, actions.capacity() * sizeof(SerializedActionByte));
    223     actions.clear();
     313    client.writeActions(WTFMove(actions), domainConditionSeen);
    224314
    225315    UniversalActionSet universalActionsWithoutConditions;
    226316    UniversalActionSet universalActionsWithConditions;
     317    UniversalActionSet universalTopURLActions;
    227318
    228319    // FIXME: These don't all need to be in memory at the same time.
    229320    CombinedURLFilters filtersWithoutConditions;
    230321    CombinedURLFilters filtersWithConditions;
    231     CombinedURLFilters conditionFilters;
     322    CombinedURLFilters topURLFilters;
    232323    URLFilterParser filtersWithoutConditionParser(filtersWithoutConditions);
    233324    URLFilterParser filtersWithConditionParser(filtersWithConditions);
     325    URLFilterParser topURLFilterParser(topURLFilters);
    234326   
    235327    for (unsigned ruleIndex = 0; ruleIndex < parsedRuleList.size(); ++ruleIndex) {
     
    257349            switch (trigger.conditionType) {
    258350            case Trigger::ConditionType::IfDomain:
     351            case Trigger::ConditionType::IfTopURL:
    259352                actionLocationAndFlags |= IfConditionFlag;
    260353                break;
    261354            case Trigger::ConditionType::None:
    262355            case Trigger::ConditionType::UnlessDomain:
     356            case Trigger::ConditionType::UnlessTopURL:
    263357                ASSERT(!(actionLocationAndFlags & IfConditionFlag));
    264358                break;
     
    274368                return ContentExtensionError::JSONInvalidRegex;
    275369            }
    276             for (const String& condition : trigger.conditions)
    277                 conditionFilters.addDomain(actionLocationAndFlags, condition);
     370            for (const String& condition : trigger.conditions) {
     371                if (domainConditionSeen) {
     372                    ASSERT(!topURLConditionSeen);
     373                    topURLFilters.addDomain(actionLocationAndFlags, condition);
     374                } else {
     375                    ASSERT(topURLConditionSeen);
     376                    status = topURLFilterParser.addPattern(condition, trigger.topURLConditionIsCaseSensitive, actionLocationAndFlags);
     377                    if (status == URLFilterParser::MatchesEverything) {
     378                        universalTopURLActions.add(actionLocationAndFlags);
     379                        status = URLFilterParser::Ok;
     380                    }
     381                    if (status != URLFilterParser::Ok) {
     382                        dataLogF("Error while parsing %s: %s\n", condition.utf8().data(), URLFilterParser::statusString(status).utf8().data());
     383                        return ContentExtensionError::JSONInvalidRegex;
     384                    }
     385                }
     386            }
    278387        }
    279388        ASSERT(status == URLFilterParser::Ok);
     
    291400    LOG_LARGE_STRUCTURES(filtersWithoutConditions, filtersWithoutConditions.memoryUsed());
    292401    LOG_LARGE_STRUCTURES(filtersWithConditions, filtersWithConditions.memoryUsed());
    293     LOG_LARGE_STRUCTURES(conditionFilters, conditionFilters.memoryUsed());
     402    LOG_LARGE_STRUCTURES(topURLFilters, topURLFilters.memoryUsed());
    294403
    295404#if CONTENT_EXTENSIONS_PERFORMANCE_REPORTING
    296     unsigned machinesWithoutConditionsCount = 0;
    297     unsigned totalBytecodeSizeForMachinesWithoutConditions = 0;
    298     unsigned machinesWithConditionsCount = 0;
    299     unsigned totalBytecodeSizeForMachinesWithConditions = 0;
    300405    double totalNFAToByteCodeBuildTimeStart = monotonicallyIncreasingTime();
    301406#endif
    302407
    303     // Smaller maxNFASizes risk high compiling and interpreting times from having too many DFAs,
    304     // larger maxNFASizes use too much memory when compiling.
    305     const unsigned maxNFASize = 75000;
    306    
    307     bool firstNFAWithoutConditionsSeen = false;
    308 
    309     auto lowerFiltersWithoutConditionsDFAToBytecode = [&](DFA&& dfa)
    310     {
    311 #if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
    312         dataLogF("filtersWithoutConditions DFA\n");
    313         dfa.debugPrintDot();
    314 #endif
    315         ASSERT_WITH_MESSAGE(!dfa.nodes[dfa.root].hasActions(), "All actions on the DFA root should come from regular expressions that match everything.");
    316 
    317         if (!firstNFAWithoutConditionsSeen) {
    318             // Put all the universal actions on the first DFA.
    319             addUniversalActionsToDFA(dfa, universalActionsWithoutConditions);
    320         }
    321 
    322         Vector<DFABytecode> bytecode;
    323         DFABytecodeCompiler compiler(dfa, bytecode);
    324         compiler.compile();
    325         LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
    326 #if CONTENT_EXTENSIONS_PERFORMANCE_REPORTING
    327         ++machinesWithoutConditionsCount;
    328         totalBytecodeSizeForMachinesWithoutConditions += bytecode.size();
    329 #endif
     408    compileToBytecode(WTFMove(filtersWithoutConditions), WTFMove(universalActionsWithoutConditions), [&](Vector<DFABytecode>&& bytecode) {
    330409        client.writeFiltersWithoutConditionsBytecode(WTFMove(bytecode));
    331 
    332         firstNFAWithoutConditionsSeen = true;
    333     };
    334 
    335     const unsigned smallDFASize = 100;
    336     DFACombiner smallFiltersWithoutConditionsDFACombiner;
    337     filtersWithoutConditions.processNFAs(maxNFASize, [&](NFA&& nfa) {
    338 #if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
    339         dataLogF("filtersWithoutConditions NFA\n");
    340         nfa.debugPrintDot();
    341 #endif
    342 
    343         LOG_LARGE_STRUCTURES(nfa, nfa.memoryUsed());
    344         DFA dfa = NFAToDFA::convert(nfa);
    345         LOG_LARGE_STRUCTURES(dfa, dfa.memoryUsed());
    346 
    347         if (dfa.graphSize() < smallDFASize)
    348             smallFiltersWithoutConditionsDFACombiner.addDFA(WTFMove(dfa));
    349         else {
    350             dfa.minimize();
    351             lowerFiltersWithoutConditionsDFAToBytecode(WTFMove(dfa));
    352         }
    353410    });
    354 
    355 
    356     smallFiltersWithoutConditionsDFACombiner.combineDFAs(smallDFASize, [&](DFA&& dfa) {
    357         LOG_LARGE_STRUCTURES(dfa, dfa.memoryUsed());
    358         lowerFiltersWithoutConditionsDFAToBytecode(WTFMove(dfa));
     411    compileToBytecode(WTFMove(filtersWithConditions), WTFMove(universalActionsWithConditions), [&](Vector<DFABytecode>&& bytecode) {
     412        client.writeFiltersWithConditionsBytecode(WTFMove(bytecode));
    359413    });
    360 
    361     ASSERT(filtersWithoutConditions.isEmpty());
    362 
    363     if (!firstNFAWithoutConditionsSeen) {
    364         // Our bytecode interpreter expects to have at least one DFA, so if we haven't seen any
    365         // create a dummy one and add any universal actions.
    366 
    367         DFA dummyDFA = DFA::empty();
    368         addUniversalActionsToDFA(dummyDFA, universalActionsWithoutConditions);
    369 
    370         Vector<DFABytecode> bytecode;
    371         DFABytecodeCompiler compiler(dummyDFA, bytecode);
    372         compiler.compile();
    373         LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
    374         client.writeFiltersWithoutConditionsBytecode(WTFMove(bytecode));
    375     }
    376     LOG_LARGE_STRUCTURES(universalActionsWithoutConditions, universalActionsWithoutConditions.capacity() * sizeof(unsigned));
    377     universalActionsWithoutConditions.clear();
    378    
    379     bool firstNFAWithConditionsSeen = false;
    380     auto lowerFiltersWithConditionsDFAToBytecode = [&](DFA&& dfa)
    381     {
    382         if (!firstNFAWithConditionsSeen) {
    383             // Put all the universal actions on the first DFA.
    384             addUniversalActionsToDFA(dfa, universalActionsWithConditions);
    385         }
    386 
    387         Vector<DFABytecode> bytecode;
    388         DFABytecodeCompiler compiler(dfa, bytecode);
    389         compiler.compile();
    390         LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
    391 #if CONTENT_EXTENSIONS_PERFORMANCE_REPORTING
    392         ++machinesWithConditionsCount;
    393         totalBytecodeSizeForMachinesWithConditions += bytecode.size();
    394 #endif
    395         client.writeFiltersWithConditionsBytecode(WTFMove(bytecode));
    396 
    397         firstNFAWithConditionsSeen = true;
    398     };
    399 
    400     DFACombiner smallFiltersWithConditionsDFACombiner;
    401     filtersWithConditions.processNFAs(maxNFASize, [&](NFA&& nfa) {
    402 #if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
    403         dataLogF("filtersWithConditions NFA\n");
    404         nfa.debugPrintDot();
    405 #endif
    406         LOG_LARGE_STRUCTURES(nfa, nfa.memoryUsed());
    407         DFA dfa = NFAToDFA::convert(nfa);
    408 #if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
    409         dataLogF("filtersWithConditions PRE MINIMIZING DFA\n");
    410         dfa.debugPrintDot();
    411 #endif
    412         LOG_LARGE_STRUCTURES(dfa, dfa.memoryUsed());
    413 
    414         ASSERT_WITH_MESSAGE(!dfa.nodes[dfa.root].hasActions(), "Filters with Conditions that match everything are not allowed right now.");
    415 
    416         if (dfa.graphSize() < smallDFASize)
    417             smallFiltersWithConditionsDFACombiner.addDFA(WTFMove(dfa));
    418         else {
    419             dfa.minimize();
    420             lowerFiltersWithConditionsDFAToBytecode(WTFMove(dfa));
    421         }
     414    compileToBytecode(WTFMove(topURLFilters), WTFMove(universalTopURLActions), [&](Vector<DFABytecode>&& bytecode) {
     415        client.writeTopURLFiltersBytecode(WTFMove(bytecode));
    422416    });
    423     smallFiltersWithConditionsDFACombiner.combineDFAs(smallDFASize, [&](DFA&& dfa) {
    424         LOG_LARGE_STRUCTURES(dfa, dfa.memoryUsed());
    425         lowerFiltersWithConditionsDFAToBytecode(WTFMove(dfa));
    426     });
    427     ASSERT(filtersWithConditions.isEmpty());
    428    
    429     if (!firstNFAWithConditionsSeen) {
    430         // Our bytecode interpreter expects to have at least one DFA, so if we haven't seen any
    431         // create a dummy one and add any universal actions.
    432 
    433         DFA dummyDFA = DFA::empty();
    434         addUniversalActionsToDFA(dummyDFA, universalActionsWithConditions);
    435        
    436         Vector<DFABytecode> bytecode;
    437         DFABytecodeCompiler compiler(dummyDFA, bytecode);
    438         compiler.compile();
    439         LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
    440         client.writeFiltersWithConditionsBytecode(WTFMove(bytecode));
    441     }
    442     LOG_LARGE_STRUCTURES(universalActionsWithConditions, universalActionsWithConditions.capacity() * sizeof(unsigned));
    443     universalActionsWithConditions.clear();
    444 
    445     conditionFilters.processNFAs(maxNFASize, [&](NFA&& nfa) {
    446 #if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
    447         dataLogF("conditionFilters NFA\n");
    448         nfa.debugPrintDot();
    449 #endif
    450         LOG_LARGE_STRUCTURES(nfa, nfa.memoryUsed());
    451         DFA dfa = NFAToDFA::convert(nfa);
    452 #if CONTENT_EXTENSIONS_STATE_MACHINE_DEBUGGING
    453         dataLogF("conditionFilters DFA\n");
    454         dfa.debugPrintDot();
    455 #endif
    456         LOG_LARGE_STRUCTURES(dfa, dfa.memoryUsed());
    457         // Minimizing this DFA would not be effective because all actions are unique
    458         // and because of the tree-like structure of this DFA.
    459         ASSERT_WITH_MESSAGE(!dfa.nodes[dfa.root].hasActions(), "There should not be any conditions that match everything.");
    460 
    461         Vector<DFABytecode> bytecode;
    462         DFABytecodeCompiler compiler(dfa, bytecode);
    463         compiler.compile();
    464         LOG_LARGE_STRUCTURES(bytecode, bytecode.capacity() * sizeof(uint8_t));
    465         client.writeConditionedFiltersBytecode(WTFMove(bytecode));
    466     });
    467     ASSERT(conditionFilters.isEmpty());
    468417   
    469418#if CONTENT_EXTENSIONS_PERFORMANCE_REPORTING
  • trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.h

    r213533 r213669  
    4141   
    4242    // Functions should be called in this order. All except writeActions and finalize can be called multiple times, though.
    43     virtual void writeActions(Vector<SerializedActionByte>&&) = 0;
     43    virtual void writeActions(Vector<SerializedActionByte>&&, bool conditionsApplyOnlyToDomain) = 0;
    4444    virtual void writeFiltersWithoutConditionsBytecode(Vector<DFABytecode>&&) = 0;
    4545    virtual void writeFiltersWithConditionsBytecode(Vector<DFABytecode>&&) = 0;
    46     virtual void writeConditionedFiltersBytecode(Vector<DFABytecode>&&) = 0;
     46    virtual void writeTopURLFiltersBytecode(Vector<DFABytecode>&&) = 0;
    4747    virtual void finalize() = 0;
    4848};
  • trunk/Source/WebCore/contentextensions/ContentExtensionError.cpp

    r213533 r213669  
    7979                return "Invalid or unsupported regular expression.";
    8080            case ContentExtensionError::JSONInvalidConditionList:
    81                 return "Invalid list of if-domain or unless-domain conditions.";
     81                return "Invalid list of if-domain, unless-domain, if-top-url, or unless-top-url conditions.";
    8282            case ContentExtensionError::JSONTooManyRules:
    8383                return "Too many rules in JSON array.";
     
    8585                return "Domains must be lower case ASCII. Use punycode to encode non-ASCII characters.";
    8686            case ContentExtensionError::JSONMultipleConditions:
    87                 return "A trigger cannot have more than one condition (if-domain or unless-domain)";
     87                return "A trigger cannot have more than one condition (if-domain, unless-domain, if-top-url, or unless-top-url)";
     88            case ContentExtensionError::JSONTopURLAndDomainConditions:
     89                return "A list cannot have if-domain and unless-domain mixed with if-top-url and unless-top-url";
    8890            }
    8991
  • trunk/Source/WebCore/contentextensions/ContentExtensionError.h

    r213533 r213669  
    5353    JSONDomainNotLowerCaseASCII,
    5454    JSONMultipleConditions,
     55    JSONTopURLAndDomainConditions,
    5556    JSONTooManyRules,
    5657   
  • trunk/Source/WebCore/contentextensions/ContentExtensionParser.cpp

    r213533 r213669  
    154154        trigger.urlFilterIsCaseSensitive = urlFilterCaseValue.toBoolean(&exec);
    155155
     156    const JSValue topURLFilterCaseValue = triggerObject.get(&exec, Identifier::fromString(&exec, "top-url-filter-is-case-sensitive"));
     157    if (topURLFilterCaseValue && !scope.exception() && topURLFilterCaseValue.isBoolean())
     158        trigger.topURLConditionIsCaseSensitive = topURLFilterCaseValue.toBoolean(&exec);
     159
    156160    const JSValue resourceTypeValue = triggerObject.get(&exec, Identifier::fromString(&exec, "resource-type"));
    157161    if (!scope.exception() && resourceTypeValue.isObject()) {
     
    197201        return makeUnexpected(ContentExtensionError::JSONInvalidConditionList);
    198202
     203    const JSValue ifTopURLValue = triggerObject.get(&exec, Identifier::fromString(&exec, "if-top-url"));
     204    if (!scope.exception() && ifTopURLValue.isObject()) {
     205        if (trigger.conditionType != Trigger::ConditionType::None)
     206            return makeUnexpected(ContentExtensionError::JSONMultipleConditions);
     207        auto ifTopURL = getStringList(exec, asObject(ifTopURLValue));
     208        if (!ifTopURL.hasValue())
     209            return makeUnexpected(ifTopURL.error());
     210        trigger.conditions = WTFMove(ifTopURL.value());
     211        if (trigger.conditions.isEmpty())
     212            return makeUnexpected(ContentExtensionError::JSONInvalidConditionList);
     213        trigger.conditionType = Trigger::ConditionType::IfTopURL;
     214    } else if (!ifTopURLValue.isUndefined())
     215        return makeUnexpected(ContentExtensionError::JSONInvalidConditionList);
     216
     217    const JSValue unlessTopURLValue = triggerObject.get(&exec, Identifier::fromString(&exec, "unless-top-url"));
     218    if (!scope.exception() && unlessTopURLValue.isObject()) {
     219        if (trigger.conditionType != Trigger::ConditionType::None)
     220            return makeUnexpected(ContentExtensionError::JSONMultipleConditions);
     221        auto unlessTopURL = getStringList(exec, asObject(unlessTopURLValue));
     222        if (!unlessTopURL.hasValue())
     223            return makeUnexpected(unlessTopURL.error());
     224        trigger.conditions = WTFMove(unlessTopURL.value());
     225        if (trigger.conditions.isEmpty())
     226            return makeUnexpected(ContentExtensionError::JSONInvalidConditionList);
     227        trigger.conditionType = Trigger::ConditionType::UnlessTopURL;
     228    } else if (!unlessTopURLValue.isUndefined())
     229        return makeUnexpected(ContentExtensionError::JSONInvalidConditionList);
     230
    199231    return WTFMove(trigger);
    200232}
  • trunk/Source/WebCore/contentextensions/ContentExtensionRule.h

    r213533 r213669  
    4444    String urlFilter;
    4545    bool urlFilterIsCaseSensitive { false };
     46    bool topURLConditionIsCaseSensitive { false };
    4647    ResourceFlags flags { 0 };
    4748    Vector<String> conditions;
     
    5051        IfDomain,
    5152        UnlessDomain,
    52     } conditionType { ConditionType::None };
     53        IfTopURL,
     54        UnlessTopURL,
     55    };
     56    ConditionType conditionType { ConditionType::None };
    5357
    5458    ~Trigger()
    5559    {
    5660        ASSERT(conditions.isEmpty() == (conditionType == ConditionType::None));
     61        if (topURLConditionIsCaseSensitive)
     62            ASSERT(conditionType == ConditionType::IfTopURL || conditionType == ConditionType::UnlessTopURL);
    5763    }
    5864
  • trunk/Source/WebCore/contentextensions/ContentExtensionsBackend.cpp

    r213533 r213669  
    9797        URL topURL = resourceLoadInfo.mainDocumentURL;
    9898        DFABytecodeInterpreter withConditionsInterpreter(compiledExtension.filtersWithConditionsBytecode(), compiledExtension.filtersWithConditionsBytecodeLength());
    99         DFABytecodeInterpreter::Actions withConditionsActions = withConditionsInterpreter.interpretWithConditions(urlCString, flags, contentExtension->cachedConditionedActions(topURL));
     99        DFABytecodeInterpreter::Actions withConditionsActions = withConditionsInterpreter.interpretWithConditions(urlCString, flags, contentExtension->topURLActions(topURL));
    100100       
    101101        const SerializedActionByte* actions = compiledExtension.actions();
  • trunk/Source/WebCore/contentextensions/DFABytecodeInterpreter.cpp

    r213533 r213669  
    9999        || getInstruction(m_bytecode, m_bytecodeLength, programCounter) == DFABytecodeInstruction::AppendActionWithIfCondition);
    100100    uint64_t action = (ifCondition ? IfConditionFlag : 0) | static_cast<uint64_t>(getBits<uint32_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecodeInstruction)));
    101     if (!m_conditionActions || matchesCondition(action, *m_conditionActions))
     101    if (!m_topURLActions || matchesCondition(action, *m_topURLActions))
    102102        actions.add(action);
    103103   
     
    120120    if (loadTypeMatches && resourceTypeMatches) {
    121121        uint64_t actionAndFlags = (ifCondition ? IfConditionFlag : 0) | (static_cast<uint64_t>(flagsToCheck) << 32) | static_cast<uint64_t>(getBits<uint32_t>(m_bytecode, m_bytecodeLength, programCounter + sizeof(DFABytecodeInstruction) + sizeof(uint16_t)));
    122         if (!m_conditionActions || matchesCondition(actionAndFlags, *m_conditionActions))
     122        if (!m_topURLActions || matchesCondition(actionAndFlags, *m_topURLActions))
    123123            actions.add(actionAndFlags);
    124124    }
     
    170170}
    171171   
    172 DFABytecodeInterpreter::Actions DFABytecodeInterpreter::interpretWithConditions(const CString& urlCString, uint16_t flags, const DFABytecodeInterpreter::Actions& conditionActions)
    173 {
    174     ASSERT(!m_conditionActions);
    175     m_conditionActions = &conditionActions;
     172DFABytecodeInterpreter::Actions DFABytecodeInterpreter::interpretWithConditions(const CString& urlCString, uint16_t flags, const DFABytecodeInterpreter::Actions& topURLActions)
     173{
     174    ASSERT(!m_topURLActions);
     175    m_topURLActions = &topURLActions;
    176176    DFABytecodeInterpreter::Actions actions = interpret(urlCString, flags);
    177     m_conditionActions = nullptr;
     177    m_topURLActions = nullptr;
    178178    return actions;
    179179}
  • trunk/Source/WebCore/contentextensions/DFABytecodeInterpreter.h

    r213533 r213669  
    6161    const DFABytecode* m_bytecode;
    6262    const unsigned m_bytecodeLength;
    63     const DFABytecodeInterpreter::Actions* m_conditionActions { nullptr };
     63    const DFABytecodeInterpreter::Actions* m_topURLActions { nullptr };
    6464};
    6565
  • trunk/Source/WebKit2/ChangeLog

    r213665 r213669  
     12017-03-09  Alex Christensen  <achristensen@webkit.org>
     2
     3        [Content Extensions] Introduce if-top-url and unless-top-url
     4        https://bugs.webkit.org/show_bug.cgi?id=169433
     5
     6        Reviewed by Brady Eidson.
     7
     8        Rename conditionedFilters to topURLFilters to reflect the fact that they are the filters
     9        that are run on the top URL, and possibly just the domain of the top url.
     10        I was a bit too aggressive when renaming domain* to condition* in r213533.
     11
     12        * Shared/WebCompiledContentExtension.cpp:
     13        (WebKit::WebCompiledContentExtension::conditionsApplyOnlyToDomain):
     14        (WebKit::WebCompiledContentExtension::topURLFiltersBytecode):
     15        (WebKit::WebCompiledContentExtension::topURLFiltersBytecodeLength):
     16        (WebKit::WebCompiledContentExtension::conditionedFiltersBytecode): Deleted.
     17        (WebKit::WebCompiledContentExtension::conditionedFiltersBytecodeLength): Deleted.
     18        * Shared/WebCompiledContentExtension.h:
     19        * Shared/WebCompiledContentExtensionData.cpp:
     20        (WebKit::WebCompiledContentExtensionData::encode):
     21        (WebKit::WebCompiledContentExtensionData::decode):
     22        * Shared/WebCompiledContentExtensionData.h:
     23        (WebKit::WebCompiledContentExtensionData::WebCompiledContentExtensionData):
     24        * UIProcess/API/APIUserContentExtensionStore.cpp:
     25        (API::encodeContentExtensionMetaData):
     26        (API::decodeContentExtensionMetaData):
     27        (API::compiledToFile):
     28        (API::createExtension):
     29        (API::UserContentExtensionStore::invalidateContentExtensionVersion):
     30        (API::userContentExtensionStoreErrorCategory):
     31        * UIProcess/API/APIUserContentExtensionStore.h:
     32        Increment CurrentContentExtensionFileVersion because we have changed the format of the binary on disk.
     33        We only added 4 bytes, but that's binary incompatible and requires re-compiling any existing content extensions.
     34
    1352017-03-09  Brent Fulgham  <bfulgham@apple.com>
    236
  • trunk/Source/WebKit2/Shared/WebCompiledContentExtension.cpp

    r213533 r213669  
    4545}
    4646
     47bool WebCompiledContentExtension::conditionsApplyOnlyToDomain() const
     48{
     49    return *reinterpret_cast<uint32_t*>(reinterpret_cast<uint8_t*>(m_data.data->data()) + m_data.conditionsApplyOnlyToDomainOffset);
     50}
     51
    4752const WebCore::ContentExtensions::DFABytecode* WebCompiledContentExtension::filtersWithoutConditionsBytecode() const
    4853{
     
    6570}
    6671
    67 const WebCore::ContentExtensions::DFABytecode* WebCompiledContentExtension::conditionedFiltersBytecode() const
     72const WebCore::ContentExtensions::DFABytecode* WebCompiledContentExtension::topURLFiltersBytecode() const
    6873{
    69     return static_cast<const WebCore::ContentExtensions::DFABytecode*>(m_data.data->data()) + m_data.conditionedFiltersBytecodeOffset;
     74    return static_cast<const WebCore::ContentExtensions::DFABytecode*>(m_data.data->data()) + m_data.topURLFiltersBytecodeOffset;
    7075}
    7176
    72 unsigned WebCompiledContentExtension::conditionedFiltersBytecodeLength() const
     77unsigned WebCompiledContentExtension::topURLFiltersBytecodeLength() const
    7378{
    74     return m_data.conditionedFiltersBytecodeSize;
     79    return m_data.topURLFiltersBytecodeSize;
    7580}
    7681
  • trunk/Source/WebKit2/Shared/WebCompiledContentExtension.h

    r213533 r213669  
    4848    const WebCore::ContentExtensions::DFABytecode* filtersWithConditionsBytecode() const final;
    4949    unsigned filtersWithConditionsBytecodeLength() const final;
    50     const WebCore::ContentExtensions::DFABytecode* conditionedFiltersBytecode() const final;
    51     unsigned conditionedFiltersBytecodeLength() const final;
     50    const WebCore::ContentExtensions::DFABytecode* topURLFiltersBytecode() const final;
     51    unsigned topURLFiltersBytecodeLength() const final;
     52    bool conditionsApplyOnlyToDomain() const final;
    5253   
    5354    const WebCore::ContentExtensions::SerializedActionByte* actions() const final;
  • trunk/Source/WebKit2/Shared/WebCompiledContentExtensionData.cpp

    r213533 r213669  
    4545    encoder << filtersWithConditionsBytecodeOffset;
    4646    encoder << filtersWithConditionsBytecodeSize;
    47     encoder << conditionedFiltersBytecodeOffset;
    48     encoder << conditionedFiltersBytecodeSize;
     47    encoder << topURLFiltersBytecodeOffset;
     48    encoder << topURLFiltersBytecodeSize;
    4949}
    5050
     
    6868    if (!decoder.decode(compiledContentExtensionData.filtersWithConditionsBytecodeSize))
    6969        return false;
    70     if (!decoder.decode(compiledContentExtensionData.conditionedFiltersBytecodeOffset))
     70    if (!decoder.decode(compiledContentExtensionData.topURLFiltersBytecodeOffset))
    7171        return false;
    72     if (!decoder.decode(compiledContentExtensionData.conditionedFiltersBytecodeSize))
     72    if (!decoder.decode(compiledContentExtensionData.topURLFiltersBytecodeSize))
    7373        return false;
    7474
  • trunk/Source/WebKit2/Shared/WebCompiledContentExtensionData.h

    r213533 r213669  
    4343    WebCompiledContentExtensionData() = default;
    4444
    45     WebCompiledContentExtensionData(RefPtr<SharedMemory>&& data, NetworkCache::Data fileData, unsigned actionsOffset, unsigned actionsSize, unsigned filtersWithoutConditionsBytecodeOffset, unsigned filtersWithoutConditionsBytecodeSize, unsigned filtersWithConditionsBytecodeOffset, unsigned filtersWithConditionsBytecodeSize, unsigned conditionedFiltersBytecodeOffset, unsigned conditionedFiltersBytecodeSize)
     45    WebCompiledContentExtensionData(RefPtr<SharedMemory>&& data, NetworkCache::Data fileData, unsigned conditionsApplyOnlyToDomainOffset, unsigned actionsOffset, unsigned actionsSize, unsigned filtersWithoutConditionsBytecodeOffset, unsigned filtersWithoutConditionsBytecodeSize, unsigned filtersWithConditionsBytecodeOffset, unsigned filtersWithConditionsBytecodeSize, unsigned topURLFiltersBytecodeOffset, unsigned topURLFiltersBytecodeSize)
    4646        : data(WTFMove(data))
    4747        , fileData(fileData)
     48        , conditionsApplyOnlyToDomainOffset(conditionsApplyOnlyToDomainOffset)
    4849        , actionsOffset(actionsOffset)
    4950        , actionsSize(actionsSize)
     
    5253        , filtersWithConditionsBytecodeOffset(filtersWithConditionsBytecodeOffset)
    5354        , filtersWithConditionsBytecodeSize(filtersWithConditionsBytecodeSize)
    54         , conditionedFiltersBytecodeOffset(conditionedFiltersBytecodeOffset)
    55         , conditionedFiltersBytecodeSize(conditionedFiltersBytecodeSize)
     55        , topURLFiltersBytecodeOffset(topURLFiltersBytecodeOffset)
     56        , topURLFiltersBytecodeSize(topURLFiltersBytecodeSize)
    5657    {
    5758    }
     
    6263    RefPtr<SharedMemory> data;
    6364    NetworkCache::Data fileData;
     65    unsigned conditionsApplyOnlyToDomainOffset { 0 };
    6466    unsigned actionsOffset { 0 };
    6567    unsigned actionsSize { 0 };
     
    6870    unsigned filtersWithConditionsBytecodeOffset { 0 };
    6971    unsigned filtersWithConditionsBytecodeSize { 0 };
    70     unsigned conditionedFiltersBytecodeOffset { 0 };
    71     unsigned conditionedFiltersBytecodeSize { 0 };
     72    unsigned topURLFiltersBytecodeOffset { 0 };
     73    unsigned topURLFiltersBytecodeSize { 0 };
    7274};
    7375
  • trunk/Source/WebKit2/UIProcess/API/APIUserContentExtensionStore.cpp

    r213533 r213669  
    8080}
    8181
    82 const size_t ContentExtensionFileHeaderSize = sizeof(uint32_t) + 4 * sizeof(uint64_t);
     82// The size and offset of the densely packed bytes in the file, not sizeof and offsetof, which would
     83// represent the size and offset of the structure in memory, possibly with compiler-added padding.
     84const size_t ContentExtensionFileHeaderSize = 2 * sizeof(uint32_t) + 4 * sizeof(uint64_t);
     85const size_t ConditionsApplyOnlyToDomainOffset = sizeof(uint32_t) + 4 * sizeof(uint64_t);
     86
    8387struct ContentExtensionMetaData {
    8488    uint32_t version { UserContentExtensionStore::CurrentContentExtensionFileVersion };
     
    8791    uint64_t filtersWithConditionsBytecodeSize { 0 };
    8892    uint64_t conditionedFiltersBytecodeSize { 0 };
     93    uint32_t conditionsApplyOnlyToDomain { false };
    8994   
    9095    size_t fileSize() const
     
    107112    encoder << metaData.filtersWithConditionsBytecodeSize;
    108113    encoder << metaData.conditionedFiltersBytecodeSize;
     114    encoder << metaData.conditionsApplyOnlyToDomain;
    109115
    110116    ASSERT(encoder.bufferSize() == ContentExtensionFileHeaderSize);
     
    132138        if (!decoder.decode(metaData.conditionedFiltersBytecodeSize))
    133139            return false;
     140        if (!decoder.decode(metaData.conditionsApplyOnlyToDomain))
     141            return false;
    134142        success = true;
    135143        return false;
     
    178186            ASSERT(!metaData.filtersWithConditionsBytecodeSize);
    179187            ASSERT(!metaData.conditionedFiltersBytecodeSize);
    180         }
    181        
    182         void writeFiltersWithoutConditionsBytecode(Vector<DFABytecode>&& bytecode) override
     188            ASSERT(!metaData.conditionsApplyOnlyToDomain);
     189        }
     190       
     191        void writeFiltersWithoutConditionsBytecode(Vector<DFABytecode>&& bytecode) final
    183192        {
    184193            ASSERT(!m_filtersWithConditionBytecodeWritten);
     
    188197        }
    189198       
    190         void writeFiltersWithConditionsBytecode(Vector<DFABytecode>&& bytecode) override
     199        void writeFiltersWithConditionsBytecode(Vector<DFABytecode>&& bytecode) final
    191200        {
    192201            ASSERT(!m_conditionFiltersBytecodeWritten);
     
    195204        }
    196205       
    197         void writeConditionedFiltersBytecode(Vector<DFABytecode>&& bytecode) override
     206        void writeTopURLFiltersBytecode(Vector<DFABytecode>&& bytecode) final
    198207        {
    199208            m_conditionFiltersBytecodeWritten += bytecode.size();
     
    201210        }
    202211
    203         void writeActions(Vector<SerializedActionByte>&& actions) override
     212        void writeActions(Vector<SerializedActionByte>&& actions, bool conditionsApplyOnlyToDomain) final
    204213        {
    205214            ASSERT(!m_filtersWithoutConditionsBytecodeWritten);
     
    208217            ASSERT(!m_actionsWritten);
    209218            m_actionsWritten += actions.size();
     219            m_conditionsApplyOnlyToDomain = conditionsApplyOnlyToDomain;
    210220            writeToFile(Data(actions.data(), actions.size()));
    211221        }
    212222       
    213         void finalize() override
     223        void finalize() final
    214224        {
    215225            m_metaData.actionsSize = m_actionsWritten;
     
    217227            m_metaData.filtersWithConditionsBytecodeSize = m_filtersWithConditionBytecodeWritten;
    218228            m_metaData.conditionedFiltersBytecodeSize = m_conditionFiltersBytecodeWritten;
     229            m_metaData.conditionsApplyOnlyToDomain = m_conditionsApplyOnlyToDomain;
    219230           
    220231            Data header = encodeContentExtensionMetaData(m_metaData);
     
    243254        size_t m_conditionFiltersBytecodeWritten { 0 };
    244255        size_t m_actionsWritten { 0 };
     256        bool m_conditionsApplyOnlyToDomain { false };
    245257        bool m_fileError { false };
    246258    };
     
    286298        WTFMove(sharedMemory),
    287299        fileData,
     300        ConditionsApplyOnlyToDomainOffset,
    288301        ContentExtensionFileHeaderSize,
    289302        metaData.actionsSize,
     
    384397    if (file == WebCore::invalidPlatformFileHandle)
    385398        return;
    386     ContentExtensionMetaData invalidHeader = {0, 0, 0, 0, 0};
     399    ContentExtensionMetaData invalidHeader;
    387400    auto bytesWritten = WebCore::writeToFile(file, reinterpret_cast<const char*>(&invalidHeader), sizeof(invalidHeader));
    388401    ASSERT_UNUSED(bytesWritten, bytesWritten == sizeof(invalidHeader));
     
    393406{
    394407    class UserContentExtensionStoreErrorCategory : public std::error_category {
    395         const char* name() const noexcept override
     408        const char* name() const noexcept final
    396409        {
    397410            return "user content extension store";
    398411        }
    399412
    400         std::string message(int errorCode) const override
     413        std::string message(int errorCode) const final
    401414        {
    402415            switch (static_cast<UserContentExtensionStore::Error>(errorCode)) {
  • trunk/Source/WebKit2/UIProcess/API/APIUserContentExtensionStore.h

    r213533 r213669  
    5151    // This should be incremented every time a functional change is made to the bytecode, file format, etc.
    5252    // to prevent crashing while loading old data.
    53     const static uint32_t CurrentContentExtensionFileVersion = 7;
     53    const static uint32_t CurrentContentExtensionFileVersion = 8;
    5454
    5555    static UserContentExtensionStore& defaultStore();
  • trunk/Tools/ChangeLog

    r213658 r213669  
     12017-03-09  Alex Christensen  <achristensen@webkit.org>
     2
     3        [Content Extensions] Introduce if-top-url and unless-top-url
     4        https://bugs.webkit.org/show_bug.cgi?id=169433
     5
     6        Reviewed by Brady Eidson.
     7
     8        * TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp:
     9        (TestWebKitAPI::TEST_F):
     10        Add tests for new functionality and new failure types.
     11
    1122017-03-09  Srinivasan Vijayaraghavan  <svijayaraghavan@apple.com>
    213
  • trunk/Tools/TestWebKitAPI/Tests/WebCore/ContentExtensions.cpp

    r213533 r213669  
    8686    Vector<ContentExtensions::DFABytecode> filtersWithoutConditions;
    8787    Vector<ContentExtensions::DFABytecode> filtersWithConditions;
    88     Vector<ContentExtensions::DFABytecode> conditionedFilters;
     88    Vector<ContentExtensions::DFABytecode> topURLFilters;
     89    bool conditionsApplyOnlyToDomain { false };
    8990};
    9091
     
    9798        EXPECT_EQ(data.filtersWithoutConditions.size(), 0ull);
    9899        EXPECT_EQ(data.filtersWithConditions.size(), 0ull);
    99         EXPECT_EQ(data.conditionedFilters.size(), 0ull);
     100        EXPECT_EQ(data.topURLFilters.size(), 0ull);
    100101    }
    101102
    102     void writeActions(Vector<ContentExtensions::SerializedActionByte>&& actions) final
     103    void writeActions(Vector<ContentExtensions::SerializedActionByte>&& actions, bool conditionsApplyOnlyToDomain) final
    103104    {
    104105        EXPECT_FALSE(finalized);
     
    106107        EXPECT_EQ(m_data.filtersWithoutConditions.size(), 0ull);
    107108        EXPECT_EQ(m_data.filtersWithConditions.size(), 0ull);
    108         EXPECT_EQ(m_data.conditionedFilters.size(), 0ull);
     109        EXPECT_EQ(m_data.topURLFilters.size(), 0ull);
     110        EXPECT_EQ(m_data.actions.size(), 0ull);
    109111        m_data.actions.appendVector(actions);
     112        m_data.conditionsApplyOnlyToDomain = conditionsApplyOnlyToDomain;
    110113    }
    111114   
     
    114117        EXPECT_FALSE(finalized);
    115118        EXPECT_EQ(m_data.filtersWithConditions.size(), 0ull);
    116         EXPECT_EQ(m_data.conditionedFilters.size(), 0ull);
     119        EXPECT_EQ(m_data.topURLFilters.size(), 0ull);
    117120        m_data.filtersWithoutConditions.appendVector(bytecode);
    118121    }
     
    121124    {
    122125        EXPECT_FALSE(finalized);
    123         EXPECT_EQ(m_data.conditionedFilters.size(), 0ull);
     126        EXPECT_EQ(m_data.topURLFilters.size(), 0ull);
    124127        m_data.filtersWithConditions.appendVector(bytecode);
    125128    }
    126129   
    127     void writeConditionedFiltersBytecode(Vector<ContentExtensions::DFABytecode>&& bytecode) final
     130    void writeTopURLFiltersBytecode(Vector<ContentExtensions::DFABytecode>&& bytecode) final
    128131    {
    129132        EXPECT_FALSE(finalized);
    130         m_data.conditionedFilters.appendVector(bytecode);
     133        m_data.topURLFilters.appendVector(bytecode);
    131134    }
    132135   
     
    172175    const ContentExtensions::DFABytecode* filtersWithConditionsBytecode() const final { return m_data.filtersWithConditions.data(); }
    173176    unsigned filtersWithConditionsBytecodeLength() const final { return m_data.filtersWithConditions.size(); }
    174     const ContentExtensions::DFABytecode* conditionedFiltersBytecode() const final { return m_data.conditionedFilters.data(); }
    175     unsigned conditionedFiltersBytecodeLength() const final { return m_data.conditionedFilters.size(); }
     177    const ContentExtensions::DFABytecode* topURLFiltersBytecode() const final { return m_data.topURLFilters.data(); }
     178    unsigned topURLFiltersBytecodeLength() const final { return m_data.topURLFilters.size(); }
     179    bool conditionsApplyOnlyToDomain() const final { return m_data.conditionsApplyOnlyToDomain; }
    176180
    177181private:
     
    789793    testRequest(backend, mainDocumentRequest("http://notwebkit.org/except-this-ignore-previous-trigger-on-scripts.html.test.html", ResourceType::Script), { ContentExtensions::ActionType::BlockCookies, ContentExtensions::ActionType::CSSDisplayNoneSelector, ContentExtensions::ActionType::BlockCookies });
    790794    testRequest(backend, mainDocumentRequest("http://webkit.org/except-this-ignore-previous-trigger-on-scripts.html.test.html", ResourceType::Script), { ContentExtensions::ActionType::BlockCookies, ContentExtensions::ActionType::CSSDisplayNoneSelector, ContentExtensions::ActionType::BlockLoad, ContentExtensions::ActionType::BlockCookies });
    791 
     795}
     796
     797TEST_F(ContentExtensionTest, TopURL)
     798{
     799    const Vector<ContentExtensions::ActionType> blockLoad = { ContentExtensions::ActionType::BlockLoad };
     800    const Vector<ContentExtensions::ActionType> blockCookies = { ContentExtensions::ActionType::BlockCookies };
     801
     802    auto ifTopURL = makeBackend("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"if-top-url\":[\"^http://web.*kit.org\"]}}]");
     803    testRequest(ifTopURL, mainDocumentRequest("http://webkit.org/test.html"), blockLoad);
     804    testRequest(ifTopURL, mainDocumentRequest("http://webkit.org/test.not_html"), { });
     805    testRequest(ifTopURL, mainDocumentRequest("http://WEBKIT.org/test.html"), blockLoad);
     806    testRequest(ifTopURL, mainDocumentRequest("http://webkit.org/TEST.html"), blockLoad);
     807    testRequest(ifTopURL, mainDocumentRequest("http://web__kit.org/test.html"), blockLoad);
     808    testRequest(ifTopURL, mainDocumentRequest("http://webk__it.org/test.html"), { });
     809    testRequest(ifTopURL, mainDocumentRequest("http://web__kit.org/test.not_html"), { });
     810    testRequest(ifTopURL, mainDocumentRequest("http://not_webkit.org/test.html"), { });
     811    testRequest(ifTopURL, subResourceRequest("http://not_webkit.org/test.html", "http://webkit.org/not_test.html"), blockLoad);
     812    testRequest(ifTopURL, subResourceRequest("http://not_webkit.org/test.html", "http://not_webkit.org/not_test.html"), { });
     813    testRequest(ifTopURL, subResourceRequest("http://webkit.org/test.html", "http://not_webkit.org/not_test.html"), { });
     814    testRequest(ifTopURL, subResourceRequest("http://webkit.org/test.html", "http://not_webkit.org/test.html"), { });
     815    testRequest(ifTopURL, subResourceRequest("http://webkit.org/test.html", "http://webkit.org/test.html"), blockLoad);
     816    testRequest(ifTopURL, subResourceRequest("http://webkit.org/test.html", "http://example.com/#http://webkit.org/test.html"), { });
     817    testRequest(ifTopURL, subResourceRequest("http://example.com/#http://webkit.org/test.html", "http://webkit.org/test.html"), blockLoad);
     818
     819    auto unlessTopURL = makeBackend("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"unless-top-url\":[\"^http://web.*kit.org\"]}}]");
     820    testRequest(unlessTopURL, mainDocumentRequest("http://webkit.org/test.html"), { });
     821    testRequest(unlessTopURL, mainDocumentRequest("http://WEBKIT.org/test.html"), { });
     822    testRequest(unlessTopURL, mainDocumentRequest("http://webkit.org/TEST.html"), { });
     823    testRequest(unlessTopURL, mainDocumentRequest("http://webkit.org/test.not_html"), { });
     824    testRequest(unlessTopURL, mainDocumentRequest("http://web__kit.org/test.html"), { });
     825    testRequest(unlessTopURL, mainDocumentRequest("http://webk__it.org/test.html"), blockLoad);
     826    testRequest(unlessTopURL, mainDocumentRequest("http://web__kit.org/test.not_html"), { });
     827    testRequest(unlessTopURL, mainDocumentRequest("http://not_webkit.org/test.html"), blockLoad);
     828    testRequest(unlessTopURL, subResourceRequest("http://not_webkit.org/test.html", "http://webkit.org/not_test.html"), { });
     829    testRequest(unlessTopURL, subResourceRequest("http://not_webkit.org/test.html", "http://not_webkit.org/not_test.html"), blockLoad);
     830    testRequest(unlessTopURL, subResourceRequest("http://webkit.org/test.html", "http://not_webkit.org/not_test.html"), blockLoad);
     831    testRequest(unlessTopURL, subResourceRequest("http://webkit.org/test.html", "http://not_webkit.org/test.html"), blockLoad);
     832    testRequest(unlessTopURL, subResourceRequest("http://webkit.org/test.html", "http://webkit.org/test.html"), { });
     833    testRequest(unlessTopURL, subResourceRequest("http://webkit.org/test.html", "http://example.com/#http://webkit.org/test.html"), blockLoad);
     834    testRequest(unlessTopURL, subResourceRequest("http://example.com/#http://webkit.org/test.html", "http://webkit.org/test.html"), { });
     835
     836    auto ifTopURLMatchesEverything = makeBackend("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"if-top-url\":[\".*\"]}}]");
     837    testRequest(ifTopURLMatchesEverything, mainDocumentRequest("http://webkit.org/test.html"), blockLoad);
     838    testRequest(ifTopURLMatchesEverything, mainDocumentRequest("http://webkit.org/test.not_html"), { });
     839    testRequest(ifTopURLMatchesEverything, mainDocumentRequest("http://not_webkit.org/test.html"), blockLoad);
     840   
     841    auto unlessTopURLMatchesEverything = makeBackend("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"unless-top-url\":[\".*\"]}}]");
     842    testRequest(unlessTopURLMatchesEverything, mainDocumentRequest("http://webkit.org/test.html"), { });
     843    testRequest(unlessTopURLMatchesEverything, mainDocumentRequest("http://webkit.org/test.not_html"), { });
     844    testRequest(unlessTopURLMatchesEverything, mainDocumentRequest("http://not_webkit.org/test.html"), { });
     845   
     846    auto mixedConditions = makeBackend("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"if-top-url\":[\"^http://web.*kit.org\"]}},"
     847        "{\"action\":{\"type\":\"block-cookies\"},\"trigger\":{\"resource-type\":[\"document\"], \"url-filter\":\"test\\\\.html\", \"unless-top-url\":[\"^http://web.*kit.org\"]}}]");
     848    testRequest(mixedConditions, mainDocumentRequest("http://webkit.org/test.html"), blockLoad);
     849    testRequest(mixedConditions, mainDocumentRequest("http://not_webkit.org/test.html"), blockCookies);
     850    testRequest(mixedConditions, subResourceRequest("http://webkit.org/test.html", "http://not_webkit.org", ResourceType::Document), blockCookies);
     851    testRequest(mixedConditions, subResourceRequest("http://webkit.org/test.html", "http://not_webkit.org", ResourceType::Image), { });
     852
     853    auto caseSensitive = makeBackend("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"if-top-url\":[\"^http://web.*kit.org/test\"], \"top-url-filter-is-case-sensitive\":true}}]");
     854    testRequest(caseSensitive, mainDocumentRequest("http://webkit.org/test.html"), blockLoad);
     855    testRequest(caseSensitive, mainDocumentRequest("http://WEBKIT.org/test.html"), blockLoad); // domains are canonicalized before running regexes.
     856    testRequest(caseSensitive, mainDocumentRequest("http://webkit.org/TEST.html"), { });
     857
     858    auto caseInsensitive = makeBackend("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"if-top-url\":[\"^http://web.*kit.org/test\"]}}]");
     859    testRequest(caseInsensitive, mainDocumentRequest("http://webkit.org/test.html"), blockLoad);
     860    testRequest(caseInsensitive, mainDocumentRequest("http://webkit.org/TEST.html"), blockLoad);
    792861}
    793862
     
    13651434    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"[\"}}]",
    13661435        ContentExtensions::ContentExtensionError::JSONInvalidRegex);
     1436
     1437    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-domain\":[\"a\"],\"unless-domain\":[\"a\"]}}]", ContentExtensions::ContentExtensionError::JSONMultipleConditions);
     1438    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-top-url\":[\"a\"],\"unless-top-url\":[]}}]", ContentExtensions::ContentExtensionError::JSONMultipleConditions);
     1439    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-top-url\":[\"a\"],\"unless-top-url\":[\"a\"]}}]", ContentExtensions::ContentExtensionError::JSONMultipleConditions);
     1440    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-domain\":[\"a\"],\"if-top-url\":[\"a\"]}}]", ContentExtensions::ContentExtensionError::JSONMultipleConditions);
     1441    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-top-url\":[],\"unless-domain\":[\"a\"]}}]", ContentExtensions::ContentExtensionError::JSONMultipleConditions);
     1442    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-top-url\":[\"a\"],\"if-domain\":[\"a\"]}}]", ContentExtensions::ContentExtensionError::JSONMultipleConditions);
     1443    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-top-url\":[\"a\"]}}, {\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-domain\":[\"a\"]}}]", ContentExtensions::ContentExtensionError::JSONTopURLAndDomainConditions);
     1444    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"if-top-url\":[\"a\"]}}, {\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\",\"unless-domain\":[\"a\"]}}]", ContentExtensions::ContentExtensionError::JSONTopURLAndDomainConditions);
     1445    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"test\\\\.html\", \"unless-top-url\":[\"[\"]}}]", ContentExtensions::ContentExtensionError::JSONInvalidRegex);
     1446    checkCompilerError("[{\"action\":{\"type\":\"block\"},\"trigger\":{\"url-filter\":\"webkit.org\", \"unexpected-identifier-should-be-ignored\":5}}]", { });
    13671447}
    13681448
Note: See TracChangeset for help on using the changeset viewer.