Changeset 60275 in webkit


Ignore:
Timestamp:
May 27, 2010 12:14:02 AM (14 years ago)
Author:
eric@webkit.org
Message:

2010-05-26 Eric Seidel <​eric@webkit.org>

Reviewed by Adam Barth.

Update our expectations now that we're handling external scripts.

  • html5lib/webkit-runner-expected-html5.txt:

2010-05-26 Eric Seidel <​eric@webkit.org>

Reviewed by Adam Barth.

Teach the HTML5 parser how to handle external scripts
​https://bugs.webkit.org/show_bug.cgi?id=39716

Make it possible for the HTML5Tokenizer to run external scripts.
I created a new class HTML5ScriptRunner to hold all of the
script-logic which is scattered throughout the old HTMLTokenizer.

The design is for the HTML5Tokenizer (the "controller") to hold
the Lexer, TreeBuilder and ScriptRunner. The Lexer returns back
to the controller, which passes tokens to the TreeBuilder. When the
treebuilder encounters a </script> tag it pauses itself and returns
back to the controller which calls the ScriptRunner. The TreeBuilder
is un-paused when the HTML5Tokenizer calls takeScriptToProcess().

The ScriptRunner attempts to process the passed script, and additionally
any blocked scripts it can. It returns to the controller indicating if
parsing should continue. If not, callbacks when external scripts load
or when stylesheets are finished parsing will cause the controller to
kick off script execution and parsing again at a later point.

  • WebCore.xcodeproj/project.pbxproj:
    • Add HTML5ScriptRunner.*
  • bindings/js/CachedScriptSourceProvider.h:
    • Add missing include discovered while building.
  • dom/ScriptElement.cpp: (WebCore::ScriptElement::finishParsingChildren):
    • Remove previous hack for inline <script> execution.
  • dom/ScriptElement.h:
    • Explain the HTML5 spec names for m_evaluated and m_createdByParser.
  • html/HTML5ScriptRunner.cpp: Added. (WebCore::HTML5ScriptRunner::HTML5ScriptRunner):
    • The HTML5Tokenizer is passed to the HTML5ScriptRunner as a CachedResourceClient. The HTML5ScriptRunner will register the HTML5Tokenizer for notifyFinished callbacks when the scripts load. The HTML5Tokenizer is expected to call the HTML5ScriptRunner to execute any loaded scripts at that point.

(WebCore::HTML5ScriptRunner::~HTML5ScriptRunner):
(WebCore::HTML5ScriptRunner::frame): Helper method.
(WebCore::createScriptLoadEvent): Helper method.
(WebCore::createScriptErrorEvent): Helper method.
(WebCore::HTML5ScriptRunner::sourceFromPendingScript):

  • Helper method for dealing with both inline and external script types.

(WebCore::HTML5ScriptRunner::isPendingScriptReady):

  • Helper for dealing with both inline and external scripts.

(WebCore::HTML5ScriptRunner::executePendingScript):

  • Execute one script. Both external and inline scripts can become m_parsingBlockingScript if they can't be executed immediately after parsing.

(WebCore::HTML5ScriptRunner::execute):

  • Takes a script element from the tree builder and tries to process it.

(WebCore::HTML5ScriptRunner::executeParsingBlockingScripts):

  • Runs the current parsing blocking script if ready.
  • Running a script could add another parsing blocking script so we loop until there is no ready-to-run parsing blocking script.

(WebCore::HTML5ScriptRunner::executeScriptsWaitingForLoad):

  • Called by HTML5Tokenizer when a script loads.

(WebCore::HTML5ScriptRunner::executeScriptsWaitingForStylesheets):

  • Called by HTML5Tokenizer when stylesheets complete.

(WebCore::HTML5ScriptRunner::requestScript):

  • Transcription of the HTML5 spec.

(WebCore::HTML5ScriptRunner::runScript):

  • Transcription of the HTML5 spec.
  • html/HTML5ScriptRunner.h: Added.
    • New class to handle script loading and execution for the HTML5 parser.
  • html/HTML5Tokenizer.cpp: (WebCore::HTML5Tokenizer::HTML5Tokenizer):
    • Create a HTML5ScriptRunner and pass it "this" as the CachedResourceClient.

(WebCore::HTML5Tokenizer::pumpLexer):

  • When the parser is paused, try to run scripts.

(WebCore::HTML5Tokenizer::write):

  • Only pump the lexer when the parser is not paused.

(WebCore::HTML5Tokenizer::end):

  • finish() tells us that we've reached EOF, not end()
  • Only pump the lexer when the parser is not paused.

(WebCore::HTML5Tokenizer::finish):

  • Mark EOF, and end() if we're not waiting on scripts.

(WebCore::HTML5Tokenizer::isWaitingForScripts):

  • isPaused() seems to mean isPausedForExternalScripts().

(WebCore::HTML5Tokenizer::resumeParsingAfterScriptExecution):
(WebCore::HTML5Tokenizer::notifyFinished):
(WebCore::HTML5Tokenizer::executeScriptsWaitingForStylesheets):

  • html/HTML5Tokenizer.h:
  • html/HTML5TreeBuilder.cpp: (WebCore::HTML5TreeBuilder::HTML5TreeBuilder):
    • Add an m_isPaused flag.

(WebCore::HTML5TreeBuilder::handleScriptStartTag):
(WebCore::HTML5TreeBuilder::handleScriptEndTag):
(WebCore::HTML5TreeBuilder::takeScriptToProcess):

  • Acknowledge that the caller has received the script element. It is the caller's responsibility to execute the script if necessary and re-pause the tree builder if necessary.

(WebCore::HTML5TreeBuilder::passTokenToLegacyParser):

  • Save off the current script tag so that it can be passed to the HTML5ScriptRunner when we're paused.
  • html/HTML5TreeBuilder.h: (WebCore::HTML5TreeBuilder::setPaused): (WebCore::HTML5TreeBuilder::isPaused):
Location:
trunk
Files:
1 added
17 edited
1 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r60269 r60275  
     12010-05-26  Eric Seidel  <eric@webkit.org>
     2
     3        Reviewed by Adam Barth.
     4
     5        Update our expectations now that we're handling external scripts.
     6
     7        * html5lib/webkit-runner-expected-html5.txt:
     8
    192010-05-26  Tony Chang  <tony@chromium.org>
    210

  • trunk/LayoutTests/html5lib/webkit-runner-expected-html5.txt

    r60172 r60275  
    1 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    2 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    3 CONSOLE MESSAGE: line 1: ReferenceError: Can't find variable: a
    4 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    5 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    6 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    7 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    8 CONSOLE MESSAGE: line 1: ReferenceError: Can't find variable: a
    9 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    10 CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    111CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    122CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    133CONSOLE MESSAGE: line 2: PASS
    144CONSOLE MESSAGE: line 2: FOO<span>BAR</span>BAZ
     5CONSOLE MESSAGE: line 1: SyntaxError: Parse error
     6CONSOLE MESSAGE: line 1: SyntaxError: Parse error
    157Content-Type: text/plain
    168resources/tests1.dat: 25, 29, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 57, 59, 78, 79, 80, 81, 82, 87, 90, 91, 92, 96, 98, 101, 104, 106, 109, 113

  • trunk/WebCore/Android.mk

    r60228 r60275  
    250250        html/HTML5Tokenizer.cpp \
    251251        html/HTML5TreeBuilder.cpp \
     252        html/HTML5ScriptRunner.cpp \
    252253        html/HTMLAllCollection.cpp \
    253254        html/HTMLCollection.cpp \

  • trunk/WebCore/CMakeLists.txt

    r60240 r60275  
    904904    html/HTML5Tokenizer.cpp
    905905    html/HTML5TreeBuilder.cpp
     906    html/HTML5ScriptRunner.cpp
    906907    html/HTMLAllCollection.cpp
    907908    html/HTMLAnchorElement.cpp

  • trunk/WebCore/ChangeLog

    r60274 r60275  
     12010-05-26  Eric Seidel  <eric@webkit.org>
     2
     3        Reviewed by Adam Barth.
     4
     5        Teach the HTML5 parser how to handle external scripts
     6        https://bugs.webkit.org/show_bug.cgi?id=39716
     7
     8        Make it possible for the HTML5Tokenizer to run external scripts.
     9        I created a new class HTML5ScriptRunner to hold all of the
     10        script-logic which is scattered throughout the old HTMLTokenizer.
     11
     12        The design is for the HTML5Tokenizer (the "controller") to hold
     13        the Lexer, TreeBuilder and ScriptRunner.  The Lexer returns back
     14        to the controller, which passes tokens to the TreeBuilder.  When the
     15        treebuilder encounters a </script> tag it pauses itself and returns
     16        back to the controller which calls the ScriptRunner.  The TreeBuilder
     17        is un-paused when the HTML5Tokenizer calls takeScriptToProcess().
     18
     19        The ScriptRunner attempts to process the passed script, and additionally
     20        any blocked scripts it can.  It returns to the controller indicating if
     21        parsing should continue.  If not, callbacks when external scripts load
     22        or when stylesheets are finished parsing will cause the controller to
     23        kick off script execution and parsing again at a later point.
     24
     25        * WebCore.xcodeproj/project.pbxproj:
     26         - Add HTML5ScriptRunner.*
     27        * bindings/js/CachedScriptSourceProvider.h:
     28         - Add missing include discovered while building.
     29        * dom/ScriptElement.cpp:
     30        (WebCore::ScriptElement::finishParsingChildren):
     31         - Remove previous hack for inline <script> execution.
     32        * dom/ScriptElement.h:
     33         - Explain the HTML5 spec names for m_evaluated and m_createdByParser.
     34        * html/HTML5ScriptRunner.cpp: Added.
     35        (WebCore::HTML5ScriptRunner::HTML5ScriptRunner):
     36         - The HTML5Tokenizer is passed to the HTML5ScriptRunner as a
     37           CachedResourceClient.  The HTML5ScriptRunner will register the
     38           HTML5Tokenizer for notifyFinished callbacks when the scripts load.
     39           The HTML5Tokenizer is expected to call the HTML5ScriptRunner to
     40           execute any loaded scripts at that point.
     41        (WebCore::HTML5ScriptRunner::~HTML5ScriptRunner):
     42        (WebCore::HTML5ScriptRunner::frame): Helper method.
     43        (WebCore::createScriptLoadEvent): Helper method.
     44        (WebCore::createScriptErrorEvent): Helper method.
     45        (WebCore::HTML5ScriptRunner::sourceFromPendingScript):
     46         - Helper method for dealing with both inline and external script types.
     47        (WebCore::HTML5ScriptRunner::isPendingScriptReady):
     48         - Helper for dealing with both inline and external scripts.
     49        (WebCore::HTML5ScriptRunner::executePendingScript):
     50         - Execute one script.  Both external and inline scripts
     51           can become m_parsingBlockingScript if they can't be executed
     52           immediately after parsing.
     53        (WebCore::HTML5ScriptRunner::execute):
     54         - Takes a script element from the tree builder and tries
     55           to process it.
     56        (WebCore::HTML5ScriptRunner::executeParsingBlockingScripts):
     57         - Runs the current parsing blocking script if ready.
     58         - Running a script could add another parsing blocking script
     59           so we loop until there is no ready-to-run parsing blocking script.
     60        (WebCore::HTML5ScriptRunner::executeScriptsWaitingForLoad):
     61         - Called by HTML5Tokenizer when a script loads.
     62        (WebCore::HTML5ScriptRunner::executeScriptsWaitingForStylesheets):
     63         - Called by HTML5Tokenizer when stylesheets complete.
     64        (WebCore::HTML5ScriptRunner::requestScript):
     65         - Transcription of the HTML5 spec.
     66        (WebCore::HTML5ScriptRunner::runScript):
     67         - Transcription of the HTML5 spec.
     68        * html/HTML5ScriptRunner.h: Added.
     69         - New class to handle script loading and execution for the HTML5 parser.
     70        * html/HTML5Tokenizer.cpp:
     71        (WebCore::HTML5Tokenizer::HTML5Tokenizer):
     72         - Create a HTML5ScriptRunner and pass it "this" as the CachedResourceClient.
     73        (WebCore::HTML5Tokenizer::pumpLexer):
     74         - When the parser is paused, try to run scripts.
     75        (WebCore::HTML5Tokenizer::write):
     76         - Only pump the lexer when the parser is not paused.
     77        (WebCore::HTML5Tokenizer::end):
     78         - finish() tells us that we've reached EOF, not end()
     79         - Only pump the lexer when the parser is not paused.
     80        (WebCore::HTML5Tokenizer::finish):
     81         - Mark EOF, and end() if we're not waiting on scripts.
     82        (WebCore::HTML5Tokenizer::isWaitingForScripts):
     83         - isPaused() seems to mean isPausedForExternalScripts().
     84        (WebCore::HTML5Tokenizer::resumeParsingAfterScriptExecution):
     85        (WebCore::HTML5Tokenizer::notifyFinished):
     86        (WebCore::HTML5Tokenizer::executeScriptsWaitingForStylesheets):
     87        * html/HTML5Tokenizer.h:
     88        * html/HTML5TreeBuilder.cpp:
     89        (WebCore::HTML5TreeBuilder::HTML5TreeBuilder):
     90         - Add an m_isPaused flag.
     91        (WebCore::HTML5TreeBuilder::handleScriptStartTag):
     92        (WebCore::HTML5TreeBuilder::handleScriptEndTag):
     93        (WebCore::HTML5TreeBuilder::takeScriptToProcess):
     94         - Acknowledge that the caller has received the script element.
     95           It is the caller's responsibility to execute the script if necessary
     96           and re-pause the tree builder if necessary.
     97        (WebCore::HTML5TreeBuilder::passTokenToLegacyParser):
     98         - Save off the current script tag so that it can be passed to
     99           the HTML5ScriptRunner when we're paused.
     100        * html/HTML5TreeBuilder.h:
     101        (WebCore::HTML5TreeBuilder::setPaused):
     102        (WebCore::HTML5TreeBuilder::isPaused):
     103
    11042010-05-26  Adam Barth  <abarth@webkit.org>
    2105

  • trunk/WebCore/GNUmakefile.am

    r60270 r60275  
    10901090        WebCore/html/HTML5TreeBuilder.cpp \
    10911091        WebCore/html/HTML5TreeBuilder.h \
     1092        WebCore/html/HTML5ScriptRunner.cpp \
     1093        WebCore/html/HTML5ScriptRunner.h \
    10921094        WebCore/html/HTMLAllCollection.cpp \
    10931095        WebCore/html/HTMLAllCollection.h \

  • trunk/WebCore/WebCore.gypi

    r60241 r60275  
    14751475            'html/HTML5TreeBuilder.cpp',
    14761476            'html/HTML5TreeBuilder.h',
     1477            'html/HTML5ScriptRunner.cpp',
     1478            'html/HTML5ScriptRunner.h',
    14771479            'html/HTMLAllCollection.cpp',
    14781480            'html/HTMLAllCollection.h',

  • trunk/WebCore/WebCore.pro

    r60240 r60275  
    612612    html/HTML5Tokenizer.cpp \
    613613    html/HTML5TreeBuilder.cpp \
     614    html/HTML5ScriptRunner.cpp \
    614615    html/HTMLAllCollection.cpp \
    615616    html/HTMLAnchorElement.cpp \

  • trunk/WebCore/WebCore.vcproj/WebCore.vcproj

    r60241 r60275  
    3095830958                        </File>
    3095930959                        <File
     30960                                RelativePath="..\html\HTML5ScriptRunner.cpp"
     30961                                >
     30962                        </File>
     30963                        <File
     30964                                RelativePath="..\html\HTML5ScriptRunner.h"
     30965                                >
     30966                        </File>
     30967                        <File
    3096030968                                RelativePath="..\html\HTMLAllCollection.cpp"
    3096130969                                >

  • trunk/WebCore/WebCore.xcodeproj/project.pbxproj

    r60240 r60275  
    29172917                A833C80D0A2CF25600D57664 /* XMLNames.h in Headers */ = {isa = PBXBuildFile; fileRef = A833C80B0A2CF25600D57664 /* XMLNames.h */; };
    29182918                A833C8520A2CF52800D57664 /* SVGElementFactory.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 656581E609D1508D000E61D7 /* SVGElementFactory.cpp */; };
     2919                A83B3AF011ADEFF500458809 /* HTML5ScriptRunner.cpp in Sources */ = {isa = PBXBuildFile; fileRef = A83B3AEE11ADEFF500458809 /* HTML5ScriptRunner.cpp */; };
     2920                A83B3AF111ADEFF500458809 /* HTML5ScriptRunner.h in Headers */ = {isa = PBXBuildFile; fileRef = A83B3AEF11ADEFF500458809 /* HTML5ScriptRunner.h */; };
    29192921                A83B78FC0CCAFF15000B0825 /* JSSVGFontFaceUriElement.h in Headers */ = {isa = PBXBuildFile; fileRef = A83B78F20CCAFF15000B0825 /* JSSVGFontFaceUriElement.h */; };
    29202922                A83B78FD0CCAFF15000B0825 /* JSSVGFontFaceUriElement.cpp in Sources */ = {isa = PBXBuildFile; fileRef = A83B78F30CCAFF15000B0825 /* JSSVGFontFaceUriElement.cpp */; };
    … …  
    83588360                A833C80A0A2CF25600D57664 /* XMLNames.cpp */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.cpp.cpp; path = XMLNames.cpp; sourceTree = "<group>"; };
    83598361                A833C80B0A2CF25600D57664 /* XMLNames.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = XMLNames.h; sourceTree = "<group>"; };
     8362                A83B3AEE11ADEFF500458809 /* HTML5ScriptRunner.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HTML5ScriptRunner.cpp; sourceTree = "<group>"; };
     8363                A83B3AEF11ADEFF500458809 /* HTML5ScriptRunner.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HTML5ScriptRunner.h; sourceTree = "<group>"; };
    83608364                A83B78F20CCAFF15000B0825 /* JSSVGFontFaceUriElement.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSSVGFontFaceUriElement.h; sourceTree = "<group>"; };
    83618365                A83B78F30CCAFF15000B0825 /* JSSVGFontFaceUriElement.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSSVGFontFaceUriElement.cpp; sourceTree = "<group>"; };
    … …  
    1332713331                        isa = PBXGroup;
    1332813332                        children = (
    13329                                 49484FAE102CF01E00187DD3 /* canvas */,
    1333013333                                2EAFAF0B10E2AF2D007ED3D6 /* Blob.cpp */,
    1333113334                                2EAFAF0C10E2AF2D007ED3D6 /* Blob.h */,
    1333213335                                2EAFAF0D10E2AF2D007ED3D6 /* Blob.idl */,
     13336                                49484FAE102CF01E00187DD3 /* canvas */,
    1333313337                                93C441ED0F813A1A00C1A634 /* CollectionCache.cpp */,
    1333413338                                93C441EE0F813A1A00C1A634 /* CollectionCache.h */,
    … …  
    1337113375                                97E8B35411A23CE200169409 /* HTML5Lexer.cpp */,
    1337213376                                97E8B35511A23CE200169409 /* HTML5Lexer.h */,
     13377                                A83B3AEE11ADEFF500458809 /* HTML5ScriptRunner.cpp */,
     13378                                A83B3AEF11ADEFF500458809 /* HTML5ScriptRunner.h */,
    1337313379                                97E8B3C211A2890800169409 /* HTML5Token.h */,
    1337413380                                A871038811A2947000DBD50E /* HTML5Tokenizer.cpp */,
    … …  
    1936419370                                B6693EEE11AD63E6003F2770 /* IDBObjectStoreRequest.h in Headers */,
    1936519371                                B6693EF311AD6486003F2770 /* JSIDBObjectStoreRequest.h in Headers */,
     19372                                A83B3AF111ADEFF500458809 /* HTML5ScriptRunner.h in Headers */,
    1936619373                        );
    1936719374                        runOnlyForDeploymentPostprocessing = 0;
    … …  
    2165821665                                B6693EF211AD6486003F2770 /* JSIDBObjectStoreRequest.cpp in Sources */,
    2165921666                                B627FB2D11AD743500E97B72 /* IDBDatabaseImpl.cpp in Sources */,
     21667                                A83B3AF011ADEFF500458809 /* HTML5ScriptRunner.cpp in Sources */,
    2166021668                        );
    2166121669                        runOnlyForDeploymentPostprocessing = 0;

  • trunk/WebCore/bindings/js/CachedScriptSourceProvider.h

    r57738 r60275  
    3030#include "CachedResourceHandle.h"
    3131#include "CachedScript.h"
     32#include "JSDOMBinding.h" // for stringToUString
    3233#include "ScriptSourceProvider.h"
    3334#include <parser/SourceCode.h>

  • trunk/WebCore/dom/ScriptElement.cpp

    r59778 r60275  
    9898    if (sourceUrl.isEmpty() && data.scriptContent().isEmpty())
    9999        data.setCreatedByParser(false);
    100     // HTML5 Requires that we execute scripts from the parser, not from
    101     // HTMLTokenizer like we currently do.
    102     // FIXME: It may not be safe to execute scripts from here if
    103     // HTMLParser::popOneBlockCommon is not reentrant.
    104     else if (useHTML5Parser(data.element()->document())) {
    105         // This is currently an incomplete implementation, see:
    106         // http://www.whatwg.org/specs/web-apps/current-work/multipage/tokenization.html#parsing-main-incdata
    107         data.evaluateScript(ScriptSourceCode(data.scriptContent(), data.element()->document()->url())); // FIXME: Provide a real starting line number here
    108     }
    109100}
    110101

  • trunk/WebCore/dom/ScriptElement.h

    r55414 r60275  
    9595    Element* m_element;
    9696    CachedResourceHandle<CachedScript> m_cachedScript;
    97     bool m_createdByParser;
     97    bool m_createdByParser; // HTML5: "parser-inserted"
    9898    bool m_requested;
    99     bool m_evaluated;
     99    bool m_evaluated; // HTML5: "already started"
    100100    bool m_firedLoad;
    101101};

  • trunk/WebCore/html/HTML5ScriptRunner.h

    r60274 r60275  
    2424 */
    2525
    26 #ifndef HTML5Tokenizer_h
    27 #define HTML5Tokenizer_h
     26#ifndef HTML5ScriptRunner_h
     27#define HTML5ScriptRunner_h
    2828
    2929#include "CachedResourceClient.h"
    30 #include "HTML5Token.h"
    31 #include "SegmentedString.h"
    32 #include "Tokenizer.h"
    33 #include <wtf/OwnPtr.h>
     30#include "CachedResourceHandle.h"
     31#include <wtf/Noncopyable.h>
     32#include <wtf/PassRefPtr.h>
    3433
    3534namespace WebCore {
    3635
    37 class HTMLDocument;
    38 class HTML5TreeBuilder;
    39 class HTML5Lexer;
     36class CachedResourceClient;
     37class CachedScript;
     38class Document;
     39class Element;
     40class Frame;
     41class ScriptSourceCode;
    4042
    41 // FIXME: This is the wrong layer to hook in the new HTML 5 Lexer,
    42 // however HTMLTokenizer is too large and too fragile of a class to hack into.
    43 // Eventually we should split all of the HTML lexer logic out from HTMLTokenizer
    44 // and then share non-lexer-specific tokenizer logic between HTML5 and the
    45 // legacy WebKit HTML lexer.
     43class HTML5ScriptRunner : public Noncopyable {
     44public:
     45    HTML5ScriptRunner(Document*, CachedResourceClient*);
     46    ~HTML5ScriptRunner();
    4647
    47 // FIXME: This class is far from complete.
    48 class HTML5Tokenizer :  public Tokenizer, public CachedResourceClient {
    49 public:
    50     HTML5Tokenizer(HTMLDocument*, bool reportErrors);
    51     virtual ~HTML5Tokenizer();
    52 
    53     virtual void begin();
    54     virtual void write(const SegmentedString&, bool appendData);
    55     virtual void end();
    56     virtual void finish();
    57     virtual bool isWaitingForScripts() const;
     48    // Processes the passed in script and any pending scripts if possible.
     49    bool execute(PassRefPtr<Element> scriptToProcess);
     50    // Processes any pending scripts.
     51    bool executeScriptsWaitingForLoad(CachedResource*);
    5852
    5953private:
    60     void pumpLexer();
     54    struct PendingScript {
     55        RefPtr<Element> element;
     56        CachedResourceHandle<CachedScript> cachedScript;
     57        // HTML5 has an isReady parameter, however isReady ends up equivalent to
     58        // m_document->haveStylesheetsLoaded() && cachedScript->isLoaded()
     59    };
    6160
    62     SegmentedString m_source;
     61    Frame* frame() const;
    6362
    64     // We hold m_token here because it might be partially complete.
    65     HTML5Token m_token;
     63    bool executeParsingBlockingScripts();
     64    void executePendingScript();
    6665
    67     OwnPtr<HTML5Lexer> m_lexer;
    68     OwnPtr<HTML5TreeBuilder> m_treeBuilder;
     66    void requestScript(Element*);
     67    void runScript(Element*);
     68
     69    bool isPendingScriptReady(const PendingScript&);
     70    ScriptSourceCode sourceFromPendingScript(const PendingScript&, bool& errorOccurred);
     71
     72    Document* m_document;
     73    CachedResourceClient* m_loadNotifier;
     74    PendingScript m_parsingBlockingScript;
     75    unsigned m_scriptNestingLevel;
    6976};
    7077

  • trunk/WebCore/html/HTML5Tokenizer.cpp

    r60257 r60275  
    2727#include "HTML5Tokenizer.h"
    2828
     29#include "Element.h"
    2930#include "HTML5Lexer.h"
     31#include "HTML5ScriptRunner.h"
     32#include "HTML5Token.h"
    3033#include "HTML5TreeBuilder.h"
     34#include "HTMLDocument.h"
    3135#include "Node.h"
    3236#include "NotImplemented.h"
    … …  
    3741    : Tokenizer()
    3842    , m_lexer(new HTML5Lexer)
     43    , m_scriptRunner(new HTML5ScriptRunner(document, this))
    3944    , m_treeBuilder(new HTML5TreeBuilder(m_lexer.get(), document, reportErrors))
    4045{
    … …  
    5257void HTML5Tokenizer::pumpLexer()
    5358{
     59    ASSERT(!m_treeBuilder->isPaused());
    5460    while (m_lexer->nextToken(m_source, m_token)) {
    5561        m_treeBuilder->constructTreeFromToken(m_token);
    5662        m_token.clear();
     63
     64        if (!m_treeBuilder->isPaused())
     65            continue;
     66
     67        // The parser will pause itself when waiting on a script to load or run.
     68        // ScriptRunner executes scripts at the right times and handles reentrancy.
     69        bool shouldContinueParsing = m_scriptRunner->execute(m_treeBuilder->takeScriptToProcess());
     70        if (!shouldContinueParsing) {
     71            // ASSERT(m_source.isEmpty() || m_treeBuilder->isPaused());
     72            // FIXME: the script runner should either make this call or return a special
     73            // value to indicate we should pause.
     74            m_treeBuilder->setPaused(true);
     75            return;
     76        }
    5777    }
    5878}
    … …  
    6080void HTML5Tokenizer::write(const SegmentedString& source, bool)
    6181{
     82    // FIXME: This does not yet correctly handle reentrant writes.
    6283    m_source.append(source);
    63     pumpLexer();
     84    if (!m_treeBuilder->isPaused())
     85        pumpLexer();
    6486}
    6587
    6688void HTML5Tokenizer::end()
    6789{
    68     m_source.close();
    69     pumpLexer();
     90    if (!m_treeBuilder->isPaused())
     91        pumpLexer();
    7092    m_treeBuilder->finished();
    7193}
    … …  
    7395void HTML5Tokenizer::finish()
    7496{
    75     end();
     97    // finish() indicates we will not receive any more data. If we are waiting on
     98    // an external script to load, we can't finish parsing quite yet.
     99    m_source.close();
     100    if (!m_treeBuilder->isPaused())
     101        end();
    76102}
    77103
    78104bool HTML5Tokenizer::isWaitingForScripts() const
    79105{
    80     notImplemented();
    81     return false;
     106    return m_treeBuilder->isPaused();
     107}
     108
     109void HTML5Tokenizer::resumeParsingAfterScriptExecution()
     110{
     111    ASSERT(!m_treeBuilder->isPaused());
     112    // FIXME: This is the wrong write in the case of document.write re-entry.
     113    pumpLexer();
     114    if (m_source.isEmpty() && m_source.isClosed())
     115        end(); // The document already finished parsing we were just waiting on scripts when finished() was called.
     116}
     117
     118void HTML5Tokenizer::notifyFinished(CachedResource* cachedResource)
     119{
     120    bool shouldContinueParsing = m_scriptRunner->executeScriptsWaitingForLoad(cachedResource);
     121    if (shouldContinueParsing) {
     122        m_treeBuilder->setPaused(false);
     123        resumeParsingAfterScriptExecution();
     124    }
     125}
     126
     127void HTML5Tokenizer::executeScriptsWaitingForStylesheets()
     128{
     129    // FIXME: We can't block for stylesheets yet, because that causes us to re-enter
     130    // the parser from executeScriptsWaitingForStylesheets when parsing style tags.
    82131}
    83132

  • trunk/WebCore/html/HTML5Tokenizer.h

    r60257 r60275  
    3535namespace WebCore {
    3636
     37class HTML5Lexer;
     38class HTML5ScriptRunner;
     39class HTML5TreeBuilder;
    3740class HTMLDocument;
    38 class HTML5TreeBuilder;
    39 class HTML5Lexer;
    4041
    41 // FIXME: This is the wrong layer to hook in the new HTML 5 Lexer,
    42 // however HTMLTokenizer is too large and too fragile of a class to hack into.
    43 // Eventually we should split all of the HTML lexer logic out from HTMLTokenizer
    44 // and then share non-lexer-specific tokenizer logic between HTML5 and the
    45 // legacy WebKit HTML lexer.
    46 
    47 // FIXME: This class is far from complete.
    48 class HTML5Tokenizer :  public Tokenizer, public CachedResourceClient {
     42// FIXME: The whole Tokenizer class system should be renamed "Parser"
     43// or "ParserController" as the job of this class is to drive parsing process
     44// but it does not itself Tokenize.
     45class HTML5Tokenizer :  public Tokenizer, CachedResourceClient {
    4946public:
    5047    HTML5Tokenizer(HTMLDocument*, bool reportErrors);
    … …  
    5653    virtual void finish();
    5754    virtual bool isWaitingForScripts() const;
     55    virtual void executeScriptsWaitingForStylesheets();
     56
     57    // CachedResourceClient
     58    virtual void notifyFinished(CachedResource*);
    5859
    5960private:
    6061    void pumpLexer();
     62    void resumeParsingAfterScriptExecution();
    6163
    6264    SegmentedString m_source;
    … …  
    6668
    6769    OwnPtr<HTML5Lexer> m_lexer;
     70    OwnPtr<HTML5ScriptRunner> m_scriptRunner;
    6871    OwnPtr<HTML5TreeBuilder> m_treeBuilder;
    6972};

  • trunk/WebCore/html/HTML5TreeBuilder.cpp

    r60137 r60275  
    2727#include "HTML5TreeBuilder.h"
    2828
    29 #include "Attribute.h"
     29#include "Element.h"
    3030#include "HTML5Lexer.h"
    3131#include "HTML5Token.h"
    … …  
    4444    : m_document(document)
    4545    , m_reportErrors(reportErrors)
     46    , m_isPaused(false)
    4647    , m_lexer(lexer)
    4748    , m_legacyHTMLParser(new HTMLParser(document, reportErrors))
    … …  
    9495}
    9596
     97void HTML5TreeBuilder::handleScriptStartTag()
     98{
     99    notImplemented(); // The HTML frgment case?
     100    m_lexer->setState(HTML5Lexer::ScriptDataState);
     101    notImplemented(); // Save insertion mode.
     102}
     103
     104void HTML5TreeBuilder::handleScriptEndTag(Element* scriptElement)
     105{
     106    ASSERT(!m_scriptToProcess); // Caller never called takeScriptToProcess!
     107    notImplemented(); // Save insertion mode and insertion point?
     108
     109    // Pause ourselves so that parsing stops until the script can be processed by the caller.
     110    m_isPaused = true;
     111    m_scriptToProcess = scriptElement;
     112}
     113
     114PassRefPtr<Element> HTML5TreeBuilder::takeScriptToProcess()
     115{
     116    // Unpause ourselves, callers may pause us again when processing the script.
     117    // The HTML5 spec is written as though scripts are executed inside the tree
     118    // builder.  We pause the parser to exit the tree builder, and then resume
     119    // before running scripts.
     120    m_isPaused = false;
     121    return m_scriptToProcess.release();
     122}
     123
    96124PassRefPtr<Node> HTML5TreeBuilder::passTokenToLegacyParser(HTML5Token& token)
    97125{
     126    if (token.type() == HTML5Token::DOCTYPE) {
     127        DoctypeToken doctypeToken;
     128        doctypeToken.m_name.append(token.name().characters(), token.name().length());
     129        doctypeToken.m_publicID = token.publicIdentifier();
     130        doctypeToken.m_systemID = token.systemIdentifier();
     131
     132        m_legacyHTMLParser->parseDoctypeToken(&doctypeToken);
     133        return 0;
     134    }
     135
     136    // For now, we translate into an old-style token for testing.
     137    Token oldStyleToken;
     138    convertToOldStyle(token, oldStyleToken);
     139
     140    RefPtr<Node> result =  m_legacyHTMLParser->parseToken(&oldStyleToken);
    98141    if (token.type() == HTML5Token::StartTag) {
    99142        // This work is supposed to be done by the parser, but
    100143        // when using the old parser for we have to do this manually.
    101         if (token.name() == scriptTag)
    102             m_lexer->setState(HTML5Lexer::ScriptDataState);
    103         else if (token.name() == textareaTag || token.name() == titleTag)
     144        if (token.name() == scriptTag) {
     145            handleScriptStartTag();
     146            m_lastScriptElement = static_pointer_cast<Element>(result);
     147        } else if (token.name() == textareaTag || token.name() == titleTag)
    104148            m_lexer->setState(HTML5Lexer::RCDATAState);
    105149        else if (token.name() == styleTag || token.name() == iframeTag
    … …  
    110154            m_lexer->setState(HTML5Lexer::PLAINTEXTState);
    111155    }
    112 
    113     if (token.type() == HTML5Token::DOCTYPE) {
    114         DoctypeToken doctypeToken;
    115         doctypeToken.m_name.append(token.name().characters(), token.name().length());
    116         doctypeToken.m_publicID = token.publicIdentifier();
    117         doctypeToken.m_systemID = token.systemIdentifier();
    118 
    119         m_legacyHTMLParser->parseDoctypeToken(&doctypeToken);
    120         return 0;
    121     }
    122 
    123     // For now, we translate into an old-style token for testing.
    124     Token oldStyleToken;
    125     convertToOldStyle(token, oldStyleToken);
    126 
    127     return m_legacyHTMLParser->parseToken(&oldStyleToken);
     156    if (token.type() == HTML5Token::EndTag) {
     157        if (token.name() == scriptTag) {
     158            if (m_lastScriptElement) {
     159                handleScriptEndTag(m_lastScriptElement.get());
     160                m_lastScriptElement = 0;
     161            }
     162        }
     163    }
     164    return result.release();
    128165}
    129166

  • trunk/WebCore/html/HTML5TreeBuilder.h

    r60095 r60275  
    3030#include <wtf/OwnPtr.h>
    3131#include <wtf/PassRefPtr.h>
     32#include <wtf/RefPtr.h>
    3233#include <wtf/unicode/Unicode.h>
    3334
    3435namespace WebCore {
    3536class Document;
     37class Element;
     38class Frame;
    3639class HTML5Lexer;
    3740class HTML5Token;
    … …  
    4548    ~HTML5TreeBuilder();
    4649
     50    void setPaused(bool paused) { m_isPaused = paused; }
     51    bool isPaused() { return m_isPaused; }
     52
    4753    // The token really should be passed as a const& since it's never modified.
    4854    PassRefPtr<Node> constructTreeFromToken(HTML5Token&);
     55    // Must be called when parser is paused before calling the parser again.
     56    PassRefPtr<Element> takeScriptToProcess();
     57
     58    // Done, close any open tags, etc.
    4959    void finished();
    5060
    … …  
    5262    PassRefPtr<Node> passTokenToLegacyParser(HTML5Token&);
    5363    PassRefPtr<Node> processToken(HTML5Token&, UChar currentCharacter = 0);
     64   
     65    void handleScriptStartTag();
     66    void handleScriptEndTag(Element*);
    5467
    55     // We could grab m_document off the lexer if we wanted to save space.
    56     Document* m_document;
     68    Document* m_document; // This is only used by the m_legacyParser for now.
    5769    bool m_reportErrors;
     70    bool m_isPaused;
    5871    // HTML5 spec requires that we be able to change the state of the lexer
    5972    // from within parser actions.
    … …  
    6275    // We're re-using logic from the old HTMLParser while this class is being written.
    6376    OwnPtr<HTMLParser> m_legacyHTMLParser;
     77    RefPtr<Element> m_lastScriptElement; // FIXME: This is a hack for <script> support.
     78    RefPtr<Element> m_scriptToProcess; // Set to a <script> tag which needs processing.
    6479};
    6580
Note: See TracChangeset for help on using the changeset viewer.