Changes between Initial Version and Version 1 of NextGenerationLayoutAndRendering

Oct 13, 2017 2:04:19 PM (5 years ago)
Jon Davis



  • NextGenerationLayoutAndRendering

    v1 v1  
     1== Next Generation Layout and Rendering ==
     2''by Simon Fraser''
     4* 2-3 years!
     5* Current layout and rendering:
     6    * Render tree exists, FrameView owns it
     7    * Hasn’t changed since KHTML
     8    * Layout is recursive across the render tree, then painting is recursive across the render tree, then hit testing is also recursive across the render tree
     9    * There is only one render tree! Used for many things
     10    * RenderLayers are used as entry points for painting and hit testing (and also handle things like opacity & clipping, scrolling, etc.)
     11    * Internally, the trees might have (effectively) different structures depending on their use. For example, the z-order tree is only used for some purpose
     12    * Bad things
     13        * Mutating the tree at a bad time causes lots of bugs
     14        * Current render tree is not read-only in any sense
     15        * The code for grid, flexbox, block, etc. all live in the same set of objects. This leads to dependency hell.
     16        * The render tree has inline trees as its leaves. The data in the inline tree is sometimes duplicated in the inline tree. This makes it difficult to think about
     17        * Repaint is difficult to reason about (because it involves mutability, but there are no immutability constraints)
     18        * No parallelism
     19        * Painting involves full paint phases that may do nothing, and involves running through the entire render tree (with some culling)
     20        * Current code often has structure of “if is block do block thing, else do inline thing”
     21* Goals of this project
     22    * We want more guarantees about what can change when. This probably involves sprinkling “const” in many places
     23    * Hackability - naming should match specs, and classes should be smaller and better scoped
     24    * Parallelism: With more strict mutability comes more likelihood of asynch work
     25        * Painting may be asynchronous (e.g. with display lists)
     26* One tree is used for layout, layerization, and painting!
     27* Instead, we want:
     28    * “Layout” tree, created the tree builder
     29    * Layout creates a “box tree” where all nodes are boxes. There is no distinction between block boxes and inline boxes
     30    * We probably still need a layerization step that works the same as it does today
     31        * Or not, maybe layerization would produce a “presentation tree”
     32    * Then, we will produce a display list thing which is used for painting
     33* Tree building! (Step 1)
     34    * RenderTreeBuilder currently does it.
     35        * This should do any mutation that currently occurs during (later) layout. We’ve been working on this
     36        * RenderTreeBuilder will also do things like anonymous box generation
     37        * The logic for this will move out of the data objects and into controller objects (like RenderTreeBuilder)
     38    * Layout! (Step 2)
     39        * Produces box tree, which includes geometry
     40        * Inline tree would be invisible, because the logic would move into iterators.
     41        * “Formatting contexts” will be the controllers for sub pieces of this.
     42            * e.g. BlockFormattingContext, InlineFormattingContext, GridFormattingContext etc.
     43            * These match what the spec language describes
     44            * Interruptible, resumable
     45                * These handle layout, so they don’t have to operate recursively. They can use whichever they use
     46                    * Nested formatting contexts still need to be recursive
     47                * Each one can be processed in parallel
     48        * Box tree: Output form Layout
     49            * Mostly geometry. Doesn’t care about formatting contexts.
     50            * Used for painting and hit testing
     51            * Immutable
     52        * Immutability is great because you can do “throwaway” layouts (not real layouts) for things like JS sync layouts
     53            * This lets you do animations where the destination is “auto”. Your throwaway layout computes the value of “auto”
     54            * Or do it off the main thread
     55                * And paint while layout
     56    * Doing these phases asynchronously decreases latency (and uses more cores)
     57    * Presentation Tree (Step 3)
     58        * These are like RenderLayers that reference into the box tree
     59        * This will implement opacity, transforms, filters, and stacking contexts / z-order tree
     60        * Clipping will be easier to implement because you will have a new tree just for it!
     61        * Don’t have to traverse the render tree for each painting phase
     62        * All this crap is currently implemented by RenderLayer. So we want to get rid of a lot of the functionality of RenderLayer and move it elsewhere
     63            * Remove a bunch of things: scrolling, Marquee, fragmentation, compositing?
     64    * Painting (Step 4)
     65        * Currently, we paint in this order:
     66            * Layers -> Box tree -> GraphicsContext -> Platform drawing API
     67        * But, if we had display lists, we do this:
     68            * Layers -> Box tree -> GraphicsContext -> Display List
     69                * Then, later, Display List -> GraphicsContext -> platform drawing API
     70        * We could do the same thing, but we get rid of GraphicsContext!
     71            * Layers -> Box tree -> Display List
     72                * Then, later, Display List -> Graphics Context -> platform drawing API
     73                * OR! Display List -> platform drawing API
     74    * Display Lists (Step 5)
     75        * Just a list of serialized drawing commands
     76        * Retained between paints! So you don’t have to rebuild it every frame
     77        * The combination of retained-mode display lists, and extent information, lets you optimize repainting by culling a bunch of crap that was painted last time
     78        * Display lists can be cached!
     79        * You can optimize the list! Eliminate redundant state changes, etc.
     80        * Downsides
     81            * They take memory, but probably less memory than compositing layer bitmaps
     82            * There is an extra phase for display lists: recording & repainting, so latency is increased
     83            * Recording and replaying can’t happen in parallel
     84* So how do we do this?
     85    * Incrementally!
     86        * Remove mutability from the render tree during layout
     87        * Hide the difference between simple line layout and real layout behind iterators
     88        * We can make formatting context objects and move code into them (this is just code moving)
     89        * Display lists (because we have an implementation)
     90        * Breaking up RenderLayer into its constituent pieces
     91    * Non-incrementally
     92        * Generational layout and box trees. So we have multiple render trees in memory at once (which is okay)
     93        * Using a box tree
     94        * Flip how repaint works to using display lists
     95            * Repaint just updates a few bytes in the display list and then plays the display list again