Changes between Initial Version and Version 1 of Accelerated rendering and compositing


Ignore:
Timestamp:
Apr 19, 2012 11:18:42 AM (10 years ago)
Author:
jonlee@apple.com
Comment:

Initial draft

Legend:

Unmodified
Added
Removed
Modified
  • Accelerated rendering and compositing

    v1 v1  
     1Outline:
     2
     3* Tree mania:
     4** RenderObject
     5** RenderLayer
     6** z-order tree
     7** Graphicslayer tree
     8** PlatformLayer tree
     9* Special considerations:
     10** overlap
     11** clipping
     12** reflections
     13** video/webgl/plugins/etc.
     14** intersection
     15* Accelerated drawing and compositing
     16
     17Notes:
     18
     19Notion that identify elements in the web page that benefit from rendering to bitmap and using gpu to composite it with other elements in the page.
     20
     21This is done through RenderLayer tree.
     22
     23RenderLayer started as convenience for rendering certain objects. RenderLayer per element with its own coordinate system: overflow, scroll. transforms, and now video.
     24
     25RenderTree with heirarchy of objs. And certain objs have renderlayer off of them. There is a renderlayer tree that mirrors render object tree.
     26
     27paint root rl, then go through child rls, use rl for clipping and overflow. hit test through rls. reverse walk through child rls for hit testing.
     28
     29another hierarchy of rls which is important, z-order tree. css props z-index, to move front-to-back ordering of pages. to support, z-order tree.  before paint, compute z-order tree by going through rl tree, find stacking contexts.  painting and hit-test use this z-order tree, not the rl tree.
     30
     31compositing: z-order tree gives back-to-front ordering. use it to generate info about which was in front of what. in compositing, we can id rls that want to be composited, and create a backing store for.
     32
     33if we make something composited, we render to backing store, then give it to teh gpu. rendered through opengl, probably. on top of regular content on page.
     34
     35ensure that any elements in front of composited layers, are also composited. RenderLyaerCompositor, per renderview. manages hierarchy of layers and decides what needs to be a layer.
     36
     37rl backing for each composited layer. rl backing creates 1+ gfx layers, glue between rendering and platform-compositing code.
     38
     39each rl backing can create another gfx layer. clipping/masks/-z indexes. slice into multiple planes and create layers for clipping.
     40
     41gfx layers are in their own tree. used by platform code to create platform layers (like corenimation).
     42
     43gfx layers can have more than one platform layer: masks and reflections.
     44
     45gfx layers have a way of dumping its tree, can get it from gdb.
     46
     47
     48test for overlap. RLCompositor list of rects of things that have been composited. Makes other layers composited if they intersect that list of rects.
     49
     50clipping: div with overflow:hidden, child with transform or video. impl clipping via platform layer. on mac: calayer add masks to apply. if a layer is composited through clipping, it doesn't have its own backing store, paints into something behind it.
     51
     52reflections: cloning layers, tracking clones. tricky.
     53
     54special layers, because of content that benefits from composites. video, rendervideo gets renderlayer, becomes composited. communication between mac media player and comp code so that media player hand pl to comp code, and pl into compositing layer tree.
     55
     56layer intersections: allows 3d transforms, to paint into a gl texture. If layers intersect (child through parent), break into subtextures along intersections.
     57
     58hit test is done in software in RL code. hit testing should be same for all paltforms.
     59
     60
     61
     62
     63overlap map is updated when there is a layotu or style change that changes compositing layer tree.
     64
     65special casing css animations. which transform animation is running, we don't updating clipping during animations. update at start and end, not during. don't change layout during. don't update scrollbars, for example. it's a cheat, but gives performance. opacity animation also.
     66
     67overlap test is used if real 3d xforms are not present in page.
     68
     69
     70platforms have their own accel drawing impls. on mac, but compositing still has value when there is accel drawing, but depends on content. feedback from paltform code to tell RLCompositor when its valuable to do compositing or not.
     71
     72
     73- plugins like flash force compositing.
     74
     75
     76
     77
     78
     79
     80how to debug
     81
     82- in safari, "show compositing borders"
     83
     84every composited layer gets a border.
     85the # represents # of times the layer has been repainted
     86yellow, not green boxes have no css properties, so use just for geometry, and doesn't need a backing store.
     87red needs a backing store.
     88
     89do you see popping? something popping in front is probably composited.
     90
     91blue outline: composited because it has overflow:hidden/scroll on it.
     92
     93- in gdb, debug only "call showLayerTree" any renderlayer tree. shows it as a z-order tree. prints whether rl is composited. says why something is a renderlayer. which GLs are associated with which RLs.
     94
     95- in inspector, turn on/off css properties. overflow, z-order, transforms.  gmail bug case: command-c the dom into a fresh document, to give you a snapshot. although there was no css, the bug still repro'd.
     96- in inspector, start deleting nodes.
     97
     98
     99
     100text: on mac there is a limitation on gfx sys where rendering text on transparent background, can't do subpixel antialiasing. no good soln to this problem right now.  if something is composited, but with opaque bg-color, then it's fine.
     101
     102
     103limitations on compositing: not smart when applying scale trasnforms. on mac, (scale:2) just blow up bitmap, looks fuzzy.
     104
     105ms support: ie10 has transforms and animations. doing stuff with direct2d when dealing with scaling.
     106
     107
     108if we redid wk, would it based on compositing primitives instead of bitmap primitives? this lives under the platform layer. only ms has done this by throwing everything out and rebuilding with direct2d.
     109
     110webcore lacks logic where knowing if elements up front obscure stuff behind it. right now it just repaints everything behind it from back-to-front.
     111
     112svg compositing? doesn't use rl at all. split up rl, piece related to compositing needs to be something svg could use.