Changes between Version 13 and Version 14 of CoordinatedGraphicsSystem


Ignore:
Timestamp:
Aug 4, 2012 5:13:13 AM (9 years ago)
Author:
kenneth@webkit.org
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • CoordinatedGraphicsSystem

    v13 v14  
    33This wiki page describes the coordinated graphics architecture currently used by Qt in WebKit2, parts of which are also used by other ports.
    44
     5[[BR]]
    56== Background ==
    67A good reading before reading this is the following from Chromium:
     
    1011Without understanding of the difference, the rest of this document might not be very useful.
    1112
    12 ----
     13[[BR]]
    1314
    1415Other blog posts that can help:
     
    2627  [http://labs.qt.nokia.com/2011/11/10/qtwebkit-accelerated-compositing-report/]
    2728
    28 
     29[[BR]]
    2930== Overview ==
    3031
     
    3233
    3334The main components described in this document:
    34 [#TextureMapper TextureMapper] - a light-weight scenegraph implementation tuned specifically for WebKit accelerated compositing. Used by Qt, EFL and GTK+.
    35 
    36 [#CoordinatedCompositing Coordinated Compositing] - an architecture for WebKit2 that allows using accelerated compositing in a dual process environment. EFL is, at the time of writing this document, in the process of switching to use coordinated compositing.
    37 
    38 [#CoordinatedBackingStore Coordinated Backing Store] - How the compositing environment deals with software-rendered buffers or WebGL buffers, e.g. from the point of view of memory and pixel-formats. This includes several components, some are used only together with coordinated compositing, some also used elsewhere.
    39 
    40 [#QtScenegraph QtScenegraph integration] - How the WebKit way of rendering is glued with the QtQuick scenegraph way of rendering. This is naturally used by the Qt port only.
    41 
    42 [#CSSAnimations CSS Animation Support]
    43 
    44 [#WebGL WebGL Support]
    45 
     35
     36  [#TextureMapper TextureMapper] - a light-weight scenegraph implementation tuned specifically for WebKit accelerated compositing. Used by Qt, EFL and GTK+.
     37
     38  [#CoordinatedCompositing Coordinated Compositing] - an architecture for WebKit2 that allows using accelerated compositing in a dual process environment. EFL is, at the time of writing this document, in the process of switching to use coordinated compositing.
     39
     40  [#CoordinatedBackingStore Coordinated Backing Store] - How the compositing environment deals with software-rendered buffers or WebGL buffers, e.g. from the point of view of memory and pixel-formats. This includes several components, some are used only together with coordinated compositing, some also used elsewhere.
     41
     42  [#QtScenegraph QtScenegraph integration] - How the WebKit way of rendering is glued with the QtQuick scenegraph way of rendering. This is naturally used by the Qt port only.
     43
     44  [#CSSAnimations CSS Animation Support]
     45
     46  [#WebGL WebGL Support]
     47
     48[[BR]]
    4649The following diagram shows the different components and classes, and how they all work together. more explanations would follow.
    4750
     51[[BR]]
    4852{{{
    4953#!html
     
    5155}}}
    5256
    53 
     57[[BR]]
    5458== [#=TextureMapper Texture Mapper] ==
    5559TextureMapper is a light-weight scenegraph implementation that is specially attuned for efficient GPU or software rendering of CSS3 composited content, such as 3D transforms, animations and canvas.
     
    7377'''[/browser/trunk/Source/WebCore/platform/graphics/texmap/TextureMapperGL.cpp TextureMapperGL]''' is the GPU-accelerated implementation of the drawing primitives. It is currently used by Qt-WebKit2, GTK+ and EFL(?). TextureMapperGL uses shaders compatible with GL ES 2.0, though we’re in the process of converting it to use the WebGL infrastructure directly, which would help with shader compatibility.
    7478
    75 == Comments ==
     79=== Comments ===
    7680
    7781TextureMapperGL tries to use scissors for clipping whenever it can, though if the clipping is not rectangular stencil is used.
     
    114118* Make it better and faster :)
    115119
     120[[BR]]
    116121== [#=CoordinatedCompositing Coordinated Compositing] ==
    117122Coordinated compositing, guarded by USE(UI_SIDE_COMPOSITING), is a WebKit2 implementation of accelerated compositing. It synchronizes the layer tree provided by WebCore in the web process with a proxied layer tree in the UI process that is responsible for the actual GPU rendering to the screen.
     
    149154* Serializing and coordinating CSS shaders, once they're implemented in TextureMapper.
    150155
     156[[BR]]
    151157== [#=CoordinatedBackingStore Coordinated Backing Store] ==
    152158Unlike coordinated compositing, which includes mainly light-weight rendering information about each layer, backing-stores contain pixel data, and thus are both memory-hungry and are expensive to copy and serialize. Backing-stores are drawing into by software or by hardware, depending on the scenario, and thus require binding to platform/OS/GPU-specific code.
     
    181187* Allow different pixel formats for tiles, e.g. 16-bit for opaque tiles and 8-bit for masks.
    182188
     189[[BR]]
    183190== [#=QtScenegraph QtScenegraph integration] ==
    184191To integrate WebKit coordinated graphics with the QtScenegraph, the following main things were necessary:
     
    194201Note that the QtScenegraph solves many things that would need to be solved separately if not using it, such as vsync.
    195202
     203[[BR]]
    196204== [#=CSSAnimations CSS Animation Support] ==
    197205Currently, in WebKit2, CSS animation frames are handled together with all other frames – the web process re-layouts for each frame, and sends the new transform/opacity/filter information together with the rest of the frame. The requestAnimationFrame feature, which allows synchronizing animations with the display refresh, is synchronized with LayerTreeCoordinator to make sure those animation frames are interpolated as accurately and smoothly as possible, avoiding choppiness and throttling.
     
    201209This is a somewhat tricky thing to achieve, mainly because animations still need to sometimes sync with the non-animated contents.
    202210
    203 
     211[[BR]]
    204212== [#=WebGL WebGL Support] ==
    205213There are two possible approaches to WebGL. The current approach uses GraphicsSurfaces, allowing the web process to render with GPU into a platform-specific buffer, later compositing it in the UI process. This approach is somewhat easier to implement, but might not be efficient on some GPU systems, and can also create security issues if the web process has direct access to the GPU.
     
    207215The other option is to serialize the WebGL display list to the UI process, making the actual GL calls there, into an FBO. This is a safer and potentially more cross-platform approach, however it's yet to be seen how much of an undertaking it is, and how well it scales.
    208216
    209 
    210 == Comments ==
    211 
    212 Q: Would be great with more infomation about how vsync and triple buffering fits into all of this (http://hardforum.com/showthread.php?t=928593)
     217[[BR]]
     218== Comments and Questions ==
     219
     220'''This section contains comments and questions. Feel free to add more!
     221'''
     222
     223Would be great with more infomation about how vsync and triple buffering fits into all of this (http://hardforum.com/showthread.php?t=928593)
    213224
    214225NR: Vsync/triple-buffering is done in the driver/QtScenegraph level. Added a comment.
     
    216227FOLLOW-UP: How does that work in practice? When the driver vsyncs we draw (or swap buffers => blit) on each sync. If we are not ready (no new back-buffer), we will paint the old front-buffer and ending up with a frame rate of vsync/N, where N is some positive number. Triple buffering fixes this. But if we are just a node in the QtSceneGraph which draws (swap => blit) on vsync, I don't see how we become triple buffered.
    217228
    218 Q: It is not so clear how TextureMapper and TextureMapperLayer etc fits together. Like it sounds as TextureMapper is a scene-graph and also a drawing context. Maybe this could be a bit more clear.
     229It is not so clear how TextureMapper and TextureMapperLayer etc fits together. Like it sounds as TextureMapper is a scene-graph and also a drawing context. Maybe this could be a bit more clear.
    219230
    220231NR: TextureMapper is a drawing context, TextureMapperLayer is a scenegraph. Updated.
    221232
    222 Q: Can UpdateAtlas be used with canvas like in http://dev.dorothybrowser.com/?p=163 ?
     233Can UpdateAtlas be used with canvas like in http://dev.dorothybrowser.com/?p=163 ?
    223234
    224235NR: yes, we can use a texture atlas for accelerated 2D canvas. However, accelerated 2D canvas is not yet supported and not discussed in this document :)
     
    235246
    236247NR: It needs the same things that WebGL needs, e.g. GraphicsSurface or serialized commands. But it's not baked enough to write anything about.
     248
     249How does the Chrome threaded compositor relate to this? Could/should we do something similar?
     250
     251Are we optimizing occluded tiles (those not visible due to some opaque layer fully covering them), ie. not painting them until needed?