Disclaimer

This is my scratch-pad for thoughts about QtWebKit for Qt 5. After spending plenty of time talking to different people, I'm slowly exceeding the capacity of my brain and it is time to digest, summarize and write down some content :)

Please feel free to edit right away or add comments with prefix between the paragraphs ("JoeDoe: I think this is all complete rubbish.")

QtWebKit for Qt 5

We have learned a lot about WebKit2 in the past months, as well as mobile web development. With QtWebKit for Qt 5 it is time to bring this experience to the main line:

• QtWebKit for Qt 5 should make it easy to write great mobile web browsers.
• QtWebKit for Qt 5 provides convenient APIs for creating Qt applications that embed and closely integrate with web content.

Design

Qt5 will be an interesting foundation for QtWebKit. The changes in Qt 5 that affect QtWebKit the most are the move of QWidget and related classes into a separate library as well as the introduction of Qt Quick 2 and the Qt Scene Graph as the primary graphics/UI API.

Consequently the QtWebKit API should change:

• The primary interface to the web engine should be through WebKit2.
• A QML based UI component offers built-in handling for gestures, touch events and scrollbars (viewport).
• Features like accelerated compositing, WebGL, inline video, etc. should work out of the box, without extra settings or configuration flags.
• A minimal C++ API allows for fine-tuning certain parameters and potentially low-level hooks (for example controlling the web process creation)
• The QWidget/QGraphicsView based WebKit1 API moves into a separate shared library that links against QtWebKit.

Implementation Details

Let's look a bit closer into possible solutions to implement the proposed design:

WebKit2, Scene Graph, Tiling and Accelerated Compositing

The QML WebView should support tiling and AC out of the box and AC should not be anymore an afterthought "on top of tiling" but we should be in a permanent state of accelerated compositing. Layers should support tiling and the root tile set should just be a layer, to avoid expensive switching between AC and non-AC mode.

We need to determine the best way to integrate tiling / AC with the Qt scene graph. An initial pragmatic solution would be to render the tiles and AC layers into an FBO first.

Another option would be to separate scene graph rendering from WebKit by rendering WebKit2 with raw OpenGL through the Texture Mapper first and then the entire QML scene on top.

The approach requiring probably the most effort is to essentially re-implement WebKit2 UI-process sided AC on top of the scene graph, i.e. let the web process serialize the layer & animation information to the UI process where the layer tree is replicated in terms of scene graph geometry nodes and texture materials that are subject to animation.

Library split

Ideally the main QtWebKit library as well as the QtWebProcess won't link against the Qt widgets library. We should investigate the option of moving all QWidget dependencies (like the old plugin code) out of WebCore and into a secondary QtWebKitWidgets library that interfaces with WebCore through a platform plugin alike mechanism and hooks up top to implement QWebView/QGraphicsWebView. In such a design, WebCore would paint primarily with the raster paint engine and load all images in QImage objects instead of QPixmaps.

WebGL

For an efficient WebGL implementation we could either

1) Use OpenGL directly in the web process, render into an FBO and implement cross-process texture sharing for AC

or

2) Serialize the OpenGL commands and execute them on the UI process side

Some research is needed here, but option 1) would have the additional benefit of also simplifying the video implementation due to the texture sharing.

Video

In order to render embedded videos efficiently and provide seamless transitions to fullscreen playback, we require from the platform the ability to render video into textures. For fullscreen playback we should piggy-back on the fullscreen element feature in WebCore/WebKit2 that integrates with AC. The video texture becomes the root layer and the controls live in their own layer on top.