Disclaimer

This is my scratch-pad for thoughts about QtWebKit for Qt 5. After spending plenty of time talking to different people, I'm slowly exceeding the capacity of my brain and it is time to digest, summarize and write down some content :)

Please feel free to edit right away or add comments with prefix between the paragraphs ("JoeDoe: I think this is all complete rubbish.")

QtWebKit for Qt 5

We have learned a lot about WebKit2 in the past months, as well as mobile web development. It is time to bring this experience to the main line of the Qt port of WebKit:

• QtWebKit for Qt 5 should be the foundation for great mobile web browsers.

Design

Qt5 will be an interesting foundation for QtWebKit. The changes in Qt 5 that affect QtWebKit the most are the move of QWidget and related classes into a separate library as well as the introduction of Qt Quick 2 and the Qt Scene Graph as the primary graphics/UI API.

Consequently the interface to the browser should change:

• Our focus should be on making the WebKit2 port fast and stable enough for a browser.
• A QML based UI component offers built-in handling for gestures, touch events and scrollbars.
• We should not spend time designing APIs that allow for feature combinations or software fallbacks that don't make sense. Features like WebGL or video should either work out of the box properly accelerated or be disabled.
• The QWidget/QGraphicsView based WebKit1 API moves into a separate shared library that links against QtWebKit.

Implementation Details

Many different approaches to different feature have been discussed. We can record them here as a starting point for the implementation work:

WebKit2, Scene Graph, Tiling and Accelerated Compositing

We need to determine the best way to integrate tiling / AC with the Qt scene graph. An initial pragmatic solution would be to port the tiling to the scene graph directly and render the AC layers into an intermediate FBO.

Another option would be to separate scene graph rendering from WebKit by rendering WebKit2 with raw OpenGL through the Texture Mapper first and then the entire QML scene on top.

The approach requiring probably the most effort is to essentially re-implement WebKit2 UI-process sided AC on top of the scene graph, i.e. let the web process serialize the layer & animation information to the UI process where the layer tree is replicated in terms of scene graph geometry nodes and texture materials that are subject to animation.

Library split

Ideally the main QtWebKit library as well as the QtWebProcess won't link against the Qt widgets library. We should investigate the option of moving all QWidget dependencies (like the old plugin code) out of WebCore and into a secondary QtWebKitWidgets library that interfaces with WebCore through a platform plugin alike mechanism and hooks up top to implement QWebView/QGraphicsWebView. In such a design, WebCore would paint primarily with the raster paint engine and load all images in QImage objects instead of QPixmaps.

Another current dependency of WebCore is QStyle, which could be replaced with the default mobile theme and a few remaining QPainter calls for what we use QWindowsStyle currently. The themeing for QWebView/QGraphicsWebView could be delegated into a plugin loaded by WebCore when widgets and QStyle is needed.

---

Jocelyn: This is going to be painful. Beside the memory overhead, is there worthy benefits of decoupling the QWidget dependency? If possible I wish we could live with it until the need is felt.

WebGL

For an efficient WebGL implementation we could either

a) Use OpenGL directly in the web process, render into an FBO and implement cross-process texture sharing for AC

or

b) Serialize the OpenGL commands and execute them on the UI process side

Some research is needed here, but option a) would have the additional benefit of also simplifying the video implementation due to the required texture sharing.

Video

In order to render embedded videos efficiently and provide seamless transitions to fullscreen playback, we require from the platform the ability to render video into textures. For fullscreen playback we should piggy-back on the fullscreen element feature in WebCore/WebKit2 that integrates with AC. The video texture becomes the root layer and the controls live in their own layer on top.

JavaScript

QtWebKit for Qt 5 uses V8 as the underlying JS engine. The implementation of QObject bindings needed for WebKit1 as well as potentially for WebKit2's WebProcess should be implemented in either in the QtScript library (based on V8) or another library in Qt that provides equivalent functionality.

---

Jocelyn: This is long term stuff, but since we want to have the main interface as QML from a different process than the page's JS runtime, I think we should take the time to re-evaluate the benefits of exposing the engine using the QtScript API. This was a dream we had before WebKit2 was introduced. Chrome's extention are allowed to manipulate pages content using ​content scripts and can communicate with the extension's script using ​messages. Doing the same kind of stuff is one way I see how to prevent the developer from having to tediously load a C++ module in the web process.

Simon: I agree, C++ modules are tedious compared to script injection. A direct export-QObject-pointer-to-JS API may not make sense anymore, but for compatibility I think we'll have to continue to offer that in the WebKit1 API library. That could still be done through V8 and QtScript-QObject bindings.

Discussion

noamr: I think that this is a good direction. A big question yet to be answered is how much hybrid support we need going forward with WebKit2. If we enable QML with WebKit2, do we enable the javaScriptWindowObjects property? or do we go with something a bit different like allowing communication with the container via channel messaging?

simon: I like the idea of a channel messaging as an API, especially when on the QML side the messages could be received in "onMessageReceived:" alike signal handlers and posted via QML's JavaScript.

romaxa: WebGL option a) will require some double buffering -> additional tx->tx copy, also creation +1 GL context on WebProcess side, sounds like serialization is better way to go