wiki:QtWebKitForQt5

Version 13 (modified by Simon Hausmann, 12 years ago) (diff)

--

Disclaimer

This is my scratch-pad for thoughts about QtWebKit for Qt 5. After spending plenty of time talking to different people, I'm slowly exceeding the capacity of my brain and it is time to digest, summarize and write down some content :)

Please feel free to edit right away or add comments with prefix between the paragraphs ("JoeDoe: I think this is all complete rubbish.")

QtWebKit for Qt 5

We have learned a lot about WebKit2 in the past months, as well as mobile web development. It is time to bring this experience to the main line of the Qt port of WebKit:

  • QtWebKit for Qt 5 should be the foundation for great mobile web browsers.

Design

Qt5 will be an interesting foundation for QtWebKit. The changes in Qt 5 that affect QtWebKit the most are the move of QWidget and related classes into a separate library as well as the introduction of Qt Quick 2 and the Qt Scene Graph as the primary graphics/UI API.

Consequently the interface to the browser should change:

  • Our focus should be on making the WebKit2 port fast and stable enough for a web browser.
  • A QML based UI component offers built-in handling for gestures, touch events (and scrollbars?).
  • We should not spend time designing APIs that allow for feature combinations or software fallbacks that don't make sense. Features like WebGL or video should either work out of the box properly accelerated or be disabled.
  • The QWidget/QGraphicsView based WebKit1 API moves into a separate shared library that links against QtWebKit.

Implementation Details

Many different approaches to different feature have been discussed. We can record them here as a starting point for the implementation work:

WebKit2, Scene Graph, Tiling and Accelerated Compositing

The implementation currently chosen is the following:

  • We're permanently in the state of composition.
  • Layers are tiled and the tiles are "managed" in the web process side.
  • Layer information is serialized and sent across to the UI process (LayerTreeHost and WebGraphicsLayer)
  • The UI process feeds the layers into the texture mapper (LayerTreeProxyHost)
  • The OpenGL texture mapper implementation renders before/after the scene graph.

Library split

Ideally the main QtWebKit library as well as the QtWebProcess won't link against the Qt widgets library. We should investigate the option of moving all QWidget dependencies (like the old plugin code) out of WebCore and into a secondary QtWebKitWidgets library that interfaces with WebCore through a platform plugin alike mechanism and hooks up top to implement QWebView/QGraphicsWebView. In such a design, WebCore would paint primarily with the raster paint engine and load all images in QImage objects instead of QPixmaps.

Another current dependency of WebCore is QStyle, which could be replaced with the default mobile theme and a few remaining QPainter calls for what we use QWindowsStyle currently. The themeing for QWebView/QGraphicsWebView could be delegated into a plugin loaded by WebCore when widgets and QStyle is needed.


Jocelyn: This is going to be painful. Beside the memory overhead, is there worthy benefits of decoupling the QWidget dependency? If possible I wish we could live with it until the need is felt.

WebGL

For an efficient WebGL implementation we could either

a) Use OpenGL directly in the web process, render into an FBO and implement cross-process texture sharing for AC

or

b) Serialize the OpenGL commands and execute them on the UI process side

Some research is needed here, but option a) would have the additional benefit of also simplifying the video implementation due to the required texture sharing.

Video

In order to render embedded videos efficiently and provide seamless transitions to fullscreen playback, we require from the platform the ability to render video into textures. Implementation bits that appear to be necessary:

  • For each platform, determine the way of sharing textures across process boundaries (for example X11: X pixmaps)
  • Add support for platform specific cross-process texture sharing to the WK2 AC implementation and the texture mapper.
  • Change the corresponding media player back-end to render into such texture-backed layers.
  • Implement support on the UI process side for establishing the UI for the temporary root layer for fullscreen elements. (maybe a dedicated QWindow subclass that renders straight with a dedicated GL texturemapper instance that small tree of texture mapper nodes that represent the layers of the fullscreen element hierarchy)

JavaScript

QtWebKit for Qt 5 uses V8 as the underlying JS engine. The implementation of QObject bindings needed for WebKit1 as well as potentially for WebKit2's WebProcess should be implemented in either in the QtScript library (based on V8) or another library in Qt that provides equivalent functionality.


Jocelyn: This is long term stuff, but since we want to have the main interface as QML from a different process than the page's JS runtime, I think we should take the time to re-evaluate the benefits of exposing the engine using the QtScript API. This was a dream we had before WebKit2 was introduced. Chrome's extention are allowed to manipulate pages content using content scripts and can communicate with the extension's script using messages. Doing the same kind of stuff is one way I see how to prevent the developer from having to tediously load a C++ module in the web process.

Simon: I agree, C++ modules are tedious compared to script injection. A direct export-QObject-pointer-to-JS API may not make sense anymore, but for compatibility I think we'll have to continue to offer that in the WebKit1 API library. That could still be done through V8 and QtScript-QObject bindings.

Discussion

noamr: I think that this is a good direction. A big question yet to be answered is how much hybrid support we need going forward with WebKit2. If we enable QML with WebKit2, do we enable the javaScriptWindowObjects property? or do we go with something a bit different like allowing communication with the container via channel messaging?

simon: I like the idea of a channel messaging as an API, especially when on the QML side the messages could be received in "onMessageReceived:" alike signal handlers and posted via QML's JavaScript.

romaxa: WebGL option a) will require some double buffering -> additional tx->tx copy, also creation +1 GL context on WebProcess side, sounds like serialization is better way to go