This page documents the current status, code overview of plugins on Qt/WebKit.

Overview

Qt/WebKit supports loading of two kinds of plugins.

• Qt based plugins - These can be QWidget or QGraphicsWidget based.
• Netscape plugins (npapi)

To enable loading of plugins, you must enable QWebSettings::PluginsEnabled.

Qt Based plugins

<object> are encapsulated in HTMLPlugInElement. FrameLoaderClient has a method createPlugin that returns the Widget that should be used as the plugin. FrameLoaderClientQt.cpp inspects the mimetype. With a mime type of application/x-qt-plugin or application/x-qt-styled-widget, the frameloader creates a QWidget using the QWebPluginFactory (QWebPluginFactory::create()). This QWidget is encapsulate in a QtPluginWidget (which is basically a WebCore Widget). The QWidget is created as a child of the QWebPageClient::pluginParent() (more information on this below).

Netscape plugins

The rest of this document focuses on Netscape plugins. Feel free to substitute 'plugins' with 'flash' when reading this document (since it's the main plugin we test with).

Hacking

The NPAPI does not have very clear documentation but the following links will help you started

• ​https://developer.mozilla.org/en/Plugins
• ​https://developer.mozilla.org/en/Gecko_Plugin_API_Reference

Netscape plugins operate in two modes:

• Windowed mode - The plugin has a native window handle of it's own. This means that the plugin has complete control over mouse, keyboard, paint events. It can do whatever it wants, whenever it wants.
• Windowless mode - The plugin does not have a native window. Instead it relies on the browser to tell it where to paint and expects the browser to provide it with keyboard/mouse events. See ​mozilla site for more information.

Flash has a wmode parameter that triggers the above modes. The default is Windowed mode. When wmode is 'transparent' or 'opaque', flash requests windowless mode. Note that the mode used ultimately resides with the plugin (i.e the browser cannot force a mode).

WebKit Plugins

Most of the code resides under WebCore/plugins. The Qt specific stuff under WebCore/plugins/qt. Here's a high level overview of the classes:

• PluginsQuirkSet.h - Plugins are buggy. WebKit has special code to workaround various bugs/features of plugins. You will find these 'quirks' listed here.
• PluginPackage.cpp - Represents the plugin library file (i.e .so or .dll). Loads the plugin, determines the mimetype and sets appropriate quirks
• PluginView.cpp - Represents an instance of the plugin. It is a Widget. When loading a page, this widget pointer is inside a RenderWidget.
• PluginStream.cpp - The plugin and browser communicate with each other using NPStreams. For example, the source file (src) is pushed to the plugin by the browser using a PluginStream.
• PluginDatabase.cpp - Takes care of locating the plugin library file. Provides PluginPackage(s).

QWebPageClient - We require access to the QWidget (QGraphicsView/QWebView) or the parent (QGraphicsWebView/QWebView). To provide this and keep the plugin code working in both cases, the chrome (hostwindow) provides a platformPageClient() that helps us get information about the client of the QWebPage. Both QWebView and QGraphicsWebView provide objects that implement the virtuals in QWebPageClient. QWebPageClient::ownerWidget() return the QWidget in which the page is displayed (i.e QWebView or the QGraphicsView). QWebPageClient::pluginParent() returns the parent QObject (i.e QWebView or QGraphicsWebView). The ownerWidget() is necessary for the popup code. The pluginParent is necessary to position the plugins.

Platform Specific

X11

​Diamondx is a good start to understand how netscape plugins are written. You can clone the source from ​git://git.forwardbias.in/diamondx.git.

PluginViewQt.cpp contains Qt specific view code for plugins.

Windowed mode

Windowed mode is implemented using XEmbed (other technique that uses Xt is not implemented). The spec is ​here.

Qt port uses QX11EmbedContainer to host the plugin. PluginView creates PluginContainerQt (which is a QX11EmbedContainer).

This mode works well in QWebView. It works in QGraphicsWebView as long as there is no transformation on the QGraphicsView or the item. To make it work with transformations, we are trying out an approach based on XComposite/XDamage (like in fennec)

Windowless mode

Windowless mode is implemented by drawing to a X Drawable. Conceptually, it's very simple, however mixing toolkits and flash bugs make it quite complex.

• Flash uses gdk. Gdk uses a different X connection from Qt. This means that we have to flush the gdk display periodically. If you find your scrolling jittery, this is the case of that. In the ideal work, flash is supposed to use the Display provided using the npapi, but it doesn't.
• Flash cannot paint on argb32 pixmaps since it does not use the visual provided using the npapi (it uses gdk_default_system_visual instead). This makes it very hard to implement transparency. For QWebView, we grab contents from the backing store to implement transparency. For all other cases, transparency is turned off (except when the default system visual is 32-bit).

Transformation, opacity of item QGraphicsWebView in QGraphicsView is supported. Windowless mode is not supported when using QWebView inside QGraphicsView. We will need XComposite/XDamage support for that.

If you are wondering why something is done a certain way, ​20081 has lots of history.

Testing

Depending on your WebKit codebase knowledge, hacking on plugins inside WebKit can be a case of having too many unknowns. We have developed npploader, a simple Qt program that can load Flash in a QWidget. You can get it by cloning from ​git://git.forwardbias.in/npploader.git.

Usage:

• Works only in X11 (as of now)
• Modify PLUGIN_* in main.cpp to the correct paths.
• By default, loads in windowed mode. Pass '-windowless' to command line for windowless mode.
• The pixmap32 branch implements windowless mode painting of plugins into a 32-bit pixmap (i.e if flash was fixed). You can test it with the diamondx's pixmap32 branch (​http://git.forwardbias.in/?p=diamondx.git).
• The flashtransparecyhack branch overrides the gdk_default_visual to make it return a 32-bit visual. Flash can now paint to 32-bit pixmaps. Note that this approach was not considered for WebKit because of -Bsymbolic-functions.

Windows

TODO

Symbian

The Symbian port of NPAPI is in ​http://trac.webkit.org/browser/trunk/WebCore/plugins/symbian .

In Symbian, dlls can be loaded only from \sys\bin, so Symbian requires a different mechanism for loading Netscape plugins. We use QPluginLoader to find and load plugins. In order for WebKit to find Netscape plugins, they must follow these rules:

• Add template=lib and config+=plugin to the pro file.
• Implement the interface NPInterface as defined in ​http://trac.webkit.org/browser/trunk/WebCore/plugins/symbian/npinterface.h .
• Export the .qtplugin placeholder to \resource\qt\plugins\npqtplugin.

PluginPackageSymbian.cpp - Uses QPluginLoader to load and unload the plugin. It also queries the plugin for its mime types, extensions and description.

PluginDatabaseSymbian.cpp - Searches for available .qtplugin placeholders in the available drives in the system.

PluginContainerSymbian.cpp - A QWidget that hosts a windowed plugin. This class should eventually merge with PluginContainerQt.cpp.

PluginViewSymbian.cpp - Does the bulk of the work interacting with the plugin. It creates and destroys the plugin, resizes the plugin and in windowless mode, sends events to the plugin.

Painting and events in windowless mode:

Painting: Plugins should draw directly to the surface of the webview. Prior to each QPaintEvent, a plugin will receive a NPN_SetWindow call, that contains the NPWindow structure. The ws_info member of NPWindow will contain a pointer to the QPainter to use for painting.

Keyboard events: When the plugin has keyboard focus, triggered e.g. by the user tapping on the plugin, WebKit will pass keyboard events as QKeyEvent to the plugin. Mouse events: When WebKit receives mouse events in the plugin's area, it will pass QMouseEvent to the plugin.

A common error is to combine NPInterface with the plugin QWidget. Please note that NPInterface represents the plugin dll. It is created and deleted when the plugin dll is loaded and unloaded, not with newp. So the same class should not be used for both NPInterface and the plugin itself.

Mac

The Qt port uses the NPAPI implementation in WebCore/plugins/mac on Mac OS X.

The idea behind WebCore/plugins/mac is that it should be usable by any port, so the port-specific bits have been put into helpers with initial implementation for PLATFORM(QT).

I've put up a brain-dump of my notes here.

TODO List

• Share code among all ports. Currently, there is a lot of code duplicaton amongst symbian, Qt ports. Gtk is still to come and will presumably require very similar code.
• X11 - Implement XComposite/XDamage in QX11EmbedContainer. Status: work in progress.
• X11 - Context menus not working in windowless mode. Status: no solution in sight.
• X11 - Windowed mode doesn't work when the same page is switched between views. In general, we need a clean way for the plugin to track changes in ownerWidget().

Testing

Use the ​communityx site for testing various modes.

Open bugs

Use this ​filter.

-- Girish (Girish Ramakrishnan)