Changeset 71393 in webkit

Timestamp:

Nov 4, 2010 10:08:02 PM (13 years ago)

Author:

tonikitoo@webkit.org

Message:

2010-11-03 Antonio Gomes <​tonikitoo@webkit.org>, Andre Pedralho <​andre.pedralho@gmail.com>

Reviewed by Kenneth Rohde Christiansen and Antti Koivisto.

[Qt] Making effective use of nodesFromRect.
​https://bugs.webkit.org/show_bug.cgi?id=44089

Patch adds a helper class (named TouchAdjuster) to improve tap actions
on mobile touch devices. TouchAdjuster makes use of the newly added rect
based hit test extension through the Document::nodesFromRect method.
Given a rectangle as input, nodesFromRect returns a z-index ordered list
of nodes whose boundaries intersect the rectangle.

Basically the TouchAdjuster class intercepts the QGraphicsSceneMouseEvent
passed to both QWebPagePrivate::mouse{Press,Release}Event methods before
they are passed down to WebCore. The goal is to infer the target click position.
For that, a rectangle is built up using the event position as a center point and
expanding it based on the values and directions set in the Platform Plugin
QWebTouchModifier::hitTestPadding.

TouchAdjuster iterates over the list of nodes returned by nodesFromRect and
picks the clickable one that has the largest intersection area with the hit
test rectangle. The target position is then the center point of this intersection
area.

In case of no clickable node intersects the hit test area, the click position
is not altered.

TouchAdjuster is *only* working for QGraphicsWebView based views.

• Api/qwebpage.cpp: (QWebPagePrivate::mousePressEvent): (QWebPagePrivate::mouseReleaseEvent): (QWebPagePrivate::adjustPointForClicking): (QWebPagePrivate::TouchAdjuster::TouchAdjuster): (QWebPagePrivate::TouchAdjuster::findCandidatePointForTouch): (isClickableElement): (hasMouseListener): (isValidFrameOwner): (nodeToElement):
• Api/qwebpage_p.h:

Location:

trunk/WebKit/qt

Files:

• Api/qwebpage.cpp (modified) (9 diffs)
• Api/qwebpage_p.h (modified) (3 diffs)
• ChangeLog (modified) (1 diff)

trunk/WebKit/qt/Api/qwebpage.cpp

@@ -32 +32 @@
 #include "qwebinspector_p.h"
 #include "qwebsettings.h"
+#include "qwebkitplatformplugin.h"
 #include "qwebkitversion.h"
 
+#include "CSSComputedStyleDeclaration.h"
+#include "CSSParser.h"
 #include "ApplicationCacheStorage.h"
 #include "BackForwardListImpl.h"
@@ -39 +42 @@
 #include "Chrome.h"
 #include "ChromeClientQt.h"
+#include "ClientRect.h"
 #include "ContextMenu.h"
 #include "ContextMenuClientQt.h"
@@ -61 +65 @@
 #include "GeolocationPermissionClientQt.h"
 #include "HTMLFormElement.h"
+#include "HTMLFrameOwnerElement.h"
 #include "HTMLInputElement.h"
 #include "HTMLNames.h"
@@ -75 +80 @@
 #include "NavigationAction.h"
 #include "NetworkingContext.h"
+#include "NodeList.h"
 #include "NotificationPresenterClientQt.h"
+#include "NotImplemented.h"
 #include "Page.h"
 #include "PageClientQt.h"
@@ -87 +94 @@
 #include "PluginPackage.h"
 #include "ProgressTracker.h"
+#include "QtPlatformPlugin.h"
 #include "RefPtr.h"
 #include "RenderTextControl.h"
@@ -658 +666 @@
 
     bool accepted = false;
+    adjustPointForClicking(ev);
     PlatformMouseEvent mev(ev, 1);
     // ignore the event if we can't map Qt's mouse buttons to WebCore::MouseButton
@@ -737 +746 @@
 
     bool accepted = false;
+    adjustPointForClicking(ev);
     PlatformMouseEvent mev(ev, 0);
     // ignore the event if we can't map Qt's mouse buttons to WebCore::MouseButton
@@ -1262 +1272 @@
 
     return frame->eventHandler()->scrollRecursively(direction, granularity);
+}
+
+void QWebPagePrivate::adjustPointForClicking(QMouseEvent*)
+{
+    notImplemented();
+}
+
+void QWebPagePrivate::adjustPointForClicking(QGraphicsSceneMouseEvent* ev)
+{
+    QtPlatformPlugin platformPlugin;
+    QWebTouchModifier* touchModifier = platformPlugin.createTouchModifier();
+    if (!touchModifier)
+        return;
+
+    unsigned topPadding = touchModifier->hitTestPaddingForTouch(QWebTouchModifier::Up);
+    unsigned rightPadding = touchModifier->hitTestPaddingForTouch(QWebTouchModifier::Right);
+    unsigned bottomPadding = touchModifier->hitTestPaddingForTouch(QWebTouchModifier::Down);
+    unsigned leftPadding = touchModifier->hitTestPaddingForTouch(QWebTouchModifier::Left);
+
+    delete touchModifier;
+    touchModifier = 0;
+
+    if (!topPadding && !rightPadding && !bottomPadding && !leftPadding)
+        return;
+
+    Document* startingDocument = page->mainFrame()->document();
+    if (!startingDocument)
+        return;
+
+    IntPoint originalPoint(QPointF(ev->pos()).toPoint());
+    TouchAdjuster touchAdjuster(topPadding, rightPadding, bottomPadding, leftPadding);
+    IntPoint adjustedPoint = touchAdjuster.findCandidatePointForTouch(originalPoint, startingDocument);
+    if (adjustedPoint == IntPoint::zero())
+        return;
+
+    ev->setPos(QPointF(adjustedPoint));
 }
 
@@ -1428 +1474 @@
 #endif
     return 0;
+}
+
+static bool hasMouseListener(Element* element)
+{
+    ASSERT(element);
+    return element->hasEventListeners(eventNames().clickEvent)
+        || element->hasEventListeners(eventNames().mousedownEvent)
+        || element->hasEventListeners(eventNames().mouseupEvent);
+}
+
+static bool isClickableElement(Element* element, RefPtr<NodeList> list)
+{
+    ASSERT(element);
+    bool isClickable = hasMouseListener(element);
+    if (!isClickable && list) {
+        Element* parent = element->parentElement();
+        unsigned count = list->length();
+        for (unsigned i = 0; i < count && parent; i++) {
+            if (list->item(i) != parent)
+                continue;
+
+            isClickable = hasMouseListener(parent);
+            if (isClickable)
+                break;
+
+            parent = parent->parentElement();
+        }
+    }
+
+    ExceptionCode ec = 0;
+    return isClickable
+        || element->webkitMatchesSelector("a,*:link,*:visited,*[role=button],button,input,select,label", ec)
+        || computedStyle(element)->getPropertyValue(cssPropertyID("cursor")) == "pointer";
+}
+
+static bool isValidFrameOwner(Element* element)
+{
+    ASSERT(element);
+    return element->isFrameOwnerElement() && static_cast<HTMLFrameOwnerElement*>(element)->contentFrame();
+}
+
+static Element* nodeToElement(Node* node)
+{
+    if (node && node->isElementNode())
+        return static_cast<Element*>(node);
+    return 0;
+}
+
+QWebPagePrivate::TouchAdjuster::TouchAdjuster(unsigned topPadding, unsigned rightPadding, unsigned bottomPadding, unsigned leftPadding)
+    : m_topPadding(topPadding)
+    , m_rightPadding(rightPadding)
+    , m_bottomPadding(bottomPadding)
+    , m_leftPadding(leftPadding)
+{
+}
+
+IntPoint QWebPagePrivate::TouchAdjuster::findCandidatePointForTouch(const IntPoint& touchPoint, Document* document) const
+{
+    if (!document)
+        return IntPoint();
+
+    int x = touchPoint.x();
+    int y = touchPoint.y();
+
+    RefPtr<NodeList> intersectedNodes = document->nodesFromRect(x, y, m_topPadding, m_rightPadding, m_bottomPadding, m_leftPadding, false);
+    if (!intersectedNodes)
+        return IntPoint();
+
+    Element* closestClickableElement = 0;
+    IntRect largestIntersectionRect;
+    FrameView* view = document->frame()->view();
+
+    // Touch rect in contents coordinates.
+    IntRect touchRect(HitTestResult::rectForPoint(view->windowToContents(IntPoint(x, y)), m_topPadding, m_rightPadding, m_bottomPadding, m_leftPadding));
+
+    // Iterate over the list of nodes hit looking for the one whose bounding area
+    // has largest intersection with the touch area (point + padding).
+    for (unsigned i = 0; i < intersectedNodes->length(); i++) {
+        Node* currentNode = intersectedNodes->item(i);
+
+        Element* currentElement = nodeToElement(currentNode);
+        if (!currentElement || (!isClickableElement(currentElement, 0) && !isValidFrameOwner(currentElement)))
+            continue;
+
+        IntRect currentElementBoundingRect = currentElement->getRect();
+        currentElementBoundingRect.intersect(touchRect);
+
+        if (currentElementBoundingRect.isEmpty())
+            continue;
+
+        int currentIntersectionRectArea = currentElementBoundingRect.width() * currentElementBoundingRect.height();
+        int largestIntersectionRectArea = largestIntersectionRect.width() * largestIntersectionRect.height();
+        if (currentIntersectionRectArea > largestIntersectionRectArea) {
+            closestClickableElement = currentElement;
+            largestIntersectionRect = currentElementBoundingRect;
+        }
+    }
+
+    if (largestIntersectionRect.isEmpty())
+        return IntPoint();
+
+    // Handle the case when user taps a inner frame. It is done in three steps:
+    // 1) Transform the original touch point to the inner document coordinates;
+    // 1) Call nodesFromRect for the inner document in case;
+    // 3) Re-add the inner frame offset (location) before passing the new clicking
+    //    position to WebCore.
+    if (closestClickableElement->isFrameOwnerElement()) {
+        // Adjust client coordinates' origin to be top left of inner frame viewport.
+        PassRefPtr<ClientRect> rect = closestClickableElement->getBoundingClientRect();
+        IntPoint newTouchPoint = touchPoint;
+        IntSize offset =  IntSize(rect->left(), rect->top());
+        newTouchPoint -= offset;
+
+        HTMLFrameOwnerElement* owner = static_cast<HTMLFrameOwnerElement*>(closestClickableElement);
+        Document* childDocument = owner->contentFrame()->document();
+        return findCandidatePointForTouch(newTouchPoint, childDocument);
+    }
+    return view->contentsToWindow(largestIntersectionRect).center();
 }
 

trunk/WebKit/qt/Api/qwebpage_p.h

@@ -32 +32 @@
 #include "qwebframe.h"
 
+#include "IntPoint.h"
 #include "KURL.h"
 #include "PlatformString.h"
@@ -44 +45 @@
     class ContextMenuItem;
     class ContextMenu;
+    class Document;
     class EditorClientQt;
     class Element;
     class InspectorController;
+    class IntRect;
     class Node;
+    class NodeList;
     class Page;
     class Frame;
@@ -127 +131 @@
     bool touchEvent(QTouchEvent*);
 
+    class TouchAdjuster {
+    public:
+        TouchAdjuster(unsigned topPadding, unsigned rightPadding, unsigned bottomPadding, unsigned leftPadding);
+
+        WebCore::IntPoint findCandidatePointForTouch(const WebCore::IntPoint&, WebCore::Document*) const;
+
+    private:
+        unsigned m_topPadding;
+        unsigned m_rightPadding;
+        unsigned m_bottomPadding;
+        unsigned m_leftPadding;
+    };
+
+    void adjustPointForClicking(QMouseEvent*);
+    void adjustPointForClicking(QGraphicsSceneMouseEvent*);
+
     void setInspector(QWebInspector*);
     QWebInspector* getOrCreateInspector();

trunk/WebKit/qt/ChangeLog

@@ +1 @@
+2010-11-03  Antonio Gomes  <tonikitoo@webkit.org>, Andre Pedralho  <andre.pedralho@gmail.com>
+
+        Reviewed by Kenneth Rohde Christiansen and Antti Koivisto.
+
+        [Qt] Making effective use of nodesFromRect.
+        https://bugs.webkit.org/show_bug.cgi?id=44089
+
+        Patch adds a helper class (named TouchAdjuster) to improve tap actions
+        on mobile touch devices. TouchAdjuster makes use of the newly added rect
+        based hit test extension through the Document::nodesFromRect method.
+        Given a rectangle as input, nodesFromRect returns a z-index ordered list
+        of nodes whose boundaries intersect the rectangle.
+
+        Basically the TouchAdjuster class intercepts the QGraphicsSceneMouseEvent
+        passed to both QWebPagePrivate::mouse{Press,Release}Event methods before
+        they are passed down to WebCore. The goal is to infer the target click position.
+        For that, a rectangle is built up using the event position as a center point and
+        expanding it based on the values and directions set in the Platform Plugin
+        QWebTouchModifier::hitTestPadding.
+
+        TouchAdjuster iterates over the list of nodes returned by nodesFromRect and
+        picks the clickable one that has the largest intersection area with the hit
+        test rectangle. The target position is then the center point of this intersection
+        area.
+
+        In case of no clickable node intersects the hit test area, the click position
+        is not altered.
+
+        TouchAdjuster is *only* working for QGraphicsWebView based views.
+
+        * Api/qwebpage.cpp:
+        (QWebPagePrivate::mousePressEvent):
+        (QWebPagePrivate::mouseReleaseEvent):
+        (QWebPagePrivate::adjustPointForClicking):
+        (QWebPagePrivate::TouchAdjuster::TouchAdjuster):
+        (QWebPagePrivate::TouchAdjuster::findCandidatePointForTouch):
+        (isClickableElement):
+        (hasMouseListener):
+        (isValidFrameOwner):
+        (nodeToElement):
+        * Api/qwebpage_p.h:
+
 2010-10-20  Chang Shu  <chang.shu@nokia.com>, Antonio Gomes <tonikitoo@webkit.org>