⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 285790 in webkit


Ignore:
Timestamp:
Nov 14, 2021, 10:38:45 AM (5 years ago)
Author:
Simon Fraser
Message:

Attach IOHIDEvent timestamps to wheel events
​https://bugs.webkit.org/show_bug.cgi?id=233051

Reviewed by Wenson Hsieh.
Source/WebCore:

On some macOS devices, there can be significant deltas between NSEvent timestamps,
and the timestamps on underlying IOHIDEvents (rdar://85309639). This makes momentum
velocity computation unpredictable; we can get better results by using IOHIDEvent
timestamps.

  • platform/PlatformWheelEvent.cpp:

(WebCore::PlatformWheelEvent::createFromGesture):

  • platform/PlatformWheelEvent.h:

(WebCore::PlatformWheelEvent::ioHIDEventTimestamp const):

  • platform/mac/PlatformEventFactoryMac.h:
  • platform/mac/PlatformEventFactoryMac.mm:

(WebCore::eventTimeStampSince1970):
(WebCore::PlatformMouseEventBuilder::PlatformMouseEventBuilder):
(WebCore::PlatformWheelEventBuilder::PlatformWheelEventBuilder):
(WebCore::PlatformKeyboardEventBuilder::PlatformKeyboardEventBuilder):

Source/WebCore/PAL:

Expose a few bits of SPI needed.

  • pal/spi/cg/CoreGraphicsSPI.h:
  • pal/spi/cocoa/IOKitSPI.h:

Source/WebKit:

On some macOS devices, there can be significant deltas between NSEvent timestamps,
and the timestamps on underlying IOHIDEvents (rdar://85309639). This makes momentum
velocity computation unpredictable; we can get better results by using IOHIDEvent
timestamps.

  • Shared/WebEventConversion.cpp:

(WebKit::WebKit2PlatformWheelEvent::WebKit2PlatformWheelEvent):

  • Shared/WebWheelEvent.cpp:

(WebKit::WebWheelEvent::WebWheelEvent):
(WebKit::WebWheelEvent::encode const):
(WebKit::WebWheelEvent::decode):

  • Shared/WebWheelEvent.h:

(WebKit::WebWheelEvent::ioHIDEventTimestamp const):

  • Shared/WebWheelEventCoalescer.cpp:

(WebKit::WebWheelEventCoalescer::coalesce):

  • Shared/ios/WebIOSEventFactory.mm:

(WebIOSEventFactory::createWebWheelEvent):

  • Shared/mac/NativeWebGestureEventMac.mm:
  • Shared/mac/WebEventFactory.mm:

(WebKit::WebEventFactory::createWebMouseEvent):
(WebKit::WebEventFactory::createWebWheelEvent):
(WebKit::WebEventFactory::createWebKeyboardEvent):

Location:
trunk/Source
Files:
16 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r285787 r285790  
     12021-11-14  Simon Fraser  <simon.fraser@apple.com>
     2
     3        Attach IOHIDEvent timestamps to wheel events
     4        https://bugs.webkit.org/show_bug.cgi?id=233051
     5
     6        Reviewed by Wenson Hsieh.
     7
     8        On some macOS devices, there can be significant deltas between NSEvent timestamps,
     9        and the timestamps on underlying IOHIDEvents (rdar://85309639). This makes momentum
     10        velocity computation unpredictable; we can get better results by using IOHIDEvent
     11        timestamps.
     12
     13        * platform/PlatformWheelEvent.cpp:
     14        (WebCore::PlatformWheelEvent::createFromGesture):
     15        * platform/PlatformWheelEvent.h:
     16        (WebCore::PlatformWheelEvent::ioHIDEventTimestamp const):
     17        * platform/mac/PlatformEventFactoryMac.h:
     18        * platform/mac/PlatformEventFactoryMac.mm:
     19        (WebCore::eventTimeStampSince1970):
     20        (WebCore::PlatformMouseEventBuilder::PlatformMouseEventBuilder):
     21        (WebCore::PlatformWheelEventBuilder::PlatformWheelEventBuilder):
     22        (WebCore::PlatformKeyboardEventBuilder::PlatformKeyboardEventBuilder):
     23
    1242021-11-13  Simon Fraser  <simon.fraser@apple.com>
    225

  • trunk/Source/WebCore/PAL/ChangeLog

    r285786 r285790  
     12021-11-14  Simon Fraser  <simon.fraser@apple.com>
     2
     3        Attach IOHIDEvent timestamps to wheel events
     4        https://bugs.webkit.org/show_bug.cgi?id=233051
     5
     6        Reviewed by Wenson Hsieh.
     7       
     8        Expose a few bits of SPI needed.
     9
     10        * pal/spi/cg/CoreGraphicsSPI.h:
     11        * pal/spi/cocoa/IOKitSPI.h:
     12
    1132021-11-13  Tim Horton  <timothy_horton@apple.com>
    214

  • trunk/Source/WebCore/PAL/pal/spi/cg/CoreGraphicsSPI.h

    r283863 r285790  
    3333#endif
    3434
     35#if PLATFORM(MAC)
     36#include <pal/spi/cocoa/IOKitSPI.h>
     37#endif
     38
    3539#if USE(APPLE_INTERNAL_SDK)
    3640
    … …  
    4347#if PLATFORM(MAC)
    4448#include <CoreGraphics/CGAccessibility.h>
     49#include <CoreGraphics/CGEventPrivate.h>
    4550#endif
    4651
    … …  
    352357CGSDisplayID CGSMainDisplayID(void);
    353358
     359IOHIDEventRef CGEventCopyIOHIDEvent(CGEventRef);
    354360#endif // PLATFORM(MAC)
    355361

  • trunk/Source/WebCore/PAL/pal/spi/mac/IOKitSPIMac.h

    r285786 r285790  
    4444
    4545WTF_EXTERN_C_BEGIN
     46
     47typedef struct __IOHIDEvent * IOHIDEventRef;
    4648typedef struct CF_BRIDGED_TYPE(id) __IOHIDServiceClient * IOHIDServiceClientRef;
    4749typedef struct CF_BRIDGED_TYPE(id) __IOHIDEventSystemClient * IOHIDEventSystemClientRef;
    … …  
    8284typedef uint32_t IOHIDEventType;
    8385
     86uint64_t IOHIDEventGetTimeStamp(IOHIDEventRef);
     87
    8488WTF_EXTERN_C_END
    8589

  • trunk/Source/WebCore/platform/PlatformWheelEvent.cpp

    r285787 r285790  
    7676
    7777#if PLATFORM(COCOA)
     78    platformWheelEvent.m_ioHIDEventTimestamp = platformWheelEvent.m_timestamp;
    7879    platformWheelEvent.m_unacceleratedScrollingDeltaY = deltaY;
    7980#endif // PLATFORM(COCOA)

  • trunk/Source/WebCore/platform/PlatformWheelEvent.h

    r285367 r285790  
    152152    unsigned scrollCount() const { return m_scrollCount; }
    153153    FloatSize unacceleratedScrollingDelta() const { return { m_unacceleratedScrollingDeltaX, m_unacceleratedScrollingDeltaY }; }
     154   
     155    WallTime ioHIDEventTimestamp() const { return m_ioHIDEventTimestamp; }
    154156#endif
    155157
    … …  
    200202#endif
    201203#if PLATFORM(COCOA)
     204    WallTime m_ioHIDEventTimestamp;
    202205    unsigned m_scrollCount { 0 };
    203206    float m_unacceleratedScrollingDeltaX { 0 };

  • trunk/Source/WebCore/platform/mac/PlatformEventFactoryMac.h

    r235158 r285790  
    5151WEBCORE_EXPORT String keyForKeyEvent(NSEvent *);
    5252WEBCORE_EXPORT String codeForKeyEvent(NSEvent *);
    53 WEBCORE_EXPORT WallTime eventTimeStampSince1970(NSEvent *);
     53WEBCORE_EXPORT WallTime eventTimeStampSince1970(NSTimeInterval);
    5454
    5555WEBCORE_EXPORT OptionSet<PlatformEvent::Modifier> modifiersForEvent(NSEvent *);

  • trunk/Source/WebCore/platform/mac/PlatformEventFactoryMac.mm

    r279059 r285790  
    608608}
    609609
    610 WallTime eventTimeStampSince1970(NSEvent* event)
    611 {
    612     return WallTime::fromRawSeconds(static_cast<double>(cachedStartupTimeIntervalSince1970() + [event timestamp]));
     610WallTime eventTimeStampSince1970(NSTimeInterval timestamp)
     611{
     612    return WallTime::fromRawSeconds(static_cast<double>(cachedStartupTimeIntervalSince1970() + timestamp));
    613613}
    614614
    … …  
    721721
    722722        m_modifiers = modifiersForEvent(event);
    723         m_timestamp = eventTimeStampSince1970(event);
     723        m_timestamp = eventTimeStampSince1970(event.timestamp);
    724724
    725725        // PlatformMouseEvent
    … …  
    756756        m_type = PlatformEvent::Wheel;
    757757        m_modifiers = modifiersForEvent(event);
    758         m_timestamp = eventTimeStampSince1970(event);
     758        m_timestamp = eventTimeStampSince1970(event.timestamp);
    759759
    760760        // PlatformWheelEvent
    … …  
    795795        m_type = isKeyUpEvent(event) ? PlatformEvent::KeyUp : PlatformEvent::KeyDown;
    796796        m_modifiers = modifiersForEvent(event);
    797         m_timestamp = eventTimeStampSince1970(event);
     797        m_timestamp = eventTimeStampSince1970(event.timestamp);
    798798
    799799        // PlatformKeyboardEvent

  • trunk/Source/WebKit/ChangeLog

    r285775 r285790  
     12021-11-14  Simon Fraser  <simon.fraser@apple.com>
     2
     3        Attach IOHIDEvent timestamps to wheel events
     4        https://bugs.webkit.org/show_bug.cgi?id=233051
     5
     6        Reviewed by Wenson Hsieh.
     7       
     8        On some macOS devices, there can be significant deltas between NSEvent timestamps,
     9        and the timestamps on underlying IOHIDEvents (rdar://85309639). This makes momentum
     10        velocity computation unpredictable; we can get better results by using IOHIDEvent
     11        timestamps.
     12
     13        * Shared/WebEventConversion.cpp:
     14        (WebKit::WebKit2PlatformWheelEvent::WebKit2PlatformWheelEvent):
     15        * Shared/WebWheelEvent.cpp:
     16        (WebKit::WebWheelEvent::WebWheelEvent):
     17        (WebKit::WebWheelEvent::encode const):
     18        (WebKit::WebWheelEvent::decode):
     19        * Shared/WebWheelEvent.h:
     20        (WebKit::WebWheelEvent::ioHIDEventTimestamp const):
     21        * Shared/WebWheelEventCoalescer.cpp:
     22        (WebKit::WebWheelEventCoalescer::coalesce):
     23        * Shared/ios/WebIOSEventFactory.mm:
     24        (WebIOSEventFactory::createWebWheelEvent):
     25        * Shared/mac/NativeWebGestureEventMac.mm:
     26        * Shared/mac/WebEventFactory.mm:
     27        (WebKit::WebEventFactory::createWebMouseEvent):
     28        (WebKit::WebEventFactory::createWebWheelEvent):
     29        (WebKit::WebEventFactory::createWebKeyboardEvent):
     30
    1312021-11-13  Per Arne Vollan <pvollan@apple.com>
    232

  • trunk/Source/WebKit/Shared/WebEventConversion.cpp

    r281610 r285790  
    170170#endif
    171171#if PLATFORM(COCOA)
     172        m_ioHIDEventTimestamp = webEvent.ioHIDEventTimestamp();
    172173        m_scrollCount = webEvent.scrollCount();
    173174        m_unacceleratedScrollingDeltaX = webEvent.unacceleratedScrollingDelta().width();

  • trunk/Source/WebKit/Shared/WebWheelEvent.cpp

    r268522 r285790  
    4545
    4646#if PLATFORM(COCOA)
    47 WebWheelEvent::WebWheelEvent(Type type, const IntPoint& position, const IntPoint& globalPosition, const FloatSize& delta, const FloatSize& wheelTicks, Granularity granularity, bool directionInvertedFromDevice, Phase phase, Phase momentumPhase, bool hasPreciseScrollingDeltas, uint32_t scrollCount, const WebCore::FloatSize& unacceleratedScrollingDelta, OptionSet<Modifier> modifiers, WallTime timestamp)
     47WebWheelEvent::WebWheelEvent(Type type, const IntPoint& position, const IntPoint& globalPosition, const FloatSize& delta, const FloatSize& wheelTicks, Granularity granularity, bool directionInvertedFromDevice, Phase phase, Phase momentumPhase, bool hasPreciseScrollingDeltas, uint32_t scrollCount, const WebCore::FloatSize& unacceleratedScrollingDelta, OptionSet<Modifier> modifiers, WallTime timestamp, WallTime ioHIDEventTimestamp)
    4848    : WebEvent(type, modifiers, timestamp)
    4949    , m_position(position)
    … …  
    5656    , m_directionInvertedFromDevice(directionInvertedFromDevice)
    5757    , m_hasPreciseScrollingDeltas(hasPreciseScrollingDeltas)
     58    , m_ioHIDEventTimestamp(ioHIDEventTimestamp)
    5859    , m_scrollCount(scrollCount)
    5960    , m_unacceleratedScrollingDelta(unacceleratedScrollingDelta)
    … …  
    9394#endif
    9495#if PLATFORM(COCOA)
     96    encoder << m_ioHIDEventTimestamp;
    9597    encoder << m_scrollCount;
    9698    encoder << m_unacceleratedScrollingDelta;
    … …  
    123125#endif
    124126#if PLATFORM(COCOA)
     127    if (!decoder.decode(t.m_ioHIDEventTimestamp))
     128        return false;
    125129    if (!decoder.decode(t.m_scrollCount))
    126130        return false;

  • trunk/Source/WebKit/Shared/WebWheelEvent.h

    r268522 r285790  
    5454    WebWheelEvent(Type, const WebCore::IntPoint& position, const WebCore::IntPoint& globalPosition, const WebCore::FloatSize& delta, const WebCore::FloatSize& wheelTicks, Granularity, OptionSet<Modifier>, WallTime timestamp);
    5555#if PLATFORM(COCOA)
    56     WebWheelEvent(Type, const WebCore::IntPoint& position, const WebCore::IntPoint& globalPosition, const WebCore::FloatSize& delta, const WebCore::FloatSize& wheelTicks, Granularity, bool directionInvertedFromDevice, Phase, Phase momentumPhase, bool hasPreciseScrollingDeltas, uint32_t scrollCount, const WebCore::FloatSize& unacceleratedScrollingDelta, OptionSet<Modifier>, WallTime timestamp);
     56    WebWheelEvent(Type, const WebCore::IntPoint& position, const WebCore::IntPoint& globalPosition, const WebCore::FloatSize& delta, const WebCore::FloatSize& wheelTicks, Granularity, bool directionInvertedFromDevice, Phase, Phase momentumPhase, bool hasPreciseScrollingDeltas, uint32_t scrollCount, const WebCore::FloatSize& unacceleratedScrollingDelta, OptionSet<Modifier>, WallTime timestamp, WallTime ioHIDEventTimestamp);
    5757#elif PLATFORM(GTK) || USE(LIBWPE)
    5858    WebWheelEvent(Type, const WebCore::IntPoint& position, const WebCore::IntPoint& globalPosition, const WebCore::FloatSize& delta, const WebCore::FloatSize& wheelTicks, Phase, Phase momentumPhase, Granularity, bool hasPreciseScrollingDeltas, OptionSet<Modifier>, WallTime timestamp);
    … …  
    7171#endif
    7272#if PLATFORM(COCOA)
     73    WallTime ioHIDEventTimestamp() const { return m_ioHIDEventTimestamp; }
    7374    uint32_t scrollCount() const { return m_scrollCount; }
    7475    const WebCore::FloatSize& unacceleratedScrollingDelta() const { return m_unacceleratedScrollingDelta; }
    … …  
    8889    uint32_t m_phase { Phase::PhaseNone };
    8990    uint32_t m_momentumPhase { Phase::PhaseNone };
     91
    9092    bool m_directionInvertedFromDevice { false };
    9193#if PLATFORM(COCOA) || PLATFORM(GTK) || USE(LIBWPE)
    … …  
    9395#endif
    9496#if PLATFORM(COCOA)
     97    WallTime m_ioHIDEventTimestamp;
    9598    uint32_t m_scrollCount { 0 };
    9699    WebCore::FloatSize m_unacceleratedScrollingDelta;

  • trunk/Source/WebKit/Shared/WebWheelEventCoalescer.cpp

    r285165 r285790  
    7979    auto mergedUnacceleratedScrollingDelta = a.unacceleratedScrollingDelta() + b.unacceleratedScrollingDelta();
    8080
    81     return WebWheelEvent(WebEvent::Wheel, b.position(), b.globalPosition(), mergedDelta, mergedWheelTicks, b.granularity(), b.directionInvertedFromDevice(), b.phase(), b.momentumPhase(), b.hasPreciseScrollingDeltas(), b.scrollCount(), mergedUnacceleratedScrollingDelta, b.modifiers(), b.timestamp());
     81    return WebWheelEvent(WebEvent::Wheel, b.position(), b.globalPosition(), mergedDelta, mergedWheelTicks, b.granularity(), b.directionInvertedFromDevice(), b.phase(), b.momentumPhase(), b.hasPreciseScrollingDeltas(), b.scrollCount(), mergedUnacceleratedScrollingDelta, b.modifiers(), b.timestamp(), b.ioHIDEventTimestamp());
    8282#elif PLATFORM(GTK) || USE(LIBWPE)
    8383    return WebWheelEvent(WebEvent::Wheel, b.position(), b.globalPosition(), mergedDelta, mergedWheelTicks, b.phase(), b.momentumPhase(), b.granularity(), b.hasPreciseScrollingDeltas(), b.modifiers(), b.timestamp());

  • trunk/Source/WebKit/Shared/ios/WebIOSEventFactory.mm

    r278253 r285790  
    162162    WebCore::FloatSize wheelTicks = delta;
    163163    wheelTicks.scale(1. / static_cast<float>(WebCore::Scrollbar::pixelsPerLineStep()));
    164 
     164    auto timestamp = MonotonicTime::fromRawSeconds(event.timestamp).approximateWallTime();
    165165    return {
    166166        WebKit::WebEvent::Wheel,
    … …  
    177177        delta,
    178178        { },
    179         MonotonicTime::fromRawSeconds(event.timestamp).approximateWallTime()
     179        timestamp,
     180        timestamp
    180181    };
    181182}

  • trunk/Source/WebKit/Shared/mac/NativeWebGestureEventMac.mm

    r267916 r285790  
    6363        webEventTypeForNSEvent(event),
    6464        OptionSet<WebEvent::Modifier> { },
    65         WebCore::eventTimeStampSince1970(event),
     65        WebCore::eventTimeStampSince1970(event.timestamp),
    6666        WebCore::IntPoint(pointForEvent(event, view)),
    6767        event.type == NSEventTypeMagnify ? event.magnification : 0,

  • trunk/Source/WebKit/Shared/mac/WebEventFactory.mm

    r242325 r285790  
    3333#import <WebCore/Scrollbar.h>
    3434#import <WebCore/WindowsKeyboardCodes.h>
     35#import <pal/spi/cg/CoreGraphicsSPI.h>
    3536#import <pal/spi/mac/NSMenuSPI.h>
    3637#import <wtf/ASCIICType.h>
    … …  
    354355    int clickCount = clickCountForEvent(event);
    355356    auto modifiers = modifiersForEvent(event);
    356     auto timestamp = WebCore::eventTimeStampSince1970(event);
     357    auto timestamp = WebCore::eventTimeStampSince1970(event.timestamp);
    357358    int eventNumber = [event eventNumber];
    358359    int menuTypeForEvent = typeForEvent(event);
    … …  
    409410
    410411    auto modifiers = modifiersForEvent(event);
    411     auto timestamp = WebCore::eventTimeStampSince1970(event);
    412 
    413     return WebWheelEvent(WebEvent::Wheel, WebCore::IntPoint(position), WebCore::IntPoint(globalPosition), WebCore::FloatSize(deltaX, deltaY), WebCore::FloatSize(wheelTicksX, wheelTicksY), granularity, directionInvertedFromDevice, phase, momentumPhase, hasPreciseScrollingDeltas, scrollCount, unacceleratedScrollingDelta, modifiers, timestamp);
     412    auto timestamp = WebCore::eventTimeStampSince1970(event.timestamp);
     413
     414    auto ioHIDEventTimestamp = [](NSEvent *event) {
     415        auto cgEvent = event.CGEvent;
     416        if (!cgEvent)
     417            return event.timestamp;
     418
     419        auto iohidEvent = adoptCF(CGEventCopyIOHIDEvent(cgEvent));
     420        if (!iohidEvent)
     421            return event.timestamp;
     422
     423        auto ioHIDEventTimestamp = IOHIDEventGetTimeStamp(iohidEvent.get()); // IOEventRef timestamp is mach_absolute_time units.
     424        return MonotonicTime::fromMachAbsoluteTime(ioHIDEventTimestamp).secondsSinceEpoch().seconds();
     425    }(event);
     426
     427    auto ioHIDEventWallTime = WebCore::eventTimeStampSince1970(ioHIDEventTimestamp);
     428
     429    return WebWheelEvent(WebEvent::Wheel, WebCore::IntPoint(position), WebCore::IntPoint(globalPosition), WebCore::FloatSize(deltaX, deltaY), WebCore::FloatSize(wheelTicksX, wheelTicksY),
     430        granularity, directionInvertedFromDevice, phase, momentumPhase, hasPreciseScrollingDeltas,
     431        scrollCount, unacceleratedScrollingDelta, modifiers, timestamp, ioHIDEventWallTime);
    414432}
    415433
    … …  
    429447    bool isSystemKey                = false; // SystemKey is always false on the Mac.
    430448    auto modifiers = modifiersForEvent(event);
    431     auto timestamp                  = WebCore::eventTimeStampSince1970(event);
     449    auto timestamp                  = WebCore::eventTimeStampSince1970(event.timestamp);
    432450
    433451    // Always use 13 for Enter/Return -- we don't want to use AppKit's different character for Enter.
Note: See TracChangeset for help on using the changeset viewer.