Changeset 206798 in webkit


Ignore:
Timestamp:
Oct 4, 2016, 5:23:51 PM (10 years ago)
Author:
Simon Fraser
Message:

[iOS WK2] Make it possible for a test to describe a user gesture as a stream of events in JSON format
​https://bugs.webkit.org/show_bug.cgi?id=162934

Reviewed by Dean Jackson.

Tools:

With this change, a test can describe a user gesture in an "event stream", which is
some JSON describing an array of events with their underlying touches. The added
test describes a single tap.

The implementation fires up an NSThread, and sleeps the thread between events to dispatch
them at close to real time.

In future, HIDEventGenerator could use this internally for all of the "compound" interactions.

  • DumpRenderTree/ios/UIScriptControllerIOS.mm:

(WTR::UIScriptController::sendEventStream):

  • TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
  • TestRunnerShared/UIScriptContext/UIScriptController.cpp:

(WTR::UIScriptController::sendEventStream):

  • TestRunnerShared/UIScriptContext/UIScriptController.h:
  • WebKitTestRunner/ios/HIDEventGenerator.h:
  • WebKitTestRunner/ios/HIDEventGenerator.mm:

(transducerTypeFromString):
(phaseFromString):
(-[HIDEventGenerator eventMaskFromEventInfo:]):
(-[HIDEventGenerator touchFromEventInfo:]):
(-[HIDEventGenerator _createIOHIDEventWithInfo:]):
(-[HIDEventGenerator dispatchEventWithInfo:]):
(-[HIDEventGenerator eventDispatchThreadEntry:]):
(-[HIDEventGenerator sendEventStream:completionBlock:]):

  • WebKitTestRunner/ios/UIScriptControllerIOS.mm:

(WTR::UIScriptController::sendEventStream):

LayoutTests:

  • fast/events/ios/event-stream-single-tap-expected.txt: Added.
  • fast/events/ios/event-stream-single-tap.html: Added.
Location:
trunk
Files:
2 added
10 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r206796 r206798  
     12016-10-04  Simon Fraser  <simon.fraser@apple.com>
     2
     3        [iOS WK2] Make it possible for a test to describe a user gesture as a stream of events in JSON format
     4        https://bugs.webkit.org/show_bug.cgi?id=162934
     5
     6        Reviewed by Dean Jackson.
     7
     8        * fast/events/ios/event-stream-single-tap-expected.txt: Added.
     9        * fast/events/ios/event-stream-single-tap.html: Added.
     10
    1112016-10-04  Chris Dumez  <cdumez@apple.com>
    212

  • trunk/Tools/ChangeLog

    r206793 r206798  
     12016-10-04  Simon Fraser  <simon.fraser@apple.com>
     2
     3        [iOS WK2] Make it possible for a test to describe a user gesture as a stream of events in JSON format
     4        https://bugs.webkit.org/show_bug.cgi?id=162934
     5
     6        Reviewed by Dean Jackson.
     7
     8        With this change, a test can describe a user gesture in an "event stream", which is
     9        some JSON describing an array of events with their underlying touches. The added
     10        test describes a single tap.
     11       
     12        The implementation fires up an NSThread, and sleeps the thread between events to dispatch
     13        them at close to real time.
     14       
     15        In future, HIDEventGenerator could use this internally for all of the "compound" interactions.
     16
     17        * DumpRenderTree/ios/UIScriptControllerIOS.mm:
     18        (WTR::UIScriptController::sendEventStream):
     19        * TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
     20        * TestRunnerShared/UIScriptContext/UIScriptController.cpp:
     21        (WTR::UIScriptController::sendEventStream):
     22        * TestRunnerShared/UIScriptContext/UIScriptController.h:
     23        * WebKitTestRunner/ios/HIDEventGenerator.h:
     24        * WebKitTestRunner/ios/HIDEventGenerator.mm:
     25        (transducerTypeFromString):
     26        (phaseFromString):
     27        (-[HIDEventGenerator eventMaskFromEventInfo:]):
     28        (-[HIDEventGenerator touchFromEventInfo:]):
     29        (-[HIDEventGenerator _createIOHIDEventWithInfo:]):
     30        (-[HIDEventGenerator dispatchEventWithInfo:]):
     31        (-[HIDEventGenerator eventDispatchThreadEntry:]):
     32        (-[HIDEventGenerator sendEventStream:completionBlock:]):
     33        * WebKitTestRunner/ios/UIScriptControllerIOS.mm:
     34        (WTR::UIScriptController::sendEventStream):
     35
    1362016-10-04  Megan Gardner  <megan_gardner@apple.com>
    237

  • trunk/Tools/DumpRenderTree/ios/UIScriptControllerIOS.mm

    r206645 r206798  
    109109}
    110110
     111void UIScriptController::sendEventStream(JSStringRef eventsJSON, JSValueRef callback)
     112{
     113}
     114
    111115void UIScriptController::typeCharacterUsingHardwareKeyboard(JSStringRef character, JSValueRef callback)
    112116{

  • trunk/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl

    r206645 r206798  
    5050    void keyUpUsingHardwareKeyboard(DOMString character, object callback);
    5151
     52    // eventsJSON describes a series of user events in JSON form. For the keys, see HIDEventGenerator.mm.
     53    // For example, this JSON describes a touch down followed by a touch up (i.e. a single tap).
     54    //  {
     55    //      "events" : [
     56    //          {
     57    //              "inputType" : "hand",
     58    //              "timeOffset" : 0,
     59    //              "touches" : [
     60    //                  {
     61    //                      "inputType" : "finger",
     62    //                      "phase" : "began",
     63    //                      "id" : 1,
     64    //                      "x" : 100,
     65    //                      "y" : 120
     66    //                  }
     67    //              ]
     68    //          },
     69    //          {
     70    //              "inputType" : "hand",
     71    //              "timeOffset" : 0.002, // seconds relative to the first event
     72    //              "touches" : [
     73    //                  {
     74    //                      "inputType" : "finger",
     75    //                      "phase" : "ended",
     76    //                      "id" : 1,
     77    //                      "x" : 100,
     78    //                      "y" : 120
     79    //                  }
     80    //              ]
     81    //          },
     82    //      ]
     83    //  }
     84    void sendEventStream(DOMString eventsJSON, object callback);
     85
    5286    // Equivalent of pressing the Done button in the form accessory bar.
    5387    void dismissFormAccessoryView();

  • trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.cpp

    r206645 r206798  
    181181}
    182182
     183void UIScriptController::sendEventStream(JSStringRef eventsJSON, JSValueRef callback)
     184{
     185}
     186
    183187void UIScriptController::typeCharacterUsingHardwareKeyboard(JSStringRef, JSValueRef)
    184188{

  • trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h

    r206645 r206798  
    6464
    6565    void longPressAtPoint(long x, long y, JSValueRef callback);
    66    
     66
     67    void sendEventStream(JSStringRef eventsJSON, JSValueRef callback);
     68
    6769    void typeCharacterUsingHardwareKeyboard(JSStringRef character, JSValueRef callback);
    6870    void keyDownUsingHardwareKeyboard(JSStringRef character, JSValueRef callback);

  • trunk/Tools/WebKitTestRunner/ios/HIDEventGenerator.h

    r206282 r206798  
    2727
    2828#import <CoreGraphics/CGGeometry.h>
     29
     30// Keys for sendEventStream:completionBlock:.
     31extern NSString* const TopLevelEventInfoKey;
     32extern NSString* const HIDEventInputType;
     33extern NSString* const HIDEventTimeOffsetKey;
     34extern NSString* const HIDEventPhaseKey;
     35extern NSString* const HIDEventTouchIDKey;
     36extern NSString* const HIDEventPressureKey;
     37extern NSString* const HIDEventXKey;
     38extern NSString* const HIDEventYKey;
     39extern NSString* const HIDEventTwistKey;
     40extern NSString* const HIDEventMajorRadiusKey;
     41extern NSString* const HIDEventMinorRadiusKey;
     42extern NSString* const HIDEventTouchesKey;
     43
     44// Values for HIDEventInputType.
     45extern NSString* const HIDEventInputTypeHand;
     46extern NSString* const HIDEventInputTypeFinger;
     47extern NSString* const HIDEventInputTypeStylus;
     48
     49// Values for HIDEventPhaseKey.
     50extern NSString* const HIDEventPhaseBegan;
     51extern NSString* const HIDEventPhaseMoved;
     52extern NSString* const HIDEventPhaseEnded;
     53extern NSString* const HIDEventPhaseCanceled;
     54
    2955
    3056@interface HIDEventGenerator : NSObject
    … …  
    5783- (void)pinchOpenWithStartPoint:(CGPoint)startLocation endPoint:(CGPoint)endLocation duration:(double)seconds completionBlock:(void (^)(void))completionBlock;
    5884
     85// Event stream
     86- (void)sendEventStream:(NSDictionary *)eventInfo completionBlock:(void (^)(void))completionBlock;
     87
    5988- (void)markerEventReceived:(IOHIDEventRef)event;
    6089

  • trunk/Tools/WebKitTestRunner/ios/HIDEventGenerator.mm

    r206282 r206798  
    3838SOFT_LINK(BackBoardServices, BKSHIDEventSetDigitizerInfo, void, (IOHIDEventRef digitizerEvent, uint32_t contextID, uint8_t systemGestureisPossible, uint8_t isSystemGestureStateChangeEvent, CFStringRef displayUUID, CFTimeInterval initialTouchTimestamp, float maxForce), (digitizerEvent, contextID, systemGestureisPossible, isSystemGestureStateChangeEvent, displayUUID, initialTouchTimestamp, maxForce));
    3939
     40NSString* const TopLevelEventInfoKey = @"events";
     41NSString* const HIDEventInputType = @"inputType";
     42NSString* const HIDEventTimeOffsetKey = @"timeOffset";
     43NSString* const HIDEventTouchesKey = @"touches";
     44NSString* const HIDEventPhaseKey = @"phase";
     45NSString* const HIDEventTouchIDKey = @"id";
     46NSString* const HIDEventPressureKey = @"pressure";
     47NSString* const HIDEventXKey = @"x";
     48NSString* const HIDEventYKey = @"y";
     49NSString* const HIDEventTwistKey = @"twist";
     50NSString* const HIDEventMajorRadiusKey = @"majorRadius";
     51NSString* const HIDEventMinorRadiusKey = @"minorRadius";
     52
     53NSString* const HIDEventInputTypeHand = @"hand";
     54NSString* const HIDEventInputTypeFinger = @"finger";
     55NSString* const HIDEventInputTypeStylus = @"stylus";
     56
     57NSString* const HIDEventPhaseBegan = @"began";
     58NSString* const HIDEventPhaseMoved = @"moved";
     59NSString* const HIDEventPhaseEnded = @"ended";
     60NSString* const HIDEventPhaseCanceled = @"canceled";
     61
    4062static const NSTimeInterval fingerLiftDelay = 0.05;
    4163static const NSTimeInterval multiTapInterval = 0.15;
    … …  
    145167        kIOHIDEventOptionNone));
    146168    [self _sendHIDEvent:eventRef.get()];
     169}
     170
     171static IOHIDDigitizerTransducerType transducerTypeFromString(NSString * transducerTypeString)
     172{
     173    if ([transducerTypeString isEqualToString:HIDEventInputTypeHand])
     174        return kIOHIDDigitizerTransducerTypeHand;
     175
     176    if ([transducerTypeString isEqualToString:HIDEventInputTypeFinger])
     177        return kIOHIDDigitizerTransducerTypeFinger;
     178
     179    if ([transducerTypeString isEqualToString:HIDEventInputTypeStylus])
     180        return kIOHIDDigitizerTransducerTypeStylus;
     181   
     182    ASSERT_NOT_REACHED();
     183    return 0;
     184}
     185
     186static UITouchPhase phaseFromString(NSString *string)
     187{
     188    if ([string isEqualToString:HIDEventPhaseBegan])
     189        return UITouchPhaseBegan;
     190
     191    if ([string isEqualToString:HIDEventPhaseMoved])
     192        return UITouchPhaseMoved;
     193
     194    if ([string isEqualToString:HIDEventPhaseEnded])
     195        return UITouchPhaseEnded;
     196
     197    if ([string isEqualToString:HIDEventPhaseCanceled])
     198        return UITouchPhaseCancelled;
     199
     200    return UITouchPhaseStationary;
     201}
     202
     203- (IOHIDDigitizerEventMask)eventMaskFromEventInfo:(NSDictionary *)info
     204{
     205    NSArray *childEvents = info[HIDEventTouchesKey];
     206    for (NSDictionary *touchInfo in childEvents) {
     207        UITouchPhase phase = phaseFromString(touchInfo[HIDEventPhaseKey]);
     208        // If there are any new or ended events, mask includes touch.
     209        if (phase == UITouchPhaseBegan || phase == UITouchPhaseEnded || phase == UITouchPhaseCancelled)
     210            return kIOHIDDigitizerEventTouch;
     211    }
     212   
     213    return 0;
     214}
     215
     216// Returns 1 for all events where the fingers are on the glass (everything but enced and canceled).
     217- (CFIndex)touchFromEventInfo:(NSDictionary *)info
     218{
     219    NSArray *childEvents = info[HIDEventTouchesKey];
     220    for (NSDictionary *touchInfo in childEvents) {
     221        UITouchPhase phase = phaseFromString(touchInfo[HIDEventPhaseKey]);
     222        if (phase == UITouchPhaseBegan || phase == UITouchPhaseMoved || phase == UITouchPhaseStationary)
     223            return 1;
     224    }
     225   
     226    return 0;
     227}
     228
     229// FIXME: callers of _createIOHIDEventType could switch to this.
     230- (IOHIDEventRef)_createIOHIDEventWithInfo:(NSDictionary *)info
     231{
     232    uint64_t machTime = mach_absolute_time();
     233
     234    IOHIDDigitizerEventMask eventMask = [self eventMaskFromEventInfo:info];
     235
     236    CFIndex range = 0;
     237    // touch is 1 if a finger is down.
     238    CFIndex touch = [self touchFromEventInfo:info];
     239
     240    IOHIDEventRef eventRef = IOHIDEventCreateDigitizerEvent(kCFAllocatorDefault, machTime,
     241        transducerTypeFromString(info[HIDEventInputType]),  // transducerType
     242        0,                                                  // index
     243        0,                                                  // identifier
     244        eventMask,                                          // event mask
     245        0,                                                  // button event
     246        0,                                                  // x
     247        0,                                                  // y
     248        0,                                                  // z
     249        0,                                                  // presure
     250        0,                                                  // twist
     251        range,                                              // range
     252        touch,                                              // touch
     253        kIOHIDEventOptionNone);
     254
     255    IOHIDEventSetIntegerValue(eventRef, kIOHIDEventFieldDigitizerIsDisplayIntegrated, 1);
     256
     257    NSArray *childEvents = info[HIDEventTouchesKey];
     258    for (NSDictionary *touchInfo in childEvents) {
     259
     260        IOHIDDigitizerEventMask childEventMask = 0;
     261
     262        UITouchPhase phase = phaseFromString(touchInfo[HIDEventPhaseKey]);
     263        if (phase != UITouchPhaseCancelled && phase != UITouchPhaseBegan && phase != UITouchPhaseEnded)
     264            childEventMask |= kIOHIDDigitizerEventPosition;
     265
     266        if (phase == UITouchPhaseBegan || phase == UITouchPhaseEnded || phase == UITouchPhaseCancelled)
     267            childEventMask |= (kIOHIDDigitizerEventTouch | kIOHIDDigitizerEventRange);
     268
     269        if (phase == UITouchPhaseCancelled)
     270            childEventMask |= kIOHIDDigitizerEventCancel;
     271
     272        IOHIDEventRef subEvent = IOHIDEventCreateDigitizerFingerEvent(kCFAllocatorDefault, machTime,
     273            [touchInfo[HIDEventTouchIDKey] intValue],               // index
     274            2,                                                      // identifier (which finger we think it is). FIXME: this should come from the data.
     275            childEventMask,
     276            [touchInfo[HIDEventXKey] floatValue],
     277            [touchInfo[HIDEventYKey] floatValue],
     278            0, // z
     279            [touchInfo[HIDEventPressureKey] floatValue],
     280            [touchInfo[HIDEventTwistKey] floatValue],
     281            touch,                                                  // range
     282            touch,                                                  // touch
     283            kIOHIDEventOptionNone);
     284
     285        IOHIDEventSetFloatValue(subEvent, kIOHIDEventFieldDigitizerMajorRadius, [touchInfo[HIDEventMajorRadiusKey] floatValue]);
     286        IOHIDEventSetFloatValue(subEvent, kIOHIDEventFieldDigitizerMinorRadius, [touchInfo[HIDEventMinorRadiusKey] floatValue]);
     287
     288        IOHIDEventAppendEvent(eventRef, subEvent, 0);
     289        CFRelease(subEvent);
     290    }
     291
     292    return eventRef;
    147293}
    148294
    … …  
    767913}
    768914
     915- (void)dispatchEventWithInfo:(NSDictionary *)eventInfo
     916{
     917    ASSERT([NSThread isMainThread]);
     918
     919    RetainPtr<IOHIDEventRef> eventRef = adoptCF([self _createIOHIDEventWithInfo:eventInfo]);
     920    [self _sendHIDEvent:eventRef.get()];
     921}
     922
     923- (void)eventDispatchThreadEntry:(NSDictionary *)threadData
     924{
     925    NSDictionary *eventStream = threadData[@"eventInfo"];
     926    void (^completionBlock)() = threadData[@"completionBlock"];
     927
     928    NSArray *events = eventStream[TopLevelEventInfoKey];
     929    if (!events.count) {
     930        NSLog(@"No events found in event stream");
     931        return;
     932    }
     933
     934    CFAbsoluteTime startTime = CFAbsoluteTimeGetCurrent();
     935   
     936    for (NSDictionary *eventInfo in events) {
     937        NSTimeInterval eventRelativeTime = [eventInfo[HIDEventTimeOffsetKey] doubleValue];
     938        CFAbsoluteTime targetTime = startTime + eventRelativeTime;
     939       
     940        CFTimeInterval waitTime = targetTime - CFAbsoluteTimeGetCurrent();
     941        if (waitTime > 0)
     942            [NSThread sleepForTimeInterval:waitTime];
     943
     944        dispatch_async(dispatch_get_main_queue(), ^ {
     945            [self dispatchEventWithInfo:eventInfo];
     946        });
     947    }
     948
     949    dispatch_async(dispatch_get_main_queue(), ^ {
     950        [self _sendMarkerHIDEventWithCompletionBlock:completionBlock];
     951    });
     952}
     953
     954- (void)sendEventStream:(NSDictionary *)eventInfo completionBlock:(void (^)(void))completionBlock
     955{
     956    if (!eventInfo) {
     957        NSLog(@"eventInfo is nil");
     958        if (completionBlock)
     959            completionBlock();
     960        return;
     961    }
     962   
     963    NSDictionary* threadData = @{
     964        @"eventInfo": [eventInfo copy],
     965        @"completionBlock": [[completionBlock copy] autorelease]
     966    };
     967   
     968    NSThread *eventDispatchThread = [[[NSThread alloc] initWithTarget:self selector:@selector(eventDispatchThreadEntry:) object:threadData] autorelease];
     969    eventDispatchThread.qualityOfService = NSQualityOfServiceUserInteractive;
     970    [eventDispatchThread start];
     971}
     972
    769973@end

  • trunk/Tools/WebKitTestRunner/ios/IOKitSPI.h

    r205618 r206798  
    118118
    119119enum {
     120    kIOHIDDigitizerTransducerTypeStylus  = 0,
     121    kIOHIDDigitizerTransducerTypeFinger = 2,
    120122    kIOHIDDigitizerTransducerTypeHand = 3
    121123};

  • trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm

    r206645 r206798  
    3636#import "UIScriptContext.h"
    3737#import <JavaScriptCore/JavaScriptCore.h>
     38#import <JavaScriptCore/OpaqueJSString.h>
    3839#import <UIKit/UIKit.h>
    3940#import <WebCore/FloatRect.h>
    … …  
    170171    auto location = globalToContentCoordinates(TestController::singleton().mainWebView()->platformView(), x, y);
    171172    [[HIDEventGenerator sharedHIDEventGenerator] stylusTapAtPoint:location azimuthAngle:azimuthAngle altitudeAngle:altitudeAngle pressure:pressure completionBlock:^{
     173        if (!m_context)
     174            return;
     175        m_context->asyncTaskComplete(callbackID);
     176    }];
     177}
     178
     179void UIScriptController::sendEventStream(JSStringRef eventsJSON, JSValueRef callback)
     180{
     181    unsigned callbackID = m_context->prepareForAsyncTask(callback, CallbackTypeNonPersistent);
     182
     183    String jsonString = eventsJSON->string();
     184    auto eventInfo = dynamic_objc_cast<NSDictionary>([NSJSONSerialization JSONObjectWithData:[(NSString *)jsonString dataUsingEncoding:NSUTF8StringEncoding] options:0 error:nil]);
     185    if (!eventInfo || ![eventInfo isKindOfClass:[NSDictionary class]]) {
     186        WTFLogAlways("JSON is not convertible to a dictionary");
     187        return;
     188    }
     189   
     190    [[HIDEventGenerator sharedHIDEventGenerator] sendEventStream:eventInfo completionBlock:^{
    172191        if (!m_context)
    173192            return;
Note: See TracChangeset for help on using the changeset viewer.