Changeset 223869 in webkit


Ignore:
Timestamp:
Oct 23, 2017 6:16:59 PM (6 years ago)
Author:
BJ Burg
Message:

[Mac] Web Automation: key modifiers for synthesized NSEvents are incorrect
https://bugs.webkit.org/show_bug.cgi?id=178615

Reviewed by Joseph Pecoraro.

In both PLATFORM(MAC) platform methods for simulating keyboard interactions,
we errantly relied on +[NSEvent modifierFlags] to get the current state of
sticky modifiers when creating synthesized events. This is incorrect for two reasons:
modifierFlags is never updated when simulating a sequence of events (because
all the events are synthesized before any are delivered); and the NSEvent class
method only reflects the modifier state of the primary physical keyboard, which
is not affected by synthesized NSEvents that happen to have modifier flags.

Instead, just keep our own m_currentModifiers state in the session and compute
the necessary NSEventModifierFlags to put on each synthesized event. This aligns
the implementation with the treatment of sticky keys in the iOS and GTK platform methods.

  • UIProcess/Automation/WebAutomationSession.h: Every port gets this variable now.
  • UIProcess/Automation/mac/WebAutomationSessionMac.mm:

(WebKit::WebAutomationSession::platformSimulateKeyStroke):
(WebKit::WebAutomationSession::platformSimulateKeySequence):
Use and update m_currentModifiers.

Location:
trunk/Source/WebKit
Files:
3 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebKit/ChangeLog

    r223861 r223869  
     12017-10-23  Brian Burg  <bburg@apple.com>
     2
     3        [Mac] Web Automation: key modifiers for synthesized NSEvents are incorrect
     4        https://bugs.webkit.org/show_bug.cgi?id=178615
     5
     6        Reviewed by Joseph Pecoraro.
     7
     8        In both PLATFORM(MAC) platform methods for simulating keyboard interactions,
     9        we errantly relied on +[NSEvent modifierFlags] to get the current state of
     10        sticky modifiers when creating synthesized events. This is incorrect for two reasons:
     11        modifierFlags is never updated when simulating a sequence of events (because
     12        all the events are synthesized before any are delivered); and the NSEvent class
     13        method only reflects the modifier state of the primary physical keyboard, which
     14        is not affected by synthesized NSEvents that happen to have modifier flags.
     15
     16        Instead, just keep our own m_currentModifiers state in the session and compute
     17        the necessary NSEventModifierFlags to put on each synthesized event. This aligns
     18        the implementation with the treatment of sticky keys in the iOS and GTK platform methods.
     19
     20        * UIProcess/Automation/WebAutomationSession.h: Every port gets this variable now.
     21        * UIProcess/Automation/mac/WebAutomationSessionMac.mm:
     22        (WebKit::WebAutomationSession::platformSimulateKeyStroke):
     23        (WebKit::WebAutomationSession::platformSimulateKeySequence):
     24        Use and update m_currentModifiers.
     25
    1262017-10-23  Alex Christensen  <achristensen@webkit.org>
    227
  • trunk/Source/WebKit/UIProcess/Automation/WebAutomationSession.h

    r222503 r223869  
    260260    bool m_permissionForGetUserMedia { true };
    261261
     262
     263    // Keep track of currently active modifiers across multiple keystrokes.
     264    // Most platforms do not track current modifiers from synthesized events.
     265    unsigned m_currentModifiers { 0 };
     266
    262267#if ENABLE(REMOTE_INSPECTOR)
    263268    Inspector::FrontendChannel* m_remoteChannel { nullptr };
    264269#endif
    265270
    266 #if PLATFORM(IOS) || PLATFORM(GTK)
    267     // Keep track of currently active modifiers across multiple keystrokes.
    268     // We don't synthesize platform keyboard events on iOS, so we need to track it ourselves.
    269     // GTK+ doesn't keep track of the active modifiers when using synthesized events.
    270     unsigned m_currentModifiers { 0 };
    271 #endif
    272271};
    273272
  • trunk/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm

    r217244 r223869  
    446446    auto eventsToBeSent = adoptNS([[NSMutableArray alloc] init]);
    447447
    448     NSEventModifierFlags existingModifiers = [NSEvent modifierFlags];
    449     NSEventModifierFlags updatedModifiers = 0;
    450 
    451448    // FIXME: this timestamp is not even close to matching native events. Find out how to get closer.
    452449    NSTimeInterval timestamp = [NSDate timeIntervalSinceReferenceDate];
     
    458455    case Inspector::Protocol::Automation::KeyboardInteractionType::KeyPress: {
    459456        NSEventType eventType = isStickyModifier ? NSEventTypeFlagsChanged : NSEventTypeKeyDown;
    460         updatedModifiers = existingModifiers | changedModifiers;
    461         [eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
     457        m_currentModifiers |= changedModifiers;
     458        [eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
    462459        break;
    463460    }
    464461    case Inspector::Protocol::Automation::KeyboardInteractionType::KeyRelease: {
    465462        NSEventType eventType = isStickyModifier ? NSEventTypeFlagsChanged : NSEventTypeKeyUp;
    466         updatedModifiers = existingModifiers & ~changedModifiers;
    467         [eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
     463        m_currentModifiers &= ~changedModifiers;
     464        [eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
    468465        break;
    469466    }
     
    474471            return;
    475472
    476         updatedModifiers = existingModifiers | changedModifiers;
    477         [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyDown location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
    478         [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyUp location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
     473        m_currentModifiers |= changedModifiers;
     474        [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyDown location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
     475        [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyUp location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
    479476        break;
    480477    }
     
    536533    NSString *text = keySequence;
    537534
    538     NSEventModifierFlags modifiers = [NSEvent modifierFlags];
    539535    NSTimeInterval timestamp = [NSDate timeIntervalSinceReferenceDate];
    540536    NSWindow *window = page.platformWindow();
     
    547543        // WebDriver expects these to be delivered as [shift-down, key-down, key-up, shift-up]
    548544        // keystrokes unless the shift key is already pressed down from an earlier interaction.
    549         BOOL shiftAlreadyPressed = modifiers & NSEventModifierFlagShift;
     545        BOOL shiftAlreadyPressed = m_currentModifiers & NSEventModifierFlagShift;
    550546        BOOL shouldPressShift = !shiftAlreadyPressed && substringRange.length == 1 && characterIsProducedUsingShift(substring);
    551         NSEventModifierFlags modifiersForKeystroke = shouldPressShift ? modifiers | NSEventModifierFlagShift : modifiers;
     547        NSEventModifierFlags modifiersForKeystroke = shouldPressShift ? m_currentModifiers | NSEventModifierFlagShift : m_currentModifiers;
    552548
    553549        if (shouldPressShift)
     
    558554       
    559555        if (shouldPressShift)
    560             [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeFlagsChanged location:eventPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:@"" charactersIgnoringModifiers:@"" isARepeat:NO keyCode:kVK_Shift]];
     556            [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeFlagsChanged location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:@"" charactersIgnoringModifiers:@"" isARepeat:NO keyCode:kVK_Shift]];
    561557    }];
    562558
Note: See TracChangeset for help on using the changeset viewer.