Changeset 209010 in webkit


Ignore:
Timestamp:
Nov 28, 2016 12:46:44 PM (7 years ago)
Author:
commit-queue@webkit.org
Message:

Support for HTML Media Capture API
​https://bugs.webkit.org/show_bug.cgi?id=43239

Patch by Andrew Gold <​agold@apple.com> on 2016-11-28
Reviewed by Tim Horton.

In order to enable media capture on iOS, we must first use the AVFoundation API to
check/request capture permission from the user for Safari. Then, Safari must request
permission on behalf of the webpage to use the capture devices. Additionally, Safari
must present UI when the capture is taking place, so WebKit needs to notify the client
of a capture session beginning and ending. To do this, we added four methods to
WKUIDelegatePrivate to request permission from the user, check for permissions, notify
that a capture session has begun, and notify that a capture session has ended. Additionally,
we added a private method to WKWebView that allows the client to stop a capture session.

  • UIProcess/API/APIUIClient.h:

(API::UIClient::didBeginCaptureSession): Notifies the client of a capture session beginning.
(API::UIClient::didEndCaptureSession): Notifies the client of a capture session ending.

  • UIProcess/API/Cocoa/WKUIDelegatePrivate.h: Added new delegate methods to request permission,

check for permission, notify of a capture session beginning, and notify of a capture session
ending.

  • UIProcess/API/Cocoa/WKWebView.mm:

(-[WKWebView _stopMediaCapture]): Cancels a media capture session.

  • UIProcess/API/Cocoa/WKWebViewPrivate.h:
  • UIProcess/Cocoa/UIDelegate.h:
  • UIProcess/Cocoa/UIDelegate.mm:

(WebKit::UIDelegate::setDelegate): Added the new delegate methods.
(WebKit::UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest):
First checks if the user has authorized the client application access capture devices,
then uses the WKUIDelegate to request permission for a site from the user.
(WebKit::UIDelegate::UIClient::checkUserMediaPermissionForOrigin): Checks the client
for permission to access capture devices.
(WebKit::UIDelegate::UIClient::didBeginCaptureSession): Notifies the client of a capture
session beginning.
(WebKit::UIDelegate::UIClient::didEndCaptureSession): Notifies the client of a capture
session ending.

  • UIProcess/WebPageProxy.cpp:

(WebKit::WebPageProxy::isPlayingMediaDidChange): Calls UIDelegate method to notify the
client of a capture session beginning/ending.

Location:
trunk/Source/WebKit2
Files:
8 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebKit2/ChangeLog

    r209008 r209010  
     12016-11-28  Andrew Gold  <agold@apple.com>
     2
     3        Support for HTML Media Capture API
     4        https://bugs.webkit.org/show_bug.cgi?id=43239
     5
     6        Reviewed by Tim Horton.
     7
     8        In order to enable media capture on iOS, we must first use the AVFoundation API to
     9        check/request capture permission from the user for Safari. Then, Safari must request
     10        permission on behalf of the webpage to use the capture devices. Additionally, Safari
     11        must present UI when the capture is taking place, so WebKit needs to notify the client
     12        of a capture session beginning and ending. To do this, we added four methods to
     13        WKUIDelegatePrivate to request permission from the user, check for permissions, notify
     14        that a capture session has begun, and notify that a capture session has ended. Additionally,
     15        we added a private method to WKWebView that allows the client to stop a capture session.
     16
     17        * UIProcess/API/APIUIClient.h:
     18        (API::UIClient::didBeginCaptureSession): Notifies the client of a capture session beginning.
     19        (API::UIClient::didEndCaptureSession): Notifies the client of a capture session ending.
     20
     21        * UIProcess/API/Cocoa/WKUIDelegatePrivate.h: Added new delegate methods to request permission,
     22        check for permission, notify of a capture session beginning, and notify of a capture session
     23        ending.
     24
     25        * UIProcess/API/Cocoa/WKWebView.mm:
     26        (-[WKWebView _stopMediaCapture]): Cancels a media capture session.
     27        * UIProcess/API/Cocoa/WKWebViewPrivate.h:
     28
     29        * UIProcess/Cocoa/UIDelegate.h:
     30        * UIProcess/Cocoa/UIDelegate.mm:
     31        (WebKit::UIDelegate::setDelegate): Added the new delegate methods.
     32        (WebKit::UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest):
     33        First checks if the user has authorized the client application access capture devices,
     34        then uses the WKUIDelegate to request permission for a site from the user.
     35        (WebKit::UIDelegate::UIClient::checkUserMediaPermissionForOrigin): Checks the client
     36        for permission to access capture devices.
     37        (WebKit::UIDelegate::UIClient::didBeginCaptureSession): Notifies the client of a capture
     38        session beginning.
     39        (WebKit::UIDelegate::UIClient::didEndCaptureSession): Notifies the client of a capture
     40        session ending.
     41
     42        * UIProcess/WebPageProxy.cpp:
     43        (WebKit::WebPageProxy::isPlayingMediaDidChange): Calls UIDelegate method to notify the
     44        client of a capture session beginning/ending.
     45
    1462016-11-28  Eric Carlson  <eric.carlson@apple.com>
    247

  • trunk/Source/WebKit2/UIProcess/API/APIUIClient.h

    r208903 r209010  
    151151
    152152    virtual void isPlayingAudioDidChange(WebKit::WebPageProxy&) { }
     153    virtual void didBeginCaptureSession() { }
     154    virtual void didEndCaptureSession() { }
    153155
    154156#if ENABLE(MEDIA_SESSION)

  • trunk/Source/WebKit2/UIProcess/API/Cocoa/WKUIDelegatePrivate.h

    r208911 r209010  
    6161- (void)_webView:(WKWebView *)webView imageOrMediaDocumentSizeChanged:(CGSize)size WK_API_AVAILABLE(macosx(10.12), ios(10.0));
    6262- (NSDictionary *)_dataDetectionContextForWebView:(WKWebView *)webView WK_API_AVAILABLE(macosx(10.12), ios(10.0));
     63- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForMicrophone:(BOOL)microphone camera:(BOOL)camera url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorizedMicrophone, BOOL authorizedCamera))decisionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
     64- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
     65- (void)_webViewDidBeginCaptureSession:(WKWebView *)webView WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
     66- (void)_webViewDidEndCaptureSession:(WKWebView *)webView WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
    6367#if TARGET_OS_IPHONE
    6468- (BOOL)_webView:(WKWebView *)webView shouldIncludeAppLinkActionsForElement:(_WKActivatedElementInfo *)element WK_API_AVAILABLE(ios(9.0));

  • trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebView.mm

    r208985 r209010  
    40414041}
    40424042
     4043- (void)_stopMediaCapture
     4044{
     4045    _page->setMuted(WebCore::MediaProducer::CaptureDevicesAreMuted);
     4046}
     4047
    40434048#pragma mark iOS-specific methods
    40444049

  • trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebViewPrivate.h

    r208926 r209010  
    259259@property (nonatomic, readonly) BOOL _isInFullscreen WK_API_AVAILABLE(macosx(WK_MAC_TBA));
    260260
     261- (void)_stopMediaCapture;
     262
    261263@end
    262264

  • trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.h

    r208903 r209010  
    9090        bool runOpenPanel(WebPageProxy*, WebFrameProxy*, const WebCore::SecurityOriginData&, API::OpenPanelParameters*, WebOpenPanelResultListenerProxy*) override;
    9191#endif
     92        bool decidePolicyForUserMediaPermissionRequest(WebKit::WebPageProxy&, WebKit::WebFrameProxy&, API::SecurityOrigin&, API::SecurityOrigin&, WebKit::UserMediaPermissionRequestProxy&) override;
     93        bool checkUserMediaPermissionForOrigin(WebKit::WebPageProxy&, WebKit::WebFrameProxy&, API::SecurityOrigin&, API::SecurityOrigin&, WebKit::UserMediaPermissionCheckProxy&) override;
     94        void didBeginCaptureSession() override;
     95        void didEndCaptureSession() override;
    9296        void printFrame(WebKit::WebPageProxy*, WebKit::WebFrameProxy*) override;
    9397#if PLATFORM(IOS)
    … …  
    131135        bool webViewDidEnterFullscreen : 1;
    132136        bool webViewDidExitFullscreen : 1;
     137        bool webViewRequestUserMediaAuthorizationForMicrophoneCameraURLMainFrameURLDecisionHandler : 1;
     138        bool webViewCheckUserMediaPermissionForURLMainFrameURLFrameIdentifierDecisionHandler : 1;
     139        bool webViewDidBeginCaptureSession : 1;
     140        bool webViewDidEndCaptureSession : 1;
    133141#if PLATFORM(IOS)
    134142#if HAVE(APP_LINKS)

  • trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.mm

    r208903 r209010  
    3131#import "CompletionHandlerCallChecker.h"
    3232#import "NavigationActionData.h"
     33#import "UserMediaPermissionCheckProxy.h"
     34#import "UserMediaPermissionRequestProxy.h"
    3335#import "WKFrameInfoInternal.h"
    3436#import "WKNavigationActionInternal.h"
    … …  
    4749#import <wtf/BlockPtr.h>
    4850
     51#if PLATFORM(IOS)
     52#import <AVFoundation/AVCaptureDevice.h>
     53#import <AVFoundation/AVMediaFormat.h>
     54#import <WebCore/SoftLinking.h>
     55
     56SOFT_LINK_FRAMEWORK(AVFoundation);
     57SOFT_LINK_CLASS(AVFoundation, AVCaptureDevice);
     58SOFT_LINK_CONSTANT(AVFoundation, AVMediaTypeAudio, NSString *);
     59SOFT_LINK_CONSTANT(AVFoundation, AVMediaTypeVideo, NSString *);
     60#endif
     61
    4962namespace WebKit {
    5063
    … …  
    102115    m_delegateMethods.webViewActionsForElementDefaultActions = [delegate respondsToSelector:@selector(_webView:actionsForElement:defaultActions:)];
    103116    m_delegateMethods.webViewDidNotHandleTapAsClickAtPoint = [delegate respondsToSelector:@selector(_webView:didNotHandleTapAsClickAtPoint:)];
     117    m_delegateMethods.webViewRequestUserMediaAuthorizationForMicrophoneCameraURLMainFrameURLDecisionHandler = [delegate respondsToSelector:@selector(_webView:requestUserMediaAuthorizationForMicrophone:camera:url:mainFrameURL:decisionHandler:)];
     118    m_delegateMethods.webViewCheckUserMediaPermissionForURLMainFrameURLFrameIdentifierDecisionHandler = [delegate respondsToSelector:@selector(_webView:checkUserMediaPermissionForURL:mainFrameURL:frameIdentifier:decisionHandler:)];
     119    m_delegateMethods.webViewDidBeginCaptureSession = [delegate respondsToSelector:@selector(_webViewDidBeginCaptureSession:)];
     120    m_delegateMethods.webViewDidEndCaptureSession = [delegate respondsToSelector:@selector(_webViewDidEndCaptureSession:)];
    104121    m_delegateMethods.presentingViewControllerForWebView = [delegate respondsToSelector:@selector(_presentingViewControllerForWebView:)];
    105122#endif
    … …  
    303320}
    304321#endif
     322
     323bool UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest(WebKit::WebPageProxy& page, WebKit::WebFrameProxy& frame, API::SecurityOrigin& userMediaOrigin, API::SecurityOrigin& topLevelOrigin, WebKit::UserMediaPermissionRequestProxy& request)
     324{
     325    auto delegate = m_uiDelegate.m_delegate.get();
     326    if (!delegate || !m_uiDelegate.m_delegateMethods.webViewRequestUserMediaAuthorizationForMicrophoneCameraURLMainFrameURLDecisionHandler) {
     327        request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::UserMediaDisabled);
     328        return true;
     329    }
     330
     331    bool requiresAudio = request.requiresAudio();
     332    bool requiresVideo = request.requiresVideo();
     333    if (!requiresAudio && !requiresVideo) {
     334        request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints);
     335        return true;
     336    }
     337
     338    __block WKWebView *webView = m_uiDelegate.m_webView;
     339    void (^uiDelegateAuthorizationBlock)(void) = ^ {
     340        const WebFrameProxy* mainFrame = frame.page()->mainFrame();
     341        WebCore::URL requestFrameURL(WebCore::URL(), frame.url());
     342        WebCore::URL mainFrameURL(WebCore::URL(), mainFrame->url());
     343
     344        [(id <WKUIDelegatePrivate>)delegate _webView:webView requestUserMediaAuthorizationForMicrophone:requiresAudio camera:requiresVideo url:requestFrameURL mainFrameURL:mainFrameURL decisionHandler:^(BOOL authorizedMicrophone, BOOL authorizedCamera) {
     345            if ((requiresAudio != authorizedMicrophone) || (requiresVideo != authorizedCamera)) {
     346                request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
     347                return;
     348            }
     349            const String& videoDeviceUID = requiresVideo ? request.videoDeviceUIDs().first() : String();
     350            const String& audioDeviceUID = requiresAudio ? request.audioDeviceUIDs().first() : String();
     351            request.allow(audioDeviceUID, videoDeviceUID);
     352        }];
     353    };
     354
     355#if PLATFORM(IOS)
     356    void (^cameraAuthorizationBlock)(void) = ^ {
     357        if (requiresVideo) {
     358            AVAuthorizationStatus cameraAuthorizationStatus = [getAVCaptureDeviceClass() authorizationStatusForMediaType:getAVMediaTypeVideo()];
     359            switch (cameraAuthorizationStatus) {
     360            case AVAuthorizationStatusDenied:
     361            case AVAuthorizationStatusRestricted:
     362                request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
     363                return;
     364            case AVAuthorizationStatusNotDetermined:
     365                [getAVCaptureDeviceClass() requestAccessForMediaType:getAVMediaTypeVideo() completionHandler:^(BOOL authorized) {
     366                    if (!authorized) {
     367                        request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
     368                        return;
     369                    }
     370                    uiDelegateAuthorizationBlock();
     371                }];
     372                break;
     373            default:
     374                uiDelegateAuthorizationBlock();
     375            }
     376        } else
     377            uiDelegateAuthorizationBlock();
     378    };
     379
     380    if (requiresAudio) {
     381        AVAuthorizationStatus microphoneAuthorizationStatus = [getAVCaptureDeviceClass() authorizationStatusForMediaType:getAVMediaTypeAudio()];
     382        switch (microphoneAuthorizationStatus) {
     383        case AVAuthorizationStatusDenied:
     384        case AVAuthorizationStatusRestricted:
     385            request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
     386            return true;
     387        case AVAuthorizationStatusNotDetermined:
     388            [getAVCaptureDeviceClass() requestAccessForMediaType:getAVMediaTypeAudio() completionHandler:^(BOOL authorized) {
     389                if (!authorized) {
     390                    request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
     391                    return;
     392                }
     393                cameraAuthorizationBlock();
     394            }];
     395            break;
     396        default:
     397            cameraAuthorizationBlock();
     398        }
     399    } else
     400        cameraAuthorizationBlock();
     401#else
     402    uiDelegateAuthorizationBlock();
     403#endif
     404
     405    return true;
     406}
     407
     408bool UIDelegate::UIClient::checkUserMediaPermissionForOrigin(WebKit::WebPageProxy& page, WebKit::WebFrameProxy& frame, API::SecurityOrigin& userMediaOrigin, API::SecurityOrigin& topLevelOrigin, WebKit::UserMediaPermissionCheckProxy& request)
     409{
     410    auto delegate = m_uiDelegate.m_delegate.get();
     411    if (!delegate || !m_uiDelegate.m_delegateMethods.webViewCheckUserMediaPermissionForURLMainFrameURLFrameIdentifierDecisionHandler) {
     412        request.setUserMediaAccessInfo(String(), false);
     413        return true;
     414    }
     415
     416    WKWebView *webView = m_uiDelegate.m_webView;
     417    const WebFrameProxy* mainFrame = frame.page()->mainFrame();
     418    WebCore::URL requestFrameURL(WebCore::URL(), frame.url());
     419    WebCore::URL mainFrameURL(WebCore::URL(), mainFrame->url());
     420
     421    [(id <WKUIDelegatePrivate>)delegate _webView:webView checkUserMediaPermissionForURL:requestFrameURL mainFrameURL:mainFrameURL frameIdentifier:frame.frameID() decisionHandler:^(NSString *salt, BOOL authorized) {
     422        request.setUserMediaAccessInfo(String(salt), authorized);
     423    }];
     424
     425    return true;
     426}
     427
     428void UIDelegate::UIClient::didBeginCaptureSession()
     429{
     430    auto delegate = m_uiDelegate.m_delegate.get();
     431    if (!delegate || !m_uiDelegate.m_delegateMethods.webViewDidBeginCaptureSession)
     432        return;
     433
     434    [(id <WKUIDelegatePrivate>)delegate _webViewDidBeginCaptureSession:m_uiDelegate.m_webView];
     435}
     436
     437void UIDelegate::UIClient::didEndCaptureSession()
     438{
     439    auto delegate = m_uiDelegate.m_delegate.get();
     440    if (!delegate || !m_uiDelegate.m_delegateMethods.webViewDidEndCaptureSession)
     441        return;
     442
     443    [(id <WKUIDelegatePrivate>)delegate _webViewDidEndCaptureSession:m_uiDelegate.m_webView];
     444}
    305445
    306446void UIDelegate::UIClient::reachedApplicationCacheOriginQuota(WebPageProxy*, const WebCore::SecurityOrigin& securityOrigin, uint64_t currentQuota, uint64_t totalBytesNeeded, Function<void (unsigned long long)>&& completionHandler)

  • trunk/Source/WebKit2/UIProcess/WebPageProxy.cpp

    r208985 r209010  
    64106410        return;
    64116411
     6412    WebCore::MediaProducer::MediaStateFlags oldMediaStateHasActiveCapture = m_mediaState & (WebCore::MediaProducer::HasActiveAudioCaptureDevice | WebCore::MediaProducer::HasActiveVideoCaptureDevice);
     6413    WebCore::MediaProducer::MediaStateFlags newMediaStateHasActiveCapture = state & (WebCore::MediaProducer::HasActiveAudioCaptureDevice | WebCore::MediaProducer::HasActiveVideoCaptureDevice);
     6414    if (!oldMediaStateHasActiveCapture && newMediaStateHasActiveCapture)
     6415        m_uiClient->didBeginCaptureSession();
     6416    if (oldMediaStateHasActiveCapture && !newMediaStateHasActiveCapture)
     6417        m_uiClient->didEndCaptureSession();
     6418
    64126419    MediaProducer::MediaStateFlags playingMediaMask = MediaProducer::IsPlayingAudio | MediaProducer::IsPlayingVideo;
    64136420    MediaProducer::MediaStateFlags oldState = m_mediaState;
Note: See TracChangeset for help on using the changeset viewer.