⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 246093 in webkit


Ignore:
Timestamp:
Jun 4, 2019, 6:03:04 PM (7 years ago)
Author:
youenn@apple.com
Message:

getUserMedia requests should be processed sequentially in UIProcess
​https://bugs.webkit.org/show_bug.cgi?id=198430
<rdar://problem/51311420>

Reviewed by Eric Carlson.

Source/WebKit:

Before the patch, we process all incoming gum/gdm requests in parallel.
We now queueu them and process them one at a time.
This allows to take into consideration state changes triggered by one request for the next one.
In particular, if a user grants a request, this might grant the next one as well.

To implement that, we keep a reference of the current request to process.
We queue other requests happening whenever another request comes.
When the request is processed, we look at the next one in the queue.
To ensure we do not stop processing the queue for no good reason, some refactoring is done:

  • queue processing happens when sending back IPC response to WebProcess.
  • denyRequest/grantRequest are consistently called in the manager proxy.
  • UIProcess/UserMediaPermissionRequestManagerProxy.cpp:

(WebKit::UserMediaPermissionRequestManagerProxy::invalidatePendingRequests):
Invalidate pregranted requests as well.
(WebKit::UserMediaPermissionRequestManagerProxy::denyRequest):
Renamed from userMediaAccessWasDenied to denyRequest.
This method is now consistently used whenever the manager proxy wants to deny the request.
It does the IPC to the WebProcess and triggers processing of the next request.
(WebKit::UserMediaPermissionRequestManagerProxy::grantRequest):
Renamed from userMediaAccessWasGranted to grantRequest.
(WebKit::UserMediaPermissionRequestManagerProxy::finishGrantingRequest):
This method is now consistently used whenever the manager proxy wants to deny the request.
It does the IPC to the WebProcess and triggers processing of the next request.
(WebKit::UserMediaPermissionRequestManagerProxy::rejectionTimerFired):
We now keep a queue of request instead of request IDs to make the deny code path more consistent.
(WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame):
(WebKit::UserMediaPermissionRequestManagerProxy::processNextUserMediaRequestIfNeeded):
(WebKit::UserMediaPermissionRequestManagerProxy::startProcessingUserMediaPermissionRequest):
(WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionRequest):
To make sure we do not process a different request, we keep a pointer to the request and compare it with the current media request.
(WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionInvalidRequest):
(WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionValidRequest):
(WebKit::UserMediaPermissionRequestManagerProxy::viewIsBecomingVisible):

  • UIProcess/UserMediaPermissionRequestManagerProxy.h:

(WebKit::UserMediaPermissionRequestManagerProxy::denyRequest):

  • UIProcess/UserMediaPermissionRequestProxy.cpp:

(WebKit::setDeviceAsFirst):
(WebKit::UserMediaPermissionRequestProxy::allow):
(WebKit::UserMediaPermissionRequestProxy::deny):

  • UIProcess/UserMediaPermissionRequestProxy.h:

Tools:

  • TestWebKitAPI/Tests/WebKit/GetUserMediaReprompt.mm:

(-[GetUserMediaRepromptUIDelegate _webView:requestMediaCaptureAuthorization:decisionHandler:]):
(TestWebKitAPI::TEST):

  • TestWebKitAPI/Tests/WebKit/getUserMedia.html:
Location:
trunk
Files:
8 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebKit/ChangeLog

    r246090 r246093  
     12019-06-04  Youenn Fablet  <youenn@apple.com>
     2
     3        getUserMedia requests should be processed sequentially in UIProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=198430
     5        <rdar://problem/51311420>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Before the patch, we process all incoming gum/gdm requests in parallel.
     10        We now queueu them and process them one at a time.
     11        This allows to take into consideration state changes triggered by one request for the next one.
     12        In particular, if a user grants a request, this might grant the next one as well.
     13
     14        To implement that, we keep a reference of the current request to process.
     15        We queue other requests happening whenever another request comes.
     16        When the request is processed, we look at the next one in the queue.
     17        To ensure we do not stop processing the queue for no good reason, some refactoring is done:
     18        - queue processing happens when sending back IPC response to WebProcess.
     19        - denyRequest/grantRequest are consistently called in the manager proxy.
     20
     21        * UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
     22        (WebKit::UserMediaPermissionRequestManagerProxy::invalidatePendingRequests):
     23        Invalidate pregranted requests as well.
     24        (WebKit::UserMediaPermissionRequestManagerProxy::denyRequest):
     25        Renamed from userMediaAccessWasDenied to denyRequest.
     26        This method is now consistently used whenever the manager proxy wants to deny the request.
     27        It does the IPC to the WebProcess and triggers processing of the next request.
     28        (WebKit::UserMediaPermissionRequestManagerProxy::grantRequest):
     29        Renamed from userMediaAccessWasGranted to grantRequest.
     30        (WebKit::UserMediaPermissionRequestManagerProxy::finishGrantingRequest):
     31        This method is now consistently used whenever the manager proxy wants to deny the request.
     32        It does the IPC to the WebProcess and triggers processing of the next request.
     33        (WebKit::UserMediaPermissionRequestManagerProxy::rejectionTimerFired):
     34        We now keep a queue of request instead of request IDs to make the deny code path more consistent.
     35        (WebKit::UserMediaPermissionRequestManagerProxy::requestUserMediaPermissionForFrame):
     36        (WebKit::UserMediaPermissionRequestManagerProxy::processNextUserMediaRequestIfNeeded):
     37        (WebKit::UserMediaPermissionRequestManagerProxy::startProcessingUserMediaPermissionRequest):
     38        (WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionRequest):
     39        To make sure we do not process a different request, we keep a pointer to the request and compare it with the current media request.
     40        (WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionInvalidRequest):
     41        (WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionValidRequest):
     42        (WebKit::UserMediaPermissionRequestManagerProxy::viewIsBecomingVisible):
     43        * UIProcess/UserMediaPermissionRequestManagerProxy.h:
     44        (WebKit::UserMediaPermissionRequestManagerProxy::denyRequest):
     45        * UIProcess/UserMediaPermissionRequestProxy.cpp:
     46        (WebKit::setDeviceAsFirst):
     47        (WebKit::UserMediaPermissionRequestProxy::allow):
     48        (WebKit::UserMediaPermissionRequestProxy::deny):
     49        * UIProcess/UserMediaPermissionRequestProxy.h:
     50
    1512019-06-04  Commit Queue  <commit-queue@webkit.org>
    252

  • trunk/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp

    r245335 r246093  
    9292void UserMediaPermissionRequestManagerProxy::invalidatePendingRequests()
    9393{
    94     for (auto& request : m_pendingUserMediaRequests.values())
     94    if (m_currentUserMediaRequest) {
     95        m_currentUserMediaRequest->invalidate();
     96        m_currentUserMediaRequest = nullptr;
     97    }
     98
     99    auto pendingUserMediaRequests = WTFMove(m_pendingUserMediaRequests);
     100    for (auto& request : pendingUserMediaRequests)
    95101        request->invalidate();
    96     m_pendingUserMediaRequests.clear();
     102
     103    auto pregrantedRequests = WTFMove(m_pregrantedRequests);
     104    for (auto& request : pregrantedRequests)
     105        request->invalidate();
    97106
    98107    m_pendingDeviceRequests.clear();
    … …  
    159168#endif
    160169
    161 void UserMediaPermissionRequestManagerProxy::userMediaAccessWasDenied(uint64_t userMediaID, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason reason)
     170void UserMediaPermissionRequestManagerProxy::denyRequest(UserMediaPermissionRequestProxy& request, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason reason, const String& invalidConstraint)
    162171{
    163172    if (!m_page.hasRunningProcess())
    164173        return;
    165174
    166     ALWAYS_LOG(LOGIDENTIFIER, userMediaID, ", reason: ", reason);
    167 
    168     auto request = m_pendingUserMediaRequests.take(userMediaID);
    169     if (!request)
    170         return;
     175    ALWAYS_LOG(LOGIDENTIFIER, request.userMediaID(), ", reason: ", reason);
    171176
    172177    if (reason == UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied)
    173         m_deniedRequests.append(DeniedRequest { request->mainFrameID(), request->userMediaDocumentSecurityOrigin(), request->topLevelDocumentSecurityOrigin(), request->requiresAudioCapture(), request->requiresVideoCapture(), request->requiresDisplayCapture() });
    174 
    175     denyRequest(userMediaID, reason, emptyString());
    176 }
    177 
    178 void UserMediaPermissionRequestManagerProxy::denyRequest(uint64_t userMediaID, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason reason, const String& invalidConstraint)
    179 {
    180     ASSERT(m_page.hasRunningProcess());
    181 
    182     ALWAYS_LOG(LOGIDENTIFIER, userMediaID, ", reason: ", reason);
    183 
    184 #if ENABLE(MEDIA_STREAM)
    185     m_page.process().send(Messages::WebPage::UserMediaAccessWasDenied(userMediaID, toWebCore(reason), invalidConstraint), m_page.pageID());
     178        m_deniedRequests.append(DeniedRequest { request.mainFrameID(), request.userMediaDocumentSecurityOrigin(), request.topLevelDocumentSecurityOrigin(), request.requiresAudioCapture(), request.requiresVideoCapture(), request.requiresDisplayCapture() });
     179
     180#if ENABLE(MEDIA_STREAM)
     181    m_page.process().send(Messages::WebPage::UserMediaAccessWasDenied(request.userMediaID(), toWebCore(reason), invalidConstraint), m_page.pageID());
    186182#else
    187183    UNUSED_PARAM(reason);
    188184    UNUSED_PARAM(invalidConstraint);
    189185#endif
    190 }
    191 
    192 void UserMediaPermissionRequestManagerProxy::userMediaAccessWasGranted(uint64_t userMediaID, CaptureDevice&& audioDevice, CaptureDevice&& videoDevice)
    193 {
    194     ASSERT(audioDevice || videoDevice);
    195 
     186
     187    processNextUserMediaRequestIfNeeded();
     188}
     189
     190void UserMediaPermissionRequestManagerProxy::grantRequest(UserMediaPermissionRequestProxy& request)
     191{
    196192    if (!m_page.hasRunningProcess())
    197193        return;
    198194
    199195#if ENABLE(MEDIA_STREAM)
    200     auto logSiteIdentifier = LOGIDENTIFIER;
    201     ALWAYS_LOG(logSiteIdentifier, userMediaID, ", video: ", videoDevice ? videoDevice.label() : "", ", audio: ", audioDevice ? audioDevice.label() : " ");
    202 
    203     auto request = m_pendingUserMediaRequests.take(userMediaID);
    204     if (!request)
    205         return;
    206 
    207     auto& userMediaDocumentSecurityOrigin = request->userMediaDocumentSecurityOrigin();
    208     auto& topLevelDocumentSecurityOrigin = request->topLevelDocumentSecurityOrigin();
    209     m_page.websiteDataStore().deviceIdHashSaltStorage().deviceIdHashSaltForOrigin(userMediaDocumentSecurityOrigin, topLevelDocumentSecurityOrigin, [this, weakThis = makeWeakPtr(*this), request = request.releaseNonNull(), logSiteIdentifier] (String&& deviceIDHashSalt) mutable {
     196    ALWAYS_LOG(LOGIDENTIFIER, request.userMediaID(), ", video: ", request.videoDevice().label(), ", audio: ", request.audioDevice().label());
     197
     198    auto& userMediaDocumentSecurityOrigin = request.userMediaDocumentSecurityOrigin();
     199    auto& topLevelDocumentSecurityOrigin = request.topLevelDocumentSecurityOrigin();
     200    m_page.websiteDataStore().deviceIdHashSaltStorage().deviceIdHashSaltForOrigin(userMediaDocumentSecurityOrigin, topLevelDocumentSecurityOrigin, [this, weakThis = makeWeakPtr(*this), request = makeRef(request)](String&&) mutable {
    210201        if (!weakThis)
    211202            return;
    212         if (!grantAccess(request))
    213             return;
    214 
    215         ALWAYS_LOG(logSiteIdentifier, deviceIDHashSalt);
    216         m_grantedRequests.append(WTFMove(request));
    217         if (m_hasFilteredDeviceList)
    218             captureDevicesChanged();
    219         m_hasFilteredDeviceList = false;
     203        finishGrantingRequest(request);
    220204    });
    221205#else
    222     UNUSED_PARAM(userMediaID);
    223     UNUSED_PARAM(audioDevice);
    224     UNUSED_PARAM(videoDevice);
    225 #endif
    226 }
    227 
    228 #if ENABLE(MEDIA_STREAM)
     206    UNUSED_PARAM(request);
     207#endif
     208}
     209
     210#if ENABLE(MEDIA_STREAM)
     211void UserMediaPermissionRequestManagerProxy::finishGrantingRequest(UserMediaPermissionRequestProxy& request)
     212{
     213    ALWAYS_LOG(LOGIDENTIFIER, request.userMediaID());
     214    if (!UserMediaProcessManager::singleton().willCreateMediaStream(*this, request.hasAudioDevice(), request.hasVideoDevice())) {
     215        denyRequest(request, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::OtherFailure, "Unable to extend sandbox.");
     216        return;
     217    }
     218
     219    if (request.requestType() == MediaStreamRequest::Type::UserMedia)
     220        m_grantedRequests.append(makeRef(request));
     221
     222    if (m_hasFilteredDeviceList)
     223        captureDevicesChanged();
     224    m_hasFilteredDeviceList = false;
     225
     226    ++m_hasPendingCapture;
     227    m_page.process().connection()->sendWithAsyncReply(Messages::WebPage::UserMediaAccessWasGranted { request.userMediaID(), request.audioDevice(), request.videoDevice(), request.deviceIdentifierHashSalt() }, [this, weakThis = makeWeakPtr(this)] {
     228        if (!weakThis)
     229            return;
     230        --m_hasPendingCapture;
     231    }, m_page.pageID());
     232
     233    processNextUserMediaRequestIfNeeded();
     234}
     235
    229236void UserMediaPermissionRequestManagerProxy::resetAccess(uint64_t frameID)
    230237{
    … …  
    288295}
    289296
    290 bool UserMediaPermissionRequestManagerProxy::grantAccess(const UserMediaPermissionRequestProxy& request)
    291 {
    292     ALWAYS_LOG(LOGIDENTIFIER, request.userMediaID());
    293     if (!UserMediaProcessManager::singleton().willCreateMediaStream(*this, request.hasAudioDevice(), request.hasVideoDevice())) {
    294         denyRequest(request.userMediaID(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::OtherFailure, "Unable to extend sandbox.");
    295         return false;
    296     }
    297 
    298     ++m_hasPendingCapture;
    299     m_page.process().connection()->sendWithAsyncReply(Messages::WebPage::UserMediaAccessWasGranted { request.userMediaID(), request.audioDevice(), request.videoDevice(), request.deviceIdentifierHashSalt() }, [this, weakThis = makeWeakPtr(this)] {
    300         if (!weakThis)
    301             return;
    302         --m_hasPendingCapture;
    303     }, m_page.pageID());
    304     return true;
    305 }
    306297#endif
    307298
    308299void UserMediaPermissionRequestManagerProxy::rejectionTimerFired()
    309300{
    310     uint64_t userMediaID = m_pendingRejections[0];
    311     m_pendingRejections.remove(0);
    312 
    313     denyRequest(userMediaID, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied, emptyString());
     301    denyRequest(m_pendingRejections.takeFirst(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied, emptyString());
    314302    if (!m_pendingRejections.isEmpty())
    315303        scheduleNextRejection();
    … …  
    348336    auto logSiteIdentifier = LOGIDENTIFIER;
    349337
     338    if (!m_page.hasRunningProcess())
     339        return;
     340
     341    ALWAYS_LOG(logSiteIdentifier, userMediaID);
     342
     343    auto request = UserMediaPermissionRequestProxy::create(*this, userMediaID, m_page.mainFrame()->frameID(), frameID, WTFMove(userMediaDocumentOrigin), WTFMove(topLevelDocumentOrigin), { }, { }, WTFMove(userRequest));
     344    if (m_currentUserMediaRequest) {
     345        m_pendingUserMediaRequests.append(WTFMove(request));
     346        return;
     347    }
     348
    350349    if (!UserMediaProcessManager::singleton().captureEnabled()) {
    351350        ALWAYS_LOG(logSiteIdentifier, "capture disabled");
    352         m_pendingRejections.append(userMediaID);
     351        m_pendingRejections.append(WTFMove(request));
    353352        scheduleNextRejection();
    354353        return;
    355354    }
    356355
    357     if (!m_page.hasRunningProcess())
    358         return;
    359 
    360     ALWAYS_LOG(logSiteIdentifier, userMediaID);
    361 
    362     auto request = m_pendingUserMediaRequests.add(userMediaID, UserMediaPermissionRequestProxy::create(*this, userMediaID, m_page.mainFrame()->frameID(), frameID, WTFMove(userMediaDocumentOrigin), WTFMove(topLevelDocumentOrigin), { }, { }, WTFMove(userRequest))).iterator->value.copyRef();
    363 
    364     auto& userMediaOrigin = request->userMediaDocumentSecurityOrigin();
    365     auto& topLevelOrigin = request->topLevelDocumentSecurityOrigin();
    366     getUserMediaPermissionInfo(frameID, userMediaOrigin, topLevelOrigin, [this, request = request.releaseNonNull(), logSiteIdentifier](Optional<bool> hasPersistentAccess) mutable {
    367         if (!request->isPending())
    368             return;
    369 
    370         if (!hasPersistentAccess) {
    371             request->deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::OtherFailure);
    372             return;
    373         }
    374 
    375         ALWAYS_LOG(logSiteIdentifier, request->userMediaID(), ", persistent access: ", *hasPersistentAccess);
    376         processUserMediaPermissionRequest(WTFMove(request), *hasPersistentAccess);
    377     });
    378 }
    379 
    380 void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionRequest(Ref<UserMediaPermissionRequestProxy>&& request, bool hasPersistentAccess)
    381 {
    382     ALWAYS_LOG(LOGIDENTIFIER, request->userMediaID());
    383 
    384     if (hasPersistentAccess)
    385         request->setHasPersistentAccess();
    386 
    387     auto& userMediaDocumentSecurityOrigin = request->userMediaDocumentSecurityOrigin();
    388     auto& topLevelDocumentSecurityOrigin = request->topLevelDocumentSecurityOrigin();
    389     m_page.websiteDataStore().deviceIdHashSaltStorage().deviceIdHashSaltForOrigin(userMediaDocumentSecurityOrigin, topLevelDocumentSecurityOrigin, [this, request = WTFMove(request)] (String&& deviceIDHashSalt) mutable {
    390         if (!request->isPending())
    391             return;
    392 
    393         RealtimeMediaSourceCenter::InvalidConstraintsHandler invalidHandler = [this, request = request.copyRef()](const String& invalidConstraint) {
    394             if (!request->isPending())
    395                 return;
    396 
    397             if (!m_page.hasRunningProcess())
    398                 return;
    399 
    400             processUserMediaPermissionInvalidRequest(request.get(), invalidConstraint);
    401         };
    402 
    403         auto validHandler = [this, request = request.copyRef()](Vector<CaptureDevice>&& audioDevices, Vector<CaptureDevice>&& videoDevices, String&& deviceIdentifierHashSalt) mutable {
    404             if (!request->isPending())
    405                 return;
    406 
    407             if (!m_page.hasRunningProcess() || !m_page.mainFrame())
    408                 return;
    409 
    410             processUserMediaPermissionValidRequest(WTFMove(request), WTFMove(audioDevices), WTFMove(videoDevices), WTFMove(deviceIdentifierHashSalt));
    411         };
    412 
    413         syncWithWebCorePrefs();
    414 
    415         RealtimeMediaSourceCenter::singleton().validateRequestConstraints(WTFMove(validHandler), WTFMove(invalidHandler), request->userRequest(), WTFMove(deviceIDHashSalt));
    416     });
     356    startProcessingUserMediaPermissionRequest(WTFMove(request));
    417357#else
    418358    UNUSED_PARAM(userMediaID);
    … …  
    424364}
    425365
    426 #if ENABLE(MEDIA_STREAM)
    427 void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionInvalidRequest(const UserMediaPermissionRequestProxy& request, const String& invalidConstraint)
    428 {
    429     ALWAYS_LOG(LOGIDENTIFIER, request.userMediaID());
    430     bool filterConstraint = !request.hasPersistentAccess() && !wasGrantedVideoOrAudioAccess(request.frameID(), request.userMediaDocumentSecurityOrigin(), request.topLevelDocumentSecurityOrigin());
    431 
    432     denyRequest(request.userMediaID(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::InvalidConstraint, filterConstraint ? String { } : invalidConstraint);
    433 }
    434 
    435 void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionValidRequest(Ref<UserMediaPermissionRequestProxy>&& request, Vector<CaptureDevice>&& audioDevices, Vector<CaptureDevice>&& videoDevices, String&& deviceIdentifierHashSalt)
    436 {
    437     ALWAYS_LOG(LOGIDENTIFIER, request->userMediaID(), ", video: ", videoDevices.size(), " audio: ", audioDevices.size());
     366void UserMediaPermissionRequestManagerProxy::processNextUserMediaRequestIfNeeded()
     367{
     368#if ENABLE(MEDIA_STREAM)
     369    if (m_pendingUserMediaRequests.isEmpty()) {
     370        m_currentUserMediaRequest = nullptr;
     371        return;
     372    }
     373    startProcessingUserMediaPermissionRequest(m_pendingUserMediaRequests.takeFirst());
     374#endif
     375}
     376
     377#if ENABLE(MEDIA_STREAM)
     378void UserMediaPermissionRequestManagerProxy::startProcessingUserMediaPermissionRequest(Ref<UserMediaPermissionRequestProxy>&& request)
     379{
     380    m_currentUserMediaRequest = WTFMove(request);
     381
     382    auto& userMediaDocumentSecurityOrigin = m_currentUserMediaRequest->userMediaDocumentSecurityOrigin();
     383    auto& topLevelDocumentSecurityOrigin = m_currentUserMediaRequest->topLevelDocumentSecurityOrigin();
     384    getUserMediaPermissionInfo(m_currentUserMediaRequest->frameID(), userMediaDocumentSecurityOrigin, topLevelDocumentSecurityOrigin, [this, request = m_currentUserMediaRequest](Optional<bool> hasPersistentAccess) mutable {
     385        if (!request->isPending())
     386            return;
     387
     388        if (!hasPersistentAccess) {
     389            denyRequest(*m_currentUserMediaRequest, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::OtherFailure);
     390            return;
     391        }
     392
     393        ALWAYS_LOG(LOGIDENTIFIER, m_currentUserMediaRequest->userMediaID(), ", persistent access: ", *hasPersistentAccess);
     394        processUserMediaPermissionRequest(*hasPersistentAccess);
     395    });
     396}
     397
     398void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionRequest(bool hasPersistentAccess)
     399{
     400    ALWAYS_LOG(LOGIDENTIFIER, m_currentUserMediaRequest->userMediaID());
     401
     402    if (hasPersistentAccess)
     403        m_currentUserMediaRequest->setHasPersistentAccess();
     404
     405    auto& userMediaDocumentSecurityOrigin = m_currentUserMediaRequest->userMediaDocumentSecurityOrigin();
     406    auto& topLevelDocumentSecurityOrigin = m_currentUserMediaRequest->topLevelDocumentSecurityOrigin();
     407    m_page.websiteDataStore().deviceIdHashSaltStorage().deviceIdHashSaltForOrigin(userMediaDocumentSecurityOrigin, topLevelDocumentSecurityOrigin, [this, request = m_currentUserMediaRequest] (String&& deviceIDHashSalt) mutable {
     408        if (!request->isPending())
     409            return;
     410
     411        RealtimeMediaSourceCenter::InvalidConstraintsHandler invalidHandler = [this, request](const String& invalidConstraint) {
     412            if (!request->isPending())
     413                return;
     414
     415            if (!m_page.hasRunningProcess())
     416                return;
     417
     418            processUserMediaPermissionInvalidRequest(invalidConstraint);
     419        };
     420
     421        auto validHandler = [this, request](Vector<CaptureDevice>&& audioDevices, Vector<CaptureDevice>&& videoDevices, String&& deviceIdentifierHashSalt) mutable {
     422            if (!request->isPending())
     423                return;
     424
     425            if (!m_page.hasRunningProcess() || !m_page.mainFrame())
     426                return;
     427
     428            processUserMediaPermissionValidRequest(WTFMove(audioDevices), WTFMove(videoDevices), WTFMove(deviceIdentifierHashSalt));
     429        };
     430
     431        syncWithWebCorePrefs();
     432
     433        RealtimeMediaSourceCenter::singleton().validateRequestConstraints(WTFMove(validHandler), WTFMove(invalidHandler), m_currentUserMediaRequest->userRequest(), WTFMove(deviceIDHashSalt));
     434    });
     435}
     436#endif
     437
     438#if ENABLE(MEDIA_STREAM)
     439void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionInvalidRequest(const String& invalidConstraint)
     440{
     441    ALWAYS_LOG(LOGIDENTIFIER, m_currentUserMediaRequest->userMediaID());
     442    bool filterConstraint = !m_currentUserMediaRequest->hasPersistentAccess() && !wasGrantedVideoOrAudioAccess(m_currentUserMediaRequest->frameID(), m_currentUserMediaRequest->userMediaDocumentSecurityOrigin(), m_currentUserMediaRequest->topLevelDocumentSecurityOrigin());
     443
     444    denyRequest(*m_currentUserMediaRequest, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::InvalidConstraint, filterConstraint ? String { } : invalidConstraint);
     445}
     446
     447void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionValidRequest(Vector<CaptureDevice>&& audioDevices, Vector<CaptureDevice>&& videoDevices, String&& deviceIdentifierHashSalt)
     448{
     449    ALWAYS_LOG(LOGIDENTIFIER, m_currentUserMediaRequest->userMediaID(), ", video: ", videoDevices.size(), " audio: ", audioDevices.size());
    438450    if (videoDevices.isEmpty() && audioDevices.isEmpty()) {
    439         denyRequest(request->userMediaID(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints, emptyString());
    440         return;
    441     }
    442 
    443     request->setDeviceIdentifierHashSalt(WTFMove(deviceIdentifierHashSalt));
    444     request->setEligibleVideoDeviceUIDs(WTFMove(videoDevices));
    445     request->setEligibleAudioDeviceUIDs(WTFMove(audioDevices));
    446 
    447     auto action = getRequestAction(request);
    448     ALWAYS_LOG(LOGIDENTIFIER, request->userMediaID(), ", action: ", action);
     451        denyRequest(*m_currentUserMediaRequest, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints, emptyString());
     452        return;
     453    }
     454
     455    m_currentUserMediaRequest->setDeviceIdentifierHashSalt(WTFMove(deviceIdentifierHashSalt));
     456    m_currentUserMediaRequest->setEligibleVideoDeviceUIDs(WTFMove(videoDevices));
     457    m_currentUserMediaRequest->setEligibleAudioDeviceUIDs(WTFMove(audioDevices));
     458
     459    auto action = getRequestAction(*m_currentUserMediaRequest);
     460    ALWAYS_LOG(LOGIDENTIFIER, m_currentUserMediaRequest->userMediaID(), ", action: ", action);
    449461
    450462    if (action == RequestAction::Deny) {
    451         denyRequest(request->userMediaID(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied, emptyString());
     463        denyRequest(*m_currentUserMediaRequest, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied, emptyString());
    452464        return;
    453465    }
    454466
    455467    if (action == RequestAction::Grant) {
    456         ASSERT(request->requestType() != MediaStreamRequest::Type::DisplayMedia);
     468        ASSERT(m_currentUserMediaRequest->requestType() != MediaStreamRequest::Type::DisplayMedia);
    457469
    458470        if (m_page.isViewVisible())
    459             grantAccess(request);
     471            grantRequest(*m_currentUserMediaRequest);
    460472        else
    461             m_pregrantedRequests.append(WTFMove(request));
     473            m_pregrantedRequests.append(m_currentUserMediaRequest.releaseNonNull());
    462474
    463475        return;
    … …  
    466478    if (m_page.isControlledByAutomation()) {
    467479        if (WebAutomationSession* automationSession = m_page.process().processPool().automationSession()) {
    468             ALWAYS_LOG(LOGIDENTIFIER, request->userMediaID(), ", page controlled by automation");
     480            ALWAYS_LOG(LOGIDENTIFIER, m_currentUserMediaRequest->userMediaID(), ", page controlled by automation");
    469481            if (automationSession->shouldAllowGetUserMediaForPage(m_page))
    470                 request->allow();
     482                grantRequest(*m_currentUserMediaRequest);
    471483            else
    472                 userMediaAccessWasDenied(request->userMediaID(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
    473 
     484                denyRequest(*m_currentUserMediaRequest, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
    474485            return;
    475486        }
    … …  
    477488
    478489    if (m_page.preferences().mockCaptureDevicesEnabled() && !m_page.preferences().mockCaptureDevicesPromptEnabled()) {
    479         ALWAYS_LOG(LOGIDENTIFIER, request->userMediaID(), ", mock devices don't require prompt");
    480         request->allow();
     490        ALWAYS_LOG(LOGIDENTIFIER, m_currentUserMediaRequest->userMediaID(), ", mock devices don't require prompt");
     491        grantRequest(*m_currentUserMediaRequest);
    481492        return;
    482493    }
    483494
    484495    // If page navigated, there is no need to call the page client for authorization.
    485     auto* webFrame = m_page.process().webFrame(request->frameID());
    486 
    487     if (!webFrame || !SecurityOrigin::createFromString(m_page.pageLoadState().activeURL())->isSameSchemeHostPort(request->topLevelDocumentSecurityOrigin())) {
    488         denyRequest(request->userMediaID(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints, emptyString());
     496    auto* webFrame = m_page.process().webFrame(m_currentUserMediaRequest->frameID());
     497
     498    if (!webFrame || !SecurityOrigin::createFromString(m_page.pageLoadState().activeURL())->isSameSchemeHostPort(m_currentUserMediaRequest->topLevelDocumentSecurityOrigin())) {
     499        denyRequest(*m_currentUserMediaRequest, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints, emptyString());
    489500        return;
    490501    }
    491502
    492503    // FIXME: Remove webFrame, userMediaOrigin and topLevelOrigin from this uiClient API call.
    493     auto userMediaOrigin = API::SecurityOrigin::create(request->userMediaDocumentSecurityOrigin());
    494     auto topLevelOrigin = API::SecurityOrigin::create(request->topLevelDocumentSecurityOrigin());
    495     m_page.uiClient().decidePolicyForUserMediaPermissionRequest(m_page, *webFrame, WTFMove(userMediaOrigin), WTFMove(topLevelOrigin), request);
     504    auto userMediaOrigin = API::SecurityOrigin::create(m_currentUserMediaRequest->userMediaDocumentSecurityOrigin());
     505    auto topLevelOrigin = API::SecurityOrigin::create(m_currentUserMediaRequest->topLevelDocumentSecurityOrigin());
     506    m_page.uiClient().decidePolicyForUserMediaPermissionRequest(m_page, *webFrame, WTFMove(userMediaOrigin), WTFMove(topLevelOrigin), *m_currentUserMediaRequest);
    496507}
    497508
    … …  
    664675void UserMediaPermissionRequestManagerProxy::viewIsBecomingVisible()
    665676{
    666     for (auto& request : m_pregrantedRequests)
    667         request->allow();
    668     m_pregrantedRequests.clear();
     677    auto pregrantedRequests = WTFMove(m_pregrantedRequests);
     678    for (auto& request : pregrantedRequests)
     679        grantRequest(request);
    669680}
    670681

  • trunk/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.h

    r245335 r246093  
    2525#include <WebCore/SecurityOrigin.h>
    2626#include <wtf/CompletionHandler.h>
     27#include <wtf/Deque.h>
    2728#include <wtf/HashMap.h>
    2829#include <wtf/LoggerHelper.h>
    … …  
    6566    void viewIsBecomingVisible();
    6667
    67     void userMediaAccessWasGranted(uint64_t, WebCore::CaptureDevice&& audioDevice, WebCore::CaptureDevice&& videoDevice);
    68     void userMediaAccessWasDenied(uint64_t, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason);
     68    void grantRequest(UserMediaPermissionRequestProxy&);
     69    void denyRequest(UserMediaPermissionRequestProxy&, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason, const String& invalidConstraint = { });
    6970
    7071    void enumerateMediaDevicesForFrame(uint64_t userMediaID, uint64_t frameID, Ref<WebCore::SecurityOrigin>&& userMediaDocumentOrigin, Ref<WebCore::SecurityOrigin>&& topLevelDocumentOrigin);
    … …  
    9495
    9596    Ref<UserMediaPermissionRequestProxy> createPermissionRequest(uint64_t userMediaID, uint64_t mainFrameID, uint64_t frameID, Ref<WebCore::SecurityOrigin>&& userMediaDocumentOrigin, Ref<WebCore::SecurityOrigin>&& topLevelDocumentOrigin, Vector<WebCore::CaptureDevice>&& audioDevices, Vector<WebCore::CaptureDevice>&& videoDevices, WebCore::MediaStreamRequest&&);
    96     void denyRequest(uint64_t userMediaID, UserMediaPermissionRequestProxy::UserMediaAccessDenialReason, const String& invalidConstraint);
    9797#if ENABLE(MEDIA_STREAM)
    98     bool grantAccess(const UserMediaPermissionRequestProxy&);
     98    void finishGrantingRequest(UserMediaPermissionRequestProxy&);
    9999
    100100    const UserMediaPermissionRequestProxy* searchForGrantedRequest(uint64_t frameID, const WebCore::SecurityOrigin& userMediaDocumentOrigin, const WebCore::SecurityOrigin& topLevelDocumentOrigin, bool needsAudio, bool needsVideo) const;
    … …  
    109109    Vector<WebCore::CaptureDevice> computeFilteredDeviceList(bool revealIdsAndLabels, const String& deviceIDHashSalt);
    110110
    111     void processUserMediaPermissionRequest(Ref<UserMediaPermissionRequestProxy>&&, bool hasPersistentAccess);
    112     void processUserMediaPermissionInvalidRequest(const UserMediaPermissionRequestProxy&, const String& invalidConstraint);
    113     void processUserMediaPermissionValidRequest(Ref<UserMediaPermissionRequestProxy>&&, Vector<WebCore::CaptureDevice>&& audioDevices, Vector<WebCore::CaptureDevice>&& videoDevices, String&& deviceIdentifierHashSalt);
     111    void processUserMediaPermissionRequest(bool hasPersistentAccess);
     112    void processUserMediaPermissionInvalidRequest(const String& invalidConstraint);
     113    void processUserMediaPermissionValidRequest(Vector<WebCore::CaptureDevice>&& audioDevices, Vector<WebCore::CaptureDevice>&& videoDevices, String&& deviceIdentifierHashSalt);
     114    void startProcessingUserMediaPermissionRequest(Ref<UserMediaPermissionRequestProxy>&&);
    114115#endif
    115116
    116117    void watchdogTimerFired();
    117118
    118     HashMap<uint64_t, RefPtr<UserMediaPermissionRequestProxy>> m_pendingUserMediaRequests;
     119    void processNextUserMediaRequestIfNeeded();
     120
     121    RefPtr<UserMediaPermissionRequestProxy> m_currentUserMediaRequest;
     122    Deque<Ref<UserMediaPermissionRequestProxy>> m_pendingUserMediaRequests;
    119123    HashSet<uint64_t> m_pendingDeviceRequests;
    120124
    … …  
    122126
    123127    RunLoop::Timer<UserMediaPermissionRequestManagerProxy> m_rejectionTimer;
    124     Vector<uint64_t> m_pendingRejections;
     128    Deque<Ref<UserMediaPermissionRequestProxy>> m_pendingRejections;
    125129
    126130    Vector<Ref<UserMediaPermissionRequestProxy>> m_pregrantedRequests;

  • trunk/Source/WebKit/UIProcess/UserMediaPermissionRequestProxy.cpp

    r243328 r246093  
    4444}
    4545
     46#if ENABLE(MEDIA_STREAM)
     47static inline void setDeviceAsFirst(Vector<CaptureDevice>& devices, const String& deviceID)
     48{
     49    size_t index = devices.findMatching([&deviceID](const auto& device) {
     50        return device.persistentId() == deviceID;
     51    });
     52    ASSERT(index != notFound);
     53
     54    if (index) {
     55        auto device = devices[index];
     56        ASSERT(device.enabled());
     57
     58        devices.remove(index);
     59        devices.insert(0, WTFMove(device));
     60    }
     61}
     62#endif
     63
    4664void UserMediaPermissionRequestProxy::allow(const String& audioDeviceUID, const String& videoDeviceUID)
    4765{
    48     ASSERT(m_manager);
    49     if (!m_manager)
    50         return;
    51 
    5266#if ENABLE(MEDIA_STREAM)
    53     CaptureDevice audioDevice;
    54     if (!audioDeviceUID.isEmpty()) {
    55         size_t index = m_eligibleAudioDevices.findMatching([&](const auto& device) {
    56             return device.persistentId() == audioDeviceUID;
    57         });
    58         ASSERT(index != notFound);
    59 
    60         if (index != notFound)
    61             audioDevice = m_eligibleAudioDevices[index];
    62 
    63         ASSERT(audioDevice.enabled());
    64     }
    65 
    66     CaptureDevice videoDevice;
    67     if (!videoDeviceUID.isEmpty()) {
    68         size_t index = m_eligibleVideoDevices.findMatching([&](const auto& device) {
    69             return device.persistentId() == videoDeviceUID;
    70         });
    71         ASSERT(index != notFound);
    72 
    73         if (index != notFound)
    74             videoDevice = m_eligibleVideoDevices[index];
    75 
    76         ASSERT(videoDevice.enabled());
    77     }
    78 
    79     m_manager->userMediaAccessWasGranted(m_userMediaID, WTFMove(audioDevice), WTFMove(videoDevice));
     67    if (!audioDeviceUID.isEmpty())
     68        setDeviceAsFirst(m_eligibleAudioDevices, audioDeviceUID);
     69    if (!videoDeviceUID.isEmpty())
     70        setDeviceAsFirst(m_eligibleVideoDevices, videoDeviceUID);
    8071#else
    8172    UNUSED_PARAM(audioDeviceUID);
    … …  
    8374#endif
    8475
    85     invalidate();
    86 }
    87 
    88 void UserMediaPermissionRequestProxy::allow(WebCore::CaptureDevice&& audioDevice, WebCore::CaptureDevice&& videoDevice)
    89 {
    90     ASSERT(m_manager);
    91     if (!m_manager)
    92         return;
    93 
    94     m_manager->userMediaAccessWasGranted(m_userMediaID, WTFMove(audioDevice), WTFMove(videoDevice));
    95     invalidate();
     76    allow();
    9677}
    9778
    … …  
    10283        return;
    10384
    104     auto audioDevice = !m_eligibleAudioDevices.isEmpty() ? m_eligibleAudioDevices[0] : CaptureDevice();
    105     auto videoDevice = !m_eligibleVideoDevices.isEmpty() ? m_eligibleVideoDevices[0] : CaptureDevice();
    106 
    107     m_manager->userMediaAccessWasGranted(m_userMediaID, WTFMove(audioDevice), WTFMove(videoDevice));
     85    m_manager->grantRequest(*this);
    10886    invalidate();
    10987}
    … …  
    11492        return;
    11593
    116     m_manager->userMediaAccessWasDenied(m_userMediaID, reason);
     94    m_manager->denyRequest(*this, reason);
    11795    invalidate();
    11896}

  • trunk/Source/WebKit/UIProcess/UserMediaPermissionRequestProxy.h

    r243328 r246093  
    4242
    4343    void allow(const String& audioDeviceUID, const String& videoDeviceUID);
    44     void allow(WebCore::CaptureDevice&& audioDevice, WebCore::CaptureDevice&& videoDevice);
    4544    void allow();
    4645

  • trunk/Tools/ChangeLog

    r246085 r246093  
     12019-06-04  Youenn Fablet  <youenn@apple.com>
     2
     3        getUserMedia requests should be processed sequentially in UIProcess
     4        https://bugs.webkit.org/show_bug.cgi?id=198430
     5        <rdar://problem/51311420>
     6
     7        Reviewed by Eric Carlson.
     8
     9        * TestWebKitAPI/Tests/WebKit/GetUserMediaReprompt.mm:
     10        (-[GetUserMediaRepromptUIDelegate _webView:requestMediaCaptureAuthorization:decisionHandler:]):
     11        (TestWebKitAPI::TEST):
     12        * TestWebKitAPI/Tests/WebKit/getUserMedia.html:
     13
    1142019-06-04  David Kilzer  <ddkilzer@apple.com>
    215

  • trunk/Tools/TestWebKitAPI/Tests/WebKit/GetUserMediaReprompt.mm

    r242339 r246093  
    3838
    3939static bool wasPrompted = false;
     40static int numberOfPrompts = 0;
    4041
    4142@interface GetUserMediaRepromptUIDelegate : NSObject<WKUIDelegate>
    … …  
    4748- (void)_webView:(WKWebView *)webView requestMediaCaptureAuthorization: (_WKCaptureDevices)devices decisionHandler:(void (^)(BOOL))decisionHandler
    4849{
     50    numberOfPrompts++;
    4951    wasPrompted = true;
    5052    decisionHandler(YES);
    … …  
    118120}
    119121
     122TEST(WebKit2, MultipleGetUserMediaSynchronously)
     123{
     124    auto configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
     125    auto processPoolConfig = adoptNS([[_WKProcessPoolConfiguration alloc] init]);
     126    auto preferences = [configuration preferences];
     127    preferences._mediaCaptureRequiresSecureConnection = NO;
     128    preferences._mediaDevicesEnabled = YES;
     129    preferences._mockCaptureDevicesEnabled = YES;
     130    auto webView = [[GetUserMediaRepromptTestView alloc] initWithFrame:CGRectMake(0, 0, 320, 500) configuration:configuration.get() processPoolConfiguration:processPoolConfig.get()];
     131    auto delegate = adoptNS([[GetUserMediaRepromptUIDelegate alloc] init]);
     132    webView.UIDelegate = delegate.get();
     133
     134    wasPrompted = false;
     135    numberOfPrompts = 0;
     136    [webView loadTestPageNamed:@"getUserMedia"];
     137    TestWebKitAPI::Util::run(&wasPrompted);
     138    EXPECT_EQ(numberOfPrompts, 1);
     139
     140    wasPrompted = false;
     141    numberOfPrompts = 0;
     142    [webView stringByEvaluatingJavaScript:@"doMultipleGetUserMediaSynchronously()"];
     143    TestWebKitAPI::Util::run(&wasPrompted);
     144    EXPECT_EQ(numberOfPrompts, 1);
     145}
     146
    120147} // namespace TestWebKitAPI
    121148

  • trunk/Tools/TestWebKitAPI/Tests/WebKit/getUserMedia.html

    r240856 r246093  
    3737                return stream !== null;
    3838            }
     39
     40            function doMultipleGetUserMediaSynchronously()
     41            {
     42                navigator.mediaDevices.getUserMedia({video: true});
     43                navigator.mediaDevices.getUserMedia({video: true});
     44                navigator.mediaDevices.getUserMedia({video: true});
     45
     46                // This one should prompt.
     47                navigator.mediaDevices.getUserMedia({audio: true});
     48
     49                navigator.mediaDevices.getUserMedia({audio: true});
     50                navigator.mediaDevices.getUserMedia({audio: true});
     51                navigator.mediaDevices.getUserMedia({audio: true, video: true});
     52            }
    3953        </script>
    4054    <head>
Note: See TracChangeset for help on using the changeset viewer.