⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 280702 in webkit


Ignore:
Timestamp:
Aug 5, 2021, 12:29:05 PM (5 years ago)
Author:
eric.carlson@apple.com
Message:

[iOS] getUserMedia sometimes doesn't capture from specified microphone
​https://bugs.webkit.org/show_bug.cgi?id=228753
rdar://79704226

Reviewed by Youenn Fablet.

Source/WebCore:

The system will always choose the "default" audio input source unless
+[AVAudioSession setPreferredInput:error:] is called first, and that only works
if the audio session category has been set to PlayAndRecord *before* it is called,
so configure the audio session for recording before we choose and configure the
audio capture device.

Tested manually, this only reproduces on hardware.

  • platform/audio/PlatformMediaSessionManager.cpp:

(WebCore::PlatformMediaSessionManager::activeAudioSessionRequired const): Audio
capture requires an active audio session.
(WebCore::PlatformMediaSessionManager::removeSession): Move #if USE(AUDIO_SESSION)
guard inside of maybeDeactivateAudioSession so it isn't spread throughout the file.
(WebCore::PlatformMediaSessionManager::sessionWillBeginPlayback): Ditto.
(WebCore::PlatformMediaSessionManager::processWillSuspend): Ditto.
(WebCore::PlatformMediaSessionManager::processDidResume): Ditto.
(WebCore::PlatformMediaSessionManager::sessionCanProduceAudioChanged): Add logging,
call maybeActivateAudioSession() so we activate the audio session if necessary.
(WebCore::PlatformMediaSessionManager::addAudioCaptureSource): Call updateSessionState
instead of scheduleUpdateSessionState so the audio session category is updated
immediately.
(WebCore::PlatformMediaSessionManager::maybeDeactivateAudioSession): Move
#if USE(AUDIO_SESSION) into the function so it doesn't need to be spread
throughout the file.
(WebCore::PlatformMediaSessionManager::maybeActivateAudioSession): Ditto.

  • platform/audio/PlatformMediaSessionManager.h:

(WebCore::PlatformMediaSessionManager::isApplicationInBackground const):

  • platform/audio/ios/AudioSessionIOS.mm:

(WebCore::AudioSessionIOS::setPreferredBufferSize): Log an error if we are unable
to set the preferred buffer size.

  • platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h:
  • platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:

(WebCore::AVAudioSessionCaptureDeviceManager::setPreferredAudioSessionDeviceUID):
New, set the preferred input so capture will use select the device we want.
(WebCore::AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices): Remove
m_recomputeDevices, setAudioCaptureDevices has been restructured so we don't need it.
(WebCore::AVAudioSessionCaptureDeviceManager::computeCaptureDevices): Ditto.
(WebCore::AVAudioSessionCaptureDeviceManager::setAudioCaptureDevices): Don't update
the list of capture devices when the default device changes, only when a device is
added, removed, enabled, or disabled.

  • platform/mediastream/mac/CoreAudioCaptureSource.cpp:

(WebCore::CoreAudioSharedUnit::setCaptureDevice): Call setPreferredAudioSessionDeviceUID
so the correct device is selected.
(WebCore::CoreAudioSharedUnit::cleanupAudioUnit): Clear m_persistentID.
(WebCore::CoreAudioCaptureSource::create): Return an error with a string, or the
web process can detect a failure.
(WebCore::CoreAudioCaptureSource::stopProducingData): Add logging.

Source/WebKit:

  • UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp: Re

(WebKit::UserMediaCaptureManagerProxy::SourceProxy::audioUnitWillStart): Delete,
we don't need it now that the web process configures the audio session before
capture begins.

Location:
trunk/Source
Files:
9 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r280700 r280702  
     12021-08-05  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [iOS] getUserMedia sometimes doesn't capture from specified microphone
     4        https://bugs.webkit.org/show_bug.cgi?id=228753
     5        rdar://79704226
     6
     7        Reviewed by Youenn Fablet.
     8
     9        The system will always choose the "default" audio input source unless
     10        +[AVAudioSession setPreferredInput:error:] is called first, and that only works
     11        if the audio session category has been set to PlayAndRecord *before* it is called,
     12        so configure the audio session for recording before we choose and configure the
     13        audio capture device.
     14       
     15        Tested manually, this only reproduces on hardware.
     16
     17        * platform/audio/PlatformMediaSessionManager.cpp:
     18        (WebCore::PlatformMediaSessionManager::activeAudioSessionRequired const): Audio
     19        capture requires an active audio session.
     20        (WebCore::PlatformMediaSessionManager::removeSession): Move `#if USE(AUDIO_SESSION)`
     21        guard inside of maybeDeactivateAudioSession so it isn't spread throughout the file.
     22        (WebCore::PlatformMediaSessionManager::sessionWillBeginPlayback): Ditto.
     23        (WebCore::PlatformMediaSessionManager::processWillSuspend): Ditto.
     24        (WebCore::PlatformMediaSessionManager::processDidResume): Ditto.
     25        (WebCore::PlatformMediaSessionManager::sessionCanProduceAudioChanged): Add logging,
     26        call `maybeActivateAudioSession()` so we activate the audio session if necessary.
     27        (WebCore::PlatformMediaSessionManager::addAudioCaptureSource): Call updateSessionState
     28        instead of scheduleUpdateSessionState so the audio session category is updated
     29        immediately.
     30        (WebCore::PlatformMediaSessionManager::maybeDeactivateAudioSession): Move
     31        `#if USE(AUDIO_SESSION)` into the function so it doesn't need to be spread
     32        throughout the file.
     33        (WebCore::PlatformMediaSessionManager::maybeActivateAudioSession): Ditto.
     34        * platform/audio/PlatformMediaSessionManager.h:
     35        (WebCore::PlatformMediaSessionManager::isApplicationInBackground const):
     36
     37        * platform/audio/ios/AudioSessionIOS.mm:
     38        (WebCore::AudioSessionIOS::setPreferredBufferSize): Log an error if we are unable
     39        to set the preferred buffer size.
     40
     41        * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h:
     42        * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:
     43        (WebCore::AVAudioSessionCaptureDeviceManager::setPreferredAudioSessionDeviceUID):
     44        New, set the preferred input so capture will use select the device we want.
     45        (WebCore::AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices): Remove
     46        m_recomputeDevices, `setAudioCaptureDevices` has been restructured so we don't need it.
     47        (WebCore::AVAudioSessionCaptureDeviceManager::computeCaptureDevices): Ditto.
     48        (WebCore::AVAudioSessionCaptureDeviceManager::setAudioCaptureDevices): Don't update
     49        the list of capture devices when the default device changes, only when a device is
     50        added, removed, enabled, or disabled.
     51
     52        * platform/mediastream/mac/CoreAudioCaptureSource.cpp:
     53        (WebCore::CoreAudioSharedUnit::setCaptureDevice): Call `setPreferredAudioSessionDeviceUID`
     54        so the correct device is selected.
     55        (WebCore::CoreAudioSharedUnit::cleanupAudioUnit): Clear m_persistentID.
     56        (WebCore::CoreAudioCaptureSource::create): Return an error with a string, or the
     57        web process can detect a failure.
     58        (WebCore::CoreAudioCaptureSource::stopProducingData): Add logging.
     59
    1602021-08-05  Alan Bujtas  <zalan@apple.com>
    261

  • trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.cpp

    r278483 r280702  
    11/*
    2  * Copyright (C) 2013-2020 Apple Inc. All rights reserved.
     2 * Copyright (C) 2013-2021 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    117117bool PlatformMediaSessionManager::activeAudioSessionRequired() const
    118118{
    119     return anyOfSessions([] (auto& session) {
    120         return session.activeAudioSessionRequired();
     119    if (anyOfSessions([] (auto& session) { return session.activeAudioSessionRequired(); }))
     120        return true;
     121
     122    return WTF::anyOf(m_audioCaptureSources, [](auto& source) {
     123        return source.isCapturingAudio();
    121124    });
    122125}
    … …  
    200203    m_sessions.remove(index);
    201204
    202 #if USE(AUDIO_SESSION)
    203205    if (hasNoSession())
    204206        maybeDeactivateAudioSession();
    205 #endif
    206207
    207208#if !RELEASE_LOG_DISABLED
    … …  
    238239    }
    239240
    240 #if USE(AUDIO_SESSION)
    241     if (activeAudioSessionRequired()) {
    242         if (!AudioSession::sharedSession().tryToSetActive(true)) {
    243             ALWAYS_LOG(LOGIDENTIFIER, session.logIdentifier(), " returning false failed to set active AudioSession");
    244             return false;
    245         }
    246 
    247         ALWAYS_LOG(LOGIDENTIFIER, session.logIdentifier(), " sucessfully activated AudioSession");
    248         m_becameActive = true;
    249     }
    250 #endif
     241    if (!maybeActivateAudioSession()) {
     242        ALWAYS_LOG(LOGIDENTIFIER, session.logIdentifier(), " returning false, failed to activate AudioSession");
     243        return false;
     244    }
    251245
    252246    if (m_interrupted)
    … …  
    395389    });
    396390
    397 #if USE(AUDIO_SESSION)
    398391    maybeDeactivateAudioSession();
    399 #endif
    400392}
    401393
    … …  
    411403
    412404#if USE(AUDIO_SESSION)
    413     if (!m_becameActive && activeAudioSessionRequired()) {
    414         m_becameActive = AudioSession::sharedSession().tryToSetActive(true);
    415         ALWAYS_LOG(LOGIDENTIFIER, "tried to set active AudioSession, ", m_becameActive ? "succeeded" : "failed");
    416     }
     405    if (!m_becameActive)
     406        maybeActivateAudioSession();
    417407#endif
    418408}
    … …  
    438428void PlatformMediaSessionManager::sessionCanProduceAudioChanged()
    439429{
     430    ALWAYS_LOG(LOGIDENTIFIER);
     431    maybeActivateAudioSession();
    440432    updateSessionState();
    441433}
    … …  
    582574    ASSERT(!m_audioCaptureSources.contains(source));
    583575    m_audioCaptureSources.add(source);
    584     scheduleUpdateSessionState();
     576    updateSessionState();
    585577}
    586578
    … …  
    604596}
    605597
     598void PlatformMediaSessionManager::maybeDeactivateAudioSession()
     599{
    606600#if USE(AUDIO_SESSION)
    607 void PlatformMediaSessionManager::maybeDeactivateAudioSession()
    608 {
    609601    if (!m_becameActive || !shouldDeactivateAudioSession())
    610602        return;
    … …  
    613605    AudioSession::sharedSession().tryToSetActive(false);
    614606    m_becameActive = false;
    615 }
    616 #endif
    617 
     607#endif
     608}
     609
     610bool PlatformMediaSessionManager::maybeActivateAudioSession()
     611{
     612#if USE(AUDIO_SESSION)
     613    if (!activeAudioSessionRequired())
     614        return true;
     615
     616    m_becameActive = AudioSession::sharedSession().tryToSetActive(true);
     617    ALWAYS_LOG(LOGIDENTIFIER, m_becameActive ? "successfully activated" : "failed to activate", " AudioSession");
     618    return m_becameActive;
     619#else
     620    return true;
     621#endif
     622}
    618623static bool& deactivateAudioSession()
    619624{

  • trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h

    r278483 r280702  
    182182
    183183    bool isApplicationInBackground() const { return m_isApplicationInBackground; }
    184 #if USE(AUDIO_SESSION)
    185184    void maybeDeactivateAudioSession();
    186 #endif
     185    bool maybeActivateAudioSession();
    187186
    188187#if !RELEASE_LOG_DISABLED

  • trunk/Source/WebCore/platform/audio/ios/AudioSessionIOS.mm

    r280462 r280702  
    11/*
    2  * Copyright (C) 2013-2019 Apple Inc. All rights reserved.
     2 * Copyright (C) 2013-2021 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    302302    float duration = bufferSize / sampleRate();
    303303    [[PAL::getAVAudioSessionClass() sharedInstance] setPreferredIOBufferDuration:duration error:&error];
     304    RELEASE_LOG_ERROR_IF(error, Media, "failed to set preferred buffer duration to %f with error: %@", duration, error.localizedDescription);
    304305    ASSERT(!error);
    305306}

  • trunk/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h

    r280400 r280702  
    5858    void disableAllDevicesQuery();
    5959
     60    void setPreferredAudioSessionDeviceUID(const String&);
     61
    6062private:
    6163    AVAudioSessionCaptureDeviceManager();

  • trunk/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm

    r280400 r280702  
    145145}
    146146
     147void AVAudioSessionCaptureDeviceManager::setPreferredAudioSessionDeviceUID(const String& deviceUID)
     148{
     149    AVAudioSessionPortDescription *preferredPort = nil;
     150    NSString *nsDeviceUID = deviceUID;
     151    for (AVAudioSessionPortDescription *portDescription in [m_audioSession availableInputs]) {
     152        if ([portDescription.UID isEqualToString:nsDeviceUID]) {
     153            preferredPort = portDescription;
     154            break;
     155        }
     156    }
     157
     158    if (!preferredPort) {
     159        RELEASE_LOG_ERROR(WebRTC, "failed to find preferred input '%{public}s'", deviceUID.ascii().data());
     160        return;
     161    }
     162
     163    NSError *error = nil;
     164    if (![[PAL::getAVAudioSessionClass() sharedInstance] setPreferredInput:preferredPort error:&error])
     165        RELEASE_LOG_ERROR(WebRTC, "failed to set preferred input to '%{public}s' with error: %@", deviceUID.ascii().data(), error.localizedDescription);
     166}
     167
    147168void AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices()
    148169{
    149     if (m_recomputeDevices)
    150         return;
    151 
    152     m_recomputeDevices = true;
    153170    computeCaptureDevices([] { });
    154171}
    … …  
    181198    }
    182199
    183     if (!m_recomputeDevices)
    184         return;
    185 
    186200    m_dispatchQueue->dispatch([this, completion = WTFMove(completion)] () mutable {
    187201        auto newAudioDevices = retrieveAudioSessionCaptureDevices();
    188202        callOnWebThreadOrDispatchAsyncOnMainThread(makeBlockPtr([this, completion = WTFMove(completion), newAudioDevices = WTFMove(newAudioDevices).isolatedCopy()] () mutable {
    189             m_recomputeDevices = false;
    190203            setAudioCaptureDevices(WTFMove(newAudioDevices));
    191204            completion();
    … …  
    221234{
    222235    bool firstTime = !m_devices;
    223     bool haveDeviceChanges = !m_devices || newAudioDevices.size() != m_devices->size();
    224     if (!haveDeviceChanges) {
    225         for (size_t i = 0; i < newAudioDevices.size(); ++i) {
    226             auto& oldState = (*m_devices)[i];
    227             auto& newState = newAudioDevices[i];
    228             if (newState.type() != oldState.type() || newState.persistentId() != oldState.persistentId() || newState.enabled() != oldState.enabled() || newState.isDefault() != oldState.isDefault())
    229                 haveDeviceChanges = true;
     236    bool deviceListChanged = newAudioDevices.size() != m_devices->size();
     237    bool defaultDeviceChanged = false;
     238    if (!deviceListChanged && !firstTime) {
     239        for (auto& newState : newAudioDevices) {
     240
     241            std::optional<CaptureDevice> oldState;
     242            for (const auto& device : m_devices.value()) {
     243                if (device.type() == newState.type() && device.persistentId() == newState.persistentId()) {
     244                    oldState = device;
     245                    break;
     246                }
     247            }
     248
     249            if (!oldState.has_value()) {
     250                deviceListChanged = true;
     251                break;
     252            }
     253            if (newState.isDefault() != oldState.value().isDefault())
     254                defaultDeviceChanged  = true;
     255
     256            if (newState.enabled() != oldState.value().enabled()) {
     257                deviceListChanged = true;
     258                break;
     259            }
    230260        }
    231261    }
    232262
    233     if (!haveDeviceChanges && !firstTime)
     263    if (!deviceListChanged && !firstTime && !defaultDeviceChanged)
    234264        return;
    235265
    … …  
    241271    m_devices = WTFMove(newDevices);
    242272
    243     if (!firstTime)
     273    if (deviceListChanged && !firstTime)
    244274        deviceChanged();
    245275}

  • trunk/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp

    r279335 r280702  
    164164void CoreAudioSharedUnit::setCaptureDevice(String&& persistentID, uint32_t captureDeviceID)
    165165{
     166    if (m_persistentID == persistentID)
     167        return;
     168
    166169    m_persistentID = WTFMove(persistentID);
    167170
    … …  
    174177#else
    175178    UNUSED_PARAM(captureDeviceID);
     179    AVAudioSessionCaptureDeviceManager::singleton().setPreferredAudioSessionDeviceUID(m_persistentID);
    176180#endif
    177181}
    … …  
    467471    m_microphoneSampleBuffer = nullptr;
    468472    m_speakerSampleBuffer = nullptr;
     473    m_persistentID = emptyString();
    469474#if !LOG_DISABLED
    470475    m_ioUnitName = emptyString();
    … …  
    620625    auto device = AVAudioSessionCaptureDeviceManager::singleton().audioSessionDeviceWithUID(WTFMove(deviceID));
    621626    if (!device)
    622         return { };
     627        return { "No AVAudioSessionCaptureDevice device"_s };
    623628
    624629    auto source = adoptRef(*new CoreAudioCaptureSource(WTFMove(deviceID), String { device->label() }, WTFMove(hashSalt), 0));
    … …  
    761766void CoreAudioCaptureSource::stopProducingData()
    762767{
     768    ALWAYS_LOG_IF(loggerPtr(), LOGIDENTIFIER);
    763769    unit().stopProducingData();
    764770}

  • trunk/Source/WebKit/ChangeLog

    r280699 r280702  
     12021-08-05  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [iOS] getUserMedia sometimes doesn't capture from specified microphone
     4        https://bugs.webkit.org/show_bug.cgi?id=228753
     5        rdar://79704226
     6
     7        Reviewed by Youenn Fablet.
     8
     9        * UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp: Re
     10        (WebKit::UserMediaCaptureManagerProxy::SourceProxy::audioUnitWillStart): Delete,
     11        we don't need it now that the web process configures the audio session before
     12        capture begins.
     13
    1142021-08-05  Youenn Fablet  <youenn@apple.com>
    215

  • trunk/Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp

    r279912 r280702  
    101101    int64_t numberOfFrames() { return m_numberOfFrames; }
    102102
    103     void audioUnitWillStart() final
    104     {
    105         // FIXME: WebProcess might want to set the category/bufferSize itself, in which case we should remove that code.
    106         auto bufferSize = AudioSession::sharedSession().sampleRate() / 50;
    107         if (AudioSession::sharedSession().preferredBufferSize() > bufferSize)
    108             AudioSession::sharedSession().setPreferredBufferSize(bufferSize);
    109         AudioSession::sharedSession().setCategory(AudioSession::CategoryType::PlayAndRecord, RouteSharingPolicy::Default);
    110     }
    111 
    112103    void start()
    113104    {
Note: See TracChangeset for help on using the changeset viewer.