⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 228310 in webkit


Ignore:
Timestamp:
Feb 8, 2018, 9:05:22 PM (9 years ago)
Author:
commit-queue@webkit.org
Message:

CoreAudioCaptureSourceIOSListener should be a WebProcess singleton
​https://bugs.webkit.org/show_bug.cgi?id=182606
<rdar://problem/37355283>

Patch by Youenn Fablet <​youenn@apple.com> on 2018-02-08
Reviewed by Eric Carlson.

Not testable right now as this relies on iOS specific Core Audio stack.
Efforts should be made to somehow mock that part and mock interruptions.

Made CoreAudioCaptureSourceFactoryIOS responsible to receive iOS interruption messages.
Before the patch, individual capture sources were receiving these messages.
When there is only one capture source per process this is fine but with more capture sources,
this is messing up things as they all share the same shared unit and we would try to interrupt it or resume it several times.

Also, if we are suspended and there is no more capture source, the shared unit will remain suspended indefinitely, as we will
not process the interruption messages.
With the patch, we always process the messages.
We still go through the active source as done now to limit the changes.
In case of no source, we go to the shared unit.
In the future, we should directly go to the shared unit and clients of the shared unit should see their state being updated.

As a way to prevent staying suspended on iOS, we unsuspend ourselves in CoreAudioCaptureSource constructor.
This is fine as this constructor is only called when getUserMedia grants access to the camera which always goes to the UIProcess.
And the UIProcess is only processing getUserMedia calls if the WebProcess is foregrounded.

  • platform/mediastream/ios/CoreAudioCaptureSourceIOS.h:
  • platform/mediastream/ios/CoreAudioCaptureSourceIOS.mm:

(-[WebCoreAudioCaptureSourceIOSListener initWithCallback:]):
(WebCore::CoreAudioCaptureSourceFactoryIOS::CoreAudioCaptureSourceFactoryIOS):
(WebCore::CoreAudioCaptureSourceFactoryIOS::~CoreAudioCaptureSourceFactoryIOS):
(WebCore::CoreAudioCaptureSourceFactory::singleton):

  • platform/mediastream/mac/CoreAudioCaptureSource.cpp:

(WebCore::CoreAudioCaptureSourceFactory::singleton):
(WebCore::CoreAudioCaptureSource::create):
(WebCore::CoreAudioCaptureSourceFactory::beginInterruption):
(WebCore::CoreAudioCaptureSourceFactory::endInterruption):
(WebCore::CoreAudioCaptureSourceFactory::scheduleReconfiguration):
(WebCore::CoreAudioCaptureSource::factory):
(WebCore::CoreAudioCaptureSource::CoreAudioCaptureSource):
(WebCore::CoreAudioCaptureSource::~CoreAudioCaptureSource):
(WebCore::CoreAudioCaptureSource::startProducingData):
(WebCore::CoreAudioCaptureSource::scheduleReconfiguration):
(WebCore::CoreAudioCaptureSource::beginInterruption):
(WebCore::CoreAudioCaptureSource::endInterruption):

  • platform/mediastream/mac/CoreAudioCaptureSource.h:

(WebCore::CoreAudioCaptureSourceFactory::setCoreAudioActiveSource):
(WebCore::CoreAudioCaptureSourceFactory::unsetCoreAudioActiveSource):
(WebCore::CoreAudioCaptureSourceFactory::coreAudioActiveSource):

Location:
trunk/Source/WebCore
Files:
5 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r228309 r228310  
     12018-02-08  Youenn Fablet  <youenn@apple.com>
     2
     3        CoreAudioCaptureSourceIOSListener should be a WebProcess singleton
     4        https://bugs.webkit.org/show_bug.cgi?id=182606
     5        <rdar://problem/37355283>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Not testable right now as this relies on iOS specific Core Audio stack.
     10        Efforts should be made to somehow mock that part and mock interruptions.
     11
     12        Made CoreAudioCaptureSourceFactoryIOS responsible to receive iOS interruption messages.
     13        Before the patch, individual capture sources were receiving these messages.
     14        When there is only one capture source per process this is fine but with more capture sources,
     15        this is messing up things as they all share the same shared unit and we would try to interrupt it or resume it several times.
     16
     17        Also, if we are suspended and there is no more capture source, the shared unit will remain suspended indefinitely, as we will
     18        not process the interruption messages.
     19        With the patch, we always process the messages.
     20        We still go through the active source as done now to limit the changes.
     21        In case of no source, we go to the shared unit.
     22        In the future, we should directly go to the shared unit and clients of the shared unit should see their state being updated.
     23
     24        As a way to prevent staying suspended on iOS, we unsuspend ourselves in CoreAudioCaptureSource constructor.
     25        This is fine as this constructor is only called when getUserMedia grants access to the camera which always goes to the UIProcess.
     26        And the UIProcess is only processing getUserMedia calls if the WebProcess is foregrounded.
     27
     28        * platform/mediastream/ios/CoreAudioCaptureSourceIOS.h:
     29        * platform/mediastream/ios/CoreAudioCaptureSourceIOS.mm:
     30        (-[WebCoreAudioCaptureSourceIOSListener initWithCallback:]):
     31        (WebCore::CoreAudioCaptureSourceFactoryIOS::CoreAudioCaptureSourceFactoryIOS):
     32        (WebCore::CoreAudioCaptureSourceFactoryIOS::~CoreAudioCaptureSourceFactoryIOS):
     33        (WebCore::CoreAudioCaptureSourceFactory::singleton):
     34        * platform/mediastream/mac/CoreAudioCaptureSource.cpp:
     35        (WebCore::CoreAudioCaptureSourceFactory::singleton):
     36        (WebCore::CoreAudioCaptureSource::create):
     37        (WebCore::CoreAudioCaptureSourceFactory::beginInterruption):
     38        (WebCore::CoreAudioCaptureSourceFactory::endInterruption):
     39        (WebCore::CoreAudioCaptureSourceFactory::scheduleReconfiguration):
     40        (WebCore::CoreAudioCaptureSource::factory):
     41        (WebCore::CoreAudioCaptureSource::CoreAudioCaptureSource):
     42        (WebCore::CoreAudioCaptureSource::~CoreAudioCaptureSource):
     43        (WebCore::CoreAudioCaptureSource::startProducingData):
     44        (WebCore::CoreAudioCaptureSource::scheduleReconfiguration):
     45        (WebCore::CoreAudioCaptureSource::beginInterruption):
     46        (WebCore::CoreAudioCaptureSource::endInterruption):
     47        * platform/mediastream/mac/CoreAudioCaptureSource.h:
     48        (WebCore::CoreAudioCaptureSourceFactory::setCoreAudioActiveSource):
     49        (WebCore::CoreAudioCaptureSourceFactory::unsetCoreAudioActiveSource):
     50        (WebCore::CoreAudioCaptureSourceFactory::coreAudioActiveSource):
     51
    1522018-02-08  Chris Dumez  <cdumez@apple.com>
    253

  • trunk/Source/WebCore/platform/mediastream/ios/CoreAudioCaptureSourceIOS.h

    r218746 r228310  
    3434namespace WebCore {
    3535
    36 class CoreAudioCaptureSourceIOS final : public CoreAudioCaptureSource {
     36class CoreAudioCaptureSourceFactoryIOS final : public CoreAudioCaptureSourceFactory {
     37public:
     38    CoreAudioCaptureSourceFactoryIOS();
     39    ~CoreAudioCaptureSourceFactoryIOS();
     40
    3741private:
    38     friend class CoreAudioCaptureSource;
    39 
    40     CoreAudioCaptureSourceIOS(const String& deviceID, const String& label);
    41     ~CoreAudioCaptureSourceIOS();
    42 
    4342    RetainPtr<WebCoreAudioCaptureSourceIOSListener> m_listener;
    4443};

  • trunk/Source/WebCore/platform/mediastream/ios/CoreAudioCaptureSourceIOS.mm

    r219191 r228310  
    5151
    5252@interface WebCoreAudioCaptureSourceIOSListener : NSObject {
    53     CoreAudioCaptureSourceIOS* _callback;
     53    CoreAudioCaptureSourceFactoryIOS* _callback;
    5454}
    5555
    … …  
    6060
    6161@implementation WebCoreAudioCaptureSourceIOSListener
    62 - (id)initWithCallback:(CoreAudioCaptureSourceIOS*)callback
     62- (id)initWithCallback:(CoreAudioCaptureSourceFactoryIOS*)callback
    6363{
    6464    self = [super init];
    … …  
    123123namespace WebCore {
    124124
    125 CoreAudioCaptureSourceIOS::CoreAudioCaptureSourceIOS(const String& deviceID, const String& label)
    126     : CoreAudioCaptureSource(deviceID, label, 0)
    127     , m_listener(adoptNS([[WebCoreAudioCaptureSourceIOSListener alloc] initWithCallback:this]))
     125CoreAudioCaptureSourceFactoryIOS::CoreAudioCaptureSourceFactoryIOS()
     126    : m_listener(adoptNS([[WebCoreAudioCaptureSourceIOSListener alloc] initWithCallback:this]))
    128127{
    129128}
    130129
    131 CoreAudioCaptureSourceIOS::~CoreAudioCaptureSourceIOS()
     130CoreAudioCaptureSourceFactoryIOS::~CoreAudioCaptureSourceFactoryIOS()
    132131{
    133132    [m_listener invalidate];
    … …  
    135134}
    136135
     136CoreAudioCaptureSourceFactory& CoreAudioCaptureSourceFactory::singleton()
     137{
     138    static NeverDestroyed<CoreAudioCaptureSourceFactoryIOS> factory;
     139    return factory.get();
     140}
     141
    137142}
    138143

  • trunk/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp

    r225395 r228310  
    5454using namespace PAL;
    5555
    56 class CoreAudioCaptureSourceFactory : public RealtimeMediaSource::AudioCaptureFactory
    57 {
    58 public:
    59     CaptureSourceOrError createAudioCaptureSource(const CaptureDevice& device, const MediaConstraints* constraints) final
    60     {
    61         return CoreAudioCaptureSource::create(device.persistentId(), constraints);
    62     }
    63 };
    64 
    65 static CoreAudioCaptureSourceFactory& coreAudioCaptureSourceFactory()
     56#if PLATFORM(MAC)
     57CoreAudioCaptureSourceFactory& CoreAudioCaptureSourceFactory::singleton()
    6658{
    6759    static NeverDestroyed<CoreAudioCaptureSourceFactory> factory;
    6860    return factory.get();
    6961}
     62#endif
    7063
    7164const UInt32 outputBus = 0;
    … …  
    698691        return { };
    699692
    700     auto source = adoptRef(*new CoreAudioCaptureSourceIOS(deviceID, device->label()));
     693    auto source = adoptRef(*new CoreAudioCaptureSource(deviceID, device->label(), 0));
    701694#endif
    702695
    … …  
    709702}
    710703
     704void CoreAudioCaptureSourceFactory::beginInterruption()
     705{
     706    if (!isMainThread()) {
     707        callOnMainThread([this] {
     708            beginInterruption();
     709        });
     710        return;
     711    }
     712    ASSERT(isMainThread());
     713
     714    if (auto* source = coreAudioActiveSource()) {
     715        source->beginInterruption();
     716        return;
     717    }
     718    CoreAudioSharedUnit::singleton().suspend();
     719}
     720
     721void CoreAudioCaptureSourceFactory::endInterruption()
     722{
     723    if (!isMainThread()) {
     724        callOnMainThread([this] {
     725            endInterruption();
     726        });
     727        return;
     728    }
     729    ASSERT(isMainThread());
     730
     731    if (auto* source = coreAudioActiveSource()) {
     732        source->endInterruption();
     733        return;
     734    }
     735    CoreAudioSharedUnit::singleton().reconfigureAudioUnit();
     736}
     737
     738void CoreAudioCaptureSourceFactory::scheduleReconfiguration()
     739{
     740    if (!isMainThread()) {
     741        callOnMainThread([this] {
     742            scheduleReconfiguration();
     743        });
     744        return;
     745    }
     746    ASSERT(isMainThread());
     747
     748    if (auto* source = coreAudioActiveSource()) {
     749        source->scheduleReconfiguration();
     750        return;
     751    }
     752    CoreAudioSharedUnit::singleton().reconfigureAudioUnit();
     753}
     754
    711755RealtimeMediaSource::AudioCaptureFactory& CoreAudioCaptureSource::factory()
    712756{
    713     return coreAudioCaptureSourceFactory();
     757    return CoreAudioCaptureSourceFactory::singleton();
    714758}
    715759
    … …  
    725769
    726770    unit.addClient(*this);
     771
     772#if PLATFORM(IOS)
     773    // We ensure that we unsuspend ourselves on the constructor as a capture source
     774    // is created when getUserMedia grants access which only happens when the process is foregrounded.
     775    if (unit.isSuspended())
     776        unit.reconfigureAudioUnit();
     777#endif
    727778}
    728779
    … …  
    730781{
    731782#if PLATFORM(IOS)
    732     coreAudioCaptureSourceFactory().unsetActiveSource(*this);
     783    CoreAudioCaptureSourceFactory::singleton().unsetCoreAudioActiveSource(*this);
    733784#endif
    734785
    … …  
    749800{
    750801#if PLATFORM(IOS)
    751     coreAudioCaptureSourceFactory().setActiveSource(*this);
     802    CoreAudioCaptureSourceFactory::singleton().setCoreAudioActiveSource(*this);
    752803#endif
    753804
    … …  
    845896void CoreAudioCaptureSource::scheduleReconfiguration()
    846897{
    847     if (!isMainThread()) {
    848         callOnMainThread([weakThis = createWeakPtr(), this] {
    849             if (!weakThis)
    850                 return;
    851 
    852             scheduleReconfiguration();
    853         });
    854 
    855         return;
    856     }
    857 
    858898    ASSERT(isMainThread());
    859899    auto& unit = CoreAudioSharedUnit::singleton();
    … …  
    875915void CoreAudioCaptureSource::beginInterruption()
    876916{
    877     if (!isMainThread()) {
    878         callOnMainThread([weakThis = createWeakPtr(), this] {
    879             if (!weakThis)
    880                 return;
    881 
    882             beginInterruption();
    883         });
    884 
    885         return;
    886     }
    887 
    888917    ASSERT(isMainThread());
    889918    auto& unit = CoreAudioSharedUnit::singleton();
    … …  
    901930void CoreAudioCaptureSource::endInterruption()
    902931{
    903     if (!isMainThread()) {
    904         callOnMainThread([weakThis = createWeakPtr(), this] {
    905             if (!weakThis)
    906                 return;
    907 
    908             endInterruption();
    909         });
    910 
    911         return;
    912     }
    913 
    914932    ASSERT(isMainThread());
    915933    auto& unit = CoreAudioSharedUnit::singleton();

  • trunk/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h

    r218746 r228310  
    111111};
    112112
     113class CoreAudioCaptureSourceFactory : public RealtimeMediaSource::AudioCaptureFactory {
     114public:
     115    static CoreAudioCaptureSourceFactory& singleton();
     116
     117    void beginInterruption();
     118    void endInterruption();
     119    void scheduleReconfiguration();
     120
     121#if PLATFORM(IOS)
     122    void setCoreAudioActiveSource(CoreAudioCaptureSource& source) { setActiveSource(source); }
     123    void unsetCoreAudioActiveSource(CoreAudioCaptureSource& source) { unsetActiveSource(source); }
     124    CoreAudioCaptureSource* coreAudioActiveSource() { return static_cast<CoreAudioCaptureSource*>(activeSource()); }
     125#else
     126    CoreAudioCaptureSource* coreAudioActiveSource() { return nullptr; }
     127#endif
     128
     129private:
     130    CaptureSourceOrError createAudioCaptureSource(const CaptureDevice& device, const MediaConstraints* constraints) final
     131    {
     132        return CoreAudioCaptureSource::create(device.persistentId(), constraints);
     133    }
     134};
     135
    113136} // namespace WebCore
    114137
Note: See TracChangeset for help on using the changeset viewer.