Changeset 113728 in webkit


Ignore:
Timestamp:
Apr 10, 2012 10:04:41 AM (12 years ago)
Author:
jer.noble@apple.com
Message:

WebAudio: propagate a silence hint through the AudioNode graph.
https://bugs.webkit.org/show_bug.cgi?id=74553

Reviewed by Chris Rogers.

No new tests; optimization of existing code path, so covered by existing tests.

Introduce the concept of a "silent" channel:

  • platform/audio/AudioChannel.h:

(WebCore::AudioChannel::AudioChannel):
(WebCore::AudioChannel::set): Clear silent bit.
(WebCore::AudioChannel::zero): Set silent bit.
(WebCore::AudioChannel::clearSilentFlag): Clear silent bit.
(WebCore::AudioChannel::isSilent): Accessor.

Optimize a few channel operations when the source or destination channels are silent.

  • platform/audio/AudioChannel.cpp:

(WebCore::AudioChannel::scale): No-op on a silent channel.
(WebCore::AudioChannel::copyFrom): zero() when source is silent.
(WebCore::AudioChannel::copyFromRange): possibly zero() when source is silent.
(WebCore::AudioChannel::sumFrom): No-op when source is silent.
(WebCore::AudioChannel::maxAbsValue): 0 on a silent channel.

Optimize a few bus operations when the source or destination channels are silent.

  • platform/audio/AudioBus.cpp:

(WebCore::AudioBus::processWithGainFromMonoStereo): No-op if source is silent and either

the destination bus is silent, or the output is not summed to the destination.

(WebCore::AudioBus::copyWithSampleAccurateGainValuesFrom): zero() if the source is silent

and the lengths of the gain values, source, and destination match.

(WebCore::AudioBus::createBySampleRateConverting): Return an empty bus if the source is silent.
(WebCore::AudioBus::createByMixingToMono): Ditto; clear the destination's silent bit otherwise.
(WebCore::AudioBus::isSilent): Return whether all channels are silent.
(WebCore::AudioBus::clearSilentFlag): Clear silent bit of constituent channels.

  • platform/audio/AudioBus.h:

Make sure that classes which generate audio clear their busses' silent bit.

  • webaudio/AudioBufferSourceNode.cpp:

(WebCore::AudioBufferSourceNode::process): Ditto.
(WebCore::AudioBufferSourceNode::renderFromBuffer): Ditto.
(WebCore::AudioBufferSourceNode::propagatesSilence): Propagate silence only when the source node

has nothing left to render.

  • webaudio/AudioBufferSourceNode.h:

(WebCore::AudioBufferSourceNode::playbackState): Made const correct.
(WebCore::AudioBufferSourceNode::isPlaying): Added simple accessor.
(WebCore::AudioBufferSourceNode::hasFinished): Ditto.

  • webaudio/AudioDestinationNode.h:

(WebCore::AudioDestinationNode::currentSampleFrame): Made const correct.
(WebCore::AudioDestinationNode::currentTime): Ditto.

Audio nodes should not process audio data when the input is silent and the nodes will propagate silences.

  • webaudio/AudioNode.cpp:

(WebCore::AudioNode::processIfNecessary):
(WebCore::AudioNode::inputsAreSilent): Convenience function which walk over the node's inputs.
(WebCore::AudioNode::silenceOutputs): Ditto.
(WebCore::AudioNode::unsilenceOutputs): Ditto.

  • webaudio/AudioNode.h:

(WebCore::AudioNode::propagatesSilence):

These Nodes can generate audio when given silent input, so return false from propagatesSilence.

  • Modules/webaudio/Oscillator.h:

(WebCore::Oscillator::propagatesSilence): Added.

Location:
trunk/Source/WebCore
Files:
12 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebCore/ChangeLog

    r113723 r113728  
     12011-12-14  Jer Noble  <jer.noble@apple.com>
     2
     3        WebAudio: propagate a silence hint through the AudioNode graph.
     4        https://bugs.webkit.org/show_bug.cgi?id=74553
     5
     6        Reviewed by Chris Rogers.
     7
     8        No new tests; optimization of existing code path, so covered by existing tests.
     9
     10        Introduce the concept of a "silent" channel:
     11        * platform/audio/AudioChannel.h:
     12        (WebCore::AudioChannel::AudioChannel):
     13        (WebCore::AudioChannel::set): Clear silent bit.
     14        (WebCore::AudioChannel::zero): Set silent bit.
     15        (WebCore::AudioChannel::clearSilentFlag): Clear silent bit.
     16        (WebCore::AudioChannel::isSilent): Accessor.
     17
     18        Optimize a few channel operations when the source or destination channels are silent.
     19        * platform/audio/AudioChannel.cpp:
     20        (WebCore::AudioChannel::scale): No-op on a silent channel.
     21        (WebCore::AudioChannel::copyFrom): zero() when source is silent.
     22        (WebCore::AudioChannel::copyFromRange): possibly zero() when source is silent.
     23        (WebCore::AudioChannel::sumFrom): No-op when source is silent.
     24        (WebCore::AudioChannel::maxAbsValue): 0 on a silent channel.
     25
     26        Optimize a few bus operations when the source or destination channels are silent.
     27        * platform/audio/AudioBus.cpp:
     28        (WebCore::AudioBus::processWithGainFromMonoStereo): No-op if source is silent and either
     29            the destination bus is silent, or the output is not summed to the destination.
     30        (WebCore::AudioBus::copyWithSampleAccurateGainValuesFrom): zero() if the source is silent
     31            and the lengths of the gain values, source, and destination match.
     32        (WebCore::AudioBus::createBySampleRateConverting): Return an empty bus if the source is silent.
     33        (WebCore::AudioBus::createByMixingToMono): Ditto; clear the destination's silent bit otherwise.
     34        (WebCore::AudioBus::isSilent): Return whether all channels are silent.
     35        (WebCore::AudioBus::clearSilentFlag): Clear silent bit of constituent channels.
     36        * platform/audio/AudioBus.h:
     37
     38        Make sure that classes which generate audio clear their busses' silent bit.
     39        * webaudio/AudioBufferSourceNode.cpp:
     40        (WebCore::AudioBufferSourceNode::process): Ditto.
     41        (WebCore::AudioBufferSourceNode::renderFromBuffer): Ditto.
     42        (WebCore::AudioBufferSourceNode::propagatesSilence): Propagate silence only when the source node
     43            has nothing left to render.
     44        * webaudio/AudioBufferSourceNode.h:
     45        (WebCore::AudioBufferSourceNode::playbackState): Made const correct.
     46        (WebCore::AudioBufferSourceNode::isPlaying): Added simple accessor.
     47        (WebCore::AudioBufferSourceNode::hasFinished): Ditto.
     48        * webaudio/AudioDestinationNode.h:
     49        (WebCore::AudioDestinationNode::currentSampleFrame): Made const correct.
     50        (WebCore::AudioDestinationNode::currentTime): Ditto.
     51
     52        Audio nodes should not process audio data when the input is silent and the nodes will propagate silences.
     53        * webaudio/AudioNode.cpp:
     54        (WebCore::AudioNode::processIfNecessary):
     55        (WebCore::AudioNode::inputsAreSilent): Convenience function which walk over the node's inputs.
     56        (WebCore::AudioNode::silenceOutputs): Ditto.
     57        (WebCore::AudioNode::unsilenceOutputs): Ditto.
     58        * webaudio/AudioNode.h:
     59        (WebCore::AudioNode::propagatesSilence):
     60
     61        These Nodes can generate audio when given silent input, so return false from propagatesSilence.
     62        * Modules/webaudio/Oscillator.h:
     63        (WebCore::Oscillator::propagatesSilence): Added.
     64
    1652012-04-10  Yi Shen  <yi.4.shen@nokia.com>
    266
  • trunk/Source/WebCore/Modules/webaudio/AudioBufferSourceNode.cpp

    r113245 r113728  
    159159            finish();
    160160        }
     161
     162        outputBus->clearSilentFlag();
     163
     164        m_processLock.unlock();
    161165    } else {
    162166        // Too bad - the tryLock() failed.  We must be in the middle of changing buffers and were already outputting silence anyway.
     
    332336            }
    333337        }
    334     }
     338    }
     339
     340    bus->clearSilentFlag();
     341
    335342    m_virtualReadIndex = virtualReadIndex;
    336343}
     
    500507    m_isLooping = looping;
    501508}
     509
     510bool AudioBufferSourceNode::propagatesSilence() const
     511{
     512    return !isPlayingOrScheduled() || hasFinished() || !m_buffer;
     513}
     514
    502515} // namespace WebCore
    503516
  • trunk/Source/WebCore/Modules/webaudio/AudioBufferSourceNode.h

    r111474 r113728  
    8585    void noteOff(double when);
    8686
    87     unsigned short playbackState() { return static_cast<unsigned short>(m_playbackState); }
     87    unsigned short playbackState() const { return static_cast<unsigned short>(m_playbackState); }
     88    bool isPlayingOrScheduled() const { return m_playbackState == PLAYING_STATE || m_playbackState == SCHEDULED_STATE; }
     89    bool hasFinished() const { return m_playbackState == FINISHED_STATE; }
    8890
    8991    // Note: the attribute was originally exposed as .looping, but to be more consistent in naming with <audio>
     
    102104    // If a panner node is set, then we can incorporate doppler shift into the playback pitch rate.
    103105    void setPannerNode(PassRefPtr<AudioPannerNode> pannerNode) { m_pannerNode = pannerNode; }
     106
     107    // If we are no longer playing, propogate silence ahead to downstream nodes.
     108    virtual bool propagatesSilence() const;
    104109
    105110private:
  • trunk/Source/WebCore/Modules/webaudio/AudioContext.h

    r113600 r113728  
    9494
    9595    AudioDestinationNode* destination() { return m_destinationNode.get(); }
    96     size_t currentSampleFrame() { return m_destinationNode->currentSampleFrame(); }
    97     double currentTime() { return m_destinationNode->currentTime(); }
    98     float sampleRate() { return m_destinationNode->sampleRate(); }
    99     unsigned long activeSourceCount() { return static_cast<unsigned long>(m_activeSourceCount); }
     96    size_t currentSampleFrame() const { return m_destinationNode->currentSampleFrame(); }
     97    double currentTime() const { return m_destinationNode->currentTime(); }
     98    float sampleRate() const { return m_destinationNode->sampleRate(); }
     99    unsigned long activeSourceCount() const { return static_cast<unsigned long>(m_activeSourceCount); }
    100100
    101101    void incrementActiveSourceCount();
  • trunk/Source/WebCore/Modules/webaudio/AudioDestinationNode.h

    r111474 r113728  
    4747    virtual void provideInput(AudioBus*, size_t numberOfFrames);
    4848
    49     size_t currentSampleFrame() { return m_currentSampleFrame; }
    50     double currentTime() { return currentSampleFrame() / static_cast<double>(sampleRate()); }
     49    size_t currentSampleFrame() const { return m_currentSampleFrame; }
     50    double currentTime() const { return currentSampleFrame() / static_cast<double>(sampleRate()); }
    5151
    5252    virtual unsigned numberOfChannels() const { return 2; } // FIXME: update when multi-channel (more than stereo) is supported
  • trunk/Source/WebCore/Modules/webaudio/AudioNode.cpp

    r111474 r113728  
    4343    , m_context(context)
    4444    , m_sampleRate(sampleRate)
    45     , m_lastProcessingTime(-1.0)
     45    , m_lastProcessingTime(-1)
     46    , m_lastNonSilentTime(-1)
    4647    , m_normalRefCount(1) // start out with normal refCount == 1 (like WTF::RefCounted class)
    4748    , m_connectionRefCount(0)
     
    178179    if (m_lastProcessingTime != currentTime) {
    179180        m_lastProcessingTime = currentTime; // important to first update this time because of feedback loops in the rendering graph
     181
    180182        pullInputs(framesToProcess);
    181         process(framesToProcess);
     183
     184        bool silentInputs = inputsAreSilent();
     185        if (!silentInputs)
     186            m_lastNonSilentTime = (context()->currentSampleFrame() + framesToProcess) / static_cast<double>(m_sampleRate);
     187
     188        if (silentInputs && propagatesSilence())
     189            silenceOutputs();
     190        else {
     191            process(framesToProcess);
     192            unsilenceOutputs();
     193        }
    182194    }
    183195}
     
    192204
    193205    input->updateInternalBus();
     206}
     207
     208bool AudioNode::propagatesSilence() const
     209{
     210    return m_lastNonSilentTime + latencyTime() + tailTime() < context()->currentTime();
    194211}
    195212
     
    201218    for (unsigned i = 0; i < m_inputs.size(); ++i)
    202219        input(i)->pull(0, framesToProcess);
     220}
     221
     222bool AudioNode::inputsAreSilent()
     223{
     224    for (unsigned i = 0; i < m_inputs.size(); ++i) {
     225        if (!input(i)->bus()->isSilent())
     226            return false;
     227    }
     228    return true;
     229}
     230
     231void AudioNode::silenceOutputs()
     232{
     233    for (unsigned i = 0; i < m_outputs.size(); ++i)
     234        output(i)->bus()->zero();
     235}
     236
     237void AudioNode::unsilenceOutputs()
     238{
     239    for (unsigned i = 0; i < m_outputs.size(); ++i)
     240        output(i)->bus()->clearSilentFlag();
    203241}
    204242
  • trunk/Source/WebCore/Modules/webaudio/AudioNode.h

    r112938 r113728  
    5555
    5656    AudioContext* context() { return m_context.get(); }
     57    const AudioContext* context() const { return m_context.get(); }
    5758
    5859    enum NodeType {
     
    144145    virtual double latencyTime() const = 0;
    145146
     147    // propagatesSilence() should return true if the node will generate silent output when given silent input. By default, AudioNode
     148    // will take tailTime() and latencyTime() into account when determining whether the node will propagate silence.
     149    virtual bool propagatesSilence() const;
     150    bool inputsAreSilent();
     151    void silenceOutputs();
     152    void unsilenceOutputs();
     153
    146154protected:
    147155    // Inputs and outputs must be created before the AudioNode is initialized.
     
    163171
    164172    double m_lastProcessingTime;
     173    double m_lastNonSilentTime;
    165174
    166175    // Ref-counting
  • trunk/Source/WebCore/Modules/webaudio/Oscillator.h

    r112938 r113728  
    7575    bool calculateSampleAccuratePhaseIncrements(size_t framesToProcess);
    7676
     77    // As a pure generator, Oscillator will never propagate silence.
     78    virtual bool propagatesSilence() const OVERRIDE { return false; }
     79
    7780    // One of the waveform types defined in the enum.
    7881    unsigned short m_type;
  • trunk/Source/WebCore/platform/audio/AudioBus.cpp

    r108604 r113728  
    425425    const float* sourceR = numberOfSourceChannels > 1 ? sourceBusSafe.channelByType(ChannelRight)->data() : 0;
    426426
     427    if (sourceBusSafe.isSilent()) {
     428        if (!sumToBus)
     429            zero();
     430        return;
     431    }
     432
    427433    float* destinationL = channelByType(ChannelLeft)->mutableData();
    428434    float* destinationR = numberOfDestinationChannels > 1 ? channelByType(ChannelRight)->mutableData() : 0;
     
    511517    }
    512518
     519    if (sourceBus.length() == numberOfGainValues && sourceBus.length() == length() && sourceBus.isSilent()) {
     520        zero();
     521        return;
     522    }
     523
    513524    // We handle both the 1 -> N and N -> N case here.
    514525    const float* source = sourceBus.channel(0)->data();
     
    540551    double sourceSampleRate = sourceBus->sampleRate();
    541552    double destinationSampleRate = newSampleRate;
     553    double sampleRateRatio = sourceSampleRate / destinationSampleRate;
    542554    unsigned numberOfSourceChannels = sourceBus->numberOfChannels();
    543555
     
    553565        return AudioBus::createBufferFromRange(sourceBus, 0, sourceBus->length());
    554566    }
    555    
     567
     568    if (sourceBus->isSilent()) {
     569        OwnPtr<AudioBus> silentBus = adoptPtr(new AudioBus(numberOfSourceChannels, sourceBus->length() / sampleRateRatio));
     570        silentBus->setSampleRate(newSampleRate);
     571        return silentBus.release();
     572    }
     573
    556574    // First, mix to mono (if necessary) then sample-rate convert.
    557575    const AudioBus* resamplerSourceBus;
     
    566584
    567585    // Calculate destination length based on the sample-rates.
    568     double sampleRateRatio = sourceSampleRate / destinationSampleRate;
    569586    int sourceLength = resamplerSourceBus->length();
    570587    int destinationLength = sourceLength / sampleRateRatio;
     
    583600    }
    584601
     602    destinationBus->clearSilentFlag();
    585603    destinationBus->setSampleRate(newSampleRate);   
    586604    return destinationBus.release();
     
    589607PassOwnPtr<AudioBus> AudioBus::createByMixingToMono(const AudioBus* sourceBus)
    590608{
     609    if (sourceBus->isSilent())
     610        return adoptPtr(new AudioBus(1, sourceBus->length()));
     611
    591612    switch (sourceBus->numberOfChannels()) {
    592613    case 1:
     
    606627                destination[i] = (sourceL[i] + sourceR[i]) / 2;
    607628
     629            destinationBus->clearSilentFlag();
    608630            destinationBus->setSampleRate(sourceBus->sampleRate());   
    609631            return destinationBus.release();
     
    615637}
    616638
     639bool AudioBus::isSilent() const
     640{
     641    for (size_t i = 0; i < m_channels.size(); ++i) {
     642        if (!m_channels[i]->isSilent())
     643            return false;
     644    }
     645    return true;
     646}
     647
     648void AudioBus::clearSilentFlag()
     649{
     650    for (size_t i = 0; i < m_channels.size(); ++i)
     651        m_channels[i]->clearSilentFlag();
     652}
     653
    617654} // WebCore
    618655
  • trunk/Source/WebCore/platform/audio/AudioBus.h

    r105431 r113728  
    8484    void zero();
    8585
     86    // Clears the silent flag on all channels.
     87    void clearSilentFlag();
     88
     89    // Returns true if the silent bit is set on all channels.
     90    bool isSilent() const;
     91
    8692    // Returns true if the channel count and frame-size match.
    8793    bool topologyMatches(const AudioBus &sourceBus) const;
  • trunk/Source/WebCore/platform/audio/AudioChannel.cpp

    r105431 r113728  
    4444void AudioChannel::scale(float scale)
    4545{
     46    if (isSilent())
     47        return;
     48
    4649    vsmul(data(), 1, &scale, mutableData(), 1, length());
    4750}
     
    5457        return;
    5558
     59    if (sourceChannel->isSilent()) {
     60        zero();
     61        return;
     62    }
    5663    memcpy(mutableData(), sourceChannel->data(), sizeof(float) * length());
    5764}
     
    5966void AudioChannel::copyFromRange(const AudioChannel* sourceChannel, unsigned startFrame, unsigned endFrame)
    6067{
     68    if (sourceChannel->isSilent() && isSilent())
     69        return;
     70
    6171    // Check that range is safe for reading from sourceChannel.
    6272    bool isRangeSafe = sourceChannel && startFrame < endFrame && endFrame <= sourceChannel->length();
     
    7484    const float* source = sourceChannel->data();
    7585    float* destination = mutableData();
    76     memcpy(destination, source + startFrame, sizeof(float) * rangeLength);
     86
     87    if (sourceChannel->isSilent()) {
     88        if (rangeLength == length())
     89            zero();
     90        else
     91            memset(destination, 0, sizeof(float) * rangeLength);
     92    } else
     93        memcpy(destination, source + startFrame, sizeof(float) * rangeLength);
    7794}
    7895
    7996void AudioChannel::sumFrom(const AudioChannel* sourceChannel)
    8097{
     98    if (sourceChannel->isSilent())
     99        return;
     100
    81101    bool isSafe = sourceChannel && sourceChannel->length() >= length();
    82102    ASSERT(isSafe);
     
    84104        return;
    85105
    86     vadd(data(), 1, sourceChannel->data(), 1, mutableData(), 1, length());
     106    if (isSilent())
     107        copyFrom(sourceChannel);
     108    else
     109        vadd(data(), 1, sourceChannel->data(), 1, mutableData(), 1, length());
    87110}
    88111
    89112float AudioChannel::maxAbsValue() const
    90113{
    91     float max = 0.0f;
     114    if (isSilent())
     115        return 0;
     116
     117    float max = 0;
    92118
    93119    vmaxmgv(data(), 1, &max, length());
  • trunk/Source/WebCore/platform/audio/AudioChannel.h

    r105431 r113728  
    4444    // Reference an external buffer.
    4545    AudioChannel(float* storage, size_t length)
    46         : m_length(length), m_rawPointer(storage) { }
     46        : m_length(length)
     47        , m_rawPointer(storage)
     48        , m_silent(false)
     49    {
     50    }
    4751
    4852    // Manage storage for us.
     
    5054        : m_length(length)
    5155        , m_rawPointer(0)
     56        , m_silent(true)
    5257    {
    5358        m_memBuffer = adoptPtr(new AudioFloatArray(length));
     
    5863        : m_length(0)
    5964        , m_rawPointer(0)
     65        , m_silent(true)
    6066    {
    6167    }
     
    6874        m_rawPointer = storage;
    6975        m_length = length;
     76        m_silent = false;
    7077    }
    7178
     
    7380    size_t length() const { return m_length; }
    7481
    75     // Direct access to PCM sample data
    76     float* mutableData() { return m_rawPointer ? m_rawPointer : m_memBuffer->data(); }
     82    // Direct access to PCM sample data. Non-const accessor clears silent flag.
     83    float* mutableData()
     84    {
     85        clearSilentFlag();
     86        return m_rawPointer ? m_rawPointer : m_memBuffer->data();
     87    }
     88
    7789    const float* data() const { return m_rawPointer ? m_rawPointer : m_memBuffer->data(); }
    7890
     
    8092    void zero()
    8193    {
     94        if (m_silent)
     95            return;
     96
     97        m_silent = true;
     98
    8299        if (m_memBuffer.get())
    83100            m_memBuffer->zero();
     
    85102            memset(m_rawPointer, 0, sizeof(float) * m_length);
    86103    }
     104
     105    // Clears the silent flag.
     106    void clearSilentFlag() { m_silent = false; }
     107
     108    bool isSilent() const { return m_silent; }
    87109
    88110    // Scales all samples by the same amount.
     
    106128    float* m_rawPointer;
    107129    OwnPtr<AudioFloatArray> m_memBuffer;
     130    bool m_silent;
    108131};
    109132
Note: See TracChangeset for help on using the changeset viewer.