⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 268826 in webkit


Ignore:
Timestamp:
Oct 21, 2020, 2:33:05 PM (6 years ago)
Author:
Chris Dumez
Message:

Update FFTFrame::realData() / imagData() to return an AudioFloatArray
​https://bugs.webkit.org/show_bug.cgi?id=218048

Reviewed by Philippe Normand.

Update FFTFrame::realData() / imagData() to return an AudioFloatArray instead of a float*.
It is risky to expose a float* since the caller has no way to know the actual size of the
array. By returning an AudioFloatArray, the caller can query AudioFloatArray::size() to
know the actual size. Note that it is not safe to assume that the size of these arrays
is FFTFrame::fftSize() because this is not true for ports using GStreamer.

  • Modules/webaudio/PeriodicWave.cpp:

(WebCore::PeriodicWave::createBandLimitedTables):

  • Modules/webaudio/RealtimeAnalyser.cpp:

(WebCore::RealtimeAnalyser::doFFTAnalysisIfNecessary):

  • platform/audio/FFTFrame.cpp:

(WebCore::FFTFrame::interpolateFrequencyComponents):
(WebCore::FFTFrame::scaleFFT):
(WebCore::FFTFrame::multiply):
(WebCore::FFTFrame::extractAverageGroupDelay):
(WebCore::FFTFrame::addConstantGroupDelay):
(WebCore::FFTFrame::print):

  • platform/audio/FFTFrame.h:

(WebCore::FFTFrame::realData):
(WebCore::FFTFrame::imagData):
(WebCore::FFTFrame::realData const):
(WebCore::FFTFrame::imagData const):

  • platform/audio/FFTFrameStub.cpp:

(WebCore::FFTFrame::realData const): Deleted.
(WebCore::FFTFrame::imagData const): Deleted.

  • platform/audio/gstreamer/FFTFrameGStreamer.cpp:

(WebCore::FFTFrame::FFTFrame):
(WebCore::FFTFrame::realData const): Deleted.
(WebCore::FFTFrame::imagData const): Deleted.

  • platform/audio/mac/FFTFrameMac.cpp:

(WebCore::FFTFrame::FFTFrame):
(WebCore::FFTFrame::doFFT):
(WebCore::FFTFrame::realData const): Deleted.
(WebCore::FFTFrame::imagData const): Deleted.

Location:
trunk/Source/WebCore
Files:
9 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r268825 r268826  
     12020-10-21  Chris Dumez  <cdumez@apple.com>
     2
     3        Update FFTFrame::realData() / imagData() to return an AudioFloatArray
     4        https://bugs.webkit.org/show_bug.cgi?id=218048
     5
     6        Reviewed by Philippe Normand.
     7
     8        Update FFTFrame::realData() / imagData() to return an AudioFloatArray instead of a float*.
     9        It is risky to expose a float* since the caller has no way to know the actual size of the
     10        array. By returning an AudioFloatArray, the caller can query AudioFloatArray::size() to
     11        know the actual size. Note that it is not safe to assume that the size of these arrays
     12        is FFTFrame::fftSize() because this is not true for ports using GStreamer.
     13
     14        * Modules/webaudio/PeriodicWave.cpp:
     15        (WebCore::PeriodicWave::createBandLimitedTables):
     16        * Modules/webaudio/RealtimeAnalyser.cpp:
     17        (WebCore::RealtimeAnalyser::doFFTAnalysisIfNecessary):
     18        * platform/audio/FFTFrame.cpp:
     19        (WebCore::FFTFrame::interpolateFrequencyComponents):
     20        (WebCore::FFTFrame::scaleFFT):
     21        (WebCore::FFTFrame::multiply):
     22        (WebCore::FFTFrame::extractAverageGroupDelay):
     23        (WebCore::FFTFrame::addConstantGroupDelay):
     24        (WebCore::FFTFrame::print):
     25        * platform/audio/FFTFrame.h:
     26        (WebCore::FFTFrame::realData):
     27        (WebCore::FFTFrame::imagData):
     28        (WebCore::FFTFrame::realData const):
     29        (WebCore::FFTFrame::imagData const):
     30        * platform/audio/FFTFrameStub.cpp:
     31        (WebCore::FFTFrame::realData const): Deleted.
     32        (WebCore::FFTFrame::imagData const): Deleted.
     33        * platform/audio/gstreamer/FFTFrameGStreamer.cpp:
     34        (WebCore::FFTFrame::FFTFrame):
     35        (WebCore::FFTFrame::realData const): Deleted.
     36        (WebCore::FFTFrame::imagData const): Deleted.
     37        * platform/audio/mac/FFTFrameMac.cpp:
     38        (WebCore::FFTFrame::FFTFrame):
     39        (WebCore::FFTFrame::doFFT):
     40        (WebCore::FFTFrame::realData const): Deleted.
     41        (WebCore::FFTFrame::imagData const): Deleted.
     42
    1432020-10-21  Zalan Bujtas  <zalan@apple.com>
    244

  • trunk/Source/WebCore/Modules/webaudio/PeriodicWave.cpp

    r268026 r268826  
    197197        // This FFTFrame is used to cull partials (represented by frequency bins).
    198198        FFTFrame frame(fftSize);
    199         float* realP = frame.realData();
    200         float* imagP = frame.imagData();
     199        auto& realP = frame.realData();
     200        auto& imagP = frame.imagData();
     201
     202        RELEASE_ASSERT(realP.size() >= numberOfComponents);
     203        RELEASE_ASSERT(imagP.size() >= numberOfComponents);
    201204
    202205        // Copy from loaded frequency data and scale.
    203         VectorMath::multiplyByScalar(realData, fftSize, realP, numberOfComponents);
    204         VectorMath::multiplyByScalar(imagData, -static_cast<float>(fftSize), imagP, numberOfComponents);
     206        VectorMath::multiplyByScalar(realData, fftSize, realP.data(), numberOfComponents);
     207        VectorMath::multiplyByScalar(imagData, -static_cast<float>(fftSize), imagP.data(), numberOfComponents);
    205208
    206209        // Find the starting bin where we should start culling.

  • trunk/Source/WebCore/Modules/webaudio/RealtimeAnalyser.cpp

    r268414 r268826  
    155155    m_analysisFrame->doFFT(tempP);
    156156
    157     float* realP = m_analysisFrame->realData();
    158     float* imagP = m_analysisFrame->imagData();
     157    auto& realP = m_analysisFrame->realData();
     158    auto& imagP = m_analysisFrame->imagData();
    159159
    160160    // Blow away the packed nyquist component.

  • trunk/Source/WebCore/platform/audio/AudioArray.h

    r268414 r268826  
    8282    }
    8383
     84    const T& at(size_t i) const
     85    {
     86        // Note that although it is a size_t, m_size is now guaranteed to be
     87        // no greater than max unsigned. This guarantee is enforced in resize().
     88        ASSERT_WITH_SECURITY_IMPLICATION(i < size());
     89        return data()[i];
     90    }
     91
    8492    T& operator[](size_t i) { return at(i); }
     93    const T& operator[](size_t i) const { return at(i); }
    8594
    8695    void zero()

  • trunk/Source/WebCore/platform/audio/FFTFrame.cpp

    r268026 r268826  
    7676    // FIXME : with some work, this method could be optimized
    7777
    78     float* realP = realData();
    79     float* imagP = imagData();
    80 
    81     const float* realP1 = frame1.realData();
    82     const float* imagP1 = frame1.imagData();
    83     const float* realP2 = frame2.realData();
    84     const float* imagP2 = frame2.imagData();
     78    auto& realP = realData();
     79    auto& imagP = imagData();
     80
     81    const auto& realP1 = frame1.realData();
     82    const auto& imagP1 = frame1.imagData();
     83    const auto& realP2 = frame2.realData();
     84    const auto& imagP2 = frame2.imagData();
    8585
    8686    m_FFTSize = frame1.fftSize();
    … …  
    176176void FFTFrame::scaleFFT(float factor)
    177177{
    178     VectorMath::multiplyByScalar(realData(), factor, realData(), fftSize());
    179     VectorMath::multiplyByScalar(imagData(), factor, imagData(), fftSize());
     178    VectorMath::multiplyByScalar(realData().data(), factor, realData().data(), realData().size());
     179    VectorMath::multiplyByScalar(imagData().data(), factor, imagData().data(), realData().size());
    180180}
    181181
    … …  
    185185    const FFTFrame& frame2 = frame;
    186186
    187     float* realP1 = frame1.realData();
    188     float* imagP1 = frame1.imagData();
    189     const float* realP2 = frame2.realData();
    190     const float* imagP2 = frame2.imagData();
     187    auto& realP1 = frame1.realData();
     188    auto& imagP1 = frame1.imagData();
     189    const auto& realP2 = frame2.realData();
     190    const auto& imagP2 = frame2.imagData();
    191191
    192192    unsigned halfSize = m_FFTSize / 2;
     193
     194    RELEASE_ASSERT(realP1.size() >= halfSize);
     195    RELEASE_ASSERT(imagP1.size() >= halfSize);
     196    RELEASE_ASSERT(realP2.size() >= halfSize);
     197    RELEASE_ASSERT(imagP2.size() >= halfSize);
     198
    193199    float real0 = realP1[0];
    194200    float imag0 = imagP1[0];
    195201
    196202    // Complex multiply
    197     VectorMath::multiplyComplex(realP1, imagP1, realP2, imagP2, realP1, imagP1, halfSize);
     203    VectorMath::multiplyComplex(realP1.data(), imagP1.data(), realP2.data(), imagP2.data(), realP1.data(), imagP1.data(), halfSize);
    198204
    199205    // Multiply the packed DC/nyquist component
    … …  
    204210double FFTFrame::extractAverageGroupDelay()
    205211{
    206     float* realP = realData();
    207     float* imagP = imagData();
     212    auto& realP = realData();
     213    auto& imagP = imagData();
    208214
    209215    double aveSum = 0.0;
    … …  
    255261    int halfSize = fftSize() / 2;
    256262
    257     float* realP = realData();
    258     float* imagP = imagData();
     263    auto& realP = realData();
     264    auto& imagP = imagData();
    259265
    260266    const double kSamplePhaseDelay = (2.0 * piDouble) / double(fftSize());
    … …  
    281287{
    282288    FFTFrame& frame = *this;
    283     float* realP = frame.realData();
    284     float* imagP = frame.imagData();
     289    auto& realP = frame.realData();
     290    auto& imagP = frame.imagData();
    285291    LOG(WebAudio, "**** \n");
    286292    LOG(WebAudio, "DC = %f : nyquist = %f\n", realP[0], imagP[0]);

  • trunk/Source/WebCore/platform/audio/FFTFrame.h

    r268559 r268826  
    6868    void scaleFFT(float factor);
    6969
    70     float* realData() const;
    71     float* imagData() const;
     70    AudioFloatArray& realData() { return m_realData; }
     71    AudioFloatArray& imagData() { return m_imagData; }
     72    const AudioFloatArray& realData() const { return m_realData; }
     73    const AudioFloatArray& imagData() const { return m_imagData; }
    7274
    7375    static int minFFTSize();
    … …  
    104106
    105107    DSPSplitComplex m_frame;
    106     AudioFloatArray m_realData;
    107     AudioFloatArray m_imagData;
    108108#endif
    109109
    … …  
    112112    GstFFTF32* m_inverseFft;
    113113    UniqueArray<GstFFTF32Complex> m_complexData;
     114#endif // USE(WEBAUDIO_GSTREAMER)
     115
    114116    AudioFloatArray m_realData;
    115117    AudioFloatArray m_imagData;
    116 #endif // USE(WEBAUDIO_GSTREAMER)
    117118};
    118119

  • trunk/Source/WebCore/platform/audio/FFTFrameStub.cpp

    r268559 r268826  
    7979}
    8080
    81 float* FFTFrame::realData() const
    82 {
    83     ASSERT_NOT_REACHED();
    84     return 0;
    85 }
    86 
    87 float* FFTFrame::imagData() const
    88 {
    89     ASSERT_NOT_REACHED();
    90     return 0;
    91 }
    92 
    9381} // namespace WebCore
    9482

  • trunk/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp

    r268559 r268826  
    8080
    8181    // Copy/setup frame data.
    82     memcpy(realData(), frame.realData(), sizeof(float) * unpackedFFTDataSize(m_FFTSize));
    83     memcpy(imagData(), frame.imagData(), sizeof(float) * unpackedFFTDataSize(m_FFTSize));
     82    memcpy(realData().data(), frame.realData().data(), sizeof(float) * realData().size());
     83    memcpy(imagData().data(), frame.imagData().data(), sizeof(float) * imagData().size());
    8484}
    8585
    … …  
    129129}
    130130
    131 float* FFTFrame::realData() const
    132 {
    133     return const_cast<float*>(m_realData.data());
    134 }
    135 
    136 float* FFTFrame::imagData() const
    137 {
    138     return const_cast<float*>(m_imagData.data());
    139 }
    140 
    141131int FFTFrame::minFFTSize()
    142132{

  • trunk/Source/WebCore/platform/audio/mac/FFTFrameMac.cpp

    r268559 r268826  
    6767// Creates a blank/empty frame (interpolate() must later be called)
    6868FFTFrame::FFTFrame()
    69     : m_realData(0)
    70     , m_imagData(0)
    7169{
    7270    // Later will be set to correct values when interpolate() is called
    … …  
    9189
    9290    // Copy/setup frame data
    93     unsigned nbytes = sizeof(float) * m_FFTSize;
    94     memcpy(realData(), frame.m_frame.realp, nbytes);
    95     memcpy(imagData(), frame.m_frame.imagp, nbytes);
     91    memcpy(realData().data(), frame.m_frame.realp, sizeof(float) * realData().size());
     92    memcpy(imagData().data(), frame.m_frame.imagp, sizeof(float) * imagData().size());
    9693}
    9794
    … …  
    104101    vDSP_fft_zrip(m_FFTSetup, &m_frame, 1, m_log2FFTSize, FFT_FORWARD);
    105102
     103    RELEASE_ASSERT(realData().size() >= halfSize);
     104    RELEASE_ASSERT(imagData().size() >= halfSize);
     105
    106106    // To provide the best possible execution speeds, the vDSP library's functions don't always adhere strictly
    107107    // to textbook formulas for Fourier transforms, and must be scaled accordingly.
    … …  
    109109    // In the case of a Real forward Transform like above: RFimp = RFmath * 2 so we need to divide the output
    110110    // by 2 to get the correct value.
    111     VectorMath::multiplyByScalar(realData(), 0.5, realData(), halfSize);
    112     VectorMath::multiplyByScalar(imagData(), 0.5, imagData(), halfSize);
     111    VectorMath::multiplyByScalar(realData().data(), 0.5, realData().data(), halfSize);
     112    VectorMath::multiplyByScalar(imagData().data(), 0.5, imagData().data(), halfSize);
    113113}
    114114
    … …  
    149149}
    150150
    151 float* FFTFrame::realData() const
    152 {
    153     return m_frame.realp;
    154 }
    155    
    156 float* FFTFrame::imagData() const
    157 {
    158     return m_frame.imagp;
    159 }
    160 
    161151} // namespace WebCore
    162152
Note: See TracChangeset for help on using the changeset viewer.