如何过滤FFT数据(用于视听)?
我正在看这个Web Audio API演示,这是这本好书的一部分
如果您看一下演示,fft峰值会平稳下降。我正在尝试使用minim库在Java模式下进行处理。我已经研究了如何在doFFTAnalysis()方法中使用Web音频api完成此操作,并尝试使用minim复制它。我还尝试移植 abs() 如何与复杂类型一起工作:
/ 26.2.7/3 abs(__z): Returns the magnitude of __z.
00565 template<typename _Tp>
00566 inline _Tp
00567 __complex_abs(const complex<_Tp>& __z)
00568 {
00569 _Tp __x = __z.real();
00570 _Tp __y = __z.imag();
00571 const _Tp __s = std::max(abs(__x), abs(__y));
00572 if (__s == _Tp()) // well ...
00573 return __s;
00574 __x /= __s;
00575 __y /= __s;
00576 return __s * sqrt(__x * __x + __y * __y);
00577 }
00578
我目前正在使用Processing(一个java框架/库)做一个快速原型。我的代码如下所示:
import ddf.minim.*;
import ddf.minim.analysis.*;
private int blockSize = 512;
private Minim minim;
private AudioInput in;
private FFT mfft;
private float[] time = new float[blockSize];//time domain
private float[] real = new float[blockSize];
private float[] imag = new float[blockSize];
private float[] freq = new float[blockSize];//smoothed freq. domain
public void setup() {
minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, blockSize);
mfft = new FFT( in.bufferSize(), in.sampleRate() );
}
public void draw() {
background(255);
for (int i = 0; i < blockSize; i++) time[i] = in.left.get(i);
mfft.forward( time);
real = mfft.getSpectrumReal();
imag = mfft.getSpectrumImaginary();
final float magnitudeScale = 1.0 / mfft.specSize();
final float k = (float)mouseX/width;
for (int i = 0; i < blockSize; i++)
{
float creal = real[i];
float cimag = imag[i];
float s = Math.max(creal,cimag);
creal /= s;
cimag /= s;
float absComplex = (float)(s * Math.sqrt(creal*creal + cimag*cimag));
float scalarMagnitude = absComplex * magnitudeScale;
freq[i] = (k * mfft.getBand(i) + (1 - k) * scalarMagnitude);
line( i, height, i, height - freq[i]*8 );
}
fill(0);
text("smoothing: " + k,10,10);
}
我没有遇到错误,这很好,但我没有得到预期的不良行为。我预计当平滑(k)接近1时,峰值下降得更慢,但据我所知,我的代码只会缩放波段。
不幸的是,数学和声音不是我的强项,所以我在黑暗中刺痛。如何复制 Web 音频 API 演示中漂亮的可视化效果?
我很想说这可能是语言不可知的,但例如使用javascript不适用于:)。但是,我很乐意尝试任何其他进行FFT分析的java库。
更新
我有一个简单的平滑解决方案(如果当前fft波段不高,则连续减少每个先前fft波段的值:
import ddf.minim.analysis.*;
import ddf.minim.*;
Minim minim;
AudioInput in;
FFT fft;
float smoothing = 0;
float[] fftReal;
float[] fftImag;
float[] fftSmooth;
int specSize;
void setup(){
size(640, 360, P3D);
minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, 512);
fft = new FFT(in.bufferSize(), in.sampleRate());
specSize = fft.specSize();
fftSmooth = new float[specSize];
fftReal = new float[specSize];
colorMode(HSB,specSize,100,100);
}
void draw(){
background(0);
stroke(255);
fft.forward( in.left);
fftReal = fft.getSpectrumReal();
fftImag = fft.getSpectrumImaginary();
for(int i = 0; i < specSize; i++)
{
float band = fft.getBand(i);
fftSmooth[i] *= smoothing;
if(fftSmooth[i] < band) fftSmooth[i] = band;
stroke(i,100,50);
line( i, height, i, height - fftSmooth[i]*8 );
stroke(i,100,100);
line( i, height, i, height - band*8 );
}
text("smoothing: " + (int)(smoothing*100),10,10);
}
void keyPressed(){
float inc = 0.01;
if(keyCode == UP && smoothing < 1-inc) smoothing += inc;
if(keyCode == DOWN && smoothing > inc) smoothing -= inc;
}
褪色的图形是平滑的图形,完全饱和的图形是实时图形。
然而,与Web Audio API演示相比,我仍然缺少一些东西:
我认为Web Audio API可能会考虑到中频和高频需要扩展以更接近我们的感知,但我不确定如何解决这个问题。
我试图阅读更多关于实时分析器类如何为WebAudioAPI执行此操作的信息,但似乎FFTFrame类的方法可能会进行对数缩放。我还没有弄清楚doFFT是如何工作的。doFFT
如何使用对数刻度缩放原始FFT图以考虑感知?我的目标是做一个体面的可视化,我的猜测是我需要:
- 平滑值,否则元素将动画化为快速/抽搐
- 缩放FFT箱/频段,以获得更好的中/高频数据
- 将流程 FFT 值映射到可视元素(查找最大值/边界)
关于我如何实现这一点的任何提示?
更新 2
我猜这部分做了我在Web Audio API中追求的平滑和缩放://归一化,所以比0dBfs处的输入正弦波寄存为0dBfs(撤消FFT缩放因子)。常数双倍幅度尺度 = 1.0 / 默认FFT大小;
// A value of 0 does no averaging with the previous result. Larger values produce slower, but smoother changes.
double k = m_smoothingTimeConstant;
k = max(0.0, k);
k = min(1.0, k);
// Convert the analysis data from complex to magnitude and average with the previous result.
float* destination = magnitudeBuffer().data();
size_t n = magnitudeBuffer().size();
for (size_t i = 0; i < n; ++i) {
Complex c(realP[i], imagP[i]);
double scalarMagnitude = abs(c) * magnitudeScale;
destination[i] = float(k * destination[i] + (1 - k) * scalarMagnitude);
}
似乎缩放是通过取复数值的绝对值来完成的。这篇文章指向同一个方向。我尝试过使用Minim使用复数的abs并使用各种窗口函数,但它看起来仍然不像我的目标(Web Audio API演示):
import ddf.minim.analysis.*;
import ddf.minim.*;
Minim minim;
AudioInput in;
FFT fft;
float smoothing = 0;
float[] fftReal;
float[] fftImag;
float[] fftSmooth;
int specSize;
WindowFunction[] window = {FFT.NONE,FFT.HAMMING,FFT.HANN,FFT.COSINE,FFT.TRIANGULAR,FFT.BARTLETT,FFT.BARTLETTHANN,FFT.LANCZOS,FFT.BLACKMAN,FFT.GAUSS};
String[] wlabel = {"NONE","HAMMING","HANN","COSINE","TRIANGULAR","BARTLETT","BARTLETTHANN","LANCZOS","BLACKMAN","GAUSS"};
int windex = 0;
void setup(){
size(640, 360, P3D);
minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, 512);
fft = new FFT(in.bufferSize(), in.sampleRate());
fft.window(window[windex]);
specSize = fft.specSize();
fftSmooth = new float[specSize];
fftReal = new float[specSize];
colorMode(HSB,specSize,100,100);
}
void draw(){
background(0);
stroke(255);
fft.forward( in.mix);
fftReal = fft.getSpectrumReal();
fftImag = fft.getSpectrumImaginary();
for(int i = 0; i < specSize; i++)
{
float band = fft.getBand(i);
//Sw = abs(Sw(1:(1+N/2))); %# abs is sqrt(real^2 + imag^2)
float abs = sqrt(fftReal[i]*fftReal[i] + fftImag[i]*fftImag[i]);
fftSmooth[i] *= smoothing;
if(fftSmooth[i] < abs) fftSmooth[i] = abs;
stroke(i,100,50);
line( i, height, i, height - fftSmooth[i]*8 );
stroke(i,100,100);
line( i, height, i, height - band*8 );
}
text("smoothing: " + (int)(smoothing*100)+"\nwindow:"+wlabel[windex],10,10);
}
void keyPressed(){
float inc = 0.01;
if(keyCode == UP && smoothing < 1-inc) smoothing += inc;
if(keyCode == DOWN && smoothing > inc) smoothing -= inc;
if(key == 'W' && windex < window.length-1) windex++;
if(key == 'w' && windex > 0) windex--;
if(key == 'w' || key == 'W') fft.window(window[windex]);
}
我不确定我是否正确使用了窗口函数,因为我没有注意到它们之间的巨大差异。复数值的 abs 是否正确?如何使可视化更接近我的目标?
更新 3
我试图应用@wakjah有用的提示,如下所示:
import ddf.minim.analysis.*;
import ddf.minim.*;
Minim minim;
AudioInput in;
FFT fft;
float smoothing = 0;
float[] fftReal;
float[] fftImag;
float[] fftSmooth;
float[] fftPrev;
float[] fftCurr;
int specSize;
WindowFunction[] window = {FFT.NONE,FFT.HAMMING,FFT.HANN,FFT.COSINE,FFT.TRIANGULAR,FFT.BARTLETT,FFT.BARTLETTHANN,FFT.LANCZOS,FFT.BLACKMAN,FFT.GAUSS};
String[] wlabel = {"NONE","HAMMING","HANN","COSINE","TRIANGULAR","BARTLETT","BARTLETTHANN","LANCZOS","BLACKMAN","GAUSS"};
int windex = 0;
int scale = 10;
void setup(){
minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, 512);
fft = new FFT(in.bufferSize(), in.sampleRate());
fft.window(window[windex]);
specSize = fft.specSize();
fftSmooth = new float[specSize];
fftPrev = new float[specSize];
fftCurr = new float[specSize];
size(specSize, specSize/2);
colorMode(HSB,specSize,100,100);
}
void draw(){
background(0);
stroke(255);
fft.forward( in.mix);
fftReal = fft.getSpectrumReal();
fftImag = fft.getSpectrumImaginary();
for(int i = 0; i < specSize; i++)
{
//float band = fft.getBand(i);
//Sw = abs(Sw(1:(1+N/2))); %# abs is sqrt(real^2 + imag^2)
//float abs = sqrt(fftReal[i]*fftReal[i] + fftImag[i]*fftImag[i]);
//fftSmooth[i] *= smoothing;
//if(fftSmooth[i] < abs) fftSmooth[i] = abs;
//x_dB = 10 * log10(real(x) ^ 2 + imag(x) ^ 2);
fftCurr[i] = scale * (float)Math.log10(fftReal[i]*fftReal[i] + fftImag[i]*fftImag[i]);
//Y[k] = alpha * Y_(t-1)[k] + (1 - alpha) * X[k]
fftSmooth[i] = smoothing * fftPrev[i] + ((1 - smoothing) * fftCurr[i]);
fftPrev[i] = fftCurr[i];//
stroke(i,100,100);
line( i, height, i, height - fftSmooth[i]);
}
text("smoothing: " + (int)(smoothing*100)+"\nwindow:"+wlabel[windex]+"\nscale:"+scale,10,10);
}
void keyPressed(){
float inc = 0.01;
if(keyCode == UP && smoothing < 1-inc) smoothing += inc;
if(keyCode == DOWN && smoothing > inc) smoothing -= inc;
if(key == 'W' && windex < window.length-1) windex++;
if(key == 'w' && windex > 0) windex--;
if(key == 'w' || key == 'W') fft.window(window[windex]);
if(keyCode == LEFT && scale > 1) scale--;
if(keyCode == RIGHT) scale++;
}
我不确定我是否按预期应用了提示。以下是我的输出外观:
但是,如果我将其与我的目标可视化进行比较,我认为我还没有达到目标:
视窗媒体播放器中的频谱
VLC播放器中的频谱
我不确定是否正确应用了对数缩放。我的假设是,我会绘制一个类似于我在使用log10后的目标的绘图(暂时忽略平滑)。
更新 4:
import ddf.minim.analysis.*;
import ddf.minim.*;
Minim minim;
AudioInput in;
FFT fft;
float smoothing = 0;
float[] fftReal;
float[] fftImag;
float[] fftSmooth;
float[] fftPrev;
float[] fftCurr;
int specSize;
WindowFunction[] window = {FFT.NONE,FFT.HAMMING,FFT.HANN,FFT.COSINE,FFT.TRIANGULAR,FFT.BARTLETT,FFT.BARTLETTHANN,FFT.LANCZOS,FFT.BLACKMAN,FFT.GAUSS};
String[] wlabel = {"NONE","HAMMING","HANN","COSINE","TRIANGULAR","BARTLETT","BARTLETTHANN","LANCZOS","BLACKMAN","GAUSS"};
int windex = 0;
int scale = 10;
void setup(){
minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, 512);
fft = new FFT(in.bufferSize(), in.sampleRate());
fft.window(window[windex]);
specSize = fft.specSize();
fftSmooth = new float[specSize];
fftPrev = new float[specSize];
fftCurr = new float[specSize];
size(specSize, specSize/2);
colorMode(HSB,specSize,100,100);
}
void draw(){
background(0);
stroke(255);
fft.forward( in.mix);
fftReal = fft.getSpectrumReal();
fftImag = fft.getSpectrumImaginary();
for(int i = 0; i < specSize; i++)
{
float maxVal = Math.max(Math.abs(fftReal[i]), Math.abs(fftImag[i]));
if (maxVal != 0.0f) { // prevent divide-by-zero
// Normalize
fftReal[i] = fftReal[i] / maxVal;
fftImag[i] = fftImag[i] / maxVal;
}
fftCurr[i] = -scale * (float)Math.log10(fftReal[i]*fftReal[i] + fftImag[i]*fftImag[i]);
fftSmooth[i] = smoothing * fftSmooth[i] + ((1 - smoothing) * fftCurr[i]);
stroke(i,100,100);
line( i, height/2, i, height/2 - (mousePressed ? fftSmooth[i] : fftCurr[i]));
}
text("smoothing: " + (int)(smoothing*100)+"\nwindow:"+wlabel[windex]+"\nscale:"+scale,10,10);
}
void keyPressed(){
float inc = 0.01;
if(keyCode == UP && smoothing < 1-inc) smoothing += inc;
if(keyCode == DOWN && smoothing > inc) smoothing -= inc;
if(key == 'W' && windex < window.length-1) windex++;
if(key == 'w' && windex > 0) windex--;
if(key == 'w' || key == 'W') fft.window(window[windex]);
if(keyCode == LEFT && scale > 1) scale--;
if(keyCode == RIGHT) scale++;
}
产生这个:
在绘制循环中,我从中心绘制,因为比例现在是负的。如果我放大这些值,结果开始看起来很随机。
更新6
import ddf.minim.analysis.*;
import ddf.minim.*;
Minim minim;
AudioInput in;
FFT fft;
float smoothing = 0;
float[] fftReal;
float[] fftImag;
float[] fftSmooth;
float[] fftPrev;
float[] fftCurr;
int specSize;
WindowFunction[] window = {FFT.NONE,FFT.HAMMING,FFT.HANN,FFT.COSINE,FFT.TRIANGULAR,FFT.BARTLETT,FFT.BARTLETTHANN,FFT.LANCZOS,FFT.BLACKMAN,FFT.GAUSS};
String[] wlabel = {"NONE","HAMMING","HANN","COSINE","TRIANGULAR","BARTLETT","BARTLETTHANN","LANCZOS","BLACKMAN","GAUSS"};
int windex = 0;
int scale = 10;
void setup(){
minim = new Minim(this);
in = minim.getLineIn(Minim.STEREO, 512);
fft = new FFT(in.bufferSize(), in.sampleRate());
fft.window(window[windex]);
specSize = fft.specSize();
fftSmooth = new float[specSize];
fftPrev = new float[specSize];
fftCurr = new float[specSize];
size(specSize, specSize/2);
colorMode(HSB,specSize,100,100);
}
void draw(){
background(0);
stroke(255);
fft.forward( in.mix);
fftReal = fft.getSpectrumReal();
fftImag = fft.getSpectrumImaginary();
for(int i = 0; i < specSize; i++)
{
fftCurr[i] = scale * (float)Math.log10(fftReal[i]*fftReal[i] + fftImag[i]*fftImag[i]);
fftSmooth[i] = smoothing * fftSmooth[i] + ((1 - smoothing) * fftCurr[i]);
stroke(i,100,100);
line( i, height/2, i, height/2 - (mousePressed ? fftSmooth[i] : fftCurr[i]));
}
text("smoothing: " + (int)(smoothing*100)+"\nwindow:"+wlabel[windex]+"\nscale:"+scale,10,10);
}
void keyPressed(){
float inc = 0.01;
if(keyCode == UP && smoothing < 1-inc) smoothing += inc;
if(keyCode == DOWN && smoothing > inc) smoothing -= inc;
if(key == 'W' && windex < window.length-1) windex++;
if(key == 'w' && windex > 0) windex--;
if(key == 'w' || key == 'W') fft.window(window[windex]);
if(keyCode == LEFT && scale > 1) scale--;
if(keyCode == RIGHT) scale++;
if(key == 's') saveFrame("fftmod.png");
}
这感觉如此接近:
这看起来比以前的版本好得多,但是光谱下/左侧的某些值看起来有点偏差,形状变化似乎非常快。(平滑值绘制零)