Some help with FFT time stretching anyone?

Question

Maybe someone would like to contribute to this Sound

 

The basic idea is:

- analyze the input with FFT using a small step size / large overlap

- read out the magnitudes at half speed (or quarter etc.)

- discard the phase and replace with random phase information (not necessary for the stretch, but sounds better)

- inv FFT with large step size / small overlap (half, quarter, etc.)

 

The problem is I hear artifacts due to windowing or wrong delay times after the inv FFT. I checked everything like a hundred times but I think I'm missing something here.. Community help? :)

 

Thanks,

Gustav

Answer 1

I came back to this and was able to solve it, it wasn't working for two reasons: - I should actually apply a window on the invFFT as well, especially if I do phase randomisation I can't expect the output to be windowed - when using a different hop size for the invFFT I need to compensate for the amplitude modulation due to the non-perfect overlap. Interestingly this has been obvious to me right now, while it wasn't at all back then

Answer 2

If you want a large overlap you cannot put it through the normal single stereo wire (ie multipul paralel FFTs with time offests). You will need many stereo wires. Then you would need some funky delays and poling switches to both time stretch them slot them into the correct time slice. Do you expect the output signal to come out at the same rate as the input signal going in and if so, how? The overlapping data means you have more data per input sample than normal. How is that extra data to be used? More output FFTs mixed together? whats the goal?

Comments

I'm replicating the FFT and set the delays accordingly for large overlaps. Did you see that you can download the Sound by clicking "this Sound" above? The output signal will come out at half speed (or quarter etc.) so I need to buffer inside which I already took care of. The extra data will be used to mask the "windowed" sound if that makes sense. The overall goal is a nice timestretching effect. Welcome to the Forum Pete! Nice to see you here :)

Answer 3

Ok, I really simplified my example now: 

Download the Sound here

Although I'm just using the magnitude I get this strange windowing artefacts.. Why is that? And how can I avoid it?

Thanks!

Answer 4

hey gustl, 

 

i made some tests with your mag-smoother and the only way i could get rid of the artifacts was by using a 4 or 8 times overlap and an aditional windowing right after the inverse fft stage. here are the  sounds:

http://kyma.symbolicsound.com/qa/?qa=blob&qa_blobid=16378956257336811082

i didnt understand why your sound creates these artifacts either, since the concept of feeding the same signal twice ( with a specific overlapfactor ) into one fft module remains the same. it might have something to do with  the feedbacking delay (that you use for the mag smoothing) in combination with a 2 times overlap. 

 maybe carla or kurt can help with that.

 

ps. iam looking forward to hear the differences between fft timestretcher and the wire in between based one.

ciao, jo

Comments

Thanks johannes, yes if you use more overlap and window the invFFT as well the artefacts are nearly gone, but not all the way.. they are still there but less audible...
The difference should be that I can do phase randomisation and I could use large windows - I'm eager to hear it myself :)

---

Hi Gustl . not fully got my head around how it's supposed to be working but the normal single Kyma FFT creates a half overlap. It does this using the two internal half's inside the module where the left input leg and the right input leg work independently but at the same time (no half frame shift hence no overlap). You have to make the overlap (half frame shift) by delaying one leg on the way in and the other leg on the way out after the inverse FFT. As you are feeding the FFT with a mono signal you are just getting two identical frame pairs and hence you are not doing a four times overlap just a standard half overlap but using two FFT modules where one would do the same thing. I hope this makes sense ? Pete

---

Totally makes sense, I understand how it works! I just made two FFT modules to make it more clear, I guess..
By now I realized what the origin of the problem is: Imagine you want to freeze a spectrum. Using Kyma Spectra it's easy because there are no overlaps so you just repeat the same frame over and over.. but if you want to do it with the fft you will have overlaps, and you get the same amount of spectra as overlaps. so for a quarter overlap you compute 4 spectra at the same time. now if you freeze you will repeat 4 overlapping spectra over and over. that's the reason why you end up with a "window artefact" sound again - at least that's what I think...