FFT on non-rectangular part of image

I need to do a FFT on an image for noise reduction, but the problem is that I do not need the complete image, but only a circle in the middle. The borders are a fixed rig, thus I am not interested in what it displays, but it has an impact on the result of the FFT.

Is there any way to just cut out a circular part and use that for the FFT? Note that if I use black background, the edge between background, and image data will have quite an impact.

Answer

Instead of having a hard edge between the image data of interest and the background, you could use a two-dimensional tapered window function, as is often done in spectral analysis. You might start by trying a Gaussian window, which for a two-dimensional case would look something like:

$$w[x,y] = e^{-\frac{\left(x-\frac{N_x-1}{2}\right)^2}{2\left(\sigma_x \frac{N_x-1}{2}\right)^2}} e^{-\frac{\left(y-\frac{N_y-1}{2}\right)^2}{2\left(\sigma_y \frac{N_y-1}{2}\right)^2}}$$

$N_x$ and $N_y$ are the dimensions of the desired transform in the $x$ and $y$ directions, respectively, and $\sigma_x$ and $\sigma_y$ are parameters that allow you to control the shape of the window; for small $\sigma$ values, most of the energy in the window function will be concentrated toward the center, with that effect decreasing as you increase $\sigma$.