There are some research papers in the literature which apply Singular Value Decomposition (SVD) on an image to decompose the image matrix into U, S and V components. The eigenvalues in the singular matrix S are multiplied by a suitable factor and the image is then reconstructed. The resulting image's luminance is enhanced. To refer to one such paper, click here.
In curiosity, I was trying to study the variation of eigenvalues with respect to the luminance of an image. For calculating luminance, I am using log-average luminance described here. I will call log-average luminance Key Value from here on.
Experiment: I began my experiment by building a data set first. I took back-to-back multiple images of the same scene with different exposures.
For each image in such a set, I calculated luminance and obtained eigenvalues for each image. Following is the kind of graph I got for R, G and B channels respectively:
To clear any ambiguity about what these graphs represent, I'll explain briefly here. These graphs are for one set of images (11 images). For each image in the set, I calculated key-value. Next, I calculated average eigenvalues for each channel. For calculating average I used only the first 20 eigenvalues.
First, I observed one thing about eigenvalues which I don't understand. Not all the eigenvalues increase with the key-value of image. When key-value was more than ~0.85, out of 20 eigenvalues, only the first two or three tend to increase; the rest started decreasing. So this led to a turning point in the graph for images with key-value more than 0.85, so I neglected those images. Does this mean only the first few eigenvalues carry the maximum information about image brightness? Is there any reason for such observation?
Lastly, if I fit a curve to the obtained data points, can I generalize a formula for all images? So that for a given key-value I can get a fair estimate of eigenvalue (maximum or average). Or is there any other recommendation to fit a curve for getting a generalized formula?
Answer
Focusing too much on the eigenvalues neglects the eigenvectors which are more important here.
In a way, it is natural to associate eigenvalues with brightness but what is actually achieved with such techniques is contrast enhancement. This is not a mere scaling of the magnitude of the pixels.
For example, given an image $A$, its pixel brightness can be changed by $2 \times A$. $A$ will now appear brighter but the relationships between the values of the pixels will have been preserved. Therefore, any objects hidden in shaddows (small pixel differences relative to their background) will remain hidden in the shadows.
To see those objects, you would have to change the contrast, that is, the relationship BETWEEN pixel values. The most simple way to achieve this is with an image window (nothing to do with spectral windows). For more information about this please see this and this link. There are other ways to modify contrast, often equivalent to a complex window, such as equalisation, normalisation and others but these are not the only ways to modify contrast.
We usually tend to think of some matrix $A$ as a whole (e.g. an image). However, for the purposes of eigendecomposition, it is more useful to look at some $A$ with dimensions $M \times N$ as a way of grouping together a set of vectors from a multidimensional space.
So, in our mind, we now abandon the view of $A$ as a spatial index of intensity (which is what an image is) and we start thinking about a huge "box" into which we see lots and lots of little vectors that are pointing towards all sorts of directions.
Now, imagine that you take a completely new vector, of equal dimensions and you stick it in this space at some orientation. You can then estimate how many vectors are aligned with this direction and make a note of it that looks like (Direction, Number of vectors aligned with that direction). Change the direction of your "controlled vector" and do it again, do it again and again and again and again and again until you have a (very) long list of (Direction, Magnitude) measurements.
Obviously, a non random "vector cloud" such as the one we are imagining here will have some sort of orientation. That is, most of the vectors will be pointing mostly (not necessarily EXACTLY) towards some specific direction. This direction will have very high "counts" (of aligned vectors) and as the direction of your "controlled" vector moves away from this your counts will be decreasing rapidly.
Let's get back to the eigendecomposition now. Eigendecomposition is a way of discovering these pairs of (Direction, Magnitude) WITHOUT having to stick a "controlled" vector at different orientations and count alignments. In the end of this process, you get pairs of (Eigenvector (a.k.a Direction), Eigenvalue (a.k.a Magnitude)).
Just as a side note, if you can decompose the "vector cloud" into (Eigenvector, Eigenvalue) pairs then you can recompose it from these pairs too.
Back to our original problem: "So what? I don't have vectors, I have an image, how does playing around with little arrows affects the image?".
Let's say that your "vector cloud" assumes a particular form (a shape) in this multidimensional space. That's not hard to do because an image that is not noise does show something so the pixels do have some sort of order and are not all over the place. BUT, the vectors in this "vector cloud" might be corrupted by some random fluctuations in their directions (a.k.a noise). So, in this case, we can take advantage of the fact that a very large number of vectors are pointing in some general direction which we can recover. All we have to do now is RECOMPOSE the vector cloud keeping only a subset of the stronger (Eigenvector, Eigenvalue) pairs.
This, is the main idea. But it is not always applied directly on a given image.
For example, in the case of eigenfaces, the eigendecomposition is applied on the covariance matrix between images. This provides the general direction of a "vector cloud" that describes covariances between images. In effect, eigenfaces are a small set of elementary images that describe a very large set of faces as linear combinations.
In the first paper that you are linking in your original post, the eigendecomposition is applied on the Discrete Cosine Transform (DCT) field of the actual satelite images, again, the main idea is the same but here authors are discovering a set of elementary spatial frequency components.
For more information, please see this and this link.
Let us now turn briefly to your experiments. It is probably obvious by now that by focusing on the eigenvalues you are obtaining a rough approximation of the scene's brightness. Furthermore, the logarithmic dependence that you see in your graph is perfectly normal and due to the characteristic curve of the imaging device you are using (e.g. a digital camera at different f-number settings.)
If all you are trying to do is...
study the variation of eigen values with respect to luminance of an image
...then there will be some dependence but it will be indirect. As you are modifying the exposure of a given scene, you will pass from a set of obvious points.
Point-1: Image severely underexposued, in this case most of the image is dark, it has very low contrast and is associated with very low eigenvalues and very few eigenvectors...There is not much to describe here.
Point-2: Image perfectly exposed, in this case you are getting the maximum of contrast that can be obtained from a given scene with a given imaging device. You will get higher number of eigevectors and eigenvalues than point-1 because every pixel of your image now contains some information about the scene being imaged.
Point-3: Image severely over-exposed, in this case you are again operating at a region of low contrast but this time it is because the sensor is saturated. Any sort of difference in luminus flux goes unnoticed by the sensor which is simply recording the maximum of value it can produce at its output. Consequently, you get a mostly washed out image which again has a lower number of eigenvectors (but possibly high eigenvalues).
Hope this helps.
No comments:
Post a Comment