INTRODUCTION
OVERVIEW
The difficulties faced by text-based retrieval became more and more severe.The efficient management of the rapidly expanding visual information became an urgent problem . The need formed the driving force behind the emergence of content-based image retrieval techniques.
A more efficient and intuitive way to represent and index visual information would be based on properties that are inherent in the images themselves. Researchers from the communities of computervision,database management, human-computer interface and information retrieval were attracted to this field. The no of research publications on the techniques of visual information extraction ,organizations,indexing,user query interaction and database management has increased enormously.Similarly a large no.of academic and commercial retrieval systems have been developed by universities ,government organizations , companies and hospitals.
CBIR uses the visual contents of an image such as colour ,shape,texture and spatial layout to represent and index the image. In typical content-based image retrieval systems ,the visual contents of the images in the database are extracted and described by multi-dimensional feature vectors. The feature vectors of the images in the database form a feature database. To retrieve images, users provide the retrieval system with example images or sketched figures.
1.2 CONTENT BASED IMAGE RETRIEVAL (CBIR)
Content-based image retrieval (CBIR) scheme searches the most-similar images of a query image that involves in comparing the feature vectors of all the images in the database with that of the query image using some pre-selected similarity measure, and then sorting of the results. On querying an image, a reduced set of candidate images which have the same features as that of the query image is obtained. After obtaining the reduced set of images, they are sorted based on their similarity values. The most similar images are then retrieved for the user. Color feature uses image histogram technique, entropy gives statistical representation of image and text feature gives polite and regularity of histogram. It is most widely used technique for managing, searching, browsing and extracting visual content of image from large collection of images. These features are stored in database for further use.
When an image is to be found out, a query image for matching is provided, the features of query image are extracted and matched with the features of the stored database images, so that a group of similar images come for the query image as a result. CBIR uses an automatic indexing scheme, to reduce search time of retrieval system from the database.
Feature extraction is a key issue in content-based image retrieval (CBIR). In the past, a number of texture features have been proposed in literature, including statistic methods and spectral methods. However, most of them are not able to accurately capture the edge information which is the most important texture feature in an image. Recent researches on multi-scale analysis, especially the curvelet research, provide good opportunity to extract more accurate texture feature for image retrieval. Curvelet was originally proposed for image de-noising and has shown promising performance. For capturing the edge information accurately, Fast Discrete Curvelet Transforms (FDCT) are used and results show that FDCT performs significantly well when compared to the widely used Gabor wavelet method.
2.2 FAST DISCRETE CURVELET TRANSFORM
The proposed work is based on the wrapping of Fourier samples that has less computational complexity as it uses fast Fourier transform instead of complex ridgelet transform. In this approach, a tight frame has been introduced as the curvelet support to reduce the data redundancy in the frequency domain. Normally, rideglets have a fixed length that is equal to the image size and a variable width, whereas curvelets have both variable width and length and represent more anisotropy. Therefore, the wrapping based curvelet transform is simpler, less redundant and faster in computation than ridgelet based curvelet transform. A Discrete curvelet transform based on wrapping Fourier samples is the most promising approach of curvelet. So far, it is intend to use for texture representation in the CBIR research.
Curvelet transform based on wrapping of Fourier samples takes a 2-D image as input in the form of a Cartesian array f [m, n] such that 0 ≤ m < M, 0 ≤ n < N and generates a number of curvelet coefficients indexed by a scale j , an orientation l and two spatial location parameters (k1, k2 ) as output. To form the curvelet texture descriptor, statistical operations are applied to these coefficients. Discrete curvelet coefficients can be defined by:
C^D (j,l,k_(1,) k_2 )= ∑_█(0≤m < M@0≤n<N )▒f(m,n) ϕ_(j,1,k_(1,) k_2)^D [m,n] (3)
Here, each Ï•j,l,k1,k2[m,n] is a digital curvelet waveform. This curvelet approach implements the effective parabolic scaling law on the sub-bands in the frequency domain to capture curved edges within an image more effectively. Curvelets exhibit an oscillating behavior in the direction perpendicular to their orientation in frequency domain.
Basically, wrapping based curvelet transform is a multi-scale transform with a pyramid structure consisting of many orientations at each scale. This pyramid structure consists of several sub-bands at different scales in the frequency domain. Sub-bands at high and low frequency levels have different orientations and positions. At high scales, the curvelet waveform becomes so fine that it looks like a needle shaped element . Whereas, the curvelet is non directional at the coarsest scale. With increase in the resolution level the curvelet becomes finer and smaller in the spatial domain and shows more sensitivity to curved edges which enables it to effectively capture the curves in an image. As a consequence, curved singularities can be well-approximated with few coefficients.
High frequency components of an image play a vital role in finding distinction between images. Curvelets at fine scales effectively represent edges by using texture features computed from the curvelet coefficients.
The frequency responses of curvelets at different scales and orientations was combined, then a rectangular frequency tiling that covers the whole image in the spectral domain will obtained. Thus, the curvelet spectra completely cover the frequency plane and there is no loss of spectral information like the Gabor filters.
Higher level of efficiency with curvelet transform is usually implemented in the frequency domain. That is, both the curvelet and the image are transformed and are then multiplied in the Fourier frequency domain. The product is then inverse Fourier transformed to obtain the curvelet coefficients. The process can be described as
Curvelet transform = IFFT [ FFT(Curvelet) × FFT(Image)] (4)
And the product from the multiplication is a wedge. The trapezoidal wedge in the spectral domain is not suitable for use with the inverse Fourier transform which is the next step in collecting the curvelet coefficients using IFFT. The wedge data cannot be accommodated directly into a rectangle of size 2j ×2j/2. To overcome this problem, researchers have formulated a wedge wrapping procedure where a parallelogram with sides 2j and 2j/2 is chosen as a support to the wedge data. The wrapping is done by periodic tiling of the spectrum inside the wedge and then collecting the rectangular coefficient area in the center. The center rectangle of size 2j × 2j/2 successfully collects all the information in that parallelogram.
C^D (j,l,k_(1,) k_2 )= ∑_█(0≤m < M@0≤n<N )▒f(m,n) ϕ_(j,1,k_(1,) k_2)^D [m,n] (5)
Where Ï•_(j,1,k_(1,) k_2)^Dis the curvelet waveform. This transform generates an array of curvelet
coefficients indexed by their scale j, orientation l and location parameters (k1, k2).
3 IMPLEMENTATION
3.1 GENERAL IDEA
In any content based image retrieval system, any combination of the features can be used to generate the feature vector and use that to compare the input query image with the image database.
In the proposed project, color feature is extracted using color histogram and texture and edge features are extracted using fast discrete curvelet transform. The proposed project consists of two sub-modules: one for extracting only color feature and other for extracting edge and texture feature.
Module 1: Figure 17 shows the basic steps involved in the first module which deals with color feature extraction using color histogram.
Module 2: The second module of the proposed system consists of extracting edge and texture
features using fast discrete curvelet transform (FDCT). Figure 18 shows the major
steps involved in this part of the proposed system.
The overall system can be summarized into the following steps:
Step 1: Creating and loading feature database A database consisting of 600 images are taken
The feature vector for module 1 and module 2 are calculated separately and stored.
Step 2: Query image selection A query image is provided to the system
Same feature vectors are calculated for the query image as that of the database
images
Step 3: Calculation of similarity value For the first module, the similarity is calculated using
equation 7. For the second module, the similarity value is calculated using equation 8.
Step 4: Similarity matching and indexing of database images Once the similarity values are
calculate for all the database images, the images of the database are then organized in
ascending order with respect to the similarity value. A mapping is performed in order
to relate the indexes of the image in the database to their corresponding similarity vaaues.
Step 5: Retrieval The similar images are sorted and stored in increasing order of their
similarity values.Out of those images, only the first ‘n’ images are taken and displayed
to the user.
DETAILED DESCRIPTION
Module 1: The first module is about taking the query image, calculating its color histogram. Same color histograms are calculated for the images in the database. Then, comparison of the color histogram of query image with the database images is performed.
For the first module, figure 19 shows some of the images present in the database of 600 images. For all these images, color histogram is computed and stored as the feature database.
Figure 4 Sample images from the image database
The detailed steps involved in calculating the color histogram are as follows:
600 images (sample images shown in figure 19) are taken to populate the database. The user inputs a query image to the system. A 3D histogram of the HSV values is calculated for both the input and database images.
The histograms of the input image and database images are compared.For each image ‘i’, distance ‘D’ is calculated between the query image histogram and one of the database image histograms using chi-square distance:
D_chisquare (H,G)= ∑_(i=1)^L▒(H(l)- (H(l)-G(l))/2)^2/((H(i)+G(i))/2) (6)
Only distances D2 among D are kept, which are greater than a predefined threshold T (=0.01). Let L2 be the number of the above distances.
A smaller set D3 out of these distances, is calculated which are greater than a second threshold T2 (=0.8). Let L3 be the number of D3 distances. Then, the similarity between the query image and the image ‘i’ is calculated as:
〖 S〗_i= (L_(2 )*D_3 (i))/(L_3^2 ) (7)
The ‘n’ images with smallest S(i) values are shown as output. The number of images given as output can be changed. In the results and analysis part, the ‘n’ value is taken as 5.
Module 2: The second module of the proposed system is about taking the input query image and then extracting the edge and texture features using fast discrete curvelet transform (FDCT).
The steps involved in generating the FDCT coefficients are shown in the figure 20 and are explained below:
First an image is taken. That image is then subjected to a filter ( s) which filters the pixels representing the rough edges present in the image.
Then, this filtered image is divided into ‘n’ small squares in order to make the processing easy. once the image is divided into tiny squares, these squares are elongated up to a level where each and every square contains a straight line (if it contains anything at all). The elongation procedure is done in order to elongate the curved edges of the image boundaries so that they become straight lines. This ensures that the edge irregularities are captured properly and accurately without missing any significant detail. After the elongation (also called smooth partitioning), the squares are separated into individual units. Out of all the individual squares, only those squares containing the edges (i.e. the straight lines) are kept and the rest of the blank squares are discarded.
For the remaining squares (with the edges), FDCT coefficients are calculated. With these coefficients, mean and standard deviations are calculated which is later used to calculate the similarity value.Once the similarity values are calculated, the database images are sorted and indexed accordingly. Out of the sorted results, most relevant images are displayed to the user. Once the curvelet coefficients are generated and stored in each sub-band, the mean and standard deviation of the coefficients associated with each sub-band are computed. Generally, these mean and standard deviation are then used as the texture feature vector elements of the image. Thus, for each curvelet, we obtain two texture features. If n curvelets are used for the transform, 2n texture features are obtained. This results in a 2n dimensional texture feature vector which represents each image in the feature database. These feature descriptors are then used to index images in the feature database, which is also known as the image ‘indexing scheme’. An internal mapping is generated to make links between images in the database to the corresponding features in the feature database
Once the database images are indexed in the feature database, search and retrieval is performed on the basis of these features. The query image is subjected to the feature descriptor generation process to obtain its feature vector. The database images are then compared to the query image using a similarity measurement technique on the feature elements in the features vectors.