CHAPTER 1
INTRODUCTION
1.1 Digital Image Processing
Digital image processing involve different techniques of image resizing. This is done by constructing mathematical metrics that measures the integrity of the input data in the image after the resampling.
Consider the most popular imaging system consisting of an optical system and digital image registering device. The optical system (lens) focuses the scene on the imaging plane. Its performance depends, among other factors like distance and focal length, on the finite aperture size. It determines the optical resolution limits. The optical transfer function (OTF), being the autocorrelation function of the aperture size, is also finite and hence cuts all spatial frequencies outside the region of its support. The next is the sampling device, transforming the captured image into pixels. In most modern digital imaging systems it is a kind of imaging sensor based on the principle of charge-coupled devices (CCD). Then, the spatial resolution is limited by the spatial sampling rate, i.e. the number of photo-detectors per unit length along a particular direction.
A digital image is a representation of a two dimensional image as a finite set of digital values, called picture elements or pixels. Pixel values specifically represent binary, gray scale and colour, etc. Digitization is only an approximation of a real scene. Digital image processing is a technology of using a number of algorithms to process digital images. It is concerned with two major tasks (a) Improvement of graphic representation for interpretation (b) Processing of image data for storage, transmission.
1.1. Sampling
The process of capturing real scenes and converting them into an image is to be a mathematical method by which image acquired and sampled. The optical system can be modeled as a spatial linear low-pass filter which can act on the continuous two-dimensional. This band-limited signal is then sampled at the sampling rate determined by the spatial distribution of the photo-detectors. While the optical system somehow blurs the signal, an insufficient sampling rate, caused by detectors spaced few and far between, can introduce aliasing. These two effects are inevitable.
1.2. Re-Sampling
Image resampling is a process by which new pixel values are interpolated from existing pixel values whenever the raster’s structure (number of rows and columns) is modified such as during projection, datum transformation, or cell resizing operations. Various resampling methods can be employed to resize an image and when an image is enlarged or reduced, changes are necessarily made to the value assigned to each pixel. To reduce an image, entire rows and columns are removed, while the enlargement of an image requires the opposite change by adding rows and columns of pixels. In both cases, the spatial extent (minimum and maximum X and Y coordinates) of the imagery is unchanged and only the raster’s structure is modified. The effect of image resampling is a concern for image quality in general, and when dealing with remotely sensed data for scientific interpretation, data integrity (i.e., how closely the interpolated value matches the original value of each pixel) becomes a concern as well. This is because raster images store data within the feature (pixel) itself as for example; each pixel from a satellite image represents a measured surface reflectance value derived from a satellite or airborne sensor.
The enlargement of satellite imagery (i.e., increasing the number of rows and columns) is typically not of concern relative to data integrity issues as the rows and columns of pixels added are simply duplicates of existing pixels. This is particularly true when the enlargement factor is a whole number, but not necessarily true when imagery is enlarged fractionally (e.g., spatial resolution of an image is changed from 15 meters per pixel (mpp) to 10 mpp) as this specific procedure does require interpolation. In contrast, the reduction of an image means fewer rows and columns (and hence fewer pixels) will be used to represent the same geographic features across the same spatial extent. A fairly common resampling task involves the conversion of satellite imagery at a relatively fine spatial resolution (e.g., 10 mpp) to a more coarse resolution (e.g., 30 mpp) to readily accommodate comparison with imagery from another satellite sensor. In this scenario, blocks of pixels (kernels) are involved in an iterative resampling process. The value of each pixel within each kernel is evaluated and a new value calculated for the output pixel in the new ‘re-sampled’ image layer. To effect this change various forms of interpolation have been developed to minimize data integrity losses as a result of resizing.
1.3. Interpolation
Image interpolation is a process that estimates a set of un-known pixels from a set of known pixels in an image. Image interpolation occurs in all digital photos at some stage- whether this to be in capturing image or in photo enlargement. It happens when we resize the image from one pixel grid to another. Image resizing is necessary to increase or decrease the total number of pixels. Remapping can occur for correcting for lens distortion, changing perspective, and rotating an image.
Roughly speaking, the role of the interpolation is to restore some missing intermediate points between the given discrete pixels. First, based on the existing discrete data, a continuous signal is generated with the aid of a proper interpolation technique. The desired interpolated samples are then obtained by resampling this continuous signal at the desired coordinates. In this general setting, the interpolation factor is arbitrary, not necessarily an integer, not even a rational number. In most cases, the continuous (analog) model fitting is performed by convolving the samples with some appropriate continuous interpolating kernel.
The interpolation does not recover the original scene continuous signal itself. In most cases we do not know the characteristics of the optical system nor the sampling rate and the effects introduced by the sampling device. What we can do is to try matching a continuous model that is consistent with the discrete data in order to be able to perform some continuous processing like differentiation or to resample into a finer grid. As such, interpolation can be regarded as ‘model-based recovery of continuous data from discrete data within a known range of abscissa’. It is not able to increase the resolution, since it does not deal with inverse problems such as de-convolution or super-resolution, although some knowledge about the acquisition and sampling device could help in the choice of the interpolating function. Its purpose is to make the discrete data adequately tractable in the particular continuous domain it originates from.
Image interpolation works in two directions, and tries to achieve a best approximation of a pixel’s color and intensity based on the values at surrounding pixels. The following example illustrates how resizing enlargement works: as shown in Fig. 1.1
Fig. 1.1: Illustration of Image Interpolation
1.3.1. Distortion by Interpolation
The distortions caused by non-ideal interpolators are as follows:
‘ Ringing
Ringing is a result of the oscillatory type of interpolators combined with the Gibbs effects due to the finite terms approximation of the continuous Fourier domain. Ringing effect occur even for the ideal sinc interpolators realized in Fourier domain. Actually, those are not true artifacts since they can arise together with the perfect recovering of the initial samples.
‘ Blurring
Blurring is a result of the non-ideality of the reconstruction function in the pass-band. Instead of preserving all frequencies in this region, non-ideal interpolators suppress some of them, especially in the high-frequency area (close to ??). As a result, the interpolated images appeared with no sharp details, i.e. blurred.
‘ Aliasing
Aliasing is more likely an effect due to improper sampling. This is the effect of the appearing of unwanted frequencies (hence the term aliasing) as a result of the repetition of the original spectrum around multiples of 2??. The aliasing artifacts resulting from insufficient sampling may appear as Moir?? patterns. For small images, they may be hardly noticeable, but after interpolation on a finer grid they can become visible.
‘ Imaging
This is the counterpart of aliasing, though the term ‘imaging’ is somehow confusing. Consider the simple case of sampling rate expansion by an integer factor of L. It can be accomplished by first inserting (L-1) zeros between the given samples (upsampling) and then smoothing the new sequence with a digital filter. The up-sampling causes ‘stretching’ of the frequency axis. As a result, in the passband the original spectrum appears together with its L ‘ 1 ‘images’. The role of the smoothing filter is to remove these unwanted frequencies. Following the model, we first apply the reconstruction operation and subsequently the resampling.
Assuming this model, unwanted frequencies can interfere into the passband during the process of resampling as a result of non-sufficient suppression of the frequency replicas during the previous step of continuous reconstruction. Hence, this effect can be again characterized as aliasing, and we will use this term in order not to confuse ‘imaging’ with digital images. The effects of possible sampling-caused and reconstruction-caused aliasing are almost undistinguishable since they appear simultaneously (and with blurring) in the re-sampled image. The reconstruction-caused aliasing effect is emphasized for short length kernels (i.e. nearest neighbor or linear) and appears in the form of blocking (pixelation) in the magnified image.
Interpolation is way through which images are enlarged. There are many different types of interpolation methods, each resulting in a different. Thus, it is best if the quality, or visible distinction for each pixel, is retained throughout the enlargement process.
Older methods of linear interpolation somewhat addressed this problem. By finding a mean pixel value between neighboring pixels, one was able to produce an effect of blurred edges and smoothed details. Bilinear re-sampling uses the values from the four surrounding pixels and new pixel values are calculated by weighting the averages of the four closest. The new pixel value is determined by calculating a weighted average of the four closest pixels ever, bilinear interpolation works better for image reduction as compare to image enlargement.
Linear interpolation methods limitations. Some non-linear interpolation methods include Bi-Cubic, Soft Directional, and non-linear interpolation through extended permutation filters. Bi-cubic interpolation uses the nearest sixteen pixels (4×4 arrays) based on distance, which produces a much better effect than linear interpolation.
Edge Enhancement
Conventional linear interpolation schemes (e.g., bilinear and bicubic) based on space-invariant models fail to capture the fast evolving statistics around edges and consequently produce interpolated images with blurred edges and annoying artifacts. Linear interpolation is generally preferred not for the performance but for computational simplicity. After linear interpolations, edges are blurred. To remedy this, spline interpolation is used.
Many algorithms [1], [2] have been proposed to improve the subjective quality of the interpolated images by imposing more accurate models. Adaptive interpolation techniques [3], [4] spatially adapt the interpolation coefficients to better match the local structures around the edges. Edge-directed interpolation (EDI) techniques [5], [6] employ a source model that emphasizes the visual integrity of the detected edges and modify the interpolation to fit the source model.
After interpolation, the edges are more visible so edge enhancement is much more successful and visible. Edge detection works by taking a weighted sum of pixels around a single pixel to determine its new value.
Super Resolution
Super-resolution (SR) methods are typically concerned with overcoming the resolution limitation resulting in aliasing. In this context, ‘resolution’ refers to the sampling interval, or pixel size. Coarse sampling results in ‘low resolution’ images, while ‘high resolution’ images correspond to fine sampling[7].Super-resolution (SR) assumes that there are some (small) differences between the input images. Most often, these differences are caused by small camera movements. Super-resolution reconstruction started as a frequency-domain technique. The original idea of dealiasing in the frequency domain dates to [8] and was improved by others. These methods are theoretically simple and computationally efficient. However, their use is restricted to the case of pure translational motion and more importantly they are sensitive to errors.
A more robust approach is solving the problem in the spatial domain. In fact, all modern techniques adopt the spatial (pixel) domain approach where the solution of a very large scale, ill-posed system of linear equations is sought. Different spatial domain methods use different assumptions and different approaches to the solution of the same matrix formulation and they are, in general, computationally expensive. This is especially true for projection type methods.
1.4. Motivation
In many applications, digital images are required to be interpolate image details. This is done by making multiple copies of the pixels in a region of interest (ROI) inside the image. Several algorithms are used in this process. The simplest and fastest method, known as a replicating zoom, displays multiple copies of each pixel. Interpolation can produce zoomed images at integral zoom factors of 2X, 3X, 4X, and higher. Depending on the spatial resolution of the image, individual pixels can become apparent at 4X or higher zooming factors.
Among biomedical applications where interpolation is quite relevant, the most obvious are those where the goal is to modify the sampling rate of pixels (picture elements) or voxels (volume elements). This operation, named rescaling, is desirable when an acquisition device’say, a scanner’has a non-homogeneous resolution, typically a fine within-slice resolution and a coarse across-slice resolution. In this case, the purpose is to change the aspect ratio of pixels in such a way that they correspond to geometric cubes in the physical space. Often, the across-slice resolution is modified to match the within-slice resolution, which is left unchanged. This results in a volumetric representation that is easy to handle because it enjoys homogenous resolution.
1.5. Applications of Image Zooming
Medical Image Processing
This technology uses Ultrasonic, X-ray, Computerized Tomography (CT) and Magnetic Resonance Imaging (MRI) to draw images of human body. For the diagnosis of different diseases. Which has extended the spun of human life.
Medical imaging helps to draw the structure of internal body parts, tissues etc. by using ultrasonic rays a pressure has been affected part which create the structure of the same. By comparison in the normal ones the disease diasgnosed. Which many a times looks impossible from outside. X-ray technology used to diagnose the defects in the solid parts of the body like bones etc. MRI and CAT scan is widely uses computer assisted imaging for exact diagnosis of the complicate diseases in brain etc. X-ray Image of palm and brain shown in Fig. 1.2.
Fig 1.2 use of interpolation medical imaging.
Surveillance
For surveillance in border of forest, state, country, airport they fit some video camera and takes image (or videos) for each person who goes through that place. For every person if doubt apply zoom the image with all angels and full features to check what the person had and what is his/her identity. Aerial surveillance methods are used to continuously keep an eye on the land and oceans. Fig. 1.3 Shown Objects location finding.
Fig. 1.3 Object location finder by radio frequency.
In synthetic aperture radar imaging, amplitude and phase of radio waves reflected by the objectis recorded in course of plain flight around the object. These flight data are then used for reconstruction of wave reflectivity distribution over the object surface. The reconstruction is carried out either optically, or, presently, in digital computers.
Face detection
The camera quality or video are very low due to data storage problem. For reliable face recognition from camera there is a need to be interpolation. For security purpose every place like ATM, Confidential dept. of Govt. face detection application applied for authentic person may go forward from this place.
Astronomy
For the research purpose scientist takes images (videos) of planets from satellite camera. The size is very big, blur, low contrast, high contrast etc so mostly image interpolation required. Interpolation corrects the image and makes them as human vision requirement.
Entertainment
In television broadcasting there is big problem of data transfer. Today every TV or news channel wants high quality videos. But if they send high quality videos then delay occurs in system. Video show with some delay so they broadcast (transfer) low size images and inbuilt feature fitted in TVs resize into screen size. And no time delay in transfer any user got High resolution images.
Industrial inspection
Today all the industries goes semi or fully automatic. In all the industries weather it a food processing, manufacturing or other type they required image processing application for bottling, refilling, packing, selecting the product and transfer to next machine.
Remote sensing
Remote sensing technology uses instruments like cameras, radiometers, lasers, radar systems
sonar, seismographs, thermal meters, etc. to gather information’s about different geological phenomena like acoustic energy, electromagnetic radiation, force fields by using electromagnetic rays. To use this techniques interpreted the objects and regions and are used to check flood, city planning, resource mobilization, monitoring, etc.
Forensics
This is a technique to diagnose suspicious objects finger print DNA or other things that are left behind by the criminal on the spot. Different techniques together are used for the investigation of the crime and criminals. Which minor details to catch the criminals and further legal procedure which supporting evidences for punishment to check further crimes.
Transportation
This is a new area that has just been developed in recent years. One of the key technological progresses is the design of automatically driven vehicles, where imaging systems play a vital role in path planning, obstacle avoidance and servo control. Digital image processing has also found its applications in traffic control and transportation planning, etc.
Military
The armies of different countries are using advanced instruments based on this technology. They are using it for three dimension Imaging and projection, for tracking of solid stable moving object like plane space ship etc. As well as tracing of energy sources by use of advance sensor technology by which one can search missing peoples and animals in remote areas like deserts, forest etc.
1.6. Thesis Organization
This thesis contains five chapters, below is the brief detail of them. Present chapter discusses the basics of Image sampling, resampling and interpolation. Further the types of distortions in non-ideal interpolation are discussed.
Chapter 2 deals with the findings of related work till date. It has also attempted to bring out some of the limitations of existing work.
Chapter 3describes the problem identified during the literature reviews.
Chapter 4 deeply explains the methodology which has been proposed in this thesis.
Chapter 5 shows the outcome of the various interpolation techniques.
Chapter 6 deals with the conclusion of the implemented work with relevant application and the possible future work.