Removing Photography Artifacts Using Flash & No-flash Image Pairs
Dr. Prashant R. Deshmukh1, Ankita R. Borkar2,
Student, Computer Science and Engineering, Sipna C.O.E.T., Amravati, India2
Abstract - In this paper, we are going to take a novel view based on a photography approach in which we are concerning the study of removing Photography Artifacts like removal of glare or the flash hot spot and remove reflection ,Highlights caused by glossy surface . so here we are using both Ambient and Flash images to. In this proposed scheme system we are going to focus on different approaches for remove the Photography artifact. We are presenting a novel Weighted Mapping scheme where we compute weighted averaging to remove glare and also introduce techniques like Diffusion tensor , cross diffusion tensors using gradient fields of both ambient and flash image to remove the reflection. We used these tensors to remove reflections in an image We analyze some experimental results that demonstrate the proficiency of algorithms to produce improved quality of flash images having glare and also reflection.
Keywords - Glare, flash, Gradient field , reflection removal, cross Diffusion tensor, high dynamic range.
I . Introduction
This over the years, a number of schemes have been introduced for removing glare and reflection which defined as a uncomfortably of bright light. In photography artifact the glare is a sensation in rcaused by bright light that can take a different numbers of forms and Flash highlight can also cause an unwanted discomfort, exasperation or even pain, and this sensation is another form of glare. In HDR image, glare is a sensation caused by bright or flash of light. In addition flashes caused disagreeable reflections. Often one sees the reflection of an object that mendacity exterior the field of view of the camera but is robustly lit by the flash[1-4]. One also sees strong highlights due to straight reflection of the flash itself by glossy objects in the scene. Former work has shown that a flash and ambient image pair can be used to de-noise and enhance the ambient image and to accomplish enhanced image fusion. Our objective is to use a flash and ambient image pair to produce a high quality flash image by reducing glare region and reflection photography artifacts. Our purpose is the design of the glare free and reflection free image by using some different techniques like Weight mapping approach where we are going to used two image one is flash one is without flash so called Ambient image and apply the weighted average method to get glare free image and similarly we need ambient and flash image to remove reflection where we are using techniques like gradient field ,diffusion tensor and cross diffusion tensor using the single threshold for the across all pixels of image cannot account for variations in high lighting and reflection through the image[5,6,7,8].
II. Literature Survey
Until the year 1997, P.E Debevec & Malik introduced a scheme that present a method which has been convalescing HDR radiance maps on various photographs of the scene. The algorithm implies that different exposed photographs has been recover by function of the imaging process. . In year 1997, Nayar, Fang and Boult represent the techniques for extrication the reflection components using color an also polarization in this paper reflection produced strong highlights in the brightness image which contain minute quantity of glare reflection of light. In year 2002, R. Fattal et. al. introduced a new techniques on conventional display for interpretation HDR images. This approach is simply easy and robust ,Where one can find the gradient field by diminution the magnitudes of large gradients of the luminance high dynamic range image by depletion . E. Eisemann & F. Durand has planned in year 2004 to enhanced qualityof Flash photography by the via intrinsic relighting in which they take photographs shot in dark region area by merging that picture taken with the available light and also with the flash. They also preserve original lighting. By using bilateral filter the images is break up into detail and large scale .
R. Raskar, K.Tan, R. Feris, J.Yu , and M.Turk, also present work on numerous flash imaging in year 2004. Where they take non-photorealistic interpretation to capture and create shape features of real world scenes using a camera with multiple depth discontinuities in the scene. The projective geometric relationship of the camera-_ash setup exploited uncovering of depth discontinuities and. The resulting images get is more better and clear to the 3D structure of the imaged scenes .
G. Petschnigg et al. Anticipated method for both flash and no flash images in year 2004 in which they represent the Digital photography which has been made it possible to quickly capture a pair of images of low-light environments with flash and without flash to take ambient illumination. Amit Agrawal et al. propose to use a flash and ambient image to produce better flash images also present a novel gradient projection scheme based on a gradient consistency model take contained Gx along x, Gy along y axis of variable gradient field for each pixel and flash intensity exposure space shows the method for adaptively sampling to minimize the quantity images for any given scene. .
In the year 2007, E. Talvala, A. Adams, M. Horowitz, & M. Levoy anticipated record of a high dynamic range image, by taking picture at one snapshot or a sequence snapshots, is limited by the presence of veiling glare the contrast has been reduced everywhere within the some field of the view and such glare arises from multiple scattering of light inside the cameraâ€™s body . In this paper, two methods used for removing glare are deco volution by a measured glare spread function and selectively block the light that contributes to veiling glare with deco volution. .
In year 2008 , R. Raskar et al. introduced the Glare aware photography in camera lenses of 4D ray sampling to minimized glare effects on image Phenomenon. This scheme explore masks for uniform and non-uniform ray sampling technique to show how a practical solution is used to analyse the 4D statistics without significantly compromising image resolution . P. Didyk, R. Mantiuk, M. Hein, and H. Seidel utilize the full potential of new high dynamic range displays, the video sequences in a system of bright luminous objects improved the quality is proposed in year 2008 .Which may classifies saturated regions as lights, reflections and diffuse surfaces by using a semi-automatic classifier .
In year 2008 , Rolf Adelsberger et.al. introduced system that employs an infrared time-of-flight rangefinder to find the distance to each object and a enhanced video projector to modulate the illumination. While this approach is limited by the power of our light source that produced illumination physically plausible.  O. Gallo et. al. projected an scheme which is capable of dealing with a large amount of movement in the scene with a reference image previously selected from the stack of images in year 2009 . It create the HDR image by average radiance estimates of all regions and also compensate for camera calibration error .
In year 2011 , C. Lau, M. Rouf, M. Trentacoste, R. Mantiuk and W. Heidrich. Proposed a novel view approach of High Dynamic Range capture, which is basically based on photography approach. Where firslyt optically encoding done with both the high light information into LDR image that can be captured with a conventional image sensor and low dynamic range part of the scene and high light information into LDR image that can be captured with a conventional image sensor. This process overcome by using a cross-screen or star filter then decode, in software .
Kanchit Pamanee, Angela Duangchit, Roungsan Chaisricharoen has anticipated a new method recovering information from the surrounding area of glare centroid in 2013 and 3DRS is assigned to extract luminance information and then separate them from intensity of edges form by material to find out edge features in animated images from Macroblock of the previous non-glare affected frames. For enhancing the distorted MB tone mapping method is applied.
Neha Hial and Somesh Dewangan introduced an approach for edge-suppressing operations on an image,which depends on affine transformation of gradient fields using cross projection tensor method which derived from another image.
Our proposed work is basically a framework design with two folded method using Ambient and Flash image which as shown below :-
1. Using weighted Mapping method for Removal of Glare
2. Using several method like diffusion tensor ,gradient and cross diffusion for Removal of Reflection
Removal Glare or flash hotspot
In this method we are going to apply Weighted average method to both Ambient and flash image and select the minimum among the three planes would give flashy region or whiteness present in image and then we apply the weighted Average method are shown as follows:-
I = w Ã— A + (1-w) Ã— F,
Let, W = min (I, , 3)
I = A + F
Where, w is the weight , A is the Ambient and F is Flash image .
Algorithm:- Glare Removal Algorithm
Input:- Ambient and Flash image of same object
Output:- Glare free image.
Step 1:- An input carrier Ambient and Flash image will be selected.
Step 2:- Determine the flash hot spot or glare region in Flash image using weighted average method
Step 3:- Find weight which indicates the undesired portion or region with ambient image and negative weight indicates the desired region with flash image .
Step 4:- Add both Ambient and Flash image using weight mapping techniques.
Step 5:- After adding both image we get Glare free image or artefact free image.
Step 6:- End.
In this section we are applying diffusion tensor and cross diffusion tensor by computing gradient field of both ambient and flash image.We will initiate process by showing how to compute the gradient at an image . Then, we compute actually does encode the gradient direction and magnitude.We form the gradient vector by combining the partial derivative of the image in the x direction and the y direction.
Î”I= (âˆ‚I/âˆ‚x, âˆ‚I/âˆ‚y)
We are also using the Diffusion Tensor , where Df is a diffusion tensor field contain at each pixel, Df (x, y) is a 2 Ã— 2 symmetric matrix, positive definite. Df is then obtained as
Df = [â– (Df11&[email protected]&Df22)]
D(y, x) = [â– (d11(y,x) &d12(y,x)@d21(y,x) &d22(y,x))]
is a 2Ã—2 symmetric, positive-definite matrix at each pixel or a field of diffusion tensors using it we also compute eigen values and eigen vector of diffusion tensor matrix. Due to reflection artifact ,the estimation of the gradient using finite differences, along with small non-zero value is used as a threshold to decide for homogeneity which vary across the image, if the image has spatially varying illumination and then used Cross Diffusion Tensor method as follows:-
D11 = L1 Ã— x1 2 + L2Ã— y12
D12 = L1 Ã— x 1 x2 + L2 Ã— y1 y2
D22 = L1 Ã— x22 + L2 Ã— y22
Algorithm: Reflection Removal Algorithm
Input: Ambient and Flash image of same object
Output: Reflection free image.
Step 1: An input carrier Ambient and Flash image will be selected.
Step 2: Convert RGB to YUV of ambient and flash image
Step 3: Apply diffusion tensor both images
Step 4: Calculate eigen values and eigen vector of diffusion tensor matrix.
Step 5: Apply thresholding term
Step 6: Determine Cross Diffusion tensors.
Step 7: Calculate gradient of Ambient image and Subtract that gradient with Cross diffusion tensor gradients.
Step 8: Convert YUV to RGB
Step 9: Finally we get Reflection free image.
In the experimental results ,to find out noise or artifact like glare and reflection we were simply used the term PSNR which stands for peak signal to noise ratio . It used to measure the quality of images. Higher the PSNR value, better the quality of image. It is estimated in decibels (db). So we are taking PSNR of artefact or noisy image like glare or reflection with Resultant recovered image.
PSNR = 10.log10 ( MAX 2I /MSE )
Where ,MSE stands for the mean square error. Lower the value of MSE, better the quality of image. Now see the following input image of computer taken with flash or no flash:-
Fig. 1 Ambient image (left) and Flash image (right)
Fig. 2 Grayscale image of Flash image with (w) and (1-w)
Fig. 3 3D visualisation of (w)weight
Fig. 4 3D visualisation of (1-w)weight
PSNR for different Image types
Ambient Image Type Size of Ambient Image Flash Image Type Size of Flash Image Processing Time for Glare free image PSNR
.bmp 2560*1920 .bmp 2560*1920 4.0021 19.9685
.png 2560*1920 .png 2560*1920 11.3193 19.9685
.jpg 2560*1920 .bmp 2560*1920 4.976 19.9693
Fig. 5 Comparison of experimental result
From above table. 1 we can see that when we take input .jpg as ambient image and .bmp as flash image we
get maximum PSNR as 19.9693 as compared to others. Finally we get glare free image as show below :-
Fig.6 Resultant image (Glarefree image)
Now similarly we compute PSNR for reflection where we experiment on different parameters like sigma ,threshold by changing its value and also we take different image types like .jpg , .png , .bmp etc. Which shows in above table 1 and fig.5.
Fig. 7 Ambient and flash image
Fig.8 Reflection free image
Image type Flash Image Type Size of both Image Iteration Processing Time PSNR
.png .png 401*301 5 5 0.4 23.72845 23.9066
.jpg .bmp 401*301 5 5 0.8 21.0125 24.6817
Fig.9 Recovered reflection
In fig.8 and fig.9, we get Reflection free and Recovered reflection image by taking diffusion tensor with its eigen values and eigen vector so here we changes the values of threshold as 5 ,15 ,25 which shown below in table 2 and table 3 means we are increasing threshold but we are getting better result when threshold value is minimum and we also experiment by changing the value of sigma as 0.4 , 0.6, 0.8 . After changing value of sigma and threshold we estimate PSNR of Flash image and the resultant image.
PSNR after changing Threshold value
T = 5
T = 15
T = 25
.png 24.5978 22.6706 22.3934
.jpg 24.5554 22.7620 22.4607
.jpg & .bmp 24.6817 22.5487 22.3450
Comparison result with Threshold 5,15, 25
Fig. 10 Comparison result with Threshold 5,15, 25
Comparison result with Threshold & Sigma
Fig. 11 Comparison of experimental result
We have noticed that in above table 4 and fig. 10 when sigma is 0.4 and threshold is 5 we get 23.6817 PSNR value which is less than the value 24.6817 when sigma set as 0.8 with threshold as 5 .
The main objective of proposed scheme is to remove Flash photography artifact like glare and reflection from so here we are taken flash photography and have addressed three well-known problems: over-illumination or under-illumination at a given flash intensity, reflections or highlights also refered as glare . We reduced these glare by exploiting information in the ambient image. We used weighted average method to remove glare for different image types and used cross diffusion tensor method to remove reflection by changing threshold and sigma value for different images type like .bmp,.png ,.jpg etc. We noticed that we get good quality of image when threshold value is minimum. Our approach is conceptually simple and can easily handle any type of color images . we hope in future our method can remove all problem of photography artifacts easily.
 P. E. Debevec and J. Malik â€œRecovering high dynamic range radiance maps from photographsâ€. In Proc. Of the 24th annual conference on Computer graphics and interactive techniques, 369â€“378 , 1997 .
 S.K. Nayar,,X.S. Fang, and T. Boult, â€œSeparation of reflection components using color and polarization.â€ Intâ€™l. Journal of Computer Vision 21, 3 (Feb.),163â€“186,1997 .
 R. Fattal, D. Lischinski, and M. Werman, â€œGradient domain high dynamic range compression.â€ ACM Transactions on Graphics 21, 3, 249â€“ 256, 2002 .
 E. Eisemann and F. Durand â€œFlash photography enhancement via intrinsic relightingâ€. ACM Transactions on Graphics 23, 3 (Aug.), 673â€“ 678 , 2004.
 R. Raskar, K.Tan, R. Feris, M.Turk and J.Yu,. â€œNon-photorealistic camera : depth edge detection and stylized rendering using multi-flash imaging.â€ ACM Transactions on Graphics 23, 3, 679â€“688 , 2004.
 G. Petschnigg , M. Agrawala, H.Hoppe, R.Szeliski, M.Cohen & K .Toyamaâ€ Digital photography with flash and no-flash image.pairs.â€ ACM Transactions on Graphics 23, 3 (Aug.) , 2004, 664â€“672.
 Amit Agrawal, Ramesh Raskar, Shree K. Nayar , & Yuanzhen Li .â€ Removing Photography Artifacts using Gradient Projection & Flash Exposure Samplingâ€, 2006.
 E. Talvala, A. Adams, M. Horowitz, and M. Levoy. â€œVeiling glare in high dynamic range imaging.â€ ACM Trans. Graph. ,, 26(3):37, 2007.
 A. Agrawal, C. A. Wilson, Ramesh Raskar & A. Veeraraghavan. â€œGlare aware photography : 4D ray sampling for reducing glare effects of camera lenses.â€ ACM Trans. Graph., 27(3 ) . 2008 .
 M. Hein, and H. Seidel. R. Mantiuk, & P. Didyk, â€œEnhancement of bright video features for HDR displays.â€ Computer Graphics Forum 27(4):1265â€“1274,2008.
 Remo Ziegler, Rolf Adelsberger, Marc Levoy, & Markus Gross,â€ Spatially Adaptive Photographic Flashâ€ ETH Zurich, Department of Computer Science, Technical Report No., 612, 2008.
 O. Gallo, N. Gelfand , W. Chen, M. Tico, & K. Pulli. â€œArtifactfree high dynamic range imaging.â€ In Proc. ICCP, 2009.
 C. Lau, M. Rouf, M. Trentacoste, R. Mantiuk & W. Heidrich. â€œGlare Encoding of High Dynamic Range Images,â€ in Computer Vision and Pattern Recognition (CVPR), pp. 289-296, 2011.
 Chaisricharoen ,Roungsan & Angela Duangchit ,â€œVision Restoration of Dangerous Head Light Glare Affected Scenario,â€ in 13th International Symposium on Communications and Information Technologies (ISCIT), , pp. 733-736 , 2013.
 Somesh Dewangan & Neha Hial ,â€ Detection of Shadow and Its Removal along with the Edge Suppressionâ€ International Journal of Science and Research (IJSR) , ISSN:2319-7064,2013.
...(download the rest of the essay above)