Confidential manuscript submitted to Radio Science
Restoration of Static JPEG Images and RGB Video Frames by
Means of Nonlinear Filtering in Conditions of Gaussian and
non-Gaussian Noise
R. I. Sokolov1, R. R. Abdullin1
1Institute of Radioelectronics and Information Technologies, Ural Federal University named after the first President of
Russia B.N.Yeltsin, 32 Mira Street, Yekaterinburg, Russian Federation
Key Points:
The developed image filtering algorithm has an advantage in SNR over adaptive and
median filters both for Gaussian and non-Gaussian noise.
The best results of nonlinear filter operation are observed when images are broken by
pulse noise with Johnson distribution.
RGB frame nonlinear filtering has an advantage over static JPEG filtering up to 5 dB
for WGN and more than 10 dB for non-Gaussian noise.
Corresponding author: Renat Abdullin, r.r.abdullin@urfu.ru
–1–
Confidential manuscript submitted to Radio Science
Abstract
The use of nonlinear Markov process filtering makes it possible to restore both video stream
frames and static photos at the stage of preprocessing. The present paper reflects the results
of research on comparison of these types image filtering quality by means of special algorithm
in conditions of Gaussian and non-Gaussian noise. Examples of filter operation with
different values of signal to noise ratio are presented. A comparative analysis has been performed
and the best filtered kind of noise has been defined. It has been shown the developed
algorithm allows one to filter RGB signal better than the adaptive algorithm for the same a
priori information about the signal. Also, an advantage over median filter takes a place when
both fluctuation and pulse noise filtering.
1 Introduction
Today the filtration of video stream frames and restoration of scanned images and text
are an actual task. The first step in single block image restoration is a preprocessing, which
allows enhancing the image quality by means of image processing methods such as filtering,
noise reduction, etc.
At the stage of preprocessing the image is rectified from scanning defects. It can be
achieved with using the standard methods of image processing, for example, various filters.
However, filtering quality significantly depends on kind of noise distribution, whereas the
most of existing filters are tuned only on statistical noise with Gaussian distribution.
Linear filtering is very widely used in the noise elimination of images. Particularly,
linear FIR (finite impulse response) filters are quite effective computationally and simple to
implement. However, in the annex to digital images, FIR filters have a number of disadvantages:
objects contours is blurred and small image parts can be lost.
The effect of blurring the contours can be reduced significantly by using nonlinear filters.
The simplest method of nonlinear filtering is median filtering. However, it was shown
experimentally the median filters have a low efficiency for fluctuation noise [1]. They give
much better effect when processing the distortion generated by impulse noise («scratches»,
failed lines, «touches», etc.).
Adaptive FIR filters are widely used to save objects contours and boundaries in the
image with the fluctuation noise [2].
Therefore, development of the filter allowing to remove effectively both fluctuation
and impulse noise is an actual problem. So the use of Markov algorithm for nonlinear image
filtering makes it possible to improve the filtering quality (compared with the adaptive
algorithm for the same a priori uncertainty) and work in nonstationary mode without loss of
quality. The Markov algorithm allows one to use the same stochastic differential equations to
describe the various signals and interference [3].
Thus, it is proposed to develop a quasi-optimal receiver algorithm based on Markov
theory of nonlinear filtering. The advanced algorithm will allow filtering in conditions of
white Gaussian noise (WGN) and non-Gaussian noises (one of three Johnson noise types) on
RGB signals without loss of quality.
In the experimental part of research we compare the filtering quality of video stream
RGB frame and frame converted to static JPEG digital image.
2 Synthesis of Quasi-Optimal Algorithm for Nonlinear Markov Filtering
The algorithm will perform process of signal nonlinear filtering, when it is necessary
to receive the sum of the signal and noise. The algorithm is based on the following assumptions:
–2–
Confidential manuscript submitted to Radio Science
1. The signal is a sequence of video pulses with 256 amplitude levels and unknown parameters
(useful signal ¹tº , the amplitude value, duration, time of arrival, carrier frequency)
and a priori defined as a random Markov process.
2. The noise n¹tº is a discrete random process, described by one of three Johnson distributions
(SL, SB, SU), that can be defined as nonlinear transform of the Markov Gaussian
process z¹tº: n¹tº = q ¹z¹tºº .
The noise of natural origin is a random process with Gaussian distribution [3]. Internal
noise of receiver is described by normal distribution (random variable with zero mean and
unitary standard deviation). The noise of artificial origin is a random process with one of
three Johnson distributions (SL, SB or SU) [3].
Johnson has described the system of random variables, which are obtained by transforms
of variables with normal distribution density. These transforms let approximate various
distributions including distributions lumped on semi-axes or compacts. Therefore, it is
possible to approximate the industrial noise distribution density with Johnson distributions,
which depend on three or four parameters and, consequently, describe more broad class of
densities, than two-parameters-dependable Gaussian distribution. Additionally, the random
variables with Johnson distribution can be transformed to normal random variables. It provides
the easiest way to carry out the statistic data processing. Figs 1-3 show the densities of
random variables at different parameters values.
Figure 1. Probability density of SL Johnson distribution at various values of variance 2 and mean
Figure 2. Probability density of Gaussian distribution at various values of variance 2 and mean
The probability density of Gaussian distribution is defined as follows
f ¹º =
1
2
exp
"
¹ m º2
22
#
: (1)
–3–
Confidential manuscript submitted to Radio Science
Figure 3. Probability density of SB and SU Johnson distributions at various values of variance and mean
The probability density of SL Johnson distribution is defined as follows
f ¹º =
2¹ m º
exp
"
1
22
+ ln ¹ m º
2#
: (2)
The random variables z with Gaussian distribution are expressed through the values of
SL Johnson distribution:
z =
+ ln m
: (3)
An expression for probability density of SB Johnson distribution looks as follows
f ¹º =
2
¹ m º¹ + m º
exp
"
1
22
+ ln
m
+ m
2#
: (4)
The random variable z with Gaussian distribution expressed through the values of
SB Johnson distribution:
z =
+ ln m
+ m
: (5)
The probability density of SU Johnson distribution is defined as follows
f ¹º =
q
2¹2 + ¹ m º2º
exp
2666664
1
22
266664
+ ln©
«
m
+
s
1 +
m
2ª®
¬
377775
23777775
: (6)
The random variable z with Gaussian distribution expressed through the values of
SB Johnson distribution:
z =
+ ln©
«
m
+
s
1 +
m
2ª®
¬
(7)
3. Mandatory condition is the slow change of the process, determining the signal in
time, compared with the change of the process describing the noise. (On two samples of signal
must be at least 10 samples of interference).
4. The signal and noise are nonstationary processes. So the input receiver signal is
y¹tº = ¹tº + z¹tº; (8)
–4–
Confidential manuscript submitted to Radio Science
where ¹tº and z¹tº are the signal and noise both presented as a Markov process [4]. The
signal ¹tº is given by a priori stochastic differential equation:
@¹tº
@t = k1¹tº + n¹tº; (9)
where k1 is a linear coefficient proportional to the duration of a single video pulse; n¹tº is a
forming white noise with one-sided spectral density N and an autocorrelation function
hn¹t1ºn¹t2ºi =
1
2N ¹t2 t1º: (10)
And analogically the noise z¹tº is given by a priori stochastic differential equation
@z¹tº
@t = k2z¹tº + nz ¹tº; (11)
where k2 is the linear coefficient inversely proportional to the noise sampling frequency;
nz ¹tº is a forming white noise with one-sided spectral density Nz and the autocorrelation
function
hnz ¹t1ºnz ¹t2ºi =
1
2Nz ¹t2 t1º: (12)
The video pulses repetition rate !, and phase ' are determined by their stochastic differential
equations. But since these parameters are only specifying for the signal ¹tº , for
the algorithm boundary evaluation it is sufficient to solve Stratonovitch integral-differential
equation for the two-component two-dimensional Markov process [5]:
@W¹t; º
@t =
d
d
»a¹; tºW¹t; º¼ +
1
2
d2
d2 »b¹; tºW¹t; º¼ + »F¹t; º hF¹t; ºi¼W¹t; º; (13)
where W¹t; º is a posteriori probability density of a random process ¹tº ; a¹; tº is a drift
coefficient; b¹; tº is a diffusion coefficient; F¹t; º is an observation time derivative of the
likelihood function logarithm. For an energy parameter ¹tº
F¹t; º =
1
N0
»y¹tº ¹tº¼2 ; (14)
where N0 is one-sided spectral density of noise.
The particular solution of (13) according to [5] is an equation system of the quasioptimal
nonlinear filtering for nonstationary mode:
@
@t = k1 +
2
N0
¹y q¹zºº¹D + Dzq0¹zºº; (15)
@z
@t = k2z
2
N0
¹y q¹zºº¹Dz + Dzq0¹zºº; (16)
@D
@t =
1
2N 2k1D
2
N0
»D2+ 2DDzq0¹zº + D2z
q02¹zº ¹y q¹zººq00¹zº
¼; (17)
@Dz
@t =
1
2Nz 2k2Dz
2
N0
»D2
z ¹q02¹zº ¹y q¹zººq00¹zºº + 2DzDzq0¹zº + D2z ¼; (18)
@Dz
@t =
1
2Nz Dz ¹k1 + k2º
2
N0
D¹Dz + Dzq0¹zºº + DzDz ¹q02¹zº¹y q¹zººq00¹zºº + Dzq0¹zº
: (19)
where and z are the random Markov processes: is an estimated value of the signal, z is
an estimated value of the noise; D and Dz are variances of the signal ¹tº and the noise z¹tº,
–5–
Confidential manuscript submitted to Radio Science
respectively; Dz is the covariance of the signal ¹tº and the noise z¹tº with one-sided spectral
density Nz .
The system of equations (15) – (19) allows synthesizing the filtering algorithm of any
video image under a priori uncertainty about the signal and noise. The system of five differential
equations defines estimation for values of the signal ¹tº, the noise z¹tº and determines
the variance parameters D and Dz , and its covariance Dz .
If WGN instead of Johnson noise is considered as interference, system of equations (8)
– (12) is simplified: q¹zº is replaced by z, and q0¹zº , q00¹zº are replaced by 1 and 0, respectively.
3 Filter Description
The technical result is achieved due to the construction of quasi-optimal receiver electrical
circuit based on nonlinear two-component Markov process filtering in transient conditions.
This circuit is obtained from conventional scheme by introduction of additional
message processing blocks, which are responsible for filtering and extraction of additional
signal y¹tº parameters: the useful message ¹tº; noise z¹tº; variance D¹tº of filtered useful
message ¹tº; covariance Dz ¹tº of filtered useful message ¹tº and noise z¹tº; variance
Dz ¹tº of filtered noise z¹tº. Additional blocks make it possible to extract the signal and noise
parameters and issue them to the useful message filtering block in feedback circuits.
To solve the task we have developed quasi-optimal receiver, which contains the filtering
block and useful signal quasi-coherent reception block. The last one differs from conventional
prototype [6] in that it contains seven blocks (Fig. 4):
1. Useful message ¹tº filtering block;
2. Extraction of filtered useful message variance D¹tº ;
3. Extraction of covariance Dz ¹tº of filtered useful message ¹tº and noise z¹tº;
4. Extraction of filtered noise variance Dz ¹tº ;
5. Noise z¹tº filtering block;
6. Input signal y¹tº processing block;
7. Intermediate processing block of message and noise parameters.
The channels are connected by the following links. Input mixture y¹tº of useful message
and noise goes to the first inputs of blocks 1, 5, and 6. The output of block 1 is the output
of quasi-optimal filter, which has a feedback with second inputs of blocks 1, 5, and 6.
The output of fifth block nonlinear transformer q¹zº is connected with the inputs 4, 3, and 5
of blocks 1, 6, and 5, respectively. The output of fifth block nonlinear transformer q0¹zº is
connected with the inputs 4 and 3 of blocks 6 and 7, respectively. The output of fifth block
nonlinear transformer q00¹zº is connected only with the input 5 of block 6. The output of input
signal processing block 6 is connected with the first ports of blocks 2, 3, and 4. From the
output of block 2 signal goes to the ports 3 and 2 of blocks 1, and 3, respectively. The output
of the third block is connected with the first and third ports of blocks 7 and 5, as well as the
second inputs of blocks 2 and 4. The fourth block output is linked with the third and second
inputs of blocks 3 and 7. The intermediate processing block 7 output is linked with ports 5
and 4 of blocks 1 and 3, respectively, as well as the third ports of second and fourth blocks.
The second output of intermediate processing block 7 is connected with inputs 5 and 4 of
blocks 3 and 5.
4 RGB Video Signal Filtering
Digital experiment was performed to determine the quality of nonlinear Markov filtering
(NMF) algorithm in comparison with adaptive filtering algorithm.
–6–
Confidential manuscript submitted to Radio Science
Figure 4. Generalized block diagram of quasi-optimal receiver based on nonlinear filtering
There is the digital model of the quasi-optimal filter of RGB image signal based on the
method of NMF is implemented in LabView software (Fig. 5). We used the standard block
from the LabView library as adaptive filter. This block creates an adaptive FIR filter with
the standard least mean squares algorithm. The filter length is equal 32. The step size of the
adaptive filter is 0.002.
The RGB signal was taken as a sequence of pulses with 256 amplitude levels. Two
pictures of .jpg format with dimensions of 200 160 pixels and 100 75 pixels were chosen
as image for processing. For both experiments an information samples repetition rate was
taken 10 times less than the noise sampling frequency.
The results of algorithm operation are presented in Table 1. It shows the original image,
image mixed with WGN or the noise with one of three Johnson distributions for differ-
–7–
Confidential manuscript submitted to Radio Science
Figure 5. The block diagram of RGB video image processing in LabView
ent values of signal to noise ratio (SNR). The images processed with developed NMF for two
component process are placed in the last column. It is necessary to note the following filter
work feature: for the small values of SNR the recovery of clear boundaries in the image leads
to the color distortion, particularly, considered images have been inverted to the green range.
The reason of images colors moving is the filter inertness. The bits sequence is defined
by strict sequence of colors: each pixel is described by three bits (R – red, G green, B –
blue). The improvement of filtering quality and image boundaries identification is carried
out by using the filtering coefficient k1 values in range from 0.1 to 0.5. At the same time we
observe a strong smoothing of an initial three-bit portions per pixel in the signal and noise
mixture. And since the first bit of each portion defines the red color intensity for each pixel,
then it is subjected to the greatest distortion with a shift in time. The third oscilloscope trace
in Fig. 6 shows the bit sequence of the filtered signal. The peaks of filtered signal pulses are
shifted in time by four samples in comparison with the peaks of original signal pulses in the
first waveform. Therefore, the maximum pulse amplitude does not come to the first «red» bit,
but to the second «green» bit. The square of relative error filtering Derr (an error variance)
is taken as a criterion of the filtering quality.
As a study result, we obtained the plots which demonstrate dependencies of the error
variance Derr of correct detection vs. SNR for different types of interference, and the parameters
of their distributions. These results are presented in Fig. 7.
5 Static Image Filtering
The digital model of JPEG image quasi-optimal filter based on NMF method is implemented
in LabView software (Fig. 8). The standard block from the LabView library is used
as median filter. This block applies a median filter to the input sequence X, where right rank
is -1 and left rank is 5.
The results of algorithm operation are presented in Table 2. Like Table 1 it contains
the original image, image mixed with WGN or the noise having one of three Johnson distributions
for different values of SNR, and images processed with developed NMF.
During the digital experiment we estimated the maximum value of cross-correlation
function (CCF) between noisy image signal and reference noise-free image in dependence to
SNR at the receiver input. The noisy image signal is defined by three-dimensional matrix of
–8–
Confidential manuscript submitted to Radio Science
Table 1. Filter Operation at Various Kinds of Noise when Processing RGB Video Image
Interference
type
SNR, dB Image with noise Filtered image
WGN -10
5
SL Johnson
Interference
-5
-10
SB Johnson
Interference
-15
SU Johnson
Interference
-5
–9–
Confidential manuscript submitted to Radio Science
Table 2. Filter Operation at Various Kinds of Noise when Processing Static Image
Interference
type SNR, dB Image with noise Filtered image
Original
Image
WGN 0
SL Johnson
Interference -5
SB Johnson
Interference -15
SU Johnson
Interference -5
–10–
Confidential manuscript submitted to Radio Science
Figure 6. Oscilloscope traces emitted by RGB signal, a mixture of signal and WGN, and the signal at the
output of the filter devices
intensity level values for every single point of spatial grid. As the noise we used white Gaussian
noise and Johnson noises with various parameters
and . The normalized value of
CCF between reference signal and noise has been taken as a criterion of the filtering quality.
As 100% we took the CCF between reference signal and signal mixed with noise.
As a study result, we obtained dependencies of the CCF vs. SNR for different types of
interference and the parameters of their distributions. These results are shown in Fig. 9.
6 Conclusion
As a result the signal processing algorithm based on nonlinear Markov filter was synthesized.
The algorithm works successfully in conditions of white Gaussian noise and artificial
non-Gaussian noises without loss of receive quality. On the basis of this algorithm the
digital models have been developed. Experiments with the device allow making the following
conclusions:
1. The gain in SNR of nonlinear Markov filtering algorithm compared with one of
the adaptive algorithms is from 5 dB for WGN and error variance Derr of correct detection
equal to 0.1 when RGB signal processing.
–11–
Confidential manuscript submitted to Radio Science
Figure 7. Error variance of correct detection vs. SNR
Figure 8. The block diagram of the image processing in LabView
2. The best results for developed algorithm are observed when the RGB frame is broken
by pulse noise with SL Johnson distribution. The worst results are obtained for the case
of fluctuation noise.
3. For the small values of SNR the recovery of clear boundaries in RGB frame leads to
the color distortion. Particularly, studied images were inverted to the green range. The reason
of image colors change is the filter inertness. Improvement of the filtering quality and image
boundaries identification is carried out by using the filtering coefficient k1 values in range
from 0.1 to 0.5.
4. The RGB frame inverted to the green range was obtained when the value of SNR
was equaled 0 dB for Gaussian noise and SL Johnson noise. In case of SU Johnson noise this
inversion was observed at SNR = 5 dB. For SU Johnson noise inversion does not occur up
to the minimum tested values of SNR = 15 dB.
–12–
Confidential manuscript submitted to Radio Science
Figure 9. Dependencies of NCCF vs. SNR for various types of noise
5. The gain in SNR of nonlinear Markov filtering algorithm compared with one of the
median algorithms is from 5 dB for WGN and about 1-2 dB for Johnson noise when JPEG
image processing.
6. The best results for developed algorithm are observed when the JPEG image is broken
by pulse noise with SB Johnson distribution. The worst results are obtained for the case
of fluctuation noise.
7. Quasi-optimal receiver based on video stream frame nonlinear filtering has an advantage
over static JPEG filtering up to 5 dB for WGN and more than 10 dB for non-Gaussian
noise. The gain is due to the signal streams parallel processing in video frames, whereas
static images have only one stream.
Acknowledgments
The work was supported by Act 211 Government of the Russian Federation, contract âDU
02.A03.21.0006
References
[1] G.R. Arce, «Nonlinear Signal Processing: A Statistical Approach,» New Jersey: Wiley,
2005.
[2] Methods of computer image processing [Metody komputernoy obrabotki izobrazheniy],
2nd ed. Moscow: Fizmatlit, 2003.
[3] D.V. Astretsov, R.I. Sokolov, «Binary signal optimal receive of joint nonlinearing filtering
method of continuous and discrete Markov process,» Science and Technology Volga
Gazette, vol. 4, pp. 41-44, April 2014.
[4] S.V. Pervachev. «Radioautomatics: Textbook for Universities». Moscow: Radio and
Communication, 1982.
[5] V.I. Tikhonov, N.K. Kulman. «Nonlinear filtering and quasi-coherent receive [Nelineynaya
filtratsiya i kvazikogerentniy priyem],» Moscow: Sov. radio, 1975.
[6] D.V. Astretsov, R.I. Sokolov, Quasi-Optimal receiver. Patent no.
2016152667/08(084364) from 29.12.2016.
–13–