Wavelets and Applications
Introduction
Wavelets are powerful tools that can be used in signal processing and data compression. Wavelet transforms are an excellent alternative to Fourier transforms in many situations. In Fourier analysis, a signal is decomposed into periodic components; in wavelet analysis, a signal is decomposed into components localized in both time and frequency domains. Thus, wavelet transforms are ideal when signals are not periodic.
The Theory
In Fourier analysis, sinn⁢π⁢x,cosn⁢π⁢x is used as an orthonormal basis of L2⁢0,1. In wavelet analysis, a father wavelet φ and a mother wavelet ψ are chosen such that:
φx,ψ2 x, ψ2 x−1,ψ4 x,...,ψ4 x−3,ψ8 x,...
form an orthonormal basis of L20,1. In theory, φ is chosen to satisfy the conditions of a multiresolutional analysis (MRA), and then ψ is determined from φ and the MRA. In practice, φ and ψ are assumed to satisfy the following functional equations, and the coefficients are computed according to the desired properties of the MRA.
φ⁡x=∑n=−∞∞hn⁢φ⁡2⁢x−n (1)
ψ⁡x=∑n=−∞∞gn⁢φ⁡2⁢x−n (2)
In fact, often ψ and φ cannot be determined symbolically, and are defined solely in terms of these coefficients. In such cases, the Cascades Algorithm can be used to obtain numerical approximations.
The fact that ψ and φ must be orthogonal reduces to the following numerical conditions on the hn and gn, when φ has norm 1.
∑n=−∞∞hn⁢hn+2⁢h=δ0,k (3)
∑n=−∞∞gn⁢gn+2⁢h=δ0,k (4)
∑n=−∞∞hn⁢gn+2⁢h=0 (5)
Usually, the hn are computed, then the gn are determined by reversing the hn and negating every term. The hn and gn are also known as the scaling and wavelet coefficients, or the low pass and high pass filters, respectively.
Z Transforms
When the Fourier transform of equation (1) is computed,
Φ⁡w=∑n=−∞∞hn⁢e−i⁢n⁢w2⁢Φ⁡w2
is obtained. This is the motivation for defining:
m0⁡x=∑n=−∞∞hn⁢xn2
This is known as the Z transform associated with φ. In this context, the orthogonality conditions reduce to:
m0⁡x⁢m0⁡−x+m0⁡x+π⁢m0⁡−x−π=2
The generation of wavelets is often phrased in this language.
Orthogonal and Biorthogonal Wavelets
Technically, the above discussion applies only to orthogonal wavelets. In a variant of this theory, different bases of L20,1 are used for the analysis and synthesis of signals. Such pairs of bases are generated by biorthogonal bases. Note that orthogonal wavelets can be viewed as biorthogonal wavelets for which the analysis and synthesis processes coincide. However, biorthogonal wavelets are not orthogonal; their main advantage is that the orthogonality conditions are relaxed, allowing more smoothness conditions to be imposed. Some authors refer to wavelets that are neither orthogonal or biorthogonal, but such wavelets are not discussed here.
Wavelet Generation
This section provides examples showing how some wavelets are generated by Maple. This section is not intended as a complete guide to the generation of these wavelets, and it is not intended as a thorough discussion of their theory. It is a simplified outline of how Maple generates these wavelets.
Symlets
Symlets are a variant of the Daubechies wavelet. In fact, they are also called Daubechies least asymmetric wavelets. They have the same vanishing moments as the Daubechies wavelets, and the same size, but they have minimal phase. A complex valued function f is said to have linear phase if there are real numbers a and b such that f⁡x=ea⁢x+b⁢I⁢f⁡x. It is known that there are no compactly supported (that is, finite length) orthogonal wavelets with linear phase. The Symlets were designed by Ingrid Daubechies to have phase as close as possible to linear.
The generation of Symlets is very similar to the generation of the Daubechies wavelets. To generate the Symlet of size 2⁢A, start by finding the roots of the polynomial P that is used in the generation of the Daubechies wavelet. Then transform these roots to get roots of the Laurent polynomial P⁡Z⁡X, where Z⁡X=2−X−X−12. The plot demonstrates that these roots come in groups of conjugates and reciprocals, so you are able to restrict your attention to those with norm less than or equal to 1 and nonzero imaginary part.
A≔7:P≔addbinomial⁡A+k−1,A−1⁢2−k⁢Xk,k=0..A−1;sols≔RootFinding:-Analytic⁡P,X,re=−100..100,im=−100..100:T:=map⁡t→op⁡1−t+−2⁢t+t212,1−t−−2⁢t+t212,sols:Tplot≔seq⁡ℜ⁡Ti,ℑ⁡Ti,i=1..nops⁡T:plotslistplotTplot,style=point;S≔selectz→z<1and0≤ℑ⁡z,T:
P≔1+72⁢X+7⁢X2+212⁢X3+1058⁢X4+23116⁢X5+23116⁢X6
Now perform spectral factorization on PZX. This means that you want to find a p⁡X such that P⁡Z⁡X=p⁡X⁢p⁡−X. This is done by using the Feyer Reiz algorithm. For each root λ of P⁡X, you simply have to assign one of the roots of PZ⁡X−λ to p⁡x. Because of the properties of P, this is sufficient. This is where the Symlets are different from the Daubechies wavelets. In the generation of the Daubechies wavelets, you can simply pick the root in the unit circle of the complex plane. This choice of spectral factorization has maximal phase. To compute the choice of spectral factorization with minimal phase, you have to compute all spectral factorizations, and pick the one where the nonlinear part has the smallest L1 norm.
phase≔ procw,ziftypez,realconsthenarctan1+z⋅tanw21−z − w2eliftype⁡z,complexconsthenarctan⁡1 −z2⋅sinw1+z2⋅cosw−2 z⋅cosargumentz − w+piecewise⁡cosw<2 z⋅cosargument⁡z1+z2,π,0end ifend proc:
minphase≔ ∞:minphaseat≔ ∞:
Note that i and n should be thought of as binary numbers whose digits encode the factorization being used.
To save time, you can assign the first value. This means that you only range over half of all possible factorizations, but this is enough. Within the loop below, save a list of the phases, PhaseList, so that you can graph them later. Remember that minimal phase is defined by minimum L1 norm.
forifrom0to2nops⁡S−1−1do n≔i; φ≔phasew,S1; for j from 2 to nopsSdo φ≔φ+2 modp⁡n,2−12⁢phasew,Sj; n≔n−modp⁡n,22; end do; PhaseListi≔φ; myInt≔evalfInt⁡φ,w=0..π,method=_Gquad; ifmyInt<minphasethen minphase≔myInt; minphaseat≔i; end if; end do:
With this done, you can now graph all of the phases that were considered, with the minimal phase displayed with a bold line.
plotsdisplay⁡plot⁡seq⁡PhaseListi,i=0..2nops⁡S−1−1,w=0..π,plot⁡PhaseListminphaseat,w=0..π,thickness=5
Now construct the spectral factorization with the minimal factorization computed above, and extract the normalized scaling (low pass) coefficients.
n≔ minphaseat:if typeS1,realconsthen p≔x−S1else p≔x−S1⁢x−S1&conjugate0; end if: for j from 2 to nopsS do if typeSj,realcons then p≔p x−Sj2⁢modp⁡n,2−12 else p≔p x−Sj2⁢modp⁡n,2−12⁢x−Sj&conjugate0;2⁢modp⁡n,2−12 end if; n≔n−modp⁡n,22; end do:
pn≔collect(p⁢1+xA⁢21−Apx=1|px=1,x): C≔ListToolsReverseseq⁡ℜcoeff⁡pn,x,k,k=0..2⁢A−1:Cadd⁡Ci2,i=1..nops⁡C
0.002681814559,−0.001047384905,−0.01263630334,0.03051551319,0.06789269328,−0.04955283505,0.01744125436,0.5361019163,0.7677643179,0.2886296316,−0.1400472406,−0.1078082374,0.004010244846,0.01026817669
Verify this against the Maple function that computes wavelets.
l,h≔ DiscreteTransforms:-WaveletCoefficientsSymlet,2⁢A:converth,list
0.00268181456826015,−0.00104738488867974,−0.0126363034032406,0.0305155131658779,0.0678926935012206,−0.0495528349370428,0.0174412550868357,0.536101917090569,0.767764317004883,0.288629631750648,−0.140047240442934,−0.107808237703290,0.00401024487152240,0.0102681767084648
The Discrete Wavelet Transform
The wavelet transform can be accomplished for discrete signals by using an algorithm known as the (fast) discrete wavelet transform. Recall the coefficients hn and gn from equations (1) to (5). The low pass filter, w2, is the hn, and the high pass filter, w1, is the gn (in vector form). In almost all useful cases, these are finite. The size of w1 and w2 is called the filter length. If w1 has n elements, w1 is often called an n-tap wavelet.
The discrete wavelet transform produces two outputs, each half the size of the input. The first output is the high detail coefficients, and the second is the low detail coefficients. They are computed by convolving w1 and w2, respectively, against the input data, and then down-sampling (throwing away every second term). The low detail coefficients are then recursively processed. It is not obvious, but this in fact computes the wavelet coefficients (the coefficients of each function in the orthonormal base of wavelets). The discrete wavelet transform is a linear transformation on the input signal. If the high and lows pass filter are:
w1=w11,w12,w13,w14
w2=w21,w22,w23,w24
then this linear transform on a signal of length 6 can be viewed as multiplication by the Matrix:
w11w12w13w1400w21w22w23w240000w11w12w13w1400w21w22w23w24w13w1400w11w12w23w2400w21w22
The result of the multiplication of this Matrix by the signal is an interlacing of the high and low pass coefficients. The orthogonality conditions on wi⁢j (recall that w1 and w2 are the wavelet and scaling coefficients from equations (3) to (5)) are equivalent to the Matrix being orthogonal! This makes the discrete wavelet transform an orthogonal linear transformation, and hence very easy to invert.
End Conditions
In the above convolutions, the filter mask "falls off" the end of the data. To maintain the orthogonality of the discrete wavelet transform (and the resulting easy invertibility), the data must be assumed to be periodic (as shown in preceding diagrams). However, two other common alternatives exist. The data can be padded with zeros, or the data can be reflected to generate extra data. In both cases, the transform is usually not invertible with orthogonal or biorthogonal wavelets, but can sometimes be modified to maintain easy invertibility. Such modifications are not discussed here. For example, given the signal [1,2,3,4,5,6], the following are the periodic, zeros, and reflection end conditions for extending the data.
Periodic:
1,2,3,4,5,6,1,2,3,...
Zeros:
1,2,3,4,5,6,0,0,0,...
Reflection:
1,2,3,4,5,6,5,4,3,...
Maple Functions for Wavelets
All of Maple's functions for wavelets are part of the SignalProcessing and DiscreteTransforms packages.
The SignalProcessing commands are: - DWT
- InverseDWT
The DiscreteTransform package commands are:
- DiscreteWaveletTransform
- InverseDiscreteWaveletTransform
- WaveletCoefficients
- WaveletPlot
See the corresponding help pages for basic information and examples. Examples from the DiscreteTransforms package are described below. For more examples from the SignalProcessing package, see the SignalProcessing examples page.
Examples
This is a quick example to explore the Daubechies length 4 wavelet.
with⁡DiscreteTransforms;withImageTools:
DiscreteWaveletTransform,FourierTransform,InverseDiscreteWaveletTransform,InverseFourierTransform,WaveletCoefficients,WaveletPlot
ExW1,ExW2≔ WaveletCoefficientsDaubechies,4
ExW1,ExW2≔−0.129409522551260−0.2241438680420130.836516303737808−0.482962913144534,0.4829629131445340.8365163037378080.224143868042013−0.129409522551260
Plot the Daubechies mother and father wavelets by using the WaveletPlot command. WaveletPlot uses a numerical algorithm called Cascades to approximate and plot functions satisfying equations (1) and (2). No explicit definition exists of the Daubechies length 4 mother and father wavelets.
WaveletPlot⁡ExW1,ExW2
The WaveletCoefficients command respects the Digits setting. In this case, if you increase the setting of Digits, you can correctly identify the symbolic expressions of the the wavelet coefficients.
Digits≔20:ExW1,ExW2≔WaveletCoefficientsDaubechies,4;map⁡identify,ExW1;Digits≔10:
ExW1,ExW2≔−0.1294095225512603811744494188120241641744−0.22414386804201338102597276224040035546800.8365163037378079055752937809168732034587−0.4829629131445341433748715998644486838167,0.48296291314453414337487159986444868381670.83651630373780790557529378091687320345870.2241438680420133810259727622404003554680−0.1294095225512603811744494188120241641744
−3⁢28+28−3⁢28+3⁢283⁢28+3⁢28−3⁢28−28
And of course, you can use the Daubechies 4 wavelet to transform data.
ExV≔ Vector10,i→i,datatype=float8:ExT≔DiscreteWaveletTransformExV,Daubechies,4;InverseDiscreteWaveletTransform⁡ExT,Daubechies,4
ExT≔2.22044604925031×10−164.44089209850063×10−164.44089209850063×10−160.−3.53553390593274,2.310789034541155.139216159287347.9676432840335310.796070408779712.6771540786184
1.2.000000000000003.000000000000004.000000000000005.000000000000006.000000000000007.000000000000008.9.10.0000000000000
You can also transform an image, by using the ImageTools package.
PreImg≔ Readcat⁡kerneloptsdatadir,/images/lichtenstein.jpg:ExImg≔MatrixToGrayscale⁡PreImg:ExImgHi,ExImgLo≔ DiscreteWaveletTransformExImg,1,Daubechies,20: ExImgHiHi,ExImgHiLo,ExImgLoHi,ExImgLoLo≔ DiscreteWaveletTransformExImgHi,2,Daubechies,20,DiscreteWaveletTransformExImgLo,2,Daubechies,20:Show≔ FitIntensity⁡ExImgLoLo|FitIntensity⁡ExImgLoHi,FitIntensity⁡ExImgHiLo|FitIntensity⁡ExImgHiHi:
EmbedCreate⁡Show
Sample Applications
Wavelets are of growing importance in a number of diverse fields, including seismology, underwater acoustics, computer vision, and signal processing. The largest application seems to be in image compression; below are three sample applications to illustrate of the capabilities of Maple's new wavelet functions.
Procedures and Initialization
First, define functions to transform (and inverse transform) a grayscale image. Discrete wavelet transforms of images are done by transforming first one dimension, and then the other. This process generates four outputs, the high-high, high-low, low-high, and low-low coefficients. The low-low coefficients can then be transformed recursively. Also define a small procedure to count the zeros in a Matrix, returning the result as a percentage of the number of elements in the Matrix, and two procedures needed for signal denoising.
restart;withDiscreteTransforms:withImageTools:
ImageDWT ≔ procimg::Or⁡Matrix,Array,w1::Vector,w2::Vector,iters::posint local rows,cols,n,final,current,temp; rows,cols≔LinearAlgebra:-Dimensionsimg; n≔1; final≔Matriximg; if modprows,2iters<>0ormodpcols,2iters<>0then error img's dimensions are not divisible by %1,2iters end if; temp≔Matrixrtable_dims⁡final,datatype=float8; for n to iters do DiscreteWaveletTransformrows2n−1,cols2n−1,final,1,w2,w1,temp,storagetype=singlearray; DiscreteWaveletTransformcols2n−1,rows2n−1,temp,2,w2,w1,final,storagetype=singlearray; end do; return final end proc:
InverseImageDWT ≔ procimg::Or⁡Matrix,Array,w1::Vector,w2::Vector,iters::posint local rows,cols,n,final,current,temp,temp2,i,j; rows,cols≔LinearAlgebra:-Dimensionsimg; n≔iters; final≔Matriximg; if modprows,2iters<>0ormodpcols,2iters<>0then error img's dimensions are not divisible by %1,2iters end if; temp≔Matrixrtable_dims⁡final,datatype=float8; for n from iters by −1 to 1 do InverseDiscreteWaveletTransformcols2n−1,rows2n−1,final,2,w2,w1,temp,storagetype=singlearray; InverseDiscreteWaveletTransformrows2n−1,cols2n−1,temp,1,w2,w1,final,storagetype=singlearray; end do; return final end proc:
count0≔ procimg, $ local rows,cols,i,j,count; rows,cols≔LinearAlgebra:-Dimensionsimg; count≔0; for j to cols do for i to rows do if imgi,j=0then count≔count+1 end if; end do; end do; return round100⋅ countrows⋅cols; end proc:
img≔ MatrixToGrayscale⁡Read⁡cat⁡kerneloptsdatadir,/images/fingerprint.jpg
VectorDWT≔ procV,iters,w1,w2, $ local i,s,temp1,temp2; s≔LinearAlgebra:-DimensionsV; if modps,2iters<>0then error the length of V is not divisible by enough powers of 2 end if; temp1≔VectorV; temp2≔VectorV; for i to iters do DiscreteWaveletTransforms2i−1,temp1,w2,w1,storagetype=singlearray,temp2; ArrayTools:-Copytemp2,temp1; end do; return temp1 end proc:
InverseVectorDWT≔ procV,iters,w1,w2, $ local s,i,temp1,temp2; s≔LinearAlgebra:-DimensionsV; if modps,2^iters<>0then error the length of V is not divisible by enough powers of 2 end if; temp1≔VectorV; temp2≔VectorV; for i from iters by −1 to 1 do InverseDiscreteWaveletTransforms/2^i − 1,temp1,w2,w1,storagetype=singlearray,temp2; ArrayTools:-Copytemp2,temp1; end do; return temp1 end proc:
Image Compression
The discrete wavelet transform is the analytical core of both the FBI fingerprint compression standard and the JPEG 2000 image compression standard. The success of wavelets in the FBI's case is nothing short of stunning: fingerprints are stored in a small fraction of the space while maintaining all of their distinguishing features. The power of wavelets in this area comes from the zero and near zero values that appear in images transformed by using wavelets. In lossy compression, these values can be set to zero, making the data much easier to compress. The examples below demonstrate the power of this method without actually performing any compression. You have already loaded an image above.
EmbedCreate⁡img
Now set the level of transform to be used, the percentage of zeros desired, and the wavelet to be used.
imglevels:=2:imgper≔85:imgWLname≔Daubechies:imgWLparam≔20:
Now you can transform and threshold the data. In the thresholding step, the smallest (in absolute value) per percent of the values in the transformed data are set to zero. As a check, output the percentage of zeros in R, the thresholded, transformed data.
imgT≔ ImageDWTimg,WaveletCoefficients⁡imgWLname,imgWLparam,imglevels:imgthresh≔ StatisticsPercentileListTools:-Flatten⁡convert⁡map⁡abs,imgT,listlist,imgper:imgR:=Matrix⁡mapz→`if`⁡z≤imgthresh,0,z,imgT,datatype=float8:
count0⁡imgR
85
EmbedCreate⁡imgT
And now you can invert the transformation, using the thresholded data.
imgNew≔ InverseImageDWTimgR,WaveletCoefficientsimgWLname,imgWLparam,imglevels:Side≔ img|imgNew:
EmbedCreate⁡Side
The quality of the new image is amazing, given that it was constructed from data that was per percent zero!
In addition to visually judging the quality of the reconstructed image, you can compute the average error (relative to the original image), view a black and white Matrix representing high error pixels, and use the ImageTools:-Quality command.
LinearAlgebra:-Norm⁡imgNew−img,∞rtable_num_elems⁡img;Er:=MatrixLinearAlgebra:-Dimensionsimg,i,j→`if`0.05<imgNewi,j−imgi,j,1,0:Quality⁡img,imgNew
0.0002205079906
0.00288222335046411059
EmbedCreate⁡Er
Given the ability to transform images to produce lots of zeroes without significantly affecting image quality, the applications of wavelets in image compression are obvious.
Signal Denoising
First create a signal, S, and a noisy version, NS.
Signal:=Vector⁡128,i→evalfsin⁡4⁢i⁢π128+1⁢sin⁡12⁢i⁢π1283,datatype=float8:NS:=Vector⁡128,i→Signali+1⁢rand⁡5000000000000,datatype=float8:plotslistplot⁡NS
Now transform the data, threshold it, and perform the inverse transform.
SDlevels≔3:SDthresh≔0.25:SDw1,SDw2 ≔WaveletCoefficientsCoiflet,4:SDT≔ VectorDWTNS,SDlevels,SDw1,SDw2:plotslistplot⁡SDT
SDR:=Vector⁡mapz→`if`⁡z≤SDthresh,0,z,SDT,datatype=float8:plotslistplot⁡SDR
SDNew:=InverseVectorDWT⁡SDR,SDlevels,SDw1,SDw2:plotslistplotSDNew
Amazing!
Matrix-Vector Multiplication
In the situation where a fixed Matrix M must be multiplied by many Vectors V, the discrete wavelet transform can sometimes be used to speed up the calculations. The algorithm presented below preprocesses M by using wavelets to exploit patterns. If M consists of random values, there will be no speed up. By thresholding, this algorithm tries to reduce the number of floating-point multiplications required, but does not represent any decrease in algorithmic complexity. Note that orthogonal wavelets must be used in this algorithm. This application exploits the fact that, for orthogonal wavelets, the discrete wavelet transform is an orthogonal linear transformation. (See the section Discrete Wavelet Transform for a brief comment on the orthogonality of the discrete wavelet transform.)
TV≔VectorcolumnLinearAlgebra:-RandomVector200,datatype=float8:TM≔ Matrix200,200,datatype=float8,i,j→evalfsin⁡i+j,datatype=float8:
Because the discrete wavelet transform is an orthogonal linear transformation, you can simultaneously transform the rows of the Matrix M and the whole Vector V, while leaving their inner product unchanged. This is related to the fact that, for Vectors x and y and an orthogonal Matrix A,
A x.A y=A.AT.x.y = x.y
The fact that the transformation does not affect M.V can be verified numerically.
Tw1,Tw2≔ WaveletCoefficientsDaubechies,4:TV1 ≔VectorDWTTV,1,Tw1,Tw2:TM1 ≔DiscreteWaveletTransformTM,2,Tw2,Tw1,storagetype=singlearray:LinearAlgebra:-Norm⁡TM1.TV1−TM.TV
1.59161572810262442×10−12
It is also possible to transform the columns of M, but then the result has to be inverse transformed.
TM2 ≔DiscreteWaveletTransformTM1,1,Tw1,Tw2,storagetype=singlearray:TRes ≔InverseDiscreteWaveletTransformTM2.TV1,Tw1,Tw2,storagetype=singlearray:LinearAlgebra:-Norm⁡TRes−TM.TV
1.70530256582424045×10−12
So this means that you can effectively apply the same techniques that were used to transform and threshold images to simplify Matrix-Vector multiplication.
References
Introductory books
Aboufadel, Edward and Schlicker, Steven. Discovering Wavelets. Interscience, 1999.
Intermediate books; note that these books use fourier transforms extensively
Mallat, Stephane. A Wavelet Tour of Signal Processing. Academic Press, 1999.
Hernandez, Eugenio and Weiss, Guido. First course on wavelets. CRC, 1996.
The definitive book on wavelets from an expert standpoint
Daubechies, Ingrid. "Ten Lectures on Wavelets." SIAM, 1992.
Return to Index for Example Worksheets
Download Help Document