# Lec-7 Solution of Difference Equation

Before we start, before we start I just like
to point out a small slip, in the last class there should have been the derivative that
you take for multiple poles; this should have been multiplied by m, please make the corrections,
z minus p K to the power m. In the example, you have used this, it was a small slip. We
are discussing in the last class; the inverse DTFT from the given frequency domain representation,
as given by this expression. So, x n will be this discrete line. Now, how
do you know, that this is going to give you the same sequence, you get my point? Given
a sequence x n, if I take its Fourier transform by this integral, do you get back the same
sequence? So, let us see let us verify this relation 1 by 2 p i minus p i to plus p I;
this is nothing but sigma x k e to the power minus j omega k into e to the power j omega
n, d omega, okay. You may write this as, sigma x k integral
minus p i to plus p i, 1 by 2 p i e to the power j omega n minus k d omega, okay. So,
that gives me if we integrate this, we will get after substitution of these values sin
n minus k p i by n minus k into p i. And this is; this particular function will be equal
to 1 when n is equal to k and equal to 0 otherwise. So, this gives me sigma sin n. So, basically
that gives you the series as x n, basically; this is a delta function n is equal to k and
n not equal to k, this function is basically delta n minus k. So, x k multiplied by delta
n minus k is x n itself, okay. Now, we take some difference equations and see what will
be the solution in the normal time domain. How to compute by the normal classical method? You have say a 0 x n plus y n is equal to
a 0 x n plus a 1 x n minus 1 plus and so on, A K x n minus k, all right. Now, the output
of the see for a sequence like this; suppose you have a term like a minus 1 x n plus 1
and so on, suppose the x n sequence is considered in the left hand side of the time domain,
can you have an output? Can you have an output which will be a function of the previous inputs,
as well as inputs which are yet to come? It is not possible, inputs which have not
yet come how can the system response, identify that, all right? So, that will be a non-causal
system, okay. So, causal system means, this kind of a term will not be present; for y
n I cannot have on the right hand side, the dependence of y n on x n plus 1 n plus 2,
okay. Sometimes we try find out, some kind of a
relationship between the present value and the future values; that is possible only when
you are having offline delta that is, the history is already available, so what is going
to come say after one hour that is already known to you, that record is there. So, can you correlate with the present value,
if it is possible? There are now causal systems also; you take an image all right, you focus
a light here, now it is not time domain function, it is phase domain function, means phase domain
this is plus all right. So, a light here will be influencing the illumination here, here,
here, it will also be influencing the illumination on this side, all right. So, if I consider in-space these as positive
side then this as negative side, then in that case at any instant of time; we are not considering
time as such, the illumination of a particular of a particular point will be depending also
on its previous values, I mean the location can be on both sides, that what I mean. So, there we can have functions with z to
the power plus 1, if you consider just one dimensional; say one line of a particular
object, of an image, then if this is z to the power minus 1, the normal backward shift
operator then this will be forward shift operator z to the power of plus 1, so both are possible.
So, this is that way something like a non-causal system, otherwise all natural processes are
causal systems. Now, suppose we have a difference equation;
k y on this side which is the summation of weighted values like this, equal to b k x
n minus k. k varying from say 1 to M, k varying from say, 0 to M and here it is 0 to N, okay.
You can write y solution y n, as b k x n minus k summation minus k varying from 0 to M minus
a k y n minus k, k varying from 1 to N, okay. So, when you have a general expression like
this; that is y n depends on the previous values of y, y n minus 1 n minus 2 because
k starts from 1 n minus 1 n minus 2 and so on and some values of the inputs then that
is known as auto-regressive process, okay. Auto-regressive and moving average, moving
average means; it is a weighted mean of the inputs average of the input, average in some
sense. You give more weightage to the present value
of the input, a little less weightage to the previous input and so on, but the previous
inputs also have an influence in the present on the present output. So, this is a moving
average part and it is regressing on itself, its previous values, so it is autoregressive. So, it is auto-regressive and moving average
more commonly known as, ARMA process, all right. ARMA, auto-regressive part what is
the order? N, ARMA N, moving average, what is the order? M. So, we write ARMA N M, it
is an ARMA, M N, N M process or somebody may choose P Q, ARMA, P Q process. When you have y n, as just one term of x n
and other terms of y; we call it simple auto regressive process, there no moving average,
just one input is that okay. When you have y n dependent on only inputs, it is called
moving average process, all right; moving average of order M, this is auto regressive
of A R process of order N and this also called FIR okay, Finite impulse response. Suppose we have, suppose we have
a system like this; will come to this very
soon, a system like this, what will be say in the Z domain, what will be the transfer
function? Y z by X z, it will be summation b k z to the power minus k, k varying from
0 to M, okay. So, this is H z, is it not? So, what will be h K impulse response? It
will be b 0, b 1 up to b K, b M okay. So, if I excite the system if excite the system,
with an impulse then this system; if I give an impulse as input then it will give you
output up to b M and after that it will be 0. So, it is an infinite length, so that is
that is why it is called finite impulse response system, FIR system. It is same as, in mathematics
we called it M A, moving average module, it one and the same thing. If on the other hand, if you have a system
like this that is Y z by X z comes out as 1 by some A z, that is basically 1 by 1 plus
a 1 z to the power minus 1 plus a 2 z to the power minus 2 and so on, the polynomial a
N z to the power minus N, okay. If you get something of this type then it
is called an IIR function, infinite impulse response. Let us see, what ii means infinite
impulse response. Let us take just one term, suppose H z equal to 1 by 1 minus 0.8 z inverse,
it can be plus or minus, it really does not matter. Then Y z is equal to X z by 1 minus
0.8 z inverse okay or multiplied by this Y K minus 0.8 Y k minus 1 is equal to x k, if
I multiply by this and then take the inverse, this will be the difference equation.
So, Y k is a solution, I can write 0.8 Y k minus 1 plus X k, this is a difference equation.
Now, I want to see its impulse response; that is X k is delta k, what will be the response,
okay. So, X k is delta k, that is x 0 is 1 then
x 1, x 2 etcetera are all 0’s, is it not? You are giving just an impulse at k equal
to 0 and after that it is all zero, so what will be y 0? If the system is initially unexcited
then there is, if it is a causal system; before the input is given there was no output, so
y at minus 1 is 0, okay. So, y 0 will be 0.8 into y at minus 1 which is 0 plus x at 0,
which means 0 plus x at 0 is 1. So, y 0 will be 1. What will be y 1? 0.8, it will be now y 0
which is 1 plus x at 1 is 0, so that will be 0.8. What will be y at 2? It will be 0.8
into again 0.8 plus 0. So, 0.64, so it will continue to give you output which will be
gradually diminishing in magnitude but it continues theoretically, it continues upto
infinite. It does not stop at a definite point. You can approximate, you can approximate truncate
it somewhere, but otherwise it is an infinite sequence; so that is why it is called IIR,
this kind of response is infinite impulse response. Now, what will be the response of
any system, in any difference equation that we have written of this kind? A difference
equation of this type, causal system; what will be the response, due to any general input?
So, the input whatever we give will decide the output but apart from that like you have
done in mathematics for a differential equation, there are two parts two components or solution,
a particular integral and a general, okay. So, y n will have y particular plus y complimentary,
okay. So, y complimentary when there is no input, when x n is 0 there is no input; there
is no input you calculate a K y n minus k to the right hand, it is all 0. So, this solution
is the complimentary solution and why particular there is particular solution will be due to
here. So, let assume a solution y n is equal to lambda to the power n, okay. So, substitute
it here. So, a K lambda n minus k summation, all right which will be a 0 plus a 1 lambda
plus a 2 lambda squared and so on, all right. And that is equal to 0, you get the solution
for lambda, is this all right, or other way? It is lambda to the power n, so okay a 0 lambda
to the power n, so it will be a 0 lambda to the power N; it should be capital N because
it is going upto a certain, see this n is a variable n and this N is a finite length,
since we are considering a finite length of this sequence, 1 to capital N, so I should
have taken N
minus k, okay. So, it will be lambda N minus 1 and so on
up to a N, that is equal to 0; so you can calculate the roots of lambda, there are N
roots all right. So, once you know the roots, linear combination of the roots will be the
solution. So, C 1 lambda 1 plus C 2 lambda 2 plus C
3 lambda 3 and so on; C N lambda N will be solution Y complimentary, lambda 1 to the
power n lambda 2 to the power n lambda 3 to the power n, because our assumed solution
was lambda to the power n as y n, okay. Y particular integral, particular solution
will be depending on the type of input, if
you give a sinusoidal input correspondingly output will also be sinusoidal, if you give
a constant input like an impulse, like a step then output will also be a constant, okay. So, you once again you assume a solution corresponding
to the type of input that you give; if it is not a regular input then of course it becomes
a little difficult. So, Y will be depending on, say we give an input of step x n is a
some k times u n then correspondingly; we can assume a solution alpha because it is
a constant, so output corresponding to this part will be a constant. So from the equation, that you have already
got say; a 0 we wrote a k, all right, a k a k plus a sorry a k y n plus a k minus 1
y n minus 1 and so on. This will be all alpha plus alpha plus alpha, so on because they
are all constant, n terms should be equal to the given value k into u n k, okay. So,
N times alpha is k, so alpha you have got k by N. The input has been given as an constant magnitude,
step function, all right; so all the time it will be equal to k. So in the difference
equation, this is given as x n which is k times u n, so you get with of course with
sorry sorry a k minus 1 etcetera, I am sorry. So, it will be summation a k into alpha is
equal to k, so alpha is summation a k, all right. Let us workout a small example, it
will be clear. Once you have got this, try to use it with the initial conditions, try
to match with the initial conditions, you can calculate all the constants. We have a system, y n plus 5 into y n minus
1 plus 4 into y n minus 2 is equal to x n, all right. Y at minus 1 is equal to 2; y at
minus 2 is equal to 1, determine the response corresponding to x n is equal to 4 times u
n, okay. So, let us first consider when x n is 0 that is a complimentary solution. Y
complimentary will be, we assume lambda to the power n, so the solution for that will
be lambda square plus 5 lambda plus 4 lambda square, sorry lambda square plus 5 lambda
plus 4, okay equal to 0. This is also known as characteristic equation;
lambda is equal to minus 4 or minus 1, okay. So, the solution
will be some C 1 times minus 4 to the power
n plus C 2 times minus 1 to the power n into of course u n, okay. Next, we take y particular solution as some
constant alpha because you are exciting it by, a step function then alpha plus 5 into
alpha plus 4 into alpha will be equal to 4, is it not? So, alpha plus 5 into alpha plus
4 into alpha is equal to 4, alpha is equal to 4 by 10, all right. Now, y n is equal to
C 1 into minus 4 to the power n plus C 2 minus 1 to the power n plus 0.4, because it is a
complimentary solution plus particular solution, all right. Substitute the initial values, initial condition
are initial conditions are y at minus 1, what be the value at minus 1? C 1 minus 4 to the
power minus 1, N is equal to minus 1 plus C 2 minus 1 to the power minus 1 plus 0.4
and that is given as 2; this is minus of 4, so minus C 1 by 4 minus C 2 is equal to 2
minus 0.4, that is 1.6, okay. If you simplify C 1 plus 4, C 2 is equal to
minus 6.4, okay this is one equation. We substitute in the second equation, second condition. So, y at minus 2 is similarly minus 4 to the
power minus 2; so it will be 1 by 16 plus 1 by 16 minus 1 to the power minus 2, is again
plus 1. So, C 1 by 16 I am just writing the final values 0.4 and that is equal to 1 or
from this C 1 plus 16 C 2 is equal to 9.6, 0.4 if I bring it to this side, it becomes
0.6 0.6 into 16.9.6, this a second equation. First equation was C 1 plus 4 C 2 equal to
minus 6.4 okay; this is was the first equation. So, if you subtract one from the other, C
1 will get cancelled. So, 12 C 2 will give me 9.6 plus 6.4. So, C 2 becomes 4 by 3, checks
whether, is it all right, okay. And then this is approximately 11.733, okay with a negative
sign, C 1 is negative. So the solution will be y n equal minus 11.733
minus 4 to the power n plus 1.33 minus 1 to the power n plus 0.4, this is a general solution
for n greater than equal to 0, is it all right. Whether same thing you can solve by Z transform
method also, that we have already seen. Sometimes when you study signals, sometimes
you want to compare one signal with other; especially in communication. Some signals
are coming, you want to check whether it is matching with some other signal or not. So,
you have got the sample values, you try correlate them, so it define a correlation function
between two signals. If it is between two different signals say, x n and Y n we call
it cross correlation. If it is a signal with it itself then it is auto-correlation So, cross correlation we define r x y of lag
l as, sigma x n y n minus l; l equal to 0 plus minus 1 plus minus 2 and so on, I can
have lag positive or negative. r y x l will be sigma Y n x n minus l, okay.
This n is over this range minus infinite plus infinite. So, this will be if you change the
variables; you will get r x y minus l, this is something very similar to which term, very
similar to convolution, is it not? You try to find out the certain difference between
convolution of a sequence up to sequences and cross correlation, there is a difference,
all right. So, here it is n, n minus l the other one
is l minus n. Now, auto-correlation
of a sequence x, it r x x l is equal to x n into x n minus l, okay. r x x 0, will be
X square n x n into x n, it is also called the energy of the signal. Now this is an even
function, r x x l is an even function, can I tell me why, this is say x n this is also
x n all right; now if I give it a shift by l all right or if I give shift to this one,
it is one at the same thing. Means, if I shifted forward or backward, it
is symmetrical, it is a same sequence, all right. So, it is an even function, is it not?
For example, if you have hundred points; x zero, x one, x two, x three, x hundred then
if I take a lag of 1, so it will be x 0 into x one, x one into x two, x two into x three
and so on, x ninety-nine to x hundred. If I take a lag on the other side; it will
be x one into x zero, x two into x one, it is just x one and x two; I mean the the two
sequences are reversed all right, but they give you the same product, so it is an even
function…..Now, let us before we go to any other discussions; let us define a positive
definite matrix, definite matrix or positive definite function. What is a positive definite function? Okay,
let us take a vector alpha 1, alpha 2, alpha 3 and a vector vector beta as beta 1, beta
2 beta 3, okay. So, what is alpha transposed beta; it is an inner product which will be
alpha 1, alpha 2, alpha 3 into beta 1, beta 2, beta 3, is alpha 1 beta 1 plus alpha 2
beta 2 plus alpha 3 beta 3, it is same as beta transposed alpha, is it not? If I would have transposed these and then
multiplied by this, I would have got the same product, is that so. Now, let us take y as
alpha transposed x okay. A vector x is converted to a is transformed to a new vector y, so
y 1, y 2, y 3 is a vector which is alpha 1 x 1 plus alpha 2 x 2 plus alpha 3 x 3, okay
where alpha is alpha 1, alpha 2, sorry sorry sorry I am sorry, this not the inner product. y is equal to, okay y is equal to A x, all
right, any matrix A. So, x 1, x 2, x 3 multiplied by A will give me vector Y, all right. The
elements of x may be positive or negative, I don not know all right. What is alpha transposed
alpha? Any vector pre multiplied by its transpose will give the will give me alpha 1 square
plus alpha 2 square plus alpha 3 squared which is always positive, okay. So, what is y transposed Y? It will be A X
transpose A X, all right which is X transpose A X all right. This A transpose A is this
entire thing is always positive, because Y transpose y is always positive. I have shown
for any vector alpha, alpha transpose alpha is always positive; so any vector Y will be
always giving you, Y transpose y as a positive quantity, okay. So, this is a positive quantity
irrespective of the vector x. So when you get such a function of X, that
is a vector x 1, x 2, x 3 etcetera multiplied by a matrix P which is transpose A is a square
matrix okay; post multiplied by the vector again, x 1, x 2, x 3 are the elements. If
it is always positive then this is called a positive definite matrix, okay and this
function is a positive definite function of X, whatever be the value of x the elements
may be positive or negative; it is always positive, okay. It is something like an energy function, say
if move my hand in this direction there is lots of frictional energy all right, if I
move it backward then also it will be positive, heat will be dissipated whether I move it
this way or that way, all right. So, it is something like energy which is always positive
so positive definite matrix and positive definite functions is defined. Positive definite function is when it is always
positive, if it is sometimes positive, sometimes zero, it is called positive semi-definite.
Similarly, if some function is always negative just put a negative sign on this, this becomes
a negative definite or negative semi-definite. If you have a finite energy sequence, X n and
a combination of a x n plus another sequence with
a with a lag, what will be the energy content
of this signal? That means, you have a linear combination
of two signals, all right you are measuring the energy content of that signal; if x n
is a finite energy, y n is a finite energy then it will also be a finite energy sequence,
all right. And let us see, what will be the energy of this sequence. Energy of this sequence
is summation of this, which will be a squared x n squared plus twice a sigma x n y n minus
l plus sigma y n minus l square, equal to a squared this is nothing but r x x 0 plus
2 a, this is nothing but r x y l and this is r y y 0, okay because there all varying
over minus infinituve to plus infinitive. We are assuming the sequence to be, extended
in both the directions all right. And that must be always greater than equal to 0, because
this is finite energy, this is finite energy. So, we can write this as a 1 r x x 0, r x
y l r x y l, r y y 0, this matrix into a 1 that is greater than equal to 0; so this is
a positive semi-definite form matrix, positive semi-definite matrix. If
it is positive semi-definite, basically for
a two by two matrix it is very simple, the determinant must be always greater than equal
to 0, determinant must be always greater than equal to 0. That means r x x 0 into r y y 0 minus r x
y l squared, should be greater than equal to 0 or r x y should be less than equal to
r x x 0 r y y 0 under root; that is under root of E x energy of signal x, energy of
signal y, okay. So by the same logic when we are considering the same signal x, therefore
r x x at any lag should be less than r x x, this is l; into r x x under root will be r
x x 0 or that is equal to E x. So, if you take a signal; if you take its
autocorrelation with a lag say, 1 or 2 or 3 that will be always less than or equal to
at the most; the value of the autocorrelation when you are taking at n equal to 0, when
you are multiplying a quantity with itself and then take the product sum, that is giving
the maximum value. So, we will stop discussion on this, at this
point. Next time, we would like to take up discrete Fourier transform, in most of the
books they do not distinguish between distinct between discrete Fourier transform, discrete
Fourier series and discrete time Fourier transform. So, we would like to bring out the certain
difference between these terms first and then we will go over to the competition of discrete
Fourier transform. Discrete Fourier transform is the most widely
used transform in the signal processing and its numerical; the algorithmic version is
fast Fourier transform which is a basically competition of DFT, very fast algorithm for competition purpose. So, we will take up DFT and FFT in the next class. Thank you very much.

### 5 thoughts on “Lec-7 Solution of Difference Equation”

• January 13, 2010 at 12:14 am

most friendly teacher i have ever seen, even though some times it hard to follow things he teaches….Smart Professor !!! thanks

• December 10, 2012 at 3:47 pm

this is so awesome…. almost all of my questions are answerred now. thanks, professor!

• May 18, 2016 at 3:43 pm

Thank you some much ðŸ˜€