 Least Mean Square (LMS) algorithm is online learning algorithm developed by Widrow and Hoff in 1960.
 Rosenblatt’s perceptron was the first learning algorithm for solving linearly separable patternclassification problem.
 LMS algorithm was the first linear adaptivefiltering algorithm for solving problems such as prediction and communicationchannel equalization.

Advantages behind LMS Algorithm:
 Computationally Efficient: Its complexity is linear with respect to adjustable parameters.
 Simple to code and easy to build.
 Robust with respect to external disturbances.
 GaussNewton Method makes a balance between computational complexity and convergence behavior.
 Diagonal loading concept.
 Stabilizer term concept.
 A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is called Markov process. The term strong Markov property is similar to this, except that the meaning of “present” is defined in terms of a certain type of random variable, which might be specified in terms of the outcomes of the stochastic process itself, known as a stopping time.
 A hidden Markov model (HMM) is a statistical model in which the system being modeled is assumed to be a Markov process with unobserved state. An HMM can be considered as the simplest dynamic Bayesian network

Questions:

128In GaussNewton Method, Why we’ve get sum over squres of error signal?
 To make the value positive.
 Why not just get the sum of the error itself.
 128 “Linearize the dependence of e(i) on w”, Is That mean, trying to find a linear function that maps the dependence between e(i) and w?
 128How equation 3.18 has linearized e(i) and w?
 129Why they call it Jacobean matrix?
 129How we’ve calculated equation 3.20
 129What’s meant by nonsingular matrix multiplication?
 129Last paragraph that’s talking about Jacobean matrix conditions.

Advertisements
[…] This post was mentioned on Twitter by Abdelrahman AlOgail. Abdelrahman AlOgail said: has a new blog post "The Least Mean Square Algorithm": http://abdelrahmanogail.wordpress.com/2010/04/18/theleastmeansquarealgorithm/ […]
[…] The Least MeanSquare Algorithm « Abdelrahman AlOgail's Blog […]
What do you mean by this >
129What’s meant by nonsingular matrix multiplication?
first time to hear of non singular Matrix Multiplication,as i know
singularity property is only important while Solving the System
of equations.
Thanks for your Knowldge sharing 😉 Ziko.
Thanks for reply 🙂