# The Least Mean-Square Algorithm

• Least Mean Square (LMS) algorithm is online learning algorithm developed by Widrow and Hoff in 1960.
• Rosenblatt’s perceptron was the first learning algorithm for solving linearly separable pattern-classification problem.
• LMS algorithm was the first linear adaptive-filtering algorithm for solving problems such as prediction and communication-channel equalization.
• Computationally Efficient: Its complexity is linear with respect to adjustable parameters.
• Simple to code and easy to build.
• Robust with respect to external disturbances.
• Gauss-Newton Method makes a balance between computational complexity and convergence behavior.
• Stabilizer term concept.
• A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is called Markov process. The term strong Markov property is similar to this, except that the meaning of “present” is defined in terms of a certain type of random variable, which might be specified in terms of the outcomes of the stochastic process itself, known as a stopping time.
• A hidden Markov model (HMM) is a statistical model in which the system being modeled is assumed to be a Markov process with unobserved state. An HMM can be considered as the simplest dynamic Bayesian network
• Questions:
• 128-In Gauss-Newton Method, Why we’ve get sum over squres of error signal?
• To make the value positive.
• Why not just get the sum of the error itself.
• 128- “Linearize the dependence of e(i) on w”, Is That mean, trying to find a linear function that maps the dependence between e(i) and w?
• 128-How equation 3.18 has linearized e(i) and w?
• 129-Why they call it Jacobean matrix?
• 129-How we’ve calculated equation 3.20
• 129-What’s meant by nonsingular matrix multiplication?
• 129-Last paragraph that’s talking about Jacobean matrix conditions.

## 4 thoughts on “The Least Mean-Square Algorithm”

1. […] The Least Mean-Square Algorithm « Abdelrahman Al-Ogail's Blog […]

2. Feras Ahmed says:

What do you mean by this ->
129-What’s meant by nonsingular matrix multiplication?

first time to hear of non singular Matrix Multiplication,as i know
singularity property is only important while Solving the System
of equations.

Thanks for your Knowldge sharing 😉 Ziko.