The Least Mean-Square Algorithm

  • Least Mean Square (LMS) algorithm is online learning algorithm developed by Widrow and Hoff in 1960.
  • Rosenblatt’s perceptron was the first learning algorithm for solving linearly separable pattern-classification problem.
  • LMS algorithm was the first linear adaptive-filtering algorithm for solving problems such as prediction and communication-channel equalization.
  • Advantages behind LMS Algorithm:
    • Computationally Efficient: Its complexity is linear with respect to adjustable parameters.
    • Simple to code and easy to build.
    • Robust with respect to external disturbances.
  • Gauss-Newton Method makes a balance between computational complexity and convergence behavior.
  • Diagonal loading concept.
  • Stabilizer term concept.
  • A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is called Markov process. The term strong Markov property is similar to this, except that the meaning of “present” is defined in terms of a certain type of random variable, which might be specified in terms of the outcomes of the stochastic process itself, known as a stopping time.
  • A hidden Markov model (HMM) is a statistical model in which the system being modeled is assumed to be a Markov process with unobserved state. An HMM can be considered as the simplest dynamic Bayesian network
  • Questions:
    • 128-In Gauss-Newton Method, Why we’ve get sum over squres of error signal?
      • To make the value positive.
    • Why not just get the sum of the error itself.
    • 128- “Linearize the dependence of e(i) on w”, Is That mean, trying to find a linear function that maps the dependence between e(i) and w?
    • 128-How equation 3.18 has linearized e(i) and w?
    • 129-Why they call it Jacobean matrix?
    • 129-How we’ve calculated equation 3.20
    • 129-What’s meant by nonsingular matrix multiplication?
    • 129-Last paragraph that’s talking about Jacobean matrix conditions.

4 thoughts on “The Least Mean-Square Algorithm

  1. What do you mean by this ->
    129-What’s meant by nonsingular matrix multiplication?

    first time to hear of non singular Matrix Multiplication,as i know
    singularity property is only important while Solving the System
    of equations.

    Thanks for your Knowldge sharing 😉 Ziko.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s