The convergence of the Ben-Israel iteration for nonlinear least squares problems

Abstract:

Ben-Israel [1] proposed a method for the solution of the nonlinear least squares problem ${\min _{x \in D}}{\left \| {F(x)} \right \|_2}$ where $F:D \subset {R^n} \to {R^m}$. This procedure takes the form ${x_{k + 1}} = {x_k} - F’{({x_k})^ + }F({x_k})$ where $F’{({x_k})^ + }$ denotes the Moore-Penrose generalized inverse of the Fréchet derivative of F. We give a general convergence theorem for the method based on Lyapunov stability theory for ordinary difference equations. In the case where there is a connected set of solution points, it is often of interest to determine the minimum norm least squares solution. We show that the Ben-Israel iteration has no predisposition toward the minimum norm solution, but that any limit point of the sequence generated by the Ben-Israel iteration is a least squares solution.