Signal processing can be characterized as the study of problems involving information that, after having been transmitted, is altered in some undesireable way before being received at its desired destination. The aim is to intelligently and efficiently separate from the collected data that part which is not desired. Applications that can be associated with signal processing are ever growing and include such diverse fields as biomedical engineering, radar, control and communications.

Particularly true for algorithms in signal processing, computational complexity and numerical reliability are of great concern. During the last two decades the introduction of so-called fast adaptive algorithms were of importance since they reduced arithmetic complexity from O(p²) to O(p), where p denotes a standard measurement that reflects the order of the underlying covariance matrix that is related to the problem. Unfortunately, reports were soon given citing cases in which many of these algorithms diverged. More troubling was the publishing of analyses stating the stability of certain algorithms that later were shown to not always produce numerically reliable solutions.

My research has concentrated on the numerical analysis of adaptive filtering algorithms in signal processing. In particular, I have worked to improve the theoretical understanding of algorithmic performance, i.e., analysing the numerical behavior of adaptive filtering algorithms. Additionally, and to address the apparant confusion regarding numerical analysis techniques employed on algorithms and their meaning with respect to the value of an algorithm's computed result, I have worked to facilitate the correct interpretation given by a numerical analysis that is employed on signal processing algorithms.
 

The analysis of the numerical properties of many signal processing algorithms (such as the recursive least-squares adaptive filtering algorithms) has, however, been hindered by problems not usually found in numerical linear algebra algorithms; nonlinear equations for parameter updates as well as recursive-in-time algorithms (and the associated long term effects of finite precision arithmetic) are examples. Furthermore, the meaning of `stability' can differ greatly between disciplines and not all definitions, when satisfied, provide a guarantee that the computed solution is accurate enough to be meaningful to the user. In numerical linear algebra, an algorithm's analysis must address the impact of the problem's conditioning as well as the expected accuracy of the computed solution before a stability analysis is said to have been completed. For signal processing algorithms, this goal has proven, thus far, to be an elusive quest. One cause, for recursive algorithms at least, is the fact that the `problem' changes with time. Furthermore, the diverse interpretations given for algorithmic stability in signal processing have had a correspondingly broadening effect on stability analyses; a stability analysis can be said to have been performed if any one of a number of definitions for stability can be proven (or disproven).

It has become evident, therefore, that extracting the true benefit from an analysis of a signal processing algorithm is not as straight forward as for an algorithm in numerical linear algebra. Nonetheless, these analyses do provide important insights into the performance capabilities of a signal processing algorithm and may, perhaps, form the basis of a much needed, coherent framework for the numerical anaysis of signal processing algorithms.

By utilizing the established techniques for numerical analysis in linear algebra whenever possible, progress has been made in furthering the theoretical understanding for recursive least squares based algorithms as well as the more aggressive goal for improving the means in which numerical analysis results can be measured as well as communicated.
 

For fast Recursive Least Squares Lattice (RLSL) algorithms, it was proposed (and heuristically shown) in the literature that the manner chosen for updating a particular parameter significantly characterized the output's robustness to machine arithmetic. In [3], a formal analysis was provided to compare the numerical effects of both implementations.

Additionally, my current research has shifted to fast recursive filters that are transversal as opposed to lattice based. Exemplifying this is the Fast Transversal Filter (FTF) algorithm that exhibits catastrophically divergent behavior. By catastrophic divergence, it is meant that the algorithm can seemingly give useful results at one iteration but then give completely meaningless results thereafter. It was previously discovered (by others) that a certain parameter exceeded its theoretical threshold just before this phenomena occurred, thus suggesting to these authors that one could "restart" the algorithm whenever the threshold was violated. Unfortunately, this allowed detection of an imminent divergence only after the algorithm's parameters had been significantly perturbed. Although the true cause of this divergence remains unanswered, in [1], the notion of consistency was used to introduce a method that is much more predictive of catastrophic divergence with respect to the FTF algorithm.
 

Future interest involves the underlying reason that explains the nature of the divergence of the FTF algorithm (catastrophic). Although it is known why the algorithm loses consistency and hence diverges, it remains unknown why the algorithm diverges catastrophically rather than more regularly. This is of theoretical importance since many fast algorithms are derived by exploiting redundancies exposed when considering multiple and related systems of equations. If catastrophic divergence is characteristic to the manner in which some fast algorithms are derived, the results of this ongoing investigation will be of importance.
 

1. Bunch, James R., Le Borne, Richard C., Proudler, K. Ian. Tracking ill-conditioning for the RLS-lattice algorithm. IEE Proceedings Vision, Image and Signal Processing 1998;145[(1)]:1-6.
2. Bunch, James R., Le Borne, Richard C., Proudler, K. Ian. A conceptual framework for consistency, conditioning and stability issues in signal processing. IEEE Transactions on Signal Processing; September 2001, v49:1971-1981.
3. Bunch, James R., Le Borne, Richard C., Proudler, K. Ian. Analysis of the direct and indirect a posteriori RLSL algorithm. Journal of Numerical Linear Algebra with Applications, September 2001, 8:453-466.
4. Bunch, James R., Le Borne, Richard C., Proudler, K. Ian. Applying Numerical Linear Algebra Techniques to Analyzing Algorithms in Signal Processing. Linear Algebra and Its Applications 2003, v.361, 133-146.