Iterative Hard Thresholding Algorithm: Core Optimization for Sparse Signal Recovery

The Iterative Hard Thresholding (IHT) algorithm serves as a fundamental optimization method in compressed sensing for sparse signal recovery. Its core mechanism involves alternating between gradient descent steps and hard thresholding operations to progressively approach the optimal solution under sparsity constraints.

The algorithm workflow comprises three key stages implemented iteratively: First, compute the gradient of the current signal estimate (typically derived from the product of the measurement matrix and residual vector) - in code this translates to gradient = A'*(y-A*x) where A is the measurement matrix, y is observed data, and x is current estimate. Second, update the temporary solution via gradient descent: x_temp = x + mu*gradient, where mu represents the step size parameter. Third, apply hard thresholding to enforce sparsity by retaining only the K largest-magnitude components (implemented as x_next = hard_threshold(x_temp, K)), effectively zeroing out all but the top K elements. This "gradient update + sparsity enforcement" cycle significantly enhances reconstruction accuracy for high-dimensional signals under undersampled conditions.

In compressed sensing applications, IHT gains widespread adoption due to its computational efficiency and noise robustness. Compared to convex optimization approaches like LASSO, IHT directly handles the non-convex L0 constraint, avoiding bias introduced by relaxation techniques. However, proper tuning of step size (mu) and sparsity level (K) parameters is critical for convergence guarantee. Enhanced variants such as accelerated IHT and adaptive IHT further improve convergence rates and stability through optimized step size selection and dynamic sparsity adaptation mechanisms.