Estimation of the number of signals impinging on an array of sensors, also known as source enumeration, is usually required prior to direction-of-arrival (DOA) estimation. In challenging scenarios such as the presence of closely-spaced sources and/or high level of noise, using the true source number for nonlinear parameter estimation leads to the threshold effect which is characterized by an abnormally large mean square error (MSE). In cases that sources have distinct powers and/or are closely spaced, the error distribution among parameter estimates of different sources is unbalanced. In other words, some estimates have small errors while others may be quite inaccurate with large errors. In practice, we will be only interested in the former and have no concern on the latter. To formulate this idea, the concept of effective source number (ESN) is proposed in the context of joint source enumeration and DOA estimation. The ESN refers to the actual number of sources that are visible at a given noise level by a parameter estimator. Given the numbers of sensors and snapshots, number of sources, source parameters and noise level, a Monte Carlo method is designed to determine the ESN, which is the maximum number of available accurate estimates. The ESN has a theoretical value in that it is useful for judging what makes a good source enumerator in the threshold region and can be employed as a performance benchmark of various source enumerators. Since the number of sources is often unknown, its estimate by a source enumerator is used for DOA estimation. In an effort to automatically remove inaccurate estimates while keeping as many accurate estimates as possible, we define the matched source number (MSN) as the one which in conjunction with a parameter estimator results in the smallest MSE of the parameter estimates. We also heuristically devise a detection scheme that attains the MSN for ESPRIT based on the combination of state-of-the-art source enumerators.

Numerous methodologies have been investigated for source enumeration in sample-starving environments. For those having their root in the framework of random matrix theory, the involved distribution of the sample eigenvalues is required. Instead of relying on the eigenvalue distribution, this work devises a linear shrinkage based minimum description length (LS-MDL) criterion by utilizing the identity covariance matrix structure of noise subspace components. With linear shrinkage and Gaussian assumption of the observations, an accurate estimator for the covariance matrix of the noise subspace components is derived. The eigenvalues obtained from the estimator turn out to be a linear function of the corresponding sample eigenvalues, enabling the LS-MDL criterion to accurately detect the source number without incurring significantly additional computational load. Furthermore, the strong consistency of the LS-MDL criterion for m,n→∞ and m/n→ c ∈ (0,∞) is proved, where m and n are the antenna number and snapshot number, respectively. Simulation results are included for illustrating the effectiveness of the proposed criterion.

A low-complexity ESPRIT algorithm for direction-of-arrival (DOA) estimation is devised in this work. Unlike the conventional subspace based methods, the proposed scheme only needs to calculate two sub-matrices of the sample covariance matrix, that is, R11∈CK×K and R21∈C(M−K)×K, avoiding its complete computation. Here, M is the number of sensors of the array, K satisfies P≤K≤min(M,N) with P being the number of source signals and N being the number of snapshots. Meanwhile, a Nyström-based approach is utilized to correctly compute the signal subspace which only requires O(MK2) flops. Thus, the proposed method has the advantage of computational attractiveness, particularly when K⪡M. Furthermore, we derive the asymptotic variances of the estimated DOAs. Numerical results are included to demonstrate the effectiveness of the developed DOA estimator.

A family of algorithms, named ℓ p -MUSIC, for direction-of-arrival (DOA) estimation in impulsive noise is proposed. The ℓ p -MUSIC estimator adopts the ℓ p -norm (1 ≤ p 2) of the residual fitting error matrix as the objective function for subspace decomposition, rather than the Frobenius norm that is used in the conventional MUSIC method. Although the matrix ℓ p -norm minimization based subspace decomposition will lead to a nonconvex optimization problem, two iterative algorithms are designed for achieving efficient solutions. The first algorithm is the iteratively reweighted singular value decomposition (IR-SVD), where the SVD of a reweighted data matrix is performed in each iteration. The second algorithm solves the nonconvex matrix ℓ p -norm minimization by alternating convex optimization. Two complex-valued Newton's methods with optimal step size in each iteration are devised to solve the resulting convex problem. The convergence of the iterative procedure is also proved. Numerical results verify that the ℓ p -MUSIC methodology outperforms the standard MUSIC scheme and several existing outlier-resistant DOA estimation approaches in terms of resolution capability and estimation accuracy.

In this paper, we propose an eigenvector-based frequency estimator for R -dimensional ( R -D) sinusoids with R ≥ 2 in additive white Gaussian noise. Our underlying idea is to utilize the tensorial structure of the received data and then apply higher-order singular value decomposition (HOSVD) and structure least squares (SLS) to perform estimation. After obtaining the tensor-based signal subspace from HOSVD, we decompose it into a set of single-tone tensors from which single-tone vectors can be constructed by another HOSVD. In doing so, the R -D multiple sinusoids are converted to a set of single-tone sequences whose frequencies are individually estimated according to SLS. The mean and variance of the frequency estimator are also derived. Computer simulations are also included to compare the proposed approach with conventional R -D harmonic retrieval schemes in terms of mean square error performance and computational complexity particularly in the presence of identical frequencies.