Targeting at sparse learning, we construct Banach spaces of functions on an input space X with the following properties: (1) possesses an norm in the sense that is isometrically isomorphic to the Banach space of integrable functions on X with respect to the counting measure; (2) point evaluations are continuous linear functionals on and are representable through a bilinear form with a kernel function; and (3) regularized learning schemes on satisfy the linear representer theorem. Examples of kernel functions admissible for the construction of such spaces are given.

Frames in a Banach space were defined as a sequence in its dual space ⁎ in some recent references. We propose to define them as a collection of elements in by making use of semi-inner products. Classical theory on frames and Riesz bases is generalized under this new perspective. We then aim at establishing the Shannon sampling theorem in Banach spaces. The existence of such expansions in translation invariant reproducing kernel Hilbert and Banach spaces is discussed.

In this paper we investigate conditions on the features of a continuous kernel so that it may approximate an arbitrary continuous target function uniformly on any compact subset of the input space. A number of concrete examples are given of kernels with this universal approximating property.

Motivated by mathematical learning from training data, we introduce the notion of refinable kernels. Various characterizations of refinable kernels are presented. The concept of refinable kernels leads to the introduction of wavelet-like reproducing kernels. We also investigate a refinable kernel that forms a Riesz basis. In particular, we characterize refinable translation invariant kernels, and refinable kernels defined by refinable functions. This study leads to multiresolution analysis of reproducing kernel Hilbert spaces.

We continue our recent study on constructing a refinement kernel for a given kernel so that the reproducing kernel Hilbert space associated with the refinement kernel contains that with the original kernel as a subspace. To motivate this study, we first develop a refinement kernel method for learning, which gives an efficient algorithm for updating a learning predictor. Several characterizations of refinement kernels are then presented. It is shown that a nontrivial refinement kernel for a given kernel always exists if the input space has an infinite cardinal number. Refinement kernels for translation invariant kernels and Hilbert-Schmidt kernels are investigated. Various concrete examples are provided.