A general theory on rates of convergence in multiple regression is developed, where the regression function is modeled as a member of an arbitrary linear function space (called a model space), which may be finite- or infinite-dimensional. A least squares estimate restricted to some approximating space, which is in fact a projection, is employed. The error in estimation is decomposed into three parts: variance component, estimation bias, and approximation error. The contributions to the integrated squared error from the first two parts are bounded in probability by $N_n/n$, where $N_n$ is the dimension of the approximating space, while the contribution from the third part is governed by the approximation power of the approximating space. When the regression function is not in the model space, the projection estimate converges to its best approximation.

The theory is applied to a functional ANOVA model, where the multivariate regression function is modeled as a specified sum of a constant term, main effects (functions of one variable), and interaction terms (functions of two or more variables). Rates of convergence for the ANOVA components are also studied. We allow general linear function spaces and their tensor products as building blocks for the approximating space. In particular, polynomials, trigonometric polynomials, univariate and multivariate splines, and finite element spaces are considered.