Recently, sample complexity bounds have been derived for problems in(cid:173) volving linear functions such as neural networks and support vector ma(cid:173) chines. In this paper, we extend some theoretical results in this area by deriving dimensional independent covering number bounds for regular(cid:173) ized linear functions under certain regularization conditions. We show that such bounds lead to a class of new methods for training linear clas(cid:173) sifiers with similar theoretical advantages of the support vector machine. Furthermore, we also present a theoretical analysis for these new meth(cid:173) ods from the asymptotic statistical point of view. This technique provides better description for large sample behaviors of these algorithms.