Abstract Classification problems have become more complex and intricate in modern applications in the face of continuous data explosion. In addition to great quantities of features and large numbers of instances, modern classification applications are continuously developed with multiple classes (objectives). The ever-increasing growth in data quantity and computation complexity has largely deteriorated the performance and accuracy of classification models. In order to deal with such situations, multivariate statistical analyses are adopted in this paper. Multivariate statistical analyses have two advantages. First, they can explore the relationships between variables and find the most characterizing features of the observed data. Second, they can solve problems which are stalled by high dimensionality. In this paper, the first advantage is applied to the selection of relevant features and the second is employed to generate the multivariate classifier. Experimental results show that our model can significantly improve classification training time by combining a compact subset of relevant features without the loss of accuracy in multi-class classification problems. In addition, the discrimination degree of our classifier outperforms other conventional classifiers.