Abstract In the paper, modulation spectral features (MSFs) are proposed for the automatic emotional recognition for speech signal. The features are extracted from an auditory-inspired long-term spectro-temporal(ST) representation. On an experiment assessing classification of 4 emotion categories, the MSFs show promising performance in comparison with features that are based on mel-frequency cepstral coefficients and perceptual linear prediction coefficients, two commonly used short-term spectral representations. The MSFs further express a substantial improvement in recognition performance when used to augment prosodic features, which have been extensively used for speech emotion recognition. Using both types of features, an overall recognition rate of 91.55% is obtained for classifying 4 emotion categories.