Speech recognition is a subjective occurrence. This work proposes a novel stochastic deep resilient network(SDRN) for speech recognition. It uses a deep neural network (DNN) for classification to predict the input speech signal. The hidden layers of DNN and its neurons are additionally optimized to reduce the computation time by using a neural-based opposition whale optimization algorithm (NOWOA). The novelty of the SDRN network is in using NOWOA to recognize large vocabulary isolated and continuous speech signals. The trained DNN features are then utilized for predicting isolated and continuous speech signals. The standard database is used for training and testing. The real-time data (recorded in ambient condition) for isolated words and continuous speech signals are additionally used for validation to increase the accuracy of the SDRN network. The proposed methodology unveils an accuracy of 99.6% and 98.1% for isolated words (standard and real-time) database and 98.7% for continuous speech signal (real-time). The obtained results exhibit the supremacy of SDRN over other techniques.