Abstract The purpose of this note is to study the assumption of “mutual information independence”, which is used by Zhou [Zhou, G.D., 2005. Direct modelling of output context dependence in discriminative hidden Markov model. Pattern Recognition Lett. 26 (5), 545–553] for deriving an output-dependent hidden Markov model, the so-called discriminative HMM (D-HMM), in the context of determining a stochastic optimal sequence of hidden states. The assumption is extended to derive its generative counterpart, the G-HMM. In addition, state-dependent representations for two output-dependent HMMs, namely HMMSDO [Li, Y., 2005. Hidden Markov models with states depending on observations. Pattern Recognition Lett. 26 (7), 977–984] and D-HMM, are presented.