Affordable Access

Information Structures of Capacity Achieving Distributions for Feedback Channels with Memory and Transmission Cost: Stochastic Optimal Control & Variational Equalities-Part I

Publication Date
Submission Date
arXiv ID: 1512.04514
External links


The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions $\big\{{\bf P}_{B_i|B^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$ and $\big\{ {\bf P}_{B_i|B_{i-M}^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$, where $M$ is the memory of the channel, $B^n = \{B_j: j=0,1, \ldots, n\}$ are the channel outputs and $A^n=\{A_j: j=0,1, \ldots, n\}$ are the channel inputs. The characterizations of FTFI capacity, are obtained by first identifying the information structures of the optimal channel input conditional distributions ${\cal P}_{[0,n]} =\big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}: i=1, \ldots, n\big\}$, which maximize directed information $C_{A^n \rightarrow B^n}^{FB} = \sup_{ {\cal P}_{[0,n]} } I(A^n\rightarrow B^n), \ \ \ I(A^n \rightarrow B^n) = \sum_{i=0}^n I(A^i;B_i|B^{i-1}) . $ The main theorem states, for any channel with memory $M$, the optimal channel input conditional distributions occur in the subset ${\cal Q}_{[0,n]}= \big\{ {\bf P}_{A_i|B_{i-M}^{i-1}}: i=1, \ldots, n\big\}$, and the characterization of FTFI capacity is given by $C_{A^n \rightarrow B^n}^{FB, M} = \sup_{ {\cal Q}_{[0,n]} } \sum_{i=0}^n I(A_i; B_i|B_{i-M}^{i-1})$ . Similar conclusions are derived for problems with transmission cost constraints. The methodology utilizes stochastic optimal control theory, to identify the control process, the controlled process, and a variational equality of directed information, to derive upper bounds on $I(A^n \rightarrow B^n)$, which are achievable over specific subsets of channel input conditional distributions. For channels with limited memory, this implies the transition probabilities of the channel output process are also of limited memory.


Seen <100 times