Feedback plays a pivotal role in control. Without feedback, even a small amount of error can destabilize control systems in stochastic environments. Communication systems, in which one ``controls'' another's state of knowledge, are notable exceptions. By encoding data with a forward error correcting code in long blocks, one can communicate reliably over a noisy channel without any feedback, as shown by Shannon.
Feedback can, however, improve the quality of communication in several important ways -- feedback can increase the capacity, reduce the probability of error, decrease communication delay, and even simplify the system design. Moreover, many common communication situations are over inherently two-way channels, such as telephone lines and the Internet, even when the information transfer is only in one direction. But the role of feedback in communication is not completely understood. For example, the feedback capacity of additive nonwhite Gaussian noise channels, despite partial results, has been open even for the simplest case.
In this talk, we focus on the nonwhite Gaussian channel as the canonical model of communication and ask the role of feedback. Using techniques from information theory, linear systems theory, convex optimization, and functional analysis, we characterize the Gaussian feedback capacity as the solution of a variational problem, which reveals an interesting interplay between control, estimation, and communication.
For first-order autoregressive moving average noise, this variational characterization gives a closed-form expression for the feedback capacity, answering a long-standing open question studied by Butman, Schalkwijk-Tiernan, Wolfowitz, Ozarow, Ordentlich, Yang-Kavcic-Tatikonda, and many others. More generally, we can show that a variant of the celebrated Schalkwijk-Kailath coding scheme achieves the feedback capacity for the general finite-order autoregressive moving-average Gaussian channel. Simply put, the optimal transmitter iteratively refines the receiver's knowledge of the intended message.