Optimally merging several informative observations on a random object is termed "information combining". Here we quantify information combining, by the mutual information between that object and all the observations, and it will also be associated with the error probability of the best detector based on the observations. We focus on binary symmetric objects (symbol bits) and address both variable nodes and check nodes information combining operators associated with the classical Sum-Product message passing algorithm (which is used for iterative decoding of LDPC codes). First we review recent results on on extremes in information combining, that is explicitly identifying the supremum and infimum of the combined mutual information, as well as the precombined extremal (conditional) densities, under the constraints of a given set of pre-combined (separated) mutual information values, and subsets of specified pre-combined channels (conditional densities). We extend this framework by adding an additional constraint on the pre-combined observations, which is associated with the error probability.To facilitate a natural application for the sum-product iterative procedure, we explore not only mutual information combining under two constraints, namely individual pre-combined, mutual information and probability of error, but also combining in terms of probability of error under the same two constraints. Specifically, optimization problems associated with the (conditional) density ofthe pre-combined observations are stated and the optimal target function value is solved in most cases or bounded in the others. A probabilistic channel decomposition setting is introduced to facilitate elegant handling of optimization problems. The theories of Tche byceff systems and Lagrange dual problems are central in the derivations of the results. Finally, we use the new results to improve previous bounds on the performance of iterative decoding of LDPC codes. We demonstrate our results for regular and irregular LDPC ensembles and show sensitivity of the bound quality to the value of the fraction of edges connected to degree-2 variable nodes.
This is joint work with Ilan Sutskover, EE Dept., Technion, Haifa and Intel Israel, ilan.sutskover@intel.com, and Jacob Ziv, EE Dept., Technion, Haifa, jz@ee.technion.ac.il.