entropic inequalities and marginal problems (fritz and chavez)

Jayant Apte ASPITRG

Entropic Inequalities and Marginal Problems*

Jayant ApteASPITRG

Drexel University

*FRITZ, T., & CHAVES, R. (2013). Entropic Inequalities and Marginal Problems. IEEE transactions on information theory, 59(2), 803-817.

Jayant Apte ASPITRG

The three coins

3

Jayant Apte ASPITRG

The three coins

4

Jayant Apte ASPITRG

The three coins

5

Jayant Apte ASPITRG

The three coins

6

Jayant Apte ASPITRG

An attempt

7

Jayant Apte ASPITRG

An attempt

8

Jayant Apte ASPITRG

An attempt

9

Jayant Apte ASPITRG

An attempt

But they are not!!!

10

Jayant Apte ASPITRG

Conclusion

● There is no joint distribution compatible with given marginals

11

Jayant Apte ASPITRG

Conclusion

● There is no joint distribution compatible with given marginals

● There is some systematic error in the coin flips

12

Jayant Apte ASPITRG

Conclusion

● There is no joint distribution compatible with given marginals

● There is some systematic error in the coin flips● OR The coins are operated by some weird

mechanism creating the required distribution as a function depending on which two are flipped together

13

Jayant Apte ASPITRG

Outline● Marginal scenario and marginal model● Contextuality● Marginal problem● Partial polymatroids● Relationship between marginal models and partial

polymatroids● Marginal Problem for polymatroids● Computations● Inference of common ancestors in Bayesian Networks● Contexuality of the n-Cycle marginal scenario● Extention to continuous random variables

14

Jayant Apte ASPITRG

Notation

15

Jayant Apte ASPITRG

Marginal Scenario

16

Jayant Apte ASPITRG

Marginal Model

S S'T

17

Jayant Apte ASPITRG

Contexuality

18

Jayant Apte ASPITRG

Example

19

Jayant Apte ASPITRG

Marginal problem

20

Jayant Apte ASPITRG

A naive probability space approach

21

Jayant Apte ASPITRG

Example: 3 coins

22

Jayant Apte ASPITRG

Example: 3 coins

This LP turns out to be infeasible

23

Jayant Apte ASPITRG

The entropy cone

24

Jayant Apte ASPITRG

Basic inequalities

25

Jayant Apte ASPITRG

Polymatroids

26

Jayant Apte ASPITRG

Partial Polymatroids

27

Jayant Apte ASPITRG

Contexuality of partial polymatroids

28

Jayant Apte ASPITRG

Example: Partial polymatroids over

Triangle inequality

29

Jayant Apte ASPITRG

Example: Modified 3-coins

30

Jayant Apte ASPITRG

References

● FRITZ, T., & CHAVES, R. (2013). Entropic Inequalities and Marginal Problems. IEEE transactions on information theory, 59(2), 803-817.