Conditional Mutual information

Question:

The theoretical approach,i want to convert this into a python program[![][1]]2Was trying to test this package for calculation of conditional mutual information from a dataset .The package name-“dit”

My code:

from __future__ import division
import numpy as np
import dit
from dit import Distribution as D
from dit.multivariate import coinformation as I
from dit.example_dists import Xor

d=Xor()
d.set_rv_names(['X','Y','Z'])
X=111010
Y=101101
Z=001011

a=dit.multivariate.coinformation(d,'XY','Z')
print(a)

While running this program,the result i always get is “1.0”.
The result come as negative or positive value like “-0.0023” or “0.120”

Basically,i want to do this
https://stats.stackexchange.com/questions/147401/estimating-mutual-information-using-r
in Python.
A little bit of help will be highly appreciated

Asked By: user10626935

||
Source

Answers:

Tell me if this is the solution for you: set_rv_names to d:

d=Xor()

## ADD THIS LINE OF CODE BELOW
d.set_rv_names(['X', 'Y', 'Z'])

X=0.052290766
Y=0.004951425
Z=0.000246642

a=dit.multivariate.coinformation(d,'XY','Z')
print(a)
Answered By: razimbres

It’s already been a long time, but… If you just want to calculate conditional mutual information for DISCRETE r.v. use below code :

import pandas as pd

def conditional_mutual_information(data, X:set, Y:set, Z:set, delta = 1):
        X = list(X); Y = list(Y); Z = list(Z)
        cmi = 0

        P_Z = data.groupby(Z).size()
        P_Z = P_Z/P_Z.sum()

        P_XZ = data.groupby(X + Z).size()
        P_XZ = P_XZ/P_XZ.sum()

        P_YZ = data.groupby(Y + Z).size()
        P_YZ = P_YZ/P_YZ.sum()

        P_XYZ = data.groupby(X + Y + Z).size()
        P_XYZ = P_XYZ/P_XYZ.sum()

        for ind in P_XYZ.index:
            x_ind = ind[:len(X)]
            y_ind = ind[len(X):len(X + Y)]
            z_ind = ind[len(X + Y):]

            xz_ind = x_ind + z_ind
            yz_ind = y_ind + z_ind
            xyz_ind = ind

            z_ind =  pd.MultiIndex.from_tuples([z_ind], names = Z) if len(Z) != 1 else pd.Index(z_ind, name = Z[0])
            xz_ind = pd.MultiIndex.from_tuples([xz_ind], names = X + Z)
            yz_ind = pd.MultiIndex.from_tuples([yz_ind], names = Y + Z)
            xyz_ind = pd.MultiIndex.from_tuples([xyz_ind], names = X + Y + Z)

            cmi += delta * P_XYZ[xyz_ind].item() * np.log2(P_Z[z_ind].item() * P_XYZ[xyz_ind].item() / (P_XZ[xz_ind].item() * P_YZ[yz_ind].item()))

        return cmi


data = pd.DataFrame()
data['X'] = [1,1,1,0,1,0]
data['Y'] = [1,0,1,1,0,1]
data['Z'] = [0,0,1,0,1,1]
conditional_mutual_information(data, {'X'}, {'Y'}, {'Z'})
>> [out] 0.25162916738782287

If you want to know how it works, "Conditional Mutual Information" part of here

Answered By: Robster