create a matrix that shows similarity

Question

I have a dataset like this: data

Person	Salary	GPA	IQ
Alisson	1	1	1
Simon	1	2	2
Michael	2	3	2
Dani	2	1	2
Brian	1	2	2
David	1	1	2
Ilan	1	1	2
Juan	2	1	2
Julius	3	3	3
Philipp	3	3	3
Joshua	2	1	3
Rick	2	1	1
Moises	3	1	1
Fabian	3	1	1
Isaac	2	2	1
Kurt	2	2	2

I now would like to create a heatmap which shows how similar this people are to each other. So if two people have in all three variables (GPA,IQ and Salary) three same number, then they get the nnumber 1 in similarity. If they have only two similar numbers in this three variables then they get another color. But I don’t know how can I visualise that.

Asked By: Jonny

||

Source

Answer 1

You can create the similarity matrix using pandas and scipy. And then plot the heatmap using seaborn. I used @root answer to create the similarity matrix.

Here is the code:

from scipy.spatial.distance import euclidean, pdist, squareform
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

data = {'Person': ['Alisson', 'Simon', 'Michael', 'Dani', 'Brian', 'David', 'Ilan', 'Juan', 'Julius', 'Philipp', 'Joshua', 'Rick', 'Moises', 'Fabian', 'Isaac', 'Kurt'],
        'Salary': [1, 1, 2, 2, 1, 1, 1, 2, 3, 3, 2, 2, 3, 3, 2, 2],
        'GPA': [1, 2, 3, 1, 2, 1, 1, 1, 3, 3, 1, 1, 1, 1, 2, 2],
        'IQ': [1, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 1, 1, 1, 1, 2]}

df = pd.DataFrame(data)
df = df.set_index("Person")

def similarity_func(u, v):
    return 1/(1+euclidean(u,v))
dists = pdist(df, similarity_func)
similarity_matrix = pd.DataFrame(squareform(dists), columns=df.index, index=df.index)


fig, ax = plt.subplots(figsize=(10,10))
sns.heatmap(similarity_matrix, annot=True, cmap='YlGnBu', linewidths=.5, ax=ax)

Output:

Option 2: Evaluate similarity only based on exact same value for column.

from scipy.spatial.distance import pdist, squareform
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

data = {'Person': ['Alisson', 'Simon', 'Michael', 'Dani', 'Brian', 'David', 'Ilan', 'Juan', 'Julius', 'Philipp', 'Joshua', 'Rick', 'Moises', 'Fabian', 'Isaac', 'Kurt'],
        'Salary': [1, 1, 2, 2, 1, 1, 1, 2, 3, 3, 2, 2, 3, 3, 2, 2],
        'GPA': [1, 2, 3, 1, 2, 1, 1, 1, 3, 3, 1, 1, 1, 1, 2, 2],
        'IQ': [1, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 1, 1, 1, 1, 2]}

df = pd.DataFrame(data)
df = df.set_index("Person")

def similarity_func(u, v):
    similarities = []
    for i in range(len(u)):
        if u[i] == v[i]:
            similarities.append(1)
        else:
            similarities.append(0)
    return sum(similarities) / len(similarities)

dists = pdist(df, similarity_func)
similarity_matrix = pd.DataFrame(squareform(dists), columns=df.index, index=df.index)

fig, ax = plt.subplots(figsize=(10,10))
sns.heatmap(similarity_matrix, annot=True, cmap='YlGnBu', linewidths=.5, ax=ax)

Output:

Answered By: Pedro Rocha

create a matrix that shows similarity

Question:

Answers: