Discover relationship between the entities
Question:
I have a dataset like below –
List((X,Set(" 1", " 7")), (Z,Set(" 5")), (D,Set(" 2")), (E,Set(" 8")), ("F ",Set(" 5", " 9", " 108")), (G,Set(" 2", " 11")), (A,Set(" 7", " 5")), (M,Set(108)))
Here X is related to A as 7 is common between them
Z is related to A as 5 is common between them
F is related to A as 5 is common between them
M is related to F as 108 is common between them
So, X, Z, A, F and M are related
D and G are related as 2 is common between them
E is not related to anybody
So, the output would be ((X, Z, A, F, M), (D,G), (E))
Order doesn’t matter here.
I have used Scala here, but solution in Scala/Python or a pseudocode would work for me.
Answers:
- Read input, creating a vector of pairs
e.g.
X 1
X 7
Z 5
...
- Sort the vector in order of the second member of the pairs
e.g
X 1
D 2
G 2
...
- Iterate over sorted vector, adding to a "pass1 group" so long as the second member does not change. If it does change, start a new pass1 group.
e.g.
X
D G
Z F A
X A
E
F
G
- merge pass1 groups with common members to give the output groups.
Here is the C++ code that implements this
#include <string>
#include <iostream>
#include <vector>
#include <algorithm>
bool merge(
std::vector<char> &res,
std::vector<char> &vg)
{
bool ret = false;
for (char r : res)
{
for (char c : vg)
{
if (c == r)
ret = true;
}
}
if (!ret)
return false;
for (char c : vg)
{
if (std::find(res.begin(), res.end(), c) == res.end())
res.push_back(c);
}
return true;
}
void add(
std::vector<std::vector<char>> &result,
std::vector<char> &vg)
{
std::vector<char> row;
for (char c : vg)
row.push_back(c);
result.push_back(row);
}
main()
{
std::string input = "List((X,Set(" 1", " 7")), (Z,Set(" 5")), (D,Set(" 2")), (E,Set(" 8")), (F,Set(" 5", " 9", " 108")), (G,Set(" 2", " 11")), (A,Set(" 7", " 5")), (M,Set("108")))";
input = "List((A,Set("0", "1")),(B,Set("1", "2")),(C,Set("2", "3")),(D,Set("3", "4")))";
std::vector<std::pair<char, int>> vinp;
int p = input.find("Set");
int q = input.find("Set", p + 1);
while (p != -1)
{
char c = input[p - 2];
int s = input.find_first_of("0123456789", p);
if( s == -1 )
break;
while (s < q)
{
vinp.push_back(std::make_pair(
c,
atoi(input.substr(s).c_str())));
s = input.find_first_of("0123456789", s + 3);
if( s == -1 )
break;
}
p = q;
q = input.find("Set", p + 1);
if( q == -1 )
q = input.length();
}
std::sort(vinp.begin(), vinp.end(),
[](std::pair<char, int> a, std::pair<char, int> b)
{
return a.second < b.second;
});
std::cout << "sortedn";
for (auto &p : vinp)
std::cout << p.first << " " << p.second << "n";
std::vector<std::vector<char>> vpass1;
std::vector<char> row;
int sec = -1;
for (auto &p : vinp)
{
if (p.second != sec)
{
// new group
if (row.size())
vpass1.push_back(row);
sec = p.second;
row.clear();
}
row.push_back(p.first);
}
std::cout << "npass1n";
for (auto &row : vpass1)
{
for (char c : row)
std::cout << c << " ";
std::cout << "n";
}
std::vector<std::vector<char>> result;
std::vector<char> pass2group;
bool fmerge2 = true;
while (fmerge2)
{
fmerge2 = false;
for (auto &vg : vpass1)
{
if (!result.size())
add(result, vg);
else
{
bool fmerge1 = false;
for (auto &res : result)
{
if (merge(res, vg))
{
fmerge1 = true;
fmerge2 = true;
break;
}
}
if (!fmerge1)
add(result, vg);
}
}
if (fmerge2)
{
vpass1 = result;
result.clear();
}
}
std::cout << "n(";
for (auto &res : result)
{
if (res.size())
{
std::cout << "(";
for (char c : res)
std::cout << c << " ";
std::cout << ")";
}
}
std::cout << ")n";
return 0;
}
It produces the correct result
((X A Z F M )(D G )(E ))
Build an undirected graph where each label is connected to each number from the corresponding set (i.e. (A, { 1, 2 }) would give two edges: A <-> 1 and A <-> 2)
Compute the connected components (using depth-first search, for example).
Filter out only the labels from the connected components.
import util.{Left, Right, Either}
import collection.mutable
def connectedComponentsOfAsc[F, V](faces: List[(F, Set[V])]): List[List[F]] = {
type Node = Either[F, V]
val graphBuilder = mutable.HashMap.empty[Node, mutable.HashSet[Node]]
def addEdge(a: Node, b: Node): Unit =
graphBuilder.getOrElseUpdate(a, mutable.HashSet.empty[Node]) += b
for
(faceLabel, vertices) <- faces
vertex <- vertices
do
val faceNode = Left(faceLabel)
val vertexNode = Right(vertex)
addEdge(faceNode, vertexNode)
addEdge(vertexNode, faceNode)
val graph = graphBuilder.view.mapValues(_.toSet).toMap
val ccs = connectedComponents(graph)
ccs.map(_.collect { case Left(faceLabel) => faceLabel }.toList)
}
def connectedComponents[V](undirectedGraph: Map[V, Set[V]]): List[Set[V]] = {
val visited = mutable.HashSet.empty[V]
var connectedComponent = mutable.HashSet.empty[V]
val components = mutable.ListBuffer.empty[Set[V]]
def dfs(curr: V): Unit = {
if !visited(curr) then
visited += curr
connectedComponent += curr
undirectedGraph(curr).foreach(dfs)
}
for v <- undirectedGraph.keys do
if !visited(v) then
connectedComponent = mutable.HashSet.empty[V]
dfs(v)
components += connectedComponent.toSet
components.toList
}
Can be used like this:
@main def main(): Unit = {
println(connectedComponentsOfAsc(
List(
("X",Set("1", "7")),
("Z",Set("5")),
("D",Set("2")),
("E",Set("8")),
("F",Set("5", "9", "108")),
("G",Set("2", "11")),
("A",Set("7", "5")),
("M",Set("108"))
)
).map(_.sorted).sortBy(_.toString))
}
Produces:
List(List(A, F, M, X, Z), List(D, G), List(E))
All steps are O(n) (scales linearly with the size of input).
This answer is self-contained, but using some kind of graph-library would be clearly advantageous here.
// I put some values in quotes so we have consistent string input
val initialData :List[(String, Set[String])] = List(
("X",Set(" 1", " 7")),
("Z",Set(" 5")),
("D",Set(" 2")),
("E",Set(" 8")),
("F ",Set(" 5", " 9", " 108")),
("G",Set(" 2", " 11")),
("A",Set(" 7", " 5")),
("M",Set("108"))
)
// Clean up the Sets by turning the string data inside the sets into Ints.
val cleanedData = initialData.map(elem => (elem._1, elem._2.map(_.trim.toInt)))
> cleanedData: List[(String, scala.collection.immutable.Set[Int])] = List((X,Set(1, 7)), (Z,Set(5)), (D,Set(2)), (E,Set(8)), ("F ",Set(5, 9, 108)), (G,Set(2, 11)), (A,Set(7, 5)), (M,Set(108)))
// Explode the Sets into a list of simple mappings. X -> 1, X -> 7 individually.
val explodedList = cleanedData.flatMap(x => x._2.map(v => (x._1, v)))
> explodedList: List[(String, Int)] = List((X,1), (X,7), (Z,5), (D,2), (E,8), ("F ",5), ("F ",9), ("F ",108), (G,2), (G,11), (A,7), (A,5), (M,108))
Group them together by the new key
val mappings = explodedList.groupBy(_._2)
> mappings: scala.collection.immutable.Map[Int,List[(String, Int)]] = Map(5 -> List((Z,5), ("F ",5), (A,5)), 1 -> List((X,1)), 9 -> List(("F ",9)), 2 -> List((D,2), (G,2)), 7 -> List((X,7), (A,7)), 108 -> List(("F ",108), (M,108)), 11 -> List((G,11)), 8 -> List((E,8)))
Print the output
mappings.foreach { case (key, items) =>
println(s"${items.map(_._1).mkString(",")} are all related because of $key")
}
> Z,F ,A are all related because of 5
> X are all related because of 1
> F are all related because of 9
> D,G are all related because of 2
> X,A are all related because of 7
> F ,M are all related because of 108
> G are all related because of 11
> E are all related because of 8
Ultimately using a simpler solution in python as below:
data=[
["X",{"1", "7"}],
["Z",{"5",}],
["D",{"2",}],
["E",{"8",}],
["F",{"5", "9", "108"}],
["G",{"2", "11"}],
["A",{"7", "5"}],
["M",{"108"}]
]
for i in range(len(data)):
for j in range(len(data)):
if(data[i][1].intersection(data[j][1])):
if(data[i][0]!=data[j][0] ):
data[i][1] = data[j][1] = (data[i][1]).union(data[j][1])
for k, g in groupby(sorted([[sorted(tuple(d[1])),d[0]] for d in data]), key=lambda x: x[0]):
print(list(l[1] for l in g))
Getting output as :
['A', 'F', 'M', 'X', 'Z']
['D', 'G']
['E']
Tested for few more datasets and it seems to be working fine.
I have a dataset like below –
List((X,Set(" 1", " 7")), (Z,Set(" 5")), (D,Set(" 2")), (E,Set(" 8")), ("F ",Set(" 5", " 9", " 108")), (G,Set(" 2", " 11")), (A,Set(" 7", " 5")), (M,Set(108)))
Here X is related to A as 7 is common between them
Z is related to A as 5 is common between them
F is related to A as 5 is common between them
M is related to F as 108 is common between them
So, X, Z, A, F and M are related
D and G are related as 2 is common between them
E is not related to anybody
So, the output would be ((X, Z, A, F, M), (D,G), (E))
Order doesn’t matter here.
I have used Scala here, but solution in Scala/Python or a pseudocode would work for me.
- Read input, creating a vector of pairs
e.g.
X 1
X 7
Z 5
...
- Sort the vector in order of the second member of the pairs
e.g
X 1
D 2
G 2
...
- Iterate over sorted vector, adding to a "pass1 group" so long as the second member does not change. If it does change, start a new pass1 group.
e.g.
X
D G
Z F A
X A
E
F
G
- merge pass1 groups with common members to give the output groups.
Here is the C++ code that implements this
#include <string>
#include <iostream>
#include <vector>
#include <algorithm>
bool merge(
std::vector<char> &res,
std::vector<char> &vg)
{
bool ret = false;
for (char r : res)
{
for (char c : vg)
{
if (c == r)
ret = true;
}
}
if (!ret)
return false;
for (char c : vg)
{
if (std::find(res.begin(), res.end(), c) == res.end())
res.push_back(c);
}
return true;
}
void add(
std::vector<std::vector<char>> &result,
std::vector<char> &vg)
{
std::vector<char> row;
for (char c : vg)
row.push_back(c);
result.push_back(row);
}
main()
{
std::string input = "List((X,Set(" 1", " 7")), (Z,Set(" 5")), (D,Set(" 2")), (E,Set(" 8")), (F,Set(" 5", " 9", " 108")), (G,Set(" 2", " 11")), (A,Set(" 7", " 5")), (M,Set("108")))";
input = "List((A,Set("0", "1")),(B,Set("1", "2")),(C,Set("2", "3")),(D,Set("3", "4")))";
std::vector<std::pair<char, int>> vinp;
int p = input.find("Set");
int q = input.find("Set", p + 1);
while (p != -1)
{
char c = input[p - 2];
int s = input.find_first_of("0123456789", p);
if( s == -1 )
break;
while (s < q)
{
vinp.push_back(std::make_pair(
c,
atoi(input.substr(s).c_str())));
s = input.find_first_of("0123456789", s + 3);
if( s == -1 )
break;
}
p = q;
q = input.find("Set", p + 1);
if( q == -1 )
q = input.length();
}
std::sort(vinp.begin(), vinp.end(),
[](std::pair<char, int> a, std::pair<char, int> b)
{
return a.second < b.second;
});
std::cout << "sortedn";
for (auto &p : vinp)
std::cout << p.first << " " << p.second << "n";
std::vector<std::vector<char>> vpass1;
std::vector<char> row;
int sec = -1;
for (auto &p : vinp)
{
if (p.second != sec)
{
// new group
if (row.size())
vpass1.push_back(row);
sec = p.second;
row.clear();
}
row.push_back(p.first);
}
std::cout << "npass1n";
for (auto &row : vpass1)
{
for (char c : row)
std::cout << c << " ";
std::cout << "n";
}
std::vector<std::vector<char>> result;
std::vector<char> pass2group;
bool fmerge2 = true;
while (fmerge2)
{
fmerge2 = false;
for (auto &vg : vpass1)
{
if (!result.size())
add(result, vg);
else
{
bool fmerge1 = false;
for (auto &res : result)
{
if (merge(res, vg))
{
fmerge1 = true;
fmerge2 = true;
break;
}
}
if (!fmerge1)
add(result, vg);
}
}
if (fmerge2)
{
vpass1 = result;
result.clear();
}
}
std::cout << "n(";
for (auto &res : result)
{
if (res.size())
{
std::cout << "(";
for (char c : res)
std::cout << c << " ";
std::cout << ")";
}
}
std::cout << ")n";
return 0;
}
It produces the correct result
((X A Z F M )(D G )(E ))
Build an undirected graph where each label is connected to each number from the corresponding set (i.e. (A, { 1, 2 }) would give two edges: A <-> 1 and A <-> 2)
Compute the connected components (using depth-first search, for example).
Filter out only the labels from the connected components.
import util.{Left, Right, Either}
import collection.mutable
def connectedComponentsOfAsc[F, V](faces: List[(F, Set[V])]): List[List[F]] = {
type Node = Either[F, V]
val graphBuilder = mutable.HashMap.empty[Node, mutable.HashSet[Node]]
def addEdge(a: Node, b: Node): Unit =
graphBuilder.getOrElseUpdate(a, mutable.HashSet.empty[Node]) += b
for
(faceLabel, vertices) <- faces
vertex <- vertices
do
val faceNode = Left(faceLabel)
val vertexNode = Right(vertex)
addEdge(faceNode, vertexNode)
addEdge(vertexNode, faceNode)
val graph = graphBuilder.view.mapValues(_.toSet).toMap
val ccs = connectedComponents(graph)
ccs.map(_.collect { case Left(faceLabel) => faceLabel }.toList)
}
def connectedComponents[V](undirectedGraph: Map[V, Set[V]]): List[Set[V]] = {
val visited = mutable.HashSet.empty[V]
var connectedComponent = mutable.HashSet.empty[V]
val components = mutable.ListBuffer.empty[Set[V]]
def dfs(curr: V): Unit = {
if !visited(curr) then
visited += curr
connectedComponent += curr
undirectedGraph(curr).foreach(dfs)
}
for v <- undirectedGraph.keys do
if !visited(v) then
connectedComponent = mutable.HashSet.empty[V]
dfs(v)
components += connectedComponent.toSet
components.toList
}
Can be used like this:
@main def main(): Unit = {
println(connectedComponentsOfAsc(
List(
("X",Set("1", "7")),
("Z",Set("5")),
("D",Set("2")),
("E",Set("8")),
("F",Set("5", "9", "108")),
("G",Set("2", "11")),
("A",Set("7", "5")),
("M",Set("108"))
)
).map(_.sorted).sortBy(_.toString))
}
Produces:
List(List(A, F, M, X, Z), List(D, G), List(E))
All steps are O(n) (scales linearly with the size of input).
This answer is self-contained, but using some kind of graph-library would be clearly advantageous here.
// I put some values in quotes so we have consistent string input
val initialData :List[(String, Set[String])] = List(
("X",Set(" 1", " 7")),
("Z",Set(" 5")),
("D",Set(" 2")),
("E",Set(" 8")),
("F ",Set(" 5", " 9", " 108")),
("G",Set(" 2", " 11")),
("A",Set(" 7", " 5")),
("M",Set("108"))
)
// Clean up the Sets by turning the string data inside the sets into Ints.
val cleanedData = initialData.map(elem => (elem._1, elem._2.map(_.trim.toInt)))
> cleanedData: List[(String, scala.collection.immutable.Set[Int])] = List((X,Set(1, 7)), (Z,Set(5)), (D,Set(2)), (E,Set(8)), ("F ",Set(5, 9, 108)), (G,Set(2, 11)), (A,Set(7, 5)), (M,Set(108)))
// Explode the Sets into a list of simple mappings. X -> 1, X -> 7 individually.
val explodedList = cleanedData.flatMap(x => x._2.map(v => (x._1, v)))
> explodedList: List[(String, Int)] = List((X,1), (X,7), (Z,5), (D,2), (E,8), ("F ",5), ("F ",9), ("F ",108), (G,2), (G,11), (A,7), (A,5), (M,108))
Group them together by the new key
val mappings = explodedList.groupBy(_._2)
> mappings: scala.collection.immutable.Map[Int,List[(String, Int)]] = Map(5 -> List((Z,5), ("F ",5), (A,5)), 1 -> List((X,1)), 9 -> List(("F ",9)), 2 -> List((D,2), (G,2)), 7 -> List((X,7), (A,7)), 108 -> List(("F ",108), (M,108)), 11 -> List((G,11)), 8 -> List((E,8)))
Print the output
mappings.foreach { case (key, items) =>
println(s"${items.map(_._1).mkString(",")} are all related because of $key")
}
> Z,F ,A are all related because of 5
> X are all related because of 1
> F are all related because of 9
> D,G are all related because of 2
> X,A are all related because of 7
> F ,M are all related because of 108
> G are all related because of 11
> E are all related because of 8
Ultimately using a simpler solution in python as below:
data=[
["X",{"1", "7"}],
["Z",{"5",}],
["D",{"2",}],
["E",{"8",}],
["F",{"5", "9", "108"}],
["G",{"2", "11"}],
["A",{"7", "5"}],
["M",{"108"}]
]
for i in range(len(data)):
for j in range(len(data)):
if(data[i][1].intersection(data[j][1])):
if(data[i][0]!=data[j][0] ):
data[i][1] = data[j][1] = (data[i][1]).union(data[j][1])
for k, g in groupby(sorted([[sorted(tuple(d[1])),d[0]] for d in data]), key=lambda x: x[0]):
print(list(l[1] for l in g))
Getting output as :
['A', 'F', 'M', 'X', 'Z']
['D', 'G']
['E']
Tested for few more datasets and it seems to be working fine.