How to write a python script to remove duplicate ip address or duplicate subnet and remove the overlaps ip address or ip_subnet

Question

I have a list,say,

ip_related_list = ['192.168.1.1', '192.168.1.2', '192.168.1.0/24', '192.168.0.0/16', '10.1.1.1', '10.1.1.1', '10.1.1.1', '10.1.1.2','10.10.0.0/16','10.20.0.0/16','10.10.0.0/16'],

The expected results should be :

192.168.0.0/16,
10.10.0.0/16, 
10.20.0.0/16,
10.1.1.1,
10.1.1.2

‘192.168.1.1’, ‘192.168.1.2’ and ‘192.168.1.0/24’ are within subnet ‘192.168.0.0/16’. So only the subnet is taken.

Asked By: marty

||

Source

Answer 1

Use netaddr.cird_merge operation to summarize groups of IP subnets and addresses:

import netaddr

ip_related_list = ['192.168.1.1', '192.168.1.2', '192.168.1.0/24', '192.168.0.0/16', '10.1.1.1', '10.1.1.1', '10.1.1.1',
                   '10.1.1.2','10.10.0.0/16','10.20.0.0/16','10.10.0.0/16']
merged_subnets = netaddr.cidr_merge(ip_related_list)
print(merged_subnets)

[IPNetwork('10.1.1.1/32'), IPNetwork('10.1.1.2/32'), IPNetwork('10.10.0.0/16'), IPNetwork('10.20.0.0/16'), IPNetwork('192.168.0.0/16')]

To get string representation you can do:

merged_subnets = list(map(str, merged_subnets))

['10.1.1.1/32', '10.1.1.2/32', '10.10.0.0/16', '10.20.0.0/16', '192.168.0.0/16']

Answered By: RomanPerekhrest

Answer 2

Here is another solution:

from ipaddress import ip_network

ip_related_list = ['192.168.1.1', '192.168.1.2', '192.168.1.0/24', '192.168.0.0/16', '10.1.1.1', '10.1.1.1', '10.1.1.1', '10.1.1.2','10.10.0.0/16','10.20.0.0/16','10.10.0.0/16']

output = []
for network in ip_related_list:
    if not any([set(ip_network(network).hosts()) <= set(ip_network(i).hosts()) for i in output]):
        output = [i for i in output if not set(ip_network(i).hosts()) <= set(ip_network(network).hosts())]
        output.append(network)

print(output)

Answered By: Celdor

How to write a python script to remove duplicate ip address or duplicate subnet and remove the overlaps ip address or ip_subnet

Question:

Answers: