How to sort IP addresses stored in dictionary in Python?
Question:
I have a piece of code that looks like this:
ipCount = defaultdict(int)
for logLine in logLines:
date, serverIp, clientIp = logLine.split(" ")
ipCount[clientIp] += 1
for clientIp, hitCount in sorted(ipCount.items(), key=operator.itemgetter(0)):
print(clientIp)
and it kind of sorts IP’s, but like this:
192.168.102.105
192.168.204.111
192.168.99.11
which is not good enough since it does not recognize that 99 is a smaller number than 102 or 204. I would like the output to be like this:
192.168.99.11
192.168.102.105
192.168.204.111
I found this, but I am not sure how to implement it in my code, or if it is even possible since I use dictionary. What are my options here?
Answers:
What are my options here?
The two obvious one that come to my mind are:
- Preformatting the strings with the IP when you store them as from the link you put in your question.
- Pass a sorting function to
sorted() function when you perform the ordering.
Which is best depends from the amount of data you have to process (you will notice an increased performance for method #1 only for very large amount of data) and from what you will need to do with said sorted list of IP (if you preformat the strings, you might then need to change them again before feeding them as arguments to other functions, for example).
Example of preformatting
Maintain the IP as a string, but uses spaces or zeroes to solve the variable number of digits problem:
>>> ip = '192.168.1.1'
>>> print('%3s.%3s.%3s.%3s' % tuple(ip.split('.')))
192.168. 1. 1
>>> print('%s.%s.%s.%s' % tuple([s.zfill(3) for s in ip.split('.')]))
192.168.001.001
Example of sorting function
Well… Ferdinand Beyer in his answer seems to have already offered an excellent solution for this approach! 🙂
Use the key parameter of sorted to convert your ip to an integer, for example:
list_of_ips = ['192.168.204.111', '192.168.99.11', '192.168.102.105']
sorted(list_of_ips, key=lambda ip: long(''.join(["%02X" % long(i) for i in ip.split('.')]), 16))
EDIT:
Gryphius proposes a solution with the socket module, and so why not use it to make the conversion from ip to long as it is cleaner:
from socket import inet_aton
import struct
list_of_ips = ['192.168.204.111', '192.168.99.11', '192.168.102.105']
sorted(list_of_ips, key=lambda ip: struct.unpack("!L", inet_aton(ip))[0])
You can use a custom key function to return a sortable representation of your strings:
def split_ip(ip):
"""Split a IP address given as string into a 4-tuple of integers."""
return tuple(int(part) for part in ip.split('.'))
def my_key(item):
return split_ip(item[0])
items = sorted(ipCount.items(), key=my_key)
The split_ip() function takes an IP address string like '192.168.102.105' and turns it into a tuple of integers (192, 168, 102, 105). Python has built-in support to sort tuples lexicographically.
UPDATE: This can actually be done even easier using the inet_aton() function in the socket module:
import socket
items = sorted(ipCount.items(), key=lambda item: socket.inet_aton(item[0]))
I think this will help you: PEP265 (sorting dictionieries by value). Just extend the sorted function.
if your application does lots of things like “find ips in range x”, “sort by ip” etc its often more convenient to store the numeric value of the ip internally and work with this one.
from socket import inet_aton,inet_ntoa
import struct
def ip2long(ip):
packed = inet_aton(ip)
lng = struct.unpack("!L", packed)[0]
return lng
convert the number back into an ip using this function:
def long2ip(lng):
packed = struct.pack("!L", lng)
ip=inet_ntoa(packed)
return ip
>>> ip2long('192.168.1.1')
3232235777
>>> ip2long('1.2.3.4')
16909060
>>> long2ip(3232235777)
'192.168.1.1'
>>> long2ip(16909060)
'1.2.3.4'
how about not working with strings at all and instead convert each octet into integer, then passing it into 4 dimensional dictionary?
ClientIps[192][168][102][105]=1
ClientIps[192][168][99][11]=1
then it is easy to just sort an array by key, isnt it?
for key1, value in sorted(ClientIps.items()):
for key2, value in sorted(ClientIps[key1].items()):
for key3, value in sorted(ClientIps[key1][key2].items()):
for key4, value in sorted(ClientIps[key][key2][key3].items()):
print(key1, key2, key3, key4)
for speed reasons it may be beneficial to also compare simple python dictionary against OrderedDict .
A clean way of handling the right order is using Pythons ipaddress module. You can transform the Strings into IPv4Address representations and sort them afterwards. Here’s a working example with list objects (Tested with Python3):
import ipaddress
unsorted_list = [
'192.168.102.105',
'192.168.204.111',
'192.168.99.11'
]
new_list = []
for element in unsorted_list:
new_list.append(ipaddress.ip_address(element))
new_list.sort()
# [IPv4Address('192.168.99.11'), IPv4Address('192.168.102.105'), IPv4Address('192.168.204.111')]
print(new_list)
Found a solution at https://www.lesinskis.com/python_sorting_IP_addresses.html
All you have to do is convert the string of ip in ipaddress
import ipaddress
sortedkey = sorted(list_of_ip_instring, key = ipaddress.IPv4Address)
in python 3
use like this:
import ipaddress
clientIp = sorted(clientIp, key=ipaddress.IPv4Address)
for ip in clientIp:
print(ip)
and when IP addresses are Classless Inter-Domain Routing (CIDR) use:
import ipaddress
clientIp = sorted(clientIp, key=ipaddress.IPv4Network)
for ip in clientIp:
print(ip)
If you want to sort a list of IPv4/6 address strings, the correct and simplest solution, for python3 at least, is to convert to ipaddress objects and sort with ipaddress.get_mixed_type_key.
import ipaddress
unsorted_ips = [
"1.1.1.1",
"1.0.0.1",
"2606:4700:4700::1111",
"2606:4700:4700::1001",
]
sorted_ips = sorted(
[ipaddress.ip_address(ip) for ip in unsorted_ips], key=ipaddress.get_mixed_type_key
)
print(sorted_ips)
I have a piece of code that looks like this:
ipCount = defaultdict(int)
for logLine in logLines:
date, serverIp, clientIp = logLine.split(" ")
ipCount[clientIp] += 1
for clientIp, hitCount in sorted(ipCount.items(), key=operator.itemgetter(0)):
print(clientIp)
and it kind of sorts IP’s, but like this:
192.168.102.105
192.168.204.111
192.168.99.11
which is not good enough since it does not recognize that 99 is a smaller number than 102 or 204. I would like the output to be like this:
192.168.99.11
192.168.102.105
192.168.204.111
I found this, but I am not sure how to implement it in my code, or if it is even possible since I use dictionary. What are my options here?
What are my options here?
The two obvious one that come to my mind are:
- Preformatting the strings with the IP when you store them as from the link you put in your question.
- Pass a sorting function to
sorted()function when you perform the ordering.
Which is best depends from the amount of data you have to process (you will notice an increased performance for method #1 only for very large amount of data) and from what you will need to do with said sorted list of IP (if you preformat the strings, you might then need to change them again before feeding them as arguments to other functions, for example).
Example of preformatting
Maintain the IP as a string, but uses spaces or zeroes to solve the variable number of digits problem:
>>> ip = '192.168.1.1'
>>> print('%3s.%3s.%3s.%3s' % tuple(ip.split('.')))
192.168. 1. 1
>>> print('%s.%s.%s.%s' % tuple([s.zfill(3) for s in ip.split('.')]))
192.168.001.001
Example of sorting function
Well… Ferdinand Beyer in his answer seems to have already offered an excellent solution for this approach! 🙂
Use the key parameter of sorted to convert your ip to an integer, for example:
list_of_ips = ['192.168.204.111', '192.168.99.11', '192.168.102.105']
sorted(list_of_ips, key=lambda ip: long(''.join(["%02X" % long(i) for i in ip.split('.')]), 16))
EDIT:
Gryphius proposes a solution with the socket module, and so why not use it to make the conversion from ip to long as it is cleaner:
from socket import inet_aton
import struct
list_of_ips = ['192.168.204.111', '192.168.99.11', '192.168.102.105']
sorted(list_of_ips, key=lambda ip: struct.unpack("!L", inet_aton(ip))[0])
You can use a custom key function to return a sortable representation of your strings:
def split_ip(ip):
"""Split a IP address given as string into a 4-tuple of integers."""
return tuple(int(part) for part in ip.split('.'))
def my_key(item):
return split_ip(item[0])
items = sorted(ipCount.items(), key=my_key)
The split_ip() function takes an IP address string like '192.168.102.105' and turns it into a tuple of integers (192, 168, 102, 105). Python has built-in support to sort tuples lexicographically.
UPDATE: This can actually be done even easier using the inet_aton() function in the socket module:
import socket
items = sorted(ipCount.items(), key=lambda item: socket.inet_aton(item[0]))
I think this will help you: PEP265 (sorting dictionieries by value). Just extend the sorted function.
if your application does lots of things like “find ips in range x”, “sort by ip” etc its often more convenient to store the numeric value of the ip internally and work with this one.
from socket import inet_aton,inet_ntoa
import struct
def ip2long(ip):
packed = inet_aton(ip)
lng = struct.unpack("!L", packed)[0]
return lng
convert the number back into an ip using this function:
def long2ip(lng):
packed = struct.pack("!L", lng)
ip=inet_ntoa(packed)
return ip
>>> ip2long('192.168.1.1')
3232235777
>>> ip2long('1.2.3.4')
16909060
>>> long2ip(3232235777)
'192.168.1.1'
>>> long2ip(16909060)
'1.2.3.4'
how about not working with strings at all and instead convert each octet into integer, then passing it into 4 dimensional dictionary?
ClientIps[192][168][102][105]=1
ClientIps[192][168][99][11]=1
then it is easy to just sort an array by key, isnt it?
for key1, value in sorted(ClientIps.items()):
for key2, value in sorted(ClientIps[key1].items()):
for key3, value in sorted(ClientIps[key1][key2].items()):
for key4, value in sorted(ClientIps[key][key2][key3].items()):
print(key1, key2, key3, key4)
for speed reasons it may be beneficial to also compare simple python dictionary against OrderedDict .
A clean way of handling the right order is using Pythons ipaddress module. You can transform the Strings into IPv4Address representations and sort them afterwards. Here’s a working example with list objects (Tested with Python3):
import ipaddress
unsorted_list = [
'192.168.102.105',
'192.168.204.111',
'192.168.99.11'
]
new_list = []
for element in unsorted_list:
new_list.append(ipaddress.ip_address(element))
new_list.sort()
# [IPv4Address('192.168.99.11'), IPv4Address('192.168.102.105'), IPv4Address('192.168.204.111')]
print(new_list)
Found a solution at https://www.lesinskis.com/python_sorting_IP_addresses.html
All you have to do is convert the string of ip in ipaddress
import ipaddress
sortedkey = sorted(list_of_ip_instring, key = ipaddress.IPv4Address)
in python 3
use like this:
import ipaddress
clientIp = sorted(clientIp, key=ipaddress.IPv4Address)
for ip in clientIp:
print(ip)
and when IP addresses are Classless Inter-Domain Routing (CIDR) use:
import ipaddress
clientIp = sorted(clientIp, key=ipaddress.IPv4Network)
for ip in clientIp:
print(ip)
If you want to sort a list of IPv4/6 address strings, the correct and simplest solution, for python3 at least, is to convert to ipaddress objects and sort with ipaddress.get_mixed_type_key.
import ipaddress
unsorted_ips = [
"1.1.1.1",
"1.0.0.1",
"2606:4700:4700::1111",
"2606:4700:4700::1001",
]
sorted_ips = sorted(
[ipaddress.ip_address(ip) for ip in unsorted_ips], key=ipaddress.get_mixed_type_key
)
print(sorted_ips)