in 1 hash get old value from dictionary and update it
Question:
i wonder if there is a way to read the content of an element in the dictionary and update the dictionary in a single hash in python ?, in C++ this is possible using the iterator returned by map.find
, but i wonder if there is a similar method in python (using a python extension is also okay if that’s the only choice)
this happens to be the problem that i am currently facing, and it’s a tight loop and a lot of time is spent hashing the key.
this is how it would be written in python, but this does two hashes, 1 for reading the value of dict_obj[key]
and another for updating the dictionary.
my_dict = {"A":5}
def read_and_update(dict_obj,key,new_value):
old_value = dict_obj[key]
dict_obj[key] = new_value
return old_value
old_value = read_and_update(my_dict,"A",3)
print(old_value)
print(my_dict)
5
{'A': 3}
all i want is to obtain the value of "A" from the dictionary (in here it is 5
) and update the dictionary so that my_dict["A"]
would give back 3
, while doing only 1 hash.
getting the old value of the element in the dictionary is important for signaling deletion and logging its deletion later on.
Edit: using the hash of the key as the key to reduce hash overhead is also not possible as it will break down on hash collision.
Answers:
answering myself for future readers, in short it’s not possible, but there are ways to improve the performance.
-
using strings as keys makes python cache the hash of the key string, so asking the dictionary for the same string key twice only involves one hash.
-
if your key isn’t a string you might want to store the hash yourself, the trick is to rely on __slots__
to improve your class performance, this pattern’s speed has improved with each python version until this moment.
class my_key_obj:
__slots__ = ["entry1","entry2","entry3","hash"]
def __init__(self,entry1,entry2,entry3):
self.entry1 = entry1
self.entry2 = entry2
self.entry3 = entry3
self.hash = hash((self.entry1,self.entry2,self.entry3))
def __hash__(self):
return self.hash
def __eq__(self, other):
return self.entry1 == other.entry1 and
self.entry2 == other.entry2 and
self.entry3 == other.entry3
note that you are always computing the hash whether or not you are using it, so you have to use it at least twice to make up for the time lost constructing it, but if you are passing this object around and using it as a key then this requirement should be satisfied.
- it’s possible to easily implement your dictionary as a c++ unordered_map using cython and wrap it in a python object, but python’s dictionaries are faster, so this idea is a bad idea.
i wonder if there is a way to read the content of an element in the dictionary and update the dictionary in a single hash in python ?, in C++ this is possible using the iterator returned by map.find
, but i wonder if there is a similar method in python (using a python extension is also okay if that’s the only choice)
this happens to be the problem that i am currently facing, and it’s a tight loop and a lot of time is spent hashing the key.
this is how it would be written in python, but this does two hashes, 1 for reading the value of dict_obj[key]
and another for updating the dictionary.
my_dict = {"A":5}
def read_and_update(dict_obj,key,new_value):
old_value = dict_obj[key]
dict_obj[key] = new_value
return old_value
old_value = read_and_update(my_dict,"A",3)
print(old_value)
print(my_dict)
5
{'A': 3}
all i want is to obtain the value of "A" from the dictionary (in here it is 5
) and update the dictionary so that my_dict["A"]
would give back 3
, while doing only 1 hash.
getting the old value of the element in the dictionary is important for signaling deletion and logging its deletion later on.
Edit: using the hash of the key as the key to reduce hash overhead is also not possible as it will break down on hash collision.
answering myself for future readers, in short it’s not possible, but there are ways to improve the performance.
-
using strings as keys makes python cache the hash of the key string, so asking the dictionary for the same string key twice only involves one hash.
-
if your key isn’t a string you might want to store the hash yourself, the trick is to rely on
__slots__
to improve your class performance, this pattern’s speed has improved with each python version until this moment.
class my_key_obj:
__slots__ = ["entry1","entry2","entry3","hash"]
def __init__(self,entry1,entry2,entry3):
self.entry1 = entry1
self.entry2 = entry2
self.entry3 = entry3
self.hash = hash((self.entry1,self.entry2,self.entry3))
def __hash__(self):
return self.hash
def __eq__(self, other):
return self.entry1 == other.entry1 and
self.entry2 == other.entry2 and
self.entry3 == other.entry3
note that you are always computing the hash whether or not you are using it, so you have to use it at least twice to make up for the time lost constructing it, but if you are passing this object around and using it as a key then this requirement should be satisfied.
- it’s possible to easily implement your dictionary as a c++ unordered_map using cython and wrap it in a python object, but python’s dictionaries are faster, so this idea is a bad idea.