Set iteration order varies from run to run
Question:
Why does the iteration order of a Python set (with the same contents) vary from run to run, and what are my options for making it consistent from run to run?
I understand that the iteration order for a Python set is arbitrary. If I put ‘a’, ‘b’, and ‘c’ into a set and then iterate them, they may come back out in any order.
What I’ve observed is that the order remains the same within a run of the program. That is, if my program iterates the same set twice in a row, I get the same order both times. However, if I run the program twice in a row, the order changes from run to run.
Unfortunately, this breaks one of my automated tests, which simply compares the output from two runs of my program. I don’t care about the actual order, but I would like it to be consistent from run to run.
The best solution I’ve come up with is:
- Copy the set to a list.
- Apply an arbitrary sort to the list.
- Iterate the list instead of the set.
Is there a simpler solution?
Note: I’ve found similar questions on StackOverlow, but none that address this specific issue of getting the same results from run to run.
Answers:
What you want isn’t possible. Arbitrary means arbitrary.
My solution would be the same as yours, you have to sort the set if you want to be able to compare it to another one.
Contrary to sets, lists have always a guaranteed order, so you could toss the set and use the list.
The set’s iteration order depends not only its contents, but on the order in which the items were inserted into the set, and whether there were deletions along the way. So you can create two different sets, using different insertions and deletions, and end up with the same set at the end, but with different iteration orders.
As others have said: if you care about the order of the set, you have to create a sorted list from it.
Use the symmetric_difference (^) operator on your two sets to see if there are any differences:
In [1]: s1 = set([5,7,8,2,1,9,0])
In [2]: s2 = set([9,0,5,1,8,2,7])
In [3]: s1
Out[3]: set([0, 1, 2, 5, 7, 8, 9])
In [4]: s2
Out[4]: set([0, 1, 2, 5, 7, 8, 9])
In [5]: s1 ^ s2
Out[5]: set()
Your question transformed into two questions: A) how to compare "the output of two runs" in your specific case; B) what’s the definition of the iteration order in a set. Maybe you should distinguish them and post B) as a new question if appropriate. I’ll answer A.
IMHO, using a sorted list in your case is not a very clean solution. You should decide whether you care for iteration order once and for all and use the appropriate structure.
Either 1) you want to compare the two sets to see if they have equal content, irrespective of the order. Then the simple == operator on sets seems appropriate. See python2 sets, python3 sets.
Or 2) you want to check whether the elements were inserted in the same order. But this seems reasonable only if the order of insertion somehow matters to the users of your library, in which case using the set type was probably inappropriate to begin with. Put another way, it is unclear what you mean exactly by "comparing the output of two runs" and why you want to do that.
In all cases, I doubt a sorted list is appropriate here.
The reason the set iteration order changes from run-to-run appears to be because Python uses hash seed randomization by default. (See command option -R.) Thus set iteration is not only arbitrary (because of hashing), but also non-deterministic (because of the random seed).
You can override the random seed with a fixed value by setting the environment variable PYTHONHASHSEED for the interpreter. Using the same seed from run to run means set iteration is still arbitrary, but now it is deterministic, which was the desired property.
Hash seed randomization is a security measure to make it difficult for an adversary to feed inputs that will cause pathological behavior (e.g., by creating numerous hash collisions). For unit testing, this is not a concern, so it’s reasonable to override the hash seed while running tests.
You can set the expected result to be also a set. And checks if those two sets are equal using ==.
Why does the iteration order of a Python set (with the same contents) vary from run to run, and what are my options for making it consistent from run to run?
I understand that the iteration order for a Python set is arbitrary. If I put ‘a’, ‘b’, and ‘c’ into a set and then iterate them, they may come back out in any order.
What I’ve observed is that the order remains the same within a run of the program. That is, if my program iterates the same set twice in a row, I get the same order both times. However, if I run the program twice in a row, the order changes from run to run.
Unfortunately, this breaks one of my automated tests, which simply compares the output from two runs of my program. I don’t care about the actual order, but I would like it to be consistent from run to run.
The best solution I’ve come up with is:
- Copy the set to a list.
- Apply an arbitrary sort to the list.
- Iterate the list instead of the set.
Is there a simpler solution?
Note: I’ve found similar questions on StackOverlow, but none that address this specific issue of getting the same results from run to run.
What you want isn’t possible. Arbitrary means arbitrary.
My solution would be the same as yours, you have to sort the set if you want to be able to compare it to another one.
Contrary to sets, lists have always a guaranteed order, so you could toss the set and use the list.
The set’s iteration order depends not only its contents, but on the order in which the items were inserted into the set, and whether there were deletions along the way. So you can create two different sets, using different insertions and deletions, and end up with the same set at the end, but with different iteration orders.
As others have said: if you care about the order of the set, you have to create a sorted list from it.
Use the symmetric_difference (^) operator on your two sets to see if there are any differences:
In [1]: s1 = set([5,7,8,2,1,9,0])
In [2]: s2 = set([9,0,5,1,8,2,7])
In [3]: s1
Out[3]: set([0, 1, 2, 5, 7, 8, 9])
In [4]: s2
Out[4]: set([0, 1, 2, 5, 7, 8, 9])
In [5]: s1 ^ s2
Out[5]: set()
Your question transformed into two questions: A) how to compare "the output of two runs" in your specific case; B) what’s the definition of the iteration order in a set. Maybe you should distinguish them and post B) as a new question if appropriate. I’ll answer A.
IMHO, using a sorted list in your case is not a very clean solution. You should decide whether you care for iteration order once and for all and use the appropriate structure.
Either 1) you want to compare the two sets to see if they have equal content, irrespective of the order. Then the simple == operator on sets seems appropriate. See python2 sets, python3 sets.
Or 2) you want to check whether the elements were inserted in the same order. But this seems reasonable only if the order of insertion somehow matters to the users of your library, in which case using the set type was probably inappropriate to begin with. Put another way, it is unclear what you mean exactly by "comparing the output of two runs" and why you want to do that.
In all cases, I doubt a sorted list is appropriate here.
The reason the set iteration order changes from run-to-run appears to be because Python uses hash seed randomization by default. (See command option -R.) Thus set iteration is not only arbitrary (because of hashing), but also non-deterministic (because of the random seed).
You can override the random seed with a fixed value by setting the environment variable PYTHONHASHSEED for the interpreter. Using the same seed from run to run means set iteration is still arbitrary, but now it is deterministic, which was the desired property.
Hash seed randomization is a security measure to make it difficult for an adversary to feed inputs that will cause pathological behavior (e.g., by creating numerous hash collisions). For unit testing, this is not a concern, so it’s reasonable to override the hash seed while running tests.
You can set the expected result to be also a set. And checks if those two sets are equal using ==.