Python list comprehension – access last created element?
Question:
Is it possible to access the previous element generated in a list comprehension.
I am working on some toy encryption stuff. Given the key as an arbitrarily large integer, an initialization value, and a list of elements as the message to encrypt. I need to xor each element with the previous ciphered element and the key. The following loop would do.
previous = initialization_value
cipher = []
for element in message:
previous = element ^ previous ^ key
cipher.append(previous)
I feel like it should be possible to turn this into a list comprehension but I am not exactly sure how to handle both the initial value or accessing the previous value generated.
Is it possible and if so what would the comprehension be?
Answers:
There isn’t a good, Pythonic way to do this with a list comprehension. The best way to think about list comprehensions is as a replacement for map and filter. In other words, you’d use a list comprehension whenever you need to take a list and
-
Use its elements as input for some expression (e.g. squaring the elements)
-
Remove some of its elements based on some condition
What these things have in common is that they each only look at a single list element at a time. This is a good rule of thumb; even if you could theoretically write the code you showed as a list comprehension, it would be awkward and unpythonic.
You could use a helper object to store all the internal state while iterating over the sequence:
class Encryption:
def __init__(self, key, init_value):
self.key = key
self.previous = init_value
def next(self, element):
self.previous = element ^ self.previous ^ self.key
return self.previous
enc = Encryption(...)
cipher = [enc.next(e) for e in message]
That being said, adding the previously encrypted element into the xor doesn’t make your algorithm any harder to break than just xor’ing every element with the key. An attacker can just xor any character in the cipher text with the previous encrypted character and so cancel out the xor that was done during encryption.
You could have done this using reduce(). It’s not list comprehension, but it’s the functional style approach:
cipher = []
def f(previous, element):
previous = element ^ previous ^ key
cipher.append(previous)
return previous
reduce(f, message, initialization_value)
It isn’t any prettier than the plain loop in this case though.
As a generator:
def cypher(message, key, seed):
for element in message:
seed = element ^ seed ^ key
yield seed
list(cypher(message, key, initial_seed))
I would prefer to use something more like enumerate generator
def emit_previous(iterable, initial=None):
previous = initial
for item in iterable:
yield previous, item
previous = item
cipher = []
for previous, element in emit_previous(message, initial=initialization_value):
seed = element ^ previous ^ key
cipher.append(seed)
Here is an example how to access the last and next created elements in a list comprehension using enumerate.
The previous value…
Here -1 is the offset and None is the default value.
A = ['A','B','C','D','E','F','G','H','I','J']
[(A[i-1] if (i-1) >= 0 else None,x) for i,x in enumerate(A)
Output as tuple (Previous, Current)
[(None, ‘A’),
(‘A’, ‘B’),
(‘B’, ‘C’),
(‘C’, ‘D’),
(‘D’, ‘E’),
(‘E’, ‘F’),
(‘F’, ‘G’),
(‘G’, ‘H’),
(‘H’, ‘I’),
(‘I’, ‘J’)]
The next value…
A = ['A','B','C','D','E','F','G','H','I','J']
[(x, A[i+1] if (i+1) < len(A) else None) for i,x in enumerate(A)]
Output as tuple (Current, Next)
[(‘A’, ‘B’),
(‘B’, ‘C’),
(‘C’, ‘D’),
(‘D’, ‘E’),
(‘E’, ‘F’),
(‘F’, ‘G’),
(‘G’, ‘H’),
(‘H’, ‘I’),
(‘I’, ‘J’),
(‘J’, None)]
This is very similar to the lag() and lead() analytical functions in sql.
So you could append to a list within the list comprehension and use the last added element as the new element for that list comprehension and then finally assigning that comprehension to the list variable.
message = [0,1,2,3,4,5,6,7]
key = 8
previous = 9
cipher = [previous]
cipher = [(cipher.append(element^cipher[-1]^key), cipher[-1])[1] for element in message]
print(cipher)
itertools.accumulate() exists exactly for that:
from itertools import accumulate
result = list(accumulate(message, lambda prev, element: prev^element^key), initial=initialization_value)
del result[0] # Remove the initialization value
A simple one-liner, no need for a comprehension or anything complex.
Since Python 3.8, there is now support for an assignment expression that can be used to implement a list comprehension that remembers the last value in the iteration:
previous = initialization_value
cipher = [previous := element ^ previous ^ key for element in message]
If you prefer to write everything in one line, you can initialize previous in a singleton list, and discard it with the and operator to let the list comprehension take over:
cipher = [previous := initialization_value] and [previous := element ^ previous ^ key for element in message]
Is it possible to access the previous element generated in a list comprehension.
I am working on some toy encryption stuff. Given the key as an arbitrarily large integer, an initialization value, and a list of elements as the message to encrypt. I need to xor each element with the previous ciphered element and the key. The following loop would do.
previous = initialization_value
cipher = []
for element in message:
previous = element ^ previous ^ key
cipher.append(previous)
I feel like it should be possible to turn this into a list comprehension but I am not exactly sure how to handle both the initial value or accessing the previous value generated.
Is it possible and if so what would the comprehension be?
There isn’t a good, Pythonic way to do this with a list comprehension. The best way to think about list comprehensions is as a replacement for map and filter. In other words, you’d use a list comprehension whenever you need to take a list and
-
Use its elements as input for some expression (e.g. squaring the elements)
-
Remove some of its elements based on some condition
What these things have in common is that they each only look at a single list element at a time. This is a good rule of thumb; even if you could theoretically write the code you showed as a list comprehension, it would be awkward and unpythonic.
You could use a helper object to store all the internal state while iterating over the sequence:
class Encryption:
def __init__(self, key, init_value):
self.key = key
self.previous = init_value
def next(self, element):
self.previous = element ^ self.previous ^ self.key
return self.previous
enc = Encryption(...)
cipher = [enc.next(e) for e in message]
That being said, adding the previously encrypted element into the xor doesn’t make your algorithm any harder to break than just xor’ing every element with the key. An attacker can just xor any character in the cipher text with the previous encrypted character and so cancel out the xor that was done during encryption.
You could have done this using reduce(). It’s not list comprehension, but it’s the functional style approach:
cipher = []
def f(previous, element):
previous = element ^ previous ^ key
cipher.append(previous)
return previous
reduce(f, message, initialization_value)
It isn’t any prettier than the plain loop in this case though.
As a generator:
def cypher(message, key, seed):
for element in message:
seed = element ^ seed ^ key
yield seed
list(cypher(message, key, initial_seed))
I would prefer to use something more like enumerate generator
def emit_previous(iterable, initial=None):
previous = initial
for item in iterable:
yield previous, item
previous = item
cipher = []
for previous, element in emit_previous(message, initial=initialization_value):
seed = element ^ previous ^ key
cipher.append(seed)
Here is an example how to access the last and next created elements in a list comprehension using enumerate.
The previous value…
Here -1 is the offset and None is the default value.
A = ['A','B','C','D','E','F','G','H','I','J']
[(A[i-1] if (i-1) >= 0 else None,x) for i,x in enumerate(A)
Output as tuple (Previous, Current)
[(None, ‘A’),
(‘A’, ‘B’),
(‘B’, ‘C’),
(‘C’, ‘D’),
(‘D’, ‘E’),
(‘E’, ‘F’),
(‘F’, ‘G’),
(‘G’, ‘H’),
(‘H’, ‘I’),
(‘I’, ‘J’)]
The next value…
A = ['A','B','C','D','E','F','G','H','I','J']
[(x, A[i+1] if (i+1) < len(A) else None) for i,x in enumerate(A)]
Output as tuple (Current, Next)
[(‘A’, ‘B’),
(‘B’, ‘C’),
(‘C’, ‘D’),
(‘D’, ‘E’),
(‘E’, ‘F’),
(‘F’, ‘G’),
(‘G’, ‘H’),
(‘H’, ‘I’),
(‘I’, ‘J’),
(‘J’, None)]
This is very similar to the lag() and lead() analytical functions in sql.
So you could append to a list within the list comprehension and use the last added element as the new element for that list comprehension and then finally assigning that comprehension to the list variable.
message = [0,1,2,3,4,5,6,7]
key = 8
previous = 9
cipher = [previous]
cipher = [(cipher.append(element^cipher[-1]^key), cipher[-1])[1] for element in message]
print(cipher)
itertools.accumulate() exists exactly for that:
from itertools import accumulate
result = list(accumulate(message, lambda prev, element: prev^element^key), initial=initialization_value)
del result[0] # Remove the initialization value
A simple one-liner, no need for a comprehension or anything complex.
Since Python 3.8, there is now support for an assignment expression that can be used to implement a list comprehension that remembers the last value in the iteration:
previous = initialization_value
cipher = [previous := element ^ previous ^ key for element in message]
If you prefer to write everything in one line, you can initialize previous in a singleton list, and discard it with the and operator to let the list comprehension take over:
cipher = [previous := initialization_value] and [previous := element ^ previous ^ key for element in message]