Partial list unpack in Python
Question:
In Python, the assignment operator can unpack a list or a tuple into variables, like this:
l = (1, 2)
a, b = l # Here goes auto unpack
But I need to specify exactly the same amount of names to the left as an item count in the list to the right. But sometimes I don’t know the size of the list to the right, for example, if I use split().
Example:
a, b = "length=25".split("=") # This will result in a="length" and b=25
But the following code will lead to an error:
a, b = "DEFAULT_LENGTH".split("=") # Error, list has only one item
Is it possible to somehow unpack the list in the example above so I can get a = “DEFAULT_LENGTH” and b equals to None
or not set? A straightforward way looks kind of long:
a = b = None
if "=" in string :
a, b = string.split("=")
else :
a = string
Answers:
Have you tried this?
values = aString.split("=")
if len(values) == 1:
a = values[0]
else:
a, b = values
This is slightly better than your solution but still not very elegant; it wouldn’t surprise me if there’s a better way to do it.
a, b = (string.split("=") + [None])[:2]
# this will result in a="length" and b="25"
a, b = "length=25".partition("=")[::2]
# this will result in a="DEFAULT_LENGTH" and b=""
a, b = "DEFAULT_LENGTH".partition("=")[::2]
You could write a helper function to do it.
>>> def pack(values, size):
... if len(values) >= size:
... return values[:size]
... return values + [None] * (size - len(values))
...
>>> a, b = pack('a:b:c'.split(':'), 2)
>>> a, b
('a', 'b')
>>> a, b = pack('a'.split(':'), 2)
>>> a, b
('a', None)
Don’t use this code, it is meant as a joke, but it does what you want:
a = b = None
try: a, b = [a for a in 'DEFAULT_LENGTH'.split('=')]
except: pass
The nicest way is using the partition string method:
Split the string at the first occurrence of sep, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing the string itself, followed by two empty strings.
New in version 2.5.
>>> inputstr = "length=25"
>>> inputstr.partition("=")
('length', '=', '25')
>>> name, _, value = inputstr.partition("=")
>>> print name, value
length 25
It also works for strings not containing the =
:
>>> inputstr = "DEFAULT_VALUE"
>>> inputstr.partition("=")
('DEFAULT_VALUE', '', '')
If for some reason you are using a version of Python before 2.5, you can use list-slicing to do much the same, if slightly less tidily:
>>> x = "DEFAULT_LENGTH"
>>> a = x.split("=")[0]
>>> b = "=".join(x.split("=")[1:])
>>> print (a, b)
('DEFAULT_LENGTH', '')
..and when x = "length=25"
:
('length', '25')
Easily turned into a function or lambda:
>>> part = lambda x: (x.split("=")[0], "=".join(x.split("=")[1:]))
>>> part("length=25")
('length', '25')
>>> part('DEFAULT_LENGTH')
('DEFAULT_LENGTH', '')
This may be of no use to you unless you’re using Python 3. However, for completeness, it’s worth noting that the extended tuple unpacking introduced there allows you to do things like:
>>> a, *b = "length=25".split("=")
>>> a,b
("length", ['25'])
>>> a, *b = "DEFAULT_LENGTH".split("=")
>>> a,b
("DEFAULT_LENGTH", [])
I.e. tuple unpacking now works similarly to how it does in argument unpacking, so you can denote “the rest of the items” with *
, and get them as a (possibly empty) list.
Partition is probably the best solution for what you’re doing however.
But sometimes I don’t know a size of the list to the right, for example if I use split().
Yeah, when I’ve got cases with limit>1 (so I can’t use partition) I usually plump for:
def paddedsplit(s, find, limit):
parts= s.split(find, limit)
return parts+[parts[0][:0]]*(limit+1-len(parts))
username, password, hash= paddedsplit(credentials, ':', 2)
(parts[0][:0]
is there to get an empty ‘str’ or ‘unicode’, matching whichever of those the split produced. You could use None if you prefer.)
Many other solutions have been proposed, but I have to say the most straightforward to me is still
a, b = string.split("=") if "=" in string else (string, None)
As an alternative, perhaps use a regular expression?
>>> import re
>>> unpack_re = re.compile("(w*)(?:=(w*))?")
>>> x = "DEFAULT_LENGTH"
>>> unpack_re.match(x).groups()
('DEFAULT_LENGTH', None)
>>> y = "length=107"
>>> unpack_re.match(y).groups()
('length', '107')
If you make sure the re.match() always succeeds, .groups() will always return the right number of elements to unpack into your tuple, so you can safely do
a,b = unpack_re.match(x).groups()
I don’t recommend using this, but just for fun here’s some code that actually does what you want. When you call unpack(<sequence>)
, the unpack
function uses the inspect
module to find the actual line of source where the function was called, then uses the ast
module to parse that line and count the number of variables being unpacked.
Caveats:
- For multiple assignment (e.g.
(a,b) = c = unpack([1,2,3])
), it only uses the first term in the assignment
- It won’t work if it can’t find the source code (e.g. because you’re calling it from the repl)
- It won’t work if the assignment statement spans multiple lines
Code:
import inspect, ast
from itertools import islice, chain, cycle
def iter_n(iterator, n, default=None):
return islice(chain(iterator, cycle([default])), n)
def unpack(sequence, default=None):
stack = inspect.stack()
try:
frame = stack[1][0]
source = inspect.getsource(inspect.getmodule(frame)).splitlines()
line = source[frame.f_lineno-1].strip()
try:
tree = ast.parse(line, 'whatever', 'exec')
except SyntaxError:
return tuple(sequence)
exp = tree.body[0]
if not isinstance(exp, ast.Assign):
return tuple(sequence)
exp = exp.targets[0]
if not isinstance(exp, ast.Tuple):
return tuple(sequence)
n_items = len(exp.elts)
return tuple(iter_n(sequence, n_items, default))
finally:
del stack
# Examples
if __name__ == '__main__':
# Extra items are discarded
x, y = unpack([1,2,3,4,5])
assert (x,y) == (1,2)
# Missing items become None
x, y, z = unpack([9])
assert (x, y, z) == (9, None, None)
# Or the default you provide
x, y, z = unpack([1], 'foo')
assert (x, y, z) == (1, 'foo', 'foo')
# unpack() is equivalent to tuple() if it's not part of an assignment
assert unpack('abc') == ('a', 'b', 'c')
# Or if it's part of an assignment that isn't sequence-unpacking
x = unpack([1,2,3])
assert x == (1,2,3)
# Add a comma to force tuple assignment:
x, = unpack([1,2,3])
assert x == 1
# unpack only uses the first assignment target
# So in this case, unpack('foobar') returns tuple('foo')
(x, y, z) = t = unpack('foobar')
assert (x, y, z) == t == ('f', 'o', 'o')
# But in this case, it returns tuple('foobar')
try:
t = (x, y, z) = unpack('foobar')
except ValueError as e:
assert str(e) == 'too many values to unpack'
else:
raise Exception("That should have failed.")
# Also, it won't work if the call spans multiple lines, because it only
# inspects the actual line where the call happens:
try:
(x, y, z) = unpack([
1, 2, 3, 4])
except ValueError as e:
assert str(e) == 'too many values to unpack'
else:
raise Exception("That should have failed.")
In Python, the assignment operator can unpack a list or a tuple into variables, like this:
l = (1, 2)
a, b = l # Here goes auto unpack
But I need to specify exactly the same amount of names to the left as an item count in the list to the right. But sometimes I don’t know the size of the list to the right, for example, if I use split().
Example:
a, b = "length=25".split("=") # This will result in a="length" and b=25
But the following code will lead to an error:
a, b = "DEFAULT_LENGTH".split("=") # Error, list has only one item
Is it possible to somehow unpack the list in the example above so I can get a = “DEFAULT_LENGTH” and b equals to None
or not set? A straightforward way looks kind of long:
a = b = None
if "=" in string :
a, b = string.split("=")
else :
a = string
Have you tried this?
values = aString.split("=")
if len(values) == 1:
a = values[0]
else:
a, b = values
This is slightly better than your solution but still not very elegant; it wouldn’t surprise me if there’s a better way to do it.
a, b = (string.split("=") + [None])[:2]
# this will result in a="length" and b="25"
a, b = "length=25".partition("=")[::2]
# this will result in a="DEFAULT_LENGTH" and b=""
a, b = "DEFAULT_LENGTH".partition("=")[::2]
You could write a helper function to do it.
>>> def pack(values, size):
... if len(values) >= size:
... return values[:size]
... return values + [None] * (size - len(values))
...
>>> a, b = pack('a:b:c'.split(':'), 2)
>>> a, b
('a', 'b')
>>> a, b = pack('a'.split(':'), 2)
>>> a, b
('a', None)
Don’t use this code, it is meant as a joke, but it does what you want:
a = b = None
try: a, b = [a for a in 'DEFAULT_LENGTH'.split('=')]
except: pass
The nicest way is using the partition string method:
Split the string at the first occurrence of sep, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing the string itself, followed by two empty strings.
New in version 2.5.
>>> inputstr = "length=25"
>>> inputstr.partition("=")
('length', '=', '25')
>>> name, _, value = inputstr.partition("=")
>>> print name, value
length 25
It also works for strings not containing the =
:
>>> inputstr = "DEFAULT_VALUE"
>>> inputstr.partition("=")
('DEFAULT_VALUE', '', '')
If for some reason you are using a version of Python before 2.5, you can use list-slicing to do much the same, if slightly less tidily:
>>> x = "DEFAULT_LENGTH"
>>> a = x.split("=")[0]
>>> b = "=".join(x.split("=")[1:])
>>> print (a, b)
('DEFAULT_LENGTH', '')
..and when x = "length=25"
:
('length', '25')
Easily turned into a function or lambda:
>>> part = lambda x: (x.split("=")[0], "=".join(x.split("=")[1:]))
>>> part("length=25")
('length', '25')
>>> part('DEFAULT_LENGTH')
('DEFAULT_LENGTH', '')
This may be of no use to you unless you’re using Python 3. However, for completeness, it’s worth noting that the extended tuple unpacking introduced there allows you to do things like:
>>> a, *b = "length=25".split("=")
>>> a,b
("length", ['25'])
>>> a, *b = "DEFAULT_LENGTH".split("=")
>>> a,b
("DEFAULT_LENGTH", [])
I.e. tuple unpacking now works similarly to how it does in argument unpacking, so you can denote “the rest of the items” with *
, and get them as a (possibly empty) list.
Partition is probably the best solution for what you’re doing however.
But sometimes I don’t know a size of the list to the right, for example if I use split().
Yeah, when I’ve got cases with limit>1 (so I can’t use partition) I usually plump for:
def paddedsplit(s, find, limit):
parts= s.split(find, limit)
return parts+[parts[0][:0]]*(limit+1-len(parts))
username, password, hash= paddedsplit(credentials, ':', 2)
(parts[0][:0]
is there to get an empty ‘str’ or ‘unicode’, matching whichever of those the split produced. You could use None if you prefer.)
Many other solutions have been proposed, but I have to say the most straightforward to me is still
a, b = string.split("=") if "=" in string else (string, None)
As an alternative, perhaps use a regular expression?
>>> import re
>>> unpack_re = re.compile("(w*)(?:=(w*))?")
>>> x = "DEFAULT_LENGTH"
>>> unpack_re.match(x).groups()
('DEFAULT_LENGTH', None)
>>> y = "length=107"
>>> unpack_re.match(y).groups()
('length', '107')
If you make sure the re.match() always succeeds, .groups() will always return the right number of elements to unpack into your tuple, so you can safely do
a,b = unpack_re.match(x).groups()
I don’t recommend using this, but just for fun here’s some code that actually does what you want. When you call unpack(<sequence>)
, the unpack
function uses the inspect
module to find the actual line of source where the function was called, then uses the ast
module to parse that line and count the number of variables being unpacked.
Caveats:
- For multiple assignment (e.g.
(a,b) = c = unpack([1,2,3])
), it only uses the first term in the assignment - It won’t work if it can’t find the source code (e.g. because you’re calling it from the repl)
- It won’t work if the assignment statement spans multiple lines
Code:
import inspect, ast
from itertools import islice, chain, cycle
def iter_n(iterator, n, default=None):
return islice(chain(iterator, cycle([default])), n)
def unpack(sequence, default=None):
stack = inspect.stack()
try:
frame = stack[1][0]
source = inspect.getsource(inspect.getmodule(frame)).splitlines()
line = source[frame.f_lineno-1].strip()
try:
tree = ast.parse(line, 'whatever', 'exec')
except SyntaxError:
return tuple(sequence)
exp = tree.body[0]
if not isinstance(exp, ast.Assign):
return tuple(sequence)
exp = exp.targets[0]
if not isinstance(exp, ast.Tuple):
return tuple(sequence)
n_items = len(exp.elts)
return tuple(iter_n(sequence, n_items, default))
finally:
del stack
# Examples
if __name__ == '__main__':
# Extra items are discarded
x, y = unpack([1,2,3,4,5])
assert (x,y) == (1,2)
# Missing items become None
x, y, z = unpack([9])
assert (x, y, z) == (9, None, None)
# Or the default you provide
x, y, z = unpack([1], 'foo')
assert (x, y, z) == (1, 'foo', 'foo')
# unpack() is equivalent to tuple() if it's not part of an assignment
assert unpack('abc') == ('a', 'b', 'c')
# Or if it's part of an assignment that isn't sequence-unpacking
x = unpack([1,2,3])
assert x == (1,2,3)
# Add a comma to force tuple assignment:
x, = unpack([1,2,3])
assert x == 1
# unpack only uses the first assignment target
# So in this case, unpack('foobar') returns tuple('foo')
(x, y, z) = t = unpack('foobar')
assert (x, y, z) == t == ('f', 'o', 'o')
# But in this case, it returns tuple('foobar')
try:
t = (x, y, z) = unpack('foobar')
except ValueError as e:
assert str(e) == 'too many values to unpack'
else:
raise Exception("That should have failed.")
# Also, it won't work if the call spans multiple lines, because it only
# inspects the actual line where the call happens:
try:
(x, y, z) = unpack([
1, 2, 3, 4])
except ValueError as e:
assert str(e) == 'too many values to unpack'
else:
raise Exception("That should have failed.")