Tips for debugging list comprehensions?
Question:
Python list comprehensions are nice, but near impossible to debug. You guys have any good tips / tools for debugging them?
Answers:
If it’s complicated enough that it’s not obvious at first glance, unpack it into multiple steps and/or for loops. It’s clearly too complicated, and making it more explicit is the easiest way to go about debugging it. Added bonus: you can now step through with the debugger or add print statements!
tip: Use list comprehension for simple tasks (1 or 2 levels). Otherwise, making it explicit is better for readability.
In Haskell I using something similar to:
def trcPV(prompt, value):
print ("%s%s" % (prompt, str(value)))
return value
xs = trcPV("xs=", [x for x in range(0,100) if trcPV("check=",(trcPV("x=",x) % 15) in [0,3,5])])
Use a debugger like pdb to walk through or break the list comprehension into a full for loop.
Haskell list comprehensions at least can be (and that is what compilers do) rewritten in terms of map, concat and filter.
So this Haskell example:
[ x*x | x<-[1..25], even x]
Works out as:
map (x-> x*x) (filter (even) [1..25])
I expect similar identities would continue to hold for Python, so similar decomposition should yield equivalent code in Python as well. The equivalent code should prove easier to debug (and run about as efficiently).
It depends on the list comprehension. You can move a part of the code to another function. This should be a clean solution which is more easy to debug.
Example:
[1.0 / i for i in [0, 2, 5, 10]]
Can be divided into
[f(i) for i in [0, 2, 5, 10]]
and a function
def f(i):
return 1.0 / i
When you do the debugging you will find out it will crash because of a “division-by-zero” error at f for the value of i = 0.
I use a function that just prints and returns a value at the same time:
from pprint import pprint
def debug(msg, item):
print('n' + msg + ':')
pprint(item)
return item
It’s very handy for debugging any part of a list/dict comprehension:
new_lines = [
debug('CUR UPDATED LINE', change(line))
for line
in debug('ALL LINES', get_lines_from_file(filename))
if debug('CUR LINE EMPTY?', not_empty(line))
]
Building on Elmex80s very nice response in https://stackoverflow.com/a/39350403/5339264, using a debug function can also help with TypeError: unsupported operand type(s) for +: 'method' and 'str' or similar errors in a list comprehension.
A classic
def debug(i):
print(f"i: {i}, {str(type(i))}")
in a list comprehension like
[debug(item) for item in list]
can be very useful to unravel what item in the list is causing the error.
Python list comprehensions are nice, but near impossible to debug. You guys have any good tips / tools for debugging them?
If it’s complicated enough that it’s not obvious at first glance, unpack it into multiple steps and/or for loops. It’s clearly too complicated, and making it more explicit is the easiest way to go about debugging it. Added bonus: you can now step through with the debugger or add print statements!
tip: Use list comprehension for simple tasks (1 or 2 levels). Otherwise, making it explicit is better for readability.
In Haskell I using something similar to:
def trcPV(prompt, value):
print ("%s%s" % (prompt, str(value)))
return value
xs = trcPV("xs=", [x for x in range(0,100) if trcPV("check=",(trcPV("x=",x) % 15) in [0,3,5])])
Use a debugger like pdb to walk through or break the list comprehension into a full for loop.
Haskell list comprehensions at least can be (and that is what compilers do) rewritten in terms of map, concat and filter.
So this Haskell example:
[ x*x | x<-[1..25], even x]
Works out as:
map (x-> x*x) (filter (even) [1..25])
I expect similar identities would continue to hold for Python, so similar decomposition should yield equivalent code in Python as well. The equivalent code should prove easier to debug (and run about as efficiently).
It depends on the list comprehension. You can move a part of the code to another function. This should be a clean solution which is more easy to debug.
Example:
[1.0 / i for i in [0, 2, 5, 10]]
Can be divided into
[f(i) for i in [0, 2, 5, 10]]
and a function
def f(i):
return 1.0 / i
When you do the debugging you will find out it will crash because of a “division-by-zero” error at f for the value of i = 0.
I use a function that just prints and returns a value at the same time:
from pprint import pprint
def debug(msg, item):
print('n' + msg + ':')
pprint(item)
return item
It’s very handy for debugging any part of a list/dict comprehension:
new_lines = [
debug('CUR UPDATED LINE', change(line))
for line
in debug('ALL LINES', get_lines_from_file(filename))
if debug('CUR LINE EMPTY?', not_empty(line))
]
Building on Elmex80s very nice response in https://stackoverflow.com/a/39350403/5339264, using a debug function can also help with TypeError: unsupported operand type(s) for +: 'method' and 'str' or similar errors in a list comprehension.
A classic
def debug(i):
print(f"i: {i}, {str(type(i))}")
in a list comprehension like
[debug(item) for item in list]
can be very useful to unravel what item in the list is causing the error.