Python – removing empty rows or columns in a list of lists

Question

I guess this problem has two parts:

Remove a list of exclusively empty elements from a list of lists
Remove an element at a certain index from every list in a list of lists if the element is always empty.

I’d like to avoid using pandas and I’m sure there’s some solution involving list comprehension. You can assume the list of lists will be rectangular.

Example:

Given:

lol = [['a',None,'c'],[None,None,None],['g',None,'i'],['j',None,None]]

(edit – wrote ‘None’ instead of None)

return:

lol = [['a','c'],['g','i'],['j',None]]

Asked By: Moses Bakst

||

Source

Answer 1

using list comprehension should work

step1: count None elements at each index position

step2: Once the elements are count we eliminate those elements if at a certain index the number of None counted elements is equal to the length of lol

from collections import Counter
# note I have change the last element of the last sub to None and not 'None' 
lol = [['a',None,'c'],[None,None,None],['g',None,'i'],['j',None, None]]

# here we count None step 1
missing_vals = Counter()
for sub in lol:
    vals = {ind:  int(not bool(val)) for ind, val in enumerate(sub)}
    counter = Counter(vals)
    missing_vals = missing_vals + counter

# step 2
out = [[a for val, a in enumerate(sub_list) if missing_vals[val] != len(lol)] 
        for sub_list in lol if any(sub_list)]
print(out)

[['a', 'c'], ['g', 'i'], ['j', None]]

Answered By: Lucas M. Uriarte

Answer 2

Probably not a very efficient answer but the best I could come up with since I don’t think it’s possible using list comprehension

for item in lol:
  if list(set(item)) == [None]: lol.remove(item)

Answered By: walker

Answer 3

Maybe you could try this simple approach:

outs = []

for l in lol:
    tmp = []
    for x in l:
        if x:         # OR if x is not None:     !<--- as comment suggested.
           tmp.append(x)
    if tmp:      # skip empty list: 2nd one
       outs.append(tmp)
    
print(outs)

Output:

[['a', 'c'], ['g', 'i'], ['j', 'None']]

Make it to a fuction:

matrix = [['a', 'b', 'c', '0'],
          ['0', 'None', 'x', 'y'],
          ['k', 'l', None, None],
          [0, '', '', 'x']]

def remove_empty(matrix):
    outs = []

    for l in matrix:
        tmp = []
    
        for x in l:
            if x:
                tmp.append(x)
        if tmp:      # skip empty list: 2nd one
           outs.append(tmp)
    
    return outs


print(remove_empty(matrix))

= Outputs:
[['a', 'b', 'c', '0'], ['0', 'None', 'x', 'y'], ['k', 'l'], ['x']]

Answered By: Daniel Hao

Answer 4

How about for a listcomp method:

[x for x in [[y for y in z if y is not None] for z in lol] if len (x) > 0]

I don’t recommend this as it’s not really readable though.

If your lists don’t contain any values apart from None which evaluate to false you could use (credit to Lucas M. Uriarte):

[[y for y in z if y is not None] for z in lol if any(z) ]

Answered By: s_pike

Answer 5

First delete empty rows:

result = [row for row in lol if not all(a is None for a in row)]

Swap rows and columns:

transpose = [list(i) for i in zip(*result)]

Second delete all empty columns (rows in transposed list):

transpose= [row for row in transpose if not all(a is None for a in row)]

And finally swap columns/rows back:

result = [list(i) for i in zip(*transpose)]

Answered By: Isem

Answer 6

This one can clean your list no matter how nested it’s

def clean_list(data):
    cleaned_list = []
    for obj in data:
        if type(obj) == list:
            sublist = clean_list(obj)  
            if sublist:
                cleaned_list.append(sublist)
        elif obj is not None:
            cleaned_list.append(obj)
    return cleaned_list

Answered By: ASH

Answer 7

This is the most efficient solution – actually doing what your want – I could come up with:

def removeEmpties(lol):
  colsCnt = len(lol[0])
  emptyRow = [None]*colsCnt
  # first remove all rows containing only None elements
  lol2 = list(lst for lst in lol if lst!=emptyRow)
  # second go through by column
  for col in range(colsCnt-1,-1,-1):
      # and now look at that column in every row
      for row in lol2:
          # if an element in the current row is not None in that column
          if row[col] is not None:
              # we do not need to look any further,
              # since this column has to remain
              break;
      else:
          # if we could not find any non-None element,
          # this column must be removed from all rows.
          for row in lol2:
              del row[col]
  return lol2

print(removeEmpties([['a' ,None,'c' ],
                     [None,None,None],
                     ['g' ,None,'i' ],
                     ['j' ,None,None]]))
                     
print(removeEmpties([['a' ,None,'c' ],
                     [None,None,None],
                     ['g' ,None,'i' ],
                     ['j' ,None,'None']]))

The output of above code is:

[['a', 'c'], ['g', 'i'], ['j', None]]
[['a', 'c'], ['g', 'i'], ['j', 'None']]

and you can see it in action at https://onlinegdb.com/g76VzPGfo

Answered By: Frank

Python – removing empty rows or columns in a list of lists

Question:

Answers:

Output:

Make it to a fuction: