How do I make a grid of empty lists in numpy that 'accepts' appending?
Question:
I am trying to use numpy.append but something goes wrong and it just doesn’t make sence to me anymore. Can someone explain why I am getting an error?
>>> np.array([[], [], []]).shape
(3, 0)
>>> a=[[], [], []]
>>> a[1].append(3)
>>> a
[[], [3], []]
>>> b=np.array(a)
>>> b[0].append(3)
array([[3], [3], []], dtype=object)
This is all logical to me, yet when I try the following it stops working.
>>> c=np.array((3,0),dtype=object)
>>> c[0].append(3)
AttributeError: 'int' object has no attribute 'append'
????
>>> np.empty((3,1))[0].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'
>>> np.empty((3,0))[1].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'
>>>np.empty((6,1),dtype=object)[0].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'
Answers:
Don’t just look at the shape; check the dtype, and if object, the nature of the elements
In [282]: np.array([[], [], []])
Out[282]: array([], shape=(3, 0), dtype=float64)
A 2d array of floats. np.array tries to make a multidimensional array of numbers; it’s only when it can’t do that it makes an object array.
In [283]: b=np.array([[],[3],[]])
In [284]: b
Out[284]: array([[], [3], []], dtype=object)
Here the 3 sublists have different size, so it can’t make a 2d array; the result is an object array, where the objects are lists, and have the append method.
In [286]: c=np.array((3,0), object)
In [287]: c
Out[287]: array([3, 0], dtype=object)
This is a (2,) object array; the 2 elements are numbers. Numbers don’t have an append method.
In [288]: np.empty((3,1))
Out[288]:
array([[ 0.],
[ 0.],
[ 0.]])
A (3,1) array of floats. No append method for numbers or arrays.
In [289]: np.empty((3,0))
Out[289]: array([], shape=(3, 0), dtype=float64)
Another 2d array of floats
In [290]: np.empty((6,1),object)
Out[290]:
array([[None],
[None],
[None],
[None],
[None],
[None]], dtype=object)
2d array of dtype object. In this case they are initialized to None. Again no append method.
More on making an array of lists
dimensions of array of arrays in numpy
and
How to keep numpy from broadcasting when creating an object array of different shaped arrays
In [305]: d=np.empty((3,),object)
In [306]: d
Out[306]: array([None, None, None], dtype=object)
In [307]: d.fill([])
In [308]: d
Out[308]: array([[], [], []], dtype=object) # array of lists
In [309]: d[0].append([1,2,3])
In [310]: d
Out[310]: array([[[1, 2, 3]], [[1, 2, 3]], [[1, 2, 3]]], dtype=object)
But oops – those lists are all the same object (pointer) 🙁 I have to put a different list in each element. Now I can append to them individually.
In [311]: d[...]=[[],[1,2,3],[2]]
In [312]: d
Out[312]: array([[], [1, 2, 3], [2]], dtype=object)
In [313]: d[0].append([2,3])
In [314]: d
Out[314]: array([[[2, 3]], [1, 2, 3], [2]], dtype=object)
I think you have to bite the bullet and use a list to initialize an object array of lists. There isn’t a short cut:
In [319]: d=np.empty((3,),object)
In [320]: d[...]=[[] for _ in range(3)]
In [321]: d
Out[321]: array([[], [], []], dtype=object)
In [323]: d
Out[323]: array([[], [3], []], dtype=object)
First of all, stop updating your question when someone answers it correctly. Accept/upvote an answer and ask a new question when you are ready. SO is a Q&A site that is meant to help future visitors viewing your question, and not just you. Invalidating all the good answers by changing the context completely for your own benefit defeats the purpose of this site, to say the least.
Secondly, np.array([[], [3], []]) ends up with dtype=object because it is a ragged array. [[], [], []] and [[3], [3], [3]], having a uniform length in all elements across all dimensions, would produce numerical arrays.
np.zeros((6,),dtype=object) produces an empty array of object references (containing NULLs). When you fill it with [], you are filling it with a reference to the same python list in every element. numpy has no knowledge of what object you are passing to ndarray.fill, so it does not call the constructor on the list type for each element as you seem to be expecting. It just copies the reference that you passed in six times. After that, it should be clear why changing the contents of that one list makes it appear that all the array elements have changed.
I know this is a rather unusual use case but in fact, it can be very handy to have an (nd)array holding lists in each cell. Imho numpy should allow passing a lambda function to the fill method. But to achieve what you want, this is what I do:
m = np.empty((12, 12), dtype=object)
for i in np.ndindex(m.shape): m[i] = []
I am trying to use numpy.append but something goes wrong and it just doesn’t make sence to me anymore. Can someone explain why I am getting an error?
>>> np.array([[], [], []]).shape
(3, 0)
>>> a=[[], [], []]
>>> a[1].append(3)
>>> a
[[], [3], []]
>>> b=np.array(a)
>>> b[0].append(3)
array([[3], [3], []], dtype=object)
This is all logical to me, yet when I try the following it stops working.
>>> c=np.array((3,0),dtype=object)
>>> c[0].append(3)
AttributeError: 'int' object has no attribute 'append'
????
>>> np.empty((3,1))[0].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'
>>> np.empty((3,0))[1].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'
>>>np.empty((6,1),dtype=object)[0].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'
Don’t just look at the shape; check the dtype, and if object, the nature of the elements
In [282]: np.array([[], [], []])
Out[282]: array([], shape=(3, 0), dtype=float64)
A 2d array of floats. np.array tries to make a multidimensional array of numbers; it’s only when it can’t do that it makes an object array.
In [283]: b=np.array([[],[3],[]])
In [284]: b
Out[284]: array([[], [3], []], dtype=object)
Here the 3 sublists have different size, so it can’t make a 2d array; the result is an object array, where the objects are lists, and have the append method.
In [286]: c=np.array((3,0), object)
In [287]: c
Out[287]: array([3, 0], dtype=object)
This is a (2,) object array; the 2 elements are numbers. Numbers don’t have an append method.
In [288]: np.empty((3,1))
Out[288]:
array([[ 0.],
[ 0.],
[ 0.]])
A (3,1) array of floats. No append method for numbers or arrays.
In [289]: np.empty((3,0))
Out[289]: array([], shape=(3, 0), dtype=float64)
Another 2d array of floats
In [290]: np.empty((6,1),object)
Out[290]:
array([[None],
[None],
[None],
[None],
[None],
[None]], dtype=object)
2d array of dtype object. In this case they are initialized to None. Again no append method.
More on making an array of lists
dimensions of array of arrays in numpy
and
How to keep numpy from broadcasting when creating an object array of different shaped arrays
In [305]: d=np.empty((3,),object)
In [306]: d
Out[306]: array([None, None, None], dtype=object)
In [307]: d.fill([])
In [308]: d
Out[308]: array([[], [], []], dtype=object) # array of lists
In [309]: d[0].append([1,2,3])
In [310]: d
Out[310]: array([[[1, 2, 3]], [[1, 2, 3]], [[1, 2, 3]]], dtype=object)
But oops – those lists are all the same object (pointer) 🙁 I have to put a different list in each element. Now I can append to them individually.
In [311]: d[...]=[[],[1,2,3],[2]]
In [312]: d
Out[312]: array([[], [1, 2, 3], [2]], dtype=object)
In [313]: d[0].append([2,3])
In [314]: d
Out[314]: array([[[2, 3]], [1, 2, 3], [2]], dtype=object)
I think you have to bite the bullet and use a list to initialize an object array of lists. There isn’t a short cut:
In [319]: d=np.empty((3,),object)
In [320]: d[...]=[[] for _ in range(3)]
In [321]: d
Out[321]: array([[], [], []], dtype=object)
In [323]: d
Out[323]: array([[], [3], []], dtype=object)
First of all, stop updating your question when someone answers it correctly. Accept/upvote an answer and ask a new question when you are ready. SO is a Q&A site that is meant to help future visitors viewing your question, and not just you. Invalidating all the good answers by changing the context completely for your own benefit defeats the purpose of this site, to say the least.
Secondly, np.array([[], [3], []]) ends up with dtype=object because it is a ragged array. [[], [], []] and [[3], [3], [3]], having a uniform length in all elements across all dimensions, would produce numerical arrays.
np.zeros((6,),dtype=object) produces an empty array of object references (containing NULLs). When you fill it with [], you are filling it with a reference to the same python list in every element. numpy has no knowledge of what object you are passing to ndarray.fill, so it does not call the constructor on the list type for each element as you seem to be expecting. It just copies the reference that you passed in six times. After that, it should be clear why changing the contents of that one list makes it appear that all the array elements have changed.
I know this is a rather unusual use case but in fact, it can be very handy to have an (nd)array holding lists in each cell. Imho numpy should allow passing a lambda function to the fill method. But to achieve what you want, this is what I do:
m = np.empty((12, 12), dtype=object)
for i in np.ndindex(m.shape): m[i] = []