Cannot cast array data from dtype('O') to dtype('float64')
Question:
I am using scipy’s curve_fit to fit a function to some data, and receive the following error;
Cannot cast array data from dtype('O') to dtype('float64') according to the rule 'safe'
which points me to this line in my code;
popt_r, pcov = curve_fit(
self.rightFunc, np.array(wavelength)[beg:end][edgeIndex+30:],
np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],
p0=[self.m_right, self.a_right])
rightFunc is defined as follows;
def rightFunc(self, x, m, const):
return np.exp(-(m*x + const))
As I understand it, the ‘O’ type refers to a python object, but I can’t see what is causing this error.
Complete Error:
Any ideas for what I should investigate to get to the bottom of this?
Answers:
From here, apparently numpy struggles with index type. The proposed solution is:
One thing you can do is use np.intp as dtype whenever things have to do with indexing or are logically related to indexing/array sizes. This is the natural dtype for it and it will normally also be the fastest one.
Does this help?
Typically these scipy functions require parameters like:
curvefit( function, initial_values, (aux_values,), ...)
where the tuple of aux_values is passed through to your function along with the current value of the main variable.
Is the dstack expression this aux_values? Or a concatenation of several. It may need to be wrapped in a tuple.
(np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],)
We may need to know exactly where this error arises, not just which line of your code does it. We need to know what value is being converted. Where is there an array with dtype object?
Just to clarify, I had the same problem, did not see the right answers in the comments, before solving it on my own. So I just repeat them here:
I have resolved the issue. I was passing an array with one element to the p0 list, rather than the element itself. Thank you for your help – Jacobadtr Sep 12 at 17:51
An O dtype often results when constructing an array from a list of sublists that differ in size. If np.array(…) can’t make a clean n-d array of numbers, it resorts to making an array of objects. – hpaulj Sep 12 at 17:15
That is, make sure that the tuple of parameters you pass to curve_fit can be properly casted to an numpy array
Just in case it could help someone else, I used numpy.array(wavelength,dtype='float64') to force the conversion of objects in the list to numpy’s float64. Works well for me.
I had the same error when attempting to run numpy.interp on data with string values. The error did not occur on data with numeric values.
So the error could be due to (some of) the data in your wavelength array being non numeric.
I am using scipy’s curve_fit to fit a function to some data, and receive the following error;
Cannot cast array data from dtype('O') to dtype('float64') according to the rule 'safe'
which points me to this line in my code;
popt_r, pcov = curve_fit(
self.rightFunc, np.array(wavelength)[beg:end][edgeIndex+30:],
np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],
p0=[self.m_right, self.a_right])
rightFunc is defined as follows;
def rightFunc(self, x, m, const):
return np.exp(-(m*x + const))
As I understand it, the ‘O’ type refers to a python object, but I can’t see what is causing this error.
Complete Error:
Any ideas for what I should investigate to get to the bottom of this?
From here, apparently numpy struggles with index type. The proposed solution is:
One thing you can do is use np.intp as dtype whenever things have to do with indexing or are logically related to indexing/array sizes. This is the natural dtype for it and it will normally also be the fastest one.
Does this help?
Typically these scipy functions require parameters like:
curvefit( function, initial_values, (aux_values,), ...)
where the tuple of aux_values is passed through to your function along with the current value of the main variable.
Is the dstack expression this aux_values? Or a concatenation of several. It may need to be wrapped in a tuple.
(np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],)
We may need to know exactly where this error arises, not just which line of your code does it. We need to know what value is being converted. Where is there an array with dtype object?
Just to clarify, I had the same problem, did not see the right answers in the comments, before solving it on my own. So I just repeat them here:
I have resolved the issue. I was passing an array with one element to the p0 list, rather than the element itself. Thank you for your help – Jacobadtr Sep 12 at 17:51
An O dtype often results when constructing an array from a list of sublists that differ in size. If np.array(…) can’t make a clean n-d array of numbers, it resorts to making an array of objects. – hpaulj Sep 12 at 17:15
That is, make sure that the tuple of parameters you pass to curve_fit can be properly casted to an numpy array
Just in case it could help someone else, I used numpy.array(wavelength,dtype='float64') to force the conversion of objects in the list to numpy’s float64. Works well for me.
I had the same error when attempting to run numpy.interp on data with string values. The error did not occur on data with numeric values.
So the error could be due to (some of) the data in your wavelength array being non numeric.