extracting days from a numpy.timedelta64 value
Question:
I am using pandas/python and I have two date time series s1 and s2, that have been generated using the ‘to_datetime’ function on a field of the df containing dates/times.
When I subtract s1 from s2
s3 = s2 – s1
I get a series, s3, of type
timedelta64[ns]
0 385 days, 04:10:36
1 57 days, 22:54:00
2 642 days, 21:15:23
3 615 days, 00:55:44
4 160 days, 22:13:35
5 196 days, 23:06:49
6 23 days, 22:57:17
7 2 days, 22:17:31
8 622 days, 01:29:25
9 79 days, 20:15:14
10 23 days, 22:46:51
11 268 days, 19:23:04
12 NaT
13 NaT
14 583 days, 03:40:39
How do I look at 1 element of the series:
s3[10]
I get something like this:
numpy.timedelta64(2069211000000000,’ns’)
How do I extract days from s3 and maybe keep them as integers(not so interested in hours/mins etc.)?
Answers:
You can convert it to a timedelta with a day precision. To extract the integer value of days you divide it with a timedelta of one day.
>>> x = np.timedelta64(2069211000000000, 'ns')
>>> days = x.astype('timedelta64[D]')
>>> days / np.timedelta64(1, 'D')
23
Or, as @PhillipCloud suggested, just days.astype(int) since the timedelta is just a 64bit integer that is interpreted in various ways depending on the second parameter you passed in ('D', 'ns', …).
You can find more about it here.
Suppose you have a timedelta series:
import pandas as pd
from datetime import datetime
z = pd.DataFrame({'a':[datetime.strptime('20150101', '%Y%m%d')],'b':[datetime.strptime('20140601', '%Y%m%d')]})
td_series = (z['a'] - z['b'])
One way to convert this timedelta column or series is to cast it to a Timedelta object (pandas 0.15.0+) and then extract the days from the object:
td_series.astype(pd.Timedelta).apply(lambda l: l.days)
Another way is to cast the series as a timedelta64 in days, and then cast it as an int:
td_series.astype('timedelta64[D]').astype(int)
Use dt.days to obtain the days attribute as integers.
For eg:
In [14]: s = pd.Series(pd.timedelta_range(start='1 days', end='12 days', freq='3000T'))
In [15]: s
Out[15]:
0 1 days 00:00:00
1 3 days 02:00:00
2 5 days 04:00:00
3 7 days 06:00:00
4 9 days 08:00:00
5 11 days 10:00:00
dtype: timedelta64[ns]
In [16]: s.dt.days
Out[16]:
0 1
1 3
2 5
3 7
4 9
5 11
dtype: int64
More generally – You can use the .components property to access a reduced form of timedelta.
In [17]: s.dt.components
Out[17]:
days hours minutes seconds milliseconds microseconds nanoseconds
0 1 0 0 0 0 0 0
1 3 2 0 0 0 0 0
2 5 4 0 0 0 0 0
3 7 6 0 0 0 0 0
4 9 8 0 0 0 0 0
5 11 10 0 0 0 0 0
Now, to get the hours attribute:
In [23]: s.dt.components.hours
Out[23]:
0 0
1 2
2 4
3 6
4 8
5 10
Name: hours, dtype: int64
First, convert the date time column in pandas date time by using:
## Convert time in pandas date time
df['Start'] = pd.to_datetime(df['Start'], errors='coerce')
Once that is done use the following command to subtract two dates:
df["Duration_after subtraction"] = (df['End_Time'] - df['Start_Time'] / np.timedelta64(1, 'm')
To convert into hour use ‘h’ instead of ‘m’
I am using pandas/python and I have two date time series s1 and s2, that have been generated using the ‘to_datetime’ function on a field of the df containing dates/times.
When I subtract s1 from s2
s3 = s2 – s1
I get a series, s3, of type
timedelta64[ns]
0 385 days, 04:10:36
1 57 days, 22:54:00
2 642 days, 21:15:23
3 615 days, 00:55:44
4 160 days, 22:13:35
5 196 days, 23:06:49
6 23 days, 22:57:17
7 2 days, 22:17:31
8 622 days, 01:29:25
9 79 days, 20:15:14
10 23 days, 22:46:51
11 268 days, 19:23:04
12 NaT
13 NaT
14 583 days, 03:40:39
How do I look at 1 element of the series:
s3[10]
I get something like this:
numpy.timedelta64(2069211000000000,’ns’)
How do I extract days from s3 and maybe keep them as integers(not so interested in hours/mins etc.)?
You can convert it to a timedelta with a day precision. To extract the integer value of days you divide it with a timedelta of one day.
>>> x = np.timedelta64(2069211000000000, 'ns')
>>> days = x.astype('timedelta64[D]')
>>> days / np.timedelta64(1, 'D')
23
Or, as @PhillipCloud suggested, just days.astype(int) since the timedelta is just a 64bit integer that is interpreted in various ways depending on the second parameter you passed in ('D', 'ns', …).
You can find more about it here.
Suppose you have a timedelta series:
import pandas as pd
from datetime import datetime
z = pd.DataFrame({'a':[datetime.strptime('20150101', '%Y%m%d')],'b':[datetime.strptime('20140601', '%Y%m%d')]})
td_series = (z['a'] - z['b'])
One way to convert this timedelta column or series is to cast it to a Timedelta object (pandas 0.15.0+) and then extract the days from the object:
td_series.astype(pd.Timedelta).apply(lambda l: l.days)
Another way is to cast the series as a timedelta64 in days, and then cast it as an int:
td_series.astype('timedelta64[D]').astype(int)
Use dt.days to obtain the days attribute as integers.
For eg:
In [14]: s = pd.Series(pd.timedelta_range(start='1 days', end='12 days', freq='3000T'))
In [15]: s
Out[15]:
0 1 days 00:00:00
1 3 days 02:00:00
2 5 days 04:00:00
3 7 days 06:00:00
4 9 days 08:00:00
5 11 days 10:00:00
dtype: timedelta64[ns]
In [16]: s.dt.days
Out[16]:
0 1
1 3
2 5
3 7
4 9
5 11
dtype: int64
More generally – You can use the .components property to access a reduced form of timedelta.
In [17]: s.dt.components
Out[17]:
days hours minutes seconds milliseconds microseconds nanoseconds
0 1 0 0 0 0 0 0
1 3 2 0 0 0 0 0
2 5 4 0 0 0 0 0
3 7 6 0 0 0 0 0
4 9 8 0 0 0 0 0
5 11 10 0 0 0 0 0
Now, to get the hours attribute:
In [23]: s.dt.components.hours
Out[23]:
0 0
1 2
2 4
3 6
4 8
5 10
Name: hours, dtype: int64
First, convert the date time column in pandas date time by using:
## Convert time in pandas date time
df['Start'] = pd.to_datetime(df['Start'], errors='coerce')
Once that is done use the following command to subtract two dates:
df["Duration_after subtraction"] = (df['End_Time'] - df['Start_Time'] / np.timedelta64(1, 'm')
To convert into hour use ‘h’ instead of ‘m’