AttributeError: Can only use .dt accessor with datetimelike values

Question:

Hi I am using pandas to convert a column to month.
When I read my data they are objects:

Date           object
dtype: object

So I am first making them to date time and then try to make them as months:

import pandas as pd
file = '/pathtocsv.csv'
df = pd.read_csv(file, sep = ',', encoding='utf-8-sig', usecols= ['Date', 'ids'])    
df['Date'] = pd.to_datetime(df['Date'])
df['Month'] = df['Date'].dt.month

Also if that helps:

In [10]: df['Date'].dtype
Out[10]: dtype('O')

So, the error I get is like this:

/Library/Frameworks/Python.framework/Versions/2.7/bin/User/lib/python2.7/site-packages/pandas/core/series.pyc in _make_dt_accessor(self)
   2526             return maybe_to_datetimelike(self)
   2527         except Exception:
-> 2528             raise AttributeError("Can only use .dt accessor with datetimelike "
   2529                                  "values")
   2530 

AttributeError: Can only use .dt accessor with datetimelike values

EDITED:

Date columns are like this:

0         2014-01-01         
1         2014-01-01         
2         2014-01-01         
3         2014-01-01         
4         2014-01-03       
5         2014-01-03         
6         2014-01-03         
7         2014-01-07         
8         2014-01-08         
9         2014-01-09 

Do you have any ideas?
Thank you very much!

Asked By: Nasia Ntalla

||

Answers:

Your problem here is that to_datetime silently failed so the dtype remained as str/object, if you set param errors='coerce' then if the conversion fails for any particular string then those rows are set to NaT.

df['Date'] = pd.to_datetime(df['Date'], errors='coerce')

So you need to find out what is wrong with those specific row values.

See the docs

Answered By: EdChum

Your problem here is that the dtype of ‘Date’ remained as str/object. You can use the parse_dates parameter when using read_csv

import pandas as pd
file = '/pathtocsv.csv'
df = pd.read_csv(file, sep = ',', parse_dates= [col],encoding='utf-8-sig', usecols= ['Date', 'ids'],)    
df['Month'] = df['Date'].dt.month

From the documentation for the parse_dates parameter

parse_dates : bool or list of int or names or list of lists or dict, default False

The behavior is as follows:

  • boolean. If True -> try parsing the index.
  • list of int or names. e.g. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column.
  • list of lists. e.g. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date column.
  • dict, e.g. {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result ‘foo’

If a column or index cannot be represented as an array of datetimes, say because of an unparseable value or a mixture of timezones, the column or index will be returned unaltered as an object data type. For non-standard datetime parsing, use pd.to_datetime after pd.read_csv. To parse an index or column with a mixture of timezones, specify date_parser to be a partially-applied pandas.to_datetime() with utc=True. See Parsing a CSV with mixed timezones for more.

Note: A fast-path exists for iso8601-formatted dates.

The relevant case for this question is the “list of int or names” one.

col is the columns index of ‘Date’ which parses as a separate date column.

Answered By: Danny Wu

When you write

df['Date'] = pd.to_datetime(df['Date'], errors='coerce')
df['Date'] = df['Date'].dt.strftime('%m/%d')

It can fixed

Answered By: Ömer Yıldız

First you need to define the format of date column.

df['Date'] = pd.to_datetime(df.Date, format='%Y-%m-%d %H:%M:%S')

For your case base format can be set to;

df['Date'] = pd.to_datetime(df.Date, format='%Y-%m-%d')

After that you can set/change your desired output as follows;

df['Date'] = df['Date'].dt.strftime('%Y-%m-%d')

#Convert date into the proper format so that date time operation can be easily performed

df_Time_Table["Date"] = pd.to_datetime(df_Time_Table["Date"])

# Cal Year
df_Time_Table['Year'] = df_Time_Table['Date'].dt.strftime('%Y')
Answered By: Prashant Mallick

train_data=pd.read_csv("train.csv",parse_dates=["date"])

Answered By: MD Gazuruddin

I encountered a similar problem when trying to use pd.Series.dt.floor, although all the elements in my pd.Series were datetime.datetime instances (absolutely no NAs). I suspect it had to do with having tz-aware instances with different timezones.

My workaround, in order to take advantage of the pd.Timestamp.floor method was to define the following function:

def floor_datetime(base_datetime_aware, freq="2H"):
    return pd.Timestamp(base_datetime_aware).floor(freq)

The I would just use pd.Series.apply to get every element of my Series through the function.

In the end, when you use the .dt accessor, the functions you would use are methods of the base classes, so using apply with a short custom function like mine may solve your problem!

Answered By: pmahuzieru

My answer might be late, but this is what worked for my data.

Parsing timestamp to datetime object went without any errors, but I still had error AttributeError: Can only use .dt accessor with datetimelike values when trying to extract minute component.
Turns out (after lengthy binary search), my data had a mixed timezone offsets, like this:

_time
2023-03-28T15:52:00.000-07:00
2023-02-08T11:16:13.000-08:00

So I added utc=True argument to to_datetime function to convert my timezone-aware timestamp strings to the UTC timestamp. Only after this exercise the error disappeared.

My full code:

df["time"] = pd.to_datetime(df["_time"], errors='coerce',utc=False)
df['minute'] = df['time'].dt.minute
Answered By: myroman
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.