Cast decimal number to datetime in Pandas

Question

I am attempting to convert a decimal number to a datatime. An example dateframe is:

     timestep_time  vehicle_angle  vehicle_id  vehicle_pos  vehicle_speed  vehicle_x  vehicle_y  Cluster
0             0.00         113.79           0         5.10           0.00     295.36     438.47        1
1             1.00         113.79           0         6.73           1.63     296.85     437.82        1
2             1.01         203.94           1         5.10           0.00     278.32     434.32        9
3             2.00         113.79           0        10.00           3.27     299.84     436.50        1
4             2.01         203.94           1         6.50           1.40     277.75     433.04        9
5             2.02          23.94           2         5.10           0.00     255.88     375.91        1

I am not super concerned with the actual units of time, I just need it as a datetime object. So, I have tried:

posBirch["timestep_time"] = pd.to_datetime(posBirch["timestep_time"], format='%M.%S')

and

posBirch["timestep_time"] = pd.to_datetime(posBirch["timestep_time"], format='%S.%f')

# Error:

ValueError: time data '0' does not match format '%M.%S' (match)

posBirch["timestep_time"] = pd.to_datetime(posBirch["timestep_time"], format='%S')

# Yields no error but drops decimal which I need to keep timesteps unique which I will later use for indexing i.e.:

1   1900-01-01 00:00:01         113.79           0         6.73           1.63     296.85     437.82        1
2   1900-01-01 00:00:01         203.94           1         5.10           0.00     278.32     434.32        9

posBirch["timestep_time"] = pd.to_datetime(posBirch["timestep_time"], format='%-S')

# Error:

ValueError: '-' is a bad directive in format '%-S'

How can I just convert this decimal to a unique datetime value? Units do not matter too much as this is a pseudo-timestep anyway, but I guess ideally seconds.

Asked By: Sam Dean

||

Source

Answer 1

This will add a new column ‘Timestamp’ based on the current column ‘timestep_time’.

def create_timestamp(val):
    mins = int(val)
    secs = int((val-mins) *60)
    return pd.Timestamp(year=2020, month=1, day=1, minute=mins, second=secs)
df['TimeStamp'] = create_timestamp(df['timestep_time'][0])

Answered By: itprorh66

Answer 2

Here’s what ended up working:

posBirch['TimeStamp'] = pd.to_datetime(posBirch['timestep_time'], unit='s')

Answered By: Sam Dean

Answer 3

I received a Decimal Epoch time in milliseconds from an AWS response. I didn’t see a simple way to convert that to a datetime with Panda unless you first cast it to an int or string.

I used this:

# cast from Decimal to int
df["creationDate"] = df["creationDate"].astype(int)

# convert Epoch with milliseconds to MONTH-YEAR
df["creationDate"] = pd.to_datetime(df["creationDate"], unit='ms').dt.strftime('%m-%Y')

The output I wanted was:

Answered By: rustyMagnet

Cast decimal number to datetime in Pandas

Question:

Answers: