Python Pandas: Convert a date string to milliseconds since epoch and back to date string?

Question

When trying to convert a date string to milliseconds since epoch and back to date string with the following code:

def date_to_millis(s) :                                                                                         
    t = pd.Timestamp(s)                                                                                         
    return time.mktime(t.timetuple())                                                                           

s = "2013-01-14 00:00:00"                                                                                       
mls = date_to_millis(s)                                                                                         
dateStr = pd.to_datetime(mls, unit='ms')                                                                        
print(s, " mls = ", mls, "  date = " , dateStr)

I get the following result where strings don’t match:

('2013-01-14 00:00:00', ' mls = ', 1358107200.0, '  date = ', Timestamp('1970-01-16 17:15:07.200000'))

Also, when re-runing this code in IPython interactive session I get an error:

AttributeError                            Traceback (most recent call last)                                     
<ipython-input-9-6a819827f8f2> in <module>()                                                                    
----> 1 time.mktime(t.timetuple())                                                                              

AttributeError: 'Timestamp' object has no attribute 'mktime'

What can be possibly wrong? How to convert a date string to milliseconds since epoch and back to date string ?

Asked By: zork

||

Source

Answer 1

Your date_to_millis function is converting to seconds not to millis.

You need to use “to_datetime64” instead of time tuple.

# %%
import pandas as pd
import time
def date_to_millis(s) :                                                                                         
    return pd.to_datetime(s).to_datetime64()                                                                                  


s = "2013-01-14 00:00:00"                                                                                       
mls = date_to_millis(s)                                                                                         
dateStr = pd.to_datetime(mls, unit='ms')                                                                        
print(s, " mls = ", mls, "  date = " , dateStr)
# %%
2013-01-14 00:00:00  mls =  2013-01-14T00:00:00.000000000   date =  2013-01-14 00:00:00

print(float(mls))
1.3581216e+18

Answered By: PabTorre

Answer 2

pd.Timestamp.timestamp() gives you a POSIX timestamp as a float. Multiply by 1000 and drop decimals to get Epoch integer milliseconds instead. The example below also makes sure to always operate in UTC to avoid time zone confusions:

from datetime import datetime
import pandas as pd

s = "2013-01-14 00:00:00+00:00"
mls = int(1000 * pd.Timestamp(s).timestamp())
assert mls == 1358121600000
pd_timestamp = pd.to_datetime(mls, unit="ms").tz_localize("UTC")
assert pd_timestamp == pd.Timestamp(datetime(2013, 1, 14), tz="UTC")

Answered By: akaihola

Python Pandas: Convert a date string to milliseconds since epoch and back to date string?

Question:

Answers: