polars native way to convert unix timestamp to date

Question

I’m working with some data frames that contain Unix epochs in ms, and would like to display the entire timestamp series as a date. Unfortunately, the docs did not help me find a polars native way to do this, and I’m reaching out here. Solutions on how to do this in Python and also in Rust would brighten my mind and day.

With pandas, for example, such things were possible:

pd.to_datetime(pd_df.timestamp, unit="ms")
# or to convert the whole col
pd_df.timestamp = pd.to_datetime(pd_df.timestamp, unit="ms")

I could loop over the whole thing and do something like I’m doing here for a single entry in each row.

datetime.utcfromtimestamp(pl_df["timestamp"][0] / 1000).strftime("%Y-%m-%d")

If I were to do this in Rust, I would then use something like chrono to convert the ts to a date. But I don’t think looping over each row is a good solution.

For now, as the best way I have found to help me is to convert pd_df = pl_df.to_pandas() and do it in pandas.

Asked By: tenxsoydev

||

Source

Answer 1

Update: See from_epoch() in @ritchie46’s answer

One way is to .cast(pl.Datetime(time_unit="ms"))

>>> df = pl.DataFrame({"timestamp": [1397392146866, 1671225446800]})
>>> df
shape: (2, 1)
┌───────────────┐
│ timestamp     │
│ ---           │
│ i64           │
╞═══════════════╡
│ 1397392146866 │
├───────────────┤
│ 1671225446800 │
└─//────────────┘
>>> df.with_column(pl.col("timestamp").cast(pl.Datetime(time_unit="ms")))
shape: (2, 1)
┌─────────────────────────┐
│ timestamp               │
│ ---                     │
│ datetime[ms]            │
╞═════════════════════════╡
│ 2014-04-13 12:29:06.866 │
├─────────────────────────┤
│ 2022-12-16 21:17:26.800 │
└─//──────────────────────┘

Answered By: jqurious

Answer 2

Adapting the answer of @jqurious. Polars has a dedicated from_epoch function for this:

(pl.DataFrame({"timestamp": [1397392146866, 1671225446800]})
   .with_column(
       pl.from_epoch("timestamp", unit="ms")
   )
)

shape: (2, 1)
┌─────────────────────────┐
│ timestamp               │
│ ---                     │
│ datetime[ms]            │
╞═════════════════════════╡
│ 2014-04-13 12:29:06.866 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 2022-12-16 21:17:26.800 │
└─────────────────────────┘

Answered By: ritchie46

polars native way to convert unix timestamp to date

Question:

Answers: