Python Polars Parse Date from Epoch

Question

How does one convert a column of i64 epoch strings into dates in polars?

I’ve got a column of i64 representing seconds since epoch and I’d like to parse them into polars native datetimes.

Asked By: Test

||

Answer 1

Polars’ Datetime is represented as unix epoch in either, nanoseconds, microseconds or milliseconds. So with that knowledge we can convert the seconds to milliseconds and cast to Datetime.

Finally we ensure polars uses the proper unit.

df = pl.DataFrame({
    "epoch_seconds": [1648457740, 1648457740 + 10]
})

MILLISECONDS_IN_SECOND = 1000;

df.select(
    (pl.col("epoch_seconds") * MILLISECONDS_IN_SECOND).cast(pl.Datetime).dt.with_time_unit("ms").alias("datetime")
)

shape: (2, 1)
┌─────────────────────┐
│ datetime            │
│ ---                 │
│ datetime[ms]        │
╞═════════════════════╡
│ 2022-03-28 08:55:40 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 2022-03-28 08:55:50 │
└─────────────────────┘

Answered By: ritchie46

Answer 2

This is for node.js, but I felt like this is still a good place to post this since I was looking here for help when doing it in node, and because the apis are similar.

const startTime = new Date('2022')
const endTime = new Date('2023')

df = pl.DataFrame().withColumns(
    pl
        .arange(startTime, endTime, 15 * 60 * 1000)
        .cast(pl.DataType.Datetime("ms"))
        .alias("datetime")
)

Somehow I had to use pl.DataType.Datetime instead of pl.Datetime, which is a bit different from what can be seen in tests: https://github.com/pola-rs/nodejs-polars/blob/main/tests/setup.ts ¯_(ツ)_/¯ but it works

edit: Having to use pl.DataType.Datetime is a side-effect of a specific project configuration. See https://github.com/pola-rs/nodejs-polars/issues/55 for more details

Answered By: sezanzeb

Python Polars Parse Date from Epoch

Question:

Answers: