Striptime unconverted data remains

Question

I am trying to make a new list from information that I’m getting from a JSON file but I’m getting this:

Exception has occurred: ValueError
unconverted data remains: 9+01:00
  File "C:Users2913843DesktopVS codeRandom ProjectsTest Folderdatetest.py", line 7, in <lambda>
    newlst = sorted(data,key=lambda x:datetime.datetime.strptime(x['startTime'],'%Y-%m-%dT%H:%M:%S.%f'),reverse=True)
  File "C:Users2913843DesktopVS codeRandom ProjectsTest Folderdatetest.py", line 7, in main
    newlst = sorted(data,key=lambda x:datetime.datetime.strptime(x['startTime'],'%Y-%m-%dT%H:%M:%S.%f'),reverse=True)
  File "C:Users2913843DesktopVS codeRandom ProjectsTest Folderdatetest.py", line 24, in <module>
    main()

The time format in the JSON is:

"startTime":"2023-03-20T07:26:37.5000623+00:00"

and the line in the code for the new list is:

newlst = sorted(data,key=lambda x:datetime.datetime.strptime(x['startTime'],'%Y-%m-%dT%H:%M:%S.%f'),reverse=True)

My desired output would be in this format:

2023-03-30 19:25:13.822945

Asked By: Faubad

||

Source

Answer 1

If you look in the table of format codes, you will see that %f is:

Microsecond as a decimal number, zero-padded to 6 digits.

So you are trying to format a string with a format specification that doesn’t handle all the characters.

You have a few options. Here are 3:

from copy import deepcopy
from datetime import datetime
from dateutil import parser

data = [
    {"startTime": "2023-03-20T07:26:37.5000623+00:00"},
    {"startTime": "2023-03-19T06:02:12.1231233+00:00"},
]

FMT = "%Y-%m-%d %H:%M:%S.%f"


def a(data: list[dict]) -> list[dict]:
    new = deepcopy(data)
    for d in new:
        d["startTime"] = datetime.strptime(
            d["startTime"], "%Y-%m-%dT%H:%M:%S.%f3%z"
        ).strftime(FMT)
    return new


def b(data: list[dict]) -> list[dict]:
    new = deepcopy(data)
    for d in new:
        d["startTime"] = parser.parse(d["startTime"], fuzzy=True).strftime(FMT)
    return new


def c(data: list[dict]) -> list[dict]:
    new = deepcopy(data)
    for d in new:
        d["startTime"] = datetime.strptime(
            d["startTime"][:26], "%Y-%m-%dT%H:%M:%S.%f"
        ).strftime(FMT)
    return new


def d(data: list[dict]) -> list[dict]:
    new = deepcopy(data)
    for d in new:
        d["startTime"] = datetime.fromisoformat(d["startTime"]).strftime(FMT)
    return new


print(sorted(a(data), key=lambda d: d["startTime"]))
print(sorted(b(data), key=lambda d: d["startTime"]))
print(sorted(c(data), key=lambda d: d["startTime"]))
print(sorted(d(data), key=lambda d: d["startTime"]))

Which yields:

[{'startTime': '2023-03-19 06:02:12.123123'}, {'startTime': '2023-03-20 07:26:37.500062'}]
[{'startTime': '2023-03-19 06:02:12.123123'}, {'startTime': '2023-03-20 07:26:37.500062'}]
[{'startTime': '2023-03-19 06:02:12.123123'}, {'startTime': '2023-03-20 07:26:37.500062'}]
[{'startTime': '2023-03-19 06:02:12.123123'}, {'startTime': '2023-03-20 07:26:37.500062'}]

a

The first option adds, a random "3" into the format string, which is no good. Basically, it has to match whatever the 7th millisecond position is, so it will fail 9 of 10 times on average.

b

In option b, the string is being parsed using a fuzzy feature in dateutil.parser.

c

Lastly, if the precision isn’t required, you can just truncate the first 26 digits and use a more simple format string.

d

As Mark noted in the comments above fromisoformat will work in Python 3.11+.

note: These functions all use deepcopy which is probably not efficient and I don’t know your use case. You should just update in place, if you don’t need both the old and new lists.

Also, I’m not sure I recommend using a static format string like this. Usually it is optimal to stick with date.isoformat() or formatting based on locale with %c.

Answered By: theherk

Striptime unconverted data remains

Question:

Answers:

a

b

c

d