Why does HTTP POST request body need to be JSON enconded in Python?

Question

I ran into this issue when playing around with an external API. I was sending my body data as a dictionary straight into the request and was getting 400 errors:

data = {
  "someParamRange": {
    "to": 1000, 
    "from": 100
  }, 
  "anotherParamRange": {
    "to": True, 
    "from": False
  }
}

When I added a json.dumps wrap, it works:

data = json.dumps({
  "someParamRange": {
    "to": 1000, 
    "from": 100
  }, 
  "anotherParamRange": {
    "to": True, 
    "from": False
  }
})

I don’t entirely understand why this is necessary, as dictionaries and JSON objects are syntactically identical. Can someone help me understand what is going on behind the scenes here?

For completeness, here are my headers:

headers = {'API-KEY': 'blerg', 'Accept-Encoding': 'UTF-8', 'Content-Type': 'application/json', 'Accept': '*/*', 'username': 'user', 'password': 'pwd'}

EDIT:

I didn’t mention this earlier but now I feel that it may be relevant. I am using the Python Requests library, and another post seems to suggest that you should never have to encode parameters to a request object: https://stackoverflow.com/a/14804320/1012040

“Regardless of whether GET/POST you never have to encode parameters again, it simply takes a dictionary as an argument and is good to go.”

Seems like serialization shouldn’t be necessary?

My request object:

response = requests.post(url, data=data, headers=headers)

Asked By: acpigeon

||

Source

Answer 1

Although they seem syntatically itentical there is a difference: JSON is a string representation of serialized object; in this case Python dict. In this example you need to send serialized data in a form of string and thus json.dumps is necessary to perform the serialization.

edit

As suggested in comments to the question it is relative to used API, but nevertheless the serialization must be done somewhere along the way to send an object over the wire.

Answered By: Piotr Hajduga

Answer 2

Apparently your API requires JSON-encoded and not form-encoded data. When you pass a dict in as the data parameter, the data is form-encoded. When you pass a string (like the result of json.dumps), the data is not form-encoded.

Consider this quote from the requests documentation:

Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the data argument. Your dictionary of data will automatically be form-encoded when the request is made.

There are many times that you want to send data that is not form-encoded. If you pass in a string instead of a dict, that data will be posted directly.

For example, the GitHub API v3 accepts JSON-Encoded POST/PATCH data:

>>> import json
>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}

>>> r = requests.post(url, data=json.dumps(payload))

Refs:

Answered By: Robᵩ

Why does HTTP POST request body need to be JSON enconded in Python?

Question:

Answers: