Python requests module sends JSON string instead of x-www-form-urlencoded param string

Question

I was under the impression that POSTSs using x-www-form-urlencoded specifications should send a URL encoded param string in the body of the post. However, when I do this

data = json.dumps({'param1': 'value1', 'param2': 'value2'})
Requests.post(url, data=data)

The body of the request on the receiving end looks like this:

{"param1": "value1", "param2": "value2"}

But I was expecting to get this

param1=value1&param2=value2

How I can get Requests to send the data in the second form?

Asked By: asolberg

||

Source

Answer 1

The reason you’re getting JSON is because you’re explicitly calling json.dumps to generate a JSON string. Just don’t do that, and you won’t get a JSON string. In other words, change your first line to this:

data = {'param1': 'value1', 'param2': 'value2'}

As the docs explain, if you pass a dict as the data value, it will be form-encoded, while if you pass a string, it will be sent as-is.

For example, in one terminal window:

$ nc -kl 8765

In another:

$ python3
>>> import requests
>>> d = {'spam': 20, 'eggs': 3}
>>> requests.post("http://localhost:8765", data=d)
^C
>>> import json
>>> j = json.dumps(d)
>>> requests.post("http://localhost:8765", data=j)
^C

In the first terminal, you’ll see that the first request body is this (and Content-Type application/x-www-form-urlencoded):

spam=20&eggs=3

… while the second is this (and has no Content-Type):

{"spam": 20, "eggs": 3}

Answered By: abarnert

Answer 2

Important to add, it does not work for nested json
So, if you have

# Wrong
data = {'param1': {'a':[100, 200]},
        'param2': 'value2',
        'param3': False}

# You have to convert values into string:
data = {'param1': json.dumps({'a':[100, 200]}),
        'param2': 'value2',
        'param3': json.dumps(False)}

Answered By: NotTooTechy

Answer 3

Short answer with example:

import requests

the_data = {"aaa": 1, "bbb": 2, "ccc": "yeah"}
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
# Execute the post
requests.post("http://bla.bla.example.com", data=the_data, headers=headers)
# You have POSTed this HTTP body: aaa=1&bbb=2&ccc=yeah (note, although the content-type is called urlencoded the data is not in the URL but in the http body)
# to this url: "http://bla.bla.example.com"

Requests library does all the JSON to urlencoded string conversion for you

References:

MDN Web docs, Requests lib post url-encoded form

Answered By: Mercury

Python requests module sends JSON string instead of x-www-form-urlencoded param string

Question:

Answers: