python requests gives 'None' response, where json data is expected
Question:
Firstly, I should add that you can find this request by doing the following:
1- Go to [airline site][1]
2- Type in "From" = "syd"
3- Type in "To" = "sin"
4- Make the departure date sep.3 and click one-way and search
5- On the search result page check your network get request when you click on an available seat option radio button
I’m trying to use the requests module to get the response for example from this site
And this is what I’m trying:
url = "http://www.singaporeair.com/chooseFlightJson.form?"
payload = {'selectedFlightIdDetails[0]':amount_data,'hid_flightIDs':'','hid_recommIDs':'','tripType':"O",'userPreferedCurrency':""}
response = requests.get(url, params=payload)
print response.json()
The response is supposed to be:
{"price":{"total":"595.34","currency":{"code":"AUD","label":""},"adult":{"count":1,"label":"Adult","cost":"328.00","total":"328.00"},"child":{"count":0,"label":"Child","cost":"0.00","total":"0.00"},"infant":{"count":0,"label":"Infant","cost":"0.00","total":"0.00"},"addOns":[{"label":"Airport / Government taxes ","cost":"83.24"},{"label":"Carrier Surcharges","cost":"184.10"}],"disclaimer":"Prices are shown in Canadian Dollar(CAD)","rate":"595.34 AUD u003d 913.80 CAD","ratehint":"Estimated","unFormattedTotal":"595.34"},"messages":{"O3FF11SYD":"A few seats left","O1FF31SYD":" ","R0FF31SYD":"A few seats left","O2FF31SYD":"A few seats left","O0FF31SYD":" ","O1FF11SYD":"A few seats left","O0FF21SYD":" ","O2FF21SYD":" ","O3FF21SYD":" ","O1FF21SYD":" "},"cabinClass":{"onwardCabin":["Economy"]}}
Answers:
The response is the value None, encoded in JSON; the server returns nullrn, which means the same as None in Python.
The content type is wrong here; it is set to text/html, but the response.json() return value is entirely correct for what the server sent:
>>> import requests
>>> url = "http://www.singaporeair.com/chooseFlightJson.form?"
>>> amount_data = 0
>>> payload = {'selectedFlightIdDetails[0]':amount_data,'hid_flightIDs':'','hid_recommIDs':'','tripType':"O",'userPreferedCurrency':""}
>>> response = requests.get(url, params=payload)
>>> response
<Response [200]>
>>> response.headers['content-type']
'text/html; charset=ISO-8859-1'
>>> response.text
'nullrn'
>>> response.json() is None
True
The solution is to use requests session, like so:
session = requests.Session()
Then to call all of the urls you need to simply do:
response = session.get(url)
This sets the cookies and session variables which are necessary to retrieve the data.
I have seen jsessionid used in different ways, in the url, or in this case in a cookie. But there are probably other session info that are required, which is taken care of by a requests session object.
while making http request make sure that response_type set to exact use case you are trying with.
In my case response_type=’object’ worked to eliminate None type in response.
change protocol from http to https.
url = "https://www.singaporeair.com/chooseFlightJson.form?"
Firstly, I should add that you can find this request by doing the following:
1- Go to [airline site][1]
2- Type in "From" = "syd"
3- Type in "To" = "sin"
4- Make the departure date sep.3 and click one-way and search
5- On the search result page check your network get request when you click on an available seat option radio button
I’m trying to use the requests module to get the response for example from this site
And this is what I’m trying:
url = "http://www.singaporeair.com/chooseFlightJson.form?"
payload = {'selectedFlightIdDetails[0]':amount_data,'hid_flightIDs':'','hid_recommIDs':'','tripType':"O",'userPreferedCurrency':""}
response = requests.get(url, params=payload)
print response.json()
The response is supposed to be:
{"price":{"total":"595.34","currency":{"code":"AUD","label":""},"adult":{"count":1,"label":"Adult","cost":"328.00","total":"328.00"},"child":{"count":0,"label":"Child","cost":"0.00","total":"0.00"},"infant":{"count":0,"label":"Infant","cost":"0.00","total":"0.00"},"addOns":[{"label":"Airport / Government taxes ","cost":"83.24"},{"label":"Carrier Surcharges","cost":"184.10"}],"disclaimer":"Prices are shown in Canadian Dollar(CAD)","rate":"595.34 AUD u003d 913.80 CAD","ratehint":"Estimated","unFormattedTotal":"595.34"},"messages":{"O3FF11SYD":"A few seats left","O1FF31SYD":" ","R0FF31SYD":"A few seats left","O2FF31SYD":"A few seats left","O0FF31SYD":" ","O1FF11SYD":"A few seats left","O0FF21SYD":" ","O2FF21SYD":" ","O3FF21SYD":" ","O1FF21SYD":" "},"cabinClass":{"onwardCabin":["Economy"]}}
The response is the value None, encoded in JSON; the server returns nullrn, which means the same as None in Python.
The content type is wrong here; it is set to text/html, but the response.json() return value is entirely correct for what the server sent:
>>> import requests
>>> url = "http://www.singaporeair.com/chooseFlightJson.form?"
>>> amount_data = 0
>>> payload = {'selectedFlightIdDetails[0]':amount_data,'hid_flightIDs':'','hid_recommIDs':'','tripType':"O",'userPreferedCurrency':""}
>>> response = requests.get(url, params=payload)
>>> response
<Response [200]>
>>> response.headers['content-type']
'text/html; charset=ISO-8859-1'
>>> response.text
'nullrn'
>>> response.json() is None
True
The solution is to use requests session, like so:
session = requests.Session()
Then to call all of the urls you need to simply do:
response = session.get(url)
This sets the cookies and session variables which are necessary to retrieve the data.
I have seen jsessionid used in different ways, in the url, or in this case in a cookie. But there are probably other session info that are required, which is taken care of by a requests session object.
while making http request make sure that response_type set to exact use case you are trying with.
In my case response_type=’object’ worked to eliminate None type in response.
change protocol from http to https.
url = "https://www.singaporeair.com/chooseFlightJson.form?"