TypeError: 'NoneType' object is not subscriptable on GAE instance assigning JSON to python variable
Question:
I’m getting an error TypeError: 'NoneType' object is not subscriptable
when assigning JSON to a python variable like this sheet_id = data["sheetID"]
.
It only happens on my Google App Engine instance. I don’t get it when running my Flask app locally and sending POST requests to the app with Postman. Here’s the code snippet
@app.route('/main',methods=["POST"])
def main():
data = request.json
print(data)
scope = [
'https://www.googleapis.com/auth/spreadsheets',
'https://www.googleapis.com/auth/gmail.send',
'https://www.googleapis.com/auth/gmail.settings.basic',
'https://www.googleapis.com/auth/gmail.settings.sharing'
]
GCP_PROJECT = os.environ['GCP_PROJECT']
GAE_SID_VERSION = os.environ['GAE_SID_VERSION']
credentials1 = access_secret_version(GCP_PROJECT, <SECRET NAME>, GAE_SID_VERSION)
credentials1 = json.loads(credentials1)
credentials = service_account.Credentials.from_service_account_info(credentials1)
scoped_credentials = credentials.with_scopes(scope)
gc = gspread.authorize(scoped_credentials)
#for sheet with categories and score
sheet_id = data["sheetID"] #this gives me the error
workbook = gc.open_by_key(sheet_id)
...
...
return "OK"
The strange thing is that the whole program runs to completion and I receive response = OK
. I’m able to send json data from a triggered script in Google Sheets to my GAE instance, then the GAE instance loads data into a Gcloud storage bucket later. I understand from this link that either data
or sheet_id
could be None but then in that case how would my full program run to completion or even just open the workbook? Here’s the Google Apps script
function runDockerImage(sheetID , mainWorksheet , emailTo , emailList , deliverToCloud ,filterOn , labelDisplay , sandbox , deliverToSlack) {
// The code below logs the HTML code of the Google home page.
var dataJSON = {
'sheetID' :sheetID,
'mainWorksheet' : mainWorksheet,
'emailTo' : emailTo,
'emailList' : emailList,
'deliverToCloud' : deliverToCloud,
'filterOn' : filterOn,
'labelDisplay': labelDisplay,
'sandbox' : sandbox,
'deliverToSlack' : deliverToSlack
};
payload = JSON.stringify(dataJSON)
var headers = {
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': 'Basic _authcode_'
};
var options = {
'method': 'post',
'contentType': 'application/json',
'headers': headers,
'payload': payload
};
var response = UrlFetchApp.fetch('https://<project-id>.uc.r.appspot.com/main', options);
Logger.log(response);
}
Here’s the tracback from App Engine Error Reporting log:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/app/app.py", line 118, in main
sheet_id = data["sheetID"]
TypeError: 'NoneType' object is not subscriptable
Finally, I’d just like to ask how is it possible that this program runs completely with this error? I think that I might run into some serious problems later if I don’t fix this.
Screenshot of the Error Reporting Log
Answers:
The error means that data
is None
$ python3
Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> d=None
>>> d['3']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable
The strange thing is that the whole program runs to completion and I receive response = OK.
Are you sure you don’t have an error handler or something that is returning OK, even though the request handler threw an exception?
Also, why are you doing this?
dataJSON = request.json
json_string = json.dumps(dataJSON)
data = json.loads(json_string)
request.json
gives you the json request payload as a python dict, but then you convert it to a json string and then parse it to a python dict again. You should just be able to do data = request.json
I know it been a while for this post, I have similar issue where the reason for this issue was the scaling (think about multiple users accessing your app) this could cause unexpected behavior where the global/local variables could contain wrong information or even could be None. so try to host your app first with only 1 instance and see if that issue resolved. if yes then the reason was due the scalability.
BTW. The solution for this issue to make your app stateless. so don’t use any variables for long time, save your data into another place (e.g. memorystore or any other DB solutions..)
I’m getting an error TypeError: 'NoneType' object is not subscriptable
when assigning JSON to a python variable like this sheet_id = data["sheetID"]
.
It only happens on my Google App Engine instance. I don’t get it when running my Flask app locally and sending POST requests to the app with Postman. Here’s the code snippet
@app.route('/main',methods=["POST"])
def main():
data = request.json
print(data)
scope = [
'https://www.googleapis.com/auth/spreadsheets',
'https://www.googleapis.com/auth/gmail.send',
'https://www.googleapis.com/auth/gmail.settings.basic',
'https://www.googleapis.com/auth/gmail.settings.sharing'
]
GCP_PROJECT = os.environ['GCP_PROJECT']
GAE_SID_VERSION = os.environ['GAE_SID_VERSION']
credentials1 = access_secret_version(GCP_PROJECT, <SECRET NAME>, GAE_SID_VERSION)
credentials1 = json.loads(credentials1)
credentials = service_account.Credentials.from_service_account_info(credentials1)
scoped_credentials = credentials.with_scopes(scope)
gc = gspread.authorize(scoped_credentials)
#for sheet with categories and score
sheet_id = data["sheetID"] #this gives me the error
workbook = gc.open_by_key(sheet_id)
...
...
return "OK"
The strange thing is that the whole program runs to completion and I receive response = OK
. I’m able to send json data from a triggered script in Google Sheets to my GAE instance, then the GAE instance loads data into a Gcloud storage bucket later. I understand from this link that either data
or sheet_id
could be None but then in that case how would my full program run to completion or even just open the workbook? Here’s the Google Apps script
function runDockerImage(sheetID , mainWorksheet , emailTo , emailList , deliverToCloud ,filterOn , labelDisplay , sandbox , deliverToSlack) {
// The code below logs the HTML code of the Google home page.
var dataJSON = {
'sheetID' :sheetID,
'mainWorksheet' : mainWorksheet,
'emailTo' : emailTo,
'emailList' : emailList,
'deliverToCloud' : deliverToCloud,
'filterOn' : filterOn,
'labelDisplay': labelDisplay,
'sandbox' : sandbox,
'deliverToSlack' : deliverToSlack
};
payload = JSON.stringify(dataJSON)
var headers = {
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': 'Basic _authcode_'
};
var options = {
'method': 'post',
'contentType': 'application/json',
'headers': headers,
'payload': payload
};
var response = UrlFetchApp.fetch('https://<project-id>.uc.r.appspot.com/main', options);
Logger.log(response);
}
Here’s the tracback from App Engine Error Reporting log:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/app/app.py", line 118, in main
sheet_id = data["sheetID"]
TypeError: 'NoneType' object is not subscriptable
Finally, I’d just like to ask how is it possible that this program runs completely with this error? I think that I might run into some serious problems later if I don’t fix this.
Screenshot of the Error Reporting Log
The error means that data
is None
$ python3
Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> d=None
>>> d['3']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable
The strange thing is that the whole program runs to completion and I receive response = OK.
Are you sure you don’t have an error handler or something that is returning OK, even though the request handler threw an exception?
Also, why are you doing this?
dataJSON = request.json
json_string = json.dumps(dataJSON)
data = json.loads(json_string)
request.json
gives you the json request payload as a python dict, but then you convert it to a json string and then parse it to a python dict again. You should just be able to do data = request.json
I know it been a while for this post, I have similar issue where the reason for this issue was the scaling (think about multiple users accessing your app) this could cause unexpected behavior where the global/local variables could contain wrong information or even could be None. so try to host your app first with only 1 instance and see if that issue resolved. if yes then the reason was due the scalability.
BTW. The solution for this issue to make your app stateless. so don’t use any variables for long time, save your data into another place (e.g. memorystore or any other DB solutions..)