TypeError: 'NoneType' object is not subscriptable on GAE instance assigning JSON to python variable

Question:

I’m getting an error TypeError: 'NoneType' object is not subscriptable when assigning JSON to a python variable like this sheet_id = data["sheetID"].
It only happens on my Google App Engine instance. I don’t get it when running my Flask app locally and sending POST requests to the app with Postman. Here’s the code snippet

@app.route('/main',methods=["POST"])
def main():
    data = request.json
    print(data)
    scope = [
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/gmail.send',
    'https://www.googleapis.com/auth/gmail.settings.basic',
    'https://www.googleapis.com/auth/gmail.settings.sharing'
    ]

    GCP_PROJECT = os.environ['GCP_PROJECT']
    GAE_SID_VERSION = os.environ['GAE_SID_VERSION']


    credentials1 = access_secret_version(GCP_PROJECT, <SECRET NAME>, GAE_SID_VERSION)
    credentials1 = json.loads(credentials1)
    credentials = service_account.Credentials.from_service_account_info(credentials1)
    scoped_credentials = credentials.with_scopes(scope)
    gc = gspread.authorize(scoped_credentials)

    #for sheet with categories and score
    sheet_id = data["sheetID"] #this gives me the error

    workbook = gc.open_by_key(sheet_id)
    ...

    ...
    return "OK"

The strange thing is that the whole program runs to completion and I receive response = OK. I’m able to send json data from a triggered script in Google Sheets to my GAE instance, then the GAE instance loads data into a Gcloud storage bucket later. I understand from this link that either data or sheet_id could be None but then in that case how would my full program run to completion or even just open the workbook? Here’s the Google Apps script

function runDockerImage(sheetID , mainWorksheet , emailTo , emailList , deliverToCloud ,filterOn , labelDisplay , sandbox , deliverToSlack) {
  // The code below logs the HTML code of the Google home page.
  var dataJSON = {
    'sheetID' :sheetID,
    'mainWorksheet' : mainWorksheet,
    'emailTo' : emailTo,
    'emailList' : emailList,
    'deliverToCloud' : deliverToCloud,
    'filterOn' : filterOn,
    'labelDisplay': labelDisplay,
    'sandbox' : sandbox,
    'deliverToSlack' : deliverToSlack 
    
  };
  
  payload = JSON.stringify(dataJSON)
  
  var headers = {
    'Accept': 'application/json',
    'Content-Type': 'application/json',
    'Authorization': 'Basic _authcode_'
  };
  
  var options = {
     'method': 'post',
     'contentType': 'application/json',
     'headers': headers,
     'payload': payload
  };
  
  
  var response = UrlFetchApp.fetch('https://<project-id>.uc.r.appspot.com/main', options);
  Logger.log(response);
 
}

Here’s the tracback from App Engine Error Reporting log:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/app/app.py", line 118, in main
    sheet_id = data["sheetID"]
TypeError: 'NoneType' object is not subscriptable

Finally, I’d just like to ask how is it possible that this program runs completely with this error? I think that I might run into some serious problems later if I don’t fix this.

Screenshot of the Error Reporting Log Error Reporting Log

Asked By: Keniboi

||

Answers:

The error means that data is None

$ python3
Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> d=None
>>> d['3']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable

The strange thing is that the whole program runs to completion and I receive response = OK.

Are you sure you don’t have an error handler or something that is returning OK, even though the request handler threw an exception?

Also, why are you doing this?

dataJSON = request.json
json_string = json.dumps(dataJSON)
data = json.loads(json_string)

request.json gives you the json request payload as a python dict, but then you convert it to a json string and then parse it to a python dict again. You should just be able to do data = request.json

Answered By: Alex

I know it been a while for this post, I have similar issue where the reason for this issue was the scaling (think about multiple users accessing your app) this could cause unexpected behavior where the global/local variables could contain wrong information or even could be None. so try to host your app first with only 1 instance and see if that issue resolved. if yes then the reason was due the scalability.

BTW. The solution for this issue to make your app stateless. so don’t use any variables for long time, save your data into another place (e.g. memorystore or any other DB solutions..)

Answered By: Mahdi Asali