Flask not processing other HTTP requests after Chrome browser accesses the web-site
Question:
Problem statement: my web server on Flask is not processing HTTP requests after I try to access non existing file from Chrome browser. When Chrome is shut down or accessing page that does exists, the backlog HTTP requests are processed at once.
Impact: web server availability is poor.
Question: why does it happen and how to fix it without running Flask in threaded mode?
The closest post I found online is this: github.com/pallets/flask/issues/2188
but could not find exactly the same problem and solution. Looking forward to your thoughts – thanks so much for your help!
Primary Hypothesis: Chrome does not read all content of the 404 response, and Flask is waiting all the content to be read
Details:
Steps to reproduce the problem:
1) run minimal Flask application (http://flask.pocoo.org/docs/0.12/quickstart/):
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'Hello, World!'
app.run()
- Running on хттп://127.0.0.1:5000/ (Press CTRL+C to quit)
2) verify you get ‘Hello world’ response in browser or curl:
curl -v localhost:5000/
3) in Chrome go to localhost:5000/pagethatdoesnotexists
observe Not Found error in the browser
4) repeat curl -v localhost:5000/ command
observe that connection is established but response is not received
e.g. :
curl -v localhost:5000/
* Trying ::1…
* Connection failed
* connect to ::1 port 5000 failed: Connection refused
* Trying 127.0.0.1…
* Connected to localhost (127.0.0.1) port 5000 (#0)
GET / HTTP/1.1
Host: localhost:5000
User-Agent: curl/7.49.0
Accept: /
5) in Chrome go to the page that exists or shut down Chrome
observe immediate response to curl:
- HTTP 1.0, assume close after body < HTTP/1.0 200 OK < Content-Type: text/html; charset=utf-8 < Content-Length: 13 < Server:
Werkzeug/0.11.10 Python/3.5.1 < Date: Tue, 28 Feb 2017 21:44:20 GMT <
- Closing connection 0 Hello, World!
it may take more than one attempt to reproduce the problem. Typically happens >8 times out of 10
Other pieces of information:
1) instead of curl I can use Safari or telnet or a python script – same problem
2) Safari does not create the problem, which Chrome does
3) tried to mimic Chrome by sending exactly the same http request as Chrome does – but cannot reproduce the problem. Chrome probably does something else.
4) when I run Flask in threaded mode (handling each request with a different thread), the problem goes away.
5) versions:
Chrome Version 56.0.2924.87 (64-bit)
Python 3.5.2 |Anaconda 4.1.1 (64-bit)| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
flask.version
‘0.11.1’
6) issue is also reproduced on AWS Ubuntu production server machine
7) tried sending custom headers in 404 http response with no luck
@app.errorhandler(404)
def page_not_found(e):
# return render_template('404.html'), 404
resp = make_response(render_template('404.html'), 404)
# resp.headers['Connection'] = 'close'
resp.headers['Cache-Control'] = 'no-cache, no-store'
return resp
UPDATE
I was able to reproduce the problem without 404 error, just with normal http requests from Chrome. There are no errors in Flask log observed.
Here is the video with the problem demonstration
Another interesting thing – if using incognito window in Chrome browser, the problem is not observed. Yet, clearing cache in normal mode Chrome does not solve the problem.
Answers:
I’ve ran into the same problem twice.
The same environment: pure Flask (no reverse proxy), the simplest application.
After you’ve open URL with Chrome/Chromium — Flask will hang and won’t respond to other clients (curl, postman, firefox, python-request, ..).
Workaround for Chrome
Disable URL-prediction services in Chrome/Chromium (Actual names of options are on the screenshot)
Real solution (for Flask)
Comming soon — contributions are welcome! (most probably I will never resolve this).
Enable threading.
app.run(host='0.0.0.0', port=80, debug=True, threaded=True)
TL;DR
The problem is still valid. It seems that Chrome does not close the connection when page prefetch is enabled, and it blocks the execution of the server, hence the processing of subsequent requests.
In my case, the problem even worst since Android-based phones also use this prefetch feature with the same results, and I can not change the settings every client.
My solution/workaround is to enable the threading option in the underlying werkzeug server (https://werkzeug.palletsprojects.com/en/0.16.x/serving/#werkzeug.serving.run_simple). Of course, it is more resources heavy on the server-side, but it allows us to separate the ill behaving requests/clients in a separate thread without blocking other requests.
if __name__ == '__main__':
logger.info('starting web server life cycle')
app.run(host='0.0.0.0', port=80, debug=True, threaded=True)
I also checked that the request processing is finished correctly, and it does, at least in the Flask side. So the problem must be either in Chrome / other clients or in the underlying werkzeug server.
@app.before_request
def filter_prefetch():
logger.debug("before request")
logger.debug(request.headers)
# uncomment these to filter Chrome specific prefetch requests.
# if 'Purpose' in request.headers and request.headers.get('Purpose') == 'prefetch':
# logger.debug("prefetch requests are not allowed")
# return '', status.HTTP_403_FORBIDDEN
@app.after_request
def debug_after(response):
logger.debug("after request")
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
response.headers["Pragma"] = "no-cache"
response.headers["Expires"] = "0"
response.headers['Cache-Control'] = 'public, max-age=0'
response.headers['Connection'] = 'close'
return response
Problem statement: my web server on Flask is not processing HTTP requests after I try to access non existing file from Chrome browser. When Chrome is shut down or accessing page that does exists, the backlog HTTP requests are processed at once.
Impact: web server availability is poor.
Question: why does it happen and how to fix it without running Flask in threaded mode?
The closest post I found online is this: github.com/pallets/flask/issues/2188
but could not find exactly the same problem and solution. Looking forward to your thoughts – thanks so much for your help!
Primary Hypothesis: Chrome does not read all content of the 404 response, and Flask is waiting all the content to be read
Details:
Steps to reproduce the problem:
1) run minimal Flask application (http://flask.pocoo.org/docs/0.12/quickstart/):
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'Hello, World!'
app.run()
- Running on хттп://127.0.0.1:5000/ (Press CTRL+C to quit)
2) verify you get ‘Hello world’ response in browser or curl:
curl -v localhost:5000/
3) in Chrome go to localhost:5000/pagethatdoesnotexists
observe Not Found error in the browser
4) repeat curl -v localhost:5000/ command
observe that connection is established but response is not received
e.g. :
curl -v localhost:5000/
* Trying ::1…
* Connection failed
* connect to ::1 port 5000 failed: Connection refused
* Trying 127.0.0.1…
* Connected to localhost (127.0.0.1) port 5000 (#0)GET / HTTP/1.1
Host: localhost:5000
User-Agent: curl/7.49.0
Accept: /
5) in Chrome go to the page that exists or shut down Chrome
observe immediate response to curl:
- HTTP 1.0, assume close after body < HTTP/1.0 200 OK < Content-Type: text/html; charset=utf-8 < Content-Length: 13 < Server:
Werkzeug/0.11.10 Python/3.5.1 < Date: Tue, 28 Feb 2017 21:44:20 GMT <- Closing connection 0 Hello, World!
it may take more than one attempt to reproduce the problem. Typically happens >8 times out of 10
Other pieces of information:
1) instead of curl I can use Safari or telnet or a python script – same problem
2) Safari does not create the problem, which Chrome does
3) tried to mimic Chrome by sending exactly the same http request as Chrome does – but cannot reproduce the problem. Chrome probably does something else.
4) when I run Flask in threaded mode (handling each request with a different thread), the problem goes away.
5) versions:
Chrome Version 56.0.2924.87 (64-bit)
Python 3.5.2 |Anaconda 4.1.1 (64-bit)| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
flask.version
‘0.11.1’
6) issue is also reproduced on AWS Ubuntu production server machine
7) tried sending custom headers in 404 http response with no luck
@app.errorhandler(404)
def page_not_found(e):
# return render_template('404.html'), 404
resp = make_response(render_template('404.html'), 404)
# resp.headers['Connection'] = 'close'
resp.headers['Cache-Control'] = 'no-cache, no-store'
return resp
UPDATE
I was able to reproduce the problem without 404 error, just with normal http requests from Chrome. There are no errors in Flask log observed.
Here is the video with the problem demonstration
Another interesting thing – if using incognito window in Chrome browser, the problem is not observed. Yet, clearing cache in normal mode Chrome does not solve the problem.
I’ve ran into the same problem twice.
The same environment: pure Flask (no reverse proxy), the simplest application.
After you’ve open URL with Chrome/Chromium — Flask will hang and won’t respond to other clients (curl, postman, firefox, python-request, ..).
Workaround for Chrome
Disable URL-prediction services in Chrome/Chromium (Actual names of options are on the screenshot)
Real solution (for Flask)
Comming soon — contributions are welcome! (most probably I will never resolve this).
Enable threading.
app.run(host='0.0.0.0', port=80, debug=True, threaded=True)
TL;DR
The problem is still valid. It seems that Chrome does not close the connection when page prefetch is enabled, and it blocks the execution of the server, hence the processing of subsequent requests.
In my case, the problem even worst since Android-based phones also use this prefetch feature with the same results, and I can not change the settings every client.
My solution/workaround is to enable the threading option in the underlying werkzeug server (https://werkzeug.palletsprojects.com/en/0.16.x/serving/#werkzeug.serving.run_simple). Of course, it is more resources heavy on the server-side, but it allows us to separate the ill behaving requests/clients in a separate thread without blocking other requests.
if __name__ == '__main__':
logger.info('starting web server life cycle')
app.run(host='0.0.0.0', port=80, debug=True, threaded=True)
I also checked that the request processing is finished correctly, and it does, at least in the Flask side. So the problem must be either in Chrome / other clients or in the underlying werkzeug server.
@app.before_request
def filter_prefetch():
logger.debug("before request")
logger.debug(request.headers)
# uncomment these to filter Chrome specific prefetch requests.
# if 'Purpose' in request.headers and request.headers.get('Purpose') == 'prefetch':
# logger.debug("prefetch requests are not allowed")
# return '', status.HTTP_403_FORBIDDEN
@app.after_request
def debug_after(response):
logger.debug("after request")
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
response.headers["Pragma"] = "no-cache"
response.headers["Expires"] = "0"
response.headers['Cache-Control'] = 'public, max-age=0'
response.headers['Connection'] = 'close'
return response