Unable to share global variable between threads with flask

Question:

Context

I’m trying to write a python flask server that answers a simple request. The data to be returned in the response is queried from a backend service. Since this query may take some time to complete, I don’t want to do it synchronously, I want to have it run periodically, in a background thread, where I can explicitly control the frequency. It should update some data structure that is shared with the flask view function, so the GET requests gets its answer from the shared data.

I’m including two sample codes below. In both codes, cnt is a global variable, starting at 0, and increased inside a separate thread. The index.html file displays the value of cnt:

<h1>cnt = {{ cnt }}</h1>

The issue I’m facing

When the view function is inside the same module, it works: every time I refresh the page with F5, the value of cnt has changed, I can see it increasing.

But when I put the view functions in a separate routes module (which I import at the end of my hello.py file), it no longer works: I can see in the server traces that cnt is being increased by the background thread, but when I refresh the page I always see

cnt = 1

It’s as if I now have two different copies of the cnt variable, even though the variable has been imported into the routes module.

Note

I’ve found countless question on SO on this topic, but none that really addresses this specific concern. Also, I’m perfectly aware that in my examples below, there is no lock protecting the shared data (which is a simple cnt variable) and I’m not handling thread termination. This is being deliberately ignored for now, in order to keep the sample code minimal.

Here are the codes.

Single module, it works

Here’s the main hello.py file, with everything inside the same module:

from flask import Flask, render_template
import threading as th
from time import sleep

cnt = 0

app = Flask(__name__)

# Run in background thread
def do_stuff():
    global cnt
    while True:
        cnt += 1
        print(f'do_stuff: cnt={cnt}')
        sleep(1)

# Create the separate thread
t = th.Thread(target=do_stuff)
t.start()

@app.route("/")
def hello():
    return render_template('index.html', cnt=cnt)

The variable is indeed shared between the background thread and the view function, as expected.

Separate modules, it no longer works

Here’s the main hello.py module, without the view function :

from flask import Flask
import threading as th
from time import sleep

cnt = 0

app = Flask(__name__)

# Run in background thread
def do_stuff():
    global cnt
    while True:
        cnt += 1
        print(f'do_stuff: cnt={cnt}')
        sleep(1)

# Create the separate thread
t = th.Thread(target=do_stuff)
t.start()

import routes

And here is the separate routes.py file (see import at the end of hello.py above):

# routes.py

from hello import app, cnt 
from flask import render_template

@app.route("/")
def hello():
    return render_template('index.html', cnt=cnt)

With this code, the web page always displays cnt = 1, as if the two modules had two distinct instances of the cntvariable.

I feel like I’m missing some basic insight into python modules, or threads, or their interaction. Any help will be appreciated, and my apologies for such a long question.

Asked By: joao

||

Answers:

globals are not shared between modules

when you say

from moduleA import count you have imported a non mutable number

if something in another file changes count in module A it will not update anything else… its overwritten that count you imported with a new variable named count effectively

if count was a mutable object than any changes would indeed reflect back … so if it was a list or dictionary or a class with fields etc..

instead you can import the module which is mutable

import moduleA

now when moduleA changes count it is changing the mutable module field and you can see it in another place by printing moduleA.count

Answered By: Joran Beasley