Question 1

The question then is, how do we get the workers to share and operate on the same object?

TL;DR

While you could share objects via something like multiprocessing, in your use case you’re probably better off using a cache, like Redis.

Explanation

I’m no expert at all in parallel/concurrent applications, but I do know that unless you need to speed up really expensive CPU-bound operations (i.e. very complex and/or long runnning calculations), you DO NOT want to share objects between processes.

You CAN do that, via dedicated libraries and modules, however it will make your app significantly more complex, having to handle all the possible race conditions and edge cases intrinsic with parallelism. If you do want to go that route, I’m sure there are plenty of libraries and tools, but you should first take a look at multiprocessing, the standard python library for handling parallelism. Check also this and this about using it to share resources among workers with gunicorn.

On the other hand, your use case doesn’t look like it requires very complex calculations, so I would suggest to use a simple cache to act as the "data hub" for your workers, instead of a class. It will give you the desired result of having a single source of truth for your processes without the complexities of shared memory.

If you want to give this approach a try, I suggest to give a look at Redis which is a very popular and well supported solution for a cache and can even persist the data if you want.

Here’s a list of Redis clients for python. redis-py is the recommended one.

Question 2

As a side question, the problem above affects the /reset endpoint too. If this endpoint is called then only one of the workers will reset its object. Is there a way to force all workers to respond to a single call on an endpoint?

If you use a cache the problem disappears. You only have a single source of truth and you just erase the data there, with whichever worker responds to the request. Then every worker will see the data has been reset.

You can create architecture without needing any external library or adding any extra complexity with databases etc.

This will be our server for sharing objects across different processes.

from multiprocessing.managers import SyncManager


class MyManager(SyncManager):
    pass

syncdict = {}

def get_dict():
    return syncdict

if __name__ == "__main__":
    MyManager.register("syncdict", get_dict)
    manager = MyManager(("127.0.0.1", 5000), authkey=b"password")
    manager.start()
    input()
    manager.shutdown()

Name this file as server.py and run this on a different process. Just python server.py should do good.

Let’s skip to our client implementation.

This will be our client implementation.

from multiprocessing.managers import SyncManager
from typing import Optional, Dict, Any, Union


class MyManager(SyncManager):
    ...


class Meta:
    def __init__(self, *, port: int) -> None:
        self.manager = MyManager(("127.0.0.1", port), authkey=b"password")
        self.manager.connect()
        MyManager.register("syncdict")

        self.syndict = self.manager.syncdict()

    def update(self, kwargs: Dict[Any, Any]) -> None:
        self.syndict.update(kwargs)

    def increase_one(self, key: str) -> None:
        self.syndict.update([(key, self.syndict.get(key) + 1)])

    def report(self, item: Union[str, int]) -> int:
        return self.syndict.get(item)


meta = Meta(port=5000)

Let’s merge it with our API.

from fastapi import FastAPI, status

from multiprocessing.managers import SyncManager
from typing import Optional, Dict, Any, Union


class MyManager(SyncManager):
    ...


class Meta:
    def __init__(self, *, port: int, **kwargs: Dict[Any, Any]):
        self.manager = MyManager(("127.0.0.1", port), authkey=b"password")
        self.manager.connect()
        MyManager.register("syncdict")

        self.syndict = self.manager.syncdict()
        self.syndict.update(**kwargs)

    def increase_one(self, key: str):
        self.syndict.update([(key, self.syndict.get(key) + 1)])

    def reset(self, key: str):
        self.syndict.update([(key, 0)])

    def report(self, item: Union[str, int]):
        return self.syndict.get(item)


app = FastAPI()

meta = Meta(port=5000, cnt=0)

# increases the count variable in the meta object by 1
@app.get("/increment")
async def increment(key: str):
    meta.increase_one(key)
    return status.HTTP_200_OK


# returns a json containing the current count from the meta object
@app.get("/report")
async def report(key: str):
    return {"count": meta.report(key)}


# resets the count in the meta object to 0
@app.get("/reset")
async def reset(key: str):
    meta.reset(key)
    return status.HTTP_200_OK

I’m gonna start two instances of our API, one will be on 8000, the other on 8001.

In: curl -X GET "http://127.0.0.1:8000/report?key=cnt"
Out: {"count": 0}

In: curl -X GET "http://127.0.0.1:8001/report?key=cnt"
Out: {"count": 0}

Both started with 0 value. Now let’s increment it

for _ in {1..10}; do curl -X GET "http://127.0.0.1:8000/increment?key=cnt" &; done

I ran the curl on port 8000, 10 times which means cnt should be 10.

Let’s check it out from port 8001:

In: curl -X GET "http://127.0.0.1:8001/report?key=cnt" 
Out: {"cnt": 10}

Working like a charm.

There are two things the consider.

1. You should start your app in a different process. More specifically, the uvicorn my_app:app and your server shouldn’t be parent processes.
2. You might want to add something like graceful shutdowns. Since this is a very simple but highly extensible example.

If you run your FastAPI service using a setup with gunicorn and uvicorn as is described in the docs you can employ the method described here by Yagiz Degimenci in a simpler way. You can use gunicorn’s --preload setting in combination with multiprocessing.Manager in order to avoid the necessity to start another server. In particular the following does need no extra setup to make it work in a single Docker Container.

import logging
from multiprocessing import Manager

manager = Manager()

store = manager.dict()

store["count"] = 0

from fastapi import FastAPI

app = FastAPI()


@app.post("/increment")
async def increment():
    store["count"] = store["count"] + 1


@app.get("/count")
async def get_count():
    return store["count"]


@app.on_event("startup")
async def startup_event():
    uv_logger = logging.getLogger("uvicorn.access")
    handler = logging.StreamHandler()
    handler.setFormatter(
        logging.Formatter(
            "%(process)d - %(processName)s - %(asctime)s - %(levelname)s - %(message)s"
        )
    )
    uv_logger.addHandler(handler)

Save this is demo.py and run via (you need fastapi, guvicorn and uvicorn libraries):

GUNICORN_CMD_ARGS="--bind=127.0.0.1 --workers=3 --preload --access-logfile=-" gunicorn -k uvicorn.workers.UvicornWorker demo:app

(the --preload is essential here!)

Try incrementing via the OpenApi UI at http://localhost:8000/docs and compare multiple calls to the /count endpoint with the process ids in access log output to see that it returns the incremented value regardless of which worker process is responding.

Note: I do not make any claims about thread / async safety here and this method should probably not be employed in production services. In case of any doubt you should always rely on a proper database / caching / memory store solution for production setups. I myself only use this in demo code!

There is a lib called UltraDict:

UltraDict uses multiprocessing.shared_memory to synchronize a dict between multiple processes.

It does so by using a stream of updates in a shared memory buffer. This is efficient because only changes have to be serialized and transferred.

You could just pip install UltraDict

Main advantage is that you don’t have to run two scripts and have less code overall than in answers with multiprocessing.Manager

Example solution could look like this:

from multiprocessing.managers import SyncManager
from typing import Any, Dict, Optional, Union

import uvicorn

from fastapi import FastAPI, status
from UltraDict import UltraDict

class Meta:
    def __init__(self, **kwargs: Dict[Any, Any]):
        self.ultradict = UltraDict(name='fastapi_dict')
        self.ultradict.update(**kwargs)

    def increase_one(self, key: str):
        self.ultradict.update([(key, self.ultradict.get(key) + 1)])

    def reset(self, key: str):
        self.ultradict.update([(key, 0)])

    def report(self, item: Union[str, int]):
        return self.ultradict.get(item)


app = FastAPI()
meta = Meta(cnt=0)


# increases the count variable in the meta object by 1
@app.get('/increment')
async def increment(key: str):
    meta.increase_one(key)
    return status.HTTP_200_OK

# returns a json containing the current count from the meta object
@app.get('/report')
async def report(key: str):
    return {'count': meta.report(key)}

# resets the count in the meta object to 0
@app.get('/reset')
async def reset(key: str):
    meta.reset(key)
    return status.HTTP_200_OK

Let’s run it and verify if it’s working:

uvicorn main:app --workers 5

curl -X GET "http://127.0.0.1:8000/report?key=cnt"
#> {"count":0}

$ for _ in {1..10} ; do curl -X GET "http://127.0.0.1:8000/increment?key=cnt" & done
#> Big bash stuff

$ curl -X GET "http://127.0.0.1:8000/report?key=cnt"
#> {"count":10}

Also let’s check fastapi logs:

INFO:     Application startup complete.
INFO:     127.0.0.1:19122 - "GET /report?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19123 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19124 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19125 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19126 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19127 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19128 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19129 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19130 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19131 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19132 - "GET /increment?key=cnt HTTP/1.1" 200 OK
INFO:     127.0.0.1:19144 - "GET /report?key=cnt HTTP/1.1" 200 OK

You could see that different workers are used (look at ports), and everything works as expected

I had this exact problem and I was able to share memory across workers by using sqlite. I created a KeyValue store that uses the same api as a dict. People say that this is simpler than installing and running a Redis process.

pip install keyvalue-sqlite

You can use it like so:

from keyvalue_sqlite import KeyValueSqlite

DB_PATH = '/path/to/db.sqlite'

db = KeyValueSqlite(DB_PATH, 'table-name')
# Now use standard dictionary operators
db.set_default('0', '1')
actual_value = db.get('0')
assert '1' == actual_value
db.set_default('0', '2')
assert '1' == db.get('0')

You can store any kind of json structure in the database.

More api docs can be found here:
https://github.com/zackees/keyvalue_sqlite