FastAPI – ValueError: raw stream has been detached
Question:
My API is working properly from the client’s side, as it returns the expected response, but it bothers me that I get a logging error as below from the server side. I failed to figure out what’s this error about and how to prevent this from happening.
Code is not attached as the error seems irrelevant with my implementation.
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/__init__.py", line 1088, in emit
stream.write(msg + self.terminator)
File "/usr/lib/python3.8/codecs.py", line 378, in write
self.stream.write(data)
ValueError: raw stream has been detached
Call stack:
File "<string>", line 1, in <module>
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 129, in _main
return self._bootstrap(parent_sentinel)
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 68, in run
return asyncio.run(self.serve(sockets=sockets))
File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
self.run_forever()
File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
self._run_once()
File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
handle._run()
File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/fastapi/applications.py", line 208, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 159, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.8/dist-packages/starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 656, in __call__
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 259, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 64, in app
await response(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/responses.py", line 149, in __call__
await send(
File "/usr/local/lib/python3.8/dist-packages/starlette/exceptions.py", line 68, in sender
await send(message)
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 156, in _send
await send(message)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/h11_impl.py", line 438, in send
self.access_logger.info(
Message: '%s - "%s %s HTTP/%s" %d'
Arguments: ('xxx.xxx.x.x:35100', 'GET', '/api/myapi', '1.1', 200)
Additional information
$ python --version
Python 3.8.10
$ cat /etc/*ease
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
A minimal example to reproduce the error.
from bisect import bisect_right
from collections import defaultdict
from datetime import datetime, timedelta
import json
from operator import ge
import sys
from pathlib import Path
import codecs
sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach())
from pprint import pprint
from typing import Optional, Set, List, Type, Dict
import inspect
import uvicorn
from fastapi import FastAPI, Request, Response
from pydantic import BaseModel
from fastapi import Form, Depends
app = FastAPI()
fake_items_db = [{"item_name": "Foo"}, {"item_name": "Bar"}, {"item_name": "Baz"}]
@app.get("/items/")
def read_item(skip: int = 0, limit: int = 10):
breakpoint()
return fake_items_db[skip : skip + limit]
Answers:
It is rather hard to figure out what the issue is without having a look at the code. The error though seems referring to the the underlying buffer being detached from the TextIOBase. The code below throws the exact same error (i.e., ValueError: raw stream has been detached), as soon as the f.close() is called after calling f.buffer.detach(). If you comment out f.close(), you won’t be seeing the error. Like I said, one will be better able to provide help, if the code is provided. However, from the below, it doesn’t seem that is a FastAPI issue, but rather—also judging from the traceback you provided, i.e., ...File "/usr/lib/python3.8/codecs.py", line 378, in write self.stream.write(data) ValueError: raw stream has been detached—seems being an issue with a writing process for either logging the messages or something that we can’t see; and maybe, attempting to close the file while the buffer has been detached.
f = open ('text.txt', 'w')
for i in range(1,15):
f.write(str(i))
buff = f.buffer.detach()
#print(buff)
f.close()
My API is working properly from the client’s side, as it returns the expected response, but it bothers me that I get a logging error as below from the server side. I failed to figure out what’s this error about and how to prevent this from happening.
Code is not attached as the error seems irrelevant with my implementation.
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/__init__.py", line 1088, in emit
stream.write(msg + self.terminator)
File "/usr/lib/python3.8/codecs.py", line 378, in write
self.stream.write(data)
ValueError: raw stream has been detached
Call stack:
File "<string>", line 1, in <module>
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 129, in _main
return self._bootstrap(parent_sentinel)
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 68, in run
return asyncio.run(self.serve(sockets=sockets))
File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
self.run_forever()
File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
self._run_once()
File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
handle._run()
File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/fastapi/applications.py", line 208, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 159, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.8/dist-packages/starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 656, in __call__
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 259, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 64, in app
await response(scope, receive, send)
File "/usr/local/lib/python3.8/dist-packages/starlette/responses.py", line 149, in __call__
await send(
File "/usr/local/lib/python3.8/dist-packages/starlette/exceptions.py", line 68, in sender
await send(message)
File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 156, in _send
await send(message)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/h11_impl.py", line 438, in send
self.access_logger.info(
Message: '%s - "%s %s HTTP/%s" %d'
Arguments: ('xxx.xxx.x.x:35100', 'GET', '/api/myapi', '1.1', 200)
Additional information
$ python --version
Python 3.8.10
$ cat /etc/*ease
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
A minimal example to reproduce the error.
from bisect import bisect_right
from collections import defaultdict
from datetime import datetime, timedelta
import json
from operator import ge
import sys
from pathlib import Path
import codecs
sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach())
from pprint import pprint
from typing import Optional, Set, List, Type, Dict
import inspect
import uvicorn
from fastapi import FastAPI, Request, Response
from pydantic import BaseModel
from fastapi import Form, Depends
app = FastAPI()
fake_items_db = [{"item_name": "Foo"}, {"item_name": "Bar"}, {"item_name": "Baz"}]
@app.get("/items/")
def read_item(skip: int = 0, limit: int = 10):
breakpoint()
return fake_items_db[skip : skip + limit]
It is rather hard to figure out what the issue is without having a look at the code. The error though seems referring to the the underlying buffer being detached from the TextIOBase. The code below throws the exact same error (i.e., ValueError: raw stream has been detached), as soon as the f.close() is called after calling f.buffer.detach(). If you comment out f.close(), you won’t be seeing the error. Like I said, one will be better able to provide help, if the code is provided. However, from the below, it doesn’t seem that is a FastAPI issue, but rather—also judging from the traceback you provided, i.e., ...File "/usr/lib/python3.8/codecs.py", line 378, in write self.stream.write(data) ValueError: raw stream has been detached—seems being an issue with a writing process for either logging the messages or something that we can’t see; and maybe, attempting to close the file while the buffer has been detached.
f = open ('text.txt', 'w')
for i in range(1,15):
f.write(str(i))
buff = f.buffer.detach()
#print(buff)
f.close()