In pytest, is there a way to mock an already imported variable without having to patch for every single test?
Question:
So to be specific, we’re using SqlAlchemy and session, and we use define it only once at say, at utils/sessions.py. like following :
#utils/sessions.py
from sqlalchemy.orm import scoped_session, sessionmaker
session = scoped_session (sessionmaker(
....
))
and we use this on our actual repository layer, for example :
#app/user/users.py
from utils.sessions import session
class UserRepo :
def get_user(self,id) :
return session.query(User).filter(User.id=id)
Now I’m trying to perform some sort of unit/integration test, and I need to mock the session variable that came from utils.sessions in practically everywhere within the test (hence I would like to use an autouse fixture mocking util.sessions.session).
so far, the only thing that seemed to be worked was to patch app.user.users.session within the test code itself , While what i want is to basically mock every single occurance of session in different tests too. like, if i want to test app/articles , I wouldn’t want to repeatedly type with patch('apps.articles.article.session' unnecessarily.
so ideally, I would like to have something like this and expect all my occurance of session imported in individual repos to work properly :
# tests/conftest.py
def mocked_session() :
return scoped_session("my_new_parameters_for_test_purpose_only")
@pytest.fixture(autouse=true)
def mock_all_sessions():
with patch("utils.sessions.session", mocked_session) as s :
yield s
but this didn’t work for me. (It’s strange for me because patching socket.socket successfully blocks networking attempt.)
it would be nice if there was a way to mock utils.sessions.session instead of having to manually patch apps/{all apps i have}/{all .py files}.session/.
Is there a way to do this? if not, would there be an alternative to having to manually write down the module path for every single test?
Answers:
The problem is that you mock session only in one namespace, i.e. utils.sessions.__dict__, but the name "session" exists in many other namespaces e.g. apps.articles.article.__dict__.
There is a low-tech solution possible by simply changing import statements – instead of
from utils.sessions import session
You could use
from utils import sessions
Then you only need to mock in the one namespace (where the name is looked up) as you’re already doing. The caveat is that all those submodules now need to use sessions.session instead of just session, so that they are all looking in the same namespace where you’re applying the patch.
See the unittest documentation https://docs.python.org/3/library/unittest.mock.html#where-to-patch
The main issue is that when you patch, you are only patching the name a variable in the context of a module. When users.py does from utils.sessions import session it creates its own reference to session. i.e. there are two variable names which point to the same object.
utils.sessions.session = <your session object>
app.user.users.session = <your session object>
When you patch just utils.sessions.session you are only replacing the contents of the name in that module. e.g.
utils.sessions.session = <the patched mock object>
app.user.users.session = <your session object>
If you instead do the import as from utils import sessions then you would reference the object as sessions.session, which means when you patch the value on the sessions module, it will find the correct override to the patched value at runtime.
IMO This becomes awkward because it imposes a certain style of performing imports on you, when both should be technically valid. An easier pattern might be to create a proxy method inside of sessions.py that will put a barrier in between importers of the module and accessing the session object. That would look like this:
_session = scoped_session(sessionmaker(...))
def session():
return _session
used like:
from utils.sessions import session
class UserRepo:
def get_user(self,id):
return session().query(User).filter(User.id=id)
Now when you go to patch, you only have to patch utils.sessions._session, and it doesn’t matter how you do the import. The reference to the function session() remains unchanged, but the object it would return is changed for everyone.
So to be specific, we’re using SqlAlchemy and session, and we use define it only once at say, at utils/sessions.py. like following :
#utils/sessions.py
from sqlalchemy.orm import scoped_session, sessionmaker
session = scoped_session (sessionmaker(
....
))
and we use this on our actual repository layer, for example :
#app/user/users.py
from utils.sessions import session
class UserRepo :
def get_user(self,id) :
return session.query(User).filter(User.id=id)
Now I’m trying to perform some sort of unit/integration test, and I need to mock the session variable that came from utils.sessions in practically everywhere within the test (hence I would like to use an autouse fixture mocking util.sessions.session).
so far, the only thing that seemed to be worked was to patch app.user.users.session within the test code itself , While what i want is to basically mock every single occurance of session in different tests too. like, if i want to test app/articles , I wouldn’t want to repeatedly type with patch('apps.articles.article.session' unnecessarily.
so ideally, I would like to have something like this and expect all my occurance of session imported in individual repos to work properly :
# tests/conftest.py
def mocked_session() :
return scoped_session("my_new_parameters_for_test_purpose_only")
@pytest.fixture(autouse=true)
def mock_all_sessions():
with patch("utils.sessions.session", mocked_session) as s :
yield s
but this didn’t work for me. (It’s strange for me because patching socket.socket successfully blocks networking attempt.)
it would be nice if there was a way to mock utils.sessions.session instead of having to manually patch apps/{all apps i have}/{all .py files}.session/.
Is there a way to do this? if not, would there be an alternative to having to manually write down the module path for every single test?
The problem is that you mock session only in one namespace, i.e. utils.sessions.__dict__, but the name "session" exists in many other namespaces e.g. apps.articles.article.__dict__.
There is a low-tech solution possible by simply changing import statements – instead of
from utils.sessions import session
You could use
from utils import sessions
Then you only need to mock in the one namespace (where the name is looked up) as you’re already doing. The caveat is that all those submodules now need to use sessions.session instead of just session, so that they are all looking in the same namespace where you’re applying the patch.
See the unittest documentation https://docs.python.org/3/library/unittest.mock.html#where-to-patch
The main issue is that when you patch, you are only patching the name a variable in the context of a module. When users.py does from utils.sessions import session it creates its own reference to session. i.e. there are two variable names which point to the same object.
utils.sessions.session = <your session object>
app.user.users.session = <your session object>
When you patch just utils.sessions.session you are only replacing the contents of the name in that module. e.g.
utils.sessions.session = <the patched mock object>
app.user.users.session = <your session object>
If you instead do the import as from utils import sessions then you would reference the object as sessions.session, which means when you patch the value on the sessions module, it will find the correct override to the patched value at runtime.
IMO This becomes awkward because it imposes a certain style of performing imports on you, when both should be technically valid. An easier pattern might be to create a proxy method inside of sessions.py that will put a barrier in between importers of the module and accessing the session object. That would look like this:
_session = scoped_session(sessionmaker(...))
def session():
return _session
used like:
from utils.sessions import session
class UserRepo:
def get_user(self,id):
return session().query(User).filter(User.id=id)
Now when you go to patch, you only have to patch utils.sessions._session, and it doesn’t matter how you do the import. The reference to the function session() remains unchanged, but the object it would return is changed for everyone.