Routes works for the first test, can not be found for the second test (hint: 1st and 2nd tests are the same)
Question:
Before asking you, I tried to search as best as I could in SO for similar questions. I’ve read https://docs.pytest.org/en/latest/explanation/fixtures.html page twice. However, I could not solve the problem I am facing with. I am new to Python and Flask. So I thought that I am missing something obvious.
I have a small micky-mouse Flask application. I am trying to write tests for this. I voted for using Pytest because fixtures seemed more flexible than unittest. To my surprise, first test case runs with no problem at all. However the exact same test case fails to run for the second time.
Here is my fixtures.py file:
from config import Config
import pytest
from app import create_app, db
from app.models import User, Role
class TestConfig(Config):
SECRET_KEY = 'Shhhhh 007! Be quiet.!'
SQLALCHEMY_DATABASE_URI = 'sqlite:///:memory:'
SQLALCHEMY_TRACK_MODIFICATIONS = False
EXPLAIN_TEMPLATE_LOADING = True
DEBUG = True
API_VERSION = 'temp'
@pytest.fixture
def client():
testConfig = TestConfig()
app = create_app(testConfig)
from app import routes, models
with app.test_client() as client:
with app.app_context():
db.create_all()
yield client
@pytest.fixture(autouse=True)
def role(client):
role = Role()
role.name = 'test role'
db.session.add(role)
db.session.commit()
yield role
db.session.delete(role)
db.session.commit()
@pytest.fixture(autouse=True)
def user(role):
user = User()
user.role_id = role.id
user.name = 'test name'
user._normalized_name = 'test name'
user.lastname = 'test lastname'
user._normalized_lastname = 'test lastname'
user.phonenumber = '+11231234567'
user.setPassword('testPassword')
db.session.add(user)
db.session.commit()
yield user
db.session.delete(user)
db.session.commit()
And here is my test_login.py file:
import logging
from tests.fixtures import client, role, user
def test_login_page_Once(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
assert '<title>Signin' in decodedRv
def test_login_page_Twice(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
assert '<title>Signin' in decodedRv
I run my tests using:
pytest -v -k test_login
And here is the outcome:
tests/functional/test_login.py::test_login_page_Once PASSED [ 50%]
tests/functional/test_login.py::test_login_page_Twice FAILED [100%]
========================================== FAILURES ===========================================
____________________________________ test_login_page_Twice ____________________________________
client = <FlaskClient <Flask 'app'>>
def test_login_page_Twice(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
> assert '<title>Signin' in decodedRv
E assert '<title>Signin' in '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">n<title>404 Not Found</title>n<h1>Not Found</h1>n<p>The requested URL was not found on the server.
If you entered the URL manually please check your spelling and try again.</p>n'
testsfunctionaltest_login.py:17: AssertionError
------------------------------------ Captured stdout call -------------------------------------
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>
=================================== short test summary info ===================================
FAILED tests/functional/test_login.py::test_login_page_Twice - assert '<title>Signin' in '<!D...
========================== 1 failed, 1 passed, 2 deselected in 1.57s ==========================
What am I missing? Any help would be highly appreciated. Thanks very much.
Answers:
Well, I found a solution to the problem for which I was banging my head against the wall for a while. Though I reached that solution by try and error, it is still beyond me why/how it works. If any experts of pytest/flask would give me an explanation, that would be lovely.
I would also appreciate if someone tells me what the differences are among fixture scopes. ( I have read https://docs.pytest.org/en/latest/reference/reference.html#pytest.fixture already but to me it is not clear.)
Thanks.
The only change I made in the code is that I added
scope=’session’
to the client fixture.
@pytest.fixture(scope='session')
def client():
testConfig = TestConfig()
app = create_app(testConfig)
from app import routes, models
with app.test_client() as client:
with app.app_context():
db.create_all()
yield client
ADDITION #2:
Further down the road, another test of mine failed with similar symptoms. The difference was this test case living in another file (I encountered previous problems while I was testing login page/behavior. Now I am dealing with registration page/behavior). I could not find my way around this error for a while. (I even considered dropping of pytest and going back to unittest. But I am still here because I am a bit of a stubborn. That’s a different story!)
Anyways, I realized that one way of grouping fixtures is to place them in a file named conftest.py. Mine was named fixtures.py. In my head, it was just a name and I thought mine was clearer. Well, I was wrong. conftest.py seems to be a magic file. Once I renamed the fixtures.py to conftest.py test all marched pass in shining green armors.
Adding my two cents here just in case anyone else feels lost in the depths of fixtures/routes/imports forests.
I had a similar issue and it turned out that I was accidentally mutating an object inside a function and that was having side effects outside the function.
First time I ran the fixture it worked fine.
Second time I ran the fixture, the function that defined the fixture was working with different data because of a side effect of the first time I ran the function. This gave an error trying to run the function.
Not sure if that is your problem, but this might help someone facing a similar issue.
Before asking you, I tried to search as best as I could in SO for similar questions. I’ve read https://docs.pytest.org/en/latest/explanation/fixtures.html page twice. However, I could not solve the problem I am facing with. I am new to Python and Flask. So I thought that I am missing something obvious.
I have a small micky-mouse Flask application. I am trying to write tests for this. I voted for using Pytest because fixtures seemed more flexible than unittest. To my surprise, first test case runs with no problem at all. However the exact same test case fails to run for the second time.
Here is my fixtures.py file:
from config import Config
import pytest
from app import create_app, db
from app.models import User, Role
class TestConfig(Config):
SECRET_KEY = 'Shhhhh 007! Be quiet.!'
SQLALCHEMY_DATABASE_URI = 'sqlite:///:memory:'
SQLALCHEMY_TRACK_MODIFICATIONS = False
EXPLAIN_TEMPLATE_LOADING = True
DEBUG = True
API_VERSION = 'temp'
@pytest.fixture
def client():
testConfig = TestConfig()
app = create_app(testConfig)
from app import routes, models
with app.test_client() as client:
with app.app_context():
db.create_all()
yield client
@pytest.fixture(autouse=True)
def role(client):
role = Role()
role.name = 'test role'
db.session.add(role)
db.session.commit()
yield role
db.session.delete(role)
db.session.commit()
@pytest.fixture(autouse=True)
def user(role):
user = User()
user.role_id = role.id
user.name = 'test name'
user._normalized_name = 'test name'
user.lastname = 'test lastname'
user._normalized_lastname = 'test lastname'
user.phonenumber = '+11231234567'
user.setPassword('testPassword')
db.session.add(user)
db.session.commit()
yield user
db.session.delete(user)
db.session.commit()
And here is my test_login.py file:
import logging
from tests.fixtures import client, role, user
def test_login_page_Once(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
assert '<title>Signin' in decodedRv
def test_login_page_Twice(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
assert '<title>Signin' in decodedRv
I run my tests using:
pytest -v -k test_login
And here is the outcome:
tests/functional/test_login.py::test_login_page_Once PASSED [ 50%]
tests/functional/test_login.py::test_login_page_Twice FAILED [100%]
========================================== FAILURES ===========================================
____________________________________ test_login_page_Twice ____________________________________
client = <FlaskClient <Flask 'app'>>
def test_login_page_Twice(client):
rv = client.get('/login')
decodedRv = rv.data.decode('utf-8')
print(decodedRv)
> assert '<title>Signin' in decodedRv
E assert '<title>Signin' in '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">n<title>404 Not Found</title>n<h1>Not Found</h1>n<p>The requested URL was not found on the server.
If you entered the URL manually please check your spelling and try again.</p>n'
testsfunctionaltest_login.py:17: AssertionError
------------------------------------ Captured stdout call -------------------------------------
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>
=================================== short test summary info ===================================
FAILED tests/functional/test_login.py::test_login_page_Twice - assert '<title>Signin' in '<!D...
========================== 1 failed, 1 passed, 2 deselected in 1.57s ==========================
What am I missing? Any help would be highly appreciated. Thanks very much.
Well, I found a solution to the problem for which I was banging my head against the wall for a while. Though I reached that solution by try and error, it is still beyond me why/how it works. If any experts of pytest/flask would give me an explanation, that would be lovely.
I would also appreciate if someone tells me what the differences are among fixture scopes. ( I have read https://docs.pytest.org/en/latest/reference/reference.html#pytest.fixture already but to me it is not clear.)
Thanks.
The only change I made in the code is that I added
scope=’session’
to the client fixture.
@pytest.fixture(scope='session')
def client():
testConfig = TestConfig()
app = create_app(testConfig)
from app import routes, models
with app.test_client() as client:
with app.app_context():
db.create_all()
yield client
ADDITION #2:
Further down the road, another test of mine failed with similar symptoms. The difference was this test case living in another file (I encountered previous problems while I was testing login page/behavior. Now I am dealing with registration page/behavior). I could not find my way around this error for a while. (I even considered dropping of pytest and going back to unittest. But I am still here because I am a bit of a stubborn. That’s a different story!)
Anyways, I realized that one way of grouping fixtures is to place them in a file named conftest.py. Mine was named fixtures.py. In my head, it was just a name and I thought mine was clearer. Well, I was wrong. conftest.py seems to be a magic file. Once I renamed the fixtures.py to conftest.py test all marched pass in shining green armors.
Adding my two cents here just in case anyone else feels lost in the depths of fixtures/routes/imports forests.
I had a similar issue and it turned out that I was accidentally mutating an object inside a function and that was having side effects outside the function.
First time I ran the fixture it worked fine.
Second time I ran the fixture, the function that defined the fixture was working with different data because of a side effect of the first time I ran the function. This gave an error trying to run the function.
Not sure if that is your problem, but this might help someone facing a similar issue.