Testing with configuration files

Question

my Google-fu has failed me by giving me results I don’t understand, so I’m asking here.

I’m working on a Python project and I currently have a configuration file, which is also .py, that holds various python objects to load when everything starts. I’m trying to get some practice unit testing with pytest and I don’t know exactly how to go about this issue. My guess is that I am probably going to make a dedicated testing config file that doesn’t change, but I have no clue how to tell my code when to use the actual config file and when to use the testing config file. My current setup in my code is just using import config and setting values from it. I would greatly appreciate some help here!

Asked By: k huang

||

Source

Answer 1

Figured out something based on this.

I ended up creating a testing config file in my folder named "tests" and having this code at the top of every file that used it:

import sys
if "pytest" in sys.modules:
    import tests.testing_config as config
else:
    import config

I don’t think it’s quite optimal since I feel something like this should be in the testing code, plus it kind of makes the new config file a dependency that should never be changed lest you break everything, but I guess it works for now if you have no clue how these cracked testing libraries work like me.

Answered By: k huang

Answer 2

An approach I’ve used to solve this problem is manipulating the path when running tests so that an additional package (/tests/resources) can shadow the normal resources package (/main/resources) that I set up for containing assets such as configuration files and making them available for loading at runtime.

Note: This structure is inspired by a pattern that I’ve brought from Java/Maven, so I won’t claim it’s Pythonic, and I don’t like the path manipulation where the tests are concerned (I know a lot of others won’t, either, so beware!). In fact, I found this question while looking for a better way to do this.

To accomplish this you first have to set up a folder to serve as your ‘resources’ package (See the docs). Once that’s done, you create a similar version for your tests, in the tests folder. The directory structure will look something like this:

project
|-main
| |-resources
| | |-__init__.py
| | -application.yml
| -[...other source modules/packages...]
|-tests
| |-resources
| | |-__init__.py
| | -application.yml
| -[...other modules/packages with tests...]
-run_tests.py

Note that main and tests are NOT packages. This is what contradicts a lot of conventional guidance in Python project structure. Instead, they’re both added to the path, with tests being inserted into the path in front of main, by the run_tests.py script using code like this (some bits borrowed from this post:

import os.path
import sys

import pytest

ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(ROOT_DIR, 'main'))
sys.path.insert(0, os.path.join(ROOT_DIR, 'tests'))

# Add any required args for testing.  
args = []

pytest.main(args)

I’ll typcially have additional options I want to feed pytest, and that’s what args is for, but it’s not relevant to this answer, so I left it empty here.

I know this is a late answer, but I hope this helps someone. Also, I know it’s a controversial approach, but it’s the most satisfying approach I’ve found so far. That may be because I’m experienced in Java as well as Python. To others, the approach of trying the import of tests.resources, and importing main.resources if it fails may be preferable.

I hope anyone with other approaches to this will share in comments, or post additional answers.

Answered By: Jason Sibre

Testing with configuration files

Question:

Answers: