How to include a .ini file in my python package when installing via pip from TFS2017?

Question:

I have a library that requires a .ini file at some point (a default config file if none provided by the user).

Here is the hierarchy, with the required file marked:


├───src
│   ├───main
│   │   │   MANIFEST.in
│   │   │   setup.py
│   │   │
│   │   ├───matmuttools
│   │   │   │   matmut_logging.py
│   │   │   │   proxy.py
│   │   │   │   PROXY_USAGE.md
│   │   │   │   singleton.py
│   │   │   │   __init__.py
│   │   │   │
│   │   │   │
│   │   │   ├───ql
│   │   │   │   │   default_config.ini    <----
│   │   │   │   │   query_language.py
│   │   │   │   │   tsql.py
│   │   │   │   │   __init__.py

I have tried several things, based on this question and this doc

Using the following in the manifest does not "package" the ini file:

include matmuttools/ql/default_config.ini

Using setup.py, the following does not work either:

include_package_data=True,
package_data={'matmuttools': ['*.ini']}

I know that the file is not included because this file is read by configparser and it throws me an error when looking for a section contained in the file.

  File "/usr/local/lib/python3.9/site-packages/matmuttools/ql/query_language.py", line 41, in __init__
    raise ValueError(f"{self.language_name} not found in config file")
ValueError: TSQL not found in config file

Moreover, using ls to check out the package when installed does not show the file:

ls -lR /usr/local/lib/python3.9/site-packages/matmuttools/*

/usr/local/lib/python3.9/site-packages/matmuttools/ql:
total 16
-rw-r--r-- 1 root root  120 Sep 30 08:44 __init__.py
drwxr-xr-x 2 root root 4096 Sep 30 08:44 __pycache__
-rw-r--r-- 1 root root 2159 Sep 30 08:44 query_language.py
-rw-r--r-- 1 root root  478 Sep 30 08:44 tsql.py

The library is installed via:

pip install "git+http:myurl"


How can I ship correctly this configuration file with my library ?


Content of the wheel:

Building the wheel with:

python setup.py bdist_wheel

It contains:

$ zipinfo dist/matmuttools-3.2.0-py3-none-any.whl 
Archive:  dist/matmuttools-3.2.0-py3-none-any.whl
Zip file size: 12646 bytes, number of entries: 14
-rw-rw-rw-  2.0 fat      153 b- defN 22-Sep-29 10:12 matmuttools/__init__.py
-rw-rw-rw-  2.0 fat     9841 b- defN 22-Jun-13 06:37 matmuttools/matmut_logging.py
-rw-rw-rw-  2.0 fat     2047 b- defN 22-Jun-13 06:37 matmuttools/proxy.py
-rw-rw-rw-  2.0 fat      928 b- defN 22-Jun-13 06:37 matmuttools/singleton.py
-rw-rw-rw-  2.0 fat      257 b- defN 22-Jun-13 06:37 matmuttools/hadoop/__init__.py
-rw-rw-rw-  2.0 fat     9891 b- defN 22-Jun-13 06:37 matmuttools/hadoop/hdfs_wrapper.py
-rw-rw-rw-  2.0 fat    10465 b- defN 22-Jun-13 06:37 matmuttools/hadoop/impala_wrapper.py
-rw-rw-rw-  2.0 fat      120 b- defN 22-Sep-29 09:22 matmuttools/ql/__init__.py
-rw-rw-rw-  2.0 fat     2159 b- defN 22-Sep-29 12:19 matmuttools/ql/query_language.py
-rw-rw-rw-  2.0 fat      478 b- defN 22-Sep-29 11:56 matmuttools/ql/tsql.py
-rw-rw-rw-  2.0 fat      454 b- defN 22-Sep-30 09:08 matmuttools-3.2.0.dist-info/METADATA
-rw-rw-rw-  2.0 fat       97 b- defN 22-Sep-30 09:08 matmuttools-3.2.0.dist-info/WHEEL
-rw-rw-rw-  2.0 fat       12 b- defN 22-Sep-30 09:08 matmuttools-3.2.0.dist-info/top_level.txt
?rw-rw-r--  2.0 fat     1155 b- defN 22-Sep-30 09:08 matmuttools-3.2.0.dist-info/RECORD
14 files, 38057 bytes uncompressed, 10720 bytes compressed:  71.8%

It seems the MANIFEST.in is not even used. Adding prune matmuttools/ql to it does nothing. The content of the wheel is the same as above.

Asked By: Itération 122442

||

Answers:

The file default_config.ini doesn’t belong to matmuttools package but to matmuttools.ql subpackage so fix it this way:

package_data={'matmuttools.ql': ['*.ini']}
Answered By: phd
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.