setup.py not installing data files

Question

I have a Python library that, in addition to regular Python modules, has some data files that need to go in /usr/local/lib/python2.7/dist-package/mylibrary.

Unfortunately, I have been unable to convince setup.py to actually install the data files there. Note that this behaviour is under install – not sdist.

Here is a slightly redacted version of setup.py

module_list = list_of_files

setup(name         ='Modules',
      version      ='1.33.7',
      description  ='My Sweet Module',
      author       ='PN',
      author_email ='email',
      url          ='url',
      packages     = ['my_module'],

# I tried this. It got installed in /usr/my_module. Not ok.

      # data_files   = [ ("my_module",  ["my_module/data1",
      #                                  "my_module/data2"])]

# This doesn't install it at all.
      package_data = {"my_module" : ["my_module/data1",
                                     "my_module/data2"] }
     )

This is in Python 2.7 (will have to run in 2.6 eventually), and will have to run on some Ubuntu between 10.04 and 12+. Developing it right now on 12.04.

Asked By: Paul Nathan

||

Source

Answer 1

http://docs.python.org/distutils/setupscript.html#installing-additional-files

If directory is a relative path, it is interpreted relative to the
installation prefix (Python’s sys.prefix for pure-Python packages,
sys.exec_prefix for packages that contain extension modules).

This will probably do it:

data_files   = [ ("my_module",  ["local/lib/python2.7/dist-package/my_module/data1",
                                 "local/lib/python2.7/dist-package/my_module/data2"])]

Or just use join to add the prefix:

data_dir = os.path.join(sys.prefix, "local/lib/python2.7/dist-package/my_module")
data_files   = [ ("my_module",  [os.path.join(data_dir, "data1"),
                                 os.path.join(data_dir, "data2")])]

Answered By: monkut

Answer 2

UPD:
package_data accepts dict in format {'package': ['list', 'of?', 'globs*']}, so to make it work, one should specify shell globs relative to package dir, not the file paths relative to the distribution root.

data_files has a different meaning, and, in general, one should avoid using this parameter.

With setuptools you only need include_package_data=True, but data files should be under version control system, known to setuptools (by default it recognizes only CVS and SVN, install setuptools-git or setuptools-hg if you use git or hg…)

with setuptools you can:

~~– in MANIFEST.im:~~

~~include my_module/data*~~

– in setup.py:

    setup(
        ...
        include_package_data = True,
        ...
    )

Answered By: podshumok

Answer 3

The following solution worked fine for me.
You should have MANIFEST.in file where setup.py is located.

Add the following code to the manifest file

recursive-include mypackage *.json *.md # can be extended with more extensions or file names.

Another solution is adding the following code to the MANIFEST.in file.

graft mypackage # will copy the entire package including non-python files. 
global-exclude __pyache__ *.txt # list files you dont want to include here.

Now, when you do pip install all the necessary files will be included.

Hope this helps.

UPDATE:
Make sure that you also have include_package_data=True in the setup file

Answered By: shmsi

setup.py not installing data files

Question:

Answers: