Pip install from zip file on private server not installing requirements and when requirements met it does not find the package

I made a command line tool to manage some actions on the server-side, and I want to easily distribute it for my coworkers using pip install from a private server we have for internal tools, and to provide them with updates on it, so we don’t have to walk around with a thumbstick and doing boring pip install . -r requirements.txt for every workstation they have.

I’d try to keep this short, but I think I have to include the project and packaging details to find the problem. I suspect a bad configuration of the setup.py file.

The server

It is a simple NGINX pointing to a folder with basic auth .htpass. Inside the folder I have the zipped project (maybe this is my first mistake?).

To download and install using pip (Python 3.5+), I simply run

pip install http://username:[email protected]/tools/mypackage.zip

The setup.py

The requirements in the setup.py below are not downloaded.

import os
from setuptools import setup
from mypackage import __version__, project_name


# Utility function to read the README file.
# Used for the long_description.  It's nice, because now 1) we have a top level
# README file and 2) it's easier to type in the README file than to put a raw
# string in below ...
def read(fname):
    return open(os.path.join(os.path.dirname(__file__), fname)).read()

setup(
  name=project_name,
  version=__version__,
  description='Command line client for maintenance and odd jobs on the server',
  long_description=read('README.md'),
  classifiers=[
        "Development Status :: 2 - Alpha",
        "Topic :: Utilities :: Rest",
        "Framework :: Click",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.5",
  ],
  keywords='cli rest client admin',
  author="Myself",
  author_email='[email protected]',
  packages=['mypackage'],
  zip_safe=True,
  install_requires=[
      'certifi==2017.7.27.1',
      'chardet==3.0.4',
      'click==6.7',
      'idna==2.6',
      'requests==2.18.4',
      'urllib3==1.22',
      'terminaltables==3.1.0',
      'tqdm==4.19.4',
      'aiohttp==2.3.3',
      'aiodns==1.1.1',
      'cchardet==2.1.1',
      'async-timeout==2.0.0',
  ],
  entry_points='''
    [console_scripts]
        stp = mypackage.MyPackage:cli
        stp-config = mypackage.MyPackage:config
        ''',
  )

It tries to download the zip file and the setup.py marks the zip "safe" (I believe this might be my first mistake, but I can’t test it atm), but it does not install the requirements.

The zip file has also the EGG folder of the installed package. I tried removing it or leaving it. Neither makes a difference whatsoever for the download/install. It still breaks because it lacks the dependencies.

If I install the requirements separately, I can install myPackage using pip (finally!) but when I try to call it by its entry point

stp --help

I receive an error saying that ‘mypackage is not present’.

C:UsersScoppio>stp --help
Traceback (most recent call last):
File "C:UsersScoppioAppDataLocalProgramsPythonPython36Scriptsstp-script.py", line 11, in <module>
load_entry_point('MyPackage==0.2.1', 'console_scripts', 'stp')()
File "c:usersscoppioappdatalocalprogramspythonpython36libsite-packagespkg_resources__init__.py", line 565, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "c:usersscoppioappdatalocalprogramspythonpython36libsite-packagespkg_resources__init__.py", line 2631, in load_entry_point
return ep.load()
File "c:usersscoppioappdatalocalprogramspythonpython36libsite-packagespkg_resources__init__.py", line 2291, in load
return self.resolve()
File "c:usersscoppioappdatalocalprogramspythonpython36libsite-packagespkg_resources__init__.py", line 2297, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "c:usersscoppioappdatalocalprogramspythonpython36libsite-packagesmypackage__init__.py", line 1, in <module>
from mypackage.my_client import myUrls
ModuleNotFoundError: No module named 'mypackage.my_client '

You may notice that this error message is from a Windows OS, but I get analogous errors on Mac OSX and Ubuntu 16.

And if I go to the folder which should have installed it inside the site-packages and open it, my package ‘mypackage’ has only the files there where inside the first folder level, for no apparent reason it discarded all the upper folders I had in place for the project.

The file structure

So my project structure was:

core/
    setup.py
    requirements.txt
    readme.md
    mypackage/
       __init__.py
       context.py
       my_package.py
       my_client/
          __init__.py
          rest_client.py
          my_urls.py
       my_plugins/
          __init__.py
          cmd_1.py
          cmd_2.py
          ...
          cmd_n.py
          utils/
             __init__.py
             print_table.py

but after installation with pip, I am left with just those files

mypackage/
   __init__.py
   context.py
   my_package.py

I explored the temporary file pip makes, but the whole project is present there, it simply does not install all the files I need.

So far I only managed to install everything by manually adding the files missing, but it moots the point of using pip to install the project.

I believe that I misunderstood the setuptools documentation and now I am tracing a few possible approaches, like adding more packages to the package entry in the setup, and using "extra" requirements instead of requirements.