Python Package with Sub-Packages for Microservice purpose
Question:
Im currently refactoring my monolith into microserivces. To make them communicate each service has as client module with client in it that call per request the other services. I want to manage the different packages as easy as possible so I created a repository which is my package. Then each folder/module is the service with the modules of it that are needed.
What I want to achieve is that I can simply call pip install package["subpackage"] and it installs only that specific module of the package. I choose a big package over small packages because of the naming problem that most services have basic names where pip packages already exist with these names.
Repository of package
repo
- payments/
client/
models/
- auth/
client/
models/
setup.py
Is there a way to provide the information what each submodule / module needs for installing like install_requires for each module ?
Is there a other good approach that I should take ? I know some companies to it with java so that each module is its own "package" but they are all under a company package. Maybe in python there is a better solution for this
Answers:
You can add additional requirements using:
setup(
name="package",
version=__version__,
install_requires=["stuff", "every", "package", "needs"],
extras_require={
"subpackage": ["dependency_1", "dependency_2"]
}
)
You can then run
pip install package[subpackage]
To install the extra dependencies.
In my organization, we
- Split each server into their own repositories
- Have a ‘mono repo’ that has all of the servers as git submodules
- Make each server pip-installable with an internal pip server
- Have a deployment repo that pins the versions of the servers we are using, as well as all the deployment scripts such as pushing docker contianers to local testing environments and terraform scripts to deploy to AWS.
Im currently refactoring my monolith into microserivces. To make them communicate each service has as client module with client in it that call per request the other services. I want to manage the different packages as easy as possible so I created a repository which is my package. Then each folder/module is the service with the modules of it that are needed.
What I want to achieve is that I can simply call pip install package["subpackage"] and it installs only that specific module of the package. I choose a big package over small packages because of the naming problem that most services have basic names where pip packages already exist with these names.
Repository of package
repo
- payments/
client/
models/ - auth/
client/
models/
setup.py
Is there a way to provide the information what each submodule / module needs for installing like install_requires for each module ?
Is there a other good approach that I should take ? I know some companies to it with java so that each module is its own "package" but they are all under a company package. Maybe in python there is a better solution for this
You can add additional requirements using:
setup(
name="package",
version=__version__,
install_requires=["stuff", "every", "package", "needs"],
extras_require={
"subpackage": ["dependency_1", "dependency_2"]
}
)
You can then run
pip install package[subpackage]
To install the extra dependencies.
In my organization, we
- Split each server into their own repositories
- Have a ‘mono repo’ that has all of the servers as git submodules
- Make each server pip-installable with an internal pip server
- Have a deployment repo that pins the versions of the servers we are using, as well as all the deployment scripts such as pushing docker contianers to local testing environments and terraform scripts to deploy to AWS.