Cloning private github repository within organisation in actions
Question:
I have 2 private GitHub repositories (say A and B) in the organization (say ORG). Repository A has repository B in requirements.txt
:
-e [email protected]:ORG/B.git#egg=B
And I have the following workflow for A (in .github/workflows/test.yml
):
name: Python package
on: push
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Install requirements
run: |
pip install -r requirements.txt
- name: Test with pytest
run: |
pytest ./tests
As B is private, it fails on installing it.
Is it possible to install B while testing A in this workflow if they are in the same organization? How?
Answers:
Either use an SSH key with no passphrase to access repo B, or create an access token for that repo and then use the access token as your password to access that repo over HTTPS: https://USERNAME:[email protected]/ORG/B.git
.
I did this way!
- uses: actions/checkout@v1
with:
repository: organization_name/repo_name
token: ${{ secrets.ACCESS_TOKEN }}
You need to provide a valid token, you can generate it following this guide
I added this line
git+https://[email protected]/ORG/REPO_NAME.git@master#egg=REPO_NAME
to my requirements.txt
and it worked. But as other people mentioned, your token will be exposed to anyone having access to this repository. It is probably best to use a secret in your repository.
Instead of check out twice, all you need is provided the TOKEN for pip
to access repo B.
- name: Install requirements
run: |
git config --global url."https://${{ secrets.ACESS_TOKEN }}@github".insteadOf https://github
pip install -r requirements.txt
Since access tokens are bound to an account and have write access to all its private repos, it’s a very bad solution.
Instead, use deploy keys.
Deploy keys are simply SSH keys that you can use to clone a repo.
- Create a new SSH key pair on your computer
- Put the public key in the private dependency repo’s Deploy keys
- Put the private key in the app repo’s Actions secrets
- Delete the keys from your computer
Once it’s set, you can set the private key in the GitHub Action’s SSH Agent. There’s no need to import a third-party GitHub Action, a 2-liner will suffice.
eval `ssh-agent -s`
ssh-add - <<< '${{ secrets.PRIVATE_SSH_KEY }}'
pip install -r requirements.txt
I found that ssh-add
command here.
Using deployment keys you can do
- uses: actions/checkout@v2
with:
ssh-key: ${{ secrets.SSH_PRIVATE_KEY }}
repository: organization_name/repo_name
For this to work you need to
- generate ssh keys locally
- add pub key as deployment key to the private repo
- add private key as a secret named
SSH_PRIVATE_KEY
There is safety way to do this in Github Actions (without storing ssh-key). You have dedicated lib for that in Github Actions
- uses: webfactory/[email protected]
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY_VOW_SHARED }}
When you use docker you have to create ssh tunneling:
docker build --ssh default=${SSH_AUTH_SOCK} .
or docker-compose.yaml:
build:
ssh:
default: ${SSH_AUTH_SOCK}
And after that in Dockerfile:
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh pip install -r requirements.txt
Remember to set DOCKER_BUILDKIT flag in ci.yaml:
env:
DOCKER_BUILDKIT: 1
I have 2 private GitHub repositories (say A and B) in the organization (say ORG). Repository A has repository B in requirements.txt
:
-e [email protected]:ORG/B.git#egg=B
And I have the following workflow for A (in .github/workflows/test.yml
):
name: Python package
on: push
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Install requirements
run: |
pip install -r requirements.txt
- name: Test with pytest
run: |
pytest ./tests
As B is private, it fails on installing it.
Is it possible to install B while testing A in this workflow if they are in the same organization? How?
Either use an SSH key with no passphrase to access repo B, or create an access token for that repo and then use the access token as your password to access that repo over HTTPS: https://USERNAME:[email protected]/ORG/B.git
.
I did this way!
- uses: actions/checkout@v1
with:
repository: organization_name/repo_name
token: ${{ secrets.ACCESS_TOKEN }}
You need to provide a valid token, you can generate it following this guide
I added this line
git+https://[email protected]/ORG/REPO_NAME.git@master#egg=REPO_NAME
to my requirements.txt
and it worked. But as other people mentioned, your token will be exposed to anyone having access to this repository. It is probably best to use a secret in your repository.
Instead of check out twice, all you need is provided the TOKEN for pip
to access repo B.
- name: Install requirements
run: |
git config --global url."https://${{ secrets.ACESS_TOKEN }}@github".insteadOf https://github
pip install -r requirements.txt
Since access tokens are bound to an account and have write access to all its private repos, it’s a very bad solution.
Instead, use deploy keys.
Deploy keys are simply SSH keys that you can use to clone a repo.
- Create a new SSH key pair on your computer
- Put the public key in the private dependency repo’s Deploy keys
- Put the private key in the app repo’s Actions secrets
- Delete the keys from your computer
Once it’s set, you can set the private key in the GitHub Action’s SSH Agent. There’s no need to import a third-party GitHub Action, a 2-liner will suffice.
eval `ssh-agent -s`
ssh-add - <<< '${{ secrets.PRIVATE_SSH_KEY }}'
pip install -r requirements.txt
I found that ssh-add
command here.
Using deployment keys you can do
- uses: actions/checkout@v2
with:
ssh-key: ${{ secrets.SSH_PRIVATE_KEY }}
repository: organization_name/repo_name
For this to work you need to
- generate ssh keys locally
- add pub key as deployment key to the private repo
- add private key as a secret named
SSH_PRIVATE_KEY
There is safety way to do this in Github Actions (without storing ssh-key). You have dedicated lib for that in Github Actions
- uses: webfactory/[email protected]
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY_VOW_SHARED }}
When you use docker you have to create ssh tunneling:
docker build --ssh default=${SSH_AUTH_SOCK} .
or docker-compose.yaml:
build:
ssh:
default: ${SSH_AUTH_SOCK}
And after that in Dockerfile:
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh pip install -r requirements.txt
Remember to set DOCKER_BUILDKIT flag in ci.yaml:
env:
DOCKER_BUILDKIT: 1