Redirecting stdout/stderr/stdin of 'git clone'
Question:
I want to write a program which will clone remote git repository and then do bunch of other stuff. The problem is that ‘git clone’ asks for password. It does not work when I open pipes to stdin/out/err to ‘git clone’ because it runs git-remote-http underneath which prompts for password on TTY.
I would like to pass the password from my program. I am using Python and Popen from subprocess. Code below does not wotk.
Popen(['git', 'clone', 'https://my.git.repo/repo.git'], shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)
How can I achieve this?
Answers:
If you do not want to use challenge authentication as the commenter said, I would use pexpect to automate this kind of interaction
You can git clone https://foo:[email protected]/repo.git to get the data, and git remote set-url origin https://my.git.repo/repo.git afterwards to clear the password. But you have an race-condition between the start of the clone, and the URL-change.
If you are fine with using a library, you could try using Dulwich.
from dulwich import porcelain
porcelain.clone("https://example.com/repo.git", username="youruser", password="yourpassword")
This does not seem to store username or password (I’ve searched for the used password within the .git directory and did not find anything).
Sources
Because it was quite a hassle to find an up-to-date Git library without any obvious problems, I want to list my sources for future reference:
- Answer to "Python Git Module experiences? [closed]" by PTBNL from 2009 that recommends (among others) to use Dulwich
- Answer to "Use Git commands within Python code" by Trishayyyy from 2019 that still recommends Dulwich (and PyGit)
- Dulwich
porcelain documentation about clone that sadly does not mention that authentication is possible at all.
- Answer to "dulwich – clone from remote repo authentication" by harvin that mentions authentication is indeed possible with Dulwich
clone, using the username:password@ syntax.
- Inspired by the last linked answer, I inspected the source code a bit further and stumbled upon the
username and password parameters. I’ve documented this in my answer to the "dulwich – clone from remote repo authentication" question.
I want to write a program which will clone remote git repository and then do bunch of other stuff. The problem is that ‘git clone’ asks for password. It does not work when I open pipes to stdin/out/err to ‘git clone’ because it runs git-remote-http underneath which prompts for password on TTY.
I would like to pass the password from my program. I am using Python and Popen from subprocess. Code below does not wotk.
Popen(['git', 'clone', 'https://my.git.repo/repo.git'], shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)
How can I achieve this?
If you do not want to use challenge authentication as the commenter said, I would use pexpect to automate this kind of interaction
You can git clone https://foo:[email protected]/repo.git to get the data, and git remote set-url origin https://my.git.repo/repo.git afterwards to clear the password. But you have an race-condition between the start of the clone, and the URL-change.
If you are fine with using a library, you could try using Dulwich.
from dulwich import porcelain
porcelain.clone("https://example.com/repo.git", username="youruser", password="yourpassword")
This does not seem to store username or password (I’ve searched for the used password within the .git directory and did not find anything).
Sources
Because it was quite a hassle to find an up-to-date Git library without any obvious problems, I want to list my sources for future reference:
- Answer to "Python Git Module experiences? [closed]" by PTBNL from 2009 that recommends (among others) to use Dulwich
- Answer to "Use Git commands within Python code" by Trishayyyy from 2019 that still recommends Dulwich (and PyGit)
- Dulwich
porcelaindocumentation aboutclonethat sadly does not mention that authentication is possible at all. - Answer to "dulwich – clone from remote repo authentication" by harvin that mentions authentication is indeed possible with Dulwich
clone, using theusername:password@syntax. - Inspired by the last linked answer, I inspected the source code a bit further and stumbled upon the
usernameandpasswordparameters. I’ve documented this in my answer to the "dulwich – clone from remote repo authentication" question.