pip install from Azure DevOps Python Artifacts feed not working
Question:
When I attempt to install a package from our Azure DevOps Artifacts feed, I get the error:
Looking in indexes: https://pypi.org/simple, https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
ERROR: Could not find a version that satisfies the requirement as-api (from versions: none)
ERROR: No matching distribution found for as-api
As using pip install -vvv potentially produces confidential information, I cannot provide the full log here. Please feel free to ask any specific questions about the log. In the meantime, I can see promising messages like:
Found index url https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
Getting credentials from keyring for https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
And some problematic messages?:
Status code 302 not in (200, 203, 300, 301)
Skipping link: not a file: ...
Given no hashes to check 0 links for project 'as-api': discarding no candidates
Reproduction details
virtualenv .venv
..venvScriptsactivate
python -m pip install -U pip
pip install keyring artifacts-keyring
pip install as-api
This link was used to produce a pipeline to publish the package and the suggested way of installing the package. My approach is now a mix of both option 1 and option 2. Note the use of a php.ini file to set --index-url and the artifacts-keyring package (installing with --pre does not make any difference to the version), so it really doesn’t make any difference. However, I have tried both options separately, it doesn’t spawn a browser, so it gives the same result.
System details:
- OS: Windows 10
- Python 2.7.17
pip list
Package Version
----------------- ----------
artifacts-keyring 0.2.8rc0
certifi 2019.11.28
chardet 3.0.4
configparser 4.0.2
entrypoints 0.3
idna 2.8
keyring 18.0.1
pip 19.3.1
pywin32-ctypes 0.2.0
requests 2.22.0
setuptools 42.0.2
urllib3 1.25.7
wheel 0.33.6
Folder structure:
test
|-- test.py
|-- .venv
|-- pip.ini
|-- ... other virtualenv folders and files
pip.ini:
[global]
extra-index-url = https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
Further analysis
-
Using a clean laptop actually works with the above reproduction details. Other computers in the company also have the same problem, so some of our set up is conflicting with the authentication.
-
If we use a pipeline (see this link) to install the as-api package, it works, so I suspect this is an authentication problem, but it’s not mentioned on any documentation.
-
Using https://username:password@… does not give any authentication error, even with wrong username and password.
-
Using the correct username but have symbols in the password triggers interactive mode to enter username and password. However, this gives this error: WARNING: 401 Error, Credentials not correct for https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/as-api/ Note that I am the owner of the Artifacts feed and the team has been added as the owner in the permission tab.
Answers:
As a workaround:
Looks like you’re using option2 from the document to do the install. I happen to see one similar issue which indicates this error message could have something to do with pip.ini(windows) or pip.conf(linux/mac), so I think you can try another approach to avoid something wrong with those configurations.
You can run pip install artifacts-keyring --pre and then run
pip install packageName --index-url https://pkgs.dev.azure.com/xxx/xxx/_packaging/xxx/pypi/simple/ -vvv --no-deps
You would meet something like this when running command pip install artifacts-keyring --pre:
After the login-in passes, you would get the package you need if it do exist in your feed.
The fix
Do one of the following:
-
Remove the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable (not very useful, not recommended).
-
Add an extra endpoint to the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable. E.g.,
{"endpointCredentials": [{"endpoint":"https://pkgs.dev.azure.com/company/_packaging/NuGetFeed/nuget/v3/index.json", ...},{"endpoint":"https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/", ...}]}
We have a script which sets up these endpoints, so this turns out to be a simple fix.
The cause
It turns out that if you have used artifacts-credprovider to set up another feed, in our case, a NuGet feed with another endpoint, the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable stores only that feed URL inside the key endpoint. artifacts-keyring will still read that environment variable even if the endpoint doesn’t exist, which causes authentication problem. The -vvv log doesn’t tell you anything about authentication and it won’t attempt to authenticate using another method.
My issue was that I had not installed artifacts-keyring. After that I could see VS Code authenticating to the feed and installing the package.
I also needed to upgrade pip (needs to be above > 19.2) with the following command:
python -m pip install --upgrade pip
Assuming your Azure DevOps artifacts is private and you have a PAT then installing a package from the artifact can be done in the following two ways
- If you have access to a terminal (only preferred in dev environment)
pip install https://<your-feed-name>:<your-PAT-key>@pkgs.dev.azure.com/<your-organization-name>/<your-project-name>/_packaging/<your-feed-name>/pypi/simple/ Your-Package-Name==x.x.x
Note: All the names (eg: feed, project) must follow the HTTPS URL convention.A simple (& actually correct) way to get to know the URL is goto Artifacts –> Select your artifact feed –> Connect to feed –> PIP –> Here you will get the correct URL. Also, use the some feed name both the place in URL
- Using
requirements.txt (this will be ideally used in prod or CI/CD pipeline) and automating the process:
Mind you it need a bit of string/URL manipulation. Add the respective line/URL in your requirements.txt in following manner:
- The URL will be mostly similar to the earlier URL used in the earlier terminal method
- In the URL after
simple everything will have to change, modified URL-
https://<your-feed-name>:<your-PAT-key>@pkgs.dev.azure.com/<your-organization-name>/<your-project-name>/_packaging/<your-feed-name>/pypi/download/<yourpackagename>/<package version>/Your-Package-Name.whl
#assuming your package is a .whl file
- So
simple changed to download; then whatever is your package name, whether it contains ‘-‘ or ‘_’ or CAPS, everything will be removed and converted into small case.
- Next is version no of your package that you want to install & finally the name of wheel or
.whl file.
As an update to @user:10097045 ‘s answer
You must add the –extra-index-url= infront of the URL with path in option 1 otherwise pip will fail to find the directory
Otherwise answer was super helpful you just get a 404 without that definition
When I attempt to install a package from our Azure DevOps Artifacts feed, I get the error:
Looking in indexes: https://pypi.org/simple, https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
ERROR: Could not find a version that satisfies the requirement as-api (from versions: none)
ERROR: No matching distribution found for as-api
As using pip install -vvv potentially produces confidential information, I cannot provide the full log here. Please feel free to ask any specific questions about the log. In the meantime, I can see promising messages like:
Found index url https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
Getting credentials from keyring for https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
And some problematic messages?:
Status code 302 not in (200, 203, 300, 301)
Skipping link: not a file: ...
Given no hashes to check 0 links for project 'as-api': discarding no candidates
Reproduction details
virtualenv .venv
..venvScriptsactivate
python -m pip install -U pip
pip install keyring artifacts-keyring
pip install as-api
This link was used to produce a pipeline to publish the package and the suggested way of installing the package. My approach is now a mix of both option 1 and option 2. Note the use of a php.ini file to set --index-url and the artifacts-keyring package (installing with --pre does not make any difference to the version), so it really doesn’t make any difference. However, I have tried both options separately, it doesn’t spawn a browser, so it gives the same result.
System details:
- OS: Windows 10
- Python 2.7.17
pip list
Package Version
----------------- ----------
artifacts-keyring 0.2.8rc0
certifi 2019.11.28
chardet 3.0.4
configparser 4.0.2
entrypoints 0.3
idna 2.8
keyring 18.0.1
pip 19.3.1
pywin32-ctypes 0.2.0
requests 2.22.0
setuptools 42.0.2
urllib3 1.25.7
wheel 0.33.6
Folder structure:
test
|-- test.py
|-- .venv
|-- pip.ini
|-- ... other virtualenv folders and files
pip.ini:
[global]
extra-index-url = https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/
Further analysis
-
Using a clean laptop actually works with the above reproduction details. Other computers in the company also have the same problem, so some of our set up is conflicting with the authentication.
-
If we use a pipeline (see this link) to install the
as-apipackage, it works, so I suspect this is an authentication problem, but it’s not mentioned on any documentation. -
Using https://username:password@… does not give any authentication error, even with wrong username and password.
-
Using the correct username but have symbols in the password triggers interactive mode to enter username and password. However, this gives this error:
WARNING: 401 Error, Credentials not correct for https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/as-api/Note that I am the owner of the Artifacts feed and the team has been added as the owner in the permission tab.
As a workaround:
Looks like you’re using option2 from the document to do the install. I happen to see one similar issue which indicates this error message could have something to do with pip.ini(windows) or pip.conf(linux/mac), so I think you can try another approach to avoid something wrong with those configurations.
You can run pip install artifacts-keyring --pre and then run
pip install packageName --index-url https://pkgs.dev.azure.com/xxx/xxx/_packaging/xxx/pypi/simple/ -vvv --no-deps
You would meet something like this when running command pip install artifacts-keyring --pre:
After the login-in passes, you would get the package you need if it do exist in your feed.
The fix
Do one of the following:
-
Remove the
VSS_NUGET_EXTERNAL_FEED_ENDPOINTSenvironment variable (not very useful, not recommended). -
Add an extra
endpointto theVSS_NUGET_EXTERNAL_FEED_ENDPOINTSenvironment variable. E.g.,
{"endpointCredentials": [{"endpoint":"https://pkgs.dev.azure.com/company/_packaging/NuGetFeed/nuget/v3/index.json", ...},{"endpoint":"https://pkgs.dev.azure.com/company/company_Software/_packaging/PyPI/pypi/simple/", ...}]}
We have a script which sets up these endpoints, so this turns out to be a simple fix.
The cause
It turns out that if you have used artifacts-credprovider to set up another feed, in our case, a NuGet feed with another endpoint, the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable stores only that feed URL inside the key endpoint. artifacts-keyring will still read that environment variable even if the endpoint doesn’t exist, which causes authentication problem. The -vvv log doesn’t tell you anything about authentication and it won’t attempt to authenticate using another method.
My issue was that I had not installed artifacts-keyring. After that I could see VS Code authenticating to the feed and installing the package.
I also needed to upgrade pip (needs to be above > 19.2) with the following command:
python -m pip install --upgrade pip
Assuming your Azure DevOps artifacts is private and you have a PAT then installing a package from the artifact can be done in the following two ways
- If you have access to a terminal (only preferred in dev environment)
pip install https://<your-feed-name>:<your-PAT-key>@pkgs.dev.azure.com/<your-organization-name>/<your-project-name>/_packaging/<your-feed-name>/pypi/simple/ Your-Package-Name==x.x.x
Note: All the names (eg: feed, project) must follow the HTTPS URL convention.A simple (& actually correct) way to get to know the URL is goto Artifacts –> Select your artifact feed –> Connect to feed –> PIP –> Here you will get the correct URL. Also, use the some feed name both the place in URL
- Using
requirements.txt(this will be ideally used in prod or CI/CD pipeline) and automating the process:
Mind you it need a bit of string/URL manipulation. Add the respective line/URL in your requirements.txt in following manner:
- The URL will be mostly similar to the earlier URL used in the earlier terminal method
- In the URL after
simpleeverything will have to change, modified URL-
https://<your-feed-name>:<your-PAT-key>@pkgs.dev.azure.com/<your-organization-name>/<your-project-name>/_packaging/<your-feed-name>/pypi/download/<yourpackagename>/<package version>/Your-Package-Name.whl
#assuming your package is a .whl file
- So
simplechanged todownload; then whatever is your package name, whether it contains ‘-‘ or ‘_’ or CAPS, everything will be removed and converted into small case. - Next is version no of your package that you want to install & finally the name of wheel or
.whlfile.
As an update to @user:10097045 ‘s answer
You must add the –extra-index-url= infront of the URL with path in option 1 otherwise pip will fail to find the directory
Otherwise answer was super helpful you just get a 404 without that definition