Windows Python (<=3.10.2) fails to run `python -m venv .venv`
Question:
This issue has been solved, resulted in a bug report to Python.org. See the my self-answer below for the workaround until it’s fixed in a future release of Python
One of my PCs got bitten by this bug which no longer allows me to create venv with the error:
Error: Command '['C:\Users\kesh\test\.venv\Scripts\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 101.
- This issue has been known, chronologically: v3.7.2, v3.8, v3.?, & v3.10.1
- The only known solution is to give up per-user install and use global install by checking "Install for all users" option
I’m trying to figure out what exactly is happening, but quickly running out of ideas. Here are what I’ve tried so far:
- On my PC, "Install for all users" works as well as per-user install on a dummy account (all using the same v3.10.2 installer). This singles out the issue to be on my Windows account. Changing the install location does not help.
- Went into
venv source by running Python with venv.main(args=('.venv',)), debugging line-by-line and noted that it copies Libvenvscriptsntpython.exe from the python install dir to the local .venvScripts folder using shutil.copyfile().
- If I run the original
Libvenvscriptsntpython.exe in command prompt, it runs with a message No pyvenv.cfg file (which makes sense as the .cfg file is in .venv folder which it couldn’t see)
- If I call the copied
.venvScriptspython.exe then it returns an error Unable to create process using 'C:UserskeshAppDataLocalProgramsPythonPython310python.exe' (note that the python.exe path for the process is that of the installed Python exe)
- If
.venv is installed successfully (on the dummy Windows account), the above run starts a Python session as you’d expect.
venvscriptsntpython.exe is different from the standard python binary and verified that this file and its source in venvScriptsnt are identical.- All this points to that something in my account configuration is bothering the
.venvScriptspython.exe to do the right thing, but my environmental variables are pretty clean and python paths are at the top of the user PATH variable.
- Currently trying to locate the source code of
.venvScriptspython.exe but not found it yet.
- Can it be something in registry?
If you have any other ideas to try, please share.
Update #1:
- Found the source of the error message
PC/launcher.c Line 814
- Possibility:
CreateProcessW(NULL, cmdline,... where cmdline is the original python path in the error message, without quote. CreateProcessW documentation states executable name is deduced from the first white space–delimited token in the cmdline string. Though I replaced my actual account name with kesh it actually comprises two words and a space…
Update #2:
Solution found as posted below
Answers:
Bingo, the finding in the update #1 was the cause. The space in my username was the culprit. Although I have no idea what triggered this behavior change on my account… (anybody with an answer, please follow up.)
Let’s say the per-user python is installed at
C:UsersUser NameAppDataLocalProgramsPythonPython310
In my case, "Microsoft Visual C++ 2015-2022 Redistributable" installer (VC_redist.x64.exe) left a log file C:UsersUser (a text file with the first part of my account name as its file name). This caused python venv to use C:UsersUser as the python executable and promptly failed (see the issue tracker link below for the full explanation).
You can fix the issue in 2 ways until Python patches the problem.
Easy Fix
Simply delete the file C:UsersUser
Note: This will work until next time another installer leaves this nasty log file.
More Involved Fix
In command console, run
DIR /X C:Users
which lists something like:
02/08/2022 11:44 AM <DIR> .
02/08/2022 11:44 AM <DIR> ..
11/19/2020 01:48 AM <DIR> Public
02/08/2022 02:47 PM <DIR> USERNA~1 User Name
Open the Environmental Variables dialog and edit Path user variable’s Python entries from
C:UsersUser NameAppDataLocalProgramsPythonPython310Scripts
C:UsersUser NameAppDataLocalProgramsPythonPython310
to
C:UsersUSERNA~1AppDataLocalProgramsPythonPython310Scripts
C:UsersUSERNA~1AppDataLocalProgramsPythonPython310
Re-open python console window or app so the path change is applied to your dev environment.
Note: This fix will work as long as you don’t update python version. When you do, you need to delete the old path entries manually and update the new path entries.
Eventual Fix
I reported this bug to python bug tracker: Issue 46686. They’ve acknowledged the bug and have it labeled as critical with a proposed fix. So, hopefully it will get fixed in a near future release. (>3.10.2)
But where can I find this the file C:UsersUser ???
This issue has been solved, resulted in a bug report to Python.org. See the my self-answer below for the workaround until it’s fixed in a future release of Python
One of my PCs got bitten by this bug which no longer allows me to create venv with the error:
Error: Command '['C:\Users\kesh\test\.venv\Scripts\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 101.
- This issue has been known, chronologically: v3.7.2, v3.8, v3.?, & v3.10.1
- The only known solution is to give up per-user install and use global install by checking "Install for all users" option
I’m trying to figure out what exactly is happening, but quickly running out of ideas. Here are what I’ve tried so far:
- On my PC, "Install for all users" works as well as per-user install on a dummy account (all using the same v3.10.2 installer). This singles out the issue to be on my Windows account. Changing the install location does not help.
- Went into
venvsource by running Python withvenv.main(args=('.venv',)), debugging line-by-line and noted that it copiesLibvenvscriptsntpython.exefrom the python install dir to the local.venvScriptsfolder usingshutil.copyfile(). - If I run the original
Libvenvscriptsntpython.exein command prompt, it runs with a messageNo pyvenv.cfg file(which makes sense as the .cfg file is in.venvfolder which it couldn’t see) - If I call the copied
.venvScriptspython.exethen it returns an errorUnable to create process using 'C:UserskeshAppDataLocalProgramsPythonPython310python.exe'(note that thepython.exepath for the process is that of the installed Python exe) - If
.venvis installed successfully (on the dummy Windows account), the above run starts a Python session as you’d expect. venvscriptsntpython.exeis different from the standardpythonbinary and verified that this file and its source invenvScriptsntare identical.- All this points to that something in my account configuration is bothering the
.venvScriptspython.exeto do the right thing, but my environmental variables are pretty clean and python paths are at the top of the user PATH variable. - Currently trying to locate the source code of
.venvScriptspython.exebut not found it yet. - Can it be something in registry?
If you have any other ideas to try, please share.
Update #1:
- Found the source of the error message
PC/launcher.cLine 814 - Possibility:
CreateProcessW(NULL, cmdline,...wherecmdlineis the original python path in the error message, without quote. CreateProcessW documentation states executable name is deduced from the first white space–delimited token in thecmdlinestring. Though I replaced my actual account name withkeshit actually comprises two words and a space…
Update #2:
Solution found as posted below
Bingo, the finding in the update #1 was the cause. The space in my username was the culprit. Although I have no idea what triggered this behavior change on my account… (anybody with an answer, please follow up.)
Let’s say the per-user python is installed at
C:UsersUser NameAppDataLocalProgramsPythonPython310
In my case, "Microsoft Visual C++ 2015-2022 Redistributable" installer (VC_redist.x64.exe) left a log file C:UsersUser (a text file with the first part of my account name as its file name). This caused python venv to use C:UsersUser as the python executable and promptly failed (see the issue tracker link below for the full explanation).
You can fix the issue in 2 ways until Python patches the problem.
Easy Fix
Simply delete the file C:UsersUser
Note: This will work until next time another installer leaves this nasty log file.
More Involved Fix
In command console, run
DIR /X C:Users
which lists something like:
02/08/2022 11:44 AM <DIR> .
02/08/2022 11:44 AM <DIR> ..
11/19/2020 01:48 AM <DIR> Public
02/08/2022 02:47 PM <DIR> USERNA~1 User Name
Open the Environmental Variables dialog and edit Path user variable’s Python entries from
C:UsersUser NameAppDataLocalProgramsPythonPython310Scripts
C:UsersUser NameAppDataLocalProgramsPythonPython310
to
C:UsersUSERNA~1AppDataLocalProgramsPythonPython310Scripts
C:UsersUSERNA~1AppDataLocalProgramsPythonPython310
Re-open python console window or app so the path change is applied to your dev environment.
Note: This fix will work as long as you don’t update python version. When you do, you need to delete the old path entries manually and update the new path entries.
Eventual Fix
I reported this bug to python bug tracker: Issue 46686. They’ve acknowledged the bug and have it labeled as critical with a proposed fix. So, hopefully it will get fixed in a near future release. (>3.10.2)
But where can I find this the file C:UsersUser ???