Why system path behaviour in pycharm seems to be different that using directly the conda env?

Question:

this is actually my first question in stack overflow :D. As background: I started learning python by myself almost 1 year ago in parallel of my work (Industrial Engineer), so feel free to point any mistakes. Any feedback will be very appreciated (including the format of this question).

I was trying to a have a project structure with multiple folders where to organize the scripts clearly. Eveything was going peachy until I wanted to schedulesome scripts using bat files.

When running my scripts (with absolute imports) in Pycharm everything works without problems, but when I try to run same scripts via bat files the imports fails!

For this question I created a new (simplified) project and created a new conda enviroment (both called test) with a example of the structure of folders where I can reproduce this error. Inside those folders I have the a script (main.py) calling a function from another script (library.py)

enter image description here

main.py :

from A.B.C import library

library.Function_Alpha('hello world ')

library.py:

def Function_Alpha(txt):
    print(txt)

main.bat

"C:LocaldataANACONDAenvstestpython.exe" "C:/Users/bpereira/PycharmProjects/test/X/main.py"
pause

When I run the script using pycharm everything goes as expected:

C:LocaldataANACONDAenvstestpython.exe C:/Users/bpereira/PycharmProjects/test/X/main.py
hello world 

Process finished with exit code 0

But when I try running the bat file:

cmd.exe /c main.bat

C:UsersbpereiraPycharmProjectstestX>"C:LocaldataANACONDAenvstestpython.exe" "C:/Users/bpereira/PycharmProjects/test/X/main.py" 
Traceback (most recent call last):
  File "C:/Users/bpereira/PycharmProjects/test/X/main.py", line 1, in <module>
    from A.B.C import library
ModuleNotFoundError: No module named 'A'

C:UsersbpereiraPycharmProjectstestX>pause
Press any key to continue . . . 

Is Pycharm doing something with the system paths that I am not aware?

How I can emulate the behaviour of pycharm using the bat files?

I tried adding the system path manually in the script and it works:

*main.py:

import sys
sys.path.append(r'C:/Users/bpereira/PycharmProjects/test')

from A.B.C import library

library.Function_Alpha('hello world ')

main.bat execution:

cmd.exe /c main.bat

C:UsersbpereiraPycharmProjectstestX>"C:LocaldataANACONDAenvstestpython.exe" "C:/Users/bpereira/PycharmProjects/test/X/main.py" 
hello world 

C:UsersbpereiraPycharmProjectstestX>pause
Press any key to continue . . . 

But I am actually trying to understand how pycharm does this automatically and if I can reproduce that without having to append the sys.path on each script.

In the actual project when I do this contaiment (sys.path.append) the scripts are able to run but I face other errors like SLL module missing while calling the request function. Again this works flawlessly within pycharm but from the bat files the request module seems to behave differently, which I think is realted to the system paths.

(Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")

For info: I am running this on the company laptop where I do not have admin rights and I am not able to edit the system paths.

Asked By: BraisPF84

||

Answers:

Solved.
After some more investigation it was clear that I was facing 2 problems:

  1. Not declaring the env path in the system
  2. Not activating the virtual enviroment properly (hence the SSL error)

Since I do not have the admin rights of the laptop (corporate one) I solved the path issue by defining the the project path in the .bat file, adding the path temporally each time the env is activated.

The one-liner I wrote in the initial .bat is not activating the enviroment. The correct way is to call the ‘activate.bat’ in the conda folder.

Hereby the solution in the .bat file:

@echo off
rem Define path to conda installation and env name.
set CONDAPATH= #CondaPath
set ENVNAME= #EnvName

rem Activate the env
if %ENVNAME%==base (set ENVPATH=%CONDAPATH%) else (set ENVPATH=%CONDAPATH%envs%ENVNAME%)
call %CONDAPATH%Scriptsactivate.bat %ENVPATH%

set PYTHONPATH= #ProjectPath

rem Run a python script in that environment
python #ScriptPath_1
python #ScriptPath_2
python #ScriptPath_3

rem Deactivate the environment
call conda deactivate

Hope this helps someone trying to automate python scripts using the windows task scheduler with .bat files

Answered By: BraisPF84
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.