globals() Doesn't Work Inside of a Package
Question:
It is sometimes convenient to source part of a .py file into the global workspace when you’re doing interactive tasks (I am aware that exec has code smell"). I have a function that takes a file and regex markers of what start and end points to source lines from and assigns the contents of the file lines to the global workspace. This works as expected when you source the function in a script interactively but when I try to put this function into a convenience package it runs but the objects aren’t added to the global workspace.
How can I make this function add the sourced objects to the global workspace when it is added to a package?
Source Lines Function (Works until you put it in a package)
from re import search
from munch import DefaultMunch
import tempfile
import os
def source_lines(path:str, start_line:str, end_line:str):
with open(path, "r") as f:
script = f.readlines()
starts = [i for i, line in enumerate(script) if search(start_line, line)]
ends = [i for i, line in enumerate(script) if search(end_line, line)]
out = []
for s, e in zip(starts, ends):
out.append(script[s:e])
out_flat = [item for sublist in out for item in sublist]
with tempfile.TemporaryDirectory() as td:
f_name = os.path.join(td, 'tempsource.py')
with open(f_name, 'w+t') as fh:
fh.write('n'.join(out_flat))
fh.seek(0)
exec(open(f_name).read(), globals())
Test Code (I want this to work if source_lines is added to a package)
pretend_file = ['## Dependenciesn', 'from datetime import datetimen',
'#n', 'n', "data = 'd'*100n", 'n', '## Main funn',
'def date_to_epoch(date: str | datetime = None) -> str:n',
'n', ' if isinstance(date, datetime):n',
' date = str(date.date())n', 'n',
' return str(int(datetime.strptime(date, "%Y-%m-%d").timestamp() * 1000))n',
'n', 'n', 'n', 'n', 'n', 'n', "data2 = 'e'*100n", '#n',
'n', 'n', "s='d'"
]
with open('delete_me.py', 'a') as f:
f.write('n'.join(pretend_file))
## Source to global
source_lines(
path = 'delete_me.py',
start_line = '^## (Dependencies|Main fun)',
end_line = '^#s*$'
)
## Now have access to date_to_epoch, datetime, data2 (This does not work if `source_lines` was added to a package)
date_to_epoch(datetime.strptime('2023-02-02', '%Y-%m-%d'))
data2
os.remove('delete_me.py')
Answers:
If we’re all absolutely clear on the fact you shouldn’t be doing this and it’s a code smell and you’re playing with fire and this might break on interpreters that aren’t CPython, you can use inspect.currentframe() to get the current stack frame, f_back to get the calling frame, and f_globals to get the globals from there:
def source_lines(path:str, start_line:str, end_line:str, namespace=None):
if not namespace:
namespace = inspect.currentframe().f_back.f_globals
# ...
exec(..., namespace)
(For the curious, here’s CPython’s globals() implementation; it calls PyEval_GetGlobals(), which has a familiar-looking statement: return current_frame->f_globals.)
It is sometimes convenient to source part of a .py file into the global workspace when you’re doing interactive tasks (I am aware that exec has code smell"). I have a function that takes a file and regex markers of what start and end points to source lines from and assigns the contents of the file lines to the global workspace. This works as expected when you source the function in a script interactively but when I try to put this function into a convenience package it runs but the objects aren’t added to the global workspace.
How can I make this function add the sourced objects to the global workspace when it is added to a package?
Source Lines Function (Works until you put it in a package)
from re import search
from munch import DefaultMunch
import tempfile
import os
def source_lines(path:str, start_line:str, end_line:str):
with open(path, "r") as f:
script = f.readlines()
starts = [i for i, line in enumerate(script) if search(start_line, line)]
ends = [i for i, line in enumerate(script) if search(end_line, line)]
out = []
for s, e in zip(starts, ends):
out.append(script[s:e])
out_flat = [item for sublist in out for item in sublist]
with tempfile.TemporaryDirectory() as td:
f_name = os.path.join(td, 'tempsource.py')
with open(f_name, 'w+t') as fh:
fh.write('n'.join(out_flat))
fh.seek(0)
exec(open(f_name).read(), globals())
Test Code (I want this to work if source_lines is added to a package)
pretend_file = ['## Dependenciesn', 'from datetime import datetimen',
'#n', 'n', "data = 'd'*100n", 'n', '## Main funn',
'def date_to_epoch(date: str | datetime = None) -> str:n',
'n', ' if isinstance(date, datetime):n',
' date = str(date.date())n', 'n',
' return str(int(datetime.strptime(date, "%Y-%m-%d").timestamp() * 1000))n',
'n', 'n', 'n', 'n', 'n', 'n', "data2 = 'e'*100n", '#n',
'n', 'n', "s='d'"
]
with open('delete_me.py', 'a') as f:
f.write('n'.join(pretend_file))
## Source to global
source_lines(
path = 'delete_me.py',
start_line = '^## (Dependencies|Main fun)',
end_line = '^#s*$'
)
## Now have access to date_to_epoch, datetime, data2 (This does not work if `source_lines` was added to a package)
date_to_epoch(datetime.strptime('2023-02-02', '%Y-%m-%d'))
data2
os.remove('delete_me.py')
If we’re all absolutely clear on the fact you shouldn’t be doing this and it’s a code smell and you’re playing with fire and this might break on interpreters that aren’t CPython, you can use inspect.currentframe() to get the current stack frame, f_back to get the calling frame, and f_globals to get the globals from there:
def source_lines(path:str, start_line:str, end_line:str, namespace=None):
if not namespace:
namespace = inspect.currentframe().f_back.f_globals
# ...
exec(..., namespace)
(For the curious, here’s CPython’s globals() implementation; it calls PyEval_GetGlobals(), which has a familiar-looking statement: return current_frame->f_globals.)