Pass Variable On Import
Question:
Let’s say you have some time-consuming work to do when a module/class is first imported. This functionality is dependent on a passed in variable. It only needs to be done when the module/class is loaded. All instances of the class can then use the result.
For instance, I’m using rpy2:
import rpy2.robjects as robjects
PATH_TO_R_SOURCE = ## I need to pass this
robjects.r.source(PATH_TO_R_SOURCE, chdir = True) ## this takes time
class SomeClass:
def __init__(self, aCurve):
self._curve = aCurve
def processCurve(self):
robjects.r['someRFunc'](robjects.FloatVector(self._curve))
Am I stuck creating a module level function that I call to do the work?
import someClass
someClass.sourceRStuff(PATH_TO_R_SOURCE)
x = someClass.SomeClass([1,2,3,4])
etc...
Answers:
Having a module init function isn’t unheard of. Pygame does it for the sdl initialization functions. So yes, your best bet is probably
import someModule
someModule.init(NECESSARY_DATA)
x = someModule.someClass(range(1, 5))
No you’re not stuck with a module level function, it’s just probably the best option. You could also use the builtin staticmethod or classmethod decorators to make it a method on someSclass that can be called before it is instantiated.
This would make sense only if everything other than someClass was usable without the initialization and I still think a module level function is better.
Could you benefit from a Proxy which implements lazy loading?
Check out the Active State “Lazy Module Imports” recipe.
There is no way to pass a variable at import.
Some ideas:
- make the module get the variable from the calling module using inspection; not very pythonic
- use an Init function for the module, this is the best way
Couple of other options that can achieve your goal (although a init() function is probably cleaner):
- Use an environment variable
- Use a separate module M to hold this variable, that the importer would set. Then the imported module could either know where to find M, or could rely on sys.meta_path to obtain it.
I had to do something similar for my project. If you don’t want to rely on the calling script to run the initialization function, you can add your own Python builtin which is then available to all modules at runtime.
Be careful to name your builtin something unique that is unlikely to cause a namespace collision (eg myapp_myvarname).
run.py
import __builtin__
__builtin__.myapp_PATH_TO_R_SOURCE = 'path.to.r.source'
import someClass
someClass module .py
import rpy2.robjects as robjects
import __builtin__
if hasattr(__builtin__, "myapp_PATH_TO_R_SOURCE"):
PATH_TO_R_SOURCE = __builtin__.myapp_PATH_TO_R_SOURCE
else:
PATH_TO_R_SOURCE = ## Some default value or Null for Exception handling
robjects.r.source(PATH_TO_R_SOURCE, chdir = True)
...
This works well for variables that may have a default but you want to allow overriding at import time. If the __builtin__ variable is not set, it will use a default value.
Edit: Some consider this an example of “Monkey patching”. For a more elegant solution without monkey patch, see my other answer.
There are two solutions I can think of, both of which are very work-around-like solutions. The first is to ditch imports and run your script like this
sys.argv[1] = "argument 1"
sys.argv[2] = "argument 2"
execfile("/path/to/dependency.py") #depreciated in python 3.x
The second is to put your arguments into an external temporary file, then read from that file in the dependency.
If there is only one configuration item to set, then I have found overriding the python __builtin__ to work just fine, but it is an example of “Monkey patching” which is frowned on by some.
A cleaner way to do it which is very useful for multiple configuration items in your project is to create a separate Configuration module that is imported by your wrapping code first, and the items set at runtime, before your functional module imports it. This pattern is often used in other projects.
myconfig/__init__.py :
PATH_TO_R_SOURCE = '/default/R/source/path'
OTHER_CONFIG_ITEM = 'DEFAULT'
PI = 3.14
mymodule/__init__.py :
import myconfig
PATH_TO_R_SOURCE = myconfig.PATH_TO_R_SOURCE
robjects.r.source(PATH_TO_R_SOURCE, chdir = True) ## this takes time
class SomeClass:
def __init__(self, aCurve):
self._curve = aCurve
if myconfig.VERSION is not None:
version = myconfig.VERSION
else:
version = "UNDEFINED"
two_pi = myconfig.PI * 2
And you can change the behaviour of your module at runtime from the wrapper:
run.py :
import myconfig
myconfig.PATH_TO_R_SOURCE = 'actual/path/to/R/source'
myconfig.PI = 3.14159
# we can even add a new configuration item that isn't present in the original myconfig:
myconfig.VERSION="1.0"
import mymodule
print "Mymodule.two_pi = %r" % mymodule.two_pi
print "Mymodule.version is %s" % mymodule.version
Output:
> Mymodule.two_pi = 6.28318
> Mymodule.version is 1.0
in any imported module you can do something like,
if ___name__!="__main__":
import __main__ as mn
x=mn.x
where you can use x as in the imported module, first by setting x in main,
also you can do something like mn.y=55 , so in main module after import you will be able to use y variable.
Let’s say you have some time-consuming work to do when a module/class is first imported. This functionality is dependent on a passed in variable. It only needs to be done when the module/class is loaded. All instances of the class can then use the result.
For instance, I’m using rpy2:
import rpy2.robjects as robjects
PATH_TO_R_SOURCE = ## I need to pass this
robjects.r.source(PATH_TO_R_SOURCE, chdir = True) ## this takes time
class SomeClass:
def __init__(self, aCurve):
self._curve = aCurve
def processCurve(self):
robjects.r['someRFunc'](robjects.FloatVector(self._curve))
Am I stuck creating a module level function that I call to do the work?
import someClass
someClass.sourceRStuff(PATH_TO_R_SOURCE)
x = someClass.SomeClass([1,2,3,4])
etc...
Having a module init function isn’t unheard of. Pygame does it for the sdl initialization functions. So yes, your best bet is probably
import someModule
someModule.init(NECESSARY_DATA)
x = someModule.someClass(range(1, 5))
No you’re not stuck with a module level function, it’s just probably the best option. You could also use the builtin staticmethod or classmethod decorators to make it a method on someSclass that can be called before it is instantiated.
This would make sense only if everything other than someClass was usable without the initialization and I still think a module level function is better.
Could you benefit from a Proxy which implements lazy loading?
Check out the Active State “Lazy Module Imports” recipe.
There is no way to pass a variable at import.
Some ideas:
- make the module get the variable from the calling module using inspection; not very pythonic
- use an Init function for the module, this is the best way
Couple of other options that can achieve your goal (although a init() function is probably cleaner):
- Use an environment variable
- Use a separate module M to hold this variable, that the importer would set. Then the imported module could either know where to find M, or could rely on sys.meta_path to obtain it.
I had to do something similar for my project. If you don’t want to rely on the calling script to run the initialization function, you can add your own Python builtin which is then available to all modules at runtime.
Be careful to name your builtin something unique that is unlikely to cause a namespace collision (eg myapp_myvarname).
run.py
import __builtin__
__builtin__.myapp_PATH_TO_R_SOURCE = 'path.to.r.source'
import someClass
someClass module .py
import rpy2.robjects as robjects
import __builtin__
if hasattr(__builtin__, "myapp_PATH_TO_R_SOURCE"):
PATH_TO_R_SOURCE = __builtin__.myapp_PATH_TO_R_SOURCE
else:
PATH_TO_R_SOURCE = ## Some default value or Null for Exception handling
robjects.r.source(PATH_TO_R_SOURCE, chdir = True)
...
This works well for variables that may have a default but you want to allow overriding at import time. If the __builtin__ variable is not set, it will use a default value.
Edit: Some consider this an example of “Monkey patching”. For a more elegant solution without monkey patch, see my other answer.
There are two solutions I can think of, both of which are very work-around-like solutions. The first is to ditch imports and run your script like this
sys.argv[1] = "argument 1"
sys.argv[2] = "argument 2"
execfile("/path/to/dependency.py") #depreciated in python 3.x
The second is to put your arguments into an external temporary file, then read from that file in the dependency.
If there is only one configuration item to set, then I have found overriding the python __builtin__ to work just fine, but it is an example of “Monkey patching” which is frowned on by some.
A cleaner way to do it which is very useful for multiple configuration items in your project is to create a separate Configuration module that is imported by your wrapping code first, and the items set at runtime, before your functional module imports it. This pattern is often used in other projects.
myconfig/__init__.py :
PATH_TO_R_SOURCE = '/default/R/source/path'
OTHER_CONFIG_ITEM = 'DEFAULT'
PI = 3.14
mymodule/__init__.py :
import myconfig
PATH_TO_R_SOURCE = myconfig.PATH_TO_R_SOURCE
robjects.r.source(PATH_TO_R_SOURCE, chdir = True) ## this takes time
class SomeClass:
def __init__(self, aCurve):
self._curve = aCurve
if myconfig.VERSION is not None:
version = myconfig.VERSION
else:
version = "UNDEFINED"
two_pi = myconfig.PI * 2
And you can change the behaviour of your module at runtime from the wrapper:
run.py :
import myconfig
myconfig.PATH_TO_R_SOURCE = 'actual/path/to/R/source'
myconfig.PI = 3.14159
# we can even add a new configuration item that isn't present in the original myconfig:
myconfig.VERSION="1.0"
import mymodule
print "Mymodule.two_pi = %r" % mymodule.two_pi
print "Mymodule.version is %s" % mymodule.version
Output:
> Mymodule.two_pi = 6.28318
> Mymodule.version is 1.0
in any imported module you can do something like,
if ___name__!="__main__":
import __main__ as mn
x=mn.x
where you can use x as in the imported module, first by setting x in main,
also you can do something like mn.y=55 , so in main module after import you will be able to use y variable.