How to fake/proxy a class in Python
Question:
I wrote some wrapper which has another object as an attribute. This wrapper proxies (forwards) all attribute requests with __getattr__ and __setattr__ to the object stored as the attribute. What else do I need to provide for my proxy so that the wrapper looks like the wrapped class under usual circumstances?
I suppose I need to fix things like inheritance, maybe __repr__, …
What else do I need to take care of and how do I fix inheritance so that instanceof() works?
EDIT: My attempt to make a function proxy, however as I don’t understand the recipe fully, it fails 🙁
setattr_=object.__setattr__
getattr_=object.__getattribute__
class Proxy(object):
__slots__=["_func", "_params", "_kwargs", "_obj", "_loaded", "__weakref__"]
def __init__(self, func, *params, **kwargs):
setattr_(self, "_func", func)
setattr_(self, "_params", params)
setattr_(self, "_kwargs", kwargs)
setattr_(self, "_obj", None)
setattr_(self, "_loaded", False)
def _get_obj(self):
if getattr_(self, "_loaded")==False:
print("Loading")
setattr_(self, "_obj", getattr_(self, "_func")(*getattr_(self, "_params"), **getattr_(self, "_kwargs")))
setattr_(self, "_loaded", True)
return getattr_(self, "_obj")
#
# proxying (special cases)
#
def __getattribute__(self, name):
return getattr(getattr_(self, "_get_obj")(), name)
def __delattr__(self, name):
delattr(getattr_(self, "_get_obj")(), name)
def __setattr__(self, name, value):
setattr(getattr_(self, "_get_obj")(), name, value)
def __nonzero__(self):
return bool(getattr_(self, "_get_obj")())
def __str__(self):
return str(getattr_(self, "_get_obj")())
def __repr__(self):
return repr(getattr_(self, "_get_obj")())
#
# factories
#
_special_names=[
'__abs__', '__add__', '__and__', '__call__', '__cmp__', '__coerce__',
'__contains__', '__delitem__', '__delslice__', '__div__', '__divmod__',
'__eq__', '__float__', '__floordiv__', '__ge__', '__getitem__',
'__getslice__', '__gt__', '__hash__', '__hex__', '__iadd__', '__iand__',
'__idiv__', '__idivmod__', '__ifloordiv__', '__ilshift__', '__imod__',
'__imul__', '__int__', '__invert__', '__ior__', '__ipow__', '__irshift__',
'__isub__', '__iter__', '__itruediv__', '__ixor__', '__le__', '__len__',
'__long__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__',
'__neg__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__',
'__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__',
'__repr__', '__reversed__', '__rfloorfiv__', '__rlshift__', '__rmod__',
'__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__',
'__rtruediv__', '__rxor__', '__setitem__', '__setslice__', '__sub__',
'__truediv__', '__xor__', 'next',
]
@classmethod
def _create_class_proxy(cls, theclass):
"""creates a proxy for the given class"""
def make_method(name):
def method(self, *args, **kw):
return getattr(getattr_(self, "_get_obj")(), name)(*args, **kw)
return method
namespace={}
for name in cls._special_names:
if hasattr(theclass, name):
namespace[name]=make_method(name)
return type("%s(%s)"%(cls.__name__, theclass.__name__), (cls,), namespace)
def __new__(cls, obj, *args, **kwargs):
"""
creates an proxy instance referencing `obj`. (obj, *args, **kwargs) are
passed to this class' __init__, so deriving classes can define an
__init__ method of their own.
note: _class_proxy_cache is unique per deriving class (each deriving
class must hold its own cache)
"""
try:
cache=cls.__dict__["_class_proxy_cache"]
except KeyError:
cls._class_proxy_cache=cache={}
try:
theclass=cache[obj.__class__]
except KeyError:
cache[obj.__class__]=theclass=cls._create_class_proxy(obj.__class__)
ins=object.__new__(theclass)
theclass.__init__(ins, obj, *args, **kwargs)
return ins
if __name__=='__main__':
def t(x, y):
print("Running t")
return x+y
a=Proxy(t, "a", "b")
print("Go")
print(a+"c")
Answers:
This problem is reasonably well addressed by this recipe:
The general idea you have to follow is that most methods on classes are accessed through some combination of __getattr__ and __getattribute__ either on the class itself, or on its metaclass, but this does not apply to python special methods, the ones that begin and end with double underscore, for those to be found, they must be actual methods on the actual class, no attribute proxying is possible.
Which of those methods you must provide will obviously depend on which of those methods the proxied class itself offers. The special methods you need to provide for isinstance() are the __instancecheck__ and __subclasscheck__ methods. for repr() to work, you must also define an appropriate __repr__() on the proxy class itself.
Generally, you can use the wrapt library (pypi), which does the heavy lifting for you:
The wrapt module focuses very much on correctness. It therefore goes way beyond existing mechanisms such as functools.wraps() to ensure that decorators preserve introspectability, signatures, type checking abilities etc. […]
To ensure that the overhead is as minimal as possible, a C extension module is used for performance critical components
It supports creating custom wrapper classes. In order to add your own attributes, you need to declare them in such a way that wrapt doesn’t try to pass them on to the wrapped instance. You can:
- Prefix the attribute with
_self_ and add properties for access
- Declare the attribute at the class level, in addition to in
__init__
-
Use slots, if appropriate for your class (not mentioned in the docs), like this:
class ExtendedMesh(ObjectProxy):
__slots__ = ('foo')
def __init__(self, subject):
super().__init__(subject)
self.foo = "bar"
It also supports function wrappers, which might suit your purpose.
in case someone else is looking for a more complete proxy implementation
Although there are several python proxy solutions similar to those OP is looking for, I could not find a solution that will also proxy classes and arbitrary class objects, as well as automatically proxy functions return values and arguments. Which is what I needed.
I’ve got some code written for that purpose as part of a full proxying/logging python execution and I may make it into a separate library in the future. If anyone’s interested you can find the core of the code in a pull request. Drop me a line if you’d like this as a standalone library.
My code will automatically return wrapper/proxy objects for any proxied object’s attributes, functions and classes. The purpose is to log and replay some of the code, so i’ve got the equivalent “replay” code and some logic to store all proxied objects to a json file.
I wrote some wrapper which has another object as an attribute. This wrapper proxies (forwards) all attribute requests with __getattr__ and __setattr__ to the object stored as the attribute. What else do I need to provide for my proxy so that the wrapper looks like the wrapped class under usual circumstances?
I suppose I need to fix things like inheritance, maybe __repr__, …
What else do I need to take care of and how do I fix inheritance so that instanceof() works?
EDIT: My attempt to make a function proxy, however as I don’t understand the recipe fully, it fails 🙁
setattr_=object.__setattr__
getattr_=object.__getattribute__
class Proxy(object):
__slots__=["_func", "_params", "_kwargs", "_obj", "_loaded", "__weakref__"]
def __init__(self, func, *params, **kwargs):
setattr_(self, "_func", func)
setattr_(self, "_params", params)
setattr_(self, "_kwargs", kwargs)
setattr_(self, "_obj", None)
setattr_(self, "_loaded", False)
def _get_obj(self):
if getattr_(self, "_loaded")==False:
print("Loading")
setattr_(self, "_obj", getattr_(self, "_func")(*getattr_(self, "_params"), **getattr_(self, "_kwargs")))
setattr_(self, "_loaded", True)
return getattr_(self, "_obj")
#
# proxying (special cases)
#
def __getattribute__(self, name):
return getattr(getattr_(self, "_get_obj")(), name)
def __delattr__(self, name):
delattr(getattr_(self, "_get_obj")(), name)
def __setattr__(self, name, value):
setattr(getattr_(self, "_get_obj")(), name, value)
def __nonzero__(self):
return bool(getattr_(self, "_get_obj")())
def __str__(self):
return str(getattr_(self, "_get_obj")())
def __repr__(self):
return repr(getattr_(self, "_get_obj")())
#
# factories
#
_special_names=[
'__abs__', '__add__', '__and__', '__call__', '__cmp__', '__coerce__',
'__contains__', '__delitem__', '__delslice__', '__div__', '__divmod__',
'__eq__', '__float__', '__floordiv__', '__ge__', '__getitem__',
'__getslice__', '__gt__', '__hash__', '__hex__', '__iadd__', '__iand__',
'__idiv__', '__idivmod__', '__ifloordiv__', '__ilshift__', '__imod__',
'__imul__', '__int__', '__invert__', '__ior__', '__ipow__', '__irshift__',
'__isub__', '__iter__', '__itruediv__', '__ixor__', '__le__', '__len__',
'__long__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__',
'__neg__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__',
'__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__',
'__repr__', '__reversed__', '__rfloorfiv__', '__rlshift__', '__rmod__',
'__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__',
'__rtruediv__', '__rxor__', '__setitem__', '__setslice__', '__sub__',
'__truediv__', '__xor__', 'next',
]
@classmethod
def _create_class_proxy(cls, theclass):
"""creates a proxy for the given class"""
def make_method(name):
def method(self, *args, **kw):
return getattr(getattr_(self, "_get_obj")(), name)(*args, **kw)
return method
namespace={}
for name in cls._special_names:
if hasattr(theclass, name):
namespace[name]=make_method(name)
return type("%s(%s)"%(cls.__name__, theclass.__name__), (cls,), namespace)
def __new__(cls, obj, *args, **kwargs):
"""
creates an proxy instance referencing `obj`. (obj, *args, **kwargs) are
passed to this class' __init__, so deriving classes can define an
__init__ method of their own.
note: _class_proxy_cache is unique per deriving class (each deriving
class must hold its own cache)
"""
try:
cache=cls.__dict__["_class_proxy_cache"]
except KeyError:
cls._class_proxy_cache=cache={}
try:
theclass=cache[obj.__class__]
except KeyError:
cache[obj.__class__]=theclass=cls._create_class_proxy(obj.__class__)
ins=object.__new__(theclass)
theclass.__init__(ins, obj, *args, **kwargs)
return ins
if __name__=='__main__':
def t(x, y):
print("Running t")
return x+y
a=Proxy(t, "a", "b")
print("Go")
print(a+"c")
This problem is reasonably well addressed by this recipe:
The general idea you have to follow is that most methods on classes are accessed through some combination of __getattr__ and __getattribute__ either on the class itself, or on its metaclass, but this does not apply to python special methods, the ones that begin and end with double underscore, for those to be found, they must be actual methods on the actual class, no attribute proxying is possible.
Which of those methods you must provide will obviously depend on which of those methods the proxied class itself offers. The special methods you need to provide for isinstance() are the __instancecheck__ and __subclasscheck__ methods. for repr() to work, you must also define an appropriate __repr__() on the proxy class itself.
Generally, you can use the wrapt library (pypi), which does the heavy lifting for you:
The wrapt module focuses very much on correctness. It therefore goes way beyond existing mechanisms such as functools.wraps() to ensure that decorators preserve introspectability, signatures, type checking abilities etc. […]
To ensure that the overhead is as minimal as possible, a C extension module is used for performance critical components
It supports creating custom wrapper classes. In order to add your own attributes, you need to declare them in such a way that wrapt doesn’t try to pass them on to the wrapped instance. You can:
- Prefix the attribute with
_self_and add properties for access - Declare the attribute at the class level, in addition to in
__init__ -
Use slots, if appropriate for your class (not mentioned in the docs), like this:
class ExtendedMesh(ObjectProxy): __slots__ = ('foo') def __init__(self, subject): super().__init__(subject) self.foo = "bar"
It also supports function wrappers, which might suit your purpose.
in case someone else is looking for a more complete proxy implementation
Although there are several python proxy solutions similar to those OP is looking for, I could not find a solution that will also proxy classes and arbitrary class objects, as well as automatically proxy functions return values and arguments. Which is what I needed.
I’ve got some code written for that purpose as part of a full proxying/logging python execution and I may make it into a separate library in the future. If anyone’s interested you can find the core of the code in a pull request. Drop me a line if you’d like this as a standalone library.
My code will automatically return wrapper/proxy objects for any proxied object’s attributes, functions and classes. The purpose is to log and replay some of the code, so i’ve got the equivalent “replay” code and some logic to store all proxied objects to a json file.