Get line number where variable is defined
Question:
I’m trying to extract the line number where a specific variable was created. This sounds like a simple question, after all, I could loop through the lines of code and try to search, or do a regular expression. But that might be a false line number and I would prefer, if possible, to parse the Python code as-is and get a real line number.
Here’s what I have:
class MyClass:
"""
MyClass.
Notice that we do override the name class attribute.
class MyClass:
name = "a real name"
"""
name = "the true name"
def __iinit__(self):
pass
Thanks to inspect.getsourcelines, I can get the lines of the class (more than one class might be defined in a file and they might all have a different name attribute). What I need from this is to find the line number where the name class attribute was defined for this class. Use ast? dis? I’m not against any suggestion.
This is useful for debugging. The class attribute in my case is a multi-line string containing some formatted text and I need to report the error with file name and line number when problems occur. So I would rather know where the attribute is defined, not just have its value.
Thanks for your help,
Edit
Based on @bsbueno ‘s answer, I tried to come up with something simple that didn’t disturb my original design, flowed as it probably is. I’m creating a simple metaclass whose sole job is to register this attribute. Notice that I’m still “wrapping” the attribute in a function call in the Child class, so this solution isn’t ideal and it’s not much better than defining the register function in a separate namespace. All in all, what I’ve done is not the best option, but it provides another example and might be useful for others, or so I hope:
import sys
class MetaRegister(type):
def __prepare__(name, bases, **kwds):
return {"register": MetaRegister.register}
@staticmethod
def register(value):
frame = sys._getframe().f_back
file = frame.f_globals["__file__"]
line = frame.f_lineno
print(f"Calling register {file}:{line}: {value!r}")
return value
class Parent(metaclass=MetaRegister):
option = None
class Child(Parent):
option = register(3)
The metaclass’ __prepare__ method is called before the other classes (both Parent and Child) are created. It adds something in their namespace (in this case, the register function). This is used directly in the Child‘s class body. The register function (which is actually a static method on the metaclass itself) doesn’t do much but printing where it was first called. This is enough for me, though as I’ve said, it’s a solution that could feel like it’s not solving much. Open opinion!
Answers:
While for functions and global variables this would be more or less straightforward. tough weird, using “dis”, for class attributes this might be a lot more convoluted – as class bodies are first compiled to a code object, that is run once when the class is defined, and all but “thrown away” once the class is properly created.
At that point you have only the class’__dict__ with no hints of order of creation or place of creation at all.
Also, using dis will make it possible to find where variables are attributed by searching the STORE_NAME opcode, you have the same problem of not knowing which of those was run – but it would be no more reliable than a textual search for patterns on the source code itself.
The only reliable way of doing this is, if instead of using regular variables, you use special descriptors – those could inspect the place where the call to set their values was made, and annotate that in a registry.
So, all you’d need is that the variables you want to trace live inside a special object, which can have a short name – the final code would look like this:
from variable_registry import R
class MyClass:
R.name = "the true name"
def __init__(self):
print(R.name)
This would need special handling if you need instance variables – but it still could be done by a more convoluted mechanism. For class and global variables you’d be good as for local variables in a non-concurrent execution.
import sys
class Registry:
def __init__(self):
self.__dict__["_registry"] = {}
def __setattr__(self, name, value):
frame = sys._getframe().f_back
file = frame.f_globals["__file__"]
self.__dict__["_registry"][name] = file, frame.f_lineno, value
super().__setattr__(name, value)
R = Registry()
from inspect import currentframe
def get_linenumber():
cf = currentframe()
return cf.f_back.f_lineno
I’m trying to extract the line number where a specific variable was created. This sounds like a simple question, after all, I could loop through the lines of code and try to search, or do a regular expression. But that might be a false line number and I would prefer, if possible, to parse the Python code as-is and get a real line number.
Here’s what I have:
class MyClass:
"""
MyClass.
Notice that we do override the name class attribute.
class MyClass:
name = "a real name"
"""
name = "the true name"
def __iinit__(self):
pass
Thanks to inspect.getsourcelines, I can get the lines of the class (more than one class might be defined in a file and they might all have a different name attribute). What I need from this is to find the line number where the name class attribute was defined for this class. Use ast? dis? I’m not against any suggestion.
This is useful for debugging. The class attribute in my case is a multi-line string containing some formatted text and I need to report the error with file name and line number when problems occur. So I would rather know where the attribute is defined, not just have its value.
Thanks for your help,
Edit
Based on @bsbueno ‘s answer, I tried to come up with something simple that didn’t disturb my original design, flowed as it probably is. I’m creating a simple metaclass whose sole job is to register this attribute. Notice that I’m still “wrapping” the attribute in a function call in the Child class, so this solution isn’t ideal and it’s not much better than defining the register function in a separate namespace. All in all, what I’ve done is not the best option, but it provides another example and might be useful for others, or so I hope:
import sys
class MetaRegister(type):
def __prepare__(name, bases, **kwds):
return {"register": MetaRegister.register}
@staticmethod
def register(value):
frame = sys._getframe().f_back
file = frame.f_globals["__file__"]
line = frame.f_lineno
print(f"Calling register {file}:{line}: {value!r}")
return value
class Parent(metaclass=MetaRegister):
option = None
class Child(Parent):
option = register(3)
The metaclass’ __prepare__ method is called before the other classes (both Parent and Child) are created. It adds something in their namespace (in this case, the register function). This is used directly in the Child‘s class body. The register function (which is actually a static method on the metaclass itself) doesn’t do much but printing where it was first called. This is enough for me, though as I’ve said, it’s a solution that could feel like it’s not solving much. Open opinion!
While for functions and global variables this would be more or less straightforward. tough weird, using “dis”, for class attributes this might be a lot more convoluted – as class bodies are first compiled to a code object, that is run once when the class is defined, and all but “thrown away” once the class is properly created.
At that point you have only the class’__dict__ with no hints of order of creation or place of creation at all.
Also, using dis will make it possible to find where variables are attributed by searching the STORE_NAME opcode, you have the same problem of not knowing which of those was run – but it would be no more reliable than a textual search for patterns on the source code itself.
The only reliable way of doing this is, if instead of using regular variables, you use special descriptors – those could inspect the place where the call to set their values was made, and annotate that in a registry.
So, all you’d need is that the variables you want to trace live inside a special object, which can have a short name – the final code would look like this:
from variable_registry import R
class MyClass:
R.name = "the true name"
def __init__(self):
print(R.name)
This would need special handling if you need instance variables – but it still could be done by a more convoluted mechanism. For class and global variables you’d be good as for local variables in a non-concurrent execution.
import sys
class Registry:
def __init__(self):
self.__dict__["_registry"] = {}
def __setattr__(self, name, value):
frame = sys._getframe().f_back
file = frame.f_globals["__file__"]
self.__dict__["_registry"][name] = file, frame.f_lineno, value
super().__setattr__(name, value)
R = Registry()
from inspect import currentframe
def get_linenumber():
cf = currentframe()
return cf.f_back.f_lineno