What does "domain" mean in Python's tracemalloc?
Question:
Python’s tracemalloc documentation defines "domain" as an "address space of a memory block" and identifies a domain using an integer. It also says:
tracemalloc uses the domain 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources
What are these address spaces to which the tracemalloc documentation is referring? As a C extension developer, how can I "use" a particular domain as the above quote suggests, and in what situations should I? As a Python package developer, in what situations should I use a DomainFilter when using tracemalloc?
Python’s memory management documentation uses the phrase "allocator domain," which seems to be unrelated. These domains are categories of allocator functions. They don’t seem to have integer identifiers like the domains in the tracemalloc documentation.
Answers:
The complexity comes from the idea that a C extension module can reuse the built-in memory tracer tracemalloc to trace memory blocks allocated by the custom allocation routine of the extension. The domain is an integer tag that can be used to distinguish address spaces. For example, a C extension can request to track a pointer 0x7f0000000001 in different address space(not one in the main memory), while there exists some object at 0x7f0000000001 in the main memory.(The pointer even doesn’t need to be an address. It can be a handle.)
You can call the C API PyTraceMalloc_Track() to request tracking a pointer. See the example in the alloc.c of NumPy.
In most cases, you deal with addresses in the main memory, so you can use the default domain, 0. And of course, you can use your own domain number to filter your memory allocations only.
Python’s tracemalloc documentation defines "domain" as an "address space of a memory block" and identifies a domain using an integer. It also says:
tracemalloc uses the domain 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources
What are these address spaces to which the tracemalloc documentation is referring? As a C extension developer, how can I "use" a particular domain as the above quote suggests, and in what situations should I? As a Python package developer, in what situations should I use a DomainFilter when using tracemalloc?
Python’s memory management documentation uses the phrase "allocator domain," which seems to be unrelated. These domains are categories of allocator functions. They don’t seem to have integer identifiers like the domains in the tracemalloc documentation.
The complexity comes from the idea that a C extension module can reuse the built-in memory tracer tracemalloc to trace memory blocks allocated by the custom allocation routine of the extension. The domain is an integer tag that can be used to distinguish address spaces. For example, a C extension can request to track a pointer 0x7f0000000001 in different address space(not one in the main memory), while there exists some object at 0x7f0000000001 in the main memory.(The pointer even doesn’t need to be an address. It can be a handle.)
You can call the C API PyTraceMalloc_Track() to request tracking a pointer. See the example in the alloc.c of NumPy.
In most cases, you deal with addresses in the main memory, so you can use the default domain, 0. And of course, you can use your own domain number to filter your memory allocations only.