Printing out traces to be read by a memory simulation application in Python Question: Trying to print out large traces of memory addresses (32 bits) for 2^32 addresses in the following manner: 0x12345678 W 0x23456789 R . . . . 0xFFFFFFFF W Basically going from one address to the other. So it could be going …
Measuring maximum memory while capturing stdout in Python using subprocess Question: Is there a clean way to measure the maximum memory consumption of a subprocess while still capturing stdout (and ideally setting a timeout) using subprocess in Python? Capturing output and setting a timeout can easily done using functions of subprocess: output = subprocess.run(cmd, capture_output=True, …
How to cleanup RAM between program iterations using Python Question: Problem I have a Python application inside Docker container. The application receives "jobs" from some queue service (RabbitMQ), does some computing tasks and uploads results into database (MySQL and Redis). The issue I face is – the RAM is not properly "cleaned up" between iterations …
How to log Python code memory consumption? Question: Question Hi, I am runnin’ a Docker container with a Python application inside. The code performs some computing tasks and I would like to monitor it’s memory consumption using logs (so I can see how different parts of the calculations perform). I do not need any charts …
python ctypes memory read stops at first zero integer Question: To all ctypes experts: I am running this code in Python to read 0x400 entries from memory. Unfortunately the returned list only contains ~20 entries. It seems to stop reading at the first entry with the value equal to 0. read_buffer = (ctypes.c_char * buffsize)() …
C++ vs Python | Read Memory Question: Have working code in C++ and would like to get equivalent results with Python. The idea is to retrieve data from memory using a specific process and a pointer. The result should look like this as it works in C++: Here is the C++ code: hProcess = SOME_HANDLER …
Python multiprocessing: sharing global real-only large data without reloading from disk for child processes Question: Say I need to read from disk a large data and do some read-only work on it. I need to use multiprocessing, but to share it across processes using multiprocessing.Manager() or Array() is way too slow. Since my operation on …
How to prevent the error "unable to allocate x.xx GiB for an array with shape (xx, xxx) and data type xxxxx" without restarting VS Code? Question: I am working with a lot of data in a jupyter notebook. From my research I understand that the memory (RAM/SWAP) is not sufficient to (temporarily) store my large …
NetCDF4 file with Python – Filter before dataframing Question: Due to a large NetCDF4 file, I get a MemoryError when I want to transform it into Pandas dataframe. But I don’t need everything from the netCDF4 file, so I wanted to know if I could cut the file priorly, and after transforming into dataframe My …
Rust BigInt memory allocation and performance compared to Python BigInt implementation Question: When working with num_bigint::BigInt I have run into some performance limitations, for context I am working on https://www.geeksforgeeks.org/how-to-generate-large-prime-numbers-for-rsa-algorithm/ [full code at bottom of page]. In the nBitRand function we essentially shuffle a vector of BigInt’s determined by the range: (2.pow(n – 1) + …
