Preface

os.system() is actually a call to C’s system().

Here is what the documentation states:

The system() function shall behave as if a child process were created
using fork(), and the child process invoked the sh utility using
execl() as follows:

execl(, "sh", "-c", command, (char *)0);

where is an unspecified pathname for the sh utility. It
is unspecified whether the handlers registered with pthread_atfork()
are called as part of the creation of the child process.

The system() function shall ignore the SIGINT and SIGQUIT signals, and
shall block the SIGCHLD signal, while waiting for the command to
terminate. If this might cause the application to miss a signal that
would have killed it, then the application should examine the return
value from system() and take whatever action is appropriate to the
application if the command terminated due to receipt of a signal.

The system() function shall not affect the termination status of any
child of the calling processes other than the process or processes it
itself creates.

The system() function shall not return until the child process has
terminated. [Option End]

The system() function need not be thread-safe.

Solution

system() creates a child process and exits, there is no stack to be resolved, therefore one would expect this to run as long as resources to do so are available. Furthermore, the operation being of creating a child process is not an intensive one— the processes aren’t using up much resources, but if allowed to run long enough the script will to start to affect general performance and eventually run out of memory to spawn a new child process. Once this occurs the processes will exit.

Example

To demonstrate this, set recursion depth limit to 10 and allow the program to run:

import os, sys, inspect

sys.setrecursionlimit(10)

args = sys.argv[1:]
arg = int(args[0]) if len(args) else 0

stack_depth = len(inspect.stack(0))

print(f"Iteration {arg} - at stack depth of {stack_depth}")

arg += 1

os.system(f"python3 main.py {arg}")

Outputs:

Iteration 0 - at stack depth of 1 - avaialable memory 43337904128 
Iteration 1 - at stack depth of 1 - avaialable memory 43370692608 
Iteration 2 - at stack depth of 1 - avaialable memory 43358756864 
Iteration 3 - at stack depth of 1 - avaialable memory 43339202560 
Iteration 4 - at stack depth of 1 - avaialable memory 43354894336 
Iteration 5 - at stack depth of 1 - avaialable memory 43314974720 
Iteration 6 - at stack depth of 1 - avaialable memory 43232366592 
Iteration 7 - at stack depth of 1 - avaialable memory 43188719616 
Iteration 8 - at stack depth of 1 - avaialable memory 43173384192 
Iteration 9 - at stack depth of 1 - avaialable memory 43286093824 
Iteration 10 - at stack depth of 1 - avaialable memory 43288162304
Iteration 11 - at stack depth of 1 - avaialable memory 43310637056
Iteration 12 - at stack depth of 1 - avaialable memory 43302408192
Iteration 13 - at stack depth of 1 - avaialable memory 43295440896
Iteration 14 - at stack depth of 1 - avaialable memory 43303870464
Iteration 15 - at stack depth of 1 - avaialable memory 43303870464
Iteration 16 - at stack depth of 1 - avaialable memory 43296256000
Iteration 17 - at stack depth of 1 - avaialable memory 43286032384
Iteration 18 - at stack depth of 1 - avaialable memory 43246657536
Iteration 19 - at stack depth of 1 - avaialable memory 43213336576
Iteration 20 - at stack depth of 1 - avaialable memory 43190259712
Iteration 21 - at stack depth of 1 - avaialable memory 43133902848
Iteration 22 - at stack depth of 1 - avaialable memory 43027984384
Iteration 23 - at stack depth of 1 - avaialable memory 43006255104
...

https://replit.com/@pygeek1/os-system-recursion#main.py

References

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html