How to execute mpirun or mpiexec command from a python script?
Question:
I have an MPI application written in C++. When I run the application using mpirun or mpiexec on my local machine shell, it works fine. The problem is that the mpi application should be executed after a request from the network (e.g. HTTP request) is received. Assuming that the application server is written in python, how can I execute the mpi application using mpiexec command from a python script? I used subprocesses.run() command but nothing happens.
My general question: What is the best way to run an MPI application in client/server architecture?
Thank you
Answers:
What options did you use with subprocess.run()? If you are trying to execute a shell command then you need to use shell=True
subprocess.run('ls', shell=True)
You would replace ‘ls’ with your mpiexec or mpirun command.
I found the solution. In the main python script, MPI should not get imported from mpi4py package. Otherwise subprocess.run("mpiexec") does not do anything.
The method that finally worked for me is:
import subprocess
Define the command like this (separate the keywords):
command = ['mpiexec', '-n', '6', 'fooprogramme']
Run the command using subprocess.run and capture the output to result:
result = subprocess.run(command, stdout=subprocess.PIPE)
Then you can print the output to the console, using:
print(result.stdout.decode('utf-8'))
Or you can also store result.stdout.decode('utf-8'), or write it into a file
I have an MPI application written in C++. When I run the application using mpirun or mpiexec on my local machine shell, it works fine. The problem is that the mpi application should be executed after a request from the network (e.g. HTTP request) is received. Assuming that the application server is written in python, how can I execute the mpi application using mpiexec command from a python script? I used subprocesses.run() command but nothing happens.
My general question: What is the best way to run an MPI application in client/server architecture?
Thank you
What options did you use with subprocess.run()? If you are trying to execute a shell command then you need to use shell=True
subprocess.run('ls', shell=True)
You would replace ‘ls’ with your mpiexec or mpirun command.
I found the solution. In the main python script, MPI should not get imported from mpi4py package. Otherwise subprocess.run("mpiexec") does not do anything.
The method that finally worked for me is:
import subprocess
Define the command like this (separate the keywords):
command = ['mpiexec', '-n', '6', 'fooprogramme']
Run the command using subprocess.run and capture the output to result:
result = subprocess.run(command, stdout=subprocess.PIPE)
Then you can print the output to the console, using:
print(result.stdout.decode('utf-8'))
Or you can also store result.stdout.decode('utf-8'), or write it into a file