How to integrate a standalone Python script into a Rails application?
Question:
I’ve got a program that has a small file structure going on and is then ran using
python do_work.py foo bar
I want my Rails users to press a button and have this happen for them, with the result either uploaded somewhere or just thrown to them as a download link or something of the sort – the output of do_work.py (say, it’s result.txt)
I also want to clarify that the script results in the creation on the filesystem of 3 separate files, which are not text files (which shouldn’t matter and isn’t really the problem here)
What is the best way to go about it? Can rake run exec Python? More importantly, is this doable on heroku?
I have Python installed on my system but the provided answer by sockmonk doesn’t seem to work – it returns nil. Mind you, other commands like ls seem to work.
Could it be a permissions problem?
def index
value = %x( python --version )
render :text => value
end
Incidentally, trying this in irb:
%x(python)
Brings up the Python terminal INSIDE of irb. It will not take params for whatever reason however.
Answers:
It partly depends on the format of the data. If it’s not too long and can be rendered directly in the browser, you can just do something like this in a rails controller:
result = `python do_work.py foo bar`
render :text => result
And assuming that result is plain ASCII text, the result will go straight to their browser. If the params to do_work.py come from the user you MUST validate them first though, so you don’t wind up creating a nasty vulnerability for yourself. Using the system() call would probably be safer in that case.
If you want to send the results back as a file, look at ruby’s Tempfile class for creating the file (in a way that won’t stick around forever), and rails’ send_file and send_data commands for some different options to send back the results that way.
Your index method does not work because python --version outputs its version to STDERR, not STDOUT. If you don’t need to separate these streams, you may just redirect STDERR to STDOUT:
value = %x(python --version 2>&1)
This call is synchronous, so after running the script (python do_work.py foo bar 2>&1), you should be able to read the files produced by it.
If the script is not able to create the files for some reason, you will now see the exception in the value variable because error messages are usually sent to STDERR.
If you want to separate STDERR from STDOUT, use the Open3 module.
Beware that the script takes some time to run, so the calls may overlap. I would use a queue here to prevent this.
And don’t forget to check the data the user enters. Never pass it directly to the script.
The answer from utapyngo is cover almost all you need to know. I’ll answer this part:
incidentally, trying this in irb:
%x(python)
Brings up the python terminal INSIDE of irb. It will not take params for whatever reason however.
To pass parameters to your python script, simply pass it. Example:
[fotanus@thing ~]$ python a.py
args:
['a.py']
[fotanus@thing ~]$ irb
1.8.7 :001 > %x(python a.py foo bar)
=> "args:n['a.py', 'foo', 'bar']n"
This works on ruby 1.8, 1.9 and 2.0.
It depends how deeply you want to integrate the python script. There are ways to actually call python modules directly from ruby.
http://www.goto.info.waseda.ac.jp/~fukusima/ruby/python-e.html
This would give you the benefit of getting the output directly from your python script instead of going over the I/O device.
I would do something like the following.
Asynchronously execute this task in the background. And once the result is ready report it to the user.
One way you can achieve this will by using Open3 and delayed_job gem.
Take a look the popen3 method in Open3 module.
Open3.popen3([env,] cmd... [, opts]) {|stdin, stdout, stderr, wait_thr|
pid = wait_thr.pid # pid of the started process.
...
exit_status = wait_thr.value # Process::Status object returned.
}
In your case you can change the Open3.popen3 statement to something like the following
Open3.popen3("python do_work.py foo bar"){
...
# mechanism for reporting like setting a flag in database system
# or queue system
}
Note: you should give the full path to your python script
And then use delayed_job gem to run this as a background task.
You should also have a polling mechanism which will poll the system to see if the flag is set which would mean the result is ready and then serve it to the user.
I’ve got a program that has a small file structure going on and is then ran using
python do_work.py foo bar
I want my Rails users to press a button and have this happen for them, with the result either uploaded somewhere or just thrown to them as a download link or something of the sort – the output of do_work.py (say, it’s result.txt)
I also want to clarify that the script results in the creation on the filesystem of 3 separate files, which are not text files (which shouldn’t matter and isn’t really the problem here)
What is the best way to go about it? Can rake run exec Python? More importantly, is this doable on heroku?
I have Python installed on my system but the provided answer by sockmonk doesn’t seem to work – it returns nil. Mind you, other commands like ls seem to work.
Could it be a permissions problem?
def index
value = %x( python --version )
render :text => value
end
Incidentally, trying this in irb:
%x(python)
Brings up the Python terminal INSIDE of irb. It will not take params for whatever reason however.
It partly depends on the format of the data. If it’s not too long and can be rendered directly in the browser, you can just do something like this in a rails controller:
result = `python do_work.py foo bar`
render :text => result
And assuming that result is plain ASCII text, the result will go straight to their browser. If the params to do_work.py come from the user you MUST validate them first though, so you don’t wind up creating a nasty vulnerability for yourself. Using the system() call would probably be safer in that case.
If you want to send the results back as a file, look at ruby’s Tempfile class for creating the file (in a way that won’t stick around forever), and rails’ send_file and send_data commands for some different options to send back the results that way.
Your index method does not work because python --version outputs its version to STDERR, not STDOUT. If you don’t need to separate these streams, you may just redirect STDERR to STDOUT:
value = %x(python --version 2>&1)
This call is synchronous, so after running the script (python do_work.py foo bar 2>&1), you should be able to read the files produced by it.
If the script is not able to create the files for some reason, you will now see the exception in the value variable because error messages are usually sent to STDERR.
If you want to separate STDERR from STDOUT, use the Open3 module.
Beware that the script takes some time to run, so the calls may overlap. I would use a queue here to prevent this.
And don’t forget to check the data the user enters. Never pass it directly to the script.
The answer from utapyngo is cover almost all you need to know. I’ll answer this part:
incidentally, trying this in irb:
%x(python)
Brings up the python terminal INSIDE of irb. It will not take params for whatever reason however.
To pass parameters to your python script, simply pass it. Example:
[fotanus@thing ~]$ python a.py
args:
['a.py']
[fotanus@thing ~]$ irb
1.8.7 :001 > %x(python a.py foo bar)
=> "args:n['a.py', 'foo', 'bar']n"
This works on ruby 1.8, 1.9 and 2.0.
It depends how deeply you want to integrate the python script. There are ways to actually call python modules directly from ruby.
http://www.goto.info.waseda.ac.jp/~fukusima/ruby/python-e.html
This would give you the benefit of getting the output directly from your python script instead of going over the I/O device.
I would do something like the following.
Asynchronously execute this task in the background. And once the result is ready report it to the user.
One way you can achieve this will by using Open3 and delayed_job gem.
Take a look the popen3 method in Open3 module.
Open3.popen3([env,] cmd... [, opts]) {|stdin, stdout, stderr, wait_thr|
pid = wait_thr.pid # pid of the started process.
...
exit_status = wait_thr.value # Process::Status object returned.
}
In your case you can change the Open3.popen3 statement to something like the following
Open3.popen3("python do_work.py foo bar"){
...
# mechanism for reporting like setting a flag in database system
# or queue system
}
Note: you should give the full path to your python script
And then use delayed_job gem to run this as a background task.
You should also have a polling mechanism which will poll the system to see if the flag is set which would mean the result is ready and then serve it to the user.