Run Python script through Java with importing just once

Question:

Update April 2020: I have accepted the answer below because it proposes a very good and simple solution to my problem but I never got my code to work! If anyone has already built something similar please contact me!


I have a simple script my-script.py that does some calculations very fast using a specific library written in Python:

import sys
import special-library

def fun(x):
  do something

fun(sys.argv[1])

In my Java code, I use/call this script a lot in different part of the code, using the ProcessBuilder approach, which means that I simply run the command python my-script.py argument.
This means that every time I call this script, I have to execute the import commands which is the most time-consuming thing in this (the calculation part is actually faster).

Is there a solution so that I call the import just once? I looked a little bit about Jython – would be possible to write a class or something that would be initiated and do the import once and then calling a function of it every time I want to do the calculations part (fun)? Has anyone done something similar or have any other solution to this?

First Attempt at Implementation

I’ve tried to write a ‘pipe’ Java Class that will execute the python script once. Now the script reads continuously from the stdin and when it gets an input argument, it does the calculations and returns the result:

import sys
import special-library

def fun(x):
  do something
  print('END')

for arg in sys.stdin:
  fun(arg)

And this works of course when I test it from the command line and supplying it with input arguments.

The Java Class is as follows:

package my.package;
import java.io.*;

public class PythonScriptExecuter {

    private static BufferedReader pythonToJavaReader;
    private static PrintWriter javaToPythonWriter;

    public PythonScriptExecuter() throws Exception {
        String MPBNScriptFile = "fullPathToScriptFile";
        ProcessBuilder pb = new ProcessBuilder("python", MPBNScriptFile);
        pb.redirectErrorStream(true);

        // Start script
        Process p = pb.start();

        pythonToJavaReader = getOutputReader(p); // from python to java
        javaToPythonWriter = getInputWriter(p); // from java to python
    }

     // Python => Java
    private static BufferedReader getOutputReader(Process p) {
        return new BufferedReader(new InputStreamReader(p.getInputStream()));
    }

    // Java => Python
    private static PrintWriter getInputWriter(Process p) {
        return new PrintWriter(new OutputStreamWriter(p.getOutputStream()));
    }

    public static Arraylist<String> getResults(String argument) throws IOException {
        // send argument to python script
        javaToPythonWriter.println(argument);
        //javaToPythonWriter.flush();

        // get results back one by one
        ArrayList<String> results = new ArrayList<>();

        int count = 0;
        String line;
        while (!(line = pythonToJavaReader.readLine()).equals("END")) {
            // process `line` string 
            results.add(line);
        }

        return results;
    }
}

So, what I do is that somewhere in the initialization part of my code I have this simple line to start the process:

new MPBNPythonScriptExecuter();

and later when I want to get some results back, I use:

String arg = "something"
Arraylist<String> res = PythonScriptExecuter.getResults(arg);

and the whole thing hangs on the reading-the-output-from-the-script line:

while (!(line = pythonToJavaReader.readLine()).equals("END"))

Any ideas what is going wrong here?

Asked By: John

||

Answers:

You could communicate between java and Python with pipes.

You run your Python as you do now (without command line args)

Python script

you write an infinite loop in python that will

  1. read data from the
    standard input (it will be your arg for the function)

  2. You call your function

  3. you write the answer to the standard output

Java

  1. Write a method for sending args to python

  2. write to the pipe the arg of your function

  3. read the answer from the pipe

You’re done.

Here is some snippet

Here how you create your tube

        Process p = Runtime.getRuntime().exec(commande);
        BufferedReader output = getOutput(p); //from python to java
        BufferedReader error = getError(p); //from python to java
        PrintWriter input  = getInput(p); //from java to python

private static BufferedReader getOutput(Process p) {
    return new BufferedReader(new InputStreamReader(p.getInputStream()));
}
private static BufferedReader getError(Process p) {
    return new BufferedReader(new InputStreamReader(p.getErrorStream()));
}    
private static PrintWriter getInput(Process p){
    return new PrintWriter (new OutputStreamWriter(p.getOutputStream()));
}
Answered By: Pascal Fares

Try the following in your Python script:

import sys

def fun(x):
  print(x)
  print('END')
  sys.stdout.flush()

while 1:
    try:
        line = sys.stdin.readline()
    except KeyboardInterrupt:
        break

    if not line:
        break

    fun(line)

Your Java code should be more or less correct.

Answered By: Stathis
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.