How to use Faktory with a ruby job and a python worker?

Question

I want to use a ruby script to submit jobs to a faktory server (https://github.com/contribsys/faktory/wiki) to launch a python worker.
Faktory should be perfect for that being language agnostic but the documentation covers only the ruby only (job and worker) or python only (job and worker) cases. Quickly, I have two questions:

How to register the worker on the python side so that it is addressable from the ruby side?
How to code correctly the ruby job (in particular, is the perform method required)?

Here is a minimal (not) working example.
First, the python worker file:

from faktory import Worker 
import logging
logging.basicConfig(level=logging.INFO)

def python_worker(var_int):
    print(var_int)

w = Worker(queues=['default'], concurrency=1)
w.register('PythonWorker', python_worker)
w.run()

Now the ruby file:

class RubyJob < ApplicationJob
    include Faktory::Job
    faktory_options retry: 5
    queue_as :default

  end

To call the job I use RubyJob.perform_later(1).

Coming back to the two questions:

Does the ruby job need a ‘perform()’ method?
How do I tell the ruby job to require from faktory the ‘PythonWorker’?

For the moment I need register the python worker as ‘ActiveJob::QueueAdapters::FaktoryAdapter::JobWrapper’.

Maybe useful would be a minimal example of a ruby job and a python worker similar to the examples for ruby (https://github.com/contribsys/faktory/wiki/Getting-Started-Ruby) or python (https://www.mikeperham.com/2019/01/08/using-faktory-with-python/) only.

Asked By: timmey

||

Source

Answer 1

Your Ruby code doesn’t need any jobs at all. The job is Python. The Ruby code just needs to create and send the job record to Faktory so the python worker can pick it up. In Ruby, you create a basic job using the plain old Faktory::Client API:

c = Faktory::Client.new
c.push('jobtype' => 'PythonWorker', 'args' => [1], 'queue' => 'default', 'jid' => SecureRandom.hex(12))

Start Faktory, start a faktory_worker_python process and run that code in your Ruby app.

Answered By: Mike Perham

How to use Faktory with a ruby job and a python worker?

Question:

Answers: