JS7 JobScheduler

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

FEATURE AVAILABILITY STARTING FROM RELEASE 2.8.2

Introduction

JS7 - GraalVM Python Jobs are based on Oracle® GraalVM.

This article explains how a GraalVM Python Job can execute an external application or script in a subprocess and forward its stdout and stderr output to the JS7 logger in real time.
This enables JS7 workflows to integrate existing scripts, tools or services while maintaining full control over log streaming and cancellation behavior.

Example

(info) For further details on the JS7 Job API, see JS7 - Job API

The following example demonstrates how to process stdout and stderr output from a subprocess in parallel using the Python threading module.
Additionally, the onProcessOrderCanceled() hook is used to terminate the subprocess if the order is canceled.

Example: Cancellable Subprocess with Live Output Logging
import subprocess
import threading

def stream_output(stream, logger, prefix):
    for line in iter(stream.readline, ""):
        logger(f"{prefix}{line.rstrip()}")
    stream.close()

class JS7Job(js7.Job):
    def processOrder(self, js7Step):
        logger = js7Step.getLogger()
        
        # e. g. '/usr/bin/python3', '/bin/bash', ...
        runtime_path = js7Step.getAllArgumentsAsNameValueMap().get("runtime_path")
        
        # Path to the application or script
        app_path = js7Step.getAllArgumentsAsNameValueMap().get("app_path")
        
        # Arguments that will be passed to the application or script
        args = ["js7Step_argument_1", "js7Step_argument_2"]

        logger.info(f"[subprocess]start...")
        
        cmd = [runtime_path]
        
        # Unbuffered mode only for Python
        if runtime_path.endswith("python") or runtime_path.endswith("python3"):
            cmd.append("-u")
        
        cmd.append(app_path)
        cmd += args
        
        # Starts the application or script as a subprocess
        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True,
            bufsize=1
        )
        
        js7Step.setCancelableResource(process)
        
        # Defines the thread for live stdout
        stdout_thread = threading.Thread(
            target=stream_output,
            args=(process.stdout, logger.info, "[LIVE STDOUT] "),
            daemon=True
        )
        
        # Defines the thread for live stderr
        stderr_thread = threading.Thread(
            target=stream_output,
            args=(process.stderr, logger.error, "[LIVE STDERR] "),
            daemon=True
        )
        
        # Starts the threads
        stdout_thread.start()
        stderr_thread.start()

        returncode = process.wait()
        
        # Waits for the threads to finish
        stdout_thread.join()
        stderr_thread.join()

        logger.info(f"[subprocess]returncode={returncode}")
        
        js7Step.getOutcome().setReturnCode(returncode)
        js7Step.getOutcome().putVariable("outcome_var_name", "outcome_var_value")

    # Uses the onProcessOrderCanceled hook after the order has been canceled
    def onProcessOrderCanceled(self, js7Step):
        process = js7Step.getCancelableResource()

        js7Step.getLogger().info(f"onProcessOrderCanceled={process}")
        
        # Kills the process if it exists
        if process is not None:
            process.kill()

Note: Subprocesses run independently of the GraalVM Python environment. Long-running scripts or tools can stream output to the JS7 logger continuously without causing the job to hang or become unresponsive.

Resources

Downloads


  • No labels