3.9 Executing Local Programs

Running local programs is one of the main reasons for scheduling joblets on resources. Although you are not allowed to run local programs on the server side job portion of JDL, there are two ways to run local programs in a joblet:

  1. Use the built-in system() function.

    This function is used for simple executions requiring no output or process handling. It simply runs the supplied string as a shell command on the resource and writes stdout and stderr to the job log.

  2. Use the Exec JDL class.

    The Exec class provides flexibility in how to invoke executables, to process the output, and to manage the process once running. There is provision for controlling stdin, stdout, and stderr values. stdout and stderr can be redirected to a file, to the job log, or to a stream object.

    Exec provides control of how the local program is run. You can choose to run as the agent user or the job user. The default is to run as the job user, but fallback to agent user if the job user does not exist on the resource.

    For more information, see Exec.

3.9.1 Output Handling

The Exec function provides controls for specifying how to handle stdout out stderr. By default, Exec discards the output.

The following example runs a program that directs stdout and stderr to the job log:

    e = Exec()
    e.setShellCommand(cmd)
    e.writeStdoutToLog()
    e.writeStderrToLog()
    e.execute()

The following example runs a program that directs stdout and stderr to files and opens the stdout file if there is no error in execution:

    e = Exec()
    e.setCommand("ps -aef")
    e.setStdoutFile("/tmp/ps.out")
    e.setStderrFile("/tmp/ps.err")
    result = e.execute()
    if result == 0:
        output = open("/tmp/ps.out").read()
        print output

3.9.2 Local Users

You can choose to run local programs and have file operations done as the agent user or the job user. The default is to run as the job user, but fallback to agent user if the job user does not exist on the resource. These controls are specified on the job. The job.joblet.runtype fact specifies how file and executable operations run in the joblet in behalf of the job user, or not.

The choices for job.joblet.runtype are defined in the following table:

Table 3-1 Job Run Type Values

Option

Description

RunAsJobUserFallingBackToNodeUser

Default. This means if the job user exists as a user on the resource, then executable and file operations is done on behalf of that user. By falling back, this means that if the job user does not exist, the agent will still execute the joblet executable and file operation as the agent user. If the executable or file operation still has a permission failure, then the agent user is not allowed to run the local program or do the file operation.

RunOnlyAsJobUser

This means resource can only run the executable or file operation as the job user and will fail immediately if the job user does not exist on the resource. You want to use this mode of operation if you wish to strictly enforce execution and file ownership. You must have your resource setup with NIS or other naming scheme so that your users will exist on the resource.

RunOnlyAsNodeUser

This means the resource will only run executables and do file operations as the agent user.

There is also a fact on the resource grid object that can override the job.joblet.runtype fact. The fact resource.agent.config.exec.asagentuseronly on the resource grid object can overwrite the job.joblet.runtype fact.

This ability to run as the job user is supported by the enhanced exec feature of the Orchestrate Agent. A resource might not support PlateSpin Orchestrate enhanced execution of running as job users. If the capability is not supported, the fact resource.agent.config.exec.enhancedused is False. This fact is provided so you can create a resource or allocation constraint to exclude such a resource if your grid mixes resource with/without the enhanced exec support and your job requires enhanced exec capabilities.

3.9.3 Safety and Failure Handling

An exception in JDL will fail the job. By default, an exception in the joblet will fail the joblet. The job.joblet.* facts provide controls on how many times a failure will fail the joblet. For more information, see Section 3.11, Improving Job and Joblet Robustness.

    try :
        < JDL >
    except:
        exc_type, exc_value, exc_traceback = sys.exc_info()
        print "Exception:", exc_type, exc_value

JDL also provides the fail() function on the Job and Joblet class for failing a job and joblet. The fail() function takes an optional reason message.

You would use fail() when you detect an error condition and wish to end the job or joblet immediately. Usage of the joblet fail() fails the currently running instance of the joblet. The actual failed state of the joblet occurs when the maximum number of retries has been reached.