Program

The Program block is intended to run a program with command line interface (e.g., solvers, CAE programs etc.). It integrates an executable or a shell script into the workflow, thus establishing a link to exchange data and files. Commands and scripts can be run on a local machine, remote host (via SSH for Linux or pSeven Agent for Windows or Linux, see pSeven Agent) or HPC cluster.

The Program block is run in one of two modes: command (simple) and script (advanced). In the command mode, you can select a program to run, specify its arguments and add input and output files without actually writing a script. In the script mode, you can type in the script right in the block or import it from a file. The Program block runs the script with bash in Linux and cmd in Windows.

There are two specific types of variables in the block: substitutions and files. Substitutions are string variables intended to insert values captured by the respective ports to the script. It means that for each substitution a port with the same name is created and the block will take the value received by the port and inject it directly into the script template, path to the executable and arguments. The places in the script to insert the value are marked as @{variable name}.

Adding file variables to the Program block results in creating the respective input or output port of the File type. Before executing the script, the block copies input files to the working directory (Sandbox) under the same name that was specified when adding the file. If the file name was not specified, then it will be generated at random, and the Program block will substitute this generated name to the command line, script template and arguments. Output files are read from the working directory and removed after the script’s execution.

Execution types

As mentioned above, the Program block launches a program or a script on a local machine, remote host or HPC cluster. The details on the required configuration and settings for each use case are given below.

Local execution

This use case implies running a program or a script on a local machine. The type 1 of executable to be launched is selected on the Script tab of the Configuration dialog 2.

../_images/page_blocks_Program_type_of_executable.png

In the command mode, specify a program by clicking b_browse. A Select file… dialog will appear.

../_images/page_blocks_Program_dialog.png

Find the necessary file in the list and click the Select button. The program’s name will be displayed in the respective field. You can also manually type in the full path to the file.

../_images/page_blocks_Program_file_name.png

In the script mode, you can write your own script using the text editor or import a ready one from a file.

../_images/page_blocks_Program_user_script.png

In the script, you can use string substitutions to replace the values of certain parameters captured by the ports or specified by the user (in the Parameters pane in Run). The substitution syntax is as follows: @{x}. As an example, there is a default substitution named “timestamp” that stands for the current date and time. In the script template, this substitution will be replaced with the actual date and time. There are some other default substitutions, for example, “project” stands for project directory, “sandbox” stands for sandbox directory etc.

../_images/page_blocks_Program_substitutions.png

String substitutions support using placeholders, for instance:

  • %d expands to the current date.
  • %t expands to current time (time when the block’s step begins).
  • %j expands to the project directory.
  • %s expands to the block’s sandbox directory.

To add a new substitution, click b_blconf_add button 1 or select Add substitutions… command 2 from the b_blconf_context menu.

../_images/page_blocks_Program_substitutions_add.png

Type substitution’s name and value and click Add.

../_images/page_blocks_Program_substitutions_add_value.png

Adding a new substitution to the block results in creating a port with the same name.

../_images/page_blocks_Program_substitutions_add_port.png

The block copies content from standard output and error streams and sends it to the corresponding port(s) (by default) or to a file(s) (if output redirection is set up).

Input and output files can be specified in the Files pane. To add a file, click b_blconf_add or select Add files… command from the b_blconf_context menu.

For the input file:

  • Specify the port name: infile.
  • Select direction: input.
  • Specify the file name: input.dat.
../_images/page_blocks_Program_infile.png

For the output file:

  • Specify the port name: outfile.
  • Select direction: output.
  • Specify the file name: output.dat.
../_images/page_blocks_Program_outfile.png

Verify file settings in the Files pane.

../_images/page_blocks_Program_files.png

Note

File names are optional. You can leave a file name empty and use port names in commands instead. The syntax is @{port_name} — for example, @{infile}; when the Program block executes the command, it substitutes actual file names where port names are specified.

See Integration Basics for more details about adding files.

In the Configuration dialog, you can also specify command line arguments 1 and redirect standard input (stdin), output (stdout) and error (stderr) streams 2.

../_images/page_blocks_Program_configuration.png

More advanced settings related to the program/script execution can be specified on the General settings tab. General 1 option details are given below.

../_images/page_blocks_Program_generalset.png

Script execution timeout

The maximum allowed time for the script to execute. The default value is 0.0 which means there is no timeout.

Fail, if timeout exceeded

The execution will be considered failed, if the timeout has occurred (enabled by default). Note that if the block encounters an error during execution, the standard Error handling mechanism will be activated. See Error Handling section for details.

Fail, if exit code is not

The execution will be considered failed if the exit code differs from the specified value. The value is user-defined; if there is more than one success code, the values can be separated by a comma or a dash, e.g. 8-20 (a dash specifies a range meaning that figures from 8 to 20, inclusive, shall be treated as success codes), 154.

Show stdout in the console

Enables logging standard output of the program or script during execution.

Fail on stdout

Defines the failing policy for the output in stdout. Possible options: never, always, if contains pattern (execution is failed, if the standard output contains the pattern, e.g. “error”, “fatal error” etc.), if matches pattern (execution is failed, if the standard output matches the pattern).

Fail pattern for stdout

The pattern (a string or a regular expression) which occurrence in stdout results in the block’s failure. Note that if “Fail on stdout” option’s value is “if contains pattern”, the pattern will be considered as string, if the option’s value is “if matches pattern”, the pattern will be considered as regular expression.

Show stderr in the console

Enables logging standard error output of the program or script during execution.

Fail on stderr

Defines the failing policy for the output in stderr. Possible options: never, always, if contains pattern (execution is failed, if the error output contains the pattern, e.g. “error”, “fatal error” etc.), if matches pattern (execution is failed, if the error output matches the pattern).

Fail pattern for stderr

The pattern (a string or a regular expression) which occurrence in stderr results in the block’s failure. Note that if “Fail on stderr” option’s value is “if contains pattern”, the pattern will be considered as string, if the option’s value is “if matches pattern”, the pattern will be considered as regular expression.

Keep temporary files

By default, generated input and output files are deleted when the block finishes execution. Selecting this option means that the respective files will not be deleted. This may be helpful for debugging purposes.

Retry count

The parameter sets the number of attempts to perform the remote operation (SSH or Agent). The default value is 0 which means there is no limit.

Retry interval

Specifies the time in seconds between each remote connection attempt.

Last two options refer to Remote Execution.

After you’ve finished configuring the block, verify the command preview (in the command mode). Click OK and close the Configuration dialog.

Remote Execution

This use case implies running a program or a script on a remote host via SSH (for Linux) or pSeven Agent (for Windows/ Linux). To configure the block for remote execution, see Local execution section for details. The information about the settings described there is also applicable to this use case.

SSH Connection

To enable SSH, the following steps are required:

  1. Switch to the Settings tab.

  2. Select SSH from the list of available options on the Execution type panel. This step is optional, as any parameter change will automatically switch the block to this mode.

  3. Specify the parameters necessary to establish a secure connection:

    • Host: remote hostname or IP address

    • Port: port number; the default port is 22

    • Username: the username to be used for the connection

    • Password: the password required to connect to a remote host

      Warning

      Note that although password characters in the block’s configuration dialog are hidden for security reasons, the password is stored in the block in plain text without any encryption and can be easily retrieved, especially if the workflow is shared with other users. To prevent this, a private key is better used as the user authentication method.

    • Private key path: the path to the RSA private key file (for example: $HOME/.ssh/id_rsa). If the private key file requires a password, the one specified in the ‘password’ field will be used.

    • Local sandbox: the local working directory for the block, see Sandbox section for details.

    • Sandbox prototype: the prototype directory which contents are copied to the local sandbox when it is initialized, see Sandbox section for details.

    • Remote sandbox: a directory on the remote machine where a program or a script is executed.

    • Synchronization: specifies the type of file synchronization between the block’s sandbox and the remote host. The files to be synchronized are to be placed in the block’s sandbox (see section Sandbox for details). Possible option values are:

      • None: no synchronization (default).
      • Two-way (“Local to remote” and “Remote to local”): a synchronized copy of the block’s sandbox will be created on the remote host. After executing the program, contents of the remote sandbox will be synced back to the local sandbox.
      • Local to remote: the local sandbox will be uploaded to the remote host before starting the program, but the remote copy will not be synced back.
      • Remote to local: local sandbox files will not be uploaded, the files located in the remote sandbox will be copied (downloaded) to the local sandbox.
    • Synchronize while running: if enabled, synchronization is performed on each step; if disabled, files will be uploaded only at workflow start and finish.

    • Remote locale: override the remote system’s locale when executing the script. Sets the LC_ALL, LANG and LANGUAGE variables in the script environment to the specified value.

  4. After specifying the parameters, always test the configuration using the Test SSH connection button. Only if this test passes, pSeven will be able to connect to the remote host when you run the workflow. The test is successful when you see the following message:

    ../_images/page_blocks_Program_remote_execution_ssh_test_passed.png

    When the test fails, pSeven shows an error message. To see additional error information, click the Details button:

    ../_images/page_blocks_Program_remote_execution_ssh_test_fail_initoutput.png

    Error details allow to find the specific cause of an error, which can be authentication failure (invalid username or password), network connection failure, or other. In particular, one possible cause of an SSH connection error is a remote host’s shell initialization script that produces output for non-interactive sessions. For details on this error, see SSH Connection Error on Unexpected Output.

    Note that the SSH connection test establishes only one connection (opens a single SSH session). When you run the workflow, it can open multiple SSH sessions if the Program block is nested into a Composite block with enabled parallelization: the number of active SSH sessions in this case is equal to the Composite block’s parallelization ratio. The remote host then has to be properly configured to accept multiple SSH connections, otherwise errors can occur (see Remote Execution and Parallelization for details).

pSeven Agent

The Program block supports remote execution on a host that runs the pSeven Agent — a cross-platform utility that is automatically installed with pSeven. A Program block in a workflow that runs on the local host can connect to the pSeven Agent running on a remote host. The pSeven Agent receives input data and commands from the block and executes a program or a script specified in the block’s configuration, then sends output data back to the block. pSeven Agent has to be installed on the remote host as standalone application or as a part of pSeven installation.

Agent Configuration

pSeven Agent configuration can be opened using a shortcut in the system menu: Start ‣ All Programs ‣ pSeven ‣ pSeven Agent.

../_images/page_blocks_Program_agent_gui.png

The configuration window contains three tabs.

  1. The Status tab shows an event log and agent status. Here you can configure and start the agent (by default it does not start automatically).
  2. The Tasks tab shows tasks received by this instance of the pSeven Agent.
  3. The About tab contains brief information and links to help and log files.

To configure the agent, click the Configure button on the Status tab.

../_images/page_blocks_Program_agent_config.png

pSeven Agent supports the passive mode only (selected by default). In this mode it listens on the specified port; this is the port that should be specified in the Program block configuration. You can select any available port, default is 8899.

Client tokens are identifiers sent by Program when it connects to the agent. The token sent by the block must be found in this list, otherwise the connection is not allowed. Note that you can specify a list of tokens in the agent’s configuration, in order to distinguish between the clients.

You can also configure the agent to start automatically using the “Autostart agent” option. When it is enabled, the agent will start immediately once you launch it with the menu shortcut. Other settings are not used.

If “Autostart agent” is not enabled, you will have to click Start button on the Status tab. After this it will show the host (server) address and port (endpoint) and will accept incoming connections.

Block Configuration

The connection to pSeven Agent is configured on the Settings tab in the block configuration dialog.

../_images/page_blocks_Program_agent_connect.png
  • Endpoint: the address and port of the host that runs pSeven Agent.
  • Authentication token: the token to send when connecting to the Agent. Note that this token has to be included into the list of client tokens in the agent configuration, otherwise the connection will not be allowed.
  • Synchronization: specifies the type of file synchronization between the block’s sandbox and the host running pSeven Agent. The files to be synchronized have to be placed in the block’s sandbox (see section Sandbox for details).
    • None: no synchronization.
    • Two-way: a synchronized copy of the block’s sandbox will be created on the remote host. After executing the program, contents of the remote sandbox will be synced back to the local sandbox.
    • Local to remote: the local sandbox will be uploaded to the remote host before starting the program, but the remote copy will not be synced back.
    • Remote to local: local sandbox files will not be uploaded, but files created on the remote host will be downloaded to the local host.

After specifying the endpoint and authentication token you can test the configuration using the connection test button.

../_images/page_blocks_Program_agent_connect_test.png

If authentication is successful, pSeven will show a message box indicating that the connection test passed.

Running Cluster Jobs

The Program block can be used to run calculations on an HPC cluster. Supported batch managers are: TORQUE, SLURM, and LSF.

The block generates a submit script consisting of two parts:

  • command to the appropriate batch manager (the parameters to be passed are configured on the Cluster settings tab);
  • user script (specified on the Script tab);

and submits the generated script to the selected batch manager.

The parameters to establish a connection to the host where the batch manager runs can be configured on the Settings tab (Execution type group). If the batch manager is installed on the remote host, specify the Execution type as SSH (see SSH Connection section for configuration details). If the batch manager is installed on a local host, specify the Execution type as local.

Job parameters common for all supported batch managers are represented as block’s options on the Cluster settings tab, other parameters can be specified as custom options.

../_images/page_blocks_Program_cluster_settings.png

A detailed description of configuration options is given below.

Node count

Defines the number of compute nodes to allocate for the job.

CPUs per node

Specifies the number of CPUs per node required for the job.

Node constraints

Specifies features of a node required by the job.

(Job) name

Job is a resource request submitted to the HPC cluster that contains one or more tasks. The option specifies a name (ID property) of a job to make it unique.

Job array

Sets IDs of near identical jobs (that perform the same work using the same script, but operate on different sets of data) to be submitted simultaneously as an array. Multiple values may be specified using a comma separated list and/or a range of values with a “-” separator. For example, 1-5 and 1, 2, 3, 4, 5 specify 5 jobs with indices 1 through 5. For range values, you may also assign an optional step size, e.g. 1-7:2 stands for 1,3,5,7.

Job array slot limit

The maximum number of jobs submitted from a job array that are allowed to run at a time. Valid values are positive integers.

Start time

The time (YYYY-MM-DD-HH-MM-SS) for the job to start.

STDOUT path

Specifies the path to a file to capture the standard output of the job. If a path is relative, it can be relative to the local or remote sandbox directory (in case of SSH connection). Supports placeholders: %j for the job ID and %i for the task ID, e.g. stdout_%j.txt.

STDERR path

Specifies the path to a file to capture the standard error output of the job. Supports placeholders: %j for the job ID and %i for the task ID, e.g. stderr_%j.txt.

Destination

Specifies a queue/partition where the job is submitted.

Restartable

Specifies whether the job can be restarted after a failure. A job is restartable by default.

Time limit

Estimated runtime for the job (in seconds). 0 stands for “no limit”.

Custom options

Any option specific for a definite batch manager (e.g. sbatch for SLURM; qsub for TORQUE, bsub for LSF, etc.). Note that custom options have priority over the rest.

The tables below list batch manager option equivalents for the Program block options.

Batch manager options

Option TORQUE SLURM LSF
Node count -l nodes=<count> -N <count> N/A
CPUs per node -l ppn=<count> -c <count> -n <count>
Node constraints -l <constraints> -C <constraint> [& <constraint>...] -R <constraint> [-R <constraint>…]

Job options

Option TORQUE SLURM LSF
Name -N <name> -J <name> -J <name>
Job array -t <array_spec> --array <array_spec> -J <name>[<array_spec>]
Job array slot limit -t <array_spec>%<slot_limit> --array <array_spec>%<slot_limit> -J <name>[<array_spec>]%<slot_limit>
Start time -a <start_time> --begin <start_time> -b <start_time>
STDOUT path -o <path> -o <path> -o <path>
STDERR path -e <path> -e <path> -e <path>
Destination -q <destination> -p <destination> -q <destination>
Restartable -r y or -r n --requeue -r
Time limit -l walltime=<HH>:<MM>:<SS> --time <D>-<HH>:<MM>:<SS> -W <minutes>
Custom options options 'as is' options 'as is' options 'as is'

Examples

Running a multi-core simple job on a cluster

Suppose you have a script that executes some solver on a TORQUE cluster, for example:

myscript.sh
#!/bin/bash
#PBS -l walltime=0:10:00
#PBS -l nodes=1:ppn=8
#PBS -l mem=8gb
#PBS -N My_calc
/sbin/SomeSolver/solver.bin --threads=8 --cmdfile=command.txt

The job parameters are as follows:

  • Estimated execution time: 10 minutes
  • 1 node and 8 cores on the node,
  • 1 node with 8GB of physical memory,
  • Job name: My_calc

The script starts a solver that reads additional parameters — for example, the names of input and output files — from a command file:

command.txt:
#solver commands

input=geometry.stl
output=results.txt

To configure the Program block for this job follow the steps below:

  1. Copy the line /sbin/SomeSolver/solver.bin --threads=8 --cmdfile=command.txt that starts the solver to the script editor of the block.
  2. Specify job requirements and parameters on the Cluster settings tab.
  • Batch manager: TORQUE
  • Node count: 1
  • CPUs per node: 8
  • Node constraints: mem=8gb
  • Job name: My_calc
  • Time limit (in seconds): 600

Leave other settings default.

../_images/page_blocks_Program_example_cluster_settings.png
  1. Enable the SSH execution on the Settings tab. Specify the parameters necessary to establish a secure connection (see SSH Connection section for details).

Pay particular attention to the following option settings:

  • Sandbox prototype: the directory where command.txt file is placed, e.g. ./calc.
  • Remote sandbox: specify the path: e.g. /tmp/remote_sbox.
  • Synchronization: set the type of file synchronization to “local to remote”. This configuration ensures that the local sandbox containing the command.txt file is uploaded to the remote host. Note that a “remote to local” synchronization is unnecessary here, as the output file will be sent to the corresponding port.
../_images/page_blocks_Program_example_SSH_settings.png
  1. Specify that the ./calc subdirectory is the sandbox prototype. To configure the sandbox, switch to the Sandbox tab in the block’s configuration. Input the path in the Prototype field: calc. Note that the block automatically transforms your input to a relative path (. means the current project directory).
../_images/page_blocks_Program_example_prototype.png

Place the command.txt file to the directory.

  1. The command.txt file contains information about the input and output files. They are not copied to the sandbox prototype, but will be added to the block’s configuration. As a result, ports of the File type with the same names will be created.

To add a file, switch to the Configuration tab and click b_blconf_add in the Files pane.

Add the input file.

  • Specify the port name: solver_in (the name of the port defined by the user).
  • Direction: input.
  • Specify the file name: geometry.stl (the input file’s name contained in the command.txt file).
../_images/page_blocks_Program_example_solver_in.png

Similarly, add the output file.

  • Port name: solver_out.
  • Direction: output.
  • File name: results.txt.
../_images/page_blocks_Program_example_solver_out.png

Verify file settings in the Files pane.

../_images/page_blocks_Program_example_files_added.png

Adding the input and output files results in creating the ports solver_in and solver_out of the File type.

  1. Click OK to close the configuration dialog.

The Program block will start when it receives a geometry.stl file to the solver_in port. The block creates a local sandbox in a temporary directory as the path to the local sandbox was not specified. The local sandbox will contain:

  • files placed in the sandbox prototype directory ./calc (. means the current project directory);
  • file captured by the solver_in port that is copied to a local sandbox under the name geometry.stl.

The local sandbox is uploaded to the remote host at the /tmp/remote_sbox directory (the one specified in the remote sandbox field on the Settings tab). The block generates the script as follows:

generated script
#!/bin/bash
#PBS -l walltime=0:10:00
#PBS -l nodes=1:ppn=8
#PBS -l mem=8gb
#PBS -N My_calc
/sbin/SomeSolver/solver.bin --threads=8 --cmdfile=command.txt

The generated script is submitted to the HPC cluster. After the job has been launched, the output file results.txt will be placed to the remote sandbox. Since the Program block knows that the results.txt file has to be output to the solver_out port, it automatically copies the results.txt file from the remote sandbox to the local one and sends it to the port.

Notes

This section answers some specific questions on using the Program block.

Environment Settings

The Program block supports per-block environment settings. To set up the environment, open the block’s configuration dialog and switch to the Environment tab.

../_images/page_blocks_Program_env.png

For details on using block environment settings, see section Environment.

Note that if the block is configured to launch remotely (uses SSH or connects to the pSeven Agent), environment changes apply to the remote process — the block automatically sets variables in the remote environment.

Catching Program Errors

Catching errors of external programs launched by the Program block requires checking for command errors in the script in order to terminate it on program error. The reason is that the block works with the script exit code, and the script can exit normally even if some command fails because by default a shell script does not fail if a command returns a non-zero status.

Though this is the default behavior of shell interpreters, in a workflow it may lead to unexpected results and gets very confusing since the error is hard to trace at the workflow level. For example, your workflow may include a Condition block which checks the Program block exit status and selects one of the two alternative workflow branches to continue. The check will not work as intended if the script in the Program block finishes with a command which always returns success, like this:

@program @fin > @fout
echo 1

With this script, the check in the Condition block will always pass, because the Program block exits with the status of the last command. Since echo always returns a success status, the script above is a “success” even when the first command (program launch) terminates with an error.

For this reason, error checking becomes essential for any non-trivial shell script. In Linux, there is a quick hack possible — adding set -e to the first line makes the script abort immediately when a command exits with an error:

set -e

@program @fin > @fout
echo 1

This is not recommended as a universal solution, and to be more correct you should check return values of important commands:

@program @fin > @fout || exit 1
echo 1

In Windows, there is no such global flag as in Linux, so the only option is to check every important command. The most simple check is to add || exit /b N after a command, where N is the exit code you want to set (make it any number except 0):

@program @fin > @fout || exit /b 42
echo 1

The command above exits the batch script with code 42 if the program execution raises an error. In practice all these methods are about exposing command errors to the Program block running the script, so it can generate the needed exit status.

Note

In Windows, the REM command used to insert comments in batch scripts (like REM This is a comment.) can also return success (this depends on the Windows version and the interpreter). As a result, comments in Windows batch scripts can interfere with the script exit status too.

A possible workaround is to start your comments with :: (for example, :: This is a comment.) — it adds an invalid GOTO label which is always ignored. However, comments like this must be placed at the beginning of a line, and cannot be inserted in code blocks.

Calling an External Script

In Windows, if you call an external batch script from Program, always use the call command. If you execute a batch file from inside another batch file (in this case, the block’s script) without using call, the “parent” batch file is terminated before the other one starts. For example, the following script stops after test.bat, and echo "BBB" is not called:

bad_call.bat
echo "AAA"
test.bat
echo "BBB"

In pSeven it will appear as if the script stops prematurely (does not run all commands), but the reason will be unclear because the calling script terminates normally.

The following script continues after call test.bat, and echo "BBB" is called:

good_call.bat
echo "AAA"
call test.bat
echo "BBB"

Note that if the called script terminates with an error, it sets ERRORLEVEL (global), but the execution continues. Then, the main script can finish, but its exit code will be the ERRORLEVEL set by the called script, and Program will catch this error code. Thus you can get lost in trace attempts like this:

bad_debugging.bat
...
echo "calling an external script"
call test.bat
echo "if you see this, the external script worked normally (THIS IS NOT TRUE)"
...

In fact, adding an echo command after a call does not verify that the called script finished normally. The main script continues even if call test.bat returns an error — only the ERRORLEVEL changes.

Consider adding || exit /b to the call (see Catching Program Errors) if you want the main script to stop immediately when the called script exits on error:

error_handling.bat
echo "calling an external script"
call test.bat || exit /b
echo "if you see this, the external script worked normally (and now this is true)"

Note that if you specify an exit code (as in || exit /b N), it will overwrite ERRORLEVEL set by the called script. Since it is more likely that you need the original exit code returned by the called script, here it is recommended to use exit /b without the exit code parameter.

SSH Connection Error on Unexpected Output

SSH (SFTP, SCP) connections are known to fail if some shell initialization script on the remote host (such as .profile, .bashrc, .bash_profile) produces output for non-interactive sessions. The simplest example is an echo command somewhere in the script, but this issue can also be caused by other commands.

The block’s SSH connection test checks for unwanted output on a remote host, so if you test your SSH connection, the block will inform you of this error. It produces a message like below, which warns that the remote command test failed due to unexpected stdout.

../_images/page_blocks_Program_remote_execution_ssh_test_fail_initoutput.png

If you have skipped the SSH connection test, this error is likely to occur when you run the workflow. The block then stops and reports it as “Garbage packet received” (this is an SSH protocol error message).

../_images/page_blocks_Program_remote_execution_ssh_runtime_initoutput_error.png

In this case, if you switch to the run log from the Run details dialog, you can see the same message about unexpected stdout in the log (at the error level).

Since this error can be caused by many different commands in various shell initialization scripts on the remote host, you will have to test them manually to find out which one produces the unexpected output, and fix these scripts accordingly. A common workaround is including a block of code like this at the very top of your initialization script (.bashrc or other):

# If not running interactively, do nothing
case $- in
    *i*) ;;
      *) return;;
esac

When the initialization script is executed in a non-interactive session, this code immediately returns without executing the rest of the script. Thus you can avoid the SSH connection error even if the script contains commands that print to stdout.

Known Issues

This section describes known issues related to using the Program block and their possible workarounds.

Network Paths in Windows

In Windows, running executables from network paths is not recommended due to system restrictions. For example, setting a network path to the block’s sandbox will cause an error when running the workflow, because cmd.exe, the Windows command line interpreter, does not support changing the working directory to a network path.

See Known IssuesNetwork Paths in Windows for more information.

Remote Execution and Parallelization

When you run a workflow containing a Program block, which is configured for remote execution over SSH (see SSH Connection) and is placed into a Composite block with parallel execution enabled (see Parallel Execution), the workflow may stop with the following error message:

Error reading SSH protocol banner.

The same message (with the Error log level) is also sent to the workflow run log.

The reason is that each parallel instance of the Program block opens a new SSH session on the remote host. However the SSH server in Linux systems is by default configured to accept at most 10 simultaneous sessions over a single network connection. So if the parallelization ratio in the Composite block’s configuration is greater than 10 (hence it can create more than 10 Program block instances), some instances of the Program block get rejected by the remote host and raise an error.

Possible ways to resolve this issue are:

  • Set a lower parallelization ratio for the Composite block, making it 10 or less.
  • Increase the session limit in the remote SSH server settings, making it greater than the Composite block’s paralellization ratio. To do this, edit the MaxSessions option in the SSH daemon configuration file on the remote host. Commonly the configuration file is /etc/ssh/sshd_config; if you cannot find it, refer to your Linux distribution documentation for a correct location. Note that editing this file usually requires root privileges.