Program

Tag: Integration

The Program block integrates any external executable in the workflow. The executable is launched by a shell script (bash script for Linux, cmd script for Windows). The user may define substitutions (script variables) which are used instead of file names in the script, and each substitution is mapped to a port (input or output) of the File type. The block also redirects standard output streams to the stdout and stderr output ports, and writes the return code of the executable to the return_code port.

If an execution timeout occurs, the block returns exit code 0 to the return_code port. By default, a timeout is considered an error, but the block can also be configured not to raise an error on timeout.

Notes

This section answers some specific questions on using the Program block.

Environment Settings

The Program block supports per-block environment settings. To set up the environment, open the block’s configuration dialog and switch to the Environment tab.

../_images/page_blocks_Program_env.png

For details on using block environment settings, see section Environment.

Note that if the block is configured to launch remotely (uses SSH or connects to the pSeven Agent), environment changes apply to the remote process — the block automatically sets variables in the remote environment.

Catching Program Errors

Catching errors of external programs launched by the Program block requires checking for command errors in the script in order to terminate it on program error. The reason is that the block works with the script exit code, and the script can exit normally even if some command fails because by default a shell script does not fail if a command returns a non-zero status.

Though this is the default behavior of shell interpreters, in a workflow it may lead to unexpected results and gets very confusing since the error is hard to trace at the workflow level. For instance, the integration wfIntegration workflow in prjGeneral Examples (see Examples) relies on the Launcher block exit status to select one of the execution branches. Try to edit the script in Launcher adding a command which always returns success, like this:

@program @fin > @fout
echo 1

With this script, the workflow will always fail, because the Launcher block now exits with the status of the last command, thus the script is a “success” even when the first command (which, actually, is the one we are interested in) terminates with an error.

For this reason, error checking becomes essential for any non-trivial shell script. In Linux, there is a quick hack possible — adding set -e to the first line makes the script abort immediately when a command exits with an error:

set -e

@program @fin > @fout
echo 1

This is not recommended as a universal solution, and to be more correct you should check return values of important commands:

@program @fin > @fout || exit 1
echo 1

In Windows, there is no such global flag as in Linux, so the only option is to check every important command. The most simple check is to add || exit /b N after a command, where N is the exit code you want to set (make it any number except 0):

@program @fin > @fout || exit /b 42
echo 1

The command above exits the batch script with code 42 if the program execution raises an error. In practice all these methods are about exposing command errors to the Program block running the script, so it can generate the needed exit status.

Note

In Windows, the REM command used to insert comments in batch scripts (like REM This is a comment.) can also return success (this depends on the Windows version and the interpreter). As a result, comments in Windows batch scripts can interfere with the script exit status too.

A possible workaround is to start your comments with :: (for example, :: This is a comment.) — it adds an invalid GOTO label which is always ignored. However, comments like this must be placed at the beginning of a line, and cannot be inserted in code blocks.

Calling an External Script

In Windows, if you call an external batch script from Program, always use the call command. If you execute a batch file from inside another batch file (in this case, the block’s script) without using call, the “parent” batch file is terminated before the other one starts. For example, the following script stops after test.bat, and echo "BBB" is not called:

echo "AAA"
test.bat
echo "BBB"

In pSeven it will appear as if the script stops prematurely (does not run all commands), but the reason will be unclear because the calling script terminates normally.

The following script continues after call test.bat, and echo "BBB" is called:

echo "AAA"
call test.bat
echo "BBB"

Note that if the called script terminates with an error, it sets ERRORLEVEL (global), but the execution continues. Then, the main script can finish, but its exit code will be the ERRORLEVEL set by the called script, and Program will catch this error code. Thus you can get lost in trace attempts like this:

...
echo "calling external script"
call test.bat
echo "external script worked normally"
...

Actually, the fact that the main script continued after call does not mean that the called script finished normally. Consider adding || exit /b to the call (see Catching Program Errors) if you want the main script to stop immediately when the called script exits on error:

echo "AAA"
call test.bat || exit /b
echo "BBB"

Note that if you specify an exit code (as in || exit /b N), it will overwrite ERRORLEVEL set by the called script. It is more likely that you will need the original exit code returned by the called script, so here it is recommended to use exit /b without the exit code parameter.

pSeven Agent

The Program block supports remote execution on a host that runs the pSeven Agent — a cross-platform utility that is automatically installed with pSeven. A Program block in a workflow that runs on the local host can connect to the pSeven Agent running on a remote host. The pSeven Agent receives input data and commands from the block and executes a program or script specified in the block’s configuration, then sends output data back to the block. pSeven Agent has to be installed on the remote host as standalone application or as a part of pSeven installation.

Agent Configuration

pSeven Agent configuration can be opened using a shortcut in the system menu: Start ‣ All Programs ‣ pSeven ‣ pSeven Agent.

../_images/page_blocks_Program_agent_gui.png

The configuration window contains three tabs.

  1. The Status tab shows an event log and agent status. Here you can configure and start the agent (by default it does not start automatically).
  2. The Tasks tab shows tasks received by this instance of the pSeven Agent.
  3. The About tab contains brief information and links to help and log files.

To configure the agent, click the “Configure...” button on the Status tab.

../_images/page_blocks_Program_agent_config.png

pSeven Agent supports the passive mode only (selected by default). In this mode it listens on the specified port; this is the port that should be specified in the Program block configuration. You can select any available port, default is 8899.

Client tokens are identifiers sent by Program when it connects to the agent. The token sent by the block must be found in this list, otherwise the connection is not allowed. Note that you can specify a list of tokens in the agent’s configuration, in order to distinguish between the clients.

You can also configure the agent to start automatically using the “Autostart agent” option. When it is enabled, the agent will start immediately once you launch it with the menu shortcut. Other settings are not used.

If “Autostart agent” is not enabled, you will have to click “Start” on the Status tab. After this it will show the host (server) address and port (endpoint) and will accept incoming connections.

Block Configuration

The connection to pSeven Agent is configured on the Settings tab in the block configuration dialog.

../_images/page_blocks_Program_agent_connect.png
  • Endpoint: the address and port of the host that runs pSeven Agent.
  • Authentication token: the token to send when connecting to the Agent. Note that this token has to be included into the list of client tokens in the agent configuration, otherwise the connection will not be allowed.
  • Synchronization: specifies the type of file synchronization between the block and the host running pSeven Agent. The files to sync have to placed in the block’s sandbox (see section Sandbox for details).
    • None: no synchronization.
    • Two-way: a synchronized copy of the block’s sandbox will be created on the remote host. After executing the program, contents of the remote sandbox will be synced back to the local sandbox.
    • Local to remote: the local sandbox will be uploaded to the remote host before starting the program, but the remote copy will not be synced back.
    • Remote to local: local sandbox files will not be uploaded, but files created on the remote host will be downloaded to the local host.

After specifying the endpoint and authentication token you can test the configuration using the connection test button.

../_images/page_blocks_Program_agent_connect_test.png

If authentication is successful, pSeven will show a message box indicating that the connection test passed.