Conditions and Error Handling

This tutorial provides an example of handling possible errors during workflow execution. It discusses two methods: adding conditional branches to the workflow and using special block options that control the block’s error handling behavior.

Note

This tutorial continues the Integration Basics tutorial and requires the workflow created there.

Before You Begin

This tutorial requires an existing prjTutorials project. If you have not created this project yet, see Tutorial Project first.

  • Open the prjTutorials project.
  • Open wfIntegrationBasics (IntegrationBasics.p7wf).
  • Select File ‣ Save Workflow As... to create a copy of the workflow. Save the workflow as wfErrorHandling.
  • Verify that you are editing wfErrorHandling and continue with the tutorial.

Task

As noted in the Integration Basics tutorial, the current version of the Toy Solver integration workflow does not handle solver errors in any way. As a result, the workflow simply aborts when any input is out of the valid range (\([0.0, 1.0]\)). While this behavior is generally correct, in many cases it is not wanted. For example, if the Toy Solver integration workflow is used as a block that evaluates objective functions in an optimization workflow, you would want to somehow inform the optimizer that an evaluation failed but let it continue solving, rather than stopping at the first error.

Avoiding invalid input is rather easy — for example, in case of optimization it would be enough to specify variable bounds. However Toy Solver can be started in a “buggy” mode using the -n option (it exists purely for example purposes and has no real meaning).

-n

Adds a small region of undefined behavior near \((1.0, 0.0)\).

We assume that the location and shape of this region are unknown, which makes us look for a more generic method of handling Toy Solver errors.

To begin, add the -n option to the Toy Solver command line in the Launcher configuration.

../_images/page_tutorials_error_handling_01_notch.png
  • Add -n before the input file name argument.
  • Verify the command preview. In Windows: toy_solver.exe -n input.dat >output.dat. In Linux: ./toy_solver -n input.dat >output.dat.
  • Click b_ok to save changes and close the configuration dialog.

Try reproducing the error: for example, specify (1.0, 0.0) input in Run and start the workflow.

../_images/page_tutorials_error_handling_02_wf_err.png

This message means that Launcher started Toy Solver, but it returned exit code 1 which is not in the list of success codes in Launcher configuration (see LauncherConfigurationSettingsSuccess codes). Thus the block decides that program execution failed and stops the workflow by default.

Solution

The solution in this tutorial consists of two steps:

  • Configure blocks in such a way that they do not interrupt the workflow in case of error but output suitable defaults.
  • Add a simple input check and create two conditional branches in the workflow — the main branch (which already exists) and the branch that activates when the input is clearly invalid (out of \([0.0, 1.0]\)).

Note that the second step is optional but can save some time because the main branch will not activate when the input check fails. It also provides an example of using the Condition block to switch branches.

Error Handling

This step requires a few changes in Launcher and OutputParser configuration to make OutputParser send predefined constants to workflow output when Launcher encounters an error.

First of all, change the behavior of Launcher so it does not interrupt the workflow when Toy Solver exits with code 1.

../_images/page_tutorials_error_handling_03_eh_opt.png
  • Open the Launcher configuration and switch to the Options tab.
  • Double-click the value of the Error handling behavior option and change it to “output defaults and signal”.
  • Click b_ok to save changes.

The error handling option has three states:

  • “stop workflow” — default, the block interrupts workflow execution on error.
  • “output signal only” — in case of error, the block does not interrupt the workflow but outputs False to the done port; nothing is output to other ports. If there was no error, the block outputs True to done. This setting is often used with a Condition block analyzing the output from done and activating an alternate workflow branch on failure (see section Fallback for an example).
  • “output defaults and signal” — the same as above, but in addition to the done signal, in case of error the block also outputs default values to other ports.

Output defaults depend on the block; for Launcher (a Program block), default output file is an empty temporary file (see Launcher configuration, Ports tab).

Try running the workflow to see the effect of changes.

../_images/page_tutorials_error_handling_04_wf_err2.png

As you can see, Launcher no longer stops on error, but now the problem is that OutputParser is not able to parse the empty file it receives from Launcher. To suppress this error, change the error handling behavior in OutputParser.

  • Open the OutputParser configuration and switch to the Options tab.
  • Double-click the value of the Error handling behavior option and change it to “output defaults and signal”.
  • Do not close the configuration dialog yet.

Lastly, you need to set default output values. By convention, evaluation errors in pSeven are indicated with NaN values (for example, Optimizer understands NaN values of objectives and constraints as undefined behavior and tries evaluating another design point).

../_images/page_tutorials_error_handling_05_opnan.png
  • Switch to the Ports tab in the OutputParser configuration.
  • Double-click the f port value in the Outputs pane and change it to (NaN, NaN).
  • Click b_ok to save changes and close the configuration dialog.

Now if OutputParser receives a file it cannot parse (such as the default empty file from Launcher), it will ignore parsing errors and output the default you have specified.

  • Try running the workflow with various invalid inputs and see the output.

Note that no matter what the input is, all blocks in the workflow start once. It means that the workflow always starts Toy Solver — however we know that if some input is out of the \([0.0, 1.0]\) range, the evaluation is actually not needed. The next section explains how to bypass the Toy Solver execution and output defaults immediately when the workflow detects an invalid input.

Input Check

To validate input values you can use a Condition block. This block can check input values against a condition you define in its configuration and change its output depending on the result of this check.

  • Add a Condition block. Name it CheckInput.

Open the CheckInput configuration. The variables to check have to be added first.

../_images/page_tutorials_error_handling_06_condvar.png
  • Click b_blconf_add on the Variables pane toolbar and add a RealVector variable.
    • Specify the name: x.
    • Select the RealVector type.

The condition should check that both input components (x[0] and x[1]) are in the valid range.

../_images/page_tutorials_error_handling_07_cond.png
  • Input condition: x[0] >= 0 and x[0] <= 1 and x[1] >= 0 and x[1] <= 1. Conditions allow basic logical syntax — see section Conditions on the Condition block page for details.
  • Finally, choose what happens when the condition is not met: select “Redirect output to else_<variable> ports”.

In this configuration the key point is output redirection. We want to create two alternative workflow branches — one (main) is triggered on a valid input (condition met), the other one — on an invalid input (condition not met). Redirection means that CheckInput adds two output ports, x and else_x; both output of them the value received to the x input with no changes, but the first one activates only if the condition is met, while the second one activates only if the condition is not met (for more details, see section Output on the Condition block page). Normally data shall flow from CheckInput.x to InputGenerator.x, starting the main branch. Otherwise, when the condition is not met, we are going to use the output of CheckInput.else_x as a signal to start a Const block that sends a (NaN, NaN) vector to the workflow output.

  • Add a Const block. Name it DefaultOutput.

Open the DefaultOutput configuration and click b_blconf_add on the Constants pane. Add a vector constant.

../_images/page_tutorials_error_handling_08_cnan.png
  • Specify the name: f_default.
  • Specify the value: (NaN, NaN).

Finally, set up the links to create a conditional node in the workflow.

  • Remove the link from the root input x to InputGenerator.x.
  • Uplink CheckInput.x.
  • Link CheckInput.x to InputGenerator.x.
  • Link CheckInput.else_x to DefaultOutput.do.
  • Uplink DefaultOutput.f_default to the existing root output f.

Note that input values are not passed to InputGenerator directly because it must not start when the values are out of the valid range. This is important because both branches output their final result to the same root output f, so you have to ensure that only one of them activates when the workflow runs.

Workflow

The final workflow contains two branches: main evaluation (top) and default (bottom).

../_images/page_tutorials_error_handling_09_fin.png

CheckInput tests whether input values are in the valid range. If the test passes, the input is sent to InputGenerator, starting the main branch. Default branch does not start because DefaultOutput.do is connected, so the block can start only if this port receives a value. Potential evaluation errors in the main branch (due to the -n option) are handled by Launcher and OutputParser. If the input check fails, CheckInput does not start the main branch but outputs a value to CheckInput.else_x — this port is connected to DefaultOutput.do, so it activates the default branch.

Results

In general, results are the same as in the Integration Basics tutorial. The point of interest is how the updated workflow processes different inputs. There are three cases:

  • At least one of the input components is out of the \([0.0, 1.0]\) range (invalid input).
  • Both components are in \([0.0, 1.0]\) but the point falls into the region of undefined behavior near \((1.0, 0.0)\) (valid input but Toy Solver fails to evaluate).
  • Normal evaluation (no errors).

The first case, input is (0.0, 2.0) for example.

../_images/page_tutorials_error_handling_10_case1.png

As you can see, the blocks in the main branch do not start. The output is (NaN, NaN).

The second case, input is (0.99, 0.01) for example.

../_images/page_tutorials_error_handling_11_case2.png

This time the input check is passed, so DefaultOutput does not start. The solver generates an error, which is handled in the main branch; the output hence is again (NaN, NaN).

The third case, input is (0.5, 0.5) for example.

../_images/page_tutorials_error_handling_12_case3.png

The input check is passed, and the main branch starts. Evaluation completes successfully (the output is (0.5, 3.841688)).

Conclusion

The workflow from this tutorial is the final version of an input-output adapter for an external program, featuring correct error handling and even some runtime optimization (skipping redundant evaluations).

  • Save the workflow you have created in this tutorial: it will be required further.

Note that this workflow is further used as a new block integrating Toy Solver to other workflows, beginning with the Workflow as a Block tutorial which explains the basics of workflow import. Other tutorials using this block include:

  • Sampling Loop — explains how to perform batch calculations with Toy Solver and provides an example of creating loops.
  • Parallelization — an improved version of the batch calculation workflow providing an example of block parallelization.
  • Caching — an example of using block input-output cache for efficient calculations.
  • Integrated Component Optimization — explains how to build an optimization workflow where Toy Solver evaluates objective functions.