Simple Workflow

This tutorial is the first in series of three tutorials that provide a general practical introduction to pSeven. Introductory tutorials do not involve any real task yet and describe a toy workflow which uses a few simple blocks purely for explanation purposes.

Note

In case you find something described in this tutorial unfamiliar, see section Introduction for general information on pSeven interface and functions.

Before You Begin

Every workflow in pSeven is a part of some project. A project can contain multiple workflows and provides a database to store workflow results. Projects are also used to organize files — for example, keeping related workflows and their input data together.

All tutorials found in this manual assume you are working with the prjTutorials project. If you have not created this project yet, see Tutorial Project.

Adding a Workflow

To add a workflow, first of all you need to open or create a project that shall contain it.

  • Open the prjTutorials project.
  • Select File ‣ New Workflow from the main menu. You can also use the new workflow shortcut on the Welcome tab or select New Workflow from the Project pane context menu.

When you add a workflow in Workspace, pSeven automatically switches to Edit.

../_images/page_tutorials_workflow_02_newwf.png

In Edit you now see a new empty workflow 1. Note that the workflow is untitled 2 — it means that it is not saved to a file yet and thus it will not yet appear in project contents in Workspace.

Note

You can also add workflows in Edit using the b_newtab button on the tab bar 3 and edit multiple workflows switching between their tabs.

Workflows in pSeven can have descriptions that are edited in workflow configuration.

  • Add a description to your workflow: click b_runconf on the edit toolbar 4 to open the workflow configuration dialog. Input some text on the General tab.
../_images/page_tutorials_workflow_03_wfdescription.png

Click b_ok to save the description and close the dialog. Later you can edit the description at any time.

To save a new workflow to a file (so that it appears in Workspace), you can use the save button b_save on the tab bar or select File ‣ Save Workflow from the main menu.

  • Save the workflow before continuing: click b_save on the tab bar and input a file name in the dialog (for example, SimpleWorkflow).
../_images/page_tutorials_workflow_04_save_wf.png

Note that workflows are always saved to the project directory. You cannot save a workflow outside the current project or to a subdirectory in the project.

When saved, the new workflow name appears on its tab in Edit. You can switch to Workspace and verify that the workflow file now exists in your project.

../_images/page_tutorials_workflow_05_wf_tooltip.png

To show the tooltip, select the workflow in the Project or the Filesystem pane. Note that file attributes (type, size and modification date) are now shown in the tooltip.

Switch back to Edit to proceed with creating a workflow: adding blocks and connecting them with links.

Adding Blocks

Blocks are added to the workflow from the Block Library pane in Edit.

../_images/page_tutorials_workflow_06_addblocks.png

The library uses tag sorting. You can expand the tag nodes 1 to see a list of blocks. To find a specific block you can also filter the library 2 by block name or tag. For example, rand will find the Random block, and tegra will find all Integration blocks (the filter is case insensitive and can match partial names).

To add a block to the workflow, you can either double-click its name in the Block Library pane or drag it from the pane to the edit area.

  • Add a Random block — for example, expand General 1 and double-click Random 3.

Every block in a workflow has a unique name in order to tell blocks of the same type from each other. When adding a new block, you have to specify its name in the Add Block dialog.

../_images/page_tutorials_workflow_07_blockname.png
  • Type the name in the dialog (for example, RandGen) and click b_ok. Note that the name has to begin with a letter, and some special symbols are not allowed.

The RandGen block is added and now you can see it in the workflow.

../_images/page_tutorials_workflow_08_randgen_added.png

Hovering the mouse cursor over a block brings up an information tooltip 1. If you select (single-click) a block, you can also find these details on the information panel 2 (this panel can be toggled with the b_edit_bip button 3).

The RandGen block has been added with the default configuration which makes it generate and output a single random value. However the majority of blocks in pSeven (and Random, as well) have various configuration options which can change block behavior, its inputs, outputs and other properties, as you can see further.

Block Configuration

The RandGen block which is currently configured to generate a single random value (default) can also fill vectors or matrices with random values. The output mode is changed in the block configuration.

  • Double-click the RandGen block to open the configuration dialog. Default configuration generates a scalar value from the uniform distribution.
../_images/page_tutorials_workflow_09_randgen_default_conf.png

If you switch to the Ports tab in the configuration dialog, you can note that currently there is a single output port value which outputs the generated scalar (do and done are special signal ports common for all blocks; these ports can be used for block synchronization).

../_images/page_tutorials_workflow_10_randgen_default_ports.png

Making changes in block configuration often results in altering its inputs and outputs. For example, let us reconfigure RandGen to output a random matrix.

../_images/page_tutorials_workflow_11_randgen_matrix_conf.png
  • Switch back to the Configuration tab in the dialog.
  • Set the output mode to matrix 1.
  • Change other options if you wish (double-click option value to edit). Non-default options are displayed in bold 2.

If you look at the Ports tab after these changes, you can see that two new input ports are added: rows and cols.

../_images/page_tutorials_workflow_12_randgen_matrix_ports.png

Values received to these ports set the number of rows and columns in the generated matrix. Note that these settings are not present in block options, so you are required to send these values to input ports — otherwise the block will not start.

  • Click b_ok in the configuration dialog to save settings and close the window.

Note that pSeven informs you that there is a problem with the workflow by sending warnings to the Issues pane.

../_images/page_tutorials_workflow_13_randgen_issues.png

Toggle the Issues pane 1 to see what the problem is. You can see two warnings 2 informing that required input ports do not receive any value; as a consequence, the block that is the source of these warnings will not start when you run the workflow. The third warning tells you that an output value is not used anywhere.

Note also that if you have changed block options, these changes (non-default option values 3) are now listed on the information panel.

Note

Reported issues can have different severity levels. A warning b_issue_pane_warn informs about a non-critical misconfiguration which does not prohibit to run the workflow. As for the current workflow, for example: you could run it right now, ignoring the warnings; there would be no block errors, but the workflow would simply do nothing. An error b_issue_pane_err is a critical issue, and a workflow with errors will cease to run.

To fix current issue with RandGen inputs, the values that determine the matrix size have to be defined somewhere and sent to the rows and cols input ports of the RandGen block. A straightforward way is to use a Const block that can define and output constants.

../_images/page_tutorials_workflow_14_size_and_randgen.png
  • Add a Const block 1 to the workflow.
  • This block shall set matrix size: name it Size 2.

Double-click Size to open block configuration. By default, it defines no constants, so the list is empty.

../_images/page_tutorials_workflow_15_size_conf_empty.png

To add constants, use the context menu 1 or the b_blconf_add button 2 to bring up the Add Constants dialog. You need to define two constants, for the number of rows and columns. Both constants can be added at once — just list them in the Name field:

../_images/page_tutorials_workflow_16_addconsts.png
  • Name: rows, cols (note that the names are case-sensitive).
  • Value: any integer value (for simplicity purposes, set a small value like 5).

Both constants will have the same value, but you can change it in block configuration afterwards. Verify that two constants have been added.

../_images/page_tutorials_workflow_17_const_ready.png

Adding constants also adds output ports (with the same names) to the Size block. The constants, and hence their ports, are intentionally named the same as RandGen inputs: this is needed for the autolinking feature explained below. If you mistyped a name, double-click it in the Name column 1 to correct. You can also change values in the Value column 2 if you wish: double-click a value to edit, or hover it and click b_editval.

Click b_ok in the block configuration dialog to save settings.

../_images/page_tutorials_workflow_18_const_bip.png

If you look at the Size block information panel, you can note that two new output ports were added, and both are assigned some values. You can now link the blocks to send these values to RandGen inputs.

Linking

Links set up the data flow between blocks. They are created in Edit by dragging a link handle from a source block to the target one.

../_images/page_tutorials_workflow_19_link_drag.png
  • Hover the source block with the mouse cursor to show the link handle 1.
  • Click the handle and drag it away from the block, holding the mouse button 2.
  • Move the handle to the target block until it is highlighted 3, then release the mouse button.

If you have named the constants correctly, pSeven will match the ports of Size and RandGen automatically and display an Autolink dialog.

../_images/page_tutorials_workflow_20_autolink_dg.png

Accept both links (default) and click b_ok to add the link.

If you get a detailed Links dialog instead, it means that there is no match between source and target ports. In this case, you can either edit the names of constants in Size (hence changing its output port names as needed), or set up the link manually.

For example, in the current workflow the Links dialog can look as follows (you can also switch to this view from the Autolink dialog):

../_images/page_tutorials_workflow_21_link_dg.png

To create a connection in the Links dialog, first select the source (output port) 1 on the Ports pane, then select the target (input port) 2 on the Connections pane. This dialog is usually used in complex workflows; for example, it allows creating one-to-many connections.

  • Either way (using autolink or manually), set up the link connecting two port pairs: Size.cols to RandGen.cols, and Size.rows to RandGen.rows.
../_images/page_tutorials_workflow_22_linked_blocks.png

Note that links also provide tooltips 1 when hovered and details on the information panel 2 when selected.

Now the workflow can run and generate data, but this data is dropped: RandGen is a simple block that can only output the generation result to its value port, and this port is not connected to anything (there is a warning on this in the Issues pane).

There are several ways to capture a block’s output in pSeven:

  1. You can send the output data to a block that can write it to a file (like CSVGenerator).
  2. You can set up monitoring for an output port so that the data is saved to the project database.
  3. You can connect a block output to a workflow output (uplink a port). Workflow output values are shown in Run after the workflow finishes, and are also saved to the project database (all workflows inputs and outputs are monitored automatically).

This tutorial further explains port monitoring, as it is the most general and flexible method. Workflow inputs and outputs are used in the Workflow as a Tool tutorial, and saving workflow data to a file is explained in several other tutorials.

Monitoring

The monitoring mechanism in pSeven allows to capture the activity of any port in a workflow, both input and output. When a port is monitored, all data it sends or receives is immediately copied to the project database available in Analyze.

Monitoring is set up in two steps:

  1. First, you select the ports that allow their monitoring.
  2. Second, you select which ports (from allowed ones) are actually monitored while the worfklow runs. This step is optional: by default, monitoring is enabled for all ports selected at the first step.

This way you can first select only the important ports, then adjust monitoring settings quickly without referring to a full list of workflow ports. In particular, it helps in the case of a complex workflow that is created and run by different users.

Ports that allow their monitoring can be selected in block configuration.

../_images/page_tutorials_workflow_23_value_monitoring.png
  • Double-click RandGen to open the configuration dialog.
  • Switch to the Ports tab 1.
  • Select the value port for monitoring 2.
  • Click b_ok to save settings and close the dialog.

Select RandGen to view its details.

../_images/page_tutorials_workflow_24_randgen_bip2.png

The information pane shows more changes done to the block: rows and cols inputs 1 are now connected; value output is disconnected (its name is grayed out) but monitored 2. Note that monitoring resolved the issue from before: there are no more warnings on the Issues panel because now pSeven knows how data from the value port is used.

A new monitor for the port you have just selected now appears in Run.

  • Switch to the Run screen and note that a new monitor “Randgen.value” appeared in the Configuration tab.
../_images/page_tutorials_workflow_25_wf_conf_monitor.png

Here you can disable and enable monitoring quickly 1 (new monitors are enabled by default). This completes the essential configuration, further settings are optional.

Note that the monitor’s name and description 2 are defaults copied from RandGen properties. pSeven allows to change them if you want to make monitoring settings in Run easier to understand. For example, you can add a description explaining the role of this port in the workflow, which will hopefully help another user to better understand what data is monitored.

Names and descriptions of monitored ports are edited in workflow configuration 3 (this is the same tool you used earlier to add a workflow description; it is available both in Run and Edit).

../_images/page_tutorials_workflow_26_wf_conf_monitor2.png
  • Open workflow configuration b_runconf and switch to the Monitoring tab 1.
  • This tab provides an overview of existing monitors. Note that if there are many of them, you can filter 2 the list by partial name (case insensitive) to find the one you need. You can also add and remove monitors here 3 instead of selecting ports in block configuration. Finally, you can change the name of any monitor (give it a new alias) and edit its description.
  • Double-click the default alias 4 to edit, remove it and type a new name. It is best to keep this name short and simple, for example: Random Data.
  • Double-click the default description 5 and replace it with your text. Description works better when you want to add more details describing the monitored value.
  • Click b_ok to save settings and close the dialog.

Note the changes in monitoring configuration.

../_images/page_tutorials_workflow_27_wf_conf_monitor3.png

One of the reasons to set meaningful aliases is that the alias becomes the name of the record in the project database that stores monitored data. So, current configuration is:

  1. Monitor the value output of the RandGen block (capture all data output to this port).
  2. Save this data to the project database for further analysis, storing it under a record named “Random Data”.

Would you not set an alias, the record would be given a default name in “Block.port” format (“RandGen.value”). This becomes important in analysis of complex tasks with many monitored values. The idea is that the data itself — its meaning, type, structure, other details which you can specify in alias and description — is more relevant to the user than its source block and port — which, if required, can be traced back from the alias using the workflow configuration b_runconf tool.

Now you have a very simple workflow that generates and saves some random data.

../_images/page_tutorials_workflow_28_pre_run.png

This workflow uses only the most basic capabilities found in pSeven, but it is ready to launch. You can now run the workflow to see the results.

  • Verify there are no validation issues 1 and correct, if any.
  • Verify the data monitor you added earlier is enabled 2.
  • Save your workflow 3 and click b_run to run it 4.

Run

When you run a workflow, the bottom panel in Run automatically switches to the run log.

../_images/page_tutorials_workflow_29_finish_ok.png

The Run Log tab 1 always shows the last run log of the current workflow. Previous run logs, if needed, can be found in the pSeven console 2. Note that the console holds a single log common for all opened workflows, while the information in Run Log is specific to the workflow you currently use (edit or run) — that is, there is a separate last run log on each workflow tab 3.

When the workflow is complete, a message regarding its status is displayed in the Statistics pane 4 (together with the details on block statistics and execution time).

The status can be either “Finished successfully” or “Finished with errors”. You may single-click details to get some extra information about error details.

../_images/page_tutorials_workflow_30_finish_error.png

The current workflow should run without errors. Both its blocks started once: Size output constants that set matrix size and RandGen generated a random matrix after receiving these settings. Size (Const) is one of the few blocks that do not require any input to start — that is why it was selected as a “workflow starter”.

Note

The workflow in this tutorial is linear, but in general workflows can contain loops. A typical example is any optimization workflow, which is always built around an optimization loop controlled by an Optimizer block (see the Optimization Basics tutorial, for instance; a generic example is also available in the Sampling Loop tutorial). Blocks inside the loop start multiple times, in which case block statistics become more useful — for example, you can tell the number of function evaluations by looking at the number of block startups inside an optimization loop.

As you can see, blocks starting order is in fact determined by links between them: a block starts as soon as it has all required data, and links control how data travels between blocks. This rule is known as the data flow concept. Note that there is no need to specify block execution order explicitly; however, you can delay the startup of a certain block until another block finishes (using the do and done ports).

Now that we have briefly described what happens during a workflow run, let us switch to Analyze to discuss its results.

Results

In Analyze you can see that the data collected from your workflow now appears on the Project Database pane.

../_images/page_tutorials_workflow_31_first_res.png

The project database is common for all workflows in the project, and each workflow creates a node 1 named the same as the workflow file. Its child nodes 2 store run results. By default, each run creates a new child node, using the workflow start timestamp to generate the run name. Also there is a special node named “<last run>” 3 — this is an alias for the most recent run record.

Alternatively, you can set a fixed run name in Run to collect all data under the same record. Such setting appends new results to previous ones without overwriting them. Note that in this case “<last run>” references the fixed name and will also contain data from all runs — not only from the actual last run.

Note

Reports are explained in more detail in the Results and Reports tutorial. In particular, it provides examples of using a fixed run name and the “<last run>” record.

A run record stores various information including basic statistics, workflow input values and so on. For now we are only interested in the data found in “Monitoring”.

  • Expand “Monitoring” to see the “Random Data” record 4 which stores the data obtained by the monitor you added earlier.

The “Random Data” record can be easily added to a report as a data sample. After this you can use it for plotting and analysis.

../_images/page_tutorials_workflow_32_rep.png
  • Click b_newtab on the tab bar 1 or select File ‣ New Report from the main menu to create a new report.
  • Drag the “Random Data” record to an empty area 2 in the report.
  • pSeven automatically creates a sample viewer 3 to show the data.

pSeven provides a quick plotting function which instantly creates a plot of desired type based on an existing sample viewer.

  • Click the parallel coordinates button 4 on the report toolbar to get a plot of “Random Data” in parallel coordinates 5 (this is one of the plot types used to visualize multidimensional data; here, each polyline represents a matrix row which can be understood as point coordinates in 5-dimensional space).

Quick plots use default configuration; after adding a plot, you can click b_config in the plot title bar to adjust it (change fonts, labels, colors and other details).

Note that a report can include many tables and plots, so there is no need to create a new report for each. Nevertheless you can add multiple reports to a project, view and edit them simultaneously (similar to workflows).

To save a report, click b_save on the tab bar or select File ‣ Save Report. Same as a new workflow, a new report is untitled until you save it to a file, and report files can be saved only to the project directory.

Conclusion

This tutorial includes only the most essential information and omits many details. It is also strongly recommended to complete the next two tutorials (Workflow as a Tool, followed by Results and Reports) before starting any other.

  • Save the workflow you have created in this tutorial: it will be required further. The report you have created when viewing the results is no longer needed, but you can also save it if you wish.

Note that the tutorials mentioned above continue with the same workflow. Workflow as a Tool shows how to customize the workflow interface in Run, using the current workflow as an example. Results and Reports uses data generated by this toy workflow when explaining how to create a report that automatically processes workflow results.