Results and Reports

This tutorial is the third and final one in the general practical introduction to pSeven. It explains how to complement a workflow with a report which plots the results and updates automatically after every run.

Note

This tutorial continues the Workflow as a Tool tutorial and requires the workflow created there.

Before You Begin

This tutorial requires an existing prjTutorials project. If you have not created this project yet, see Tutorial Project first. You will also need the workflow created in the Workflow as a Tool tutorial.

  • Open the prjTutorials project.
  • Open wfSimpleWorkflowTool (SimpleWorkflowTool.p7wf).
  • Select File ‣ Save Workflow As... to create a copy of the workflow. Save the workflow as wfSimpleWorkflowAnalyze.
  • Verify that you are editing wfSimpleWorkflowAnalyze and continue with the tutorial.

Task

In pSeven you can create reports which automatically update their contents when new results appear in the project database. Configuring such reports requires some test data, but after the report is ready, the test data can be removed from the project database. Combined with workflow configuration capabilities explained in Workflow as a Tool, this allows to prepare clean projects with simple Run interface and preconfigured reports to show results in Analyze.

This tutorial provides an example of configuring a report for the workflow created in the Workflow as a Tool tutorial. The report shall plot samples and update the plot when new samples are generated. For illustrative purposes, let us assume that samples are always two-dimensional. Also, the report will rely on the fact that sample dimension is fixed, so a few changes to the workflow are required.

Solution

The first step in solution is to fix the dimension of generated samples. Currently the workflow allows the user to change it at his or her own discretion (the Dimension input). This input should be removed and replaced with a constant. One obvious way is to disconnect RandGen.cols from the root cols port, add a Const block and use it to send a constant value to RandGen.cols. However there is a better way which does not require extra blocks: add a value on RandGen.cols and do not connect this port anywhere, so the value on port is accepted as input.

After this you will have to do a few test runs to gather data and configure a report in Analyze.

Port Values

Since you are no longer allowing to change the sample dimension, the Dimension input should be removed from Run. The only way to do it is to delete the related root port.

  • Open the root block configuration and remove the cols port.

The input link remains, but now it connects only one port pair.

../_images/page_tutorials_analyze_01_rowsin.png

RandGen.cols input is now disconnected, and a warning appears on the Issues pane.

../_images/page_tutorials_analyze_02_colsin_issue.png

The warning will be removed when you add a value on port.

../_images/page_tutorials_analyze_03_const_cols.png
  • Open RandGen configuration and switch to the Ports tab.
  • Double-click the field in the Value column and type 2.
  • Click b_apply to save the setting and switch to the Configuration tab.
  • Set the “Deterministic” option value to “disable”.
  • Click b_ok and close the dialog.

Note

Setting the “Deterministic” option value specified in RandGen configuration to “disable” is required to properly solve the specific task outlined in this tutorial.

Note that the report will rely on the fact that the sample dimension is constant. For this reason, RandGen.cols should not be a parameter: it would make the dimension setting available to the user in Run.

  • Run the workflow once with default inputs and parameters, then switch to Analyze.

Report

The SimpleWorkflowAnalyze node in the project database should now contain two child records.

../_images/page_tutorials_analyze_04_report.png

Begin with adding a sample viewer.

  • Click b_newtab on the tab bar to create a new report.
  • Find the “sample” record 1 in <last run>, drag and drop it to an empty area 2 in the report to create a new viewer 3. Note that it is important to use values from <last run>, not from a timestamped record.

The viewer you just created can be quickly updated to the most recent results when you run the workflow next time: just click b_refresh on the report toolbar 4 after run. Same goes for any plot created in a similar way from the <last run> record.

Note

Sample viewers and plots in a report can be created and configured while a workflow is running, and refresh also works at the run-time. This feature is often used to plot the evaluation history in optimization workflows which can take hours to complete.

You can find more on what you can do with Sample Viewer in prj2.02 General - Sample Viewer project in wfSample Viewer (Sample Viewer.p7rep) report.

  • Save the report as repSimpleReport and continue.

Report Database

In order to describe how refresh works, let us first explain what happens in the background when you add a sample viewer by dragging data from the project database to the report.

Each report (a .p7rep file) has its own database. It is stored in a separate file (a .p7rdb file with the same name), but pSeven handles the report and database as a pair, and never shows .p7rdb files in Workspace.

../_images/page_tutorials_analyze_05_ds1.png

Report database contains data series — 1D arrays automatically formed from multidimensional project database records. This data is shown on the Data Series pane 1 (click 2 to toggle). Note that data series are parts of the report, so contents of the Data Series pane change if you switch to another report.

Every window in the report (sample, plot) is just another viewer for data series. Viewers do not connect to the project database but use the report database as a source. However, the report database is synchronized with the project database. Each data series remembers its source 3 — the original project database record. When you click b_refresh 4 on the report toolbar, the report updates its data series (re-reads data from the project), then refreshes all viewers (which re-read data series).

As a safety measure, the report database does not synchronize deletions. That is, if original data is deleted from the project database, or the project database gets corrupt, the report still keeps a copy of data. In such a case this copy is no longer synchronized (nowhere to sync from) and the refresh function stops working. However it does not break the report — even if you delete the project database and refresh the report, it will not delete data from the report.

Note

Data series can be removed only manually, using the context menu b_context on the Data Series pane.

Data series are not deleted even if they are not used by any viewer. For example, if you add a sample viewer by dragging records from the project database and then remove it, the data series that were created in background remain in the report database.

Plotting

This section explains the intended usage of data series with an example of a plot that shows results from all workflow runs. It is recommended to rename existing data series first.

../_images/page_tutorials_analyze_06_ds2.png
  • Double-click the name of data series to edit it (or click b_blconf_write).
  • For example, rename “Sample[0]” to “X”, and “Sample[1]” to “Y”.

Note that the column names in the sample viewer change automatically. They were not specified, and by default the column name is the same as the name of its source data series.

Add a 2D plot, this time using two existing series as data sources.

../_images/page_tutorials_analyze_07_ds_plotadd.png
  • Select both data series 1 (hold Ctrl or Shift for multiselection).
  • Click the 2D plot button 2 and select the Point plot option (first label).
  • A new quick plot 3 is added automatically.

Click b_config on the plot title bar to adjust settings.

../_images/page_tutorials_analyze_08_plotconf1.png
  • In General Settings 1 you can change plot title, axis labels and ranges, legend location and other common settings.
  • Adjust the following general settings:
    • Replace the default title 2 with something more specific. This is the main plot title shown on the plot title bar.
    • For testing, it is recommended to set fixed axis ranges 3: \([-0.1, 1.1]\) if you are running with the default parameter preset which generates values in \([0.0, 1.0]\). If you have changed generation parameters, adjust axis ranges accordingly. If you leave axis ranges as default, the plot will determine them automatically and the size of the plotted area can change sometimes.
  • Click b_apply and switch to the next tab.
../_images/page_tutorials_analyze_09_plotconf2.png
  • Tabs like Dataset #0 (default name) 1 contain dataset-specific settings. If a plot supports multiple datasets, each of them creates a “layer” with its own settings for colors, line styles and so on. Plot legends show names of datasets.
  • Adjust the following settings on the Dataset #0 tab:
    • Name: change to all points. This name 2 is shown in the plot legend (and on the dataset tab 1 after you click b_apply).
    • Point marker 3 settings: it is recommended to use medium solid markers of some dark color.
  • Click b_ok to save settings and close the plot configuration dialog.
../_images/page_tutorials_analyze_10_plot1.png

You can now test the plot.

  • Switch to Run and start the workflow.
  • Switch back to Analyze and click b_refresh on the report toolbar to update the plot.
  • Try running the workflow a few times, refreshing the plot after each run. You can change the Sample Size, Lower and Upper Bound values between runs. If you decide to change value range, adjust the axis ranges on plot or clear these settings so the plot determines them automatically.

Note that you did not have to reconfigure the plot’s data source. It still uses data series which update from the <last run> record.

../_images/page_tutorials_analyze_12_ds3.png
  • The <last run> record 1 works as a shortcut.
  • Data series grow in size 2 because they update from <last run>.

Each time you refresh the plot, it changes to display the recent results, but older points are not shown anymore. To keep them on the plot, you can set a fixed run name in Run.

../_images/page_tutorials_analyze_11_runname.png
  • Switch to Run and single-click the Run name field on the Monitoring settings pane 1.
  • This will bring up a Set run name dialog 2. Specify a run name. Run names support placeholders for the current day ``%d`` and time ``%t`` (so the default run name is simply ”%d %t”).
Input a name without placeholders, for example: all points.

Now the workflow data is saved to the same record every time.

Note

Setting a fixed run name enables users to store all outputs to the same record in the project database after each run. If it is not required to store data from previous runs, select Overwrite run history option from the context menu on Monitoring settings pane. The old workflow data will be removed.

On the same plot you can mark a certain batch of points by adding another dataset.

../_images/page_tutorials_analyze_13_ds4.png
  • Select one point batch from last run or all points (for example, Sample[1] 1).
  • Drag it to the Data Series pane. This creates two new data series 2 without adding a new sample viewer to the report. Note that their size is different from the X and Y data series.

Click b_config in the plot title bar to add the new data series to plot’s data sources.

../_images/page_tutorials_analyze_14_plotconfadd.png
  • Click Add dataset 1. A new dataset tab 2 will be added to the window.
  • In the Data Source column 3, select two data series you have just created.
  • Specify a name for this dataset 4 (will appear in plot legend).
  • Configure markers 5: use large circle markers of some bright color.
  • Remove lines connecting point markers 6 (lines are enabled by default, but not needed now).
  • Leave other settings default and сlick b_ok to close the dialog.

The plot now shows all generated points with solid black markers and points from the selected batch are outlined in red.

../_images/page_tutorials_analyze_15_plot2.png

Note that you can delete the project database now, and the plot will not break (report database does not synchronize deletions, see above).

When you run the workflow again, new data will appear under the same name in the project database, and it will be automatically synchronized.

Conclusion

This tutorial concludes the practical introduction to pSeven. If you have completed all introductory tutorials, you will have a small usable project with a simple workflow and a preconfigured report.

To finalize the project, you can clean it up — for example:

  • Clear axis ranges in plot configuration, if you have not done it before. The workflow allows to select any generation range, so it is better to allow the plot to determine its area automatically.
  • Update the workflow description: you can include usage instructions and other details.
  • Add comments explaining the workflow structure, in case you or someone else will edit it later. Comments are added in Edit: select New Annotation from the context menu b_context on the edit toolbar to add an empty note; double-click it to add text. For example, it makes sense to add a note near the RandGen block which explains why one of its inputs is constant.
  • Save the workflow and report you have edited in this tutorial.
  • Remove previous versions of this workflow (wfSimpleWorkflow and wfSimpleWorkflowTool).
  • Clear the project database (assuming it contains only test data). Deleting project database does not break the preconfigured report, and report refresh will work when workflow results become available.

Note that the same technique is used to prepare all examples that are installed with pSeven. If you wish, you can study these examples to learn more about the capabilities found in pSeven, or follow other tutorials which explain advanced features and using pSeven to solve typical engineering tasks.