Uncertainty quantification

The Uncertainty quantification block performs sensitivity and reliability analysis studies. It is based on the OpenTURNS methodology. Supported methods are:

  • Simulation: the block generates randomized inputs for the studied model and analyzes its responses to estimate their statistical characteristics. Optionally it can also fit each output to known distributions, finding the distribution parameters and the best matching distribution. Input values can be either taken from parametric distributions which you specify in the block’s configuration, or generated using an input sample which the block automatically fits to some distribution, and then takes values from this internally defined distribution.

  • Reliability analysis: the block generates inputs for the model and analyzes responses, which have bounds specified in the block’s configuration. The main result of this study is a reliability estimate which is the probability that the value of a response is within its specified bounds (in case of multiple responses, estimates probability that all responses satisfy bounds simultaneously). That is, the estimate shows how robust or reliable the model is when it receives inputs from a certain range. The range is defined, implicitly or explicitly, by the distribution you specify for each input, or by the given input sample.

    Reliability analysis may use the following methods:

    • Sampling: the direct sampling method which generally works as described above. Input generation can be random (Monte-Carlo sampling) or use values taken from isoprobabilistic intervals (LHS sampling).
    • FORM: the first order reliability method. This method performs blackbox sampling but uses the collected data to create an internal probability model. The internal model is then used to estimate reliability.

To perform analysis, the Uncertainty quantification block has to be connected to a blackbox block — the block which evaluates responses of some computational model. Basic configuration is:

  • Add variables and responses to the Uncertainty quantification block configuration.
  • For each variable, the block adds an output port named Blackbox.<variable_name>. Connect these ports to your blackbox inputs.
  • For each response, the block adds an input port named Blackbox.<response_name>. Connect output ports of your blackbox to these inputs.
  • Select the analysis method. Note that the Uncertainty quantification block supports SmartSelection, so by default it will select an appropriate method automatically based on properties of variables and responses.
  • Set Number of designs. This is the maximum number of design points which the block is allowed to generate during analysis, or the maximum number of model evaluations (blackbox calls).

The block also provides additional input ports which you can use to set properties of variables and responses, instead of specifying these properties in the configuration dialog.

For variables, the <variable_name>.Sample input ports accept samples with variable values. Note that when you send a sample for some variable, the block ignores the distribution which you set for the variable in its configuration, and always derives the distribution from the sample.

Main ports which output analysis results are the following:

  • Report — outputs an analysis report in the HTML format.
  • All designs — outputs all data collected during the analysis as a single matrix. Each matrix row represents a single design point. Matrix columns contain values of variables and responses sorted in the same order as they are listed on the Variables and Responses panes in block’s configuration (variables first). Variables and responses with Size 2 or more occupy multiple columns.
  • Variable designs and Response designs — similar to All designs, but output the variables and responses data as two separate matrices.

Monitoring is enabled by default for the main result ports. To view the block’s report, which is output to the Report port, you can use the Page viewer in Analyze:

  1. Create a new report and open the Report database pane.
  2. Drag the record with the report from the Project database pane to the Data series pane in the report database. This creates a string data series, which usually contains 1 element — a long string with the HTML code.
  3. Select this new data series and click the Page viewer button on the report toolbar to load the report into a new viewer.

To export the report as HTML, you can either use the Export page… command in a Page viewer, or configure the block to save the report to a file on disk by specifying the export path to the Report path port. The path can be relative to the project directory — for example, ./results/My report.html saves the file to the results subdirectory in the project. Absolute paths can be used to save the report outside the project. In both cases, if the specified file already exists, it will be replaced.

In addition to the main ports, there are the following outputs:

  • Reliability estimate — outputs the reliability estimate value, if the block runs in the reliability analysis mode.
  • Confidence interval — outputs the confidence interval for the above estimate as a RealVector with 2 values. The confidence level used when calculating the interval is set by the Confidence level option.
  • Result — outputs a Dict containing all result data so it can be post-processed in various blocks, for example PythonScript.

The additional ports listed above are always present on the block, but monitoring is disabled for them by default. To enable it, use the Ports and parameters dialog.

Known Issues

This section describes known issues in the Uncertainty quantification block.

Reliability Analysis (FORM)

The Reliability analysis (FORM) method sometimes does not respect the Number of designs setting and performs a few more evaluations when it should. This is a bug in the OpenTURNS library used by the block, and will be fixed in future versions.

Column Order in Results

In pSeven 6.16 and older versions, a bug in the block configuration dialog could invalidate the configuration in some cases — for example, when you reordered variables, then deleted one of them and created a new one with the same name. Due to the bug, pSeven did not correctly save the order of variables and responses, making their order shown on the Variables and Responses panes inconsistent with the column order in result samples. This led to various obscure issues in results, such as “mixed” columns in the matrix output to All designs, and also caused incorrect behavior of the variable and response reordering commands (the b_blconf_up and b_blconf_down buttons).

This bug is fixed since pSeven 6.16.1, but workflows created in pSeven 6.16 and older can still contain a block saved with invalid configuration. When you open such a workflow in pSeven 6.16.1 or newer, you will get validation errors in the Issues pane, like the following:

The order of variables may be inconsistent due to a configuration bug. Reopen the block to resolve the issue.
The order of responses may be inconsistent due to a configuration bug. Reopen the block to resolve the issue.

The workflow cannot run unless these errors are fixed. To remove the errors:

  1. Open the block which reports the error — see its name in the Source column in the Issues pane.

  2. When you open the block, an error dialog will pop up:

    ../_images/page_blocks_common_indexbug_message.png

    The error resolution dialog shown when you open a block with errors in variable or response configuration. pSeven can automatically fix these errors, but you will have to verify configuration afterwards, as the order of variables or responses may change.

  3. Click Resolve in the error dialog to fix the block. Note that the order of variables or responses may change after this: unfortunately, it is not possible to restore the correct order from the invalid configuration.

    • If you want to check the configuration without any changes, click Cancel in the error resolution dialog.
  4. After pSeven resolves errors, verify the order of variables and responses in the block configuration dialog, and make your adjustments. Usually there is a problem with one variable or response, and it is automatically moved to the bottom position on the pane.

  5. Click OK or Apply to save the fixed configuration, then save the workflow.

Note that clicking Cancel in the error resolution dialog or the block configuration dialog keeps the current (invalid) configuration. It does not remove the validation error, so pSeven will continue to show the error in the Issues pane and the block’s validation pane. No changes are done unless you select Resolve and save the block afterwards, so you can review the issue and make backup copies of the block or workflow before allowing pSeven to fix errors.