Sample-Based Approximation

This tutorial explains how to train an approximation model with ApproxBuilder, given a file with the training data, and save the model to disk.

Before You Begin

This tutorial requires an existing Tutorials project. If you have not created this project yet, see Tutorial Project first.

  • Open the Tutorials project.
  • Create a new empty workflow. If you have never created a workflow before, see the Simple Workflow tutorial.
  • Save the workflow as ApproxTrainSample.
  • Switch to Workspace and verify that the workflow file (ApproxTrainSample.p7wf) is added to the project. You can see it in the Project pane.
  • Switch to Edit and select the workflow you have just created to continue with the tutorial.

Task

The task in this tutorial is to obtain an approximation model using a sample of data containing inputs (values of variables) and responses (function values) of some unknown dependency.

The training sample is a file named approx_train.csv located in the samples subdirectory of the Tutorials project. You can open this file in Workspace to view its contents: open samples in the Project pane and double-click approx_train.csv.

../_images/page_tutorials_approx_sample_01_csv.png

The file contains 100 points sampled from a function we assume to be unknown (in fact, it is the well-known Branin function). Note that both inputs and responses are in the same file: the first two columns are values of variables, and the third column contains function values. The first three lines, beginning with #, are text comments which can be ignored when reading data. The next line (x1,x2,f) is a header containing column names — in this case, the names of variables and responses. These names can be read from the file and saved to the model as the names of its inputs and outputs.

Solution

Training a sample-based approximation model in pSeven can be divided into the following general steps:

  1. Configure a CSVParser block to read the sample data from the file correctly. pSeven does not require a specific file format: for example, the file can use tabs instead of commas to separate fields, use a different character to comment lines, contain no column headers, and so on. CSVParser provides additional settings which allow to parse any text file which represents a data table. This block reads the sample and outputs all data as a single matrix.
  2. Prepare the sample for ApproxBuilder. This block has separate ports for the input and response parts of the training sample, so when all data is contained in the same file, the matrix output by CSVParser has to be split vertically to get two separate samples.
  3. Send the training data to ApproxBuilder. This block trains the model and outputs it to a port by default. In this tutorial, ApproxBuilder is reconfigured to save the model to a file on disk. Note that you can also combine both — for example, if the model is sent to another block in the workflow, but you also want to save its copy after training.

Loading the Sample

Begin with configuring a CSVParser block to load and parse the training sample, approx_train.csv.

  • Add a CSVParser block. Name it Sample.

Open the Sample block configuration. When you load a file, the block displays a results preview (parsed sample) in the Preview pane. Use it to check whether the file is parsed correctly and adjust settings when needed.

../_images/page_tutorials_approx_sample_02_parser.png
  • In the Input file pane, click b_browse and browse to the sample file in your project (samples/approx_train.csv). Note that the block automatically saves a relative path to the file, so it will work even if you move the project to another location.
  • Verify that the file is parsed correctly using the Preview pane. Since samples/approx_train.csv uses the delimiter and comment characters which are default in CSVParser configuration, you do not need to change any settings here.
  • Note the Parse first row as header setting. It should be enabled to read names from the first non-commented line and use them in the workflow. When the names are parsed correctly, you can see them in the Preview pane as column titles.
  • Leave other settings default and click OK.

When the workflow starts, Sample will read the CSV file, convert the data to a RealMatrix and output this matrix to the matrix port.

Additionally, names from the header will be parsed as strings and form a StringVector which is output to the header port.

Preparing the Sample

The matrix output by the Sample block cannot be sent directly to ApproxBuilder because it contains both variable and response (function) values. ApproxBuilder accepts variable and response values to different ports, so the sample should be split into two parts.

Note

ApproxBuilder has separate inputs for variables and responses because in many cases these samples come from different sources.

To separate the input and response values you can use a Submatrix block.

  • Add a Submatrix block. Name it Splitter.

Open the Splitter block configuration. This block allows to define submatrices that include specified rows and columns from the input matrix. To add a submatrix, click b_blconf_add in the toolbar (default configuration is empty).

Select the input part of the training sample.

../_images/page_tutorials_approx_sample_03_submatrix_x.png
  • Specify the name: x_sample.
  • Leave rows default (: selects all rows).
  • Specify columns: 0-1 (the values of variables are first two columns, indexed 0 and 1).

Similarly, select the response part.

../_images/page_tutorials_approx_sample_03_submatrix_f.png
  • Name: f_sample.
  • Rows: : (default, all rows).
  • Columns: 2 (the third column in the file, indexed 2, contains the response values).

Verify the Splitter block configuration.

../_images/page_tutorials_approx_sample_04_splitter.png

Adding a submatrix automatically adds a corresponding output port to the block, so Splitter now has two outputs: x_sample and f_sample, and one default input: matrix which you can see on the Ports tab.

  • Finally, link Sample.matrix to Splitter.matrix.

This completes the sample preparations. Splitter will receive a matrix from Sample, split it according to your submatrix settings, and output the input and response parts to the Splitter.x_sample and Splitter.f_sample ports. Next, these ports have to be connected to ApproxBuilder inputs. Another link has to be created from Sample directly to ApproxBuilder to pass the names of variables (columns) from the file.

Model Training

Approximation models are trained by the ApproxBuilder block. Basically you only need to send training data to ApproxBuilder, other inputs and settings are optional.

  • Add an ApproxBuilder block. Name it Builder.
  • Link Splitter.x_sample to Builder.x_sample, Splitter.f_sample to Builder.f_sample.
  • Link Sample.header to Builder.sample_names.

By default, the trained model is only output to Builder.model. To save the model to disk, select the output file in the Builder block configuration.

  • Open the Builder block configuration.
../_images/page_tutorials_approx_sample_block_config.png

Note that by default the model is trained in the SmartSelection mode 1 which means that pSeven automatically selects and tunes the approximation technique in order to obtain the most accurate approximation model.

  • Click b_browse in the Output model pane 2 to bring up the Configure file dialog.
../_images/page_tutorials_approx_sample_05_modelfile.png
  • In the File origin pane select the Project origin.
  • In the File path field, input ./model.gtapprox.
  • Leave other settings default and click OK to close the dialog.

As a result, the saved model (model.gtapprox) will be found in the project directory after the workflow finishes. A path that begins with ./ is a relative path where . represents the project directory. For more details on this file configuration, you may see FAQ: How do the files of project origin work with absolute and relative paths?.

The model is saved to a binary format native to pSeven. To evaluate the model, you can then load it into an Approximation model block and send it an input sample to get predictions on function values. If you want to evaluate the model out of pSeven, you can also export it after training, using an Approximation model block.

Model Export

Note

This part is optional. Using exported models is out of the scope of this tutorial. The instructions below only explain how to add the model export step to a workflow.

pSeven supports exporting approximation models to various formats. There are several export functions available:

  • You can use the Approximation model block to export a model from the workflow automatically. See section Model Export for more details on available export formats.
  • It is also possible to set up model export directly from an ApproxBuilder block (Builder), although it is less convenient than using an Approximation model block. See the ApproxBuilder block documentation for details.
  • You can manually export your model in Analyze. See section Model Import and Export for details.

This section explains how to automate model export from a workflow using the recommended configuration.

  • Add an Approximation model block. Name it Export.

The Approximation model block is mostly used to evaluate models which are loaded from disk. However, it can also accept a model to the Model file input. Note that despite you have reconfigured Builder to save the model to disk, the Builder.model output is not disabled, so you can send the model from Builder directly to Export.

  • Connect the Builder block to Export. Although there are no exact matches in port names, pSeven will suggest creating a link from Builder.model to Export.Model file, displaying the Autolink dialog.
../_images/page_tutorials_approx_sample_links.png
  • Click OK to accept the suggested link and close the dialog.

Note

Loading the model from a file would require explicit synchronization between Builder and Export. That is, Export should start only after Builder finishes and writes the model to disk, otherwise Export will not be able to find the model file. Block synchronization is easy in fact: if you connect Builder.done to Export.do, it will make Export to wait until Builder finishes. You can test this version of the workflow later.

The Export block now receives a model from Builder, but export is not configured yet.

  • Double-click the Export block to open its configuration. On the Configuration tab, switch to Advanced. Note that there is no model loaded at the moment, so the configuration dialog does not show any model details. This block will receive a model only when you run the workflow.
  • Use the Export to selector to specify the export format — for example, select Octave (MATLAB-compatible script). Note that multiple formats can be selected, and the block will export to all selected formats at once.
../_images/page_tutorials_approx_sample_06_exportfile.png

When you select the format, a few export-related ports are added to the block. For each export format, there will be 3 inputs for additional export settings and 1 port which outputs a file with the exported model in this format.

  • Switch to the Ports tab to see the new ports.

Ports are named after the export format they are related to. For example, the ports for the Octave format are:

  • Model export.Octave script.Function description input: optional, can be used to add a comment describing the model to the exported source code.

  • Model export.Octave script.Function name input: optional, can be used to change the name of the model function in the exported code. If you leave this empty, function name will be exportedModel (default).

  • Model export.Octave script.Path input: sets the path where to save the exported model file on disk. By default, the path is ./Model.fmt, so the model is exported to the current project directory under the name “Model”, and .fmt is the format-specific file extension.

    The export path can be relative to the project directory as shown, or absolute to save the exported code outside the project. If the specified file already exists, it will be replaced. If you input a path leading to a directory which does not exist, this directory will be automatically created by the block.

  • Model export.Octave script output: outputs a temporary file with the exported model. Note that this port is not used to save the exported model to disk: it is intended to send the model code to another block in the workflow.

../_images/page_tutorials_approx_sample_ports.png

Finalize your export settings:

  • Set Model export.Octave script.Path to "./gtapprox_model.m" (.m is the extension of MATLAB scripts). This will export the model code to the gtapprox_model.m file in the project directory.
  • Optionally, change the model function name and add a description. For example:
    • Model export.Octave script.Function description: "Model from the Sample-Based Approximation tutorial.".
    • Model export.Octave script.Function name: "tutorialModel".
  • Click OK in the Export block configuration to save settings.

Workflow

Finished workflow is a typical representation of a sample-based approximation task.

../_images/page_tutorials_approx_sample_07_wf.png

The Sample and Splitter blocks are used to prepare the training sample and metadata (names of variables) for Builder. When Builder receives the prepared input and response samples from Splitter and names from Sample, it starts model training. When it is completed, Builder outputs the model to its model output and saves its copy to model.gtapprox in the project directory. These models have a binary format specific to pSeven. Optionally, the model from Builder is also sent to the Export block which exports it to one of the common source code formats. The exported code can be used in any other application.

  • Save the workflow and run it to see the results.

Results

The main result of this tutorial is the approximation model (model.gtapprox) saved to the Tutorials project directory in the pSeven binary format. After the workflow finishes, you can switch to Workspace to see this file in the Project pane.

  • Verify that model.gtapprox exists after running the workflow. Note that this model is required in the Model Evaluation tutorial.

If you have completed the optional Model Export part, you will also find the gtapprox_model.m file in the project directory. This is a text file containing the model code in the Octave format. To view the code, you can open this file in Workspace.

../_images/page_tutorials_approx_sample_08_mfile.png

Here you can see the function name 1 which you have specified as Model export.Octave script.Function name. The function is preceded by an automatically generated comment 2 explaining its usage.

Conclusion

This tutorial shows only the basics of training approximation models in pSeven. It does not discuss advanced ApproxBuilder options which allow controlling the quality of approximation and can have a noticeable effect on model performance and behavior. For more details, you can see the ApproxBuilder block page, in particular section Options which details various settings available in ApproxBuilder.

The model trained by ApproxBuilder can be used in pSeven to predict function values at points not found in the training sample. However note that values of variables should be within the range which was present in the training data — the model’s bounding box. The model has no information about the input-output dependency outside the bounding box, because the training data did not contain this information. Due to this, its predictions for inputs outside the training sample bounds are very inaccurate; in brief, the model tends to linear extrapolation. The Model Evaluation tutorial explains how to find the bounding box of a model and generate an input sample inside the bounds.

Depending on the task, approximation models are either saved to disk or sent directly to an Approximation model block for evaluation. This tutorial suggests saving the model to a file (model.gtapprox) so it can be used in tutorials later. In your tasks, you can run the model in the same workflow where it is trained by sending it to a Approximation model block which is configured to evaluate the model. You can even reconfigure the Export block in this tutorial to both evaluate and export the model in a single run.