PythonScript

The PythonScript block integrates a Python script into a workflow. The script is stored in the block and may be either edited directly or imported from a file on disk.

Introduction

PythonScript configuration allows to create block variables which are used to read and write port data in the script. Adding a variable creates a port with the same name — input, output, or a pair of both. Block variables are associated with script variables: their names become the names of global variables in the script’s scope.

The block starts only after it receives values to all input ports. These values are assigned to variables associated with input ports, including those associated with an input-output port pair. A variable that is associated only with an output port is assigned its default value on block start (the default value of a variable is actually a value assigned to the port; see Ports). If no value is assigned to an output port, the associated variable in the script is initialized with None. After the script executes, final values of all variables which have an associated output port are sent to these ports.

Variables associated with ports are reinitialized on every block startup. Other (unassociated) variables are temporary unless you have the Keep globals option enabled — see option description for details.

Python Modules

The PythonScript block does not require you to install Python, because the block runs your scripts using the pSeven’s built-in Python interpreter. It includes a number of modules available for import (see Built-in Modules). If you need to use a module which is not included with pSeven, you can point the block to the directory containing this module to make it available — see Additional Modules.

Built-in Modules

pSeven’s Python distribution includes a number of useful modules, which you can import in your scripts:

  • Scientific computing: numpy, pandas, scipy, sympy, scikit-learn, and networkx.
  • Plotting: bokeh and matplotlib with cycler.
  • PyWin32 modules — see PyWin32 Documentation.
  • Excel support: openpyxl, xlrd and xlwt. Note that pSeven also provides the Excel integration block.
  • Open Database Connectivity (ODBC) support: pyodbc. Note that using this module requires a working ODBC driver and related libraries installed in the system which runs pSeven.
  • Markup and parsing: et_xmlfile, fortranformat, markupsafe, pyparsing, whoosh, and yaml.
  • Networking: paramiko, requests, and tornado with tornado_json.
  • Cryptography: Crypto (PyCrypto) and ecdsa.
  • Various utilities: astor, decorator, and six; psutil and lockfile; dateutil, jdcal, and pytz.

Additional Modules

In addition to the modules available in pSeven by default, PythonScript also supports importing external modules. This is possible for pure Python modules — that is, the module should not include compiled code.

Two steps are required to make an external module available for import in the PythonScript block:

  1. Set the block’s sandbox directory (see section Sandbox for details). If you want a portable project, select a subdirectory inside the project as the sandbox.
  2. Copy the module into the directory you have specified.

PythonScript automatically adds its working directory (which is the sandbox) to sys.path, so after copying your module there, you can import it in the script as usual.

Testing Scripts

You can test a script contained in the PythonScript block before running the workflow to verify that it works correctly. To test the script, use the b_blconf_test button 1 on the Script pane toolbar or the Test script command from the pane’s b_blconf_context menu 2.

Testing a script requires you to specify values on the Variables pane 3. When you run the script, block variables (both input and output) are initialized with these values. If you do not specify a value for some variable, it is initialized with None. Note that specifying a value on the Variables pane also assigns this value to the port associated with the variable (you can see it on the Ports tab). Similarly, changes in port values you make on the Ports tab automatically apply to values on the Variables pane too.

../_images/page_blocks_PythonScript_test_script.png

While the script runs, you can see its console output in the Script output pane 4. To interrupt the script, use the b_terminate button on the Script pane toolbar or the Terminate script command from the pane’s b_blconf_context menu. The Test results pane 5 shows final values of variables after the script finishes. If you interrupt the script or it exits with an error, the Test results pane remains empty.

Note

When testing the script, settings from the Environment and Sandbox tabs apply, but settings from the Options tab are ignored.

Tip

If you want to quickly test a script saved to a .py file, you can run it in Workspace without creating a workflow. This feature is similar to running python my_script.py [arguments] from a command line: double-clicking the .py file on the Filesystem pane in Workspace opens the Run script dialog where you can specify arguments and start the script. Script output is sent to the main pSeven console.

Options

Error handling behavior

Common option that controls the block’s behavior when it encounters an error during execution. See section Error Handling for details.

Keep globals

Controls the lifespan of global variables created (assigned) in the script but not associated with block ports (not defined in the block configuration). If disabled (default): global script variables are deleted and their values are lost when the script finishes. If enabled: global variables are not deleted if they were assigned in the script; their values are kept in the block memory, making them available on the next startup.

Value:Boolean
Default:off

Enable this option if you want to add a steps counter to your Python script:

try:
  step_index += 1
except NameError:
  step_index = 0

Here, on the first step step_index variable does not exist and its usage causes NameError exception. Then it is catched and step_index is initialized with 0. If Keep globals is enabled, step_index variable is preserved between steps, so on the next steps NameError is not thrown and step_index is incremented. Note that this code will not work if there is a block variable named step_index.

Notes

This section answers some specific questions on using the PythonScript block.

Environment Settings

PythonScript supports per-block environment settings. To set up the environment, open the block’s configuration dialog and switch to the Environment tab.

../_images/page_blocks_PythonScript_env.png

For details on using block environment settings, see section Environment.

Opening Files in PythonScript

This question arises when for some reason you do not want to use an absolute file path. For files with absolute paths, PythonScript allows using the Python’s open() builtin. The disadvantage of using an absolute path is that the project becomes not portable — if you move it to another host or another directory, PythonScript may be unable to find the file because its path will be different.

Note

If you are going to create a custom parser or generator (that is, you want to open a text file and do something with its contents), consider using the Text block. This block automates file opening, and supports Python scripting, too.

If you want the project to be portable, then common solutions are:

  • The file is generated by another block, but is not output to a port.

    Typical case is when an external executable launched by a Program block does not accept the output file name as a command line parameter, and its output does not contain the necessary information so redirecting it to a file is of no use. In this situation, you will have to configure both blocks (the one generating the file and the PythonScript block) so that they share the same sandbox — this is done in block configuration dialog on the Sandbox tab. The sandbox is the block’s working directory, so the generated file appears there when the Program block finishes. In the Python script, just use a relative path in open(): script working directory is the PythonScript block’s sandbox, so a relative path is interpreted as a path inside the sandbox. This works also if you add a StringScalar input port to the PythonScript block and send the filename to it.

  • The file has to be created somewhere inside the project directory (or already exists there), and you want to use a path relative to the project.

    In this case, you should add a File block which creates a file object, and send this object to PythonScript. Configure the File block as described in FAQ: How do the files of project origin work with absolute and relative paths?, then add an input port of the File type to your PythonScript block and connect it with the file output port of the File block. See the next case for details on how to work with this file in your script.

  • The file comes from another block’s port.

    First, you need to add a new input port to your PythonScript block. Set its type to File, so the block knows that this port receives a file. Let us assume you have added this port and named it myfile (and linked it to the output port that sends the file, of course). As soon as you add a port to PythonScript, its name becomes an object name in the script scope, so you now have a myfile object in your script. This object is a pSeven file-like: it is somewhat different from the Python’s file-like, but basic methods are the same. For example:

    myfile.open("r")
    data = myfile.read()
    myfile.close()
    
    # the rest of code
    

    Note that the file has to be opened and closed correctly. Also, myfile.open() is a pSeven file method, and the opening mode parameter ("r") is required, unlike the Python’s open() builtin.

NaN Output Values

Sometimes the PythonScript block has to output NaN as the value of some variable, usually to indicate that there was an error during script execution. This feature is typically used when PythonScript calculates responses values for a Design space exploration block (works as a blackbox). There are two methods available:

  1. If you want to use NaN as an error value of an output variable, it is recommended to assign a NaN value to the port which outputs the variable and set the Error handling behavior option to output defaults and signal. With this configuration, when an error occurs in the script, the block will suppress it and output values assigned to ports. The workflow continues to run normally in this case.

  2. If it is not possible to use the Error handling behavior option, you can assign NaN to a variable inside the script. Note that numpy.nan should be used, for example:

    import numpy
    my_var = numpy.nan  # a RealScalar variable
    my_var = numpy.full([5, 2], numpy.nan)  # a RealMatrix variable (5 rows, 2 columns)
                                            # filled with ``NaN`` values
    

Warning

Do not use Python’s float() constructor to get a NaN value. It may lead to errors due to incorrect type conversion.

Old Block Versions

PythonScript was completely redesigned after the pSeven 2.0 release. New version of the block is not compatible with the old one, but pSeven kept compatibility by allowing to load workflows containing the old version of PythonScript. However, the old block in such workflows was not replaced with the new one because it could not be upgraded automatically.

The support for the old version of PythonScript was finally discontinued in pSeven 6.6. Since this version, you can still open workflows containing the old PythonScript block, but they will have to be updated manually.

Note that prior to pSeven 6.6 the old block was always kept intact if it existed in the workflow. Due to this it is possible to have an old block version in your workflow originating from pSeven 2.0 or below, even if you later edited the workflow in a newer pSeven version.

If you open such a workflow in pSeven 6.6 or above, the old version of the PythonScript block will keep the embedded script, but all block ports will be removed (consequently, links to this block will also be lost). The ports and links then have to be restored manually. The workflow will not run until you fix the block: on attempt to start the workflow, pSeven will issue an error message informing you of the situation. pSeven will also add a comment to the block script starting with the following:

# This block was originally created in an old version which is
# no longer supported. It could not be completely upgraded to
# the current version. The original Python script was preserved, but
# block ports were removed and will have to be re-created manually.
# You will also need to restore links to these ports.
#
# Other contents of the workflow are not affected. The workflow will
# run normally after you fix this block.

The comment will then provide instructions on updating the block and list all ports and links that were removed, so you can add them back manually. After fixing the block, you should remove the comment and the line above it (that line raises an exception in order to prevent the broken workflow from running).