Skip to content

Changelog

v2022.11

Updates

  • Design space exploration block (beta):

    • New optimization technique - Gradient-free optimization, which can provide better quality solutions in tasks with many discrete variables. The technique uses the Nevergrad solver by Meta AI Research.

      The Gradient-free optimization technique is not enabled in SmartSelection yet, so you have to select it manually to try it.

    • If you close a block configuration dialog with unsaved changes, you get a related warning.

  • Updated the Python runtime environment version to 3.9 and updated versions of pre-installed Python packages. This version of Python is used in Python script blocks, user blocks, and on Windows extension nodes.

  • Updated the top navigation bar in pSeven Enterprise Studio:
    • You can go from Studio to AppsHub using the dropdown menu at the left side of the bar.
    • Added a separate Help menu with links to the Getting started page and changelogs.
  • Updated and fixed the list editor in the Edit value dialog.
  • Upgraded the embedded pSeven Core package to version 6.39. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • If you are logged into pSeven Enterprise Studio, you can use the new dropdown menu in the top navigation bar at its left to go to the admin interface.
  • Updated the pSeven Enterprise deployment guide with details on using a reverse proxy to secure the deployment - see section Reverse proxy in Prerequisites and updated deployment diagrams.

User block development

  • Fixed a regression issue from v2022.10 where a user block that does not handle the shutdown request from pSeven Enterprise caused incorrect workflow run behavior (the run never finished). Now, if a block did not stop within 1 minute after receiving the shutdown request, it is unconditionally forced to stop.

    Note that pSeven Enterprise will not allow any block to process the shutdown request longer than 1 minute. When implementing the shutdown request handler, you will have to make sure it can finish its job before that timeout expires.

  • Fixed an issue where pSeven Enterprise did not set the idle timeout value specified in the block manifest, when you add the block to the workflow.

Bugfixes

  • Fixed a regression issue from v2022.10 where you could not set workflow parameter values after you have created a run in the "Configuration" state.
  • Fixed an issue where you were not redirected to the sign-in page after you have logged out.
  • Fixed an issue with incorrect workflow run behavior after you rename the folder that contains your workflow or the run directory. Now you get a warning dialog when you try to rename a folder that contains a workflow run in progress.
  • Fixed an issue where adding a .p7history file to a report caused an error.
  • Fixed issues with the Runs sidebar in AppsHub:
    • If you delete an ongoing app run, it was stopped but not deleted, so you had to click again.
    • The sidebar did not show all existing app runs (created by any app user) to the app owner.
  • Fixed an issue with editing Enumeration type values where the Edit value dialog required you to input the value manually instead of selecting it from a predefined list as intended.
  • Fixed various minor UI issues in the Run setup pane and the Edit value dialog.

v2022.10

Updates

  • Design space exploration block (beta):

    • Added optional per-variable and per-response result ports, which you can use to get the result values for specific input and output components of your task.
    • Added optional per-component initial sample ports.
    • Enable those ports in the Ports and parameters dialog in the block configuration.
    Optional per-component ports in the Ports and parameters dialog

  • Python script block:

    • You can now rename request (argument) and response (function) ports - for example, to display human-readable names in the Block properties pane and the Links dialog, instead of Python variable identifiers.

      Port name settings in Python script

    • Added the code autoformatting toggle in the Configuration pane, which you can use to disable the default formatting if you find that more convenient.

      Code autoformatting toggle in Python script

    • Added the support for missing (N/A) values: it is now possible to pass arrays with missing values through ports, and pass N/A or None as a value of a float (Real), int (Integer), or str (String) variable.

  • You can copy and paste table and matrix data between the Edit value dialog and Excel or another spreadsheet editor that uses a similar clipboard format (tab-delimited tabular data).

  • Added the support for tables and matrices with missing (N/A) values.
  • Added the support for string matrices.
  • Various small improvements in the table and matrix editors in the Edit value dialog.
  • Minor UI improvements in the Condition block.
  • pSeven Enterprise is now case insensitive when checking names for uniqueness (workflows, reports, blocks, ports, and so on) - for example, you cannot add two inputs named Force and force to the same block, because those names are considered identical. This helps in avoiding various usability issues related to case-sensitive name validation.
  • Upgraded the embedded pSeven Core package to version 6.38. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • Added experimental support for NFSv4 file storages. Disabling NFSv4 on the file storage NFS server is now optional.

    If you are going to enable NFSv4 on the file storage NFS server during an upgrade of an existing deployment (which currently uses NFSv3), you will have to deploy with the storage check enabled (the -⁠-⁠set fixStorages=true option). See Enabling NFSv4 in the pSeven Enterprise upgrade notice below for details.

  • Updated pSeven Enterprise for compatibility with Kubernetes versions 1.15.5 through 1.25.2, removing the "batch/v1beta1 CronJob is deprecated" warning that appeared during deployment on Kubernetes 1.21 and newer versions.

  • To reduce some of the security risks related to allowing users to connect to a pSeven Enterprise deployment directly, all X-Forwarded-* headers in HTTP requests are now ignored by default.

    If your deployment is protected by a trusted reverse proxy, it is advisable to enable parsing the X⁠-⁠Forwarded⁠-⁠* headers in the deployment configuration file values.yaml. For details, see Deployment configuration in the pSeven Enterprise upgrade notice below.

  • Improved stability by resolving several memory cache issues that could lead to restarts and unavailability of various pSeven Enterprise services.

  • The SSH key for extension nodes is now stored in the deployment configuration file values.yaml (the new extensionNode.blockSSHGateClientKey parameter) and is saved on each extension node during its deployment, which slightly improves startup times of workflow blocks running on extension nodes.

    When preparing the values.yaml file for pSeven Enterprise v2022.10, you will have to generate the extension node SSH key and add it to values.yaml. For instructions, see Deployment configuration in the pSeven Enterprise upgrade notice below.

  • In addition to the custom logos and Getting started page, you can set a custom favicon for pSeven Enterprise - see section Branding for details.

    There are a few changes in the branding file structure in pSeven Enterprise v2022.10. For differences, see Branding in the pSeven Enterprise upgrade notice below.

  • In user workflow and app runs that complete without errors, pSeven Enterprise performs a log files cleanup, so successful runs take slightly less disk space than in previous versions.

  • Fixed a compatibility issue with LDAP over SSL (LDAPS), which caused the sun.security.provider.certpath.SunCertPathBuilderException error during setup of user account federation with a user identity storage provider using LDAPS.

    Using LDAPS requires importing the public key of the LDAP server into pSeven Enterprise, which is easily done during the version upgrade to v2022.10. For instructions, see Using LDAPS in the pSeven Enterprise upgrade notice below.

  • Fixed a compatibility issue with the PgBouncer database connection pooler, which led to malfunction if your PostgreSQL database provider uses PgBouncer.

  • Fixed an issue where a user who runs a workflow that contains a block with a high CPU limit could leave pSeven Enterprise unusable for other users because their workflows could not start until that workflow containing a CPU-intensive block is stopped or a 10 minute timeout is exceeded.
  • Fixed a regression issue with pSeven Enterprise branding from v2022.07, where your custom logo did not apply to the pSeven Enterprise sign-in page.
  • Various updates and fixes in the pSeven Enterprise deployment and administration guides.

User block development

  • Fixed an issue with user blocks and the REST API where you could not start a workflow run using the REST API commands if that workflow contains a user block with an empty description of some input port, and that port is a workflow parameter.

Bugfixes

  • Fixed an issue with the Design space exploration block (beta) where it did not stop as intended when receiving an invalid response from a blackbox block - for example, a matrix with incorrect number of columns.
  • Fixed an issue with the table and matrix editors in the Edit value dialog where the button was usable only to delete the bottom row.
  • Fixed an AppsHub issue where you could not get the workflow from an app due to incorrect workflow name validation.
  • Fixed a usability issue with the Explorer pane when viewing the Runs or Files special folders that contain files or subfolders with long names.
  • Fixed an issue where you could not use slashes (/ or \) in the name of a composite block.
  • Fixed a usability issue where you could not delete a link from a workflow using the Remove command from the menu in the workflow edit toolbar.
  • Fixed a minor UI issue with the Explorer pane navigation bar, which displayed pSeven Enterprise reports as common folders.

pSeven Enterprise upgrade notice

pSeven Enterprise v2022.10 features a number of updates that may require additional preparation steps for the v2022.10 upgrade, depending on your current configuration. The summary of those steps is as follows:

  • In values.yaml, set the new entrypoint.usingTrustedReverseProxy parameter.
  • Generate a new SSH key for Windows extension nodes and add it to values.yaml as the new extensionNode.blockSSHGateClientKey parameter.
  • If you are going to connect pSeven Enterprise with a LDAP server using LDAPS, place the server public key certificate into the pSeven Enterprise shared data storage so it is imported during the upgrade.
  • If your deployment is branded, update the branding file structure in the shared data storage.
  • If you are going to use NFSv4 (experimental), run deployment with the -⁠-⁠set fixStorages=true option.

Each of the additional steps is described in detail in paragraphs below. Note that they are mostly done while you prepare for the upgrade, prior to running the installation. The descriptions below supplement the general updating instructions found in section Version upgrade.

Deployment configuration

When preparing the deployment configuration file values.yaml for pSeven Enterprise v2022.10, note the new entrypoint.usingTrustedReverseProxy and extensionNode.blockSSHGateClientKey parameters:

  1. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.
  2. If your deployment is protected by a trusted reverse proxy that sets the X⁠-⁠Forwarded⁠-⁠* headers for user HTTP requests, and users cannot establish HTTP connections directly to pSeven Enterprise, set the entrypoint.usingTrustedReverseProxy parameter in the prepared values.yaml file to true.

    With that setting, pSeven Enterprise will consider the X⁠-⁠Forwarded⁠-⁠* HTTP headers to be trustworthy, and will use the header information for identification of clients (users). Parsing the the X⁠-⁠Forwarded⁠-⁠* headers is disabled by default as a safe setting, and should be enabled only if your pSeven Enterprise deployment is accessible to users through a trusted reverse proxy, and not also accessible directly.

  3. The prepared values.yaml file also contains a new extensionNode.blockSSHGateClientKey parameter. This key is required to establish connections with Windows extension nodes and must be specified in values.yaml.

    1. Generate a new file with pSeven Enterprise v2022.10 secrets using the following command (replace <version> with the full version number):

      docker run --rm -it docker.pseven.datadvance.net/pseven-secretgen:<version> > pseven-secrets-2022-10-extnodessh.txt
      
    2. From the generated pseven-secrets-2022-10-extnodessh.txt, copy only the extensionNode.blockSSHGateClientKey parameter to the values.yaml file you prepare for the upgrade deployment. Do not change other secrets in values.yaml (the jobcluster.* and extensionNode.* parameters, except extensionNode.blockSSHGateClientKey) - those must be copied from the existing configuration, as described in Upgrading pSeven Enterprise.

    3. Also copy the extensionNode.blockSSHGateClientKey parameter to your backup copy of pSeven Enterprise secrets.
    4. Remove the generated pseven-secrets-2022-10-extnodessh.txt file.

Using LDAPS

pSeven Enterprise v2022.10 supports connecting to user identity storage providers using LDAP over SSL (LDAPS), which requires importing the public key of the LDAP server. The easiest way to import that key is to place the public key certificate of the LDAP server into the pSeven Enterprise shared data storage before you run the deployment:

  1. Obtain the public key certificate from the LDAP server. The certificate must be exported to a DER or Base-64 encoded X.509 file. Request the certificate file from your LDAP server administrator.
  2. Locate the pSeven Enterprise shared data storage (see the storage.shareddata.nfs.* parameters in values.yaml). Create a directory named certs in the shared data storage directory specified by the storage.shareddata.nfs.path parameter.
  3. Rename the certificate file to ldaps.cert and copy it to the certs directory. The expected path to the ldaps.cert file is <shared data>/certs/ldaps.cert, where <shared data> is the path specified by the storage.shareddata.nfs.path parameter in values.yaml.

When you run the deployment, pSeven Enterprise will import the key from <shared data>/certs/ldaps.cert. Now when you set up the user identity storage provider (see section Provider setup in the administration guide), you can specify the ldaps:// prefix in the Connection URL field to use LDAPS.

Branding

There are a few changes in the branding file structure in pSeven Enterprise v2022.10. The Branding section describes the current structure; compared to previous versions, the differences are:

  • The logo-100x266.png file, as well as the <shared data>/branding/auth/ folder, are no longer used.
  • All logo customizations should now be placed into the <shared data>/branding/pseven/ folder.
  • The logo for the sign-in screen is now named block-logo-200x530.png. For best results, this logo image should be 200×530 pixels.
  • There is an additional logo used in navigation bars, named header-logo-64x64.png. This logo image should be 64×64 pixels.
  • If you want to use a custom favicon, place your favicon.ico file into the <shared data>/branding/pseven/ folder alongside the custom logos.

Enabling NFSv4

pSeven Enterprise versions prior to v2022.10 are compatible only with NFSv3 file storage servers. The experimental NFSv4 support is added only in v2022.10, so all existing deployments of prior versions are configured to use NFSv3 only, and NFSv4 is disabled on the NFS server.

If you want to start using NFSv4, enable the NFSv4 protocol on your file storage server before deploying the v2022.10 upgrade, and run deployment with the -⁠-⁠set fixStorages=true option:

  1. Before you run the deployment, revert the changes described in section Disabling NFSv4 of the pSeven Enterprise deployment guide. Use the test described there to verify that NFSv4 is now enabled:

    1. Create a test directory and try mounting it using NFSv4.
    2. Verify that a command like the following runs without errors:

      $ sudo mount -t nfs -o vers=4 localhost:<test directory path> <mountpoint>
      
    3. Additionally you can run the mount | grep <test directory name> command to check that the test directory is mounted using NFSv4: look for the mountvers option in the command output, it should read mountvers=4.

  2. When running the v2022.10 deployment, enable the storage check by adding the -⁠-⁠set fixStorages=true option to the helm install command, as noted in the upgrade guide - see the command example in section Upgrading pSeven Enterprise. Note that the storage check may take some time depending on the amount of storage data.

  3. Once the deployment finalization message appears, verify that the storage check has run. The message should contain the following note:

    Fixing storages enabled: deployment was run with the "--set fixStorages=true" option, fixing user permissions in data storages.

v2022.09

Updates

  • pSeven Enterprise can email you when your workflow run completes or interrupts. Enable mail notifications in the pSeven Enterprise user menu.

    Notifications toggle in the user menu

  • The Share dialog supports username autocompletion so you no longer have to input full names or email addresses of users with whom you want to share your files.

    Autocompletion suggestions in the Share dialog

    Autocompletion settings are managed by your administrators, who may decide to disable username suggestions for security reasons.

  • You can specify the following block run settings in the Block properties pane:

    Block run settings in the Block properties pane

    • Memory limit (MB) - the amount of memory reserved for a block. By specifying a higher memory limit than default, you can request more memory for a block that has high memory consumption.
    • Idle timeout (sec.) - the timeout (in seconds) after which a ready block, which captures runtime resources and waits for input data, is stopped by pSeven Enterprise to free up the resources. In particular, you can set the timeout to NEVER to keep a block running until the workflow run stops - this is useful in workflows that run as a service, where certain blocks have to run continuously in order to improve the service response times.

    Block run settings are always editable, though they have effect only if the workflow runs on a pSeven Enterprise deployment where administrators have enabled the resource management features. This may be useful if you edit and configure your workflow on a test deployment with resource management disabled and then move the ready workflow to another, working deployment where resource management is enabled.

  • The Workflow contents pane on a workflow tab provides an overview of memory limits you have set for workflow blocks.

    Block memory limits overview in the Workflow contents pane

  • Updated and fixed the matrix editor in the Edit value dialog: that editor now provides all matrix editing functions that were available in previous versions (fill, transpose, and so on).

  • When you create a run of a read-only shared workflow and select a folder where to create the run directory, the default location is the folder you currently have open in Explorer. In previous versions the default location was the root of your home folder, which is often inconvenient.
  • Upgraded the embedded pSeven Core package to version 6.37. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • Since v2022.09, pSeven Enterprise supports integration with Active Directory and LDAP services - see section Users from Active Directory or LDAP server in the administration guide for instructions.

    If you want to enable Active Directory or LDAP integration during an upgrade of an existing deployment, where users already have accounts created by means of the pSeven Enterprise admin interface, you will have to perform account matching as a part of that setup in order to prevent users from losing access to their data. For instructions, see the pSeven Enterprise upgrade notice below.

  • You can set up single sign-on (SSO) for pSeven Enterprise users, integrating with an external OpenID provider, such as a Keycloak-based identity provider. See section Setting up single sign-on (SSO) in the administration guide for instructions.

  • You can enable username and email autocompletion in pSeven Enterprise UI - for example, in the Share dialog, so the user who shares a folder to others will not have to input their full usernames or email addresses. Autocompletion is disabled by default; to enable, set the exposeUsersInfo parameter in the deployment configuration file values.yaml to true when upgrading your deployment to pSeven Enterprise v2022.09.

    Enabling autocompletion is a security risk

    A malicious user can exploit autocompletion to gather information about names and email addresses of other users. It is strongly recommended to disable this feature in public deployments, and enable it only if you are deploying pSeven Enterprise in a private environment that allows user contact sharing.

  • Fixed an issue with blocks configured to run on a Windows extension node - such a block was unable to launch a software package installed on the node, if that package requires certain environment variables (a typical requirement of various engineering software). Note that in order to avoid this issue, an additional node configuration step is needed - you must add all required variables to the node's block runtime environment as described in section Engineering software on extension nodes of the administration guide.

  • To avoid stability issues when many users actively edit blocks (open block configuration windows) and run workflows, the block edit sessions are now handled by the new appworkerblockjobs service while workflow run tasks are handled by the appworkerworkflowjobs service. Previously tasks of both kinds were handled by appworkerworkflowjobs, which caused issues for users - in particular, when a user cannot start any workflow run because other users have many block configuration windows open, thus overloading the appworkerworkflowjobs service so it cannot start new workflow run tasks.
  • Improved run resource management for Windows extension nodes to avoid issues caused by a high number of blocks running at the same time on a given Windows node.
  • Fixed a regression issue from v2022.07, which caused incompatibility with certain PostgreSQL providers that use PgBouncer as the database connection pooler: pSeven Enterprise deployment stopped with an error when initializing the Keycloak database (executing the initkeycloakdb Helm hook).
  • Fixed an issue where a < or > character in the database password (the postgres.password parameter value in values.yaml) caused a deployment error.
  • Fixed an issue with run resource management where, if there are insufficient free resources to start a workflow run, that run started and immediately failed with an "Insufficient resources to run the workflow" error, instead of being put into the run queue as intended.
  • Updated the pSeven Enterprise administration guide:
  • Updated the service interaction diagram in the pSeven Enterprise deployment guide: added the new appworkerblockjobs service and changed the layout to make the relationships between diagram elements more clear.

Bugfixes

  • Fixed a regression issue from v2022.08 where workflow runs failed with "The following IDs are unknown" or "Locked document must be a directory" errors.
  • Fixed an issue where a prolonged workflow run created too many temporary files and auxiliary logs, due to which it occupied much disk space and caused performance and responsiveness issues when you navigate that run's results in Explorer or view run logs.
  • Fixed a regression issue from v2022.06 where a composite block with batch mode ports did not actually enable batch processing for arrays and lists sent to those ports, if the @Number of parallel workers setting is 1.
  • Fixed a regression issue from v2022.08 where uploading a file using drag and drop caused incorrect behavior of the Explorer pane.
  • Fixed a regression issue from v2022.08 where the AppsHub Runs sidebar was displayed when you open any HTML file from the Explorer pane.
  • Fixed an issue with the pSeven Enterprise load balancing system that caused confusing behavior: if other users have many block configuration windows open at the same time, you could not start any workflow run with no apparent reason. As a part of that fix, pSeven Enterprise now limits the number of block configuration windows you can open at once, and keeping them open does not affect workflow runs.
  • Fixed a regression issue from v2022.08 where pSeven Enterprise could not open a workflow run in a tab and displayed a "variable referenced before assignment" error.
  • Fixed a REST API regression issue from v2022.06: with some workflows, the response to a correct PATCH request to set workflow run parameters returned was an Internal server error (status code 500).
  • Fixed a number of issues with the Design space exploration block (beta):
    • Inconvenient data types used by request ports, which send values of variables to blackbox blocks for response evaluation.
    • Incorrect behavior when you configure a space-filling DoE technique and disable batch mode in run options: when sending generated DoE points to blackbox blocks evaluating responses, the Design space exploration block output all points as a single batch instead of the expected sequential output mode.
    • Incorrect behavior of the problem definition ports (for example, variable bounds ports): if you assign a value to such a port and connect a link to it, the block always used the assigned value while it should wait for and use a value received from the link.
    • An error when saving a block with a valid initial sample, which contains NaN or missing (N/A) response values.
    • Minor UI issue when you set the Maximum batch size option and the Gradient-based optimization technique is selected - that option does not have any effect in this case as the technique does not provide the support for real concurrency in response evaluations, but the block UI did not make it clear.
  • Fixed an issue where a workflow run continued indefinitely if the workflow contains a composite block with a batch mode port, and a value is assigned to that port.
  • Fixed an issue with the Python script block that sometimes caused a "block init was called more than once" error when starting a workflow run.
  • Fixed UI responsiveness issues in the Block properties and Composite properties panes.
  • Fixed a convenience issue with editing input values in block properties and Run setup panes, where a data editor window always popped up even if the value is a plain scalar number or a string.
  • Fixed an incomprehensible text in the error message that appears when you undo changes in a workflow but pSeven Enterprise cannot apply the undo immediately.
  • Fixed several minor UI issues in the Explorer, Block properties, and Composite properties panes.

pSeven Enterprise upgrade notice

Since version v2022.09, users can sign in to pSeven Enterprise with their accounts from an external storage such as Active Directory Domain Services or an LDAP server. To enable this feature after upgrading pSeven Enterprise, user account federation must be set up and run as described in Users from Active Directory or LDAP server. As a result, for each user account from the external storage that matches the federation settings, the associated account of the pSeven Enterprise user will be created. If any pSeven Enterprise users have previously created non-federated accounts, you must link their new federated accounts to existing authorization accounts as described below. Otherwise, upon the first sign-in with the new federated account, those users will get new authorization accounts that consume the pSeven Enterprise license and do not provide access to the existing user data associated with the old authorization accounts.

Account matching

Running federation results in creating new accounts in the pSeven Enterprise authentication service. Upon the first sign-in to pSeven Enterprise, an authorization account is created for the user account, which provides access to the user data such as their home directory, files, workflows, and apps. If the user already has an account created by means of the pSeven Enterprise admin interface (see Adding accounts), you must link the respective authorization account to the new user account, created by running federation. This ensures that having signed in with the new account, the user can access their data in pSeven Enterprise.

The authorization account of the user in question is initially linked to their old account, which was created by means of the pSeven Enterprise admin interface. Before allowing the user to sign in to pSeven Enterprise with their new account, you must complete the following steps:

  1. Disable the old account of that user in the pSeven Enterprise authentication service.
  2. Link the authorization account of that user to the new user account, which federation has created for that user.

Follow the instructions below to complete each of these steps.

To disable the old account in the pSeven Enterprise authentication service:

  1. Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
  2. Look up the old account: on the Users page of the authentication service interface that appears, enter the username or a part of the username in the Search box, and then press Enter.
  3. In the lookup results list, click the ID of the old account to open its edit page.
  4. On the Details tab of the old account edit page, turn the User Enabled switch OFF and click Save.

Copy and save the username provided in the Username field on the Details tab of the old account edit page. This username will be required to locate the authorization account.

The old account might prevent the creation of the new federated account. This happens, for example, if the account in the external storage has the same email or username as the old account in pSeven Enterprise. In this case, having copied and saved the username, you should delete the old account, and then run federation again.

To link the authorization account to the new user account:

  1. On the Users page of the authentication service interface, look up the new user account and open its edit page.
  2. Copy and save the contents of the ID field displayed on the Details tab of the edit page - this is the unique identifier of the new user account.
  3. Click the icon in the upper right corner of the authentication service interface page, and then select the Manage user permissions menu item to open the list of authorization accounts.
  4. In the list that appears, click the username of the old user account.
  5. In the Keycloak user ID field on the Change user page that appears, enter the identifier of the new user account and save the changes.

The user can now sign in to pSeven Enterprise with the new account. The account matching you performed ensures that the user keeps access to their data.

v2022.08

Updates

  • You can view and manage app runs in AppsHub using the new Runs sidebar available in all apps, including those published with a custom app UI.

    • Click at the upper left of the app page to open the Runs sidebar.
    • Click a run in the sidebar to view its results.
    • To delete runs you no longer need, select them in the sidebar and click the button that appears.
    • By default, you can view and delete only your runs. The app owner can view and delete all runs.
    The app Runs sidebar

  • Updated the Design space exploration block (beta), resolving a number of usability and compatibility issues. Since this version, future pSeven Enterprise updates will keep compatibility with the current Design space exploration block version.

    An optimization problem set up in a Design space exploration block

  • You can use the table editor in the Edit value dialog to create any custom table, as it now supports adding, removing, renaming and reordering table columns.

    Column settings in the table editor

  • Improved Edit value dialog usability for simple values (numbers and strings).

Deployment and administration

  • Windows extension node setup no longer requires Internet access.
  • In a deployment with the "on-demand" block execution strategy enabled, a deployment administrator can control the block shutdown timeout with the new Inactive block timeout setting.

    The "on-demand" block execution strategy is an experimental feature.

  • Updated the pSeven Enterprise administration guide:

  • Fixed a user management issue where pSeven Enterprise deployment could stop with an error if some users have changed their usernames since the previous version upgrade or restart.
  • Fixed a stability issue in workflow run management where a run that failed to start was blocking subsequent workflow runs for all users, and all those runs got stuck in the "Configuration" state until you restart pSeven Enterprise.
  • Fixed a stability issue with the user interface service (appserver) where it could run out of memory if there are users who monitor run logs during execution, which caused a service restart, creating UI issues for other Studio users.

User block development

  • You can implement a user block that runs an existing pSeven Enterprise workflow as the current user running a workflow with your block. Such user blocks work with the pSeven Enterprise REST API described in the REST API guide. pSeven Enterprise provides access to the REST API through the following block environment variables:
    • DA__P7__BLOCK__REST_API_URL - contains the REST API URL.
    • DA__P7__BLOCK__REST_API_TOKEN - contains the REST API authorization token of the current user.
  • Fixed an issue with development of Windows-only blocks, where a user who opens such a block was receiving a "Can not submit job to the cluster!" error.

Bugfixes

  • Fixed an issue where you could not sign in to pSeven Enterprise after waiting at the sign-in page for a certain time.
  • Fixed an issue where file upload, copy, and other file operations in Explorer caused a "connection already closed" error.
  • Fixed an issue with logging in long-running workflows, which caused slowdowns and unresponsiveness when working with such workflow runs:
    • Disabled extended block logging by default to reduce the amount of logs.
    • Logs displayed in the Run log pane are truncated for faster loading.
    • To get a full log, you can download the log file with the Download command from the Run log pane menu.
  • Fixed an issue with Windows-only blocks where a "Can not submit job to the cluster!" error appeared when you open the block configuration dialog.
  • Fixed an issue where sending a Matrix value to a batch mode port of some composite block caused an error, which stopped the workflow run.
  • Fixed an issue with delays in workflow loop execution, that became noticeable after a certain high number of loop iterations.
  • Fixed an issue with workflow loops where, if the loop completed but did not send some of the required responses to the loop driver block, that block waited indefinitely instead of raising an error and interrupting the workflow.
  • Fixed an AppsHub issue with the stock app UI that caused incorrect app behavior if your network connection with AppsHub got interrupted while the app runs.
  • Fixed an issue where a Python script block configured to output a NumPy array of objects (for example, an array of lists) caused an error during a workflow run.
  • Fixed a UI issue with the Workflow library pane where, if you add a block to your workflow and close its tab or hide the pane before it finishes loading the block, the added block was hidden from the pane until you add one more block.
  • Fixed an issue where an error occurred when you copy and paste a block into the same workflow, if the name of that block is a number.
  • Fixed a UI issue with the Block properties pane where it could display incorrect variable names in port tooltips for Condition blocks.
  • Fixed an issue with the Explorer pane where pSeven Enterprise did now display an appropriate error message if you input a non-existing path in the Explorer address bar.

v2022.07

Updates

  • You can manage your account settings - change email, password, and other - in your user profile. To open it, use the new Edit profile command from the pSeven Enterprise user menu.

    The Edit profile command in user menu

    This feature is enabled by the new user authentication system introduced in pSeven Enterprise v2022.07. Due to changes in authentication, you will be required to verify your account once after the upgrade to v2022.07. Until you verify your account, sign-in attempts to pSeven Enterprise v2022.07 will result in an "Invalid username or password" message. To verify, click the "Forgot Password?" link in the pSeven Enterprise v2022.07 sign-in form. Upon clicking that link, you will receive an email with further instructions.

  • To make it easier to find run tabs related to a particular workflow, pSeven Enterprise opens new run tabs next to the related workflow tab, instead of opening them after the last existing tab.

    Run tabs open next to their related workflow tab

  • You can quickly show or hide all block presets, expand and collapse all block groups in the block library using the new and buttons in the Workflow library and Common library panes.

  • Upgraded the embedded pSeven Core package to version 6.36. For details on recent improvements and changes, see the pSeven Core changelog.
  • Updated the REST API guide in the pSeven Enterprise Help to describe REST API v2.

Deployment and administration

Important

When upgrading to pSeven Enterprise v2022.07, follow the instructions found in the pSeven Enterprise upgrade notice below. Otherwise, users may be unable to sign in after the upgrade due to deployment configuration errors, and fixing those errors will require you to re-deploy v2022.07.

  • pSeven Enterprise v2022.07 works with a new authentication service based on Keycloak, which improves user account security and provides additional account management options.
  • Added the number of remaining Studio and AppsHub user licenses to the Licenses page in the pSeven Enterprise admin interface.
  • Fixed a regression issue from v2022.06 where pSeven Enterprise deployment stopped with an error, if you set the entrypoint.allowedHosts parameter in the deployment configuration file values.yaml.
  • Fixed a stability issue with the user interface service (appserver) where it could become unresponsive if many users actively switch tabs in Studio, causing UI slowdowns and errors for all Studio users.
  • Updated and extended the pSeven Enterprise deployment and Administration guides following the introduction of a new authentication service based on Keycloak:
  • Added section Backing up and restoring user data to the administration guide.

User block development

  • Fixed an issue where loading a block prototype to the workflow library by dragging its folder from the Explorer pane onto the Workflow library pane caused a "Device or resource busy" or "Directory not empty" error.
  • Fixed an issue where a value in scientific notation (for example, 1e-08) in block manifest caused an error when you add that block to the workflow library.

Bugfixes

  • Fixed a regression issue from v2022.06 where a workflow run stopped with an error, if the workflow contains a block that outputs a large data array.

    If you encounter that issue, update blocks in the workflow library by clicking the update notification next to the block name in the Workflow library section of the Block library pane. See Updating blocks to a newer version for more details.

  • Fixed a regression issue from v2022.06 where you could not run an app in AppsHub due to a workflow run error, if app parameters have no default values.

pSeven Enterprise upgrade notice

Due to the introduction of a new authentication service based on Keycloak, an upgrade to pSeven Enterprise v2022.07 requires a few additional configuration steps, which are summarized below.

Before you upgrade to pSeven Enterprise v2022.07:

  1. It is recommended to create a backup of the pSeven Enterprise database. You may need that backup if you decide to rollback to a previous pSeven Enterprise version: the v2022.07 upgrade adds new database objects, which may cause issues if the database once upgraded to v2022.07 is used with an older version. The pSeven Enterprise database is specified by the postgres.dbname parameter in your deployment configuration file values.yaml. For database backup instructions, see Backup and Restore in the PostgreSQL server documentation.

Additional upgrade steps required for pSeven Enterprise v2022.07:

  1. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.
  2. Check the entrypoint.allowedHosts and entrypoint.allowedAuthPorts parameter settings in the prepared values.yaml file (see the comments to those parameters for details). Note that entrypoint.allowedAuthPorts is a new parameter, which must be set correctly, considering the domain name and port numbers used to access your pSeven Enterprise deployment. Otherwise, you users will be unable to sign in to pSeven Enterprise after the upgrade.
  3. The prepared values.yaml file also contains a new keycloak.clientSecret parameter. This secret is required to establish connection to the new authentication service (keycloak) and must be specified in values.yaml, otherwise user authentication will be disabled, and users will be unable to sign in.

    1. Generate a new file with pSeven Enterprise v2022.07 secrets using the following command (replace <version> with the full version number):

      docker run --rm -it docker.pseven.datadvance.net/pseven-secretgen:<version> > pseven-secrets-2022-07-keycloak.txt
      
    2. From the generated pseven-secrets-2022-07-keycloak.txt, copy only the keycloak.clientSecret parameter to the values.yaml file you prepare for the upgrade deployment. Do not change other secrets in values.yaml (the jobcluster.* and extensionNode.* parameters) - those must be copied from the existing configuration, as described in Upgrading pSeven Enterprise.

    3. Also copy the keycloak.clientSecret parameter to your backup copy of pSeven Enterprise secrets.
    4. Remove the generated pseven-secrets-2022-07-keycloak.txt file.
  4. Verify your entrypoint.allowedHosts, entrypoint.allowedAuthPorts, and keycloak.clientSecret parameter settings in the updated values.yaml file you prepare. Then continue with the upgrade deployment, following the Version upgrade guide.

After the upgrade to pSeven Enterprise v2022.07:

  1. All admin accounts will have their password reset to admin. Once the upgrade deployment completes, sign in as administrator immediately and change the default password for all admin accounts. To sign in, specify an existing admin username (email address) and the admin password. For password reset instructions, see Managing user account data.
  2. The default administrator account is assigned the username admin instead of admin@admin.admin.

    If user authorization is configured for that account on an extension node (see Configuring user authorization on extension nodes), you will have to change the regular expression in the DA__P7__ALLOWED_USERS environment variable to match the new username admin. Otherwise the user admin will lose existing permission to run blocks on that specific extension node.

  3. All user accounts will have their password unset, and users will have to set a password. Notify your users that they need to pass a one-time account verification by resetting their passwords using the "Forgot Password?" link at the sign-in page. Users who have not set a password will be unable to sign in to pSeven Enterprise v2022.07, and their sign-in attempts will result in an "Invalid username or password" message. Otherwise, you can also set user passwords in the pSeven Enterprise admin interface (see Managing user account data).

Upon the v2022.07 upgrade, existing user accounts keep their usernames, so existing users should sign-in with an email address. The exception is the default administrator account, which is assigned the username admin after the upgrade. New users added after the upgrade are no longer required to use an email address as a username.

If there are no active admin accounts in your deployment, the v2022.07 upgrade will also re-enable the default admin account (username: admin, password: admin).

v2022.06

Updates

  • In the Share dialog, you can press Enter to add another user instead of clicking every time.
  • Enabled renaming block working directories in run result folders of completed workflow runs.
  • Added the support for N-dimensional NumPy array outputs in the Python script block:
    • 2-dimensional arrays are output as matrices by default, as earlier
    • Other arrays are output as nested lists by default
    • A flat (1-dimensional) array is also compatible with the matrix data type and can be converted to a single-column matrix
  • The pSeven Enterprise REST API protocol is updated to v2 with minor changes in the run parameters and run results structures (see sections Run parameters and Run results in the REST API guide for details). The new protocol version is used only if you access the REST API methods by URL /pseven/.rest/v2/. pSeven Enterprise keeps compatibility with the REST API protocol v1 (URL /pseven/.rest/v1/), so existing REST clients, which use REST API v1, will continue to work with no issues. However it is advisable to update existing clients to REST API v2, as well as use only REST API v2 in new client implementations.
  • You can write data tables for reports from a Python script block using the api.write_report_table() function from the block's embedded api module. Function parameters are write_report_table(path, data), where path is the table file full path with an optional .p7table extension, and data is the table data as pandas.DataFrame.
  • You can view shell scripts (.cmd, .bat, .sh) in pSeven, in addition to previously supported file formats. To view a file, double-click it in Explorer, or select it and press Space.
  • Added a few commonly used shortcuts for keyboard navigation in Explorer.
  • Removed the Show active document folder command from the menu in Explorer as it is more conveniently available from the tab bar: while viewing a workflow, run, or report click in the tab bar or double-click the tab title to open the related folder in Explorer.
  • Upgraded the embedded pSeven Core package to version 6.34. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • Increased the number of the block configuration UI router service (blockrouter) replicas to 2 to avoid interrupting user work in case of possible blockrouter failure.
  • Fixed a deployment issue where possible errors in configuration of the database server or NFS storage server were reported only at the final deployment stage, leading to additional downtime during pSeven Enterprise version upgrades or redeployment. pSeven Enterprise now tests connection to the database and file storages at the start of the deployment.
  • Fixed an issue where Windows extension nodes could not connect to pSeven Enterprise if it is deployed on a Kubernetes cluster that uses the Calico CNI plugin.
  • Revised and updated admin guides in the pSeven Enterprise Help:
    • Added a separate Version upgrade section to help locate the appropriate instructions, replacing the upgrade sections previously included in the pSeven Enterprise administration and extension node deployment guides.
    • Expanded the administration guide with section Understanding accounts to provide a better understanding of account setup in pSeven Enterprise.
    • Moved the section describing user access settings on extension nodes from the extension node deployment guide to Administration - see Configuring user authorization on extension nodes.
    • Moved all uninstallation instructions to a separate Uninstallation guide.
    • Several other minor updates and additional explanations.

User block development

  • Redesigned the Edit value and View value dialogs to better support custom tables with predefined columns and other custom data types.
  • Fixed a regression issue from v2022.03 where pSeven Enterprise did not properly remove files from the block data directory (DA__P7__BLOCK_DATA_DIR) after they had been deleted by the block.
  • Updated section Block assets in the user blocks guide with more information about publishing and updating assets, and also using assets to add Python modules to blocks.

Bugfixes

  • Fixed a regression issue from v2022.05 where you could not download large files from pSeven Enterprise.
  • Fixed a regression issue from v2022.04 where moving a block to another workflow using cut and paste would not remove the block from the source workflow, causing errors when you edit the source workflow.
  • Fixed an issue where passing complex List or Dict values between blocks caused delays in workflow execution.
  • Fixed an issue with editing workflow notes where they could reset position or disappear if you are actively editing notes.
  • Fixed an issue with renaming subfolders in the Files special folder in workflows where you could add the reserved .p7wd extension to any folder, which caused various filesystem error messages.
  • Fixed a UI issue with the workflow editor where, if you remove some port that has a link connected to it, that link was automatically removed as intended but was still displayed in the workflow until you refresh a browser tab, which caused confusion.
  • Fixed several issues with the Run setup pane where it incorrectly displayed the current parameter preset.
  • Fixed incorrect behavior of the info pane in Explorer when selecting a shared folder (workflow, report) that is no longer available - for example, was moved or deleted by its owner.
  • Fixed an issue with the View as text command in Explorer where viewing a binary file caused UI unresponsiveness.
  • Design space exploration block (beta):
    • Fixed slowdowns when adding or removing the block from a workflow.
    • Fixed incorrect order of ports in the Block properties pane.
    • Fixed several issues that caused block execution to stop with an error.

Compatibility issues

  • Due to changes in data type conversion rules that were required to add the support for N-dimensional NumPy arrays in the Python script block, in certain cases the arrays that were previously converted to a single-column matrix may now instead be converted to flat lists if you pass them through untyped ports (the "Any" port type). Typically this causes a "'float' object is not subscriptable" error in the block that receives such an array. In affected workflows, that error appears only after you update the Python script block in the workflow library to the most recent version (block version 20). To revert to the old behavior, you can change port type in affected blocks to the matrix type - to force conversion of a 1-dimensional array to a single-column matrix.
  • The typesystem.File() constructor previously used in the Disk optimization example to create data tables for reports is no longer supported in Python script blocks, as it was replaced with the more convenient api.write_report_table() function from the block's embedded api module.

v2022.06.07

v2022.06.07 is a hotfix release for several issues related to the "Failed to send interrupt request to block" error appearing in v2022.05.

Bugfixes

  • Fixed a regression issue from v2022.05 where numerous "Failed to send interrupt request to block" error messages appeared in the Run log pane when you stop the workflow.
  • Fixed a regression issue from v2022.05 where the stop button on a running block did not work.
  • Python script block: fixed a regression issue from v2022.05 where api.interrupted() returned False even if the workflow run is stopping when the block calls this function, instead of returning True in that case, as intended.

User block development

  • Fixed a regression issue from v2022.05 where, when a user stops a workflow run, pSeven Enterprise did not send an interrupt request (the interrupt control endpoint) to running blocks, so a block could not handle the run stop properly.

v2022.05

Updates

  • To update the stock app UI in a workflow created in previous pSeven Enterprise versions or to revert UI changes in a workflow, you can use the new app UI reset command from the menu in Explorer. To update an app published with the stock UI, reset the app UI in the workflow you published as that app, then re-publish the workflow.

    The app UI reset command in the Explorer menu

  • Timestamps in the Run log pane use the time format from your system (browser).

  • Added a couple of convenience features for the tree view in the Explorer pane:
    • If you navigate between folders that are displayed in tree view (for example, run results), the pane now remembers the tree state in a few folders you have recently visited.
    • If you input a path pointing to a file or folder in a tree, the pane expands the corresponding tree node upon navigating to that path.
  • The Explorer pane uses icon and background colors to indicate status of workflow runs. The color scheme is consistent with the one used on run tabs.

    Colored run icons in Explorer indicating run status

  • Added an early preview version of the Design space exploration block, which provides methods for design of experiments and optimization.

    The Design space exploration block in the library

    This early preview version of the Design space exploration block may appear to be incompatible with future pSeven Enterprise updates and should be used for testing only.

  • Upgraded the embedded pSeven Core package to version 6.33. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • To prevent situation when some user opens too many blocks or starts too many workflow runs at the same time, which interferes with other users' work, you can now set limits for the number of open blocks and the number of workflow (or app) runs - common ones on the Settings page, which apply to every user by default, and individual limits for specific users in their user settings.
  • Added information about storage space used and storage quota to the Users page in the pSeven Enterprise admin interface.
  • Fixed an issue with pSeven Enterprise version upgrade where the deployment of a new version could take several hours, depending on the amount of stored user data, because it always ran a filesystem check procedure, which is often not required. That procedure is now optional, does not run by default, and you can enable it by adding --set fixStorages=true to the helm install deployment command.
  • Fixed an issue where pSeven Enterprise could stop responding to user actions and sign-ins, which was caused by appserver service errors and required you to manually restart appserver. Now if any of the appserver replicas stops responding for longer than a minute, that replica is restarted automatically, which may interfere with work of some users but avoids a complete functional failure.
  • Fixed an issue where user workflow runs were blocked by the license limit if a number of runs had failed earlier - sometimes the run failure status was not set correctly, so failed runs could count as still active.
  • Updates for pSeven Enterprise deployments with run-time resource management enabled (optional feature, disabled by default):
    • Added a block execution scheduler to avoid overallocation of resources when running user workflows.
    • Fixed an issue where pSeven Enterprise could cease to run user workflows until you restart the task execution service.

Bugfixes

  • Fixed an issue where you could not sign in to pSeven Enterprise or it stopped responding after certain errors - for example, when someone opens a corrupt workflow or report - which could be resolved only manually by a deployment administrator.
  • Fixed a security issue in the REST API where a malicious user could set permissions on a folder owned by other user.
  • Fixed a UI issue where the Block properties and Composite properties panes incorrectly displayed unconnected optional input ports (gray dot) as required (red dot).
  • Fixed a convenience issue with the list of recent runs for shared workflows in the Start tab where the list included runs of all users who have access to that workflow instead of including only your runs.
  • Fixed an AppsHub issue with the stock app UI where an attempt to run an app that was deleted after you opened the app page produced an incomprehensible error message.
  • Fixed a UI issue where the Run setup pane did not display long values of parameters in full even if you extend pane width.
  • Fixed a UI issue where refreshing the Explorer pane in the tree view collapsed all tree nodes.
  • Fixed a UI issue with editing workflow notes where the browser context menu did not appear when you right-click a note.
  • Fixed a UI issue where special folders like Files or Runs were not always sorted to the top in the Explorer pane as intended.

v2022.04

Updates

  • Updated the stock app UI for AppsHub: if you open an app page directly by its URL, the app recognizes query string parameters as its arguments and automatically sets parameter values in the UI. This is useful, for example, if you integrate pSeven Enterprise apps to some other environment where users select parameter values before they launch an app: now you can pass user values to app in the query string, so users do not have to repeat parameter input in the app. See section App URL parameters for details. Note that previously published apps do not get this update automatically. To update an existing app, update the stock app UI files in the UI folder of that app's source workflow, then re-publish the workflow.
  • If a workflow run has stopped with an error, you can see error details in the run status tooltip on the run toolbar.

    Run status tooltip with error details

  • Several improvements in the Block properties and Composite properties panes:

    • If you are viewing contents of a composite block, its Composite properties pane displays port connection status regarding uplinks inside that composite block. Previously the pane was always showing status of external connections to the composite block, which was inconvenient when checking uplinks from nested blocks to ports of their parent composite block.
    • The sorting buttons work independently - each button switches the sort order in a specific list, not everywhere in the pane. For example, you can use different sort order in the input and output port lists.
    • Better alphabetical sorting: it is now case insensitive, and numbered ports are sorted in natural order - for example, port 9 is sorted before port 10.

      Port sorting

    • Sorting now applies to the order of port groups. Previously, sorting applied only to ports inside each group but never changed the group order, and port groups were always displayed before non-grouped ports in the list.

    • When you edit a port name in the Composite properties pane, leading and trailing space characters are trimmed to avoid accidentally creating ports with names that differ in the number of surrounding spaces only and look confusingly the same in the pane.
    • Port name validation in composite blocks is now case insensitive. In the Composite properties pane, it is no longer possible to create ports with names that differ by case only, like value and Value.
    • Improved pane UI responsiveness when working with large lists of ports.
  • Upgraded the embedded pSeven Core package to version 6.32. For details on recent improvements and changes, see the pSeven Core changelog.

  • Updated pSeven Enterprise Help:
    • Updated the Python script block guide with a new section explaining how to get a complete list of Python modules available in the block - see Listing modules.
    • Added a few notes on troubleshooting module import errors to section Python modules in the Python script block guide.
    • Updated section Publishing the workflow as an app with information about using query string parameters in the app URL to pass parameter values to an app (see App URL parameters).

Deployment and administration

  • pSeven Enterprise v2022.04 contains updates in the license mechanism, which add flexibility to license settings, enabling certain optimization of license usage (see below). Due to these updates, previously issued licenses are not compatible with v2022.04. You will need to request an updated license key and have it ready before you begin the update - so you can update the license immediately after deploying v2022.04. Otherwise, your users will be unable to sign in to pSeven Enterprise until you update the license key in your deployment. For further details, see the pSeven Enterprise upgrade notice below.
  • You can configure an admin account so that it does not count as a licensed user. See sections User permissions and Admin account configuration in the pSeven Enterprise administration guide.
  • pSeven Enterprise license since v2022.04 counts the number of AppsHub users and Studio users separately. AppsHub users can sign in to AppsHub and run apps from there. Studio users can sign in to both Studio and AppsHub, can edit and run workflows, publish them to AppsHub and run apps. AppsHub and Studio access require different user permissions, so you can configure user accounts to optimize license usage - see sections User permissions and End user account configuration in the pSeven Enterprise administration guide.
  • If a workflow or an app run has failed, you can now see the fail reason on the run page in the admin interface.
  • Removed the "Job info" column from the Workflow runs page in the admin interface as it was taking too much screen space. Job info is available on each run page.
  • Fixed an issue where numerous pint.util warnings appeared in logs of every Kubernetes pod.
  • Fixed an internal configuration issue, which could cause zombie processes appearing in pSeven Enterprise service containers, potentially leading to various failures that would be hard to investigate.
  • Updated section Prerequisites in the pSeven Enterprise deployment guide with notes about additional system requirements considering Python modules in user code (see in section Kubernetes cluster).
  • Slightly cleaned up and improved the layout of the deployment infrastructure diagram in the pSeven Enterprise deployment guide (see Deployment diagram).
  • Updated section User management in the pSeven Enterprise administration guide.

User block development

  • Fixed an issue where names of assets in the block manifest (see Block assets) were not checked properly, which allowed using assets with names that can lead to a name clash in block environment variables that store asset paths.

Bugfixes

  • Fixed several internal issues with task management services that led to instability being one cause of incorrect behavior when running workflows - such as stopping with an error because a block failed to start, or inability to start another run after some run has stopped with an error.
  • Fixed a bug in workflow execution, which contributed to the issue where a workflow run just started by a user could get stuck in the "Submitting" state. The issue is not fixed completely yet, although that situation is more rare now.
  • Fixed an issue where a workflow run could get stuck in the "Queued to run" state if the user who started that run is near the disk quota limit.
  • Fixed the primary cause of the issue where the message "Failed to send updates to platform: [Errno 32] Broken pipe!" appeared repeatedly in the run log. That warning may still appear rarely, although it does not indicate a run error.
  • Fixed an issue where saving a block in its configuration dialog reset or unset its input values specified in the Block properties pane.
  • Fixed an issue with the Run setup pane where you could unintentionally unset a parameter, because for parameters of certain types (matrix, list, and some others) clicking outside the input field or pressing Enter while editing the parameter reset its value instead of accepting edits as intended.
  • Fixed an issue where a driver block in a running workflow started a loop that cannot complete because some of the loop body blocks do not receive all their input values, after which the run waited indefinitely for that loop. Now driver blocks check their loops before start and cease to run if required input data cannot enter the loop - this is possible, for example, if a block in the loop body receives some input from a Condition block. Also, if some block in the loop body cannot run (for example, one of its required input ports is not connected), a warning is issued to the run log pointing to the faulty block.
  • Fixed an issue with the Block properties and Composite properties panes where adding many ports to workflow parameters could cause incorrect UI behavior or unresponsiveness.
  • Fixed an issue with the Composite properties pane where you could create a port with a name that contains only space characters.
  • Fixed an AppsHub issue with the stock app UI where you could not set a value to a matrix type parameter if that parameter has no default value.
  • Fixed an issue where you could not open, move, or delete a folder copied from a run results folder.
  • Fixed a UI issue with the tab bar, which sometimes did not display the left and right shift buttons and the scrollbar when you have many tabs open.
  • Fixed a minor UI issue with workflow navigation in run tabs where you could not go inside a composite block with a double-click (like in workflow tabs) and had to always click the icon on the block to view its contents.

pSeven Enterprise upgrade notice

Since v2022.04, pSeven Enterprise license counts the number of AppsHub users and Studio users separately - the limits for them are set independently by the license. The type of a user depends on access permissions you set for that user. This also allows creating admin-only accounts, which do not have user access permissions - that is, cannot sign in to pSeven Enterprise as users. Admin-only accounts are not counted by the license, so you can create a number of such accounts as needed.

Licenses issued for previous pSeven Enterprise versions count only the total number of users, so they are not compatible with v2022.04. You will need to request an updated license key and have it ready before you begin the update - so you can update the license immediately after deploying v2022.04.

Important

Get an updated license key for pSeven Enterprise v2022.04 before you begin the upgrade. You will have to remove the existing key and add the new key immediately after upgrading to pSeven Enterprise v2022.04. Otherwise, user sign-in to pSeven Enterprise after the upgrade will be blocked until you replace the license key in your deployment.

The advisable upgrade sequence for pSeven Enterprise v2022.04 is:

  1. pSeven Enterprise licenses are bound to a specific Installation ID. Get and save your Installation ID as described in section License of the pSeven Enterprise deployment guide.
  2. Request a license update for your Installation ID. Postpone the upgrade to v2022.04 until you receive an updated license key.
  3. Having received the new license key, upgrade to v2022.04 as described in the Version upgrade guide.
  4. Immediately after the v2022.04 deployment finishes, sign in as admin and open the Licenses page of the administrative interface (URL: <admin URL>/app_auth/license/).
  5. Remove the existing license. Licenses from previous versions are not compatible with pSeven Enterprise v2022.04 and may cause issues later.
  6. Add the updated license key at the Add license page (URL: <admin URL>/app_auth/license/add/).

After replacing the license key, verify that you can sign in as a user. If you cannot sign in or encounter other issues during the upgrade, contact the DATADVANCE support team for advice (see Technical support).

After you complete the v2022.04 deployment and license update, consider also updating user permissions and configuring user and admin accounts as described in section User management of the pSeven Enterprise administration guide.

v2022.03

Updates

  • More tab bar enhancements:
    • When adding a large number of tabs, their widths are proportionally reduced in order to fit more tabs into the bar, until tab width reaches a certain minimum.
    • When tabs fill the pane, left and right shift buttons appear to enable mouse-click scrolling through tabs.
    • The tab bar menu now contains a list of all open tabs, so you can simply select a tab name from the list to navigate to the desired tab.
    • In addition to scrolling tabs with your mouse wheel, you can also use swipe gestures - for example, on Mac.
    • The new tab button is no longer stuck at the far right.
  • Upgraded the embedded pSeven Core package to version 6.31.1. For details on recent improvements and changes, see the pSeven Core changelog.
  • Tooltips on links that may cause errors or unexpected behavior when running the workflow now display all warning and error details at once, instead of displaying only one of those messages.

Deployment and administration

  • Updated guidelines. The pSeven Enterprise deployment guide was becoming overloaded, so it was split into several guides:
    • The pSeven Enterprise deployment guide now describes only important deployment steps. This guide was also updated to address a number of issues that may arise when installing pSeven Enterprise.
    • The Administration section contains an initial version of the support and maintenance guide.
    • Technical support contains support contacts and troubleshooting instructions.
    • Branding explains branding-related customizations.
  • Updated the sizing.* parameter descriptions in the pSeven Enterprise deployment configuration file values.yaml to better explain their role.

User block development

  • Your block can now read data from the shared data storage - see section Block assets for details.
  • Fixed an issue where pSeven Enterprise sometimes deleted folders created by the block in its data storage (DA__P7__BLOCK__DATA_DIR).
  • Fixed an issue where port values in the Results pane were always highlighted red (invalid) if the port data type is a custom table with predefined columns.

Bugfixes

  • Resolved the most prominent of stability issues that caused the "Task scheduler service restart" and "No such file or directory" errors when running a workflow. The fixes applied do not remove those errors completely yet, though significantly reduce the probability of them appearing during a workflow run.
  • Fixed an issue that caused the "Block process failed to start" error when running a workflow.
  • Fixed an issue where exceeding the user's disk quota made it impossible to stop a workflow, and the workflow continued to spam errors.
  • Fixed an issue that caused damage to the workflow when in a situation where the user's disk quota is exceeded.
  • Fixed an issue where the block configuration dialog failed to open, displaying an empty window along with a "Block edit session failed" error message.
  • Fixed an issue with the Functions pane in the Python script block configuration dialog where pressing Enter while the cursor is in the Function name field removed one of the function arguments.
  • Fixed a regression issue from v2022.02 where moving a block to another workflow using cut and paste would not remove the block from the source workflow, until you refresh the browser tab with pSeven Enterprise or reopen that workflow.
  • Fixed UI issues with long names in the Links dialog. Long names now have an ellipsis to fit in the dialog width, and full names can be viewed by hovering over the name in the list.
  • Fixed an issue where a workflow run tab could remain in the loading or waiting state until you refresh a web browser tab with pSeven Enterprise.
  • Fixed an issue that resulted in an error message when double-clicking a workflow run tab while the run is initializing.
  • Fixed an issue with the Block library pane where a block update notification in Workflow library did not appear until you select that block in the pane.
  • Fixed an issue where the workflow run log sometimes did not update for about half a minute due to a log file initialization error.
  • Fixed an issue due to which the command for automatic layout of blocks in the workflow could place some blocks very far from the rest.
  • Fixed an issue where the search function in the Russian language version of Help ignored English words.

v2022.02

Updates

  • Updated Links and Autolink dialogs:

    • The Links dialog now supports port groups. You can collapse and expand the list of ports in each group.
    • When linking a driver block, you can select which ports to display: driver only, input (output) only, or all.
    • The Autolink dialog now suggests links in one direction only, from source to target block. Its old version, which suggested links in both directions, proved to be inconvenient for linking blocks with many similarly named ports.
    Port grouping in the Links dialog

  • Upgraded the embedded pSeven Core package to version 6.30. For details on recent improvements and changes, see the pSeven Core changelog.

Bugfixes

  • Fixed an error when running a workflow with a Composite block that has a batch input port added to workflow results.
  • Fixed an issue where the recent list on the Start tab was emptied if you launch many workflow runs in succession.
  • Fixed an issue where editing a workflow after applying automatic block layout could unpredictably change the layout.
  • Fixed an issue where UI was becoming unresponsive if you move blocks in the workflow using keyboard.
  • Fixed an incorrect "You are trying to access an item that no longer exists" warning appearing when you copy blocks or notes in the workflow.
  • Fixed an issue with renaming workflow runs and items in the workflow Files folder where it was impossible to select text with a mouse during rename.
  • Fixed an issue where sometimes you could not rename a folder immediately after copying it.
  • Fixed a UI error that could be seen in the browser console when you open a block or edit port values in the Block properties pane.

v2022.01

Updates

  • Redesigned the tab bar for more convenience:

    • No more multi-row tab stack clutter. Tab titles always have the same width and are always displayed in a single line, which you can scroll with your mouse wheel, when there are too many open tabs to show all their titles at once.
    • Run tab titles use colored icons, which indicate run status.
    • Added tab bar menu with a few useful commands.
    The new tab bar

  • The main toolbar in a run tab displays current run status.

    Run status in the run toolbar

  • The Explorer pane displays your total and available storage space in its title bar.

    Storage information in Explorer

  • When you select one or a few blocks to view their log messages, the Run log pane title displays names of selected blocks, so you can always tell which logs you are viewing at the moment.

  • You can rename workflow runs in Explorer - for example, to tag important or debug runs.

    Note that you cannot rename a run in progress. To create a new run with a custom name, use the New run command, which prepares a run but does not immediately start it. Then while your new run is in configuration, rename the run folder in Explorer.

  • You can view log files, Python scripts, CSV files, JSON and YAML data files in pSeven, in addition to previously supported Markdown and plain text files. To view a file, double-click it in Explorer.

  • Upgraded the embedded pSeven Core package to version 6.29. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • User storage quotas. A deployment administrator can set file storage quota which applies to all users (as default), and override that default with user-specific quota settings. Quota management requires disk quota support on the file storage server - see section Enabling user storage quota in the pSeven Enterprise deployment guide for details.

Bugfixes

  • Fixed a workflow publishing issue where the published app could not run, if a block in the app's workflow reads files from the workflow directory.
  • Fixed an issue where refreshing a browser tab with pSeven Enterprise or AppsHub sometimes entered a loop, indefinitely redirecting you to the sign-in page and back.
  • Resolved a stability issue that could lead to errors appearing when you open a block, after which the block UI did not load - but could load successfully the next time you open the block.
  • Fixed a UI regression issue from v2021.12 where pSeven sometimes displayed a run that has already finished as a queued run or a run in progress, and you had to reopen the run tab to fix that.

v2021.12

Updates

  • Added initial support for the TensorFlow machine learning (ML) platform: in Python script blocks, you can use TensorFlow (import tensorflow) and Keras API (from tensorflow import keras) to develop and train ML models. Also added lasio (import lasio) to read and write Log ASCII Standard (LAS) files.

    Using TensorFlow, Keras API, or lasio in a Python script block that is configured to run on a Windows extension node is currently not supported.

  • You can now test Python script blocks without running the workflow - see section Testing the script in the Python script block guide.

  • Upgraded the embedded pSeven Core package to version 6.28. For details on recent improvements and changes, see the pSeven Core changelog.
  • Added logging to the stock app UI in AppsHub, so you can see progress of an app run, its error details, and so on. To view or download app logs, click the Log button in the app page header. Previously published apps do not get this update automatically, but you can re-publish them with the new version of the stock app UI.
  • You can now see folder (workflow, run) size in Explorer - select a folder, then expand the properties pane at the bottom and click calculate to get the selected folder size.
  • To prevent clutter in the workflow run log, the Run logs pane displays memory usage messages from blocks only if you enable showing the debug log in the pane's menu.
  • Error toast messages no longer disappear from screen by timeout - you close them manually, so there is no risk of missing an important error message. Information messages are automatically closed after a certain time, same as in previous versions.
  • Links that may cause errors or unexpected behavior when running a workflow are highlighted orange (warnings) and red (errors). Warning and error details are available in link tooltips.
  • Updated pSeven Enterprise Help:

Deployment and administration

  • Fixed a deployment issue where the command to generate secrets failed due to a configuration error.
  • Fixed an issue with pSeven Enterprise admin interface where searching for a user caused an error.

Bugfixes

  • Fixed an issue where autolinking blocks could unexpectedly remove manually created links between those blocks.
  • Fixed an issue with the Run logs pane where it was not showing logs of the selected block, or was showing logs of a different block than the selected one.
  • Fixed an issue where error messages in a workflow run log were too wordy and always included full traceback, often making it harder to identify the error cause.
  • Fixed block sorting in the Block library pane.
  • Fixed various issues with keyboard navigation in the Explorer pane.
  • Fixed a UI issue with the Explorer pane, which was too wide even when resized to its minimum.
  • Fixed a UI issue with Run overview on the Start tab where it kept displaying the "Loading..." message instead of the usage tip text if there are no workflow runs that a user can view, causing confusion.

v2021.11

Updates

  • Mass uplinking and adding ports to workflow parameters is now more convenient: in the Block properties pane, you can select multiple ports in Inputs or Outputs section, then use the and buttons that appear in the section header. adds uplinks by default, however if all selected ports are uplinked already, removes their uplinks. Similarly, excludes selected ports from parameters, if all of them are parameters already. And, since an output port cannot be a parameter, the button appears in Inputs only.
  • The Block library pane loads noticeably faster in the case when the library contains blocks with many ports and a number of configuration presets.
  • During a workflow run, the Workflow contents pane shows block memory usage, and blocks also regularly report their peak and average memory consumption to the run log.

    Block memory usage in the Workflow contents pane

  • Run status indicators on the Start tab are slightly redesigned to unify their style and make color scheme consistent with the background color scheme used on run tabs.

    Run status indicators redesigned in v2021.11

  • Completed workflow runs take less storage space because intermediate data files are automatically deleted when a run finishes.

  • Changed the default parallelization ratio of batch-processing Composite blocks from 4 to 2 (the @Number of parallel workers setting, see Batch mode service ports). This is done to avoid situation where a workflow run that includes nested parallel Composite blocks requests too much resources by default, forcing you to manually edit settings of all Composite blocks to reduce the number of block instances created due to parallelization.
  • Partially resolved workflow run performance issues caused by using List and Dict data types to pass big volumes of data through links between blocks. As a result, delays in workflow execution were reduced but not completely removed yet.
  • Removed a distracting warning pop-up that appeared in certain cases when you add or remove an uplink in the Block properties pane.

Deployment and administration

  • A deployment administrator can now control resource usage with the following new settings:
    • The Max blocks per run setting limits the number of block instances that may be created in a single workflow run started by any user. Use it to avoid situations where a single user consumes all cluster resources by running a workflow that creates a high number of block instances - for example, due to extensive usage of Composite blocks with enabled parallel processing. Note that each block nested into Composite with parallel processing creates multiple instances when running, and the number of those instances is controlled by user through the Composite block settings.
    • The Max open blocks setting limits the total number of blocks that can be open for editing at the same time, considering all blocks opened by every one of the users. Use it to avoid situation where the number of open blocks is high enough to consume most of cluster resources, which causes block edit and save errors as well as various other issues.
  • The task executor settings in the pSeven Enterprise deployment configuration file (the sizing.htcondorexecuteResources.* parameters in values.yaml) now set resource requests for that service instead of resource limits. This improves the service stability in cases when its actual resource consumption (the amount of resources used by running blocks) exceeds the expected one (the request): the service can now use additional resources as long as they are not claimed by other services, instead of shutting down as soon as it reaches the limit. For further information about new settings, see the pSeven Enterprise upgrade notice below.
  • Adjusted several deployment settings in the Helm chart to improve pSeven Enterprise stability:
    • Increased the number of user interface service (appserver) replicas from 2 to 3.
    • Increased the number of task submit service (htcondorsubmit) replicas from 1 to 2.
    • Increased amount of requested resources for some services.
  • Removed the disableFileLock parameter from the pSeven Enterprise deployment configuration file (values.yaml) as it is intended only for troubleshooting in a few rare cases, while being dangerous and essentially useless in production deployments.
  • Optimized UI to lower the number of requests to the pSeven Enterprise user interface service (appserver), reducing its load.
  • Fixed an issue where the user interface service (appserver) repeatedly ran out of memory and restarted while there is a user who keeps open a tab with an ongoing workflow run. Frequent restarts caused UI issues noticeable to all users and, in some cases, errors during sign in.
  • Fixed an issue with workflow run status notifications, due to which a few concurrently running workflows could overload the pSeven Enterprise user interface service (appserver), causing UI unresponsiveness and incorrect run statuses being displayed on the Start tab.
  • Fixed an issue with published user blocks where pSeven Enterprise deployment failed if the folder name of some published block contains a space.
  • Fixed extension node deployment issues:
    • After installing Poetry, node deployment could fail to install Python packages due to an issue with Poetry cache.
    • Node deployment could fail if you specify a relative path to the installation or temporary files directory when running bootstrap.bat.

User block development

  • Fixed section Publishing a custom block in the block development guide: added missing instructions about setting access permissions for published blocks.

Bugfixes

  • Fixed an issue where watching an ongoing run in a tab disrupted general UI functionality, so the UI became unresponsive for all users, and some users also experienced errors during sign in.
  • Fixed unexpected slowdowns in workflow editing, which were noticeable when you have several workflow tabs open.
  • Fixed an issue where a large workflow with many links open in a background tab caused unexpected slowdowns and UI unresponsiveness when editing workflows in other tabs.
  • Resolved a stability issue that appeared when several users actively edit different blocks in their workflows at the same time. In particular, that issue caused errors when opening a block or saving block configuration changes.
  • Fixed an issue with the Links dialog where it did not show ports that you have recently added to one of the blocks you are linking.
  • Fixed an issue with the Links dialog where lists of ports disappeared when linking two blocks of the same type.
  • Fixed issues with the Block properties pane where uplinking many ports caused slowdowns, incorrect UI behavior and errors.
  • Fixed an issue with the Block properties pane where it did not properly update the list of ports when you select different blocks in the workflow and displayed ports of a previously selected block.
  • Fixed an issue with the Block properties pane where it displayed uplinked ports as not uplinked.
  • Fixed an issue where you could not uplink multiple ports to the same port of a Composite block using buttons in the Block properties pane.
  • Fixed an issue where a block input value specified in the Block properties pane was not saved if you deselect that block or select another block immediately after input.
  • Fixed an issue where input parameter values in the Run setup pane were not updated when you change parameter presets.
  • Fixed an issue where uploading large files caused a 502 Bad Gateway error.
  • Fixed delays in log output in the Run logs pane.
  • Fixed an issue where a running workflow, which contains Composite blocks with enabled parallel processing, did not stop when a block nested into one of those Composite blocks fails to start, which caused various workflow errors later on.
  • Fixed an issue with block working directory synchronization, due to which workflow runs occasionally stopped with a file access error.
  • Fixed an issue where a workflow run stopped with a "Task failed" error immediately after start.
  • Fixed a UI issue where a run tab stopped updating after you switch to another tab and back - for example, when you start several runs and switch between them.
  • Fixed an issue with the Start tab where it displayed workflow run status incorrectly due to unexpected delays in status update.
  • Fixed a UI issue where the Start tab did not show the list of runs in Run overview if you do not interact with UI after opening a new Start tab.
  • Fixed some of the UI issues that appear on low-resolution displays (tablets and laptops), or when you resize the browser window with pSeven Enterprise.
  • Fixed an issue with high resolutions (for example, on 4K displays), which caused delays when switching between tabs in the pSeven Enterprise UI.
  • Fixed an issue with the Block library pane where the Workflow library section sometimes did not show block update notifications even if a newer version of a block is available from the Common library.
  • Fixed several issues that caused incorrect behavior when editing workflow annotations - for example, the text you edit could be lost or added to another annotation instead of the one you are editing.
  • Fixed a workflow copying issue where active workflow runs were "copied" with the workflow. A copy of the workflow, which was created while that workflow is running, contained runs that were incorrectly displayed as active (running) while in fact those runs in the copy are interrupted.
  • Fixed an issue where dragging a workflow in a run tab reset block startup counters to 0.
  • Fixed an issue where a workflow run that contains a Composite block with enabled parallel processing failed with an "unknown value id" error.
  • Fixed an issue where you could accidentally clear some result value in the Run setup pane, because the value reset button was displayed when you click a result despite that button is intended for input parameters only.
  • Fixed issues with the stock app UI in AppsHub:
    • The Stop button did not work after you refresh the app page.
    • The app UI did not clear the list of files in the Results pane once you start a new calculation.
  • Fixed a UI issue in the Explorer pane where a file upload progress bar remained at 0 if the cursor is away from the pane.

pSeven Enterprise upgrade notice

In the pSeven Enterprise v2021.11 deployment configuration file (values.yaml), resource settings for task executors specify resource requests instead of limits: the sizing.htcondorexecuteResources.limits.* parameters are replaced with the sizing.htcondorexecuteResources.requests.* parameters. Note that requests (sizing.htcondorexecuteResources.requests.*) are now required while limits (sizing.htcondorexecuteResources.limits.*) are optional, and the v2021.11 deployment configuration file sets only requests parameters by default.

When transferring parameter settings from an existing deployment to the v2021.11 configuration file (see Upgrading pSeven Enterprise), there are two advisable ways to update resource settings for task executors:

  • To improve pSeven Enterprise stability under load, specify requests only. Set requests to the same values that were previously specified as limits. This allows task executors to temporarily exceed the expected resource consumption, using additional resources as long as they are not claimed by other services.
  • To reproduce behavior of previous pSeven Enterprise versions, specify requests and limits, and set both to the same values that were previously specified as limits. Note that with such settings, Kubernetes pods where task executors run will be given a QoS class of Guaranteed (see QoS classes in the Kubernetes documentation).

v2021.10

Updates

  • The code editor in Python script blocks now supports search and a few other common functions - see section Keyboard shortcuts in the Python script block guide.
  • Ctrl+S or Cmd+S in Python script now saves the block.
  • Newly created workflows now contain the README.md file template. When publishing a workflow to AppsHub, you can use README.md to add descriptions of the app, its parameters and results. See Adding descriptions to the stock app UI for more information.
  • Improved editing and display of parameters and results in the stock app UI in AppsHub:
    • Non-default parameter values are now displayed in bold, so you can easily tell which parameters you have already changed.
    • Each parameter now has a button, which unsets the parameter value.
    • If a result value is too long to display it in the result field (for example, the value is a matrix, a long list or string), a button appears. Clicking it opens a dialog where you can view the full value.
    • Increased value preview length in result tooltips.
  • You can now unset input values in the Block properties and Run setup panes: click a value to edit, then click in the input field.
  • Pan action in a workflow tab now works more smoothly.
  • Blocks in workflows of example apps in AppsHub are now automatically updated to current versions upon a pSeven Enterprise version upgrade.
  • Added a beta tag to the GraphQL API command in the user menu to indicate that pSeven Enterprise GraphQL API is unstable and is not yet ready to be used in production.
  • Updated pSeven Enterprise Help:
    • Added section Keyboard shortcuts to the Python script block guide.
    • Updated license information and instructions in section License of the deployment guide as the pSeven Enterprise license is now deployment specific and tied to the Installation ID.
    • The Russian language version of pSeven Enterprise Help now contains a full changelog, the same as the English version.

Deployment and administration

  • pSeven Enterprise licenses are now deployment specific. Each new deployment has a unique Installation ID (which is now generated during installation) and requires a valid license bound to this specific Installation ID. An existing Installation ID is kept upon a pSeven Enterprise version upgrade, so a license previously issued for that Installation ID remains valid after the upgrade, within its licensing period. However if you upgrade an existing deployment that does not yet have an Installation ID, you will need to request an updated license, as the upgrade will generate a new Installation ID for your deployment. For further details, see the pSeven Enterprise upgrade notice below.
  • Fixed an issue where file storage requirements (see section File storage in Prerequisites) were not checked properly during installation, so deployment with an unsupported file storage continued but failed after a long timeout.
  • Fixed an issue with the extension node setup package, due to which a node could appear working after installation but stopped functioning later - for example, after a reboot or after some of the pSeven Enterprise services on the node had been restarted.
  • Fixed an issue where the OpenVPN port setting for extension nodes specified in the pSeven Enterprise deployment configuration file (the entrypoint.gateOpenvpn parameter in values.yaml) did not apply, requiring you to manually edit the OpenVPN configuration file (winnode.ovpn) on every extension node if you change the entrypoint.gateOpenvpn parameter during deployment.

User block development

  • It is now possible to implement and use a Windows-only block, which starts on a Windows extension node both in edit (configuration) and run (execution) modes. A user configuring such a block should select a Windows node to run on in the Block properties pane prior to opening block configuration. Thus a block in edit mode will be able to access node-specific resources - for example, Windows-only software tools installed on the selected node.

Bugfixes

  • Fixed an issue where saving a block was taking too much time.
  • Fixed unexpected slowdowns when deleting files in Explorer, deleting blocks in the workflow or in the Workflow library pane.
  • Fixed an issue where pSeven Enterprise was becoming unresponsive if you edit a complex workflow containing many blocks and links.
  • Fixed an issue where changes in workflows and apps were not showing until you delete the browser cache, which happened after a pSeven Enterprise version upgrade.
  • Fixed an issue where a workflow failed to start after you update a block in the Workflow library to a newer version.
  • Fixed slow loading of the Block properties pane.
  • Fixed an issue with the Run logs pane where it sometimes did not show the log until you reopen the pane.
  • Fixed an issue where scroll position in the Run logs pane was reset if you switch between document (workflow, report) tabs.
  • Fixed an issue with the Block properties pane, which was incorrectly highlighting an optional input port as a port required to start the block.
  • Fixed an AppsHub issue where refreshing an app page reset the app UI and unexpectedly stopped the app if it was running calculation.
  • Fixed issues with the stock app UI in AppsHub:
    • The UI sometimes did not show results of a completed calculation.
    • The UI blocked editing of a parameter if it has no default value.
    • A parameter value could be ignored if that parameter is an input port of some Composite block in the app's workflow.
    • The UI did not clear results of a previous calculation when you start a new one, causing confusion.
  • Fixed an issue with environment settings on extension nodes where pSeven Enterprise incorrectly set MKL_NUM_THREADS, NUMEXPR_NUM_THREADS, OMP_NUM_THREADS, and several other environment variables for the process it starts on the node. Those settings limited the number of CPU cores provisioned to the process, which could cause errors when running multithreaded CAD or CAE packages.
  • Fixed an issue where you could not copy a shared workflow if another user is editing a block in that workflow at the moment.
  • Fixed an issue with REST API where results of a finished run could be empty.
  • Fixed an issue where a workflow run could enter a deadlock, if the workflow contains a Python script block that declares a function but never calls it in the script, and the response port corresponding to that function has a link connected to it and actually receives function values, which the block does not use.
  • Fixed several bugs that sometimes caused unexpected behavior of common file management commands in Explorer - for example, the sequence of Ctrl+C and Ctrl+V hotkeys, which commonly creates a copy of the selected file or directory, was not working as intended.
  • Fixed minor UI issues in the Composite properties pane.
  • Fixed unexpected UI behavior when quickly opening and closing block configuration dialogs.

pSeven Enterprise upgrade notice

Since v2021.10, pSeven Enterprise licenses become bound to a specific Installation ID. A unique Installation ID is automatically generated during installation, and each deployment requires a license key valid for its Installation ID. The license key contains information about the ID, and can only be used for the deployment with that ID.

Deployments of pSeven Enterprise v2021.10 and future versions will keep their once generated Installation ID upon a version upgrade, so a deployment specific license for them has to be issued only once.

Deployments of prior versions (v2021.09 and earlier) have no Installation ID. For such deployments, a new ID is generated during a version upgrade, so the previously used license becomes incompatible as it was not generated for a deployment with a specific Installation ID.

If you upgrade an existing deployment to v2021.10, you will need to request an updated license valid for your newly generated Installation ID. To get the ID, sign in as admin after v2021.10 installation finishes and copy the ID string from the admin page header (see section License in the pSeven Enterprise deployment guide for more details). This is a one-time update required to switch to the new version of license, which supports Installation ID; future upgrades will keep the same ID.

v2021.09

Updates

  • Added initial ONNX support: the Python script block now provides the ONNX Runtime (import onnxruntime), which enables the usage of ONNX models from deep learning frameworks such as PyTorch and TensorFlow as well as classical machine learning libraries such as scikit-learn, XGBoost, and other.

    Using the ONNX Runtime in a Python script block that is configured to run on a Windows extension node is currently not supported.

  • The stock app UI in AppsHub now supports uploading input files and downloading files from app results.

  • Extended the Python script block functionality:
    • Connect to MongoDB databases with PyMongo (import pymongo).
    • Retry function calls (for example, HTTP requests) with backoff (import backoff).
    • Establish SSH connections with Paramiko (import paramiko).
  • Added caching support in the Python script block. For example, if your block runs in a loop, you can cache certain data or objects to avoid initializing them every time the block runs. See Caching Python objects in the Python script block guide for details.

    Note that the cache should not be used as a persistent storage, because it may be reset between block runs.

  • Added a few common convenience features to the code editor in the Python script block:

    • code folding support
    • highlight all occurrences of a word under the caret
    • Ctrl+/ to comment the current line or selected lines
  • NumPy arrays sent to output ports of a Python script block are now required to be 2-dimensional in order to avoid errors caused by inability to convert an array to the matrix data type. The support for N-dimensional arrays may be added in future versions.
  • Upgraded the embedded pSeven Core package to version 6.24. For details on recent improvements and changes, see the pSeven Core changelog.
  • The Run setup pane now supports tree-like grouping of parameters and results. For example, parameters named geometry.x and geometry.L are displayed as a collapsible group geometry containing inputs x and L.
  • Blocks in the Workflow library pane are now sorted by name, with different versions of the same block sorted in the version decreasing order.
  • Added block help shortcuts to the Block library pane: hover over a block in the library and click the icon that appears.
  • Improved AppsHub search to find search term occurrences in app descriptions. Also, you can search by username to find all apps published by a certain user.
  • Updated design of title bars in block configuration dialogs for consistency. Block title bar now always contains the Save and Help buttons. The Save button now saves configuration but does not close the dialog (use Alt+F4 to close). The Apply button is removed.
  • Added port tooltips in the Block properties and Composite properties panes, explaining the current state of each port.
  • If you open a folder in the Explorer pane and it takes a long time to load and display folder contents, the pane now shows a loading message.
  • To avoid errors, a driver block can now be grouped into a Composite block only if you also group to the same Composite the entire body of a loop controlled by that driver block (select the driver block and all blocks in the loop before grouping).
  • Added a message explaining the workflow run error caused by incorrect usage of conditional branching inside a Composite block with enabled batch processing.
  • Updated tooltips in the Explorer pane to better explain the use of commands, and added missing tooltips.
  • Removed the Open button from the collapsible information pane in Explorer as redundant.
  • Disabled the workflow run hotkeys F9 and F10 as they often conflict with browser hotkeys.
  • Updated pSeven Enterprise Help:

Deployment and administration

  • The Technical support section now explains what information you should collect and provide when contacting support with deployment issues.
  • Optimized resource requirements for pSeven Enterprise services in the Helm chart so the services are now easier to distribute among the Kubernetes cluster resources. Requirements for non-replicated services are now set in a specific way as required to give their pods a QoS class of Guaranteed, improving their stability (see QoS classes in the Kubernetes documentation).
  • In the pSeven Enterprise deployment configuration file (values.yaml), you can now set annotations which are added only to Kubernetes jobs and pods created by jobs - see the annotations.job parameter. The annotations added by annotations.job override annotations from annotations.all.
  • Removed memory limits for post-install jobs started by Helm, which run once to check and finalize the pSeven Enterprise installation. This change is done to avoid deployment failures, which sometimes occurred because a one-time post-install job exceeded its memory limit and was stopped by Kubernetes. Also, post-install jobs are now allowed to restart up to 3 times in case of errors.
  • Increased the ephemeral storage size for the pSeven Enterprise edge router service to avoid its restarts, which sometimes occurred under a peak load because the service log size exceeded storage size and Kubernetes destroyed the service.
  • Added branding options: you can now set a custom logo on the pSeven Enterprise sign-in page and in the navigation bar, and add a custom Getting Started page. See Branding for details.

User block development

  • Getting block source code (a .p7protoblock folder) from the block library now requires the app_auth | user | Can develop blocks user permission. New and existing users will have this permission by default, and a deployment administrator can revoke it. Users without this permission will be unable to drag blocks from the block library to the Explorer pane as well as add blocks from Explorer to the Workflow library pane.
  • Fixed an issue where you could not add a block from the Explorer pane to the workflow library if the block manifest did not define default values of all ports.

Bugfixes

  • Fixed an issue where the block UI sometimes did not load when you open the block configuration dialog.
  • Fixed the "Task scheduler service restart" error, which sometimes occurred while running a workflow.
  • Fixed errors and slowdowns when viewing large run logs.
  • Fixed an issue where running a workflow with a Python script block that produces many log messages resulted in a "Memory limit exceeded" error, regardless of the memory limit specified in block configuration.
  • Fixed an issue in the Python script block where function arguments in the Functions pane were listed in alphabetical order instead of their declaration order, causing confusion.
  • Fixed an issue in the Python script block where using the concurrent call syntax with 3 or more functions (for example, (F1&F2&F3)(x1, x2, x3)) caused an error.
  • Fixed an issue with the Python script block where writing a large amount of data to the block's temporary storage directory (api.tempdir) could cause an error in all running workflows.
  • Fixed an issue in the Python script block where it was possible to send an N-dimensional NumPy array to an output port, causing an error because an N-dimensional array cannot be converted to the matrix data type.
  • Fixed an issue with the variable quick add syntax in the Python script block where ranges in names of variables were sometimes parsed incorrectly.
  • Fixed an issue in the Python script block where names of variables added using the quick add syntax were not properly validated.
  • Fixed an issue where changing the value assigned to an uplinked port did not change the value assigned to the corresponding port of a parent block as intended.
  • Fixed the inability to group a block into a Composite block, if there are multiple links connected to the same port of the block you are grouping.
  • Fixed an error while running a workflow containing two or more Composite blocks with similar names, which have ports added to workflow results.
  • Fixed an AppsHub issue where re-publishing a workflow changed its app ID, which required you to also update all REST clients using that app.
  • Fixed an AppsHub issue where you could overwrite an existing app by publishing another workflow under the same name, even if you are not the owner of that app.
  • Fixed an issue with the stock app UI in AppsHub where descriptions of results appeared only after you run the app.
  • Fixed an AppsHub issue where you could not open an app if its description was saved with encoding different from UTF-8.
  • Fixed incorrect behavior of the Publish workflow dialog - for example, it allowed you to click the Publish button multiple times, resulting in repeated publishing of the workflow.
  • Fixed an issue where you could not create a copy of a read-only shared folder.
  • Fixed an issue where deleting a workflow sometimes caused an error, after which you could not delete that workflow.
  • Fixed an issue where workflow run folders were occupying more storage space than intended because they contained hidden system files.
  • Fixed a sign-in issue where re-opening the pSeven Enterprise sign-in page after a successful sign-in did not properly redirect you to pSeven Enterprise or AppsHub.
  • Fixed an issue where double-clicking a workflow, run, or report tab title did not open the corresponding folder in the Explorer pane as intended if the pane is collapsed.
  • Fixed an issue where you could not preview the app UI in a workflow (open the index.html file from the workflow UI folder in Explorer) if the workflow name or path to the workflow contain special characters.
  • Fixed an issue where you could not open an HTML file located in a run directory or the workflow Files folder.
  • Fixed incorrect status code and error message in a response to an invalid REST API file upload request.
  • Fixed an issue with renaming files and folders in the Explorer pane where it was impossible to select text with a mouse during rename because this started a drag-and-drop action.
  • Fixed a UI issue with the parameter preset selector in the Run setup pane, which sometimes displayed duplicated presets.

v2021.07

Updates

  • You can now enable or disable run-time resource management in your deployment when installing pSeven Enterprise or upgrading it to a new version. In pSeven Enterprise v2021.07, resource management is disabled by default. See the pSeven Enterprise upgrade notice below for details.
  • Adjusted resource requirements for pSeven Enterprise services in the Helm chart to their actual resource consumption based on the recent resource usage statistics gathered from real pSeven Enterprise deployments.
  • Improved pSeven Enterprise stability in many cases where users experienced issues when configuring blocks, editing or running workflows.
  • pSeven Enterprise administrators can now create users who have AppsHub access only. Such users may run apps from AppsHub but cannot create, edit, or run workflows.
  • When you download a workflow or a folder containing some workflows, you can now skip downloading the run results stored with workflows to speed up download and reduce its size.
  • You can now upload workflows or folders in ZIP archives and extract them in pSeven Enterprise using the new Unzip command from the menu in Explorer.
  • Upgraded the embedded pSeven Core package to version 6.23. For details on recent improvements and changes, see the pSeven Core changelog.
  • The Python script block now provides optional pandas dependencies like pytables, tabulate, and other.
  • Updated pSeven Enterprise Help:
    • Added section Workflow guides, which explains pSeven Enterprise concepts and provides a general user guide.
    • Added section Block guides, which provides help for the Python script and Condition blocks.
    • Added section User blocks guide, which provides an updated block development guide.
    • Minor improvements in the pSeven Enterprise deployment guide.

User block development

  • Fixed an issue where pSeven Enterprise displayed an empty Common library pane after a block with an invalid manifest had been published.
  • Fixed an issue due to which it was impossible to implement a block that allows changing default values of its ports.
  • Added the block development guide to pSeven Enterprise Help - see User blocks guide.
  • Removed the outdated User Block Example from AppsHub. Note that if you upgrade an existing deployment, this example is not removed automatically. It is recommended to delete it manually after the upgrade - see section Compatibility issues for details.

Bugfixes

  • Fixed an issue where run-time resource management did not work as intended when running a workflow that contains a block configured to run on an extension node.
  • Fixed a deployment issue where annotations specified in the pSeven Enterprise deployment configuration file (the annotations.all parameter in values.yaml) were not added to some of the Kubernetes pods during deployment.
  • Fixed an issue where saving a script with a syntax error in the Python script block produced a generic error message instead of a message that locates the error.
  • Fixed an issue where you could add a function with no arguments in the Python script block, which caused an error when running the workflow.
  • Fixed an issue where you could not change access rights of a folder you have shared earlier.
  • Fixed an issue where you could not preview the app UI in a workflow (open the index.html file from the workflow UI folder in Explorer) if the workflow had been shared to you by another user.
  • Fixed incorrect UI behavior and various issues that occurred when you can no longer access a shared workflow because its owner had deleted the workflow or revoked your access.
  • Fixed errors caused by deleting a workflow that is currently running.
  • Fixed an AppsHub issue where you could not open an app if its name contains a dot.
  • Fixed an issue where the pop-up message displayed after publishing a workflow to AppsHub contained an invalid link to the app.
  • Fixed an issue where you sometimes could not save configuration in the Example user block.
  • Fixed an issue where the Ctrl+V hotkey was not working in the workflow editor.

Compatibility issues

  • pSeven Enterprise v2021.07 disables run-time resource management by default. To enable the resource management features, which were introduced in v2021.06, set the sizing.enableResourceManagement parameter in the pSeven Enterprise deployment configuration file (values.yaml) to true. For details, see the pSeven Enterprise upgrade notice below.
  • The default maximum number of users (the maxUsers parameter in values.yaml) in pSeven Enterprise v2021.07 is increased to 256 (previously the default was 128). When upgrading, it is recommended to keep the new default value, unless a non-default value was specified in your existing deployment.
  • pSeven Enterprise v2021.07 no longer installs the User Block Example to AppsHub, as this example is outdated and is completely replaced by the new block development guide (see User blocks guide) and the example implementation of a user block available from the block library. However if you upgrade an existing deployment to v2021.07, the old example is not removed from AppsHub automatically. To avoid using that outdated example, it is recommended to manually delete it from the AppsHub data storage after you upgrade to v2021.07.

    • The AppsHub data storage location is specified by the storage.appdata.* parameters in the pSeven Enterprise deployment configuration file (values.yaml). Connect to the server specified by the storage.appdata.nfs.server parameter and locate the directory specified by storage.appdata.nfs.path. This is the AppsHub data storage directory.
    • To delete the outdated User Block Example, remove the User Block Example.p7wf subdirectory and all its contents from the AppsHub data storage directory.

      Warning

      Do not remove any subdirectories other than User Block Example.p7wf from the AppsHub data storage. Those subdirectories contain more example apps, as well as apps published by pSeven Enterprise users.

pSeven Enterprise upgrade notice

pSeven Enterprise v2021.07 deployment configuration file (values.yaml) is updated with a new parameter, sizing.enableResourceManagement, which you can use to enable or disable the run-time resource management features introduced in v2021.06 (default is disabled). This setting applies once during installation and cannot be changed afterwards, unless you re-install.

If you choose to enable resource management, configure it as described in the pSeven Enterprise upgrade notice for v2021.06.

If you choose to keep resource management features disabled, you can limit the maximum load by adjusting the sizing.htcondorexecuteSlots parameter, which sets the number of tasks that may run on a task execution service instance.

v2021.06

Updates

  • Improved run-time resource management:

    • Blocks now declare their memory and CPU requirements, and pSeven Enterprise uses this information to schedule block execution.
    • pSeven Enterprise sets run-time memory and CPU limits for blocks, allowing a block to use no more resources than it requires.
    • In Python script blocks, you can set per-block resource requirements in the new Resources section of the block configuration dialog.
    • User blocks can declare resource requirements in block manifest and can also implement per-block resource settings in their configuration dialogs.
    • If a block does not declare its resource requirements, default limits apply, set by an administrator in deployment configuration.

    Warning

    The resource management updates outlined above require you to update all blocks in workflows created in previous versions of pSeven Enterprise. To update a block, click the update notification next to the block's name in the Workflow library section of the Block library pane.

    If you do not update a block, the workflow will keep using an old block version, which does not declare resource requirements, so default limits always apply. Since the default limits are rather low, you may experience various issues if you run a workflow that contains old block versions.

  • You can now reduce the size of a workflow folder by removing the history of changes - open the workflow and use the Clear history command from the menu in the workflow edit toolbar.

  • Plotly is now available in Python script blocks.
  • To access files in Python script blocks, you can now use the api.rundir and api.wfdir variables that contain, respectively, the path to the current run directory and the path to the workflow folder. Import the api module (import api) to use. Both variables are pathlib.Path objects.

    Join paths

    The join operator for pathlib paths is /. For example, to get the full path to the file named file_name.ext, located at the root of the run directory:

    import api
    
    file_path = api.rundir / 'file_name.ext'
    
    Files in the block's working directory

    To work with files located inside the block's working directory, you do not need any special methods - just use relative paths. This works because the block sets its working directory as the current working directory for the script it runs.

  • Workflow run folders now occupy less storage space.

  • Improved responsiveness and loading speed of the Block properties pane.
  • Improved autolinking: the Autolink dialog suggests links between ports with partially matching names, and can match ports from different port groups.
  • Recently closed workflows and reports now appear on top in the list of recent documents, so they are easier to find.
  • Helm post-install hooks and jobs no longer disappear during deployment, so you can review their logs when needed.
  • pSeven Enterprise deployment no longer creates persistent volumes that require additional privileges.
  • Upgraded the embedded pSeven Core package to version 6.22. For details on recent improvements and changes, see the pSeven Core changelog.
  • Added more details about NFS server configuration to section File storage.
  • Reworked the Version upgrade guide to avoid issues with re-using configuration files from an existing deployment. Until further notice, the recommended method to prepare a configuration file for an update is to manually transfer settings from the existing configuration.
  • Updated the REST API guide:
  • Updated the Russian language version of pSeven Enterprise Help to bring it in line with the English language version. To select your language, use the menu in the Help's top bar.

User block development

  • Due to the resource management updates, blocks should also be updated so they declare resource requirements, and users should be additionally notified to update blocks in their workflows. See the User block upgrade notice for details.
  • Fixed an issue where users could not update a block in their workflows if the name of the folder where developers have published the updated block version was not following a specific pattern.

Bugfixes

  • Fixed an issue where pSeven Enterprise could stop responding due to a malfunctioning task scheduler service.
  • Fixed an issue where a workflow with blocks configured to run on a Windows extension node often stopped with an error, if these blocks were nested into a Composite block with parallel processing enabled.
  • Fixed performance issues caused by adding a large file to the workflow Files folder.
  • Fixed an issue where editing a block always reset its previously assigned port values to default, as well as resetting related workflow parameters.
  • Fixed an issue where pSeven did not correctly create working directories for a Composite block with working directory set to indexed and parallel processing enabled.
  • Fixed an issue where pSeven sometimes did not update a run tab properly, displaying a run as active (running) while it has already finished.
  • Fixed the unexpected UI behavior after opening several workflow tabs in rapid succession.
  • Fixed errors when deleting, moving or copying a folder that contains many subfolders.
  • Fixed an issue where it was possible to create duplicate links - for example, two links connecting the same port pair.
  • Fixed an issue where pSeven displayed incorrect values of block start-up counters on a run tab.
  • Fixed a few minor issues with the Links and Autolink dialogs.
  • Fixed an issue where pSeven kept displaying long-running operation pop-ups at the bottom.
  • Fixed an issue where a side panel with debugging information (Django admin panel) appeared in AppsHub and in pSeven Enterprise admin interface.

Compatibility issues

  • pSeven Enterprise v2021.06 changes the requirements to the file storage (NFS) server: now it supports NFSv3 only and requires disabling NFSv4 - see section File storage in the pSeven Enterprise deployment guide for details.

    Important

    Reconfigure the NFS server before upgrading to v2021.06, otherwise the v2021.06 deployment will fail.

  • Due to the resource management updates in pSeven Enterprise v2021.06, user blocks should also be updated so they declare resource requirements. Otherwise, the general memory and CPU usage limits set in the deployment configuration will apply. The general limits may be too low for some user blocks - for example, if a block processes large data arrays, it may run out of memory. Also, users should be additionally notified to update blocks in their workflows created in previous versions of pSeven Enterprise. For details, see the User block upgrade notice below.

  • pSeven Enterprise v2021.06 uses an updated deployment configuration file, which contains new parameters related to resource management. These parameters should be adjusted when upgrading to v2021.06. For configuration details, see the pSeven Enterprise upgrade notice below.

pSeven Enterprise upgrade notice

pSeven Enterprise v2021.06 deployment configuration file (values.yaml) contains several new parameters related to resource management. To ensure correct resource allocation and to avoid issues when running workflows, these parameters should be set deliberately and with regard to resources available in your deployment.

Follow the guidelines in section Upgrading pSeven Enterprise to generate a new configuration file for v2021.06 and transfer parameter settings from your existing deployment to the new file. In addition to this, find the following sizing.* parameters in the new file and adjust their values:

  • sizing.blockrunDefaultResources.memory - default memory usage limit for a block run task.
  • sizing.blockrunDefaultResources.cpu - default CPU usage limit for a block run task.
  • sizing.htcondorexecuteReplicas - the number of task execution service instances to create.
  • sizing.htcondorexecuteResources.limits.memory - total memory usage limit for a task executor.
  • sizing.htcondorexecuteResources.limits.cpu - total CPU usage limit for a task executor.

Limits set by the sizing.blockrunDefaultResources.* parameters apply to blocks that do not declare their specific resource requirements. Do not set these default limits too high as this leads to inefficient resource usage, which causes performance issues.

Values of the sizing.htcondorexecuteReplicas and sizing.htcondorexecuteResources.limits.* parameters should be selected carefully with regard to expected load and to resources that are in fact available in your deployment. For details, see comments to these parameters in the v2021.06 configuration file (values.yaml).

User block upgrade notice

User blocks developed for prior pSeven Enterprise versions do not declare their resource requirements as expected in pSeven Enterprise v2021.06. Due to this, default run-time memory and CPU limits will apply to these user blocks unless you prepare an updated block version and add it to the block library. The default limits may be too low for some user blocks, so users may experience issues when running workflows that contain old block versions.

Before you upgrade pSeven Enterprise to v2021.06, it is recommended to update each user block, adding run-time resource requirements to the block's manifest, and upload the updated version to the block library beforehand - so the block update is readily available to users once you upgrade to v2021.06.

  • Make a copy of your .p7protoblock directory, which will be the new block version. For example, if you have a directory named TheBlock-v0.p7protoblock, copy it to TheBlock-v1.p7protoblock.
  • In the copy (TheBlock-v1.p7protoblock) open the block's manifest (manifest.yaml) and edit the following parameters:

    • Under default_preset.template, add the resources parameter specifying the block's requirements as in the following example:

      default_preset:
          template:
              # Do not copy this example as is.
              resources:
                  run:
                      memory: 1024
                      cpu: 250
              # Other template parameters follow...
      

      Note that the memory and CPU requirements must be set according to the real block memory and CPU usage. Memory is specified in MB, so memory: 1024 sets the memory requirement to 1 GB. CPU requirement is specified in millicores, so cpu: 250 sets it to 0.25 vCPU.

      When adding the resources parameter, do not remove any of the existing parameters under default_preset.template.

    • After specifying the resource requirements, increase the version parameter by 1 - for example:

      # version: 2
      version: 3
      

    Important

    Do not change the id parameter in the manifest - otherwise pSeven Enterprise will recognize your update as an entirely different block, and users will not be able to update blocks in their workflows automatically.

    Also do not change the manifest_version and protocol_version parameters - they are reserved for future use and should not be edited.

  • Save your changes to manifest.yaml and upload the new block version (TheBlock-v1.p7protoblock) to the pSeven Enterprise block library (the /protoblocks directory in the shared data storage). This updated version will appear in the Common library section of the Block library pane in the pSeven Enterprise user interface. In workflows that contain an old version of the block, the Workflow library section of the Block library pane will show the old version along with an update notification.

  • After upgrading to pSeven Enterprise v2021.06, notify your users that they need to update all blocks in their workflows created in previous versions of pSeven Enterprise - otherwise they may experience issues if they run a workflow that contains old block versions.

    To update a block in a workflow, click the update notification next to the block's name in the Workflow library section of the Block library pane.

v2021.05

Updates

  • Workflow and report downloads now run faster and have less size: when archiving the document to download, pSeven no longer adds the history of document changes to the archive. Previously, downloads always kept a full history, which could greatly increase the archive size.
  • Added a few convenience improvements related to configuring and linking blocks that have driver ports - for example, Python script blocks with function and argument ports:
    • Driver ports and generic ports are now sorted separately in the Links and Autolink dialogs.
    • Output driver ports are labeled "Request": a block uses them to request evaluations from other blocks connected to it.
    • Input driver ports are labeled "Response": they accept responses (evaluation results) from connected blocks.
  • Upgraded the embedded pSeven Core package to version 6.21. For details on recent improvements and changes, see the pSeven Core changelog.
  • Reduced the extension node setup package size. Removed a few unused versions of packaged components, which could cause issues during a node deployment.
  • Updated the pSeven Enterprise deployment guide for more clarity:
  • Updated the Extension node deployment guide, added section Allowed users (moved: see Configuring user authorization on extension nodes).
  • Added the Russian language version to pSeven Enterprise Help. To select your language, use the menu in the Help's top bar.

Note

pSeven Enterprise Help is now also available at the DATADVANCE website. See www.datadvance.net/product/pseven-enterprise/docs/en/ for the latest version.

User block development

  • When block developers update a block in the common library, users can now update this block in their workflows in a few clicks - instead of manually replacing old versions of the block. pSeven shows block update notifications in the Workflow library pane; click the notification to start the update.
  • Fixed issues in block manifest validation, due to which you could add an unusable block with an invalid name to the block library, or add a block with an invalid id, which you then could not delete.
  • Fixed an issue where the Block library pane sometimes displayed wrong icons for user blocks.

Bugfixes

  • Fixed an issue where any user with admin access could see workflow runs of other users on the Start tab, and an attempt to open one of these runs led to an error.
  • Fixed a rare issue where a workflow run did not start but pSeven did not show any error, so the run appeared to be waiting indefinitely.
  • Fixed a stability issue which could disable editing and running a workflow if there was an error while editing this workflow, so you had to create its copy to continue.
  • Fixed an issue where pSeven was showing an incorrect inline preview for a matrix value - for example, port values in the Block properties pane or parameter values in the Run setup pane.
  • Removed dummy test blocks from the block library.
  • Fixed Enter not working as a confirmation in the file deletion dialog.

v2021.04.20

Updates

User block development

  • Deployment administrators can now publish user blocks. Copy a user block to the directory named protoblocks in the shared data storage to make it available to all pSeven users.

    The storage server address and the shared data directory path are specified by the storage.shareddata.* parameters in the deployment configuration file (values.yaml).

Bugfixes

  • Fixed an issue in the Python script block, due to which it sometimes did not save configuration changes.
  • Fixed a bug due to which unfinished Kubernetes jobs were building up and could consume all resources, potentially leading to a general failure.
  • Fixed a bug in UDP package routing, which could cause instability of the OpenVPN connection between pSeven and Windows extension nodes.

Compatibility issues

  • Since this release, pSeven supports only NFS file storages. If your current deployment uses a persistent volume claim or a custom storage class, you will have to set up an NFS storage and transfer all files there before upgrading to v2021.04.20. Note the requirements listed in section File storage of the pSeven Enterprise deployment guide.
  • If you use extension nodes, you will have to manually remove HTCondor on each node before updating it with the new node setup package - see the Extension node upgrade notice below.

Extension node upgrade notice

The version of the HTCondor service in the v2021.04.20 extension node setup package is not compatible with previous installations, so you will have to remove HTCondor on each of the extension nodes manually before upgrading.

  • Open the Windows Control Panel - Programs and Features (or run appwiz.cpl to open Programs and Features directly). Find HTCondor in the list and uninstall it. If the uninstaller prompts for a restart, reboot the node before you continue.
  • Get the new pSevenExtensionNode.zip setup package from pSeven (see Setup package).
  • Unpack pSevenExtensionNode.zip to a temporary directory. The path to this directory must not contain spaces, for example: C:\Temp\pSevenExtensionSetup.
  • From this directory, run bootstrap.bat without parameters. The setup scripts will use settings from the previous installation, which are stored into environment variables on the node.
  • When bootstrap.bat finishes, check its output for errors. If there are none, reboot. If you see errors, copy the script's output from the command prompt and send it to support@datadvance.net including a brief description of the problem.
  • After the reboot, check the node as described in the Extension node deployment guide, section Node check.

v2021.04

Updates

  • It is now possible to restrict user access to a Windows extension node connected with pSeven - see section Configuring user authorization on extension nodes for details.
  • Added the Stop button to the stock app UI in AppsHub. You can use that button to stop a running app. Also improved the stock app UI stability in case of connection issues.
  • Workflow and report copy now works faster, and newly created copies take less space because pSeven no longer copies the history of changes with the document. The original document keeps a full history, but the history in a copy will contain only newer changes.
  • Added the Getting started page, which provides a quick overview of pSeven's functions and features and links to more guides. To open it, click the Getting started button on the Start tab.
  • Since reports in pSeven are currently limited, and many of their planned features are missing, the New report dialog now contains a beta notice to avoid misunderstanding.

User block development

  • Fixed an issue where adding a user block to a workflow created duplicates of this block in the workflow library.
  • Fixed an issue where you could not copy and paste a user block in a workflow if the block contains a large amount of code or embedded data.

Bugfixes

  • Fixed slowdown issues, which could occur when running a workflow where a Composite block receives or outputs a large data array - for example, a 100×100 matrix.
  • Fixed the inability to save a Python script block that contains a very long script.
  • Fixed an issue with the stock app UI in AppsHub where the UI reported a timeout error if an app run took longer than 10 minutes.
  • Fixed a bug due to which a block in a running workflow failed to start if you have copied this block from another workflow.
  • Fixed an issue where pSeven sometimes was not showing notifications until you refresh its tab in your browser.

v2021.03

Updates

  • Convenience improvements in the Links and Autolink dialogs:
    • Quicker linking for ports with matching names. Ctrl-click and drag when creating a link to instantly open the Autolink dialog. In this case, OK in the Autolink dialog links selected port pairs and closes the dialog.
    • Switch between autolinking and manual linking. In the Links dialog, click Autolink to open the simplified dialog. In this case, OK in the Autolink dialog selects the links to create and switches back to Links, where you can continue with manual link editing.
  • The Python script block now automatically formats the script code to comply with PEP 8 and checks the code for syntax errors, when you click Apply or Save in the block's configuration dialog.
  • OpenTURNS support. You can now use the openturns module in Python script blocks and user blocks implemented in Python.
  • The Block properties pane now shows inputs and outputs sorted in the order defined by the block. You can also switch to alphabetical order using the sorting buttons in the pane. For grouped ports, alphabetical sort order applies to port names in a group but does not change the order of groups.
  • When a message receive timeout is exceeded, the api.message_pop() REST API method in the Python script block now raises a TimeoutError instead of a generic exception. If you use messaging in Python script, it is recommended to update your scripts to specifically handle the TimeoutError from api.message_pop().
  • Added logging for block editing operations. When you add, reconfigure, or delete a block, pSeven writes debug information to edit log.txt in the workflow folder.

User block development

  • User blocks can now access the Redis database that provides the REST API messaging functionality. This makes it possible to add your custom messaging implementation to a user block, functionally similar to the api.message_push() and api.message_pop() methods available in Python script blocks. The database hostname and port are available to blocks as the DA__P7__BLOCK__REDIS_HOST and DA__P7__BLOCK__REDIS_PORT environment variables.

Bugfixes

  • Fixed various issues with incorrect run order and statuses displayed on the Start tab.
  • Fixed an AppsHub issue where you could not edit some parameters in the stock app UI.
  • Fixed an issue with the stock app UI where an app could not start if there is a parameter that accepts any data type.
  • Fixed a vulnerability issue in the messaging mechanism, which is in particular used by the Python script block REST API implementation. Before the fix, exploiting this vulnerability allowed a user to access messages of Python script blocks from workflows of other users, and to access messages used internally by pSeven. This could cause data loss in workflows and general malfunctioning. The fix properly isolates messaging for every user.
  • Fixed a bug in the Python script block REST API, due to which api.message_pop() returned a bytearray instead of a string.
  • Fixed an issue where pSeven could show a failed workflow run as active (running) indefinitely if the run fails to start because of a block start-up error.
  • Fixed a similar issue where pSeven could not handle a block error when you open the block to edit its configuration.
  • Fixed pSeven showing a workflow run as active while it has failed due to an internal error. Now pSeven automatically cancels runs that stopped responding and shows them as failed.
  • Fixed a similar issue where pSeven did not properly stop a block if there was an error while editing its configuration. Now pSeven automatically stops a block running in the edit mode if it becomes unresponsive.

Compatibility issues

  • Since api.message_pop() in a Python script block now returns a string as intended (was bytearray), you may be required to update your scripts to correctly handle the return value, depending on how it is processed.

v2021.02

Updates

  • Enabled the intended functionality of the Files folder in a workflow. This special folder is a prototype of the workflow's working directory:
    • When you configure block's working directories, pSeven creates folders in Files - so you can preview the working directory structure while editing a workflow. When you move, rename, or reconfigure blocks, pSeven automatically updates the folder structure in Files.
    • You can upload data to Files and to folders inside it to make this data available to blocks during a workflow run. When a workflow starts, pSeven copies the files and folders from Files to the workflow run directory, creating the initial state of block working directories.
    • Running blocks work with copies of files in the run directory, so you can continue editing the workflow and Files as needed, even while a workflow run is active.
    • Note that if a block's working directory type is "indexed", the block creates a new (numbered) working directory every time it starts. These directories have a single prototype in Files - a folder with the [####] name suffix. When you run a workflow, pSeven copies this folder's contents to every new working directory created by the block.
    • To avoid needless copying, place read-only files somewhere outside of the Files folder.
  • Simplified behavior of the block linking dialogs. The Autolink dialog no longer opens when you create a link, and you can switch to autolinking with the Autolink button in the Links dialog. Creating or editing a link now always opens the full Links dialog.
  • The Python script block now validates names of variables, functions, and arguments to avoid syntax issues and name conflicts.
  • Added a command to download the run log file, available in the Run logs pane menu.
  • REST API clients can now download workflow run logs. To get a download URL, request run details and read logfile_url from the response.

User block development

  • Fixed the inability to set a port value if its data type is a custom table with predefined columns.

Bugfixes

  • Fixed licensing issues, due to which any user could start an unlimited number of workflow runs or calculations in AppsHub, potentially consuming all resources and disrupting other user's activity.
  • Fixed a bug with messaging support in AppsHub, due to which an app did not work if its workflow contained a Python script block that uses messaging (the api.message_push() and api.message_pop() REST API methods).
  • Fixed an AppsHub issue where you could not open the Edit value dialog for an app parameter that currently has an invalid value.
  • Fixed an AppsHub issue where an app could not start due to an error in processing a port that is both linked and selected as a parameter.
  • Fixed a GUI error that sometimes occurred when deleting a block from a workflow.

Compatibility issues

  • Due to bugfixes in the license mechanism, this version requires an updated license valid for v2021.02 or above.

v2021.01

Initial release.