Release Notes 2021.4.3

Release Highlights

The goal of the Incorta Cloud 2021.4.3 release is to enhance predictive analytical capabilities, data management, security, automation, and performance. To that end, the 2021.4.3 release introduces a new Incorta Component Software Development Kit that enables you to develop custom visualizations to use in building insights and dashboards. In addition, this release introduces new Analyzer features, time series forecasting and anomaly detection, to build insights using predefined machine learning models. It also introduces a new Cluster Management Console (CMC) infrastructure monitoring tool, a new data lake connector for Google Cloud Storage (GCS) as well as enhancements to the SAP ERP connector, Analyzer, and Materialized Views (MVs).

New Features

In addition, this release includes several enhancements and fixes, and a few known issues and workarounds.

Incorta Component Software Development Kit

In this release, Incorta introduces a new major feature: Incorta Component Software Development Kit, or Incorta Component SDK. The Incorta Component SDK enables Incorta customers and partners to develop custom visual and non-visual components that plug-in to Incorta Analyzer and Dashboards. Using this new feature requires React knowledge as you will be developing HTML, CSS, JSON, Javascript, Node.js, NPM, React, and Typescript.

Important

You can currently test component samples, but you cannot yet add them to the marketplace. This capability will be available in a subsequent release.

For more details and information about the Incorta Component SDK and how to use it, refer to References → Incorta Component SDK.

Time Series Forecasting and Anomaly Detection using Machine Learning for Line visualizations

The Analyzer now supports time series forecasting and anomaly detection using machine learning for line visualizations. You can generate multiple forecasts and detect anomalies in time series data for different product lines or divisions.

Here are the steps to use the Machine Learning features:

In the Insight panel, under Charts, select Line.
From the Data Panel, drag and drop the following columns to the respective tray:
- A date or timestamp column to the Grouping Dimension tray.
- A double or integer column to the Measure tray.
In the Data panel, expand Add Machine Learning.
Drag and drop Time Series Forecasting and Anomaly Detection to the Machine Learning tray.
In the Time Series Forecasting Properties panel, for Model, choose Auto (includes all models), Auto Arima, FB Prophet (Facebook Prophet), or Exponential Smoothing.
In the Action bar, select Settings(gear icon) to open the panel.
In the Machine Learning section,
- Auto-run ML Models and View Forecast are enabled by default.
If you disable Auto-run ML Models, select Run ML Models (flower icon) in the Analyzer’s Action bar or on the Dashboard to run the machine learning models, and then view the summary.
- For Forecast Data Points, enter a value in the spin box. The default value is 10.
Select Save.
On the Dashboard tab, hover over the Line insight, and select ML Summary (flower icon) to view the forecasting and anomaly detection summaries.

For more information on creating a line insight, refer to Visualizations → Line.

Important

To use the Machine learning features, perform the following steps:

Start the Cluster and Analytics service in the CMC.
Contact the Support team to enable One-click ML on the Cloud Admin portal.
Restart the Analytics service.

Note

Machine Learning is an Incorta Labs feature that is currently experimental.

Visualization enhancements for Donut Total and Table style and appearance

The Analyzer now supports the following features in the Settings panel:

A new Total setting for the Donut visualization, which enables you to show the total of a measure in the white space inside of the Donut. In addition, you can also choose whether to hide or show the measure name using the setting Show Measure. For more information, refer to the Visualizations → Donut.
A new Style and Appearance section for configuring tables, such as font size, color, and alignment of headers and values. For more information, refer to the Settings Panel section in the Listing Table, Aggregated Table, and Pivot Table documents.

Backslash support as an escape character

The Engine now supports using the backslash mark \ as an escape character that you can use within a string in a formula to treat the next character as literal text. For strings that you enclose in double quotes and include double quotation marks within, you must precede each quotation mark that you want to include in the string with a backslash \. The same applies to single quotation marks within a string enclosed in single quotes. In addition, to include a backslash in a string, precede it with another backslash as an escape character.

Here is an example:

The formula:

concat(
   "Place a string between two double quotes: \"TEXT\"",
   ' or two single quotes: \'TEXT\'.', "Use a backslash \\ as an escape character."
)

The Result:

Place a string between two double quotes: "TEXT" or two single quotes: 'TEXT'. Use a backslash \ as an escape character.

Important

After upgrading to the 2021.4.3 release, existing formula columns with a backslash \ as part of the formula text will display no data. In addition, when you try to save a formula with a single backslash in the Formula Builder, this will show errors. You must precede each backslash by another backslash. The same error appears with one single quote in a string enclosed in single quotes and one double quote in a string enclosed in double quotes without a preceding backslash.

Infrastructure Monitoring tool in the Cluster Management Console

Incorta is introducing a new Infrastructure Monitoring tool in the Cluster Management Console (CMC) that enables you to view your infrastructure utilization graphs and insights per cluster. You can monitor the Metadata Database, Spark, Zookeeper, and Notebook services. For more information, refer to Tools → CMC Monitoring.

Google Cloud Storage Connector

A new connector to support the use of Google Cloud Storage (GCS) as a data lake data source is now available. To learn more, refer to Connectors → Google Cloud Storage.

SAP ERP Connector enhancements

The SAP ERP connector now includes the following:

A Datasource Timezone property, which preserves the date, time, or timestamp from the datasource to Incorta.
Support for different full and incremental load queries with the same number of columns. The Full Load Derived Columns and Incremental Load Derived Columns properties are available to add columns post-extract for cases in which the full load query and incremental load query have differing numbers of columns.
The option to choose the backend database fetch method for full and incremental loads, which now includes OPEN_CURSOR for improved performance.
New Chunking Column Type options:
- Chunking by Year and Period columns
- Full Load by Filter

For more information, refer to Connectors → SAP ERP.

Automatically force load jobs stuck in the interruption process to abort

In this release, Incorta automatically aborts any load job that is interrupted in the extraction phase and exceeds the configured interruption time interval. This removes the stuck physical schema from the Schema Pool, which allows users to start a new load job of the related physical schema and allows loading other physical schemas as well. The status of this load job is Aborted.

The default time interval to automatically abort stuck interrupted load jobs is 60 minutes. You can ask the Incorta Support team to edit the service.properties file that exists in the directory of each Loader Service and define another time interval in minutes. The following key controls the time interval: abort.interrupted.job.wait.time.

Different incremental load strategies for Materialized Views

This release supports two different incremental load strategies for MVs. In incremental load jobs, the Loader Service instructs Spark to fetch only new and updated records. You can load MVs incrementally depending on one of the following:

The MV’s last successful transformation time; available in previous releases
The maximum value of a column in the MV

For more information, refer to Concepts → Materialized View → Incremental load strategies.

Limited access to the Business Schema Manager for Analyze and Individual Analyzer users

In this release, users with only the Analyze User or Individual Analyzer roles will have limited access to the Business Schema Manager where they can view a list of business schemas shared with them without the need to be assigned the Schema Manager role. They can also export shared business schemas and view their description and sharing configurations.

These users can also access the runtime business views in the Business Schema Designer View Mode where they can view the runtime business view columns and invoke the Analyzer to explore their data and create insights.

Encryption and decryption support in Scala Materialized Views

In this release, you can reference an encrypted column in a materialized view that is defined using Spark Scala. In addition, you can turn on the encryption property of a column in a Scala MV using the Table Editor.

When reading an encrypted column referenced in a Scala MV or an encrypted column in a Scala MV, while rendering a dashboard, for example, decrypted data is correctly displayed.

Additional Enhancements and Fixes

Physical Schema

Fixed the error message that displays when a schema column is changed to an invalid data type to always include the column name
Fixed a case where two consecutive update jobs within a small time interval caused the failure of one of the update jobs
Fixed an issue with the table columns in which selecting View Details (eye icon) of a column did not preview sample data

Dashboards

Enhanced the dashboard performance after materializing formula columns
Fixed an issue in which exporting tables to a PDF format did not draw the table gridlines
Fixed an issue with the drill down option in the Heatmap layers of an Advanced Map insight
Fixed an issue in which filtering a dashboard that has a presentation variable with a valid session variable does not return data
Fixed an issue in which an error message appears when a dashboard is moved from one content folder to another
Fixed an issue in which no error message appears when you remove a dashboard that is linked as a drill down, and then attempt to drill down on the same dashboard for Listing, Pivot, and Aggregated tables
Fixed an issue with the AR Summary dashboard of the EBS data application in which the dashboard displays an error due to an undefined session variable

Analyzer

Enhanced the performance of the ago and toDate time series analytic functions
Fixed an issue in which an error occurs while trying to open the Query Plan Viewer of a formula in the Aggregated Filter tray
Fixed an issue in which a join between a Materialized View table and an Alias table is broken in the insight’s Query Plan after updating the physical schemas

Data Agent

Fixed an issue in which the data agent could not be started

Scheduler

Fixed an issue in which users where unable to delete emails from the CC and BCC properties of scheduled dashboards

Materialized Views and SQLi

Fixed an intermittent issue with ODBC connections, in which the “message contents do not agree with length in message type "N"” error occurs

Miscellaneous

Fixed an issue in which logs consumed the disk space
Fixed an issue with sending dashboards via email in Excel or CSV format, in which the default value of a presentation variable is not equal to the session variable value in the email body
Fixed an issue in which an SSO application could not sign in to an Incorta tenant when the tenant name is case insensitive in the URL
Fixed an issue with previewing data in Dark Mode
Fixed an issue in which long usernames on the Home page banner became truncated
Fixed an issue with the Analytics Service in which the log displays error messages when a formula on prompt is created with a constant value
Fixed an issue with the search in the Individual Filter or Runtime Filter, in which a keyword preceded by a comma and space does not retrieve any search results
Fixed an issue with the Boolean functions in which false is always returned when the boolean function is used with an SQL Business schema view

Known Issues and Workarounds

The following list illustrates the known issues in this release, with a workaround if it is available:

Loading a physical schema after removing columns that are used in joins will show a Success load status instead of Finished with errors, even when the Stop on Error toggle is enabled. As a workaround, do not delete any columns that are used in joins to successfully load your physical schema.
Opening the SQL Query dialog of a Table with NetSuite Suite Analytics as a data source takes several minutes to load.
Email addresses added as CC or BCC in a scheduled data notification are not saved.
Synchronization jobs that occur during rendering a related insight on a dashboard cause the rendered insight to fail.
Adding columns in a specific order to the Grouping Dimension tray of a Listing table, and then enabling Subtotal cause the “INC_04050735: Unexpected error while sorting groups” error message to appear in the Visualization canvas of the Analyzer.
Some physical schemas experience a prolonged loading time.