Release Notes 2023.4.x

Release Highlights

In this release, Incorta is introducing the dashboard free form layout, as well as major changes in the dashboard manager’s user experience. This release provides improved resilience for the Loader Service, resulting in a huge enhancement in memory handling during load jobs.

Incorta is also introducing the query timeout feature that’s available through a configuration in the Cluster Management Console (CMC). You can also now disable users through the Security Manager.

In addition, Incorta is introducing other new features that aim to enhance Incorta’s performance, decrease exceptions, and empower analytics capabilities.

The 2023.4.1 maintenance pack include additional features and enhancements.

Upgrade considerations

SQLi queries with user-defined joins or conditions

Incorta has implemented an enhancement to address SQLi queries involving user-defined or unresolved joins within Incorta Engine. In scenarios where the Engine cannot identify a suitable query plan, it falls back to Spark and executes the original user-defined query. However, it's important to note that as a result of these improvements, the outcomes of existing queries may vary compared to previous releases. To maintain consistency with previous results, it is recommended to review these queries accordingly.

API authentication

Starting with 2023.4.0, Incorta introduces a new public API version (v2) that you can use with one of two new authentication methods: personal access tokens (PATs) or Java Web Tokens (JWTs) generated by an OAuth 2.0 authentication server. Public API v1 isn't compatible with the new authentication methods. You can't use the 2023.4.0 release to generate new API keys or renew existing ones that are used in Public API v1. However, you can refresh existing access tokens, or you can use the latest API key that you generated before upgrading to 2023.4.0 and use it to create new access tokens for Public API v1 as usual. If you don't have an API key generated before the upgrade, you can use only Public API v2.

New Features

Dashboards, Components, and Analytics

• Dashboard free form layout
• Individual and dashboard filter optimization
• Query timeout
• Support Aggregate filter in Pivot table components
• Support abbreviating measure values
• Tooltip tray in components
• Background conditional formatting for KPI components

Architecture and Application Layer

• Enhancing memory handling during load jobs
• Enabling Incorta Engine to read from compacted parquet files
• Solving the discrepancy between compacted and committed versions
• Deactivate users
• Allow Cloud Diskspace Monitoring configuration
• Faster formula calculations

Data Management Layer

• Business Schema Designer enhancements
• A new API version with enhanced authentication
• New API endpoints
• Replacing the Load Job Viewer
• Displaying the load job summary
• Data lineage enhancements

Dashboard free form layout

With this release, Incorta is introducing a new way of creating and managing dashboards. It is also enhancing the user experience when creating or editing a new dashboard. With this change, you have more flexibility and easiness to design your dashboard. For more information, refer to Dashboard Manager.

Individual and dashboard filter optimization

You can now enable Individual and dashboard filter optimization in the Tenant Configurations > Tuning to improve the performance of filtering data, and accordingly reduce the overall rendering time of insights and dashboards. This feature optimizes filtering data using insight individual filters and when applying runtime security filters and dashboard filters (including dashboard runtime filters, applied filters, dashboard prompts, and filter options).

This feature is disabled by default. To enable it, contact Incorta Support.

Query timeout

New configurations are now available in the Cluster Management Console (CMC), Tenant Configurations > Tuning > Query timeout and Query timeout (in minutes), to set a time limit after which running queries will be terminated automatically, releasing the read lock acquired on the related entities and freeing the resources reserved by these queries. These configurations affect querying data in the following scenarios:

• Creating an insight
• Rendering a dashboard
• Sending a dashboard via email or to a data destination
• Downloading a dashboard as CSV or Excel
• Previewing a chart as a listing or aggregated table
• Creating or updating an Incorta Analyzer table or view
• Exploring the data of a table or view
• Creating a data notification condition

The system tracks queries that are automatically terminated, and it records them in the system logs. In addition, a message will be displayed if applicable.

Support Aggregate filter in Pivot table components

Incorta now supports aggregate filters for Pivot table components. For more information, refer to Visualizations → Pivot Table.

Support abbreviating measure values

In this release, Incorta is supporting abbreviating measures and enabling you to configure the how the abbreviation behaves. For more information, refer to the 2023.4.0 What’s New guide.

Tooltip tray in components

The tooltip tray is now available in more components. The supported components in this release are:

• Sunburst
• Combination
• Time series
• Line time series
• Area range
• Stacked column and line
• Dual x-axis
• Waterfall
• Bubble map
• Funnel
• Pyramid
• Packed bubble
• Organizational

For more information, refer the corresponding document for each visualization.

Background conditional formatting for KPI components

Background conditional formatting is now available for KPI components. You can apply the conditional formats to a measure column in a KPI component for both font color and background. For more information, refer to Visualizations → KPI.

Deactivate users

Incorta administrators (Super User / User Manager) can now deactivate Incorta analytics users instead of totally deleting them. Deactivating users will not delete users’ data, instead, it will keep the data saved and usable for others. Also, deactivated users’ scheduled jobs will remain active unless the administrator chooses to suspend them.

For more information, refer Tools → Security Manager.

Enhancing memory handling during load jobs

This release provides improved resilience for the Loader Service, which enhances memory handling during load jobs. The out-of-memory (OOM) exceptions will be reduced significantly by pausing the memory-bound tasks during a load job if the off-heap memory reaches a defined threshold and applying a retrial mechanism if a memory-bound task fails because of an OOM exception.

Enabling Incorta Engine to read from compacted parquet files

Incorta Engine can now fully read columns from compacted parquet files if the compacted version is consistent with the committed version. However, it will read columns from the source parquet files when reading columns incrementally or when there is a discrepancy between the compacted version and the committed version. This feature unifies the source that both Incorta Engine and Spark read from. It also decreases the I/O operations and enhances the performance of reading columns when a table has too many duplicates.

Solving the discrepancy between compacted and committed versions

To ensure that Incorta Engine and Spark have access to the same version of data at a point in time, the Loader Service will make sure that both the compacted and committed (loaded) files of a physical schema object share the same version. The Loader Service synchronously integrates the compaction step of performance-optimized tables in the load job process. If the compaction of an object fails, loading the object fails as well.

Deactivate Users

Incorta administrators (Super Role / User Manager) can now deactivate Incorta analytics users instead of totally deleting them. Deactivating users will not delete user’s data, instead it will keep the data saved and usable for others. Also, deactivated user’s scheduled jobs will remain active unless the administrator chooses to suspend them.

Allow Cloud Diskspace Monitoring configuration

Incorta is adding a new configuration to the CMC. The Allow Cloud Diskspace Monitoring option is under Clusters > Cluster Configurations > Server Configurations > Diagnostics and is disabled by default. For more information, refer to Guides → Configure Server.

Faster formula calculations

This release improves the parallel formula materialization process to ensure faster calculations of grouping dimension formula columns that reference string columns with a large number of records. This enhancement affects dashboard insights, Analyzer tables, Analyzer views, and data notification conditions.

Business Schema Designer enhancements

This release enhances the Business Schema Designer by introducing a new user interface (UI) and improved features. The new UI enriches the user experience and facilitates managing views in a business schema. The new UI has the same look and feel as other components, such as the Analyzer and the Catalog (Content Manager), which provides a consistent user experience.

Two modes are available when dealing with a business schema: View mode and Edit mode. In addition, you can now add, edit, and delete business schemas when the Loader Service isn’t available.

A new API version with enhanced authentication

The 2023.4.0 release brings multiple enhancements to Incorta’s public API. These enhancements include the following:

• A new version of Incorta Public API (v2) is now available for seamless integration.
• Enhanced public API authentication, where you can create and maintain up to 10 personal access tokens (PATs) that you can directly use in the request header for authentication. PATs are compatible with Incorta Public API v2 only. Active access keys created before upgrading to 2023.4.0 work only with Public API v1.
• Optionally, you can use OAuth 2.0 as your authenticator for public APIs, knowing that OAuth 2.0 works with Public API v2 only.

For more information, refer to References → Public API v2.

New API endpoints

The following are the new public API endpoints in 2023.4.0:

• Get SQL connection (/configs/sqlConnection): Gets the connection string and port number of Incorta SQL Interface (SQLi).
• Query a business view (/query): Queries a single business view into a listing table, aggregated table, or Pivot table.
• List schemas (/schema/list): Returns the list of physical and business schemas that the user owns or has access rights to.
• List schema objects (/schema/{schemaName}/list): Returns the tables, views, and included columns in a specific physical or business schema that the user owns or has access rights to.

Replacing the Load Job Viewer

The Load Job Details Viewer introduced in 2022.12.0 replaces the Load Job Viewer starting this release. Now, when you select the status or time of the latest or current load job on the Schema Manager → Schemas or the Schema Designer or you select Job Details of a failed, interrupted, or finished with errors job on the Home page, the job opens in the Load Job Details Viewer.

Displaying the load job summary

The Load Job Details Viewer now shows the summary of the selected load job. This summary includes the duration of the load job as a whole and of each stage, the number of schemas, and the number of tables.

Data lineage enhancements

You can now find out whether your columns are referenced in an insight or in the dashboard filters or prompts. The Data Lineage Viewer shows the insight name, if any, or the dashboard name depending on the dependency type.

Additional enhancements and fixes

In addition to the new features and major enhancements mentioned above, this release introduces some additional enhancements and fixes that help make Incorta more stable, engaging, and reliable.

Enhancements

Fixes

2023.4.1 Maintenance Pack

Copy dashboard link

Incorta is introducing a new method to share a dashboard with other users. Now you can share a dashboard link via copy and paste.

You can find the Copy link option in the following places:

• The More options (⋮ vertical ellipsis) icon in the Content manager page beside a dashboard.
• Share icon in a dashboard’s Action bar.
• Dashboard’s tab More options (⋮ vertical ellipsis) icon.

In sharing the dashboard link, dashboards filters and bookmarks by default. By selecting “Include current filters in the URL” option, the filters will be added to the URL string. This can be especially useful when leveraging URLs from third-party tools.

SAP ERP connector callback extraction mode

Incorta is introducing a new callback extraction mode in the existing SAP ERP connector. The new extraction mode is faster and easier to set up.

For more information refer to Connectors → SAP ERP.

Performance enhancement for rendering and downloading insights

• Queueing of download and render requests: The Analytics Service now queues requests for the same insight with the same format, preventing simultaneous execution of duplicate requests.
• In-memory caching of downloaded insights: the Cluster Management Console (CMC) admin can configure the Analytics Service to maintain an in-memory cached version of insights downloaded, sent to data destinations, saved in a download folder, or shared via email in XLSX and CSV formats.

The Analytics Service checks for a cached version before executing the insight query, regardless of the requesting user. The new Export to CSV/XLSX caching limit (In megabytes) option, located in CMC > Tenant Configurations > Tuning, controls the maximum data size of the query result that the Analytics Service caches for downloaded insights. By default, this feature is disabled. To enable it, set the caching limit to a value greater than 0. Insights with a query result size exceeding the defined limit will not be cached. Additionally, if any of the Maximum Cached Memory (%) or Maximum Cached Entries options are disabled or exceeded, the Analytics Service won’t cache downloaded insights.

These improvements collectively enhance performance, optimize resource utilization, and improve the overall user experience within the Analytics Service.

For more details, refer to Tenant Configurations > Tuning.

Security roles modification

In this release, Incorta is enabling you to limit the roles of the User and Individual Analyzer user roles. Now you can disable downloading insights for these user roles using the Users with “User” or “Individual Analyzer” roles can download insights option under Default Tenant Configurations > Security in the Cluster Management Console (CMC).

This option is only available to the CMC admin, to disable it and prevent users with "User" and "Individual Analyzer" user roles from downloading insights, please contact Incorta Support.

For more information about security roles, refer to References → Security Roles.

Enhancements to the query endpoint

You can now specify format options for dates and numbers in measures, rows, columns, and hierarchy attributes in the query endpoint. Additionally, you don't need to add a label to formula columns as the endpoint will use "New Formula" as a default header.

For more details, refer to the Query Endpoint documentation.

Fixed Issues and Enhancements

Known issues

The following are known issues in 2023.4.x.

For all of the known issues and workarounds in Incorta Cloud releases, refer to Known Issues in 2023 Releases.