1. On-premises 6.0
  2. Guides
  3. Concepts
  4. Tools
  5. References
  6. Integrations
  7. Data applications
  8. Releases
    1. Release Model
    2. Release Notes 6.0
    3. Release Notes 6.0.1
    4. Release Notes 6.0.2
    5. Release Notes 6.0.3
    6. Known Issues

Release Notes 6.0.2

Release Highlights

The 6.0.2 release introduces significant features, including the ability to select tables to send to your data destination, a new Tenant Management Tool (TMT) command to delete old Parquet data, and allowing direct communication between SQL Interface (SQLi) and data agent after introducing SQLi as a stand-alone service. Additionally, more enhancements and fixed issues are available in this release.

Upgrade considerations

  • As a result of fixing an issue with the like() function, you must load all tables that use this function from staging.

What’s new?

Data delivery tables selection

Now you can select which tables to send to your data destination while setting up a data destination.

The new feature enables you to send specific tables only to the configured data destination. By default, all tables are selected. Knowing that, you must select at least one table to be able to proceed.

In case you have numerous amounts of tables within a schema, Incorta added a search option while setting up the data destination to filter the available tables. In addition, the “Show only selected” toggle to display only the table you have selected.

A new TMT command to remove old Parquet data

You can use this new TMT command (trunc) to effectively manage your disk space by removing old data based on Parquet files’ extraction dates and a specific retention period, allowing for efficient data storage usage and optimized performance. You can also specify schemas or objects (tables) to remove their data.

To use this command, follow these steps:

  1. Before running the command, make sure that the Loader Service will not start any load jobs during the truncation process.

  2. Open your command prompt, navigate to /IncortaAnalytics/cmc/tmt/, and then run the following command after providing the required parameters and options:
    /tmt.sh -clnm <cluster-name> -trunc <tenant-name> <number-of-days> -sn <schema-names> -tn <table-name>

    Example: /tmt.sh -clnm customCluster -trunc default 90 -sn HR,SALES

  3. After the truncation process, you must load the related schemas or objects from staging.

Notes
  • The default behavior of this command is to delete all the extracted data in the tenant that exceeds the specified period.
  • Use the - sn option to specify selected schemas; use the -tn option to specify selected objects (enter the object name in the following format: schemaName.objectName).
  • The command’s log file, trunc-tool.log, exists under the TMT directory. You can also check the CSV file that the command generates in the /tmt/work directory. The file name is the command execution timestamp, and it contains the deleted increments per object.
  • Running this command in addition to the Parquet merge tool may result in some old data not being deleted as the merge tool changes the timestamp of the data folders on disk.

Allowing direct communication between SQLi and Data Agent

After introducing the stand-alone SQLi Service in 6.0, this release introduces an enhancement to allow the direct communication between the data agent and SQLi services so that the SQLi service can successfully:

  • Discover and query tables and views that contain external session variables based on a data source utilizing the data agent.
  • Resolve external session variables based on a data source utilizing the data agent.
Important

This release uses the Data Agent version 6.2.0. You must upgrade to this version to avoid data loading failures due to the communication issue between SQLi and data agent. Contact Incorta Support to get the installation files.

After upgrading to 6.0.2, follow these steps:

  1. Stop the old data agent version.
  2. Go to the installation path of SQLi service: /<InstallationPath>/IncortaNode/sqli/services/<service_guid>/incorta/
  3. Open the service.properties file, add the following line, and save your changes. service.type = sqli
  4. Under The Cluster Management Console (CMC) → Server Configurations → Data Agent, set the following new properties:
    • SQLi Data Agent Port: The port where the SQLi service listens to the data agent connection requests.
    • SQLi Public Hosts and Ports: The host and port used by the data agent to establish a connection with the SQLi service.
  5. Restart the SQLi service.
  6. Sign in to Incorta Analytics platform, go to the Data Manager, select the Data Agents tab, and for the data agent you want, select Regenerate Authentication File.
  7. Deploy the new data agent.
  8. Copy the newly generated authentication file to the /conf directory under the new data agent installation folder.
  9. Start the data agent.

Enhancements and fixed issues

Enhancements

EnhancementArea
Limit the accepted rang of values for the following Spark configurations in the CMC so that they do not allow for zero or negative values:
  ●  SQL App driver memory
  ●  SQL App cores
  ●  SQL App memory
  ●  SQL App executors		CMC
When downloading listing and aggregated tables in Excel format, the column size is automatically adjusted to display the entire value without showing hashes.
Note:
You may still need to resize columns manually if the value format exceeds 20 characters.		Dashboards

Fixed issues

IssueArea
Average aggregations on decimal columns with the same scale and different precisions returned wrong results due to caching issues.Analytics
For dashboards that require a long time to load, saving insights took longer than expected and the Analyzer waited for all the dashboard insights to load.Analyzer
The like() function did not handle trailing patterns properly as it returned true when the field contained the pattern regardless of the place of the matching characters.
Note:
Fixing this issue requires loading all tables that use this function from staging after upgrading to 6.0.2.		Built-in Functions
SQL Interface (SQLi) failed to initiate due to setting the cores assigned to the SQL App executor to zero.CMC
Formatting and color options applied to dashboards were not maintained when sharing dashboards via email in PDF format.Dashboards
Issues with the system overall performance, especially when rendering dashboards due to excessive logging operations related to the Data Lineage featureData Lineage
Table extraction failed due to a communication issue between the SQLi service and the data agent causing the SQL query to time out.
Follow these steps to solve the issue.		SQL Interface (SQLi)