References → Engine Audit
Overview
In releases before 2024.7.x, the Engine audit files track essential information only, including the tenant, the logged-in user, and actions performed, such as viewing or analyzing insights, applying filters, or downloading a dashboard. These details are saved on CSV files in the following location: /home/incorta/IncortaAnalytics/Tenants/<tenantName>/data/audit.
In 2024.7.x, Incorta has introduced a new enhanced version of the audit files. These files capture detailed tracking information, including the request source, service, off-heap memory used for processing the request, and task state time. The new files are saved under the /home/incorta/IncortaAnalytics/Tenants/<tenantName>/engine_audit.2.0 folder.
After upgrading to 2024.7.x, Incorta records user activities in both files by default. However, you can specify one type of them to use. In the Cluster Management Console (CMC) > Clusters > your_cluster > Server Configurations > Analytics Workload Management, for the Analytics Auditing option, specify if you want to log activities in the Standard audit files, the Enhanced ones, or Both.
- To start logging user actions in the audit files, Incorta requires at least one of the actions it tracks, such as opening a dashboard or downloading an insight, to be executed manually or via a scheduled job.
- You can create schemas on top of these files and build dashboards to analyze the recorded details.
Standard Audit files
User activities recorded in the Standard audit files include:
- Insight actions, such as analyzing or viewing an insight
- Variable evaluation
- Applying filters
- Downloading insights or dashboards
Columns available on these files include the following:
- Dashboard GUID
- Date
- Timestamp
- Tenant
- User
- Command
- Elapsed Time
Enhanced Audit files
Folder structure
The engine_audit.2.0 folder contains four sub-folders as follows:
action: Files in this folder include a row for each HTTP request or Scheduler job.action_details: Files in this folder include the action state updates: one row per update.tasks: Files in this folder include a list of tasks included in an action: one row per task. For example, downloading a dashboard tab may contain 10 queries (insights), representing 10 tasks.tasks_details: Files in this folder include the task state updates: one row per update.
The following section describes the columns in each file type.
Action
| Column | Description |
|---|---|
| action_id | The action ID |
| user_name | The name of the user who triggered the action |
| tenant_id | The tenant ID |
| source | The action source: ● PRIVATE_API_RENDER ● PRIVATE_API_DOWNLOAD ● PRIVATE_API_SEARCH ● SQI_X ● SCHEDULER_RENDER ● SCHEDULER_DOWNLOAD ● UNDEFINED (Includes Public API, SQLi, and variable evaluation requests) |
| format | The requested action format: ● JSON ● CSV ● XLSX ● UNDEFINED |
| isSampled | A boolean represented in 1 and 0 options |
| mode | The requested mode from the action: ● VIEW ● ANALYZE |
| dashboard_id | The ID of the dashboard the action is related to. For some actions, such as Analyzer queries, this value is null. |
| dashboard_name | The dashboard name |
| insight_id | The ID of the insight related to the action. For some actions, such as Analyzer queries and searches, this value is null. |
| start_time | The action start time |
| end_time | The action end time |
| tasks_count | The number of tasks included in this action |
| termination_state | The action termination state: ● DONE ● FAILED ● ANOTHER_REQUEST_OVERRIDE |
| service_name | The name of the service that executed the action |
Action details
| Column | Description |
|---|---|
| parent_id | The parent action ID |
| state | The action state: ● INITIALIZED ● RUNNING ● PENDING_MATCHING_RUNNING_REQUEST ● SERVED_FROM_MATCHING_REQUEST_RESULT ● REJECTED ● DONE ● FAILED ● ANOTHER_REQUEST_OVERRIDE |
| start_time | The state start time |
| duration | The state duration in Milliseconds |
Tasks
| Column | Description |
|---|---|
| action_id | The parent action ID |
| task_id | The task ID |
| insight_id | The ID of the insight related to the action. For some actions, such as Analyzer queries and searches, this value is null. |
| insight_name | The insight name |
| task_type | The task type: ● QUERY ● SEARCH |
| thread_id | The ID of the thread assigned to execute this task |
| start_time | The task start time |
| end_time | The task end time |
| termination_state | The task terminal state: ● DONE ● FAILED |
Tasks details
| Column | Description |
|---|---|
| action_id | The parent action ID |
| parent_id | The parent task ID |
| state | The task state, for example: ● INITIALIZED ● QUEUED ● PROCESSING ● DONE_PROCESSING ● INTERRUPTING |
| state_suffix | An identifier for the state if it is repeated in the same task |
| start_time | The state start time |
| duration | The state duration in milliseconds |
| off_heap_usage_bytes | The off-heap memory used while executing the task (Doesn’t include the memory used for loading the columns) |