1. Home
  2. Previous Releases
  3. Latest Release
  4. Guides
  5. Concepts
  6. Tools
  7. References
    1. Apache Spark
    2. Built-in Functions
    3. Incorta Component SDK
    4. Incorta Connector SDK
    5. Connectors
    6. Data Studio
      1. Enabling the Data Studio
      2. Aggregation Recipe
      3. Change Type Recipe
      4. Data Quality Recipe
      5. Filter Recipe
      6. Formula Recipe
      7. Fuzzy Join Recipe
      8. Join Recipe
      9. Python Recipe
      10. Rename Recipe
      11. Sample Recipe
      12. Save MV Recipee
      13. Select Recipe
      14. Sort Recipe
      15. Split Recipe
      16. SQL Recipe
      17. Union Recipe
      18. Unpivot Recipe
    7. Public API
    8. Security
    9. SQL Interface
    10. Visualizations
    11. Other
  8. Integrations
  9. Data applications
  10. Releases
  11. Incorta Premium

References → SQL Recipe

The code tool is an excellent way to inject Spark SQL into your dataflow. This can be used for simple or complex operations that are not currently available as a recipe.

ConfiguratioDescription
Recipe NameA freeform name of how a user would like to name a recipe
Multiple InputsSelect a previously constructed recipe(s). The SQL tool can ingest multiple inputs.
CodeEnter Spark SQL code to run within the recipe

Important pySpark commands

Reference Input Sources

The inputs assigned in the Multiple Inputs setting will populate a new configuration section called Input Variable Name. This will showcase the true name of the underlying data frame. Users can copy these variable names as table names within their SQL. Note that no quotations are needed for specifying tables or columns.

Output Data

For each Code recipe, the code must contain output as a data frame for the recipe to save properly. Please use the function output_df() to save the data frame.

Example: output_df(my_sample_df)