References → Filter Recipe
The filter recipe is a tool focused on keeping data where the configuration of the recipe is true. Use this recipe to remove rows from your working data set or split it into another flow based on one or multiple conditions.
Configuration
| Configuration | Description |
|---|---|
| Recipe Name | A freeform name of how a user would like to name a recipe |
| Input | Select a previously constructed recipe to process |
| Advanced Mode | Enter custom pySpark logic. Selecting this mode will remove the below configuration options. |
| Keep Only Rows That Satisfy | Select between the two choices: At least one of the following conditions ● If any of the conditions are evaluated as true, then keep the row. This functions as an OR condition. All of the following conditions ● All conditions must be evaluated as true, to keep a row. This functions as an AND condition. |
| Conditions | Select a field to evaluate, a condition, and a value if applicable. |
Conditions
Below is a table of operators and a description of the intended result when a condition is evaluated as true.
| Operator | Compatible Data Types | Description |
|---|---|---|
| Is Null | Date, Measure, String | Returns null values |
| Is Not Null | Date, Measure, String | Returns non-null values |
| Is Empty | String | Returns blank strings |
| Is Not Empty | String | Returns strings that are not blank |
| Sames As | Date, Measure | Equivalent to 'equals’, but the value must be defined as another column |
| Different From | Date, Measure | Equivalent to 'does not equal’ the value must be defined as another column |
| Equal | Date, String | Returns matching values (not case-sensitive) |
| Equal Case-sensitive | String | Returns matching values (case-sensitive) |
| Contains | String | Returns strings where specified characters are found |
| Does Not Contain | String | Returns values were specified characters are not found |
| In | String | Returns values where value matched to values in a list |
| Not in | String | Returns values where matched to values not in a list |
| Not Equal | String | Returns rows where operand value is not matched |
| Before | Date | Returns dates before the specified operand value |
| After | Date | Returns dates after the specified operand value |
| Between | Date | Returns dates between the specified operand values |
| == | Measure | Equivalent to 'equal' |
| != | Measure | Equivalent to 'does not equal' |
| > | Measure | Returns rows with a value greater than the operand value |
| >= | Measure | Returns rows with a value equal to or greater than the operand value |
| < | Measure | Returns rows with a value less than the operand value |
| <= | Measure | Returns rows with a value equal to or less than the operand value |
| == (Column) | Measure | Same as ‘==’, but a column is selected as the operand. |
| > (Column) | Measure | Same as ‘>’, but a column is selected as the operand. |
| >= (Column) | Measure | Same as ‘>=’, but a column is selected as the operand. |
| < (Column) | Measure | Same as ‘<’, but a column is selected as the operand. |
| <= Column | Measure | Same as ‘<=’, but a column is selected as the operand. |