Dbt

dbt is an open-source command line tool that helps analysts and engineers transform data in their warehouse.

The connector is available here and supports

• Synchronizing tags defined in DBT models, snapshots and seeds.
• Creating and managing Access Controls defined in DBT models, snapshots and seeds. Please note that these access controls will be marked as ‘Managed in Raito’ in the UI.

Prerequisites

dbt manifest.json file

The DBT plugin uses the target/manifest.json file to find the tags and the access controls. A valid manifest.json file, generated by dbt build or dbt run is required.

dbt-specific CLI parameters

To see all parameters, type

$> raito info raito-io/cli-plugin-dbt

in a terminal window.

Currently, the following configuration parameters are available:

• manifest (mandatory): The filepath to the manifest.json file generated by dbt to be used by the plugin.

Import tags

Models, snapshot and seeds can have tags applied within dbt. They can be defined either in the project file, in the config property file or in a config block as defined in the dbt documentation

All tags defined on dbt models, snapshots and seeds (both on top level or on column level) will be imported.

Example

In the following schema.yml file, table model1 will be tagged with tags tag:raito_tag_1 and tag:raito_tag_2. Additionally, column1 of model1 will be tagged with tag:raito_tag_2 and tag:raito_tag_3.

version: 2

models:
  - name: model1
    config:
      tags: ['raito_tag_1', 'raito_tag_2']
    columns:
      - name: column1
        config:
          tags: ['raito_tag_2', 'raito_tag_3']

Resource Provider

The plugin is also able to create access provider resources (grants, masks and filters) defined within dbt. These access control resources should be defined within the meta section of a model, snapshot, seed (top level or column level depending on the access control) as defined in the dbt documentation with with a key raito. The lifecycle of access controls defined by a dbt plugin, is managed by the dbt plugin. This includes creation and deletion of the access control.

The what-part of every access control, created by the dbt plugin, will be locked. This indicates that the what-part of the access control is managed by the dbt plugin. The who-part of each access control should be managed within Raito Cloud.

Grants

Grants can be defined on models, seeds and snapshots. Within the raito object, defined in the meta property, a grant array can be defined. A grant can be defined with the following properties:

• name (mandatory): The name of the grant. All grants, defined in the dbt project, with the same name will be combined into one Raito Cloud grant.
• permissions: Set of permissions that should be granted within this grant on the current resource.
• global_permissions: Set of global permissions (Read, Write, Admin) that should be granted with this grant on the current resource.

Example

Based on the following schema.yml file, two grants will be created. grant_1 will contain two data objects model_table_1 and model_table_2 with global permissions Read+Write and Read respectively.

version: 2

models:
  - name: model_table_1
    meta:
      raito:
        - name: grant_1
          global_permissions:
            - Read
            - Write
        - name: grant_2
          global_permissions:
            - Read
  - name: model_table_2
    meta:
      raito:
        - name: "grant_1"
          global_permissions:
            - Read

The next example shows how to define grants within the model (sql) file.

\{\{ config(meta = {
    'raito': {
        'grant': [
            {
                'name': 'grant_1', 'global_permissions': ['READ', 'WRITE']
            }
        ]
    }
}) \}\}

SELECT ...

Filters

(Feature coming soon)

Masks

Masks can be defined on the columns of models, seeds and snapshots. Within the raito object, defined in the meta property, a mask can be defined. A mask can be defined with the following properties:

• name (mandatory): A name of the mask. This name should be unique within the dbt project.
• type: The mask type that should be used to mask the data. The possible types are defined within the plugin of the corresponding data source. If no type is defined, the default mask of the plugin will be used.

Example

Within a BigQuery project the following mask could be defined within the schema.yml file.

version: 2

models:
  - name: model_table_1
    columns:
      - name: column_1
        meta:
           raito:
              mask:
                  name: mask_column_1
                  type: SHA256