An open source framework for data engineers and analysts

The Dataform SDK helps data teams manage and automate SQL-based operations in your data warehouse.
Read the docs or view on GitHub github logo.

How it works

1. Develop

You develop queries, operations, and assertions in SQL with Dataform's built-in templating syntax and APIs

2. Compile

Dataform compiles your project into a DAG (Directed Acyclic Graph) of every action in SQL

3. Run

Dataform connects to your data warehouse and executes the DAG (or the actions you specify)

4. View logs

You receive detailed execution logs of the datasets created and the other actions performed

config { type: "table" }

SELECT
  country,
  device_type,
  country,
  SUM(revenue) AS revenue,
  SUM(sessions) AS sessions,
  SUM(pageviews) AS pageviews
FROM raw_data
GROUP BY 1, 2
logo
table
dataform.my_table
FieldTypeDescription
countrySTRINGDescription
device_typeSTRINGDescription
revenueDOUBLEDescription
sessionsDOUBLEDescription
pageviewsDOUBLEDescription
config { type: "table" }

SELECT
  country,
  device_type,
  country,
  SUM(revenue) AS revenue,
  SUM(sessions) AS sessions,
  SUM(pageviews) AS pageviews
FROM raw_data
GROUP BY 1, 2
Each .sqlx file in your definitions/ folder contains a SQL query or statement enriched with Dataform's template and configuration syntax. Dataform then generates the boilerplate required to create corresponding tables and views in your data warehouse.

Benefits

Write less SQL thanks to reusable code snippets and an integrated suite of APIs

Highly productive

Write less SQL thanks to reusable code snippets and an integrated suite of APIs

Develop reusable functions in JavaScript - to the benefit of your entire team

Fully extensible

Develop reusable functions in JavaScript - to the benefit of your entire team

View compiled SQL with real-time compilation

Iterate quickly

View compiled SQL with real-time compilation

Ready to get started?

Get in touch or create an account.

Already using Dataform? Log in