A framework for ambitious data engineers and analysts.

Dataform is a productive open-source framework for building modern SQL based data workflows. It includes everything you need to develop, test and deploy complex data workflows easily in your cloud data warehouse.

How does dataform work?

You develop your dataworkflows in SQL with dataform's built-in templating syntax and APIs

Dataform compiles your scripts into a DAG (Directed Acyclic Graph) of every transformation in SQL

Dataform connects to your warehouse and executes the DAG or the nodes that you specify

You get logs of the runs

Publish datasets

Dataform make the definition of datasets easy. Simply write a select queries, add a few parameters and our compiler generates the boilerplate for you.

Custom snippets and functions

Standardize definitions across all your datasets with includes and JavaScript based SQL templates.

Ensure high data quality

Have confidence in your data and diagnose data quality issues before they impact production with assertions that validate your data continuously

Knowledge sharing

Write descriptions of your datasets and fields while you develop and get complete overview of your datasets in the data catalog.

Benefits

Highly productive

Write dramatically less SQL with Dataform's integrated set of APIs and reusable snippets

Extentable

Develop your own functions in plain Javascript that benefit your entire team

Iterate fast

View compiled SQL and errors in real time while you develop