Dataform | Manage data in Snowflake, BigQuery, Redshift

The data modeling platform

Dataform brings open source tooling, best practices and software engineering inspired workflows to advanced data teams that are looking to scale, helping you deliver reliable data to the entire organization.

Kaleva
Livup
Echo
Charlotte Tilbury
Curology
Big Tree
Teatime Games
Slite
Tasman Analytics
Butternut Box
Outshine
Dreamdata
Mittelbayerische
SME Capital
Mixcloud

How Dataform works

A software engineering inspired approach to data modeling, pipelines and data quality management.

1

Write data transformations as code

You develop your data workflows in SQL with dataform built-in templating syntax and APIs

2

Deploy to your warehouse with Dataform

Dataform deploys your SQL table definitions to your warehouse, creates tables, views and runs tests.

3

Create a single source of truth for your data

You’re left with trusted, documented, well tested datasets ready to be consumed by other teams, or fed in to dashboards.

For teams

An end to end solution for growing data teams looking to scale their cloud data warehouse operations. Collaboratively develop and deploy, schedule and alert when things go wrong.

For developers

Designed for data professionals looking to adopt software engineering best practices. Define, document and test datasets and pipelines with SQLX and deploy to your warehouse with one command.

Define transformations with SQLX


config { type: "view" }

select
  country         as country
  device_type     as device_type
  sum(page_views) as page_views
  sum(sessions)   as sessions
from ${ref("sessions")}
group by 1, 2

Deploy to your warehouse


$ dataform run
Compiling... Compiled successfully (0.5s).

The following datasets will created (78):
  dataform_dev.sessions [table]
  dataform_dev.users [table]
  ...

The following assertions will be run (20):
  dataform_dev.sessions_browser_type_not_null
  dataform_dev.users_unique_user_id

Where Dataform fits in your stack

Dataform helps analysts manage all data processes in your warehouse, turning raw data into the datasets you need for your analytics.

What our customers say

Benoit Photo
Benoit Machefer

Director of Data

If your business is scaling fast and you want to ensure data quality, make your life easier, leverage engineering best practices and remain BI tool agnostic then don’t hesitate to use Dataform for a second!

Echo Logo
speech marks
Saadat Photo
Saadat Qadri

Analytics Practice Lead

Having modeled data using other tools in the past, this is much simpler and an easier environment to code in. The code compiles in real time and lets you know if there are errors in the syntax. It also helps generate a dependency graph for the data pipeline which is insanely useful.

Outshine Logo
speech marks
Will Photo
Will Misslin

Data Engineer

After using Dataform for a while I really discovered the power of integrating an IDE with an ETL tool. The web based IDE completely eliminates the hassle of maintaining local dev environments.

Curology Logo
speech marks

Getting started is easy

Connect to your warehouse and publish your first data model in 5 minutes.