Ensuring data quality in a cloud data warehouse | Dataform

How to ensure data quality in a cloud data warehouse

You can’t make informed decisions if you don’t trust your underlying data. Dataform helps your team automate data quality testing to ensure tahat data in your warehouse is accurate for all your analytics projects.

</>

Write tests as you develop transformations

Test for nulls, uniqueness, expected values or custom logic against all your columns in the same file as your SQL code.

</>

Extensible

Write tests against any error condition by writing custom assertions in SQL.

</>

Run assertions as part of your pipeline

Optionally add assertions as dependencies to your tables and views to ensure bad data doesn’t get populated further down your pipeline.

my_dataset.sqlx


tests {
  assertions: {
    uniqueKeys: [
      "column1",
      ["column1", "column2"],
    ],
    nonNull: ["column1"],
    rowConditions: [
      "column1 > 0",
      "column2 is null or column2 >= column1"
    ]
  }
}


select * from ...
Kaleva
Livup
Echo
Charlotte Tilbury
Curology
Outshine
Dreamdata

Learn more data quality quality testing with Dataform

Dataform brings open source tooling, best practices and software engineering inspired workflows to advanced data teams that are looking to scale, helping you deliver reliable data to the entire organization.

What our customers say

Benoit Photo
Benoit Machefer

Director of Data

If your business is scaling fast and you want to ensure data quality, make your life easier, leverage engineering best practices and remain BI tool agnostic then don’t hesitate to use Dataform for a second!

Echo Logo
speech marks
Saadat Photo
Saadat Qadri

Analytics Practice Lead

Having modeled data using other tools in the past, this is much simpler and an easier environment to code in. The code compiles in real time and lets you know if there are errors in the syntax. It also helps generate a dependency graph for the data pipeline which is insanely useful.

Outshine Logo
speech marks
Will Photo
Will Misslin

Data Engineer

After using Dataform for a while I really discovered the power of integrating an IDE with an ETL tool. The web based IDE completely eliminates the hassle of maintaining local dev environments.

Curology Logo
speech marks