Amazon Redshift, part of the AWS suite of products, is a powerful data warehouse with petabyte-scale capacity, massively parallel processing, and columnar database architecture. Dataform allows you to manage all data processes happening in your Redshift warehouse, turning raw data into datasets that power your company’s analytics. Redshift pricing starts at $0.25 per hour and scales from there to support petabytes of data and thousands of concurrent users.
The Dataform web IDE is natively integrated with GitHub and GitLab. Version controlling your SQL has never been easier: create branches, commit changes, revert files and create pull requests without ever needing to touch the command line.
Redshift Spectrum lets you read and analyze data from S3 buckets, without ever having to read ingest the data into the warehouse. Dataform provides a version controlled environment to manage and update your Spectrum table definitions.
Keep your Redshift warehouse up to date with Dataform’s powerful scheduling features. Schedules can be triggered by API, webhook or a time of your choosing. Success and failure alerts are sent to your team by Slack or email. Detailed run logs show exactly which SQL statements ran when, making debugging simple. And our parallel execution strategy minimises schedule durations.
If your business is scaling fast and you want to ensure data quality, make your life easier, leverage engineering best practices and remain BI tool agnostic then don’t hesitate to use Dataform for a second!
Having modeled data using other tools in the past, this is much simpler and an easier environment to code in. The code compiles in real time and lets you know if there are errors in the syntax. It also helps generate a dependency graph for the data pipeline which is insanely useful.
After using Dataform for a while I really discovered the power of integrating an IDE with an ETL tool. The web based IDE completely eliminates the hassle of maintaining local dev environments.