1. Home
  2. Platforms
  3. dbt & Airflow

dbt & Airflow

dbt and Airflow, set up the way teams actually maintain.

dbt and Airflow are the two tools that turn a collection of scripts into a data platform. We implement both with the parts teams usually skip: tests, CI, documentation, and DAGs that fail safely instead of silently.

  • Tested dbt models
  • Reliable DAGs
  • CI/CD for data

What you get

The tools, with the practices that make them work.

Tested models

Assumptions asserted, so silent wrongness fails the run.

DAGs that fail safely

Idempotent tasks, sensible retries and alerts that reach a person.

CI for data

dbt build on every pull request, so broken models never ship.

A project your analysts own

Docs, lineage and working sessions at handover.

The problem

Adopted the tools, kept the problems.

Installing dbt does not make a project maintainable. We regularly review dbt repos with one flat directory of 200 models, no tests, and business logic copy-pasted across a dozen files. Airflow tends to arrive with the same pathology: DAGs with hard-coded dates, tasks that are not idempotent so a retry corrupts data, and no alerting, so failures are noticed when someone opens the UI. The tools are correct; the practices around them are what break.

What we implement

What's included.

  • A dbt project structure with staging, intermediate and mart layers, plus clear naming conventions
  • Test coverage on the assumptions models depend on: uniqueness, referential integrity, accepted values and freshness
  • Incremental model strategies, so large tables do not fully rebuild every run
  • Airflow DAG design with idempotent tasks, correct dependencies, sensible retries and real alerting
  • CI that runs dbt build on every pull request, so a broken model never reaches production
  • Generated documentation and lineage, plus enablement so your analysts own the project

How we work

How we run it.

  1. Step 1

    Review the current project or scripts and identify what is fragile versus merely untidy

  2. Step 2

    Restructure the transformation layer with tests, refactoring duplicated logic into shared models

  3. Step 3

    Rebuild orchestration with idempotent tasks and alerting that reaches a person

  4. Step 4

    Add CI, then train the team on the workflow so it holds after we leave

What we use

Tools, platforms and methods.

  • dbt
  • Airflow
  • Dagster
  • Snowflake
  • BigQuery
  • GitHub Actions
  • Python

FAQ

Common questions.

Airflow or Dagster for a new project?

Airflow has the larger ecosystem, more managed options and a much bigger hiring pool, which is why we default to it. Dagster has a genuinely better developer experience and treats data assets as first-class, which suits teams with strong engineering practices starting fresh. Either is defensible; the deciding factor is usually who maintains it.

Can dbt replace our stored procedures?

Almost always, and it is usually worth doing. dbt gives you version control, testing, documentation and lineage, none of which stored procedures offer. The migration is mostly mechanical, and the real gain is that transformation logic becomes reviewable in pull requests instead of living undocumented in the database.

How much test coverage is enough?

Every model should assert that its primary key is unique and not null, and every relationship it depends on should be tested. Beyond that, test the assumptions that would cause a wrong number rather than a failed run, since those are the ones that reach clients. Perfect coverage is not the goal; catching silent wrongness is.

Do you train our team or just build it?

Both, and the training matters more. A dbt project nobody on your team can extend has just moved the bottleneck to us, which is not a good outcome for you. Handover includes the repo, the CI, the docs and working sessions with your analysts.

Review your dbt and Airflow setup.

Point us at the repo or describe how transformations run today, and we will report what is fragile and what to fix first.

Talk to an engineer