See it as a diagram

Everything below, as a diagram you can edit. Describe yours and see it in seconds.

107/2000• 3 credits left

Try:

No account needed · Editable canvas, not a picture

1. Why Data Quality Matters

Every data team has a horror story: a dashboard showed revenue up 40%, the CEO announced it in an all-hands, and then someone noticed a duplicate join. Data quality isn't about perfection—it's about catching issues before they reach decision-makers.

The Cost of Bad Data

Gartner estimates that poor data quality costs organizations an average of $12.9 million per year. But the hidden cost is worse: eroded trust. Once stakeholders lose confidence in your data, they stop using it—and start making gut decisions instead.

Data quality delivers three outcomes:

Trust

Stakeholders use data confidently because they know it's been validated. No more "let me double-check that number."

Speed

Catch issues at the source instead of debugging dashboards. Minutes to fix vs. hours of investigation.

Scale

Automated checks scale with your data. Manual spot-checks don't work when you have 500 tables.

From experience

The teams I've seen succeed treat data quality like unit tests in software: you don't ship code without tests, and you don't ship data without checks. It's not overhead—it's insurance.

2. The Six Dimensions of Data Quality

Not all quality issues are the same. These six dimensions give you a framework for thinking about what can go wrong—and what to test for.

Completeness

Is all required data present?

Test: Null rate on required fields should be 0%

Accuracy

Do values reflect reality?

Test: Cross-reference with source systems

Consistency

Same format everywhere?

Test: Date formats, enum values, naming conventions

Timeliness

Is data fresh enough?

Test: max(updated_at) within SLA window

Validity

Conforms to business rules?

Test: Status in ('active', 'inactive'), price > 0

Uniqueness

No unwanted duplicates?

Test: Primary key is unique, no duplicate orders

Dimension	Question It Answers	Common Failure Mode
Completeness	Is it all there?	Nulls in required fields
Accuracy	Is it correct?	Stale cache, wrong joins
Consistency	Is it the same everywhere?	Different date formats
Timeliness	Is it fresh enough?	Pipeline delays, SLA breaches
Validity	Does it make sense?	Negative prices, future dates
Uniqueness	Are there duplicates?	Duplicate IDs from fan-out joins

3. Where to Test in Your Pipeline

The "test at the boundaries" principle: validate data when it enters your system and when it leaves. This catches both upstream issues and transformation bugs.

Stage 1: Source Validation (Ingestion)

Test data as it arrives from external systems. This is your first line of defense against upstream issues.

What to check:

• Schema matches expected (no new/missing columns)
• Row count within expected range
• Required fields are non-null
• Data arrived on time (freshness)

Stage 2: Transformation Tests

Test the output of your transformations. This catches logic bugs in your SQL/Python.

What to check:

• Primary keys are unique (no fan-out)
• Aggregations sum correctly
• Business rules are enforced
• Referential integrity maintained

Stage 3: Output Validation (Delivery)

Test data before it reaches dashboards and consumers. This is your last chance to catch issues.

What to check:

• Metrics are within expected bounds
• No anomalies vs. historical patterns
• Critical dashboards have data
• SLAs are met

Pro Tip

Fail Fast, Fail Loud

It's better for a pipeline to fail at 6 AM with a clear error than to silently produce bad data that gets discovered in a board meeting. Configure tests to block bad data from propagating downstream.

4. What to Test (Practical Checks)

Start with these high-value, low-effort checks. They catch 80% of issues with 20% of the work.

Row count bounds

Check that row count is within expected range (e.g., 900K-1.1M). Catches truncated loads and duplicate explosions.

expect_table_row_count_to_be_between(min=900000, max=1100000)

Null rate thresholds

Required fields should have 0% nulls. Optional fields should have stable null rates (alert if rate changes by >10%).

expect_column_values_to_not_be_null(column="customer_id")

Unique constraints

Primary keys must be unique. This catches the dreaded fan-out join that multiplies your metrics.

expect_column_values_to_be_unique(column="order_id")

Freshness SLA

Data should be no older than your SLA allows. Check max(updated_at) or partition date.

expect_column_max_to_be_between(column="updated_at", min=now()-24h)

Value ranges

Numeric fields should be within valid bounds. Revenue > 0, age between 0-150, percentage 0-100.

expect_column_values_to_be_between(column="price", min=0, max=100000)

Enum validation

Status fields should only contain expected values. Catches typos and unexpected states from upstream.

expect_column_values_to_be_in_set(column="status", value_set=["active", "inactive", "pending"])

Example: dbt schema test

# models/marts/orders.yml
version: 2
models:
  - name: orders_mart
    description: "Order-level facts for analytics"
    columns:
      - name: order_id
        tests:
          - unique
          - not_null
      - name: customer_id
        tests:
          - not_null
          - relationships:
              to: ref('dim_customers')
              field: customer_id
      - name: order_total
        tests:
          - not_null
          - dbt_utils.accepted_range:
              min_value: 0
              max_value: 1000000

5. Alerting Without Alert Fatigue

The biggest risk with data quality monitoring isn't missing issues—it's alert fatigue. Too many false positives and your team starts ignoring alerts. Here's how to get it right.

Do This

• Tier your alerts: P1 (page), P2 (Slack), P3 (email digest)
• Use anomaly detection for metrics that vary naturally
• Include context in alerts: what broke, impact, runbook link
• Set appropriate thresholds based on historical data
• Review and tune alert thresholds monthly

Avoid This

• Paging on every failed test
• Hard-coded thresholds that don't adapt
• Alerts without clear ownership
• Ignoring "flaky" tests instead of fixing them
• Missing runbooks for common failures

Alert Severity Framework

P1 - Page Now

Data is wrong in production dashboards. Executives are affected. Revenue impact.

P2 - Slack Alert

Data quality degraded but not critically broken. Should fix within 4 hours.

P3 - Daily Digest

Minor issues, informational. Review in next sprint. No immediate action needed.

From experience

I once joined a team with 200+ data quality alerts per day. Everyone ignored them. We cut it to 15 high-signal alerts and suddenly the team started fixing issues the same day. Less is more.

6. Tools for Data Quality

Tool	Best For	Approach	Pricing
dbt tests	Transformation testing	Schema + custom tests	Free (OSS)
Great Expectations	Comprehensive validation	Expectation suites	Free (OSS)
Monte Carlo	Data observability	ML anomaly detection	Enterprise
Soda	Data monitoring	SodaCL checks	Free tier + paid
elementary	dbt-native observability	Anomaly detection	Free (OSS)

Proactive Testing

You define the rules. Tests run on every pipeline execution. Explicit, deterministic.

Tools: dbt tests, Great Expectations, Soda

Anomaly Detection

ML learns "normal" patterns and alerts on deviations. Catches unknown unknowns.

Tools: Monte Carlo, elementary, Bigeye

7. Best Practices Checklist

Test at the boundaries

Validate data at ingestion (source issues) and at delivery (transformation issues). Catch problems early.

Start with high-value tables

Focus on tables that feed executive dashboards and critical reports first. Prove value, then expand.

Use both rules and anomaly detection

Rules catch known issues (nulls, duplicates). Anomaly detection catches unknown issues (sudden volume changes).

Tier your alerts

Not every failure deserves a page. P1 for revenue impact, P2 for degraded quality, P3 for informational.

Include runbooks in alerts

Every alert should link to a doc explaining: what it means, how to investigate, how to fix common causes.

Track quality metrics over time

Measure: test pass rate, mean time to detection, mean time to resolution. Improve systematically.

Make quality part of CI/CD

Run tests on every PR. Block merges that break data quality. Shift left.

Review and tune monthly

Alert thresholds drift. Data patterns change. Schedule monthly reviews to keep tests relevant.

Pro Tip

The "Would I Bet Money On It?" Test

Before publishing a metric, ask: "Would I bet $1,000 that this number is correct?" If the answer is no, you need more tests. Data quality is about confidence, and confidence comes from validation.

8. Frequently Asked Questions

What are the key dimensions of data quality?

The six key dimensions are: Completeness (no missing values), Accuracy (correct values), Consistency (same format across systems), Timeliness (data is fresh enough), Validity (conforms to business rules), and Uniqueness (no duplicates where there should be none).

How do you measure data quality?

Measure through automated tests: row counts vs expected ranges, null rates per column, unique constraint violations, freshness (time since last update), schema drift detection, and business rule validation. Track these metrics over time and set alerting thresholds.

What tools are best for data quality monitoring?

Popular tools include: Great Expectations (open-source validation), dbt tests (for transformation testing), Monte Carlo (data observability), Soda (data monitoring), and elementary (dbt-native observability). Choose based on your stack and whether you need proactive testing or anomaly detection.

Should data quality checks run before or after transformations?

Both. Run source data quality checks before transformations to catch upstream issues early (fail fast). Run output checks after transformations to validate business logic. This "test at the boundaries" approach catches issues at both ingestion and delivery points.

Document Your Data Quality Strategy

Build clear diagrams of your data pipelines with quality checkpoints marked. Show stakeholders exactly where and how data gets validated.

Create Pipeline Diagram Pipeline Documentation Guide

1. Why Data Quality Matters

The Cost of Bad Data

Data quality delivers three outcomes:

Trust

Speed

Scale

2. The Six Dimensions of Data Quality

Completeness

Accuracy

Consistency

Timeliness

Validity

Uniqueness

3. Where to Test in Your Pipeline

Stage 1: Source Validation (Ingestion)

Stage 2: Transformation Tests

Stage 3: Output Validation (Delivery)

Fail Fast, Fail Loud

4. What to Test (Practical Checks)

Row count bounds

Null rate thresholds

Unique constraints

Freshness SLA

Value ranges

Enum validation

5. Alerting Without Alert Fatigue

Do This

Avoid This

Alert Severity Framework

6. Tools for Data Quality

Proactive Testing

Anomaly Detection

7. Best Practices Checklist

Test at the boundaries

Start with high-value tables

Use both rules and anomaly detection

Tier your alerts

Include runbooks in alerts

Track quality metrics over time

Make quality part of CI/CD

Review and tune monthly

The "Would I Bet Money On It?" Test

8. Frequently Asked Questions

What are the key dimensions of data quality?

How do you measure data quality?

What tools are best for data quality monitoring?

Should data quality checks run before or after transformations?

Document Your Data Quality Strategy

Related Guides

Data Pipeline Documentation

Data Lineage Best Practices

Data Contracts

Medallion Architecture

ML Pipeline Architecture