Data Engineering Design Patterns

by Bartosz Konieczny

Data Science

Book Details

Book Title

Data Engineering Design Patterns

Author

Bartosz Konieczny

Publisher

O'Reilly Media, Inc

Publication Date

2025

ISBN

9781098165819

Number of Pages

493

Language

English

Format

PDF

File Size

8.6MB

Subject

Data Engineering

Table of Contents

  • Preface
  • Chapter 1. Introducing Data Engineering Design Patterns
  • What Are Design Patterns?
  • Yet More Design Patterns?
  • Common Data Engineering Patterns
  • Case Study Used in This Book
  • Summary
  • Chapter 2. Data Ingestion Design Patterns
  • Full Load
  • Incremental Load
  • Replication
  • Data Compaction
  • Data Readiness
  • Event Driven
  • Summary
  • Chapter 3. Error Management Design Patterns
  • Unprocessable Records
  • Duplicated Records
  • Late Data
  • Filtering
  • Fault Tolerance
  • Summary
  • Chapter 4. Idempotency Design Patterns
  • Overwriting
  • Updates
  • Database
  • Immutable Dataset
  • Summary
  • Chapter 5. Data Value Design Patterns
  • Data Enrichment
  • Data Decoration
  • Data Aggregation
  • Sessionization
  • Data Ordering
  • Summary
  • Chapter 6. Data Flow Design Patterns
  • Sequence
  • Fan-In
  • Fan-Out
  • Orchestration
  • Summary
  • Chapter 7. Data Security Design Patterns
  • Data Removal
  • Access Control
  • Data Protection
  • Connectivity
  • Summary
  • Chapter 8. Data Storage Design Patterns
  • Partitioning
  • Records Organization
  • Read Performance Optimization
  • Data Representation
  • Summary
  • Chapter 9. Data Quality Design Patterns
  • Quality Enforcement
  • Schema Consistency
  • Quality Observation
  • Summary
  • Chapter 10. Data Observability Design Patterns
  • Data Detectors
  • Time Detectors
  • Data Lineage
  • Summary
  • Afterword
  • Appendix. Summary of Patterns
  • Index
  • About the Author