Data Engineering Design Patterns
Data Science
Book Details
Book Title
Data Engineering Design Patterns
Author
Bartosz Konieczny
Publisher
O'Reilly Media, Inc
Publication Date
2025
ISBN
9781098165819
Number of Pages
493
Language
English
Format
File Size
8.6MB
Subject
Data Engineering
Table of Contents
- Preface
- Chapter 1. Introducing Data Engineering Design Patterns
- What Are Design Patterns?
- Yet More Design Patterns?
- Common Data Engineering Patterns
- Case Study Used in This Book
- Summary
- Chapter 2. Data Ingestion Design Patterns
- Full Load
- Incremental Load
- Replication
- Data Compaction
- Data Readiness
- Event Driven
- Summary
- Chapter 3. Error Management Design Patterns
- Unprocessable Records
- Duplicated Records
- Late Data
- Filtering
- Fault Tolerance
- Summary
- Chapter 4. Idempotency Design Patterns
- Overwriting
- Updates
- Database
- Immutable Dataset
- Summary
- Chapter 5. Data Value Design Patterns
- Data Enrichment
- Data Decoration
- Data Aggregation
- Sessionization
- Data Ordering
- Summary
- Chapter 6. Data Flow Design Patterns
- Sequence
- Fan-In
- Fan-Out
- Orchestration
- Summary
- Chapter 7. Data Security Design Patterns
- Data Removal
- Access Control
- Data Protection
- Connectivity
- Summary
- Chapter 8. Data Storage Design Patterns
- Partitioning
- Records Organization
- Read Performance Optimization
- Data Representation
- Summary
- Chapter 9. Data Quality Design Patterns
- Quality Enforcement
- Schema Consistency
- Quality Observation
- Summary
- Chapter 10. Data Observability Design Patterns
- Data Detectors
- Time Detectors
- Data Lineage
- Summary
- Afterword
- Appendix. Summary of Patterns
- Index
- About the Author