Implementing Data Mesh - Helion
ISBN: 9781098156183
stron: 268, Format: ebook
Data wydania: 2024-09-04
Księgarnia: Helion
Cena książki: 228,60 zł (poprzednio: 278,78 zł)
Oszczędzasz: 18% (-50,18 zł)
As data continues to grow and become more complex, organizations seek innovative solutions to manage their data effectively. Data mesh is one solution that provides a new approach to managing data in complex organizations. This practical guide offers step-by-step guidance on how to implement data mesh in your organization.
In this book, Jean-Georges Perrin and Eric Broda focus on the key components of data mesh and provide practical advice supported by code. Data engineers, architects, and analysts will explore a simple and intuitive process for identifying key data mesh components and data products. You'll learn a consistent set of interfaces and access methods that make data products easy to consume.
This approach ensures that your data products are easily accessible and the data mesh ecosystem is easy to navigate. This book helps you:
- Identify, define, and build data products that interoperate within an enterprise data mesh
- Build a data mesh fabric that binds data products together
- Build and deploy data products in a data mesh
- Establish the organizational structure to operate data products, data platforms, and data fabric
- Learn an innovative architecture that brings data products and data fabric together into the data mesh
About the authors:
Jean-Georges "JG" Perrin is a technology leader focusing on building innovative and modern data platforms.
Eric Broda is a technology executive, practitioner, and founder of a boutique consulting firm that helps global enterprises realize value from data.
Osoby które kupowały "Implementing Data Mesh", wybierały także:
- Data Science w Pythonie. Kurs video. Przetwarzanie i analiza danych 149,00 zł, (67,05 zł -55%)
- Excel 2013. Kurs video. Poziom drugi. Przetwarzanie i analiza danych 79,00 zł, (35,55 zł -55%)
- Zarz 99,00 zł, (49,50 zł -50%)
- Eksploracja danych za pomoc 67,00 zł, (33,50 zł -50%)
- Google Analytics od podstaw. Analiza wp 69,00 zł, (34,50 zł -50%)
Spis treści
Implementing Data Mesh eBook -- spis treści
- Foreword
- Preface
- Who Is This Book For?
- Overview of the Parts and Chapters
- What This Book Isnt
- Conventions Used in This Book
- OReilly Online Learning
- How to Contact Us
- Acknowledgments
- I. The Basics
- 1. Understanding Data Mesh: The Essentials
- Making Data Agile
- Local Autonomy + Speed = Agility
- Solving Todays Data Challenges
- Bridging Data Silos
- Shifting Toward Higher Quality Data
- Transforming Data Governance
- Data Volume, Variety, and Variability
- Turning Principles into Practice
- Summary
- 2. Applying Data Mesh Principles
- Data Mesh Principles
- Data as a Product
- Decentralized Domain Ownership
- Self-Serve Data Platform
- Federated Computational Governance
- Defining a Good Data Product
- Defining a Principled Data Product
- Defining a FAIR Data Product
- Defining an Enterprise-Grade Data Product
- Defining a Valuable Data Product
- Defining a Balanced Data Product
- Defining a Modern Data ProductMore than Just Regular Data
- Defining a Practical Data Product Lifecycle
- Defining a Practical Data Mesh Roadmap
- A Good Data Product Has an Empowered Data Product Owner
- Identifying a Data Product
- Summary
- Data Mesh Principles
- 3. Our Case Study: Climate Quantum Inc.
- Making Climate Data Easier to Find, Consume, Share, and Trust
- Introducing Climate Quantum Inc.
- Climate Quantum Data Landscape
- Applying Climate Quantum Inc. to Your Enterprise
- Summary
- II. Designing, Building, and Deploying Data Mesh
- 4. Defining the Data Mesh Architecture
- Data Product Architecture
- Data Product Artifacts
- Development Architecture Components
- Data product definition
- Artifact definition
- Policy definition
- Runtime Architecture Components
- Data product gateway
- Ingestion interfaces
- Consumption interfaces
- Operations Architecture Components
- Discoverability interfaces
- Observability interfaces
- Governance interfaces
- Control interfaces
- Data Mesh Architecture
- Data Mesh Marketplace and Registry
- Data Mesh Backbone Services
- Climate Quantum Use Case Considerations
- Summary
- Data Product Architecture
- 5. Driving Data Products with Data Contracts
- Bringing Value Through Trust
- Navigating the Data Contract
- Going Through the Theory
- Stacking Up Good Information
- Its All About Proper Versioning
- Keeping It Simple and Semantic
- Walking Through an Example: Complementing Tribal Knowledge
- Documenting in a slightly better way
- Creating a human lineage
- What is Data QoS and Why Is It Critical?
- Representation
- Periods
- Groups
- Why Does It Matter?
- Why Data Quality Is Not Enough
- Accuracy (Ac)
- Completeness (Cp)
- Conformity (Cf)
- Consistency (Cs)
- Coverage (Cv)
- Timeliness (Tm)
- Uniqueness (Uq)
- Service Levels Complement Quality
- Availability (Av)
- Throughput (Th)
- Error rate (Er)
- General availability (Ga)
- End of support (Es)
- End of life
- Retention (Re)
- Frequency of update (Fy)
- Latency (Ly)
- Time to detect (an issue) (Td)
- Time to notify (Tn)
- Time to repair (Tr)
- Representation
- Applying Data QoS to the Data Contract
- Checking Conformity of Measurements
- Completeness
- Accuracy
- Engaging Service Levels
- Summary
- 6. Building Your First Data Product
- Anatomy of a Data Product
- The Data Contract to the Rescue
- Connecting Your Data Sources
- Ensuring Higher Data Quality with Observability
- Getting Faster Data Discovery
- Enabling Operability and Control
- Building and Packaging
- Summary
- 7. Aligning with the Experience Planes
- The Three Planes
- The Infrastructure Plane Remains Key
- The Data Product Experience Plane Is Independent
- The Mesh Experience Plane Is About Synergy
- Building a Capability Model for Each Plane
- Capabilities of the Infrastructure Experience Plane
- Capabilities of the Data Product Experience Plane
- Capabilities of the Mesh Experience Plane
- Gathering Key Metrics Through Feedback Loops
- Summary
- The Three Planes
- 8. Meshing Your Data Products
- Registering Your Data Product
- Connecting to Your Products in the Mesh
- Meshing Data Products Together
- Describing Your Data Lineage
- Notification Through the Mesh
- Summary
- III. GenAI, Teams, Operating Model, and Roadmap for Data Mesh
- 9. Running and Operating Your Data Mesh
- Making Data Products Discoverable, Observable, and Secure
- Data Product Discovery Interface
- Data Product Observability Interface
- Data Product Control Interface
- Data Product Security
- Climate Quantum Use Case Considerations
- Toward Dynamic Data Products
- The Power of Dynamic Data Products
- Summary
- Making Data Products Discoverable, Observable, and Secure
- 10. Creating a Data Mesh Marketplace
- Challenges with Traditional Data Catalogs
- The Data Mesh Marketplace
- A Window into Data Mesh
- The Consumer User Journey
- The Producer User Journey
- Self-Publishing of Data Products
- The Power of Discoverability
- Bootstrapping Your Marketplace
- Climate Quantums Data Mesh Marketplace
- Summary
- 11. Establishing Data Mesh Governance
- Data Governance
- Data Product Certification
- Federated Data Product Certification
- Implementing Data Product Certification
- Data Contracts and Certification
- Climate Quantums Certification Approach
- Summary
- 12. Understanding Data Product Supply Chains
- The Modern Manufacturing Supply Chain
- The Modern Software Factory
- The Data Product Factory
- The Data Product Supply Chain
- Climate Quantums Data Product Supply Chain
- Summary
- 13. Integrating Data Mesh and Generative AI
- Generative AI Background
- Large Language Models
- Embeddings
- Vector Databases
- Prompts
- Challenges
- Data Mesh and Generative AI
- Improve Data Quality with Generative AI
- Elevate Models as Primary Artifacts
- Consume Models Inside Data Products
- Generate Data Product Code
- Complement Data Product Capabilities
- Climate Quantum and Generative AI
- Simplifying Climate Data Search
- Analyzing and Summarizing Climate Data
- Creating Disclosure Reports
- Generating Code to Simplify Climate Data Processing
- Summary
- Generative AI Background
- 14. Establishing Data Mesh Teams
- Team Topologies in Data Mesh
- The Data Product Team
- Key roles and responsibilities
- Data product owner
- Release management
- Metadata engineering
- Data and security engineering
- Consumption engineering
- Ingestion engineering
- Platform Teams
- Enabling Teams
- The Data Product Team
- Climate Quantum Teams
- Summary
- Team Topologies in Data Mesh
- 15. Defining a Data Mesh Operating Model
- Characteristics of an Operating Model
- The Operating Model Continuum
- Centralized Organizations
- Matrixed Organizations
- Federated Organizations
- Distributed Organizations
- Data Mesh Operating Model
- Data Mesh Maturity Model
- Climate Quantums Data Mesh Operating Model
- Summary
- 16. Establishing a Practical Data Mesh Roadmap
- Roadmap Structure
- Strategy and Roadmap Stream
- Technology Stream
- Foundational phase deliverables
- Industrialization phase deliverables
- Factory Stream
- Operating Model Stream
- Socialization Stream
- Rollout Stream
- Climate Quantums Data Mesh Roadmap
- Summary
- Roadmap Structure
- Index