Organizations collecting and using personal data must now heed a growing body of regulations, and the penalties for noncompliance are stiff. The ubiquity of the cloud and the advent of generative AI have only made it more crucial to govern data appropriately. Thousands of companies have turned to Databricks Unity Catalog to simplify data governance and manage their data and AI assets more effectively. This practical guide helps you do the same.

Databricks data specialists Kiran Sreekumar and Karthik Subbarao dive deep into Unity Catalog and share the best practices that enable data practitioners to build and serve their data and AI assets at scale. Data product owners, data engineers, AI/ML engineers, and data executives will examine various facets of data governance—including data sharing, auditing, access controls, and automation—as they discover how to establish a robust data governance framework that complies with regulations.

• Explore data governance fundamentals and understand how they relate to Unity Catalog
• Utilize Unity Catalog to unify data and AI governance
• Access data efficiently for analytics
• Implement different data protection mechanisms
• Securely share data and AI assets internally and externally with Delta Sharing

Osoby które kupowały "Data Governance with Unity Catalog on Databricks. Implement Data and AI Governance with Databricks Data Intelligence Platform", wybierały także:

• Jak zhakowa 125,00 zł, (10,00 zł -92%)
• Biologika Sukcesji Pokoleniowej. Sezon 3. Konflikty na terytorium 125,00 zł, (15,00 zł -88%)
• Windows Media Center. Domowe centrum rozrywki 66,67 zł, (8,00 zł -88%)
• Podręcznik startupu. Budowa wielkiej firmy krok po kroku 93,75 zł, (15,00 zł -84%)
• Ruby on Rails. Ćwiczenia 18,75 zł, (3,00 zł -84%)

Spis treści

Data Governance with Unity Catalog on Databricks. Implement Data and AI Governance with Databricks Data Intelligence Platform eBook -- spis treści

• Foreword
• Preface
  • Why We Wrote This Book and Why Now
  • Who This Book Is For
  • How This Book Is Organized
  • Conventions Used in This Book
  • OReilly Online Learning
  • How to Contact Us
  • Acknowledgments
• Prologue: Governance by Choice
  • The Journey to Being a Multicloud Data Platform
  • Databricks Lakehouse and Unification of the Data Estate
  • Leveraging Unity Catalog
• 1. The Modern Governance Stack
  • Introducing Data Governance
    • Benefits of Effective Data Governance
    • The Lifecycle of Data
    • Governing the Ungoverned
    • Complying with Increasing Regulations
  • The Dawn of the Lakehouse
    • Deriving Value from Data
      • Business intelligence and advanced analytics
      • Big data begins
    • Data Warehouses and Data Lakes
      • Cloud data lake
      • Data silos and fractured governance
    • The Lakehouse Paradigm
      • The lakehouse architecture
      • Medallion lakehouse architecture
      • Databricks Data Intelligence Platform
  • Databricks Unity Catalog: Enabling Unified Governance
    • Introducing Unity Catalog
      • Access control for data and AI assets
      • Core governance features
      • Advanced governance features
    • Databricks Platform Architecture
      • Control plane
      • Compute plane
      • Unity Catalog components
    • Data Sharing and Collaboration
  • Summary
• 2. Unity Catalog Under the Hood
  • The Governance Story So Far
    • Hive Metastore as the Default Catalog
    • The Dilemma of Governance in Hive Metastore
      • Isolation at cluster level
      • Table Access Control List
      • Credential passthrough
  • Unity Catalog Architecture
    • Centralized Governance with Unity Catalog
      • Databricks account
      • The metastore regional construct
  • The Governance Model of Unity Catalog
    • Decoupled Storage Credentials
    • External Location for Cloud Object Storage
    • Compute Modes in Unity Catalog
      • Isolation using Unity Catalog Lakeguard in standard or shared access mode
      • Single user or dedicated access mode for elevated privileges
  • Data Management Features
    • Catalog as the Namespace
    • Data Isolation at Catalog and Schema Level
    • Catalog to Workspace Binding
  • Summary
• 3. Identity Management
  • Databricks Constructs
    • Cloud-Specific Details
      • Databricks on AWS
      • Azure Databricks
      • Databricks on GCP
    • Access to Databricks and Beyond
    • Databricks Securables
  • Databricks Identities
    • Databricks Identity Types
    • Predefined Admin Roles and Responsibilities
      • Account admin
      • Metastore admin
      • Workspace admin
  • Interfaces to Access the Platform
    • Databricks UI
    • Databricks REST API
  • Identity Provisioning
    • Syncing Identities from Identity Provider to Databricks Account
    • Automatic Identity Management with Microsoft Entra ID
    • Databricks Workspace Assignment
    • User Access Provisioning and De-provisioning
  • Single Sign-On
  • Programmatic Authentication Methods
    • Cloud-Specific Authentication: Azure Databricks
    • Cloud-Specific Authentication: Databricks on GCP
    • OAuth Token Federation
  • Identity Best Practices
  • Summary
• 4. Unity Catalog and Compute
  • Implementing Governance: A Two-Part Problem
  • Classic Compute in Databricks
    • Standard or Shared Access
      • Unity Catalog Lakeguard for standard cluster
      • Unity Catalog guardrails and service credentials
      • Limitations of standard access mode
    • Dedicated or Single User Access
      • Unity Catalog Lakeguard for dedicated cluster
      • Limitations of dedicated access mode
    • Assigned to Group Cluster
  • Going Serverless with Databricks
    • Serverless Generic Compute
    • Serverless Data Warehouse
    • Serverless Model Serving
    • Serverless Databricks Apps
    • Serverless Lakeflow Declarative Pipelines
  • Summary
• 5. Access Controls and Permissions Model
  • Access Management
  • Access Controls
    • Workspace Access Controls
    • Unity Catalog Access Controls
    • Permissions Model
    • Access Controls on Nontabular Data
    • Managed and Unmanaged Datasets
    • Advanced Access Controls
      • Binding Unity Catalog securables with workspaces
      • Fine-grained access controls
      • Attribute-based access controls
      • Governed tags
      • ABAC policies
  • Data Governance Models
    • Centralized Data Governance
    • Distributed Data Governance
    • Federated Data Governance
  • Data Storage and Distribution
    • Catalog Layout and Nomenclature
    • Data Sharing and Distribution
      • In-place publishing
      • Dedicated publishing catalog
      • Centralized publishing
  • Bringing It All Together
  • Summary
• 6. Governing AI
  • What Is AI Governance?
  • AI Model Lifecycle
    • Model Training
    • Model Serving
  • Governing AI Systems on Databricks
    • MLOps
    • Large Language Models
    • Mosaic AI Gateway
  • Components of an AI System
  • Implementing an AI System
  • Summary
• 7. Observability and Discoverability
  • Unity Catalog System Tables
    • Architecture
    • Audit Observability
    • Lineage Observability
    • Cost Observability
    • Compute Observability
    • Jobs Observability
    • Marketplace Observability
    • Model Serving Observability
    • Query History and Storage Observability
    • Observability Assistant
  • Data Quality in Databricks
  • Lakehouse Monitoring
    • The Profiles
    • The Baseline Table
    • The Monitoring Artifacts
  • Data Quality Monitoring
  • Discoverability in Unity Catalog
    • Asset Description
    • Tagging
    • AI-Powered Search
    • BROWSE Privilege
    • Insights and Popularity
    • Lineage
    • Lakehouse Federation
    • Enterprise Catalogs
    • Certification and Deprecation
      • Certified tag
      • Deprecated tag
  • Summary
• 8. Data Sharing and Collaboration
  • Databricks Data Access Patterns
  • Data Sharing and Publishing with Delta Sharing
  • Data Governance Beyond Metastores
    • Why Delta Sharing?
      • Simplicity
      • Scalability
      • Security
    • D2D Sharing Under the Hood
    • Ownership and Privileges
    • Catalog Layout
    • Challenges
      • Propagating access controls across metastores
      • Cross-metastore data discovery and lineage
      • FGACs across metastores
  • Internal and External Sharing
    • Data Mesh with Delta Sharing
    • External Sharing
    • Databricks Marketplace and Clean Rooms
  • Summary
• 9. Open Access
  • Managed Versus External Table
    • Why Use External Tables?
    • Data Independence
    • Managed Tables for the Win
  • Open Source Unity Catalog
  • External Access
    • Unity and Iceberg REST Catalog
    • Credential Vending
    • Catalog Interoperability
  • Summary
• 10. Being Compliant with Regulatory Standards
  • GDPR Compliance
  • The Platform Decision
  • Simplifying the Compliance Journey on Databricks
    • Treating Data and AI Assets as Products
    • Detecting and Securing Sensitive Data
    • Architecture Best Practices for Handling Sensitive Data
  • Summary
• 11. Accelerating Unity Catalog Adoption
  • Automatic Enablement of Unity Catalog
    • Default Metastore
    • Default Catalog
    • Default Schema
  • Migrating from HMS to Unity Catalog
    • Upgrade Wizard
    • UCX
      • Workspace group migration
      • Table migration
      • Reconciliation
      • Code migration
  • HMS Federation
    • Supported HMS Variants
      • Internal Databricks HMS
      • External HMS
    • How to Federate HMS
      • Create connection
      • Register locations
      • Create a foreign catalog
      • Fallback mode
  • Summary
• 12. The Future of Unity Catalog
  • Advanced Data Governance
  • Catering to Business Users
    • Unity Catalog Metrics
    • Business User-Friendly Interface
  • Doubling Down on Openness and Interoperability
  • Summary
• Index