Mastering Azure Analytics. Architecting in the Cloud with Azure Data Lake, HDInsight, and Spark - Helion

ebook

Autor: Zoiner Tejada
ISBN: 978-14-919-5660-1
stron: 412, Format: ebook
Data wydania: 2017-04-06
Księgarnia: Helion

Cena książki: 143,65 zł (poprzednio: 167,03 zł)
Oszczędzasz: 14% (-23,38 zł)

Osoby, które kupiły tę książkę, wybierały także »

Microsoft Azure has over 20 platform-as-a-service (PaaS) offerings that can act in support of a big data analytics solution. So which one is right for your project? This practical book helps you understand the breadth of Azure services by organizing them into a reference framework you can use when crafting your own big data analytics solution.

You’ll not only be able to determine which service best fits the job, but also learn how to implement a complete solution that scales, provides human fault tolerance, and supports future needs.

Understand the fundamental patterns of the data lake and lambda architecture
Recognize the canonical steps in the analytics data pipeline and learn how to use Azure Data Factory to orchestrate them
Implement data lakes and lambda architectures, using Azure Data Lake Store, Data Lake Analytics, HDInsight (including Spark), Stream Analytics, SQL Data Warehouse, and Event Hubs
Understand where Azure Machine Learning fits into your analytics pipeline
Gain experience using these services on real-world data that has real-world problems, with scenarios ranging from aviation to Internet of Things (IoT)

Osoby które kupowały "Mastering Azure Analytics. Architecting in the Cloud with Azure Data Lake, HDInsight, and Spark", wybierały także:

Windows Media Center. Domowe centrum rozrywki 66,67 zł, (8,00 zł -88%)
Ruby on Rails. Ćwiczenia 18,75 zł, (3,00 zł -84%)
DevOps w praktyce. Kurs video. Jenkins, Ansible, Terraform i Docker 190,00 zł, (39,90 zł -79%)
Przywództwo w świecie VUCA. Jak być skutecznym liderem w niepewnym środowisku 58,64 zł, (12,90 zł -78%)
Scrum. O zwinnym zarządzaniu projektami. Wydanie II rozszerzone 58,64 zł, (12,90 zł -78%)

Spis treści

Mastering Azure Analytics. Architecting in the Cloud with Azure Data Lake, HDInsight, and Spark eBook -- spis treści

Foreword
Preface
- Conventions Used in This Book
- Using Code Examples
- OReilly Safari
- How to Contact Us
- Acknowledgments
1. Enterprise Analytics Fundamentals
- The Analytics Data Pipeline
- Data Lakes
- Lambda Architecture
- Kappa Architecture
- Choosing Between Lambda and Kappa
- The Azure Analytics Pipeline
- Introducing the Analytics Scenarios
- Example Code and Example Data Sets
- What You Will Need
  - Broadband Internet Connectivity
  - Azure Subscription
  - Visual Studio 2015 with Update 1
  - Azure SDK 2.8 or Later
- Summary
2. Getting Data into Azure
- Ingest Loading Layer
- Bulk Data Loading
  - Disk Shipping
    - Azure Import/Export Service
      - Requirements for import job
      - Preparing a disk
      - Run the WAImportExport tool
      - Creating the import job
  - End User Tools
    - Graphical clients
      - Using Visual Studio Cloud Explorer and Server Explorer to bulk-load to Blob Storage
      - Using Visual Studio Cloud Explorer to bulk-load into your Data Lake Store
      - Microsoft Azure Storage Explorer
      - Azure Portal
      - Third-party clients
      - SSIS Feature Pack for Azure
    - Programmatic clients: Command-line and PowerShell clients
      - AZCopy
      - Using AZCopy to bulk-load files into Blob Storage
      - Azure Command-Line Interface
      - AdlCopy
      - PowerShell cmdlets
      - Bulk loading into Azure Storage account blobs
      - Bulk loading into Azure Data Lake Store
  - Network-Oriented Approaches
    - FTP
    - UDP transfers
    - SMB network shares
    - Hybrid connections and Azure Data Factory
    - Ingesting from a file share to Blob Storage
    - Ingesting from a file share to Azure Data Lake Store
    - Site-to-site networking
      - Express Route
      - Virtual private networks
- Stream Loading
  - Stream Loading with Event Hubs
    - Stream loading with IoT Hub
- Summary
3. Storing Ingested Data in Azure
- File-Oriented Storage
  - Blob Storage
    - Blob Storage capabilities
    - Blob Storage capacity
    - How do we use it in the scenario?
    - Getting the sample data
    - Transfer the files
  - Azure Data Lake Store
    - Whats the capacity?
    - How do we use it in the BYA scenario?
    - Provisioning Data Lake Store
    - Transferring the data
    - Exploring the data via the portal
  - HDFS
    - How do you use HDFS in Azure?
    - How do we use HDFS in the scenario?
- Queue-Oriented Storage
  - Blue Yonder Scenario: Smart Buildings
  - Event Hubs
    - Ingest and storage with Event Hubs
    - Whats the capacity?
    - How do we use Event Hubs in the scenario?
    - Exploring the sensor simulators
      - Introducing the Sensors and SimpleSensorConsole projects
      - Running the Event Hubs load simulator
      - Creating an Event Hubs instance
  - IoT Hub
    - Ingest and storage with IoT Hub
    - Whats the capacity?
    - How do we use IoT Hub in the BYA scenario?
    - Revisiting the Sensors project
      - Running the IoT Hub load simulator
      - Creating an IoT Hub instance
- Summary
4. Real-Time Processing in Azure
- Stream Processing
  - Consuming Messages from Event Hubs
- Tuple-at-a-Time Processing in Azure
  - Introducing HDInsight
  - Storm on HDInsight
    - Applying Storm to Blue Yonder Airports
    - Alerting with Storm on HDInsight (Java + Linux Cluster)
      - Dev environment setup
      - Topology implementation
      - Provisioning the Linux HDI cluster
      - Running the topology on HDI
    - Alerting with Storm on HDInsight (C# + Windows cluster)
      - Dev environment setup
      - Topology implementation
      - Provisioning the Windows HDI cluster
      - Running the topology on HDI
  - EventProcessorHost
    - EventProcessorHost in Web Jobs
  - Azure Machine Learning
- Summary
5. Real-Time Micro-Batch Processing in Azure
- Micro-Batch Processing in Azure
  - Spark Streaming on HDInsight
    - Implementing a Spark Streaming application
    - Running the Spark Streaming application locally
    - Provisioning the Spark Streaming application on an HDInsight cluster
    - Running the Spark Streaming application on HDInsight
  - Storm on HDInsight
    - Storm with Trident
    - Storm with tick tuples
  - Azure Stream Analytics
    - Provisioning a Stream Analytics job
- Summary
6. Batch Processing in Azure
- Batch Processing with MapReduce on HDInsight
  - Apache Hadoop MapReduce
- Batch Processing with Hive on HDInsight
  - Internal and External Tables
  - Partitioning Tables
  - Views
  - Indexes
  - Databases
  - Using Hive on HDInsight
  - Storage on HDInsight
  - Batch Processing Blue Yonder Airports Data
  - Creating an External Table
  - Creating an Internal Table
- Batch Processing with Pig on HDInsight
- Batch Processing with Spark on HDInsight
  - Batch Processing Blue Yonder Airports Data
  - Creating an External Table
- Batch Processing with SQL Data Warehouse
  - Using SQL Data Warehouse
  - Batch Processing Blue Yonder Airports Data
  - Storing the Credentials to Azure Storage
- Batch Processing with Data Lake Analytics
  - Using Data Lake Analytics
  - Batch Processing Blue Yonder Airports Data
  - Processing with U-SQL
- Batch Processing with Azure Batch
- Orchestrating Batch Processing Pipelines with Azure Data Factory
- Summary
7. Interactive Querying in Azure
- Interactive Querying with Azure SQL Data Warehouse
  - Partitions and Distributions
  - Indexes
    - Clustered columnstore index
    - Clustered index
    - Nonclustered index
    - Heap table
  - Interactive Exploration of the Blue Yonder Airports Data
- Interactive Querying with Hive and Tez
  - Indexes
  - Partitions
  - Interactive Exploration of the Blue Yonder Airports Data
- Interactive Querying with Spark SQL
  - Indexes
  - Partitions
  - Interactive Exploration of the Blue Yonder Airports Data
- Interactive Querying with USQL
  - Interactive Exploration of the Blue Yonder Airports Data
- Summary
8. Hot and Cold Path Serving Layer in Azure
- Azure Redis Cache
  - Redis in the Speed Serving Layer
- Document DB
  - Document DB in the Speed Serving Layer
  - Document DB in the Batch Serving Layer
- SQL Database
  - SQL Database in the Speed Serving Layer
  - SQL Database in the Batch Serving Layer
- SQL Data Warehouse
- HBase on HDInsight
- Azure Search
- Summary
9. Intelligence and Machine Learning
- Azure Machine Learning
- R Server on HDInsight
- SQL R Services
- Microsoft Cognitive Services
- Summary
10. Managing Metadata in Azure
- Managing Metadata with Azure Data Catalog
  - Data Catalog in the Blue Yonder Airports Scenario
  - Add an Azure Data Lake Store Asset
  - Add Azure Storage Blobs
  - Add a SQL Data Warehouse
- Summary
11. Protecting Your Data in Azure
- Identity and Access Management
- Data Protection
- Auditing
- Summary
12. Performing Analytics
- Analytics with Power BI
  - Real-Time Power BI in the Blue Yonder Scenario
- Batch Analytics Reporting with Power BI in the Blue Yonder Scenario
- A Look Ahead
  - Real Time
  - Lower Batch Latencies
  - IoT
  - Security
  - More Linux
Index