Programming Pig - Helion

ebook

Autor: Alan Gates
ISBN: 978-14-493-1768-3
stron: 224, Format: ebook
Data wydania: 2011-09-29
Księgarnia: Helion

Cena książki: 118,15 zł (poprzednio: 137,38 zł)
Oszczędzasz: 14% (-19,23 zł)

Osoby, które kupiły tę książkę, wybierały także »

This guide is an ideal learning tool and reference for Apache Pig, the open source engine for executing parallel data flows on Hadoop. With Pig, you can batch-process data without having to create a full-fledged application—making it easy for you to experiment with new datasets.

Programming Pig introduces new users to Pig, and provides experienced users with comprehensive coverage on key features such as the Pig Latin scripting language, the Grunt shell, and User Defined Functions (UDFs) for extending Pig. If you need to analyze terabytes of data, this book shows you how to do it efficiently with Pig.

Delve into Pig’s data model, including scalar and complex data types
Write Pig Latin scripts to sort, group, join, project, and filter your data
Use Grunt to work with the Hadoop Distributed File System (HDFS)
Build complex data processing pipelines with Pig’s macros and modularity features
Embed Pig Latin in Python for iterative processing and other advanced tasks
Create your own load and store functions to handle data formats and storage mechanisms
Get performance tips for running scripts on Hadoop clusters in less time

Osoby które kupowały "Programming Pig", wybierały także:

Windows Media Center. Domowe centrum rozrywki 66,67 zł, (8,00 zł -88%)
Ruby on Rails. Ćwiczenia 18,75 zł, (3,00 zł -84%)
DevOps w praktyce. Kurs video. Jenkins, Ansible, Terraform i Docker 190,00 zł, (39,90 zł -79%)
Przywództwo w świecie VUCA. Jak być skutecznym liderem w niepewnym środowisku 58,64 zł, (12,90 zł -78%)
Scrum. O zwinnym zarządzaniu projektami. Wydanie II rozszerzone 58,64 zł, (12,90 zł -78%)

Spis treści

Programming Pig eBook -- spis treści

Programming Pig
- SPECIAL OFFER: Upgrade this ebook with OReilly
- Preface
  - Data Addiction
  - Who Should Read This Book
  - Conventions Used in This Book
  - Code Examples in This Book
  - Using Code Examples
  - Safari Books Online
  - How to Contact Us
  - Acknowledgments
- 1. Introduction
  - What Is Pig?
    - Pig on Hadoop
      - MapReduces hello world
    - Pig Latin, a Parallel Dataflow Language
      - Comparing query and dataflow languages
      - How Pig differs from MapReduce
    - What Is Pig Useful For?
    - Pig Philosophy
  - Pigs History
- 2. Installing and Running Pig
  - Downloading and Installing Pig
    - Downloading the Pig Package from Apache
    - Downloading Pig from Cloudera
    - Downloading Pig Artifacts from Maven
    - Downloading the Source
  - Running Pig
    - Running Pig Locally on Your Machine
    - Running Pig on Your Hadoop Cluster
    - Running Pig in the Cloud
    - Command-Line and Configuration Options
    - Return Codes
- 3. Grunt
  - Entering Pig Latin Scripts in Grunt
  - HDFS Commands in Grunt
  - Controlling Pig from Grunt
- 4. Pigs Data Model
  - Types
    - Scalar Types
    - Complex Types
      - Map
      - Tuple
      - Bag
    - Nulls
  - Schemas
    - Casts
- 5. Introduction to Pig Latin
  - Preliminary Matters
    - Case Sensitivity
    - Comments
  - Input and Output
    - Load
    - Store
    - Dump
  - Relational Operations
    - foreach
      - Expressions in foreach
      - UDFs in foreach
      - Naming fields in foreach
    - Filter
    - Group
    - Order by
    - Distinct
    - Join
    - Limit
    - Sample
    - Parallel
  - User Defined Functions
    - Registering UDFs
      - Registering Python UDFs
    - define and UDFs
    - Calling Static Java Functions
- 6. Advanced Pig Latin
  - Advanced Relational Operations
    - Advanced Features of foreach
      - flatten
      - Nested foreach
    - Using Different Join Implementations
      - Joining small to large data
      - Joining skewed data
      - Joining sorted data
    - cogroup
    - union
    - cross
  - Integrating Pig with Legacy Code and MapReduce
    - stream
    - mapreduce
  - Nonlinear Data Flows
  - Controlling Execution
    - set
    - Setting the Partitioner
  - Pig Latin Preprocessor
    - Parameter Substitution
    - Macros
    - Including Other Pig Latin Scripts
- 7. Developing and Testing Pig Latin Scripts
  - Development Tools
    - Syntax Highlighting and Checking
    - describe
    - explain
    - illustrate
    - Pig Statistics
    - MapReduce Job Status
    - Debugging Tips
  - Testing Your Scripts with PigUnit
- 8. Making Pig Fly
  - Writing Your Scripts to Perform Well
    - Filter Early and Often
    - Project Early and Often
    - Set Up Your Joins Properly
    - Use Multiquery When Possible
    - Choose the Right Data Type
    - Select the Right Level of Parallelism
  - Writing Your UDF to Perform
  - Tune Pig and Hadoop for Your Job
  - Using Compression in Intermediate Results
  - Data Layout Optimization
  - Bad Record Handling
- 9. Embedding Pig Latin in Python
  - Compile
  - Bind
    - Binding Multiple Sets of Variables
  - Run
    - Running Multiple Bindings
  - Utility Methods
- 10. Writing Evaluation and Filter Functions
  - Writing an Evaluation Function in Java
    - Where Your UDF Will Run
    - Evaluation Function Basics
      - Interacting with Pig values
    - Input and Output Schemas
    - Error Handling and Progress Reporting
    - Constructors and Passing Data from Frontend to Backend
      - Loading the distributed cache
      - UDFContext
    - Overloading UDFs
    - Memory Issues in Eval Funcs
  - Algebraic Interface
  - Accumulator Interface
  - Python UDFs
  - Writing Filter Functions
- 11. Writing Load and Store Functions
  - Load Functions
    - Frontend Planning Functions
      - Determining InputFormat
      - Determining the location
      - Getting the casting functions
    - Passing Information from the Frontend to the Backend
    - Backend Data Reading
      - Getting ready to read
      - Reading records
    - Additional Load Function Interfaces
      - Loading metadata
      - Using partitions
      - Casting bytearrays
      - Pushing down projections
  - Store Functions
    - Store Function Frontend Planning
      - Determining OutputFormat
      - Setting the output location
      - Checking the schema
    - Store Functions and UDFContext
    - Writing Data
      - Preparing to write
      - Writing records
    - Failure Cleanup
    - Storing Metadata
- 12. Pig and Other Members of the Hadoop Community
  - Pig and Hive
  - Cascading
  - NoSQL Databases
    - HBase
    - Cassandra
  - Metadata in Hadoop
- A. Built-in User Defined Functions and Piggybank
  - Built-in UDFs
    - Built-in Load and Store Functions
    - Built-in Evaluation and Filter Functions
      - Built-in math UDFs
      - Built-in aggregate UDFs
      - Built-in chararray and bytearray UDFs
      - Built-in complex type UDFs
      - Built-in filter functions
      - Miscellaneous built-in UDF
  - Piggybank
- B. Overview of Hadoop
  - MapReduce
    - Map Phase
    - Combiner Phase
    - Shuffle Phase
    - Reduce Phase
    - Output Phase
    - Distributed Cache
    - Handling Failure
  - Hadoop Distributed File System
- Index
- About the Author
- Colophon
- SPECIAL OFFER: Upgrade this ebook with OReilly