Learning Spark Jules Damji, Brooke Wenig, and Tathagata Das is an English-language printed study book aimed at data engineers and scientists. It covers how to efficiently process data workloads with Apache Spark, including machine learning and open source tools.
This edition highlights new features in Spark 2.4 and provides practical insights into using Python, SQL, Scala, and Java through DataFrames and Datasets. You will learn, among other things:
- How the Spark SQL engine works and optimizations
- How to tune and debug Spark performance with configurations and the UI
- Connecting to various data formats and sources such as JSON, Parquet, Hive, and Kafka
- Batch and streaming analytics with Structured Streaming
- Building data pipelines with Delta Lake
- Machine learning pipelines with MLlib and using MLflow
- Data transformation and feature engineering with the open source Pandas framework Koalas
This book connects to topics such as databases, data management, algorithms, data structures, and data science.