Data Engineering with Scala and Spark

Most data engineers know that performance issues in a distributed computing environment can easily lead to issues impacting the overall efficiency and effectiveness of data engineering tasks. While Python remains a popular choice for data engineering due to its ease of use, Scala shines in scenarios where the performance of distributed data processing is paramount.
This book will teach you how to leverage the Scala programming language on the Spark framework and use the latest cloud technologies to build continuous and triggered data pipelines. You’ll do this by setting up a data engineering environment for local development and scalable distributed cloud deployments using data engineering best practices, test-driven development, and CI/CD. You’ll also get to grips with DataFrame API, Dataset API, and Spark SQL API and its use. Data profiling and quality in Scala will also be covered, alongside techniques for orchestrating and performance tuning your end-to-end pipelines to deliver data to your end users.
By the end of this book, you will be able to build streaming and batch data pipelines using Scala while following software engineering best practices.

J-Novel Club is a digital publishing company started by translators and fans like you! Our mission is to translate and release the coolest, funnest, and newest light novels from Japan to the world. By focusing on digital releases, and providing a membership service to let people read the books as soon as they are translated, our goal is to build a community of light novel readers and to grow the market, so that more and more releases can be officially licensed and translated. We won't just publish the big hit light novels that get anime adaptations, but also newer titles or books from small publishers and web novels... as long as it's a blast to read, we'll bring it to you! So pull out your tablet or ereader, sit down in a comfy chair, and join the club! weniger anzeigen expand_less