Section outline

• Select section Day 1 – Hadoop & Hive Fundamentals

  Day 1 – Hadoop & Hive Fundamentals

  Collapse all Expand all

  • HDFS, YARN, Namenode/DataNode Architecture

  • MapReduce Basics & Hive Setup

  • ETL with Hive, Hive SerDes, Partitioning

  • Lab: HDFS commands, MapReduce job, Hive ETL queries
• Select section Day 2 – Spark Core & Performance Tuning

  Day 2 – Spark Core & Performance Tuning

  • Spark vs Hadoop, Lambda Architecture

  • RDD Operations, Caching, Checkpointing

  • Spark Internals: DAG, Partitions, Shuffling

  • Performance Tuning and Cluster Setup

  • Lab: RDD manipulation, caching, metrics analysis
• Select section Day 3 – Spark SQL & DataFrames

  Day 3 – Spark SQL & DataFrames

  • DataFrame APIs, Catalyst Optimizer, Tungsten Engine

  • Working with CSV, JSON, XML, Parquet, ORC

  • Advanced SQL: Joins, Window Functions, Aggregations

  • Performance Optimizations in Spark SQL

  • Lab: SQL queries, UDFs, Hive Integration
• Select section Day 4 – Streaming with Spark & Kafka

  Day 4 – Streaming with Spark & Kafka

  • DStream, Windowing, Fault Tolerance

  • Kafka Architecture and Cluster Configuration

  • Kafka-Spark Streaming Integration

  • Lab: Real-time analysis with Twitter & Kafka streams
• Select section Day 5 – Apache Iceberg

  Day 5 – Apache Iceberg

  • Iceberg Architecture, Schema Evolution, Time Travel

  • Integration with Hive and Hadoop

  • Performance Optimizations & GDPR Compliance

  • Lab: Create, Query, Optimize Iceberg Tables using Spark SQL