Skip to main content

Section outline

    • What is Apache Flink and stream processing

    • Job Manager, Task Manager, Flink deployments (Local, YARN, Kubernetes)

    • DataStream vs Dataset API, Web UI overview

    • Lab: Set up Flink and run your first Flink job

    • Kafka integration, Flink transformations, event-time processing

    • Checkpointing, fault tolerance, and Flink savepoints

    • Stateful operators and windowing (Tumbling, Sliding, Session)

    • Lab: Implement custom sources/sinks, use Watermarks and state

    • Table API and SQL queries on streaming data

    • Flink Connectors (Kafka, S3, JDBC, Elasticsearch)

    • Authentication (Kerberos, OAuth), RBAC, TLS configuration

    • Lab: SQL queries on Kafka streams, secure Flink job deployment

    • Resource optimization, memory and GC tuning, Flink metrics

    • Complex Event Processing (CEP), Flink ML use cases

    • Final capstone project: end-to-end Flink pipeline (fraud detection, analytics, etc.)

    • Quiz and best practices for production