Experienced data engineers who build batch pipelines and now need to design streaming systems, operate lakehouse storage, enforce contracts between teams and run data platforms with SLOs and on-call.
Expert level
Data Engineering
Kafka, Flink, Iceberg and Data Contracts
Engineer real-time and large-scale data systems: event time, windows and state in stream processing, Kafka and change data capture, exactly-once pipelines, how Iceberg and Delta tables work internally, data contracts and schema evolution, privacy and deletion at scale, and reliability engineering for data platforms.
- 46hours of work
- 13parts
- Reviewedcapstone
- Lifetimeaccess
- Stream processing
- Kafka and CDC
- Lakehouse table formats
- Data contracts
- Privacy and deletion at scale
- Data reliability engineering
At a glance
The intermediate programme in this track or equivalent: Airflow, dbt, Spark, medallion layers, data quality and observability. Comfort with Python or Java or Scala, Docker Compose, SQL and basic distributed systems ideas (partitions, replication, consensus) is expected. A machine with 16 GB of RAM is recommended for the labs.
Pass the quizzes and labs, then submit a capstone project. The NIBVEDA team reviews it and either accepts it or sends it back with comments you can act on.
What you will be able to do
- 01Reason about event time, watermarks, windows, state and delivery guarantees and design correct streaming computations
- 02Operate Kafka topics and consumers and build change data capture pipelines with Debezium and schema registries
- 03Explain how Iceberg and Delta tables achieve transactions, time travel and schema evolution, and maintain them in production
- 04Define and enforce data contracts and manage schema evolution across producers and consumers
- 05Run a data platform with SLOs, incident response, privacy and deletion workflows and cost governance
Curriculum 46 hours in 13 parts
Select a kind to highlight its parts.
Stage 1An exactly-once streaming lakehouse pipeline with backfill, maintenance and measured guarantees7 parts · 20 h
- 1Streaming fundamentals: events, time, windows, state and guaranteesWorkshop · Follow along and build, with the steps and the traps written out.3 hFree
- 2Kafka in depth and change data captureWorkshop · Follow along and build, with the steps and the traps written out.3 h
- 3A CDC pipeline from PostgreSQL through Kafka into the lake with fault injectionWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 4Check yourself: streaming concepts, Kafka and CDCQuiz · Twelve questions, marked at once, with the reason behind each answer.1 h
- 5Stream processing engines: Flink, Spark Structured Streaming and Kafka StreamsWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 6Lakehouse table formats: how Iceberg and Delta work, and how to run themWorkshop · Follow along and build, with the steps and the traps written out.3 h
- 7An exactly-once streaming lakehouse pipeline with backfill, maintenance and measured guaranteesLab · Hands-on work. Many labs are checked automatically against what must exist.6 h
Stage 2Data contracts, quality gates, SLOs and a game day for a multi-team platform4 parts · 11 h
- 8Check yourself: stream engines and lakehouse tablesQuiz · Twelve questions, marked at once, with the reason behind each answer.1 h
- 9Data contracts, schema evolution and platform governanceWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 10Reliability engineering for data platformsWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 11Data contracts, quality gates, SLOs and a game day for a multi-team platformLab · Hands-on work. Many labs are checked automatically against what must exist.6 h
Stage 3Data contracts, governance and reliability1 part · 1 h
- 12Check yourself: data contracts, governance and reliabilityQuiz · Twelve questions, marked at once, with the reason behind each answer.1 h
FinishCapstone project1 part · 14 h
- 13A governed real-time data platform with contracts, exactly-once guarantees and operational evidenceCapstone · A full project for your portfolio.14 h
What you take away
An internship certificate with your skills on it.
Made for you automatically when you finish the last part, it names the internship programme, the level and the 46 hours, lists what you showed, and carries a number and a QR code. Anyone can check it on the NIBVEDA verification page, and it shows as withdrawn there if it is ever revoked.
- Proof you completed all 46 hours of this internship programme
- An Internship badge for your profile, CV or portfolio
- Not a job, a stipend or an academic degree, and it says so
Keep building.
3 programmes in this track, from beginner to expert. Buy the learning path and save 15%.
Build production-style batch data platforms: ELT and lakehouse architecture, workflow orchestration with Apache Airflow, tested and documented SQL transformations with dbt, distributed processing with Apache Spark, data quality and observability, and warehouse performance and cost control.
- ELT and lakehouse design
- Apache Airflow
- dbt
Learn how data engineering works from the ground up: SQL for analytics, Python and pandas for wrangling, file formats, ETL pipeline design, dimensional modelling, data quality checks, scheduling and Git, using only free local tools (SQLite, DuckDB and PostgreSQL), and build a warehouse with a dashboard.
- SQL
- Python and pandas
- ETL design
See all 3 programmes in Data Engineering Take the placement test
Quick answers.
The policies have the full detail.
Is this a real internship?
It is a virtual internship programme: a structured, self-paced course built around a real project, with checked assessments and a certificate. It is not employment, it does not carry a stipend and it is not an academic or AICTE-approved qualification. What it gives you is verified, project-based evidence of skills.
How does self-paced work?
You start when you like and move at your own speed. Each part unlocks when you finish the one before it. There are no live classes and no deadlines. Your access is shown on each programme page.
How is my work checked?
Quizzes are marked as soon as you submit them. Labs use a tick-list of tasks. Intermediate and Expert programmes finish with a capstone that the NIBVEDA team reviews and either accepts or sends back with comments.
Can I get a refund?
Please read the refund policy before you buy. It explains when a refund is possible.
Ready to start?
Rs. 1,178.82 including GST. Lifetime access. Read the first lesson free before you decide.