Learners who can write analytical SQL, use Python and pandas, design a star schema and build a simple tested ETL pipeline, and now need the tools and practices that teams use for reliable pipelines at scale.
Intermediate level
Data Engineering
Orchestration with Airflow, Transformations with dbt and Processing with Spark
Build production-style batch data platforms: ELT and lakehouse architecture, workflow orchestration with Apache Airflow, tested and documented SQL transformations with dbt, distributed processing with Apache Spark, data quality and observability, and warehouse performance and cost control.
- 41hours of work
- 13parts
- Reviewedcapstone
- Lifetimeaccess
- ELT and lakehouse design
- Apache Airflow
- dbt
- Apache Spark
- Data observability
- Warehouse optimisation
At a glance
The beginner programme in this track or equivalent: SQL with joins and window functions, Python and pandas, Parquet, dimensional modelling, idempotent loads, Git and basic Docker. A machine with 8 GB of RAM that can run Docker Compose is needed for the labs; a free cloud warehouse account is optional.
Pass the quizzes and labs, then submit a capstone project. The NIBVEDA team reviews it and either accepts it or sends it back with comments you can act on.
What you will be able to do
- 01Explain ELT, lakehouse and medallion architectures and choose appropriate designs
- 02Build reliable Airflow DAGs with correct scheduling, idempotency, retries, backfills and alerting
- 03Develop dbt projects with sources, staging and mart layers, tests, incremental models, snapshots and documentation
- 04Process large datasets with Spark DataFrames, tune shuffles and joins and handle skew and small files
- 05Monitor data quality, freshness and lineage and control warehouse cost and performance
Curriculum 41 hours in 13 parts
Select a kind to highlight its parts.
Stage 1An end-to-end ELT platform with Airflow, dbt and a warehouse6 parts · 16 h
- 1Modern data architecture: ELT, lakes, warehouses and the lakehouseWorkshop · Follow along and build, with the steps and the traps written out.2 hFree
- 2Orchestration with Apache AirflowWorkshop · Follow along and build, with the steps and the traps written out.3 h
- 3A production-style DAG with backfills, sensors and alertsWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 4Check yourself: modern data architecture and AirflowQuiz · Twelve questions, marked at once, with the reason behind each answer.1 h
- 5SQL transformations with dbtWorkshop · Follow along and build, with the steps and the traps written out.3 h
- 6An end-to-end ELT platform with Airflow, dbt and a warehouseLab · Hands-on work. Many labs are checked automatically against what must exist.5 h
Stage 2A Spark batch pipeline on a lakehouse table format with tuning and tests4 parts · 10 h
- 7Apache Spark and distributed data processingWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 8Check yourself: dbt and SparkQuiz · Twelve questions, marked at once, with the reason behind each answer.1 h
- 9Data observability, lineage, governance and cost controlWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 10A Spark batch pipeline on a lakehouse table format with tuning and testsLab · Hands-on work. Many labs are checked automatically against what must exist.5 h
Stage 3Observability, governance, cost and operating model2 parts · 3 h
- 11Putting it together: platform operating model, reviews and team practicesWorkshop · Follow along and build, with the steps and the traps written out.2 h
- 12Check yourself: observability, governance, cost and operating modelQuiz · Twelve questions, marked at once, with the reason behind each answer.1 h
FinishCapstone project1 part · 12 h
- 13A governed ELT platform with orchestration, tests, lakehouse processing and cost controlCapstone · A full project for your portfolio.12 h
What you take away
An internship certificate with your skills on it.
Made for you automatically when you finish the last part, it names the internship programme, the level and the 41 hours, lists what you showed, and carries a number and a QR code. Anyone can check it on the NIBVEDA verification page, and it shows as withdrawn there if it is ever revoked.
- Proof you completed all 41 hours of this internship programme
- An Internship badge for your profile, CV or portfolio
- Not a job, a stipend or an academic degree, and it says so
Keep building.
3 programmes in this track, from beginner to expert. Buy the learning path and save 15%.
Learn how data engineering works from the ground up: SQL for analytics, Python and pandas for wrangling, file formats, ETL pipeline design, dimensional modelling, data quality checks, scheduling and Git, using only free local tools (SQLite, DuckDB and PostgreSQL), and build a warehouse with a dashboard.
- SQL
- Python and pandas
- ETL design
Engineer real-time and large-scale data systems: event time, windows and state in stream processing, Kafka and change data capture, exactly-once pipelines, how Iceberg and Delta tables work internally, data contracts and schema evolution, privacy and deletion at scale, and reliability engineering for data platforms.
- Stream processing
- Kafka and CDC
- Lakehouse table formats
See all 3 programmes in Data Engineering Take the placement test
Quick answers.
The policies have the full detail.
Is this a real internship?
It is a virtual internship programme: a structured, self-paced course built around a real project, with checked assessments and a certificate. It is not employment, it does not carry a stipend and it is not an academic or AICTE-approved qualification. What it gives you is verified, project-based evidence of skills.
How does self-paced work?
You start when you like and move at your own speed. Each part unlocks when you finish the one before it. There are no live classes and no deadlines. Your access is shown on each programme page.
How is my work checked?
Quizzes are marked as soon as you submit them. Labs use a tick-list of tasks. Intermediate and Expert programmes finish with a capstone that the NIBVEDA team reviews and either accepts or sends back with comments.
Can I get a refund?
Please read the refund policy before you buy. It explains when a refund is possible.
Ready to start?
Rs. 765.82 including GST. Lifetime access. Read the first lesson free before you decide.