I had a nice experience at Creative-I-Lab. I liked the way the classes were conducted and the learning environment was comfortable. The teachers were supportive and I learned a lot during the course.
Azure Data Engineering
Master end-to-end data engineering on Microsoft Azure by learning how to build real-world data pipelines from scratch. This 90-hour program combines Azure Data Factory and Azure Databricks to help you understand how data is ingested, processed, and transformed into actionable insights.
19 Lessons
18 Focused Modules & Capstone Project
Intermediate Level
Production-Grade Cloud Architecture
ADF & Databricks
PySpark, Delta Lake & Pipeline Orchestration
Capstone Project
End-to-End Real-World Pipeline Implementation
What you’ll learn
Ingest, transform, and orchestrate big data using Azure Data Factory (ADF).
Process large-scale datasets with Azure Databricks and PySpark.
Implement modern Delta Lake and Lakehouse architectures.
Secure data solutions with Azure Key Vault and query data using Azure Data Explorer.
Build, run, and schedule real-world production-grade ETL pipelines.
Design end-to-end data pipelines through a practical Capstone Project.
Skills you'll gain
Data Pipeline Architecture
ETL/ELT Development
Big Data Processing
Data Lakehouse Architecture
Orchestration & Triggering
State Management
Real-Time Data Analysis
Production Monitoring
Tools you'll learn
Azure Data Factory
Azure Databricks
PySpark
Delta Lake
Azure Key Vault
Azure Data Explorer
SQL / Scripting
Prepare for a career in cloud data engineering with comprehensive hands-on training. You will start with the fundamentals of data engineering and Azure architecture, move into building robust ingestion pipelines using Azure Data Factory, and then dive into large-scale data processing using Azure Databricks and PySpark. The course also covers Delta Lake, modern data lake architecture, and real-world pipeline orchestration.
By the end of this program, you will be able to design, build, and manage complete data pipelines used in production environments.
- Course Level: Intermediate
- Total Duration: 90 Hours
- Total Curriculum: 19 Lessons
- Training Mode: Classroom (Offline)
Curriculum
90 Hours
|
Intermediate
What you’ll learn
Foundations of cloud-based data integration and ingestion workflows.
Navigating the Azure Data Factory interface and ecosystem.
Skills you'll gain
ADF Basics
Cloud Architecture
What you’ll learn
Configure Copy Activities for data transfer across disparate storage systems.
Build visual, code-free data transformations using Data Flow Activities.
Skills you'll gain
Data Ingestion
Data Transformation
ADF Copy Activity
Data Flows
What you’ll learn
Loop through dataset arrays using For Each and Until activities.
Control pipeline execution branching with If and Switch logic.
Skills you'll gain
Pipeline Logic
Conditional Branching
Control Flow
What you’ll learn
Store and dynamically update state variables across pipeline executions using Set and Append activities.
Skills you'll gain
State Management
Variable Handling
Dynamic Pipelines
What you’ll learn
Query external metadata and lookup values to inform pipeline flow.
Apply filter criteria to process specific dataset arrays dynamically.
Skills you'll gain
Dynamic Lookups
Data Filtering
Metadata Processing
What you’ll learn
Connect to external REST APIs and HTTP endpoints via Web activities.
Execute custom SQL or script operations within ADF pipelines.
Skills you'll gain
API Integration
SQL Scripting
External Services
What you’ll learn
Manage execution pauses with Wait activities.
Implement custom error catching and pipeline termination using Fail activities.
Skills you'll gain
Error Handling
Exception Management
Pipeline Guardrails
What you’ll learn
Automate post-processing cleanups and data retention policies using Delete activities.
Skills you'll gain
Data Retention
Storage Cleanup
What you’ll learn
Trigger and run external Databricks PySpark notebooks directly from ADF pipelines.
Skills you'll gain
Pipeline Integration
Notebook Execution
What you’ll learn
Set up event-based, schedule-based, and tumbling window triggers for pipeline automation.
Skills you'll gain
Automated Scheduling
Event-driven Pipelines
What you’ll learn
Securely manage pipeline credentials, access tokens, and connection strings.
Skills you'll gain
Secrets Management
Data Security
What you’ll learn
Perform rapid real-time telemetry and log data analytics using KQL.
Skills you'll gain
Real-time Analytics
KQL Querying
What you’ll learn
Present end-to-end ETL design architectures and best practices.
Skills you'll gain
Architecture Presentation
ETL Design
What you’ll learn
Navigate the Databricks Lakehouse, configure Spark clusters, and handle PySpark DataFrames.
Skills you'll gain
Spark Engine
PySpark DataFrames
Lakehouse Architecture
What you’ll learn
Perform complex data transformations, aggregations, and joins at scale.
Skills you'll gain
Distributed Computing
PySpark Transformations
What you’ll learn
Understand Spark architecture, process streaming data, and implement Delta Lake features.
Skills you'll gain
Structured Streaming
Delta Lake
ACID Transactions
What you’ll learn
Query, optimize, and analyze Delta tables for downstream analytics reporting.
Skills you'll gain
Delta Lake Analysis
Performance Tuning
What you’ll learn
Solve real-world data engineering scenarios and practice complex query logic.
Skills you'll gain
Hands-on Problem Solving
Pipeline Troubleshooting
What you’ll learn
Architect and build a complete end-to-end data pipeline from ingestion to analytics storage.
Skills you'll gain
Production Pipeline Design
End-to-End Implementation
Success Stories That Inspire
See how our students transformed their skills, built strong portfolios, and launched successful creative careers.
I joined Creative-I-Lab to improve my skills and I am happy with my decision. The classes are easy to understand and the trainer explains the topics with good examples. I also liked that the batches are small.
Sachin Ghule
I really liked learning at Creative-I-Lab. The teachers are friendly and explain everything in a simple way. The classes are easy to understand and the overall experience was good.
