Azure Data Factory For Data Engineers - Project on Covid19
Sunday, December 11, 2022
Azure Data Factory For Data Engineers - Project on Covid19 - Real world project for Data Engineers using Azure Data Factory, SQL, Data Lake, Databricks, HDInsight, CI/CD [DP203]
Major updates to the course since the launch
November 2022 - Addition of sections 15 & 16 focusing on Continuous Integration & Continuous Delivery (CI/CD)
What you'll learn
- You will learn how to build a real-world data pipeline in Azure Data Factory (ADF). This course has been taught using real world data used to report Covid-19 trends.
- You will acquire good Data Engineering skills in Azure using Azure Data Factory (ADF), Azure Data Lake Storage Gen2, Azure SQL Database, Azure Blob Storage and Azure Monitor
- You will learn how to ingest data from sources such as HTTP and Azure Blob Storage into Azure Data Lake Gen2 using Azure Data Factory (ADF)
- You will learn how to transform data using Data Flows in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
- You will learn how to transform data using Databricks Notebook Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
- You will learn how to transform data using Azure HDInsight Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
- You will learn how to load transformed data from Azure Data Lake Storage Gen2 to Azure SQL Database using Azure Data Factory (ADF)
- You will learn extensively about Triggers in Azure Data Factory (ADF) and how to use them to schedule the data pipelines.
- You will learn how to monitor pipelines using Azure Data Factory (ADF), Azure Monitor and Log Analytics with a real-world project.
- You will learn how to build production ready pipelines and good practices and naming standards
- You will gain most of the skills required to pass the Azure Data Engineer Associate certification exams DP200 and DP200, but the primary objective of the course is not to teach you to pass the exams.
2022 Azure Data Factory Basics for Azure Data Engineer DP203
Saturday, November 5, 2022
2022 Azure Data Factory Basics for Azure Data Engineer DP203 - Learn Data engineering on Azure Data Factory real world projects in 1.5 hour. Start your career as Azure Data Engineer
- New
- Created by Vijay S
In this course, you are going to learn how to build data pipelines with azure data factory for real-world projects from an industry expert with hands-on Labs.
Anyone can spend 1.5 hours time and become an Azure Data engineer!
This course will provide what is data factory, what are the various core components in azure data factory.
This course is useful for those aspiring to become azure data engineers.
Those who want to switch their career from soft engineering to data engineer
To expand their skills from AWS, GCP data engineer to AZURE data engineer
to gain an in-depth understanding of the latest trends in azure data engineering
This course covers the following topics.
Basic concepts of various components in ADF
What are ETL and ELT
What is Linked service,
Demo for creating Linked service
What is a data set,
Demo for creating Data set
What is Integration run time
What is an Azure Data lake
Demo for creating Azure data lake storage account
What is Cosmos DB
Demo for creating Cosmos DB
What is COPY activity
Demo for creating a data pipeline to move data from ADLS 2 to Cosmos DB using COPY Activity
What are control flow activities-
Get Metadata activity
Demo for getting Metadata activity
Filter activity
Demo for Filter activity
ForEach activity
Demo for ForEach activity
Demo for other Control flow activities
What is DATA FLOW Activity
Demo for removing Duplicate data using DATA FLOW Activity
The course is useful not only for freshers but also for experienced data engineers to gain more skills in azure data engineering
What you'll learn
- Data engineering with Azure data factory in real world projects to process large scale data
- Data movement and migration using copy data activity from various sources
- Data transformations and processing using Data flow activity
- Creating linked services to connect with various data sources
- Creating Integration runtime to run the ADF pipeline
- Implementing different types of triggers in Azure Data Factory
- Data monitoring and and alerts in Azure Data factory