Data Engineering with MS Azure Synapse Apache Spark Pools
In this course, you will learn how to perform data engineering with Azure Synapse Apache Spark Pools, which enable you to boost the performance of big-data analytic applications by in-memory cluster computing. You will learn how to differentiate between Apache Spark, Azure Databricks, HDInsight, and SQL Pools and understand the use-cases of data-engineering with Apache Spark in Azure Synapse Analytics. You will also learn how to ingest data using Apache Spark Notebooks in Azure Synapse Analytics and transform data using DataFrames in Apache Spark Pools in Azure Synapse Analytics.
Monthly Subscription Starting at AED 99 + VAT
- Level Professional
- Duration 8 hours
- Course by Microsoft
-
Offered by
About
In this course, you will learn how to perform data engineering with Azure Synapse Apache Spark Pools, which enable you to boost the performance of big-data analytic applications by in-memory cluster computing. You will learn how to differentiate between Apache Spark, Azure Databricks, HDInsight, and SQL Pools and understand the use-cases of data-engineering with Apache Spark in Azure Synapse Analytics. You will also learn how to ingest data using Apache Spark Notebooks in Azure Synapse Analytics and transform data using DataFrames in Apache Spark Pools in Azure Synapse Analytics. You will integrate SQL and Apache Spark pools in Azure Synapse Analytics. You will also learn how to monitor and manage data engineering workloads with Apache Spark in Azure Synapse Analytics. This course is part of a Specialization intended for Data engineers and developers who want to demonstrate their expertise in designing and implementing data solutions that use Microsoft Azure data services for anyone interested in preparing for the Exam DP-203: Data Engineering on Microsoft Azure (beta). You will take a practice exam that covers key skills measured by the certification exam. This is the sixth course in a program of 10 courses to help prepare you to take the exam so that you can have expertise in designing and implementing data solutions that use Microsoft Azure data services. The Data Engineering on Microsoft Azure exam is an opportunity to prove knowledge expertise in integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building analytics solutions that use Microsoft Azure data services. Each course teaches you the concepts and skills that are measured by the exam. By the end of this Specialization, you will be ready to take and sign-up for the Exam DP-203: Data Engineering on Microsoft Azure (beta).Modules
Welcome to the course
-
1
Videos
-
2
Readings
-
1
Discussion
1
Discussions
- Meet and greet
1
Videos
- Introduction to the course
2
Readings
- Course syllabus
- How to be successful in this course
Understand big data engineering with Apache Spark in Azure Synapse Analytics
-
3
Videos
-
1
Readings
-
1
Assignment
1
Assignment
- Knowledge check
3
Videos
- What is an Apache Spark pool in Azure Synapse Analytics?
- How do Apache Spark pools work in Azure Synapse Analytics?
- Lesson summary
1
Readings
- When do you use Apache Spark pools in Azure Synapse Analytics?
Ingest data with Apache Spark notebooks in Azure Synapse Analytics
-
4
Videos
-
8
Readings
-
1
Assignment
1
Assignment
- Knowledge check
4
Videos
- Introduction to spark notebooks
- Understand the use-cases for spark notebooks
- Run spark notebooks
- Lesson summary
8
Readings
- Create a spark notebook in Azure Synapse Analytics
- Discover supported languages in spark notebooks
- Develop spark notebooks
- Develop spark notebooks
- Run spark notebooks
- Load data in Spark notebooks
- Load data in Spark notebooks
- Save Spark notebooks
Transform data with DataFrames in Apache Spark Pools in Azure Synapse Analytics
-
4
Videos
-
3
Readings
-
2
Assignment
2
Assignment
- Knowledge check
- Test prep
4
Videos
- Introduction to DataFrames in Spark pools in Azure Synapse Analytics
- Load data in a Spark DataFrame
- Flatten nested structures and explode arrays with Apache Spark
- Lesson summary
3
Readings
- Load data into a Spark DataFrame
- Create an Apache Spark table
- Flatten nested structures and explode arrays with Apache Spark in synapse
Integrate SQL and Apache Spark pools in Azure Synapse Analytics
-
6
Videos
-
4
Readings
-
1
Assignment
1
Assignment
- Knowledge check
6
Videos
- Describe the integration methods between SQL and Spark pools in Azure Synapse Analytics
- Understand the use-cases for SQL and Spark pools integration
- Authenticate in Azure Synapse Analytics
- Externalize the use of Spark pools within Azure Synapse Workspace
- Transfer data outside the Synapse workspace using the PySpark connector
- Lesson summary
4
Readings
- Transfer data between SQL and Spark pool in Azure Synapse Analytics
- Authenticate between Spark and SQL pool in Azure Synapse Analytics
- Integrate SQL and Spark pools in Azure Synapse Analytics
- Transfer data outside the Synapse workspace using the PySpark connector
Monitor and manage data engineering workloads with Apache Spark in Azure Synapse Analytics
-
3
Videos
-
2
Readings
-
2
Assignment
2
Assignment
- Knowledge check
- Test prep
3
Videos
- Monitor Spark pools in Azure Synapse Analytics
- Optimize Apache Spark jobs in Azure Synapse Analytics
- Lesson summary
2
Readings
- Base-line Apache Spark performance with Apache Spark history server in Azure Synapse Analytics
- Automate scaling of Apache Spark pools in Azure Synapse Analytics
Course practice exam
-
1
Videos
-
1
Readings
-
1
Assignment
1
Assignment
- Course practice exam
1
Videos
- Course 6 recap
1
Readings
- About the practice exam
Course wrap up
-
1
Videos
-
1
Readings
-
1
Discussion
1
Discussions
- Reflect on learning
1
Videos
- Course wrap up
1
Readings
- Next steps
Auto Summary
Unlock the potential of big-data analytics with the "Data Engineering with MS Azure Synapse Apache Spark Pools" course, designed for data science and AI enthusiasts. Guided by expert instructors, this course empowers you to elevate your data engineering skills using Azure Synapse Apache Spark Pools, known for their in-memory cluster computing capabilities. Dive deep into differentiating Apache Spark from Azure Databricks, HDInsight, and SQL Pools, and explore practical use-cases of data engineering within Azure Synapse Analytics. Learn to ingest and transform data using Apache Spark Notebooks and DataFrames, integrate SQL and Apache Spark pools, and efficiently manage data engineering workloads. This course is the sixth installment in a comprehensive 10-course specialization aimed at preparing you for the Exam DP-203: Data Engineering on Microsoft Azure. It's tailored for aspiring data engineers and developers looking to showcase their expertise in designing and implementing data solutions with Microsoft Azure. With a total duration of 480 minutes, the course offers flexible subscription options, including Starter and Professional plans, to accommodate your learning needs. By the end of this specialization, you'll be thoroughly prepared to take the certification exam and demonstrate your proficiency in integrating, transforming, and consolidating data for analytics solutions using Azure data services. Targeted at professional-level learners, this engaging course is presented by Coursera, ensuring you receive top-quality instruction and resources to advance your career in data engineering.
Microsoft