Monthly Subscription Starting at AED 99 + VAT

Scripting with Python and SQL for Data Engineering
  • Level Professional
  • Duration 18 hours
  • Course by Duke University
  • Offered by

About

In this third course of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will explore techniques to work effectively with Python and SQL. We will go through useful data structures in Python scripting and connect to databases like MySQL. Additionally, you will learn how to use a modern text editor to connect and run SQL queries against a real database, performing operations to load and extract data. Finally, you will use extracted data from websites using scraping techniques. These skills will allow you to work effectively when data is not readily available, or when spatial queries are required to extract useful information from databases.

Modules

About the Course

    • 3 Videos
    • 3 Readings
    • 1 Discussion
Show info about module content

1 Discussions

  • Meet and Greet (optional)

3 Videos

  • Welcome to Scripting with Python and SQL for Data Engineering
  • Meet your Course Instructor: Alfredo Deza
  • Overview of Key Concepts

3 Readings

  • Connect with your instructor
  • Meet your Supporting Instructors: Kennedy Behrman and Noah Gift
  • Course Structure and Discussion Etiquette

Exploring Data Structures in Python

    • 6 Videos
    • 2 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Using Data Structures in Python

6 Videos

  • Introduction to Working with Data in Python
  • Introduction to Python Data Structures
  • Using Lists to Save and Retrieve Data in Python
  • Using Dictionaries to Save and Retrieve Data in Python
  • Overview of Less Common Data Structures in Python
  • Recap of Data Structures in Python

2 Readings

  • Key Terms
  • Python Lists and Dictionaries

Mapping Data Structures in Python

    • 5 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Looping Over Lists and Dictionaries in Python

5 Videos

  • Introduction to Choosing Data Structures in Python
  • Iterating Over Lists and Dictionaries in Python
  • Iterating Over Other Data Structures in Python
  • Storing Data Between Data Structures in Python
  • Recap of Mapping Data Structures in Python

1 Readings

  • Key Terms

Persisting and Loading Data in Python

    • 5 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Loading and Saving JSON Files

5 Videos

  • Introduction to Data Sources and Formats in Python
  • Loading Data from Files and File Paths in Python
  • Working with JSON in Python
  • Saving Data from Python to Disk
  • Recap of Persisting and Loading Data in Python

1 Readings

  • Key Terms

Putting it all Together: JSON Formatting Jupyter Notebook

    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Working with Data in Python

1 Readings

  • Build a notebook that reformats data into JSON

Python Scripting Basics

    • 5 Videos
    • 2 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Build a Script to Find Large Files

5 Videos

  • Introduction to Python Scripting and SQL
  • Introduction to Scripting in Python
  • Creating a Script as a Module in Python
  • Traversing the File System with a Script in Python
  • Recap of Python Scripting Basics

2 Readings

  • Key Terms
  • Minimal Python, Chapter 2: Learn to Store Data

SQLite and Python

    • 5 Videos
    • 2 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Create a SQLite Database and Store Data

5 Videos

  • Introduction to Embedded Databases
  • What is SQLite?
  • Creating and Connecting to a SQLite Database in Python
  • Saving and Querying from a SQLite Database in Python
  • Recap of SQLite and Python

2 Readings

  • Key Terms
  • Example GitHub Repository

Querying Databases with SQL

    • 5 Videos
    • 2 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Querying a SQLite Database

5 Videos

  • Introduction to Querying with SQL in Python
  • Basic SQL Commands in Python
  • Extracting Distinct Data using SQL in Python
  • Searching with SQL in Python
  • Recap of Querying with SQL in Python

2 Readings

  • Key Terms
  • Example GitHub Repository

Putting it all Together: Create a Reporting Script for File Sizes

    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Python Scripting and SQL

1 Readings

  • Create a Reporting Script for File Sizes

Introduction to Web Scraping

    • 5 Videos
    • 2 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Parse HTML with HTMLParser

5 Videos

  • Introduction to Web Scraping using Python
  • Introduction to Extracting Data from Unstructured HTML
  • Challenges with Web Data
  • Parsing HTML with HTMLParser in Python
  • Recap of Web Scraping Techniques in Python

2 Readings

  • Key Terms
  • Parsing Techniques with HTMLParser

Using Scrapy and XPath

    • 5 Videos
    • 2 Readings
Show info about module content

5 Videos

  • Introduction to Scrapy and XPath in Python
  • Creating a Web Scraping Project with Scrapy in Python
  • Parsing Data with XPath and Scrapy Shell
  • Using Scrapy Spider for Web Scraping
  • Recap of Scrapy and XPath in Python

2 Readings

  • Key Terms
  • Codespaces Lab: Parse HTML with Scrapy and XPath

Persistence and Efficiency with Scraping

    • 5 Videos
    • 2 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Parse HTML and Persist it to a SQLite Database

5 Videos

  • Overview of Challenges with Web Scraping
  • Scraping Locally
  • Persisting Data in CSV and JSON Formats
  • Persisting Data to a SQLite Database
  • Recap of Persistence and Efficacy with Web Scraping

2 Readings

  • Key Terms
  • Efficient Scraping Techniques

Putting it all Together: Build a Web Scraping Tool

    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Web Scraping using Python

1 Readings

  • Build a Web Scraping Tool

Using VSCode and MySQL

    • 4 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Connect to a Running MySQL Server

4 Videos

  • Introduction to Working with MySQL
  • Introduction to VSCode with MySQL
  • Connecting to a MySQL Server
  • Recap of VSCode with MySQL

1 Readings

  • Key terms

Running SQL Queries from VSCode

    • 3 Videos
    • 2 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Running Queries Repository

3 Videos

  • Challenges of Running MySQL Queries
  • Using VSCode to Execute MySQL Queries
  • Recap of Running MySQL Queries

2 Readings

  • Key terms
  • Working with VSCode and MySQL

Importing and Exporting Data in MySQL

    • 11 Videos
    • 2 Readings
    • 2 Lab
Show info about module content

2 Labs

  • Export and Import Data in MySQL
  • Linux Hacking with MySQL

11 Videos

  • Challenges with Importing to Databases
  • Importing CSV Data into MySQL
  • Exporting Data from MySQL
  • Recap of Importing and Exporting Data in MySQL
  • MySQL Hacking Overview
  • MySQL from Terminal
  • Archive and Drop Database
  • Import external database Sakila
  • Modify database Sakila
  • Bash pipelines with MySQL
  • MySQL to Python Standard Library Web Server

2 Readings

  • Key Terms
  • Lesson Reflection

Putting it all Together: Loading and Exporting Data using MySQL

    • 2 Readings
    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Working with MySQL

1 Labs

  • MySQL Sandbox

2 Readings

  • Loading and Exporting Data using MySQL
  • Next Steps

Auto Summary

Explore effective data engineering techniques with "Scripting with Python and SQL for Data Engineering." This professional-level course, taught by Coursera, focuses on Python scripting, MySQL databases, and web scraping. Over 1080 minutes, learners will master essential skills for handling and extracting data, available via Starter and Professional subscriptions. Ideal for those in Big Data and Analytics.

Alfredo Deza
Instructors

Alfredo Deza

Kennedy Behrman
Instructors

Kennedy Behrman

Noah Gift
Instructors

Noah Gift