Monthly Subscription Starting at AED 99 + VAT

Web Applications and Command-Line Tools for Data Engineering
  • Level Professional
  • Duration 15 hours
  • Course by Duke University
  • Offered by

About

In this fourth course of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will build upon the data engineering concepts introduced in the first three courses to apply Python, Bash and SQL techniques in tackling real-world problems. First, we will dive deeper into leveraging Jupyter notebooks to create and deploy models for machine learning tasks. Then, we will explore how to use Python microservices to break up your data warehouse into small, portable solutions that can scale. Finally, you will build a powerful command-line tool to automate testing and quality control for publishing and sharing your tool with a data registry.

Modules

About the Course

    • 2 Videos
    • 3 Readings
    • 1 Discussion
Show info about module content

1 Discussions

  • Meet and Greet (optional)

2 Videos

  • Introduction to Web Applications and Command-Line Tools for Data Engineering
  • Overview of Key Concepts

3 Readings

  • Meet your Instructors
  • Welcome
  • Course Structure and Etiquette

Introduction to Jupyter Notebooks

    • 6 Videos
    • 1 Readings
    • 3 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Introduction to Jupyter

3 Labs

  • Code Cells in Jupyter
  • Text Cells in Jupyter
  • Magics in Jupyter

6 Videos

  • Introduction to Jupyter Notebooks
  • Getting Started with Jupyter
  • Code Cells in Jupyter
  • Text Cells in Jupyter
  • Magics in Jupyter
  • Overview of Jupyter Lab

1 Readings

  • Key Terms

Google Colab

    • 3 Videos
    • 2 Readings
    • 3 Assignment
Show info about module content

3 Assignment

  • Introduction to Colab
  • Colab Features
  • Data and Documents in Colab

3 Videos

  • Introduction to Colab
  • Tour of Colab Features
  • Data and Documents in Colab

2 Readings

  • Key Terms
  • Important Notebook Links

AWS SageMaker

    • 3 Videos
    • 2 Readings
    • 3 Assignment
Show info about module content

3 Assignment

  • Introduction to SageMaker
  • SageMaker Studio
  • SageMaker Pipelines

3 Videos

  • Introduction to SageMaker
  • Tour of SageMaker Studio
  • Overview of SageMaker Pipelines

2 Readings

  • Key Terms
  • Get started with Code Editor in Amazon SageMaker Studio

Putting it all Together: Notebooks for Data Engineering

    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Jupyter Notebooks

1 Labs

  • Notebook Review

Exploring the Benefits and Characteristics of Microservices

    • 4 Videos
    • 2 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Quiz-Exploring the Benefits and Characteristics of Microservices

4 Videos

  • Introduction to Building Python Microservices
  • What are the Benefits of Microservices?
  • Setting up Python Project Structure for Continuous Integration
  • Building a Random Fruit Web App with Python

2 Readings

  • Key Terms
  • Lesson Reflection

Building Python Microservices

    • 3 Videos
    • 2 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Quiz-Building Python Microservices

3 Videos

  • Introduction to Python Microservices with FastAPI
  • Building FastAPI Microservices for Machine Learning Predictions
  • Deploying a Python Lambda Microservice

2 Readings

  • Key Terms
  • Lesson Reflection

Building Containerized Microservices

    • 4 Videos
    • 3 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Quiz-Building Containerized Microservices

4 Videos

  • Introduction to Building Containerized Microservices
  • Why use Containers for Microservices?
  • Deploying a Containerized .NET 6 API
  • Deploying a Containerized Machine Learning Microservice

3 Readings

  • Key Terms
  • Containers and Container Services
  • Lesson Reflection

Putting it All Together: Python Microservices

    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • What are the key components of Python Microservices?

1 Labs

  • Building a Python Microservice

Building Command-Line Tools

    • 7 Videos
    • 2 Readings
    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Practice Quiz

1 Labs

  • Install an editable Python CLI tool

7 Videos

  • Introduction to Python Packaging and Command-Line Tools
  • Introduction to Building Command-Line Tools
  • Getting Started with Python Projects
  • Overview of Command-Line Tool Frameworks
  • Using Click to Build a Command-Line Tool
  • Exploring Advanced Command-Line Tool Features
  • Recap of Building Command-Line Tools

2 Readings

  • Key Terms
  • Building Command-Line Tools

Packaging your Python Project

    • 5 Videos
    • 1 Readings
    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Packaging

1 Labs

  • Install a Python CLI tool

5 Videos

  • Introduction to Packaging and Distributing your Python Project
  • Introduction to Python Packaging
  • Working with Python Setup Tools
  • Uploading to a Python Registry
  • Recap of Packaging and Distributing your Python Project

1 Readings

  • Key Terms

Reviewing Continuous Integration for Command-Line Tools

    • 5 Videos
    • 1 Readings
    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Continuous Integration

1 Labs

  • Test and validate a Python CLI tool

5 Videos

  • Introduction to Continuous Integration for Command-Line Tools
  • Introduction to Linting
  • Automating Testing with GitHub Actions
  • Automating Publishing of your Python Project
  • Recap of Continuous Integration for Command-Line Tools

1 Readings

  • Key Terms

Building a Rust Command Line Tool

    • 7 Videos
    • 4 Readings
Show info about module content

7 Videos

  • Introduction
  • Setting up your development environment for Command-line development
  • Your first Command-line tool in Rust
  • Working with user input: arguments and options
  • Expanding your tool's functionality with modules and libraries
  • Managing output: logging, errors, and panics
  • Optimizing your Command-line tools: Performance and best practices

4 Readings

  • Key terms
  • External lab: Setup your development environment
  • Introduction to Rust command line tools
  • External lab: Build your first Rust CLI

Putting it All Together: Python Packaging and Command Line Tools

    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Command-Line Tools and Packaging

1 Labs

  • Updating a Command-Line Tool

Final Challenge-Big O Notation

    • 1 Videos
    • 3 Readings
    • 1 Lab
    • 1 Assignment
Show info about module content

1 Assignment

  • Quiz-Big O Notation

1 Labs

  • Big O Notation Final Challenge

1 Videos

  • Big O Notation-Final Challenge Walkthrough

3 Readings

  • Key Terms
  • Lesson Reflection
  • Next Steps

Auto Summary

Enhance your data engineering skills with Coursera's "Web Applications and Command-Line Tools for Data Engineering." This professional-level course delves into Python, Bash, and SQL techniques for real-world applications. You'll learn to create machine learning models using Jupyter notebooks, develop scalable Python microservices, and build command-line tools for automation and quality control. Available through Starter and Professional subscriptions, this 900-minute course is perfect for IT and computer science professionals aiming to advance their expertise.

Noah Gift
Instructors

Noah Gift

Kennedy Behrman
Instructors

Kennedy Behrman

Alfredo Deza
Instructors

Alfredo Deza