Monthly Subscription Starting at AED 99 + VAT

Custom and Distributed Training with TensorFlow
  • Level Professional
  • Duration 25 hours
  • Course by DeepLearning.AI
  • Offered by

About

In this course, you will: • Learn about Tensor objects, the fundamental building blocks of TensorFlow, understand the difference between the eager and graph modes in TensorFlow, and learn how to use a TensorFlow tool to calculate gradients. • Build your own custom training loops using GradientTape and TensorFlow Datasets to gain more flexibility and visibility with your model training. • Learn about the benefits of generating code that runs in graph mode, take a peek at what graph code looks like, and practice generating this more efficient code automatically with TensorFlow’s tools. • Harness the power of distributed training to process more data and train larger models, faster, get an overview of various distributed training strategies, and practice working with a strategy that trains on multiple GPU cores, and another that trains on multiple TPU cores. The DeepLearning.AI TensorFlow: Advanced Techniques Specialization introduces the features of TensorFlow that provide learners with more control over their model architecture and tools that help them create and train advanced ML models. This Specialization is for early and mid-career software and machine learning engineers with a foundational understanding of TensorFlow who are looking to expand their knowledge and skill set by learning advanced TensorFlow features to build powerful models.

Modules

A conversation with Andrew Ng

    • 1 Videos
Show info about module content

1 Videos

  • A conversation with Andrew Ng: Overview of course 2

Tensor Basics

    • 4 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Basic Tensors

4 Videos

  • What is a tensor?
  • Creating tensors in code
  • Math operations with tensors
  • Basic Tensors code walkthrough

1 Readings

  • [IMPORTANT] Have questions, issues or ideas? Join our Forum!

Working with Tensors in Eager Mode

    • 2 Videos
Show info about module content

2 Videos

  • Broadcasting, operator overloading and Numpy compatibility
  • Evaluating variables and changing data types

Gradient Tape

    • 5 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Gradient Tape Basics

5 Videos

  • Gradient Tape
  • Gradient Descent using Gradient Tape
  • Calculate gradients on higher order functions
  • Persistent=true and higher order gradients
  • Gradient Tape basics code walkthrough

1 Readings

  • Reference: CNN for visual recognition

Lecture Notes (Optional)

    • 1 Readings
Show info about module content

1 Readings

  • Lecture Notes Week 1

Quiz

    • 1 Assignment
Show info about module content

1 Assignment

  • Tensors and Gradient Tape

Assignment: Basic Tensor Operations

    • 1 Programming
Show info about module content

1 Programming

  • Basic Tensor Operations

Custom Training Loops

    • 4 Videos
    • 1 Lab
Show info about module content

1 Labs

  • Training Basics

4 Videos

  • Custom Training Loop steps
  • Loss and gradient descent
  • Define Training Loop and Validate Model
  • Training Basics code walkthrough

Custom Training with TensorFlow Datasets

    • 4 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • Fashion MNIST using Custom Training Loop

4 Videos

  • Training steps and data pipeline
  • Define the training loop
  • Gradients, metrics, and validation
  • Fashion MNIST Custom Training Loop code walkthrough

1 Readings

  • Reference: tf.keras.metrics

Lecture Notes (Optional)

    • 1 Readings
Show info about module content

1 Readings

  • Lecture Notes Week 2

Week 2 Quiz: Custom training

    • 1 Assignment
Show info about module content

1 Assignment

  • Custom Training

Assignment: Breast Cancer Prediction

    • 1 Programming
Show info about module content

1 Programming

  • Breast Cancer Prediction

AutoGraph

    • 3 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • AutoGraph Basics

3 Videos

  • Benefits of graph mode
  • Generating graph code
  • AutoGraph Basics code walkthrough

1 Readings

  • Reference: Fizz Buzz

Creating Graphs for Complex Code

    • 3 Videos
    • 1 Lab
Show info about module content

1 Labs

  • AutoGraph

3 Videos

  • Control dependencies and flows
  • Loops and tracing variables
  • AutoGraph code walkthrough

Lecture Notes (Optional)

    • 1 Readings
Show info about module content

1 Readings

  • Lecture Notes Week 3

Week 3 Quiz: Autograph

    • 1 Assignment
Show info about module content

1 Assignment

  • AutoGraph

Assignment: AutoGraph

    • 1 Programming
Show info about module content

1 Programming

  • Horse or Human?

Overview of Distribution Strategies

    • 2 Videos
Show info about module content

2 Videos

  • Intro to distribution strategies
  • Types of distribution strategies

Mirrored Strategy

    • 2 Videos
    • 1 Lab
Show info about module content

1 Labs

  • Mirrored Strategy

2 Videos

  • Converting code to the Mirrored Strategy
  • Mirrored Strategy code walkthrough

Multiple GPU Mirrored Strategy

    • 2 Videos
    • 1 Lab
Show info about module content

1 Labs

  • Multi GPU Mirrored Strategy

2 Videos

  • Custom Training for Multiple GPU Mirrored Strategy
  • Multi GPU Mirrored Strategy code walkthrough

TPU Strategy

    • 2 Videos
    • 1 Lab
Show info about module content

1 Labs

  • TPU Strategy

2 Videos

  • TPU Strategy
  • TPU Strategy code walkthrough

Other Distributed Strategies

    • 1 Videos
    • 1 Readings
    • 1 Lab
Show info about module content

1 Labs

  • One Device Strategy

1 Videos

  • Other Distributed Strategies

1 Readings

  • References used in Other Distributed Strategies

Lecture Notes (Optional)

    • 1 Readings
Show info about module content

1 Readings

  • Lecture Notes Week 4

Quiz

    • 1 Assignment
Show info about module content

1 Assignment

  • Distributed Strategy

End of Access to Lab Notebooks

    • 1 Readings
Show info about module content

1 Readings

  • [IMPORTANT] Reminder about end of access to Lab Notebooks

Assignment: Distributed Strategy

    • 2 Programming
Show info about module content

2 Programming

  • Distributed Strategy
  • Upload your model (optional)

Course Resources

    • 1 Readings
Show info about module content

1 Readings

  • References

Acknowledgments

    • 1 Readings
Show info about module content

1 Readings

  • Acknowledgments

Auto Summary

"Custom and Distributed Training with TensorFlow" is a professional-level course designed for software and machine learning engineers eager to deepen their TensorFlow expertise. Led by DeepLearning.AI on Coursera, it covers Tensor objects, custom training loops, graph mode, and distributed training across GPUs and TPUs. With a focus on advanced techniques, this 1500-minute course caters to early and mid-career professionals aiming to build powerful ML models. Subscription options include Starter and Professional.

Laurence Moroney
Instructor

Laurence Moroney