Monthly Subscription Starting at AED 99 + VAT

Scalable Machine Learning on Big Data using Apache Spark
  • Level Professional
  • المدة 7 ساعات hours
  • الطبع بواسطة IBM
  • Offered by

عن

This course will empower you with the skills to scale data science and machine learning (ML) tasks on Big Data sets using Apache Spark. Most real world machine learning work involves very large data sets that go beyond the CPU, memory and storage limitations of a single computer. Apache Spark is an open source framework that leverages cluster computing and distributed storage to process extremely large data sets in an efficient and cost effective manner. Therefore an applied knowledge of working with Apache Spark is a great asset and potential differentiator for a Machine Learning engineer. After completing this course, you will be able to: - gain a practical understanding of Apache Spark, and apply it to solve machine learning problems involving both small and big data - understand how parallel code is written, capable of running on thousands of CPUs. - make use of large scale compute clusters to apply machine learning algorithms on Petabytes of data using Apache SparkML Pipelines. - eliminate out-of-memory errors generated by traditional machine learning frameworks when data doesn’t fit in a computer's main memory - test thousands of different ML models in parallel to find the best performing one – a technique used by many successful Kagglers - (Optional) run SQL statements on very large data sets using Apache SparkSQL and the Apache Spark DataFrame API. Enrol now to learn the machine learning techniques for working with Big Data that have been successfully applied by companies like Alibaba, Apple, Amazon, Baidu, eBay, IBM, NASA, Samsung, SAP, TripAdvisor, Yahoo!, Zalando and many others. NOTE: You will practice running machine learning tasks hands-on on an Apache Spark cluster provided by IBM at no charge during the course which you can continue to use afterwards. Prerequisites: - basic python programming - basic machine learning (optional introduction videos are provided in this course as well) - basic SQL skills for optional content The following courses are recommended before taking this class (unless you already have the skills) https://www.coursera.org/learn/python-for-applied-data-science or similar https://www.coursera.org/learn/machine-learning-with-python or similar https://www.coursera.org/learn/sql-data-science for optional lectures

الوحدات

Course Introduction

    • 2 Videos
    • 2 Readings
Show info about module content

2 Videos

  • Introduction to Apache Spark for Machine Learning on BigData
  • What is Big Data?

2 Readings

  • Course Syllabus
  • Setup of the grading and exercise environment

Understanding how Apache Spark works

    • 4 Videos
    • 4 Readings
    • 2 Assignment
Show info about module content

2 Assignment

  • Practice Quiz (Ungraded) - Apache Spark concepts
  • Apache Spark and parallel data processing

4 Videos

  • Data storage solutions
  • Parallel data processing strategies of Apache Spark
  • Functional programming basics
  • Resilient Distributed Dataset and DataFrames - ApacheSparkSQL

4 Readings

  • Exercise 1 - working with RDD
  • Exercise 2 - functional programming basics with RDDs
  • Exercise 3 - working with DataFrames
  • Programming Lanuage Options for Apache Spark (optional)

Experience parallel programming on Apache Spark

    • 5 Videos
    • 1 Readings
    • 2 Assignment
Show info about module content

2 Assignment

  • Practice Quiz (Ungraded) - Statistics and API usage on Spark
  • Parallelism in Apache Spark 

5 Videos

  • Averages
  • Standard deviation
  • Skewness
  • Kurtosis
  • Covariance, Covariance matrices, correlation

1 Readings

  • Exercise 1 - statistics and transfomrations using DataFrames

Data Visualization of Big Data

    • 3 Videos
    • 2 Readings
    • 2 Assignment
Show info about module content

2 Assignment

  • Questions on Plotting
  • Questions on PCA

3 Videos

  • Plotting with ApacheSpark and python's matplotlib
  • Dimensionality reduction
  • PCA

2 Readings

  • Exercise on Plotting
  • Exercise on PCA

Introduction to Apache SparkML

    • 3 Videos
    • 1 Readings
    • 2 Assignment
Show info about module content

2 Assignment

  • Practice Quiz (Ungraded) - ML Pipelines
  • SparkML concepts 

3 Videos

  • How ML Pipelines work
  • Introduction to SparkML
  • Extract - Transform - Load

1 Readings

  • Exercise 1: Modifying a Apache SparkML Feature Engineering Pipeline

Unsupervised Learning with Apache SparkML

    • 2 Videos
    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Practice Quiz (Ungraded) - SparkML Algorithms

2 Videos

  • Introduction to Clustering: k-Means
  • Using K-Means in Apache SparkML

1 Readings

  • Exercise 2 - Working with Clustering and Apache SparkML

Supervised Learning with Apache SparkML

    • 4 Videos
    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Practice Quiz (Ungraded) - SparkML Algorithms (2)

4 Videos

  • Linear Regression
  • LinearRegression with Apache SparkML
  • Logistic Regression
  • LogisticRegression with Apache SparkML

1 Readings

  • Exercise 1 - Improving Classification performance

Course Project

    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Course Project Quiz

1 Readings

  • Course Project

Auto Summary

Unlock the power of scalable machine learning on Big Data with this comprehensive course on Apache Spark. Designed for data science and AI enthusiasts, this course equips you with practical skills to handle large datasets efficiently using cluster computing. Led by expert instructors, you'll delve into Apache Spark, learn to write parallel code, and apply ML algorithms on massive data. The course spans 420 minutes and offers hands-on practice on an IBM-provided cluster. Ideal for professionals with basic Python, ML, and SQL skills, you can subscribe via Starter or Professional plans. Join now to enhance your data science capabilities!

Romeo Kienzler
Instructor

Romeo Kienzler