Monthly Subscription Starting at AED 99 + VAT

Recommender Systems: Evaluation and Metrics
  • Level Foundation
  • Duration 7 hours
  • Course by University of Minnesota
  • Offered by

About

In this course you will learn how to evaluate recommender systems. You will gain familiarity with several families of metrics, including ones to measure prediction accuracy, rank accuracy, decision-support, and other factors such as diversity, product coverage, and serendipity. You will learn how different metrics relate to different user goals and business goals. You will also learn how to rigorously conduct offline evaluations (i.e., how to prepare and sample data, and how to aggregate results). And you will learn about online (experimental) evaluation. At the completion of this course you will have the tools you need to compare different recommender system alternatives for a wide variety of uses.

Modules

Preface

    • 2 Videos
Show info about module content

2 Videos

  • Introduction to Evaluation and Metrics
  • The Goals of Evaluation

Basic Prediction and Recommendation Metrics

    • 5 Videos
    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Basic Prediction and Recommendation Metrics Assignment

5 Videos

  • Hidden Data Evaluation
  • Prediction Accuracy Metrics
  • Decision Support Metrics
  • Rank-Aware Top-N Metrics
  • Assignment Intro Video

1 Readings

  • Metric Computation Assignment Instructions

Advanced Metrics and Offline Evaluation

    • 5 Videos
    • 1 Assignment
Show info about module content

1 Assignment

  • Offline Evaluation and Metrics Quiz

5 Videos

  • Beyond Basic Evaluation
  • Additional Item and List-Based Metrics
  • Experimental Protocols
  • Unary Data Evaluation
  • Temporal Evaluation of Recommenders (Interview with Neal Lathia)

Honors

    • 1 Videos
    • 1 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Programming Assignment Quiz

1 Videos

  • Programming Assignment Introduction

1 Readings

  • Evaluating Recommenders

Online Evaluation

    • 4 Videos
    • 1 Assignment
Show info about module content

1 Assignment

  • Online Evaluation Quiz

4 Videos

  • Introduction to Online Evaluation and User Studies
  • Usage Logs and Analysis
  • A/B Studies (Field Experiments)
  • User-Centered Evaluation (Interview with Bart Knijnenburg)

Evaluation Design

    • 3 Videos
    • 2 Readings
    • 1 Assignment
Show info about module content

1 Assignment

  • Assignment: Evaluation Design Cases

3 Videos

  • Matching Evaluation to the Problem/Challenge
  • Case Examples
  • Assignment Intro Video

2 Readings

  • Intro to Assignment: Evaluation Design Cases
  • Quiz Debrief

Auto Summary

Enhance your Data Science & AI skills with "Recommender Systems: Evaluation and Metrics" on Coursera. This foundational course, led by expert instructors, explores various metrics for evaluating recommender systems, including prediction accuracy, rank accuracy, and diversity. You'll learn to align metrics with user and business goals and conduct both offline and online evaluations. Ideal for beginners, the course spans 420 minutes and offers Starter and Professional subscription options.

Michael D. Ekstrand
Instructors

Michael D. Ekstrand

Joseph A Konstan
Instructors

Joseph A Konstan