Monthly Subscription Starting at AED 99 + VAT

Information Theory
  • Level Foundation
  • المدة 33 ساعات hours
  • الطبع بواسطة The Chinese University of Hong Kong
  • Offered by

عن

The lectures of this course are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008). This book and its predecessor, A First Course in Information Theory (Kluwer 2002, essentially the first edition of the 2008 book), have been adopted by over 60 universities around the world as either a textbook or reference text. At the completion of this course, the student should be able to: 1) Demonstrate knowledge and understanding of the fundamentals of information theory. 2) Appreciate the notion of fundamental limits in communication systems and more generally all systems. 3) Develop deeper understanding of communication systems. 4) Apply the concepts of information theory to various disciplines in information science.

الوحدات

Welcome

    • 1 Videos
    • 1 Readings
Show info about module content

1 Videos

  • About this course

1 Readings

  • Grading Scheme

Week 1 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 1 Introduction

Chapter 1 The Science of Information

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 1

2.1 Independence and Markov Chain

    • 3 Videos
Show info about module content

3 Videos

  • Chapter 2 - Section 2.1 A
  • Chapter 2 - Section 2.1 B
  • Chapter 2 - Section 2.1 C

2.2 Shannon’s Information Measures

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.2

2.3 Continuity of Shannon’s Information Measures for Fixed Finite Alphabets

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.3

Week 1 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 1 Submission

1 Videos

  • Assignment 1 Video Preview

1 Readings

  • Homework Assignment 1 Solution

Week 2 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 2 Introduction

2.4 Chain Rules

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.4

2.5 Informational Divergence

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.5

2.6 The Basic Inequalities

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.6

2.7 Some Useful Information Inequalities

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.7

2.8 Fano's Inequality

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.8

2.10 Entropy Rate of a Stationary Source

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 2 - Section 2.10

Week 2 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 2 Submission

1 Videos

  • Assignment 2 Video Preview

1 Readings

  • Homework Assignment 2 Solution

Week 3 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 3 Introduction

3.1 Preliminaries & 3.2 The I-Measure for Two Random Variables

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 3 - Section 3.1

3.3 Construction of the I-Measure μ*

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 3 - Section 3.3 A
  • Chapter 3 - Section 3.3 B

3.4 μ* can be Negative

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 3 - Section 3.4

3.5 Information Diagrams

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 3 - Section 3.5 A

Week 3 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 3 Submission

1 Videos

  • Assignment 3 Video Preview

1 Readings

  • Homework Assignment 3 Solution

Week 4 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 4 Introduction

3.5 Information Diagrams

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 3 - Section 3.5 B

3.6 Examples of Applications

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 3 - Section 3.6 A
  • Chapter 3 - Section 3.6 B

4.1 The Entropy Bound

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 4 - Section 4.1

4.2 Prefix Codes

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 4 - Section 4.2 A

Week 4 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 4 Submission

1 Videos

  • Assignment 4 Video Preview

1 Readings

  • Homework Assignment 4 Solution

Week 5 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 5 Introduction

4.2 Prefix Codes

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 4 - Section 4.2 B
  • Chapter 4 - Section 4.2 C

4.3 Redundancy of Prefix Codes

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 4 - Section 4.3

5.1 The Weak AEP

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 5 - Section 5.1

5.2 The Source Coding Theorem

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 5 -Section 5.2

Week 5 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 5 Submission

1 Videos

  • Assignment 5 Video Preview

1 Readings

  • Homework Assignment 5 Solution

Week 6 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 6 - Introduction

6.1 Strong AEP

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 6 - Section 6.1 A
  • Chapter 6 - Section 6.1 B

6.2 Strong Typicality vs Weak Typicality

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 6 - Section 6.2

6.3 Joint Typicality

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 6 - Section 6.3 A
  • Chapter 6 - Section 6.3 B

6.4 An Interpretation of the Basic Inequalities

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 6 - Section 6.4

Week 6 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 6 Submission

1 Videos

  • Assignment 6 Video Preview

1 Readings

  • Homework Assignment 6 Solution

Week 7 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 7 Introduction

7.1 Definition and Capacity

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 7 - Section 7.1 A
  • Chapter 7 - Section 7.1 B

7.2 The Channel Coding Theorem

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 7 - Section 7.2

7.3 The Converse

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 7 - Section 7.3 A
  • Chapter 7 - Section 7.3 B

Week 7 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 7 Submission

1 Videos

  • Assignment 7 Video Preview

1 Readings

  • Homework Assignment 7 Solution

Week 8 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 8 Introduction

7.4 Achievability

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 7 - Section 7.4 A
  • Chapter 7 - Section 7.4 B

7.5 A Discussion

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 7 - Section 7.5

7.6 Feedback Capacity

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 7 - Section 7.6

7.7 Separation of Source and Channel Coding

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 7 - Section 7.7

Week 8 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 8 Submission

1 Videos

  • Assignment 8 Video Preview

1 Readings

  • Homework Assignment 8 Solution

Week 9 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 9 Introduction

8.1 Single-Letter Distortion Measure

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 8 - Section 8.1

8.2 The Rate-Distortion Function

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 8 - Section 8.2

8.3 The Rate Distortion Theorem

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 8 - Section 8.3 A
  • Chapter 8 - Section 8.3 B

8.4 The Converse

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 8 - Section 8.4

Week 9 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 9 Submission

1 Videos

  • Assignment 9 Video Preview

1 Readings

  • Homework Assignment 9 Solution

Week 10 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 10 Introduction

8.5 Achievability of RI(D)

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 8 - Section 8.5 A
  • Chapter 8 - Section 8.5 B

9.1 Alternating Optimization

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 9 - Section 9.1

9.2 The Algorithms

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 9 - Section 9.2 A
  • Chapter 9 - Section 9.2 B

Week 10 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 10 Submission

1 Videos

  • Assignment 10 Video Preview

1 Readings

  • Homework Assignment 10 Solution

Week 11 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 11 Introduction

9.2 The Algorithms

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 9 - Section 9.2 C

9.3 Convergence

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 9 - Section 9.3

10.1 Preliminaries

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 10 - Section 10.1 A
  • Chapter 10 - Section 10.1 B

Week 11 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 11 Submission

1 Videos

  • Assignment 11 Video Preview

1 Readings

  • Homework Assignment 11 Solution

Week 12 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 12 Introduction

10.2 Definition

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 10 - Section 10.2

10.3 Joint Differential Entropy, Conditional (Differential) Entropy, and Mutual Information

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 10 - Section 10.3 A
  • Chapter 10 - Section 10.3 B

10.4 AEP for Continuous Random Variables

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 10 - Section 10.4

10.5 Informational Divergence

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 10 - Section 10.5

10.6 Maximum Differential Entropy Distributions

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 10 - Section 10.6

Week 12 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 12 Submission

1 Videos

  • Assignment 12 Video Preview

1 Readings

  • Homework Assignment 12 Solution

Week 13 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 13 Introduction

11.1 Discrete-Time Channels

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 11 - Section 11.1

11.2 The Channel Coding Theorem

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 11 - Section 11.2

11.3 Proof of the Channel Coding Theorem

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 11 - Section 11.3 A
  • Chapter 11 - Section 11.3 B

Week 13 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 13 Submission

1 Videos

  • Assignment 13 Video Preview

1 Readings

  • Homework Assignment 13 Solution

Week 14 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 14 Introduction

11.4 Memoryless Gaussian Channels

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 11 - Section 11.4

11.5 Parallel Gaussian Channels

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 11 - Section 11.5

11.6 Correlated Gaussian Channels

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 11 - Section 11.6

Week 14 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 14 Submission

1 Videos

  • Assignment 14 Video Preview

1 Readings

  • Homework Assignment 14 Solution

Week 15 Introduction

    • 1 Videos
Show info about module content

1 Videos

  • Week 15 Introduction

11.7 The Bandlimited White Gaussian Channel

    • 2 Videos
Show info about module content

2 Videos

  • Chapter 11 - Section 11.7 A
  • Chapter 11 - Section 11.7 B

11.8 The Bandlimited Colored Gaussian Channel

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 11 - Section 11.8

11.9 Zero-Mean Gaussian Noise Is the Worst Additive Noise

    • 1 Videos
Show info about module content

1 Videos

  • Chapter 11 - Section 11.9

Week 15 Homework Assignment

    • 1 Videos
    • 1 Readings
    • 1 PeerReview
Show info about module content

1 Peer Review

  • Homework Assignment 15 Submission

1 Videos

  • Assignment 15 Video Preview

1 Readings

  • Homework Assignment 15 Solution

Auto Summary

Embark on an enlightening journey into the world of information theory with this foundational course, meticulously designed for enthusiasts in the fields of Mathematics and Statistics. Guided by the esteemed Prof. Raymond Yeung, this course is anchored in the first 11 chapters of his acclaimed textbook "Information Theory and Network Coding" (Springer 2008), a pivotal resource adopted by over 60 universities globally. Throughout this comprehensive course, learners will gain a robust understanding of the core principles of information theory. Participants will explore the concept of fundamental limits in communication systems, enhancing their insight into the constraints and capabilities of various systems. Furthermore, the course will deepen your knowledge of communication systems and their applications across multiple disciplines within information science. This foundational course, available through Coursera, spans an extensive duration, providing ample time to thoroughly grasp the subject matter. Learners can choose between Starter and Professional subscription plans to suit their educational needs. Ideal for those keen on building a strong base in information theory, this course promises to equip you with the skills and knowledge to apply these concepts across a range of information science domains. Join now and elevate your understanding of this crucial field.

Prof. Raymond W. Yeung
Instructor

Prof. Raymond W. Yeung