Skip to main content Skip to navigation

ES4E7-15 Information Theory and Coding

Department
School of Engineering
Level
Taught Postgraduate Level
Module leader
Mark Leeson
Credit value
15
Module duration
10 weeks
Assessment
50% coursework, 50% exam
Study location
University of Warwick main campus, Coventry
Introductory description

The subject of Information Theory underpins all of modern communications and hence the connected world in which we live. This module provides insight into this important topic plus the compression and error-control coding schemes used in communication systems. It provides the means to quantify information; introduces entropy; presents a quantitative approach to the capacity of communication channels; investigates the methods and limits of source coding and reliable communications.

Module web page

Module aims

To provide the understanding and analytical tools necessary to apply information theory to a range of relevant modern problems in communication engineering. In particular: to convey the source coding and noisy channel theorems; to furnish students with the means to compute information theoretic quantities; to deliver the principles and applications of source codes; to convey the principles and applications of channel codes; to provide exposure to the latest developments in the area.
Students will gain: Substantial knowledge of information and entropy, and their use in information theory. Principles and practice of data compression. Channel capacity of common communication channels. Design and performance evaluation of error correcting codes. Knowledge of lossy compression. Exposure to emerging topics in information theory, coding and compression.

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

Introduction to information theory; information and entropy; data compression; source coding theorem; symbol codes, stream codes; noisy channels; noisy channel coding theorem; channel capacity; example channels – e.g. Gaussian, binary symmetric and binary erasure; error correcting codes – block codes (e.g. Hamming and Reed Solomon) – convolutional codes; lossy compression –principles and practice (e.g. JPEG, MPEG, H.264); selection of emerging topics such as Stochastic Resonance, fountain codes, molecular communications, physical layer security.

Learning outcomes

By the end of the module, students should be able to:

  • Demonstrate advanced knowledge and understanding of information and entropy, and their use in information theory.
  • Comprehend the principles of data compression, and use these to design and evaluate specific data compression techniques.
  • Design and evaluate the error performance of specific error correcting codes.
  • Apply the principles of noisy channel analysis to calculate the capacity of common communication channels.
  • Demonstrate sound and systematic comprehension of the principles and implementations of lossy compression.
Indicative reading list

S M Moser and P-N Chen, A student's guide to coding and information theory, CUP, 2012. ISBN 9781107015838 [Q 360.M67 and online access]
T M Cover and J A Thomas, Elements of Information Theory (2nd edition), Wiley, 2006. ISBN 9780471241959 [Q360.C6 and online access]
M. Kelbert and Y. Suhov, Information theory and coding by example, CUP, 2013. ISBN 9780521139885 [Q360.K45 and online access]
M Borda, Fundamentals in Information Theory and Coding, Springer, 2011. e-ISBN 9783642203473 [online access]
D Salomon and G Motta, Handbook of Data Compression (5th edition), Springer, 2010. eBook ISBN
9781848829039 [online access]

Subject specific skills

Comprehensive understanding of the principles of data transmission, compression and error correction.
Critical awareness of current developments in information theory and coding.
Ability to apply appropriate engineering analysis methods for solving problems of information theory and coding, and to assess the limitations of the methods employed.
Thorough understanding of modern data transmission practice, its limitations, and appreciation of likely new developments.

Transferable skills

Ability to express advanced technical concepts concisely and accurately and comment on their applications, limitations and implications.
Ability to select, adapt and apply a range of mathematical techniques to solve advanced problems and explain the implications of the answer.

Study time

Type Required
Lectures 20 sessions of 1 hour (13%)
Tutorials 5 sessions of 1 hour (3%)
Other activity 2 hours (1%)
Private study 123 hours (82%)
Total 150 hours
Private study description

Self-study, problem sheets, background reading and revision - total of 123 hours.

Other activity description

2x1 hour Revision Session

Costs

No further costs have been identified for this module.

You must pass all assessment components to pass the module.

Assessment group C
Weighting Study time
Assignment 50%
Online Examination 50%

QMP

~Platforms - AEP,QMP


  • Online examination: No Answerbook required
  • Students may use a calculator
  • Engineering Data Book 8th Edition
  • Graph paper
Feedback on assessment

Model solutions are published for past examination papers.
Cohort level feedback on examinations is provided.

Past exam papers for ES4E7

Courses

This module is Core for:

  • Year 1 of TESA-H641 Postgraduate Taught Communications and Information Engineering

This module is Optional for:

  • Year 1 of TCSA-G5PA Postgraduate Taught Data Analytics

This module is Option list A for:

  • Year 4 of UESA-H63X MEng Electronic Engineering
  • Year 5 of UESA-H636 MEng Electronic Engineering with Intercalated Year
  • Year 4 of UESA-H114 MEng Engineering
  • Year 4 of UESA-H606 Undergraduate Electrical and Electronic Engineering MEng