ES4E7-15 Information Theory and Coding
Introductory description
The subject of Information Theory underpins all of modern communications and hence the connected world in which we live. This module provides insight into this important topic plus the compression and error-control coding schemes used in communication systems. It provides the means to quantify information; introduces entropy; presents a quantitative approach to the capacity of communication channels; investigates the methods and limits of source coding and reliable communications.
Module aims
To provide the understanding and analytical tools necessary to apply information theory to a range of relevant modern problems in communication engineering. In particular: to convey the source coding and noisy channel theorems; to furnish students with the means to compute information theoretic quantities; to deliver the principles and applications of source codes; to convey the principles and applications of channel codes; to provide exposure to the latest developments in the area.
Students will gain: Substantial knowledge of information and entropy, and their use in information theory. Principles and practice of data compression. Channel capacity of common communication channels. Design and performance evaluation of error correcting codes. Knowledge of lossy compression. Exposure to emerging topics in information theory, coding and compression.
Outline syllabus
This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.
Introduction to information theory; information and entropy; data compression; source coding theorem; symbol codes, stream codes; noisy channels; noisy channel coding theorem; channel capacity; example channels – e.g. Gaussian, binary symmetric and binary erasure; error correcting codes – block codes (e.g. Hamming and Reed Solomon) – convolutional codes; lossy compression –principles and practice (e.g. JPEG, MPEG, H.264); selection of emerging topics such as Stochastic Resonance, fountain codes, molecular communications, physical layer security.
Learning outcomes
By the end of the module, students should be able to:
- Demonstrate advanced knowledge and understanding of information and entropy, and their use in information theory.
- Comprehend the principles of data compression, and use these to design and evaluate specific data compression techniques.
- Design and evaluate the error performance of specific error correcting codes.
- Apply the principles of noisy channel analysis to calculate the capacity of common communication channels.
- Demonstrate sound and systematic comprehension of the principles and implementations of lossy compression.
Indicative reading list
S M Moser and P-N Chen, A student's guide to coding and information theory, CUP, 2012. ISBN 9781107015838 [Q 360.M67 and online access]
T M Cover and J A Thomas, Elements of Information Theory (2nd edition), Wiley, 2006. ISBN 9780471241959 [Q360.C6 and online access]
M. Kelbert and Y. Suhov, Information theory and coding by example, CUP, 2013. ISBN 9780521139885 [Q360.K45 and online access]
M Borda, Fundamentals in Information Theory and Coding, Springer, 2011. e-ISBN 9783642203473 [online access]
D Salomon and G Motta, Handbook of Data Compression (5th edition), Springer, 2010. eBook ISBN
9781848829039 [online access]
Subject specific skills
Comprehensive understanding of the principles of data transmission, compression and error correction.
Critical awareness of current developments in information theory and coding.
Ability to apply appropriate engineering analysis methods for solving problems of information theory and coding, and to assess the limitations of the methods employed.
Thorough understanding of modern data transmission practice, its limitations, and appreciation of likely new developments.
Transferable skills
Ability to express advanced technical concepts concisely and accurately and comment on their applications, limitations and implications.
Ability to select, adapt and apply a range of mathematical techniques to solve advanced problems and explain the implications of the answer.
Study time
| Type | Required |
|---|---|
| Lectures | 20 sessions of 1 hour (13%) |
| Tutorials | 5 sessions of 1 hour (3%) |
| Other activity | 2 hours (1%) |
| Private study | 123 hours (82%) |
| Total | 150 hours |
Private study description
Self-study, problem sheets, background reading and revision - total of 123 hours.
Other activity description
2x1 hour Revision Session
Costs
No further costs have been identified for this module.
You must pass all assessment components to pass the module.
Assessment group B1
| Weighting | Study time | Eligible for self-certification | |
|---|---|---|---|
| In-person Examination | 100% | No | |
|
Unseen Examination
|
|||
Feedback on assessment
Model solutions are published for past examination papers.
Cohort level feedback on examinations is provided.
Courses
This module is Core for:
- Year 1 of TESA-H641 Postgraduate Taught Communications and Information Engineering
This module is Optional for:
- Year 4 of UESA-H116 MEng Engineering with Exchange Year
- Year 5 of UESA-H115 MEng Engineering with Intercalated Year
- Year 1 of TCSA-G5PA Postgraduate Taught Data Analytics
- Year 5 of UESA-H607 Undergraduate Electrical and Electronic Engineering with Intercalated Year
This module is Option list A for:
- Year 4 of UESA-H63X MEng Electronic Engineering
- Year 5 of UESA-H636 MEng Electronic Engineering with Intercalated Year
- Year 5 of UESA-H63Y MEng Electronic Engineering with Intercalated Year
- Year 4 of UESA-H114 MEng Engineering
- Year 1 of TESA-H644 Postgraduate Taught Electrical and Electronic Engineering
- Year 4 of UCSA-G408 Undergraduate Computer Systems Engineering
- Year 5 of UCSA-G409 Undergraduate Computer Systems Engineering (with Intercalated Year)
- Year 4 of UESA-H606 Undergraduate Electrical and Electronic Engineering MEng