Introductory description

Modern programming approaches use high level constructs which abstract away the system's architecture. These high levels of abstraction use code generation programs such as compilers and assemblers to take the human author's input, and produce code that will execute as output. The modern programmer rarely needs to consider the underlying architecture of the machine that will execute the code.
There are situations where, rather than creating an executable from source, you need to go in the opposite direction; you need to infer what the source code might look like by analysing the executable. Perhaps you have some potential malware; or perhaps you have to analyse and exploit a vulnerability which an executable might have. Either way, you want to know what the program will do, were it to run on your system.
In order to reverse engineer an executable, you need to understand the typical idioms that an operating system, architecture and code generation programs will adopt to convert high level constructs into low level executables.
In addition, if the executable is malware, then it is likely the authors will have used some obfuscation in order to make the analysis more difficult. Under these circumstances you need to understand the typical idioms of obfuscation.

Module aims

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

The content of this module will be taught from a cyber security perspective.

• executable code from a variety of perspectives
• assembly language programming
• machine-level instruction set and organisation
• code generation
• reverse engineering techniques
• de-obfuscation
• common tools for reverse engineering
• anti-debugging mechanisms
• fuzzing

Learning outcomes

By the end of the module, students should be able to:

• Demonstrate a critical understanding of common idioms and patterns used during code transformation and explain the origin and organisation of arbitrary code and/or data fragments within an executable program [CITP 2.1.8, 3.1.2]
• Apply tools and techniques as appropriate to infer the executable's overall high-level function, potentially obfuscated, potentially malicious code [CITP 2.1.3, 2.1.4, 2.1.5, 2.1.8, 2.2.2, 3.1.1, 3.1.2, 3.2.2]
• Perform malicious code analysis, vulnerability identification and evaluation independently from the findings generated by automated analysis tools [CITP 2.1.3, 2.1.4, 2.1.5,2.1.8, 2.2.2, 3.1.1, 3.1.2, 3.2.2]
• Identify system vulnerabilities, and consequently develop and deploy custom exploits [CITP 2.1.3, 2.1.4, 2.1.5,2.1.8,2.2.2, 3.1.1, 3.1.2, 3.2.2]

Indicative reading list

Reading lists can be found in Talis

Specific reading list for the module

Subject specific skills

Code transformation pattern recognition;
Executable code and data fragment analysis;
Advanced tool use for code function inference;
Obfuscated and malicious code analysis.

Transferable skills

Critical thinking, problem solving