Introductory description

Modern programming approaches use high level constructs which abstract away the system's architecture. These high levels of abstraction use code generation programs such as compilers and assemblers to take the human author's input, and produce code that will execute as output. The modern programmer rarely needs to consider the underlying architecture of the machine that will execute the code.
There are situations where, rather than creating an executable from source, you need to go in the opposite direction; you need to infer what the source code might look like by analysing the executable. Perhaps you have some potential malware; or perhaps you perhaps you have to analyse and exploit a vulnerability which an executable might have. Either way, you want to know what the program will do, were it to run on your system.
In order to reverse engineer an executable, you need to understand the typical idioms that an operating system, architecture and code generation programs will adopt to convert high level constructs into low level executables.
In addition, if the executable is malware, then it is likely the authors will have used some obfuscation in order to make the analysis more difficult. Under these circumstances you need to understand the typical idioms of obfuscation.

Module aims

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

The content of this module will be taught from a cyber security perspective.

• executable code from a variety of perspectives
• assembly language programming
• machine-level instruction set and organisation
• code generation
• reverse engineering techniques
• de-obfuscation
• common tools for reverse engineering
• anti-debugging mechanisms
• fuzzing

Learning outcomes

By the end of the module, students should be able to:

• Identify common idioms and patterns used during code transformation and explain the origin and organisation of arbitrary code and/or data fragments within an executable program.
• Apply tools and techniques as appropriate to infer the executable's overall high-level function, potentially obfuscated, potentially malicious code.
• To perform malicious code analysis, vulnerability identification and evaluation independently from the findings generated by automated analysis tools.

Indicative reading list

Aho, A. V., Lam, Monica S., Sethi, R. and Ullman, Jeffrey D., “Compilers: Principles, Techniques, and Tools”, 2 Ed, Pearson (2013)
Sikorski, Michael and Honig, Andrew “Practical Malware Analysis”, No Starch Press (2012)
Szor, Peter, “The Art of Computer Virus Research and Defense”, Addison-Wesley (2005)

View reading list on Talis Aspire

Subject specific skills

1 - Identify common idioms and patterns used during code transformation and so explain the origin and organisation of arbitrary code and / or data fragments within an executable program.

2 - Apply tools and techniques as appropriate to infer the overall high level function of executable, potentially obfuscated, potentially malicious code.

Transferable skills

Critical thinking, problem solving