Skip to main content Skip to navigation

PX915-30 Modelling of Heterogeneous Systems Group Project and Peer-to-peer Project Evaluation

Department
Physics
Level
Taught Postgraduate Level
Module leader
Peter Brommer
Credit value
30
Module duration
52 weeks
Assessment
100% coursework
Study location
University of Warwick main campus, Coventry
Introductory description

N/A.

Module aims

Part I: Group Project
As a project team, create a piece of research software following the principles learned in the HetSys Core modules: PX911, PX912, PX913 and PX914.
Part II: Peer-to-Peer Project Evaluation
Validate the error estimate and portability of two codes from a library of current or past individual projects or existing projects.

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

Part I: Group Project

  1. Project teams of 5-6 students and 2 project supervisors (1 academic, 1 research software engineer).
  2. Students will design, specify, implement and document a small-scale simulation package to solve a problem within the modelling of heterogeneous systems: examples include DFT, molecular dynamics or computational fluid dynamics.
  3. Projects must meet all 3 key CDT challenges: Cross-disciplinarity, robust software design, uncertainty quantification.
  4. Students will present and demonstrate their software to the CDT.
    Part II:
  5. Software created in individual projects is distributed to students. Each student receives two distinct codes.
  6. Students examine the compliance of the codes with research software engineering principles, with a particular focus on usability (e.g. by examining documentation), portability (e.g. by running on a different platform) and sustainability (e.g. by recapitulating parts of the code). Software authors and assessors are encouraged to engage with each other. ILO II.2.
  7. Students run the codes with an ensemble of input values and study the variation of the results (sensitivity analysis). ILO II.1.
  8. Students compare the outcome of this analysis with the error estimates provided by the software authors. ILO II.3.
  9. Students provide critical feedback on the error estimates to the software authors in an error estimate assessment report (3 pages per software). ILO II.4.
  10. Students present their feedback to the authors. Authors judge the quality of the feedback and award a peer mark.
Learning outcomes

By the end of the module, students should be able to:

  • Part I: Group Project Create a piece of research software implementing methods and using software design principles introduced in previous modules in the course, addressing a cross-discipline research computing challenge.
  • Part I: Group Project Plan and manage the research software engineering project from the specification phase to a deliverable outcome, including documentation.
  • Part I: Group Project Analyse and validate research software results, both by using automated testing and manual comparisons.
  • Part I: Group Project Formulate and quantify the uncertainty in the output of research software, thus giving an measure of the reliability of a computed quantity.
  • Part I: Group Project Demonstrate effective communication and documentation skills, both within the group and for the benefit of a future software user.
  • Part I: Group Project Demonstrate the ability to work as a member of a team to achieve shared objectives within the scope of the project and monitor and adjust a personal programme of work on an ongoing basis.
  • Part II: Peer-to-Peer Project Evaluation Construct error estimates of third-party simulation codes using a simulation ensemble with varied input parameters.
  • Part II: Peer-to-Peer Project Evaluation Validate a code’s compliance with research software engineering principles, including usability, portability, and sustainability.
  • Part II: Peer-to-Peer Project Evaluation Critically assess the error estimates provided by the software authors.
  • Part II: Peer-to-Peer Project Evaluation Provide critical feedback to software authors on software design and uncertainty quantification.
Indicative reading list

J. C. Carver, N. P. Chue Hong, G. K. Thiruvathukal, Software Engineering for Science
S. Oliveira, D.E. Stewart: Writing scientific software : a guide for good style
R. Stephens: Beginning software engineering

Interdisciplinary

Mixes domain-specific simulation and mathematics with implementation and software engineering.

Subject specific skills

Scientific software development, domain-specific knowledge.

Transferable skills

Teamwork, collaborative code development, presentation skills, portfolio writing.

Study time

Type Required
Lectures 5 sessions of 1 hour (2%)
Project supervision 8 sessions of 1 hour (3%)
Supervised practical classes 15 sessions of 1 hour (5%)
Other activity 2 hours (1%)
Private study 110 hours (37%)
Assessment 160 hours (53%)
Total 300 hours
Private study description

Part 1: 60 hours free study (researching, programming, group meetings,...)
Part 2: 50 hours individual work

Other activity description

Feedback presentation session where students present their findings to the cohort and give feedback on the performance and uncertainty estimates of the codes.

Costs

No further costs have been identified for this module.

You do not need to pass all assessment components to pass the module.

Assessment group A2
Weighting Study time
Software Development Plan 15% 10 hours

(Part I) Software Specification, Design Document, Workplan for Research Software Project. Includes peer assessment component.

Portfolio of Project Outcomes: Documentation, Tutorial and Code 40% 100 hours

(Part I) Group assessment: 40%. Project code, tutorial and documentation (user- and developer-facing). Includes peer assessment component.

Presentation and Usage demonstration 15% 20 hours

(Part 1) 1h final presentation session & usage demonstration, to the CDT.

Individual Contribution Statement 10% 10 hours

(Part 1) Individual contribution based on contribution statement (supported by revision system logging data).

Uncertainty Quantification Report 20% 20 hours

(Part 2) Construct error estimates of third-party simulation codes using a simulation ensemble with varied input parameters.
Validate compliance of two sets of code with research software engineering principles, including usability, portability, and sustainability.
Critically assess the error estimates provided by the software authors.
Provide critical feedback to software authors in an error estimate assessment report and oral presentation, thus demonstrating effective written and oral communication skills.
Mark includes peer assessment from original software authors.

Feedback on assessment

Part I: Group Project

Extensive written feedback on software specification document
Q & A session with examinersduring Software Demonstration / Presentation session
Written feedback on Individual Contribution Statement
Written feedback on Performance Assessment of delivered code

Part II: Peer-to-Peer Project Evaluation
Assessment reports will be returned with detailed feedback
Peer assessment will be received (subject to moderation) on the reports.

Courses

This module is Core for:

  • PG Diploma and MSc in Modelling of Heterogenous Systems