Syllabus
CS491 High-Performance Concurrent Computing
Concurrency is key to high performance in computers today, whether exposed to the programmer, as in multi-threaded programming on multi-core computers, or hidden, as in out-of-order execution and instruction level parallelism in modern CPU cores.
This class covers the design of high-performance programs, based on an understanding of the various forms of concurrency present in in a contemporary computer, as well as the tools and techniques needed to measure, diagnose and improve performance of programs in general, and concurrent programs in particular.
The focus throughout this class will be on individual computer systems, as opposed to supercomputing architectures, distributed/cloud computing, or networking.
Lectures will be accompanied by multiple significant programming assignments in C and/or Rust, featuring the various systems covered in class. Tentatively, and pending some hardware availability constraints, the the class may cover the following topics:
- profiling and debugging tools
- high-performance multi-threading in SMP systems
- instruction level parallelism
- performance implications of SMP system cache coherence
- OpenMP as an alternative to explicit threading
- review of explicit multi-threaded programming and synchronization primitives
- scalable concurrent data structure design
- performance considerations in memory management
- delegation as an alternative to synchronization
- parallel programming paradigms and pitfalls
- performance and scalability considerations with 100’s of x86 threads on one chip
- GPU programming with CUDA or OpenCL
- SIMD programming
- weaker memory consistency models
examples of topics we will not be covering:
- cloud computing
- mapreduce
- MPI
- complexity of concurrent algorithms
- distributed systems
- distributed shared memory
Course and Instructor Details Heading link
| Meets | TR 12:30-1:45 in BH B6 |
| Instructor | Jakob Eriksson |
| - office hours | Tue 1:45-2:45 pm, Wed 1 pm-2 pm, in SEO 1309 |
| TA |
Ben Baenen |
| - office hours | Wed 3-5 pm, SEO 120 |
Goals Heading link
The goal of this class is to introduce students to the various forms of concurrent computing broadly available today, including several hands-on exercises to get experience with the tools we use to harness this concurrency.
While the class will cover safety and correctness, the focus of the class will be performance. As in, providing a fundamental understanding of each technology and/or method, so that students will be know when and how to leverage various concurrent computing technologies to speed up their application.
Prerequisites Heading link
CS361 Computer Systems is a pre-requisite for undergraduate students, and CS385/461 Operating Systems provides very good background.
For graduate students, CS361 or equivalent experience is strongly recommended, and significant C programming experience is necessary.
Textbook Heading link
The course has no official textbook. Materials will be disseminated throughout the class, and a set of partial class notes will be made available.
For diagnosing system performance, the following textbook is recommended:
Systems Performance by Brendan Gregg and his accompanying website http://www.brendangregg.com
Academic Integrity Policy Heading link
All course projects are individual, and you are expected to turn in only your own work. Discussing course projects with classmates is encouraged, but sharing code or other products is not allowed.
Occasionally, it will be expedient to copy “found code” from online sources. This is allowed, but only with a citation. To be safe, always accompany any third-party code included in your turn-in by a reference to its origins. A simple comment like this, will suffice to avoid disciplinary action:
/* from http://stackoverflow.com/ntoeuh8.html */
Note, however, that wholesale copying of solutions, even with a citation, does not earn homework credit. For the best grade outcome, and the best retention of knowledge and skills, plan to do all work on your own.
Grading Heading link
The course grade will be determined based on exam and homework assignment/project outcomes. There are two exams, the midterm and the final, worth 20% and 30% of the course grade respectively. There are a yet-to-be-determined number of assignments, that in aggregate are worth 50% of the grade.
The grading curves/cut-offs are set after the final exam results are in. Graduate and undergraduate students are graded on separate curves, with expectations higher on graduate students.
Students are advised that exams tend to be hard, while homework assignments are merely hard work. Moreover, the best way to study for the exams is to complete the assignments on your own, and to explore beyond.