Advanced Parallel Programming

This subject will increase the knowledge on parallel programming acquired by the students in the previous quarter in subjects such as "Parallel Programming" and "Programming of heterogeneous architectures". The aim will be that the students learn to optimize parallel codes for big parallel architectures or current supercomputers, using for their tests the resources provided by the Centro de Supercomputación de Galicia (CESGA) and the Group of Architecture of Computers (GAC) of the Universidade da Coruña (UDC).

We will focus on those aspects of the parallel applications that usually penalize performance, such as the communications, load unbalance, memory access patterns or the management of I/O. We will also tackle multiplatform computing, which allows taking advantage of the task level parallelism by using several hardware accelerators, as well as hybrid computing, where the same application uses several parallel programming paradigms in order to obtain good performance in clusters with multi-core computers and/or hardware accelerators.

- Advanced techniques for the optimization of parallel codes.
- Affinity control and load balance.
- Optimization of communications in distributed memory systems.
- Parallel I/O.
- Hybrid programming for systems with several hardware accelerators.
- Hybrid programming for systems with shared/distributed memory.

Basic bibliography:

- Using Advanced MPI: Modern Features of the Message-Passing Interface. 2014. W. Gropp, T. Hoefler, R. Thakur, E. Lusk. MIT Press
- Using OpenMP: The Next Step: Affinity, Accelerators, Tasking, and SIMD (Scientific and Engineering Computation). 2017. R. They go der Pas, And. Stotzer, C. Terboven. MIT Press
- OpenCL Programming Guide. 2011. To. Munshi, B. Gaster, T. G. Mattson, J. Fung, D. Ginsburg. Addison-Wesley/Pearson Education

Complementary bibliography:

- Multi-core programming. 2006. S. Akhter And J. Roberts. Intel Press.
- Professional CUDA C Programming. 2014. J. Cheng, M. Grossman, T. McKercher. Wross.

Online bibliography:
- https://www.mpi-forum.org/docs/
- https://www.openmp.org/resources/

Competences of the degree that are developed (see memory title):
- Basic: CB6, CB7, CB10
- General: CG1, CG4
- Transversal: CT1
- Specific: CE1, CE2, CE4, CE5 and CE7

- Theory classes, in which the content of each subject is exposed. The student will have all the necessary material before the class and the professor will promote an active attitude, making questions that allow clarifying concrete aspects while leaving open questions for the reflection of the student.

- All teaching materials will be available to students on a virtual platform, which this course will be the Aula Cesga, https://aula.cesga.es/.

- Practical classes in the laboratory and in the classroom, in which students perform directed tasks that allow them to get acquainted from a practical point of view with the contents exposed in the theory classes.

- Development of assignments, in which the students have to apply the knowledge acquired in order to solve different problems in an autonomous way.

- Follow-up support: orientation for the development of the assignments, resolution of doubts, etc.

Formative activities of face-to-face nature and their relation with the competences of the degree:

Theory classes CB6, CE1, CE4, CE7
Practical classes in laboratory CT1, CE2, CE5
Follow-up support CB6, CB7

Formative activities of no face-to-face nature and their relation with the competences of the degree:

Practical classes in laboratory CB10, CT1, CE2, CE5
Development of academically directed assignments CB6, CB7, CB10, CT1, CG1, CG4, CE1, CE2, CE4, CE5, CE7

Evaluation of autonomous and academically directed assignments: 100%

Second opportunity: Evaluation of autonomous and academically directed assignments: 100%

In the case of fraudulent performance of exercises or tests, the regulations of the Normativa de avaliación do rendemento académico dos estudantes e de revisión de cualificacións will be applied.
In the application of the Normativa da ETSE sobre plaxio (approved by the ETSE Council on 12/19/2019), the total or partial copy of any exercise will mean failure on both opportunities of the course, with a grade of 0.0 in both cases.

- Theory classes: 20h face-to-face + 0h autonomous work (total 20h)
- Practical classes in laboratory: 21h face-to-face + 63h autonomous work (total 84h)
- Follow-up support: 1h face-to-face + 0h autonomous work (total 1h)
- Development of assignments: 0h face-to-face + 45h autonomous work (total 45h)

TOTAL: 42h face-to-face + 108h autonomous work, for a total of 150h

Because of the strong interrelationship between the theoretical and the practical part, and because of the progressiveness in the presentation of very related concepts in the theoretical part, it is recommended to devote a daily time of study or review.

In this subject, the intensive use of tools for online communication such as videoconference, email, chat, etc. will be made.

In the non face-to-face evaluation activities, students may be required to apply mechanisms that guarantee their identity as well as their authorship of the elements presented for evaluation purposes.