Neural Networks and Deep Learning

Neural networks are a method of supervised machine learning. This course covers the most important neural network models, from classical networks to deep neural networks, analyzing training methods and their theoretical and practical aspects. Recurrent neural networks for sequential information processing will also be studied. Additionally, some of the most successful applications of deep neural networks will be addressed, including computer vision through convolutional neural networks.

THEORY
Topic 1: Introduction to neural networks.
Topic 2: Convolutional neural networks.
Topic 3: Recurrent neural networks.
Topic 4: Transformers.
Topic 5: Autoencoders.
Topic 6: Generative adversarial networks.

PRACTICALS
A set of practical bulletins will be carried out related to convolutional networks; recurrent networks and transformers; and autoencoders and generative adversarial networks.

BASIC BIBLIOGRAPHY
- A. Bosch Rué, J. Casas-Roma, T. Lozano Bagén (2019): Deep learning : principios y fundamentos. https://elibro-net.ezbusc.usc.gal/es/ereader/busc/126167/
- A. Zhang, Z.C. Lipton, A.J. Smola (2023): Dive into Deep Learning. https://d2l.ai

COMPLEMENTARY BIBLIOGRAPHY
- R. Atienza (2018): Advanced deep learning with Keras : apply deep learning techniques, autoencoders, GANs, variational autoencoders, deep reinforcement learning, policy gradients, and more. https://ebookcentral-proquest-com.ezbusc.usc.gal/lib/buscsp/detail.acti…
- M. Phi (2018): Illustrated Guide to LSTM’s and GRU’s A step by step explanation by Michael Phi Towards Data Science. https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-s…
- M. Steward (2019): Comprehensive Introduction to Autoencoders. https://towardsdatascience.com/generating-images-with-autoencoders-77fd…
- X. Mao, Q. Li (2021): Generative Adversarial Networks for Image Generation. Springer. https://link-springer-com.ezbusc.usc.gal/book/10.1007/978-981-33-6048-8
- J. Langr, W. Bok (2019): GANs in Action: Deep Learning with Generative Adversarial Networks.

Students will acquire a set of generic competencies, others specifically associated with Artificial Intelligence, and others more transversal:

BASIC AND GENERAL COMPETENCIES
[CB2] Students should be able to apply their knowledge to their work or vocation in a professional way and possess the competencies typically demonstrated through the development and defense of arguments and the resolution of problems within their field of study.
[CB3] Students should have the ability to gather and interpret relevant data (usually within their field of study) to make judgments that include reflection on relevant social, scientific, or ethical issues.
[CB5] Students should develop those learning skills necessary to undertake further studies with a high degree of autonomy.
[CG3] Ability to design and create quality models and solutions based on Artificial Intelligence that are efficient, robust, transparent, and responsible.
[CG4] Ability to select and justify the appropriate methods and techniques to solve a specific problem, or to develop and propose new methods based on Artificial Intelligence.
[CG5] Ability to conceive new computational systems and/or evaluate the performance of existing systems that integrate models and techniques of Artificial Intelligence.

SPECIFIC COMPETENCIES
[CE12] Know the fundamentals of artificial intelligence algorithms and models for solving problems of certain complexity, understand their computational complexity, and have the ability to design new models.
[CE15] Know and know how to correctly apply and explain the validation techniques of artificial intelligence solutions.

TRANSVERSAL COMPETENCIES
[TR2] Ability to work in a team, in interdisciplinary environments and managing conflicts.
[TR4] Ability to introduce the gender perspective in models, techniques, and solutions based on artificial intelligence.
[TR5] Ability to develop models, techniques, and solutions based on artificial intelligence that are ethical, non-discriminatory, and reliable.

Additionally, the learning outcomes will be as follows:

- Knowing how to develop and configure different neural network architectures, selecting the most appropriate for the different problems to be addressed.
- Understanding the structure and applications of recurrent and convolutional neural networks.
- Knowing the different tools for the development of deep learning networks.

The teaching methodology is aimed at focusing the course on the theoretical and practical aspects of neural networks and deep learning techniques and architectures. Therefore, students should be able to understand the advantages of these machine learning techniques and develop a solution to the different problems that will be shown. Taking this into account, two types of learning activities are distinguished: lectures and small group sessions. Thus:

(1) Lectures (30 hours) are aimed at explaining the characteristics of neural networks and the different deep learning architectures, with special emphasis on the situations in which these architectures should be applied and the concepts and mathematics that support them.

(2) Practical sessions in small groups (20 hours) are aimed at students acquiring skill in implementing the different deep learning architectures. Therefore, it is important that in these practices a set of exercises are carried out in which these types of architectures are tested, and the context in which they should be applied is studied given the characteristics of the problem to be solved. Attendance at these classes is MANDATORY.

The assessment of the course will take place in two different, although complementary ways, which aim to evaluate the competence in the theoretical and practical domain of the concepts of neural networks and deep learning. On the other hand, a distinction will be made between the ordinary opportunity assessment and the recovery opportunity:

ORDINARY OPPORTUNITY
(1) Exam to demonstrate the mastery of neural networks and deep learning, focusing on the characteristics and problems solved by each of the different architectures studied throughout the course. In this exam, students must answer a set of questions about the course content. This part will represent 60% of the final grade for the course.

(2) Completion of a set of bulletins to practically demonstrate the mastery of the concepts of neural networks and the different deep learning architectures. All bulletins will be completed individually. This part will constitute 40% of the final grade for the course. Finally, if any of the bulletins are submitted, it will be considered as presented in the course.

It should be noted that to pass the course EACH PART MUST BE PASSED SEPARATELY.

RECOVERY OPPORTUNITY
The evaluation criteria for the theory and practice parts in the recovery opportunity will be exactly the same as for the ordinary opportunity. Therefore, in addition to passing the theory exam and the bulletins, it will be necessary to have attended the interactive practical sessions (with the attendance criteria indicated below) to pass the course.

ATTENDANCE CONTROL
As mentioned earlier, attendance at the interactive practical sessions is mandatory because key concepts of the course are addressed in them, and attendance control will be carried out through sign-in sheets that must be completed at the end of each session. Additionally, if less than 80% of the interactive practical sessions are attended, it will be considered that the course has not been passed.

In the event of fraudulent completion of exercises or tests, the provisions of the Regulations on the evaluation of students' academic performance and grade review will apply. In application of the ETSE Regulations on plagiarism (approved by the ETSE Board on 19/12/2019), the total or partial copying of any practical or theoretical exercise will result in a fail in both opportunities of the course, with a grade of 0.0 in both cases.

As indicated earlier, attendance at the practical sessions (20 hours) is mandatory, and this participation should be active to adequately utilize the time, while attendance at the theoretical sessions (30 hours) is highly recommended, as the concepts of the course are introduced and explained in them. In addition to this, additional time will be needed to work on the following aspects:

(1) Autonomous study of the concepts of neural networks and deep learning (30 hours). The time dedicated to this study includes not only the preparation for the theoretical exam but also the time needed to understand the theoretical concepts of the course.

(2) Completing the exercises in the bulletins (65 hours). This time is necessary to complete the exercises in the bulletins that are not finished in the practical sessions. During this time, the way to solve the problem posed in the exercise can be internalized, as these sessions place more emphasis on understanding the problem and the general way it will be solved, while the details necessary to complete the exercises must be done in additional practical work time.

(3) Theoretical study of matter (30). This time is necessary for the preparation of the final exam as to establish the theoretical concepts based on the results and reflection carried out in the exercises of the bulletins.

(4) Evaluation (5 hours). This is the time dedicated to the evaluation of both the practicals and the theoretical exam.

It is recommended that the theoretical part is studied using the practical content, in which the proposed neural networks and deep learning architectures are programmed and used. This should help establish the relationships between the mathematical models that support these techniques and their practical implementation.

The languages of instruction for the lectures and interactive sessions will be Galician and Spanish. Some of the course content may be in English.