Psychometrics

INTRODUCTION

Psychometrics is taught in the first quadrimester of the second year of the Bachelor’s degree in Psychology. Together with Data Analysis in Psychology, Research Designs and Research in Psychology (1st year) and Multivariate Analysis (2nd quadrimester, 2nd year), it is a part of the Methodology of Behavioural Sciences area. These subjects constitute the methodological training provided to students of the Bachelor’s degree in Psychology at the USC.

OBJECTIVES

Although Psychometrics is concerned with everything pertaining to measurements in Psychology, this subject focuses on so-called “Test Theory”, which deals with the logic behind the most salient mathematical models for constructing and using tests in this field, namely: Classical Test Theory (CTT), which is taught in depth, and Item Response Theory (IRT), which is only introduced briefly here.

The primary goal is to have students know the procedure for constructing a test, and the methods and techniques to be used, and also to acquire the abilities needed to understand and appreciate psychometric information from commercially available tests and apply them properly.

I. INTRODUCTION

Unit 1. Delimitation of the field of Psychometrics: Measurement in Psychology. Definition of Psychometrics. Origins and development of Psychometrics. Theories of tests.
Unit 2.- Tests: Definition, history and classification: Definition of tests. Classification of tests. Stages in the construction of a test.

II. DESIGN AND ANALYSIS OF ITEMS

Unit 3. Design and development of items: Introduction. Item construction. Response formats. Quantification of responses.
Unit 4. Item analysis: Difficulty index. Discrimination index. Homogeneity index. Reliability index. Validity index. Item analysis in relation to the test. Distractor analysis.

III. CLASSICAL MODEL AND RELIABILITY

Unit 5. Theoretical framework, TCT and IRT: Classical Test Theory. Item Response Theory.
Unit 6. Reliability indicators of a test: Concept of reliability. Reliability as precision. The reliability coefficient. Reliability estimation procedures. Cronbach's alpha. Standard error of measurement and reliability index. Estimation of reliability as precision.
Unit 7. Factors affecting the reliability of a test: Test length. Variability of the test.

IV. VALIDITY

Unit 8. Concept and evidence of validity. Introduction. Concept of validity. Evidence of validity: evidence based on the content of the test, evidence based on the internal structure, and on the relationship with other variables.
Unit 9. Exploratory factor analysis. Introduction. Overview of AFE. Steps of AFE: correlation matrix, extraction, rotation and factorials scores.
Unit 10. Validity coefficient. Introduction. Validity coefficient and interpretation. Using tests to predict criteria: the correlation and regression analysis. Factors influencing validity.

V. STANDARIZATION AND REPORT

Unit 11. Interpretation of scores. Introduction. Transformations of direct test scores: Percentiles, standard Z scores and standard derived scores.
Unit 12. Report of the psychometrics proprieties of test.


INTERACTIVE CLASS PROGRAMME

1. Obtaining and preparing data for the psychometric analysis of a test.
2. Item analysis.
3. Reliability.
4. Factor analysis.
5. Criterion-referenced validity and baremation.
6. Report of the psychometric properties of a test.

BASIC

Abad, F.J. Olea, J., Ponsoda, V., & García, C. (2011). Medición en ciencias sociales y de la salud. Madrid: Síntesis.

Colegio Oficial de Psicólogos (COP). Comisión Internacional de Tests (ITC). Directrices internacionales para el uso de los tests. http://www.cop.es/index.php?page=directrices-internacionales

Fernández Muñoz, J.J. & Marquínez Rengifo, J.M. (2019). Introducción a la Psicometría. Madrid: OMMPress.

Martínez Arias, R., Hernández Lloreda, M.J., & Hernández Lloreda (2014). Psicometría. Madrid: Alianza. (Libro-e: Texto completo disponible vía PRELO).

Meneses, J., Barrios, M., Bonillo, A., Cosculluela, A., Lozano, L.M., Turbany, J. & Valero, S. (2013). Psicometría. Barcelona: UOC.

Muñiz, J. (2018). Introducción a la psicometría. Teoría clásica y TRI. Madrid: Pirámide.

Prieto, G. & Delgado, A.R. (2010). Fiabilidad y validez. Papeles del Psicólogo, 31(1), 67-74.

Seoane, G. & Rodríguez, M.S. (2001). Análisis factorial. En C. Arce y E. Real, Introducción al análisis estadístico con SPSS para Windows (pp. 189-204). Barcelona: PPU.


COMPLEMENTARY

American Psychological Association, American Educational Research Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Barbero García, M.I., Holgado Tello, F.P. & Vila Abad, E. (2015). Psicometría. Madrid: Sanz y Torres.

de Ayala, R. J. (2022). The theory and practice of item response theory (2nd ed.). New York: Guilford Press.

Hidalgo Montesinos, M.D., French, B.F., Finch, H. & Immekus, J.C. (2022). Psicometría aplicada usando SPSS y AMOS. Charlotte, NC: Information Age Publishing.

Miller, L.A. & Lovier, R.L. (2016). Foundations of psychological testing. A practical approach (5th ed.). Thousand Oaks, CA: Sage.

Muñiz, J. (1997). Introducción a la teoría de respuesta a los ítems. Madrid: Pirámide.

Muñiz, J. & Fonseca-Pedrero, E. (2019). Diez pasos para la construcción de un test. Psicothema, 31(1), 7-16. doi: 10.7334/psicothema2018.29

Santiesteban Requena, C. (2009). Principios de psicometría. Madrid: Síntesis.

Specific competences of the degree to which he contributes this subject:

EC1 • To know and understand the functions, characteristics, contributions and shortcomings of existing theoretical methods in Psychology.
EC7 • To know and understand the different methods available for psychological assessment, diagnosis and treatment in different applied fields of Psychology (clinical and medical, social, educational).
EC9 • Identify the most salient characteristics of behaviour in individuals, groups, organizations and contexts by using the typical methods, techniques and instruments for psychological assessment.
EC11 • Select and implement the typical techniques and instruments of Psychology.
EC14 • Produce psychological reports in different contexts, both for the recipient and for other professionals.
EC15 • Comply with the deontological duties of the Psychology profession.


Specific competences of the subject. The specific competences of the degree make concrete in the following specific competences of the subject:

• To know available methods for constructing measurement instruments in Psychology, as well as their application and existing techniques for their psychometric assessment.
• To know the main strategies for designing, quantifying and analysing the metric properties of the different types of items; to select the most suitable procedures for assessing reliability and validity; and to optimize the psychometric properties of measurement instruments.
• To be able to use basic software for analysing the psychometric properties of tests (SPSS).
• To be able to produce technical reports on the psychometric properties of measurement instruments by using accurate language.
• To know and understand the psychometric indicators derived with the assessment instruments used in the different areas of Psychology, as well as those identified in scientific publications.

EXHIBITION SESSIONS: in which the different topics of the programme will be explained (topics 1 to 12), distributed throughout the first four-month period in two weekly sessions, with a duration of 1h 15m.

INTERACTIVE SESSIONS: whose purpose will be to complement the contents of the lectures by means of practical applications. They will take place in a laboratory of the Faculty with computer equipment where the statistical package SPSS will be used. These sessions will have a duration of 2 hours in alternate weeks.

To follow these classes, students will have support material prepared by the professors and other documents that will be provided, with sufficient time in advance, in the Virtual Campus of the subject. In addition, they will be able to consult the basic bibliography of free access or in electronic format, available through the library services of the Faculty.

AUTONOMOUS WORK OF THE STUDENT: in addition to attending the activities indicated in the previous paragraphs, the student will have to dedicate time to read complementary materials, carrying out practical exercises, and studying the contents that will be developed during the course. It is necessary that the student masters the theoretical contents required for the adequate development and use of the interactive sessions.

ATTENDANCE TO CLASSES AND ACTIVITIES: it is considered essential to achieve the objectives of the subject. Attendance to all interactive sessions will be controlled. The consideration of attendance in the evaluation of the student is indicated in the corresponding section of the programme.

TUTORIALS: the teachers of the subject are available in tutorials (through the means, places and times that will be indicated at the beginning of the course) to answer doubts and any question regarding the development of the subject and its contents.

The necessary curricular adaptations will be made to guarantee equal opportunities and universal accessibility for people with special educational needs, with the support and advice of the "Servicio de Participación e Integración Universitaria" (SEPIU) of the University of Santiago de Compostela.

The evaluation of learning and acquired competences will be based on several elements, whose relative weight in the final result is expressed below:

1. Continuous assessment (30%) by carrying out:

- A formative assessment of the first half of the subject (topics 1 to 7) to be carried out in the middle of the first quarter, with a weight of 1 point on the final grade.

- A group work that will consist of the elaboration of a report based on the data analysed in the practical classes and the theoretical/practical contents taught, the delivery of which is planned for the month of December at the end of the four-month period. This work will have a weight of 2 points on the final grade and once the deadline has passed, it will be evaluated and feedback will be provided to the student through the virtual campus.

The planning of the continuous assessment tasks will be conditioned to the calendar of classes as well as the activities of the other subjects of the second year, whose coordination will be carried out at the beginning of the course.

This maximum score of 3 points will only be applied to those students who have attended at least 60% of the practical classes and pass the final exam (with a score of 3.5 or higher). In addition, the mark obtained in the continuous assessment will be retained for two academic years, in the event that the subject is not passed.

2. Final exam (70%) which will consist of questions on theoretical/practical contents included in the syllabus related to the results of the psychometric analysis of a test (SPSS). The form and the calculator may be used as support material. The date of the exam corresponds to the official timetable published on the Faculty's website and on the centre's notice board.

The completion/submission of any of the tests listed in the preceding sections implies the incorporation of the evaluation process, so it will lead to a final grade of the subject.

In order to pass the subject, students must pass the exam and obtain a total score (sum of continuous assessment + final exam) equal to or higher than 5.

Those students who do not take part in the continuous assessment may take the exam for the whole subject in which both theoretical and practical contents are assessed. In order to pass the exam they will have to obtain a score of 5 out of 7 points.

In the second opportunity to pass the subject, scheduled for the month of June/July for those students who do not pass or do not sit the exam at the first opportunity, a final exam will be held with the same characteristics as described above. In addition, students who wish to improve or obtain their score in the continuous assessment will have a second period (in June) to present the work planned for the first opportunity.

Students who are granted dispensation from class attendance (following the Instruction No. 1/2017 of the General Secretariat on the dispensation of class attendance in certain circumstances), will be evaluated with a final exam that will account for 70% of the grade, in addition to an autonomous work (30%) that will be specified in tutorials with the teachers of the subject.

In cases of fraudulent performance of exercises or tests will apply the provisions of the Regulations for the evaluation of the academic performance of students and review of grades (Resolution June 15, 2011, DOG July 21, 2011).

The subject is allocated 6 ETCS credits, which corresponds to an estimated total of 150 hours of student work distributed as follows:

Attendance of in-class activities (lectures, assessment): 50 hours

Out-of-class work: 100 hours, including preparation of lecture and class contents,resolution of exercises and study of the subject materials.

In order to properly learn this subject, students are advised to:

(1) Fulfil the objectives of the first year of the Bachelor’s degree, especially as regards the subjects most closely related to Psychometrics (Data Analysis in Psychology and Research Designs).

(2) To keep up with work on the subject by completing the proposed activities, reading the recommended materials, studying the taught contents, and attending tutorial sessions to solve their doubts or obtain guidance on their learning process.

The tutoring schedule during the teaching period of the subject is:

Tuesday from 12h to 14,30h

Wednesday from 12h to 14,30h

It is advisable to send an e-mail to the teacher in advance to communicate the attendance to the tutorial.