Кто читает:: Департамент социологии

Статус:: Курс по выбору

Когда читается:: 4-й курс, 1, 2 модуль

Преподаватель

Немировская Анна Валентиновна

Full Syllabus Ask Question

Abstract

The course aims to develop students' deep understanding of the cognitive, sociocultural, and measurement aspects underlying sociological questions, as well as applied skills in designing complex survey instruments.

Learning Objectives

To equip students with systematic theoretical knowledge and applied skills in designing survey instruments (questionnaires and standardized interview guides), taking into account measurement methodology, cognitive psychology, and the sociocultural characteristics of respondents.
To master the techniques of operationalizing concepts and transitioning from theoretical constructs to empirical indicators.
To analyze the nature of measurement errors (within the Total Survey Error framework) and their sources.
To examine respondents' cognitive processes when answering questions (comprehension, retrieval, judgment, and response formulation).
To develop skills in designing various types of questions (regarding behavior, attitudes, and knowledge) and measurement scales for academic and applied research.
To familiarize students with the ethical standards of conducting sociological surveys and the specifics of designing sensitive questions.
To build skills in adapting survey instruments for various data collection modes (online surveys, mobile interfaces, telephone interviews, and CAWI/CAPI/СATI), taking into account visual design and the specifics of respondent interaction.
To learn how to apply methods for piloting and testing survey instruments (cognitive interviewing, pre-testing).

Expected Learning Outcomes

Apply foundational measurement theories and cognitive psychology principles to operationalize complex theoretical concepts into empirical indicators.
Analyze respondent cognitive processes and identify sources of measurement bias within the Total Survey Error framework.
Design and develop complex survey instruments tailored to specific research objectives and varied data collection modes.
Formulate effective survey questions, including sensitive items, by applying ethical standards and specialized techniques to minimize social desirability bias.
Evaluate and refine survey instruments by conducting rigorous pilot testing, including cognitive interviews and pre-testing.
Develop practical surveys in specialized programming platforms, integrating optimal visual design and user experience principles.
Critically audit and assess the validity, reliability, and overall methodological quality of applied and academic survey tools.

Course Contents

Section 1: Foundations of survey design and conceptualization. Lecture 1: Introduction to survey methodology
Section 1: Foundations of survey design and conceptualization. Seminar 1: Formulating and critiquing research questions in different types of research.
Section 1: Foundations of survey design and conceptualization. Seminar 2: Conceptualization and operationalization.
Section 1: Foundations of survey design and conceptualization. Seminar 3: Institutional context and the function of questions.
Section 1: Foundations of survey design and conceptualization. Seminar 4: Cross-national and comparative survey design.
Section 1: Foundations of survey design and conceptualization. Seminar 5: The total survey error (TSE) framework.
Section 2: Cognitive aspects of question design and response. Lecture 2: Cognitive psychology of the survey response.
Section 2: Cognitive aspects of question design and response. Seminar 6: Designing questions about behavior and events.
Section 2: Cognitive aspects of question design and response. Seminar 7: Designing questions about attitudes.
Section 2: Cognitive aspects of question design and response. Seminar 8: Sensitive questions and social desirability.
Section 2: Cognitive aspects of question design and response. Seminar 9: Interviewer effects.
Section 2: Cognitive aspects of question design and response. Seminar 10: Cognitive analysis in practice.
Section 3: Instrument architecture, scaling, and visual design. Lecture 3: Anatomy of the Survey Instrument.
Section 3: Instrument architecture, scaling, and visual design. Seminar 11: Constructing Closed-Ended Questions and Scales.
Section 3: Instrument architecture, scaling, and visual design. Seminar 12: Open-Ended Questions and Self-Ratings.
Section 3: Instrument architecture, scaling, and visual design. Seminar 13: Respondent Burden and Visual Design.
Section 3: Instrument architecture, scaling, and visual design. Seminar 15: Multivariate Metrics and Construct Validation.
Section 3: Instrument architecture, scaling, and visual design. Seminar 14: Mode Effects.
Section 4: Pretesting, evaluation, and ethical implementation. Lecture 4: Methodology of Survey Instrument Testing.
Section 4: Pretesting, evaluation, and ethical implementation. Seminar 16: Designing and Conducting Cognitive Interviews.
Section 4: Pretesting, evaluation, and ethical implementation. Seminar 17: Cognitive Lab Practice: Understanding Complex Concepts.
Section 4: Pretesting, evaluation, and ethical implementation. Seminar 18: Behavior Coding and Pilot Analysis.
Section 4: Pretesting, evaluation, and ethical implementation. Seminar 19: Ethical dilemmas in survey research.
Section 4: Pretesting, evaluation, and ethical implementation. Seminar 20: Data preparation for analysis.

Assessment Elements

Class activity
The in-class participation
Homework
Homework is an individual assignment requiring the justification and design of a research instrument.
Exam
The exam is conducted in the form of an individual project assignment (research instrument design). Final individual project assignment: design, cognitive audit, and piloting of a survey instrument. Context and conditions: students act as research methodologists tasked with developing an instrument to measure a complex concept for studying a sociological problem or practical case. The assignment is to be completed individually. Execution specifications (work stages): the assignment must be formatted as a single analytical report (portfolio) consisting of 4 mandatory sections: Stage 1: Conceptualization and operationalization (theory) • Select specific theories applicable to the analysis of the chosen sociological problem. • Construct an "operationalization tree" in the form of a table: theoretical construct → dimensions → empirical indicators → question wording. Stage 2: Questionnaire architecture (construction) • Develop a draft of a formalized questionnaire (20–30 questions). • Mandatory requirements for the questionnaire:  inclusion of different question types (e.g., questions about behavior, attitudes, and knowledge/awareness).  use of at least 3 different types of measurement scales (e.g., Likert scale, semantic differential, ordinal scale).  inclusion of at least 3 sensitive questions (e.g., regarding shadow income, conflicts with management, or intention to quit) utilizing techniques to mitigate social desirability bias. Stage 3: Cognitive audit and error forecasting (TSE) • Prior to fieldwork, conduct a self-audit of the questionnaire based on the Total Survey Error (TSE) paradigm. • Select the 5 most complex questions from the questionnaire and describe the potential cognitive difficulties respondents might face at the following stages: comprehension → retrieval → judgment → response selection. Stage 4: Piloting and content analysis of transcripts (practice) • Conduct 2–3 cognitive interviews with representatives of the target audience, applying "think-aloud" and verbal probing techniques. • Produce verbatim transcripts of the audio recordings. • Perform a content analysis of these transcripts by themes and categories of difficulties (e.g., terminology difficulties, lack of response options, resistance to sensitive topics). • Submit the final, revised version of the questionnaire along with a modification log table (original / revised / justification based on content analysis).

Interim Assessment

2026/2027 2nd module
Class activity * 0.3 + homework (home assignments) * 0.3 + Exam work (research instrument design) * 0.4

Bibliography

Recommended Core Bibliography

Constructing research questions: doing interesting research, Alvesson, Mats, 2024
Presser, S. (2004). Methods for Testing and Evaluating Survey Questionnaires. Hoboken, NJ: Wiley-Interscience. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=535693

Recommended Additional Bibliography

Когнитивный анализ опросного инструмента, Рогозин, Д. М., 2002

Authors

Nemirovskaia Anna Valentinovna

Бакалаврская программа «Социология и социальная информатика»

Контакты

Коронавирус Live

Survey Instrument Design