Description

Examining the consequential validity of standardized examinations via public perceptions: a review of mixed methods survey design considerations

John H. Hitchcock+, Anthony J. Onwuegbuzie++ and Heba B. Khoshaim+++

+Instructional Systems Technology Department, Indiana University, Bloomington, IN, USA;

++Department of Educational Leadership, Sam Houston State University, Huntsville, TX, USA;

+++Department of General Sciences, Prince Sultan University, Riyadh, Saudi Arabia

ABSTRACT

The use of standardized examinations to inform high-stakes decisions is a process that is followed in many countries, and there is merit to understanding the impact that such testing has on any given society. For this reason, some researchers have studied the societal impact of testing to understand the social validity and consequences of assessment. Information gleaned from such efforts can be used to inform testing practices as well as communication efforts with stakeholders. Assessing social validity, and aspects of the consequences of measurement, can be handled in part via survey studies. Because the use of testing is becoming more widespread, and across several nations, anyone interested in such inquiry will need to be able to develop culturally relevant surveys. For this reason, this methodological article describes an exploratory and sequential mixed methods research approach for survey development.

KEYWORDS: Standardized testing; mixed methods; measurement; culturally relevant surveys; social validity; consequential validity