There is a never ending need for better quality and comparable international social sample survey data. The risk of non-response bias affecting research estimates is real as response rates decline over time. The most common ways to approach non-response in literature are the comparison of the characteristics of different types of respondents, call-backs as a practice to increase responses and response rates. However, the study of non-response presupposes the collection of relevant data and there are only a few surveys that they are doing so as is the European Social Survey (ESS).
Τo assess the importance of non-response in large-scale sample survey data, the originality of this dissertation lies in the investigation for the first time in the literature of the existence (or not) of differences in the demographic "profile" of respondents, refusals, and reluctant respondents. The socio-political "profile" of respondents and reluctant respondents is also examined. The method of call-backs is evaluated, a rate for measuring refusals conversion (Converted Refusals Rate-CRR) is proposed, and the contribution of call-backs to this rate is assessed. Finally, the response rate applied by ESS is evaluated according to Kish's definitions. The analysis is based on the first eight rounds (2002-2016) of the ESS for the following 11 countries: Belgium, Germany, Switzerland, United Kingdom, Spain, Norway, the Netherlands, Poland, Slovenia, Sweden, Finland.
The findings showed that there were statistically significant differences for gender and age between respondents, refusals, and reluctant respondents in all countries for at least one round. The reluctant respondents differ significantly from the respondents and are mainly women, older on average, having completed fewer years of compulsory education, higher feeling of religiousness, lower family income, varying degrees of urbanization of their permanent residence, less social and political trust, less interest in politics, are not actively offering to volunteer at the same degree, but they abstain less from their voting rights. The analysis showed that four or five visits is adopted as a minimum number of call-backs in most countries. The CRR values range from 8.72% to 25.73%, while the contribution of call-backs to refusals after two call-backs does not exceed 1% in most countries. Finally, the comparison of the official ESS response rate index to that based on Kish’s definitions resulted in statistically significant differences for all 11 countries, with Germany, Spain, Poland and Slovenia having the largest systematic deviations (1.01- 7.91%). In spite of ESS’s efforts to address non-response, which is an example of a reliable large-scale sampling survey with strict comparability protocols, evidence of bias is found in the data. This indicates that the problem of non-response will continue to be of concern in the international literature.
This methodological study aims to contribute to the continually growing field of large-scale sample surveys by discussing issues of further ensuring the quality of international comparable data collection. The findings are useful to the scientific community as they confirm the magnitude of the non-response problem in modern social research and the practices adopted in the field to address it adequately. Moreover, the new scientific tool that is proposed may be used by scientists in any sample survey collecting data for refusals.
Καθ. Clive Richardson