Methods of Public Opinion Research in New Technological and Social Environments

Identification of a Social Problem

The increasing internet penetration and evolving technological possibilities have made online and/or self-administered questionnaire surveys easily accessible. While these surveys may reach an impressive sample size (i.e., a sufficient number of people responding to the survey), the reliability and generalizability of the data are significantly lower compared to probability samples (e.g., face-to-face surveys). In Hungary, based on the current internet coverage, online-collected samples lead to unreliable estimates in 80% of cases and significant distortions when generalizing the results to the entire Hungarian population. This is due to the substantial differences between internet-accessible and internet-inaccessible segments of the population in the majority of the examined questions. In recent years, a significant portion (40%) of social science research has relied on low-quality online-collected samples, and the majority of online news articles (70%) report statistically weak data. Thus, the problem is not only scientific but also relevant from a public perspective, considering the increasing intensity of data generation. Proper handling and interpretation of data are crucial for avoiding misleading research results. In social science, adhering to proper data collection protocols and generalizing from the data to the entire population under strict conditions are essential for further development. This requires appropriate and in-depth data related to the population.

Research Outcome

The research findings are utilized as a concrete, tangible guideline specifically designed for the research field, based on the demographic parameters of the Hungarian population. It provides more precise considerations for research design and ensures more reliable results in research outcomes.

Introduction of the Developed Method

A publication has been prepared to present the research, discussing the mathematical-statistical dimensions that are compromised in non-probability sampling and explaining why this leads to unreliable results. It emphasizes the issue of quota sampling and the misinterpretation of representativeness, which can be applied to telephone and other convenience samples, and are applicable in research practice. In online data collection, key aspects include determining the population and monitoring internet coverage. The study reveals that the population of internet users and daily internet users significantly differs from the total adult population in terms of basic demographic aspects. For example, individuals aged 55 and above and those with a lower level of education are underrepresented, while young people, highly educated individuals, and residents of the central Hungarian region are overrepresented. The study examines the potential estimation errors of online samples through simulation analysis. In three-quarters of cases, the true value is either underestimated or overestimated, and even small overall differences between internet users and the total population are sufficient to draw incorrect conclusions. In addition to simulation results, real data are also examined. Based on the European Social Survey (ESS) sample, each of the four variables examined produces estimates significantly different from the total sample when only internet users are surveyed, even when adjusting educational proportions to match population proportions.

Contact

Blanka Szeitl (Eörtvös Loránd University, Facuty of Social Sciences)
E-mail