I

ndependent pollsters and opinion polls are undeniably important in any democratic country, even though they are rather new to Indonesia.

A number of pollsters have introduced their quick count results in the recent presidential election and based on these results, both sides claimed victory. The mass media published different quick count results extracted from several pollsters.

The reports have raised an earlier debate about pollsters all over again. Some argue that pollsters are mere money-oriented bodies, while pollsters try very hard to convince the public that they use reliable scientific methods.

Fadli Zon, supporter and close confidante of presidential candidate Prabowo Subianto, has claimed those pollsters announcing Prabowo-Hatta Rajasa as the winner to be absolutely reliable. So can we completely trust these pollsters? How can we make sure these pollsters are reliable?

Let'€™s try to discriminate the trustworthy pollsters from the unreliable and unethical street-surveys. There are several fundamental principles in conducting survey research that anyone should take into consideration when processing information about opinion polls.

First, survey methods have limitations. Most people are already quite familiar with the margin of error, which reflects the confidence level estimated by the pollster. A margin of error also estimates the discrepancy between actual results and survey outcomes.

It can be easily calculated if the pollsters take their samples randomly, which means every individual in the population has an equal chance of being selected as a sample.

However, even if the pollsters do not use simple random sampling techniques, a margin of error can be calculated using more advanced statistical techniques. By giving the appropriate consideration to a margin of error, we can assess the general quality of the survey.

Most pollsters have paid appropriate attention to and have been properly cautious about margins of error, but there is still another form of error that is not so easy to estimate but has a greater impact on survey quality '€” non-sampling errors.

Non-sampling errors in survey research lie in problems related to the interviewers, interviewees, questionnaire design, data coding, data cleaning or even in the wording of the questions. Sometimes the context of the questions can make a significant difference, for example, asking participants '€œWould you support Joko '€˜Jokowi'€™ Widodo as Indonesian president?'€ and '€œWhich presidential candidate would you vote for?'€ will definitely lead to various answers.

In regards to research interviewers, recent research has confirmed that female interviewers are more likely to get more responses than male interviewers, which means the gender of the interviewers can affect the participants'€™ responses. Interviewers'€™ personality, interview techniques and their expertise are some other factors that may also determine non-sampling errors.

Alas, nobody knows how to estimate non-sampling errors. Sometimes the best thing that a pollster can do is to make sure that it has tried to reduce or avoid the potential errors rather than estimate the impact of these errors.

Even though some statisticians have introduced the '€œtotal error'€ concept, which combines sampling errors (a margin of error) with non-sampling errors, so far there is no widely acceptable statistical formula that is able to take non-sampling errors into account. So we cannot solely rely on a margin of error to assess survey/quick counts quality as it has proved to be a spurious factor in determining opinion poll quality.

Prior presidential and legislative elections have shown that quick count results were not far different from the actual results '€” something that we have all easily predicted through a margin of error '€” but we must remember that this presidential election is completely different.

Both candidates are rather alike in their dominance of the surveys, so non-sampling errors made by the pollsters will introduce significantly different results. And different quick count results can dangerously lead to confusion or public outcries. Regrettably, both candidates have unwisely claimed victory on the basis of inadequate quick counts.

Secondly, we must consider that carrying out a massive research survey is not cheap. A pollster needs a significant amount of money to fund its research, so that is why a financier is needed. Common research ethics obligates researchers to acknowledge the person and/or company that funds their research to avoid conflicts of interest. This should have been done by the pollsters.

Transparency in research funding is an incredibly important matter in carrying out ethical survey research, but sadly, we are not yet aware of it.

Therefore, it is not surprising at all that some people accuse the pollsters of being mere money-oriented research groups.

Having learned from the recent presidential election, we need to design a system that regulates these pollsters, so they will be more responsible in carrying out and publishing their research.

Transparency should be the first and foremost principle in extracting opinion polls.

Pollsters should be transparent in regards to sampling-population proportions, sampling techniques, social-demographic descriptions of survey participants, questionnaire/interview design, margin of error, the wording of the questions and most importantly their financiers.

The Indonesian Institute of Sciences (LIPI) can hopefully be a watchdog for these pollsters by accrediting them, as most people will not focus on all information given by the pollsters, except their opinion polls or quick count results.

The writer is a social research graduate at the University of Edinburgh, Scotland, and is a social psychology lecturer in the Faculty of Psychology, Airlangga University, Surabaya.