B

efore the recent election, nearly all survey outfits had predicted a win for the Indonesian Democratic Party of Struggle (PDI-P), with a doubling of its vote if Joko '€œJokowi'€ Widodo was nominated, Islamic parties as a group would see their vote decline and the Prosperous Justice Party (PKS) would lose the most because of the corruption scandals. However, the survey organizations'€™ own quick-count results indicate the surveys were only right on the PDI-P topping the polls, but they were widely and wildly off the mark, and by a sizable percentage, on Jokowi'€™s coattail effect and the fate of the Islamic parties and the PKS.

The discrepancy between the results of their surveys and their quick counts can be easily explained by the fact that in the quick counts (or exit polls), the voters were real, warm-bodied voters while in the surveys, the surveyed '€œvoters'€ were hypothetical (who might or might not have voted in the real election).

But the real culprit for the mixed prediction results may lie elsewhere.

The main source of flawed predictions in any survey can usually be traced to sampling errors but the above errors came from faulty assumptions about Indonesian political culture.

Election prediction is a hit-and-miss business even in homogeneous societies, let alone in Indonesia, an archipelago of some 17,000 islands and 300 dialects, with each island varying in statistics on population distribution, economic development, poverty, education and even beliefs.

To think one can take a representative sample out of these is to ignore the facts on the ground and to assume too much, especially if the standard sample size used in most surveys is only around 2,000, certainly much too small for a population close to 300 million scattered along a huge territorial spread consisting of 70 percent water.

Even in America (the pioneering country of public opinion surveys and with a relatively homogeneous population where rural-urban or other socioeconomic and regional splits are less prominent), survey results over the years have been touch-and-go with quite a few surprises.

In Indonesia, all the above differentials are severe: poor farmers or poor fishermen in remote areas or day laborers (or itinerant and migrant workers) in urban areas are as night-and-day compared to their rich, well-heeled upper and middle-class counterparts in urban centers. These latter groups are well-endowed (with two or three modern communication gadgets in their hands and pockets) and have knowledge and opinions about everything including politics, so they tend to get included in sampling easily, resulting in their being over-represented.

But come election day, they are less likely to vote; they are more apathetic and cynical about politics and the electoral process. In contrast, their poor counterparts are less aware politically and may not even have opinions per se, let alone '€œcorrect'€ ones. And, by virtue of their poverty and inaccessibility (coupled with the disjointed communication system in the country), they are less likely to be polled or to be included in sampling, resulting in their being under-represented.

But, come election time, they come out to vote in droves because going to the polls for them is like going to a festival. This is the pesta demokrasi, a unique feature of Indonesian democracy that also explains the low levels of election violence and high turnover. To the rural and urban poor, voting is a real civic duty which gives them a sense of pride, identity and self-worth.

Both urban over-representation and rural and urban poor under-representation constitute the Achilles'€™ heel of opinion surveys because they create a biased sample, leading to biased and even incorrect conclusions.

This is because sampling works on the '€œwisdom of the crowd'€ theory: it is hoped that while one answer may be wrong, put together, all the wrong and right answers can somehow produce the right
results.

While this can happen (because of the effects of the bell curve and the Law of Large Numbers), the unknown in the process is how large is large enough. Including a margin of error (no matter how small) shields the survey results somewhat but still the size of the margin of error is essentially a guessing game.

There is also what is known as the level-of-analysis problem and the error of anthropomorphizing in surveys; they interview a few (sample) people then they call them a group and then they add or attach all the characteristics and attributes of one unit to the group (or the whole is equated as the sum total of its part fallacy).

The fact of the matter is a unit, or a voter, is a living person with perceptions, emotions and beliefs; the concept of '€œIndonesian voters'€ (as a whole) has no validity and yet survey people and observers routinely use the term '€” as when they often say, '€œIndonesian voters will punish parties for their indiscretions'€ when in fact each voter goes to the booth for different reasons or even for reverse motivations.

But even the most cohesive of groups cannot do such a thing (i.e., '€œpunish'€ anything), only individuals can. By giving a '€œsoul'€ to a collectivity that in fact has none, the errors of anthropomorphism and mixing up levels of analysis are committed in one go; they collect their information on the characteristics of individual voters but they extrapolate or conclude at a group or general-voter level, resulting in generalization or stereotyping.

These weaknesses notwithstanding, opinion surveys have an important role to play because ascertaining public opinion is part of the working of democracy and mature national politics, especially as Indonesians have come to rely on them (especially the quick counts) to satisfy their thirst for same-day election results, a feat not many democracies in the world can achieve.

But this privileged position enjoyed by survey institutes must not be considered a license to be sloppy about their findings. Because there have been charges of data manipulation and innuendos about made-to-order survey results in the past, the survey institutes, like all other organs of democracy, must be open and transparent '€” not only about their methodologies but also their finance and funding sources '€” so the '€œhazards'€ of election prediction can be kept to a minimum and the public can take their predictions with the correct grain of salt.

The writer is a retired international civil servant.