The pandemic has caused significant disruption of business processes and employment. It has also revealed our reliance on technology to run, manage and organize our work. Telecommuting, for instance, enables us to work remotely from home amid the need to enforce social distancing to slow down the spread of the virus.

The same applies to the task of producing official statistics. The disruption here is particularly intense in so far as, in the Indonesian context, producing government statistics relies on traditional surveys. In conventional surveys, data is largely collected through face-to-face interviews or field observation.

At present, a considerable portion of socioeconomic data produced by Statistics Indonesia (BPS) is obtained by running massive traditional surveys on a regular bases, such as the National Socioeconomic Survey (Susenas) for poverty estimation, the National Labor Force Survey for unemployment estimation, the area sampling frame to estimate the harvested area of paddy and the Price Survey for the monthly computation of inflation.

The Susenas and labor force surveys are household-based surveys that need face-to-face interviews to gather information. That means there are no data without knocking on doors.

The need to enforce social distancing and working from home poses a tough challenge for those surveys in the coming months.

The response rate will be meager, which would drag down the accuracy of estimation substantially. The worst scenario is that no data is available at all.

Yet the availability of statistics to measure the socioeconomic impacts of the pandemic is critical. There is an urgent need for statistics that allow us to make so-called nowcasting, a prediction of the present, the very near future and the very recent past for the sake of mitigation measures.

The Australian Bureau of Statistics (ABS) has provided an excellent example in this regard. It has produced retail turnover and an interactive map of employment impacts of COVID-19 in Australia.

This pandemic starkly shows that reliance on traditional surveys and conventional methods of collecting data must be lessened in the future.

Although it is compelling for capturing the facts for post-analysis, one limitation of traditional surveys is the time lag between the data collection and dissemination of the results, which is sometimes too long. As a consequence, the information provided tends to be outdated.

To reduce that reliance, innovative methodologies and cost-effective technology for data collection are indispensable. Moreover, the use of other data sources, such as big data and administrative data, must be improved.

The rapid development of big data in recent years can be considered a disruptive innovation in the production of official statistics by BPS, which still relies on traditional surveys and censuses to collect data.

Big data, which pertains to large amounts of data produced very quickly by a high number of diverse resources, offers new sources of data that were not available before.

Big data could address some of the challenges for official statistics. Those challenges include respondent burdens, the high cost of traditional censuses and surveys, the need for a small area analysis and timely statistics to do in-process monitoring and nowcasting instead of a postevent analysis.

The last issue requires a high frequency in producing statistics, which is almost impossible with traditional surveys. Big data can potentially complement, replace, improve and add to official statistics.

However, new technologies and innovative methodologies, as well as data science expertise, are required to take advantage of big data as a new data source. On top of that, a governance system (policies, rules and monitoring) must be put in place. This could ensure that some critical issues around the use of big data for official statistics, such as ethics, legality, confidentiality, security, continuity and quality assurance, are well managed.

Some success stories in using big data for official statistics have proven that harvesting tremendous opportunities provided by this technology, as insisted by President Joko “Jokowi” Widodo, is possible and should be the future of official statistics.

In constructing the Consumer Price Index to calculate the inflation rate, for instance, ABS has used big data to monitor the prices of one-quarter of its commodities basket. Statistics Canada has succeeded in using satellite data for crop monthly estimations.

In Indonesia, active collaboration between BPS and Telkomsel has produced inbound tourism statistics in border areas by using mobile positioning data (MPD) since 2016.

Earlier, BPS had relied on immigration records and border surveys. The latter is very costly and cannot be done at all unattended border gates. The innovation has reduced both the workload and the burden on respondents. Moreover, the use of MPD has been expanded to domestic tourism and population statistics on migration and commuting.

Unfortunately, the use of big data for official statistics in Indonesia is still in its infancy. Hopefully, this devastating pandemic will at least trigger the use of big data and other innovative methodologies for official statistics.

***

Head of evaluation and reports of food crops at Statistics Indonesia (BPS). The opinions expressed are his own.