The many unknown aspects of the disease, in combination with political and economic factors, weak leadership and asymptomatic air-borne infectiousness, enabled COVID-19 to grow into a public health emergency of international concern – the World Health Organization’s highest level of alarm – in the month following China’s formal report of the illness to the WHO. 

Medical research is critical to understanding the essential characteristics of a new disease in order to prevent, treat and contain it. The ability to generate a vast body of evidence alone is not sufficient. The evidence must be unbiased, credible and analyzed in a scientifically robust and practically useful way.

COVID-19 galvanized researchers worldwide and many switched their focus to it. Ultimately, a significant body of data was generated, but much of it was either flawed, irrelevant or inadequately translated to practice. Resources and, critically, time were thus inefficiently spent as the health crisis unfolded in 2020. The scientific responses to such crises can be made more dynamic, critical and open to rapid evidence-driven adjustments.

Several examples highlight the power of scientifically robust evidence as a tool to help curb the COVID-19 pandemic. Vaccines are a prime example. Existing research into coronaviruses in combination with novel vaccine technology platforms helped accelerate vaccine development, and today at least 11 coronavirus vaccines are in use globally.

We applaud the Indonesian government’s timely research response, which prioritized resources to support critical research activities, along with incubating biotechnical innovation. However, this pandemic has also exposed some vulnerabilities in the country’s research infrastructure and practices.

While high-income countries (HIC) have contributed most of the scientific evidence, evidence generated by/in collaboration with low- and middle-income countries (LMIC), including Indonesia, will help identify responses relevant to less well-resourced settings.

Initiatives within Indonesia have been in place to help harmonize efforts from various stakeholders in academic, industrial, domestic and international institutions in controlling the pandemic, but there have been dubious investigations conducted, funded or promoted by the government. In fact, despite extensive criticism by many scientists, some "innovations" have been authorized and implemented despite poor methodological rigor and a lack of transparency.

Transparency is essential to ensuring that scientific evidence is the soul and body of evidence-based clinical medicine and public health interventions. That evidence is especially critical when working in uncharted territory, such as the COVID-19 pandemic.

Last year, the Food and Drug Monitoring Agency (BPOM) found critical problems in the design of a drug cocktail trial conducted by researchers at Airlangga University in partnership with the State Intelligence Agency (BIN) and the Indonesian Military (TNI). Despite the researchers' claim of the drug combination’s effectiveness, BPOM determined that the trial was not of sufficient quality to support authorization for clinical use.

Concerns emerged surrounding the randomization process used to achieve comparability between intervention arms, which is critical to demonstrating a cause-effect relationship of an intervention with health outcomes. It is questionable how a late-phase randomized trial involving 754 participants was allowed to proceed with inadequate methodological and statistical protocols in place.

While it is without a doubt that ethical clearance and informed consent were sought and obtained, whether the trial was ethical may be questioned by some. If a trial either uses an unnecessary excess of subjects or its design cannot yield valid findings, its conduct cannot be considered ethical.

Another example is the so-called Nusantara vaccine. BPOM did not recommend the vaccine to continue to phase 2 human trials due to, among others, noncompliance with good clinical practice and good manufacturing practice standards by the investigators and manufacturer. In addition, a lack of transparency reflected by the failure to publish a detailed protocol was highlighted.

The investigators somehow managed to continue experimenting under the umbrella of research-based service regulated and supervised by the Health Ministry. The exact definition of such “service” remains vague and the regulations that govern it are unclear.

Regarding the screening and diagnostic initiatives, we have witnessed the hasty implementation of the GeNose screening test in the general population (e.g. at airports and train stations), while the principal investigator himself has acknowledged that independent validation studies in this target population have not been completed and existing results are based solely on hospital outpatients. No findings have yet been published in a peer-reviewed journal.

In fact, the most comprehensive systematic review to date on prediction models for the diagnosis and prognosis of COVID-19 indicates that almost all published prediction models are at high risk of bias and poorly reported. When validated, such studies tended to show poorer predictive performance indeed.

Despite all this, GeNose has been cleared for use by the Health Ministry and has been installed as a screening instrument for train passengers in many cities in the country. The urgency of this need is not debated, but unproven diagnostics put into wide use risk exacerbating the crisis.

Thankfully, the BPOM’s professionalism and courage in protecting us from dubious products has been steadfast. The weakness is the poor scientific culture that brings unqualified research teams and products to the BPOM. The real danger is in the vested use of political influence to push those products forward by maneuvering around those objective professionals. 

We recommend the use of available risk of bias tools to evaluate the validity of study results. There is also value in consulting research methodologists to judge the robustness of study designs. The WHO has published a coordinated global research road map to tackle COVID-19 that can help prioritize important research questions. Researchers and ethics committees need to work closely with experts in the field to gain insights on whether the research results will be applicable in daily practice.

Finally, equipping the ethics committee with expertise in study designs to be able to judge if a proposal is scientifically sound – and therefore ethical – is crucial. To ensure transparency, the decision making process should be made available publicly.

Researchers must agree to abide by the consensus in the standard reporting of medical research and adhere to best practices. These can be enforced by including this in the fund contract.

Despite all the challenges, we view this pandemic as a wake-up call and an opportunity to evaluate and reform the biomedical research ecosystem as an invaluable investment in the health and prosperity of Indonesians.

 ***

Ihsan Fadilah is an applied statistician at the Eijkman-Oxford Clinical Research Unit, Jakarta, and Kartika Saraswati is a research physician currently pursuing her doctorate in clinical medicine at the Nuffield Department of Medicine, University of Oxford, the United Kingdom. Lira Mirandus, a general physician currently pursuing his master's degree in infection and immunity at the University College London, and Ines Atmosukarto, chief scientific officer and managing director of Lipotek Pty Ltd, Canberra, contributed to the piece.