Reporting back to communities: Accountability to populations in research practice – A case study from Timor-Leste

How can we strengthen accountability to population as part of our research work? In Timor-Leste, an evaluation commissioned by the Ministry of Health and supported by UNICEF integrated an innovative process of reporting back on the evaluation results to the communities that had been surveyed.

In July and August 2021, the enumerators went back to 18 locations where focus group discussions had taken place, in 6 municipalities, in order to report back to communities. Three locations of one municipality were not accessible, due to the lack of transportation options to that municipality at that time.

Flexibility was key

The plan to report back to communities about the research results was not in the original terms of reference of the evaluation, developed by UNICEF with the Ministry of Health, but was suggested, in a broad way, in the proposal received from the institution that was selected for carrying out the evaluation, FH Designs, under the ethical consideration section: “We believe that it is important that individuals who give up their time to assist in this evaluation learn of the results and the uses of the evaluation. To that end, we will work with the evaluation management team to ensure that summary results are delivered back to aldeias in a form that local residents can access.”

How exactly to do it was refined later, through discussions between UNICEF and the lead evaluator, who agreed that the team of local enumerators employed FH Designs would visit the communities again, once the evaluation was completed. This was an additional activity, designed after the signature of the evaluation contract between UNICEF and FH Designs, and the flexibility of FH Designs was key here, as the company was willing to pay a few more days of work to the local enumerators. The availability of flexible funds, thanks to support from the UNICEF East Asia and Pacific Regional Office, also helped make this process possible, as it enabled UNICEF to cover the extra transportation cost.

A young, local team

Another important factor had to do with the composition of the evaluation research team. The foreign lead evaluator worked with a local researcher and  12 Timorese research assistants / enumerators, most of them young university graduates. The nationality and age of the research assistants helped – they would remain in Timor after data collection was complete, and they were not seasoned researchers who would be in high demand, thus they were available for this extra work that happened several months after the evaluation was concluded. The lead evaluator and local researcher maintained contact with the team through a dedicated WhatsApp channel while the feedback was being completed.

Reporting back – but what?

The question of what to report back to the communities was an interesting one, in the sense that giving them a copy of the evaluation report would not have been very effective. Timor-Leste is a country with an oral culture, literacy may be limited, particularly in rural communities, and the evaluation report was a complex document designed to provide recommendations and evidence to inform policy. It was important that the research team could go back with a product that would make sense and would be useful to the communities.

The content that was handed back to them was a short document that the research team presented orally in local languages and also gave as a hard copy in the national language. That document included some of the evaluation findings, using plain language. For example, one of the evaluation findings was rephrased as follows: “We are very concerned the young children are allowed to poo on the ground and it is usually left for the dogs or pigs. The reason that we are worried is that, even though the children are healthy, their poo has a lot of diseases in it and flies will carry those diseases to people.”

The document also highlighted some interesting, innovative practices that the evaluation team had documented during their field work, and that could be useful to other communities. One such practical tip was that “pit latrines can be stopped from smelling bad by, every day, adding a handful of ash and a handful of fresh cut up papaya leaves and a handful of either dry grass or dry leaves or dry rice husks or dry corn husks.”

The team also came with printed copies of the photos that had been taken during the research, and this brought a lot of joy. In Timor-Leste, group photos are very popular. Rural communities may be visited by people coming from the capital to get information, including visual information, but community members rarely get to see the photos, let alone have a hard copy of it, so this gesture was very much appreciated.

The challenge of reaching communities after a research

It proved more difficult than had been anticipated to give back to the exact people who had given time to the research through focus group discussions or the household survey. When the research team went back to the communities, in most of the areas they found out that the groups of people that they had interviewed were not available, as they were busy with agricultural work. Instead, they reported back to local leaders (village chiefs or sub-village chiefs), who had also provided data for the evaluation. The huge majority of local leaders appreciated hearing about the evaluation findings. Some said they would keep the evaluation results and photos in their offices as important documents, another that he would stick the photos and summary of the report on the village townhall’s wall where everyone could see and read them.

Taking accountability one step further

Ideally, we should not only report back on the findings of the research but also on the impact of the research – in other words, on whether it has been used, and more importantly on what decisions and actions it has triggered.

One way to put this into practice would be to report back once more, at a later stage, to share what happened since the research was published. From a logistical standpoint, it may be challenging, as the teams that were involved in the research, including the UNICEF staff, may no longer be there, or be available, and if they are would probably have other priorities. But it would also provide an incentive, for UNICEF and its partners, particularly governmental ones, to put extra attention on the use of research, so as to have positive outcomes to report to the communities who donated their time during the research, foregoing income-generating and other critical activities to contribute to the evidence generation, and hopefully social change.