Data gaps towards health development goals, 47 low- and middle-income countries

Abstract Objective To assess the availability and gaps in data for measuring progress towards health-related sustainable development goals and other targets in selected low- and middle-income countries. Methods We used 14 international population surveys to evaluate the health data systems in the 47 least developed countries over the years 2015–2020. We reviewed the survey instruments to determine whether they contained tools that could be used to measure 46 health-related indicators defined by the World Health Organization. We recorded the number of countries with data available on the indicators from these surveys. Findings Twenty-seven indicators were measurable by the surveys we identified. The two health emergency indicators were not measurable by current surveys. The percentage of countries that used surveys to collect data over 2015–2020 were lowest for tuberculosis (2/47; 4.3%), hepatitis B (3/47; 6.4%), human immunodeficiency virus (11/47; 23.4%), child development status and child abuse (both 13/47; 27.7%), compared with safe drinking water (37/47; 78.7%) and births attended by skilled health personnel (36/47; 76.6%). Nineteen countries collected data on 21 or more indicators over 2015–2020 while nine collected data on no indicators; over 2018–2020 these numbers reduced to six and 20, respectively. Conclusion Examining selected international surveys provided a quick summary of health data available in the 47 least developed countries. We found major gaps in health data due to long survey cycles and lack of appropriate survey instruments. Novel indicators and survey instruments would be needed to track the fast-changing situation of health emergencies.


Introduction
The United Nations (UN) Transforming our world: the 2030 agenda for sustainable development is a global plan shared by Member States for a far healthier, more prosperous world. 1,2 Fifty-seven of the 232 UN sustainable development goal (SDG) indicators were subsequently identified as health-related indicators by the World Health Organization (WHO) in 2019 with two more added afterwards. 3,4 In alignment with the SDGs, WHO Member States approved the Thirteenth general programme of work 2019−2023 in 2018 and set the triple billion targets to accelerate delivering impact in countries. 5 The results framework of the general programme of work identified 46 health outcome indicators to track countries' progress towards the triple billion targets, 6 including 39 SDG indicators and seven non-SDG indicators. 7 The seven non-SDG indicators were approved in World Health Assembly (WHA) resolutions including two that relate to health emergencies. These 46 indicators assist WHO Member States to stay focused and accelerate their progress towards achieving the SDGs and the triple billion targets.
The recommendations of the thirteenth general programme of work would require gathering data in many dimensions, yet little is known about the efforts needed by countries to measure these 46 indicators. Researchers have estimated that 12 data systems would be required in each country to monitor 57 health-related SDG indicators. 3 A similar number of systems would likely be needed to measure the 46 indicators. Representative household surveys, civil registration and vital statistics, and other administrative health data systems are the primary and preferred data source for monitoring health indicators. 8 Regular population-based health surveys are a key component of well-functioning data systems for surveying populations and health risks. 9 Household surveys can provide data for an estimated 29 of the 57 health-related indicators. 3 Additionally, valid measurement of some indicators, such as out-of-pocket health expenditure, are mostly achieved through surveys. Many countries, however, especially lowand middle-income countries, lack the technical capacity and financial resources to perform these surveys, and have relied on development partners to implement surveys. 10 The WHO SCORE (survey, count, optimize, review, enable) global report in 2020 revealed that most low-income Health data gaps for SDGs Luhua Zhao et al.
countries relied heavily on external funding for surveys, with only 3% of the surveys fully funded by a government. 9 International surveys provided essential health data and this was sometimes the only data available in these countries to monitor some important health trends. Reviewing the survey methods and procedures in low-and middle-income countries would offer valuable insight into whether and to what extent key health data, measurable by the 46 indicators, were available in these countries. Our literature search showed that not much work has been done in this area. The UN Intersecretariat Working Group on Household Surveys conducted a review to map the SDG global indicators to household surveys. 11 The report found that 77 of the SDG indicators could be sourced from household surveys; however, the availability of the data was not assessed.
We therefore aimed to assess the availability and gaps in the relevant data and to explore new methods for improving health data in low-and middle-income countries. Our study focused on reviewing international surveys that have been implemented in these countries. We reviewed each survey to determine if it could measure any of the 46 health outcome indicators defined by the WHO. We also reviewed the implementation of the surveys between the years 2015 and 2020 in 47 low-and middle-income countries that are designated by the UN as the least developed countries. 12 These countries lack adequate local capacity, and depend heavily on surveys sponsored by international partners to provide essential health data. We therefore found it feasible to assess their health data systems through examining the international surveys used. The results provide evidence on countries' progress towards the health-related SDGs. We also propose exploring new methods to reduce the data gaps through surveys, particularly for timely and rapid data collection in health emergency settings.

Methods
We identified potential international population surveys based on a literature review and consultation with experts across WHO technical programmes. We included surveys in our study if they covered health topics; were coordinated or facilitated by international agencies or partners; had been implemented in multiple least developed countries since 2000 using nationally representative samples; and had a core questionnaire or instrument and a standard protocol. We included surveys concerning human immunodeficiency virus (HIV), tuberculosis, malaria and hepatitis B only if rapid diagnosis tests or laboratory testing were used in the surveys in addition to data collected through oral interviews.
For each type of survey, we conducted an extensive online review to identify standard and country-specific survey instruments. We examined the instruments to determine whether they contained tools (such as survey questions, health examinations or collection of biomarkers) that could potentially be used to measure any of the 46 health outcome indicators. When the country-specific survey instrument was not available, we used the standard international questionnaire for assessment. We categorized each indicator into one of three groups according to its measurability: (i) measurable (if the surveys we studied contained suitable questions, health examinations or biomarkers); (ii) potentially measurable (if the surveys we studied did not contain suitable questions, health examinations or biomarkers but such tools could be formulated and added); and (iii) not measurable (if suitable questions, health examinations or biomarkers could not be formulated for any surveys). If surveys containing tools to measure an indicator were conducted in a country between 2015 and 2020, we assumed that data were available for this indicator in this country for this period. We judged that an indicator was not measurable by surveys if its calculation involved important information unavailable from surveys. For SDG 3.4 (incidence of HIV, tuberculosis, malaria and hepatitis B infections), for example, we judged that surveys needed to include rapid diagnostic tests or laboratory testing to measure the indicators.
For all 47 least developed countries 12 we assessed the availability of data for all indicators that were deemed measurable by the surveys we studied. We did this by mapping the indicators each survey can measure. We first assessed data availability and the number of indicators measured by the surveys for the 47 countries for the period of 2015-2020. We conducted a second analysis using the period 2018-2020 to evaluate the impact of survey frequency on availability of health data in these countries. For each indicator, we recorded the number of countries that could measure it and then calculated the percentage out of the 47 countries.

Results
We identified 14 different surveys that could measure at least one of the 46 health outcome indicators from 2015 to 2020 (Table 1). The surveys included comprehensive multi-topic surveys such as demographic and health surveys (DHS), labour force surveys, living standards measurement surveys and multiple indicator cluster surveys (MICS), as well as single-topic surveys such as household income and expenditure survey and reproductive health surveys. More details of these surveys are provided in the authors' data repository. 13 DHS were able to measure the greatest number of indicators (22 indicators), followed by MICS (14 indicators), living standards measurement surveys (11 indicators), reproductive health surveys (eight indicators), household income and expenditure surveys (five indicators) and labour force surveys (five indicators). There were major overlaps in survey contents among them. Nine indicators measured by DHS could also be measured by MICS and living standards measurement surveys. Out of 14 indicators measurable by MICS, 11 indicators could also be measured by DHS. Similarly, 10 of 11 indicators that are measured by living standards measurement surveys are also measured by DHS.
We found that 27 indicators were measurable by the selected surveys. The percentages of countries that used the surveys to measure each indicator between 2015 and 2020 are listed in Thirteen indicators were not measurable by surveys. These indicators fell into six groups: (i) indicators that concern health facility and health workers, including SDG 3.b.3 (health facility operation) and SDG 3.c.1 (health workforce); (ii) indicators that involve government policies, including WHA 66.10 (trans-fats regulation), SDG 3.d.1 (International Health Regulations compliance) and SDG 1.a.2 (government spending); (iii) indicators that involve sophisticated testing and diagnosis, including WHA 68.3 (poliomyelitis) and WHA 67.25 (antimicrobial resistance); (iv) indicators that are usually reported through civil registration and vital statistics and administrative data systems, including SDG 1.5.1 (death from disasters) and SDG 11.6.2 (fine particulate matter); (v) indicators that involve complex calculation algorithms, including SDG 3.9.1 (mortality rate attributable to household and ambient environment); and (vi) composite indicators that involve multiple tracer variables, including SDG 3.8.1 (coverage of universal health care) and the two indicators that are related to readiness and response to health emergencies.
The remaining six indicators were potentially measurable by surveys, but were not measured by the surveys we studied. These indicators were SDG 3.3.5 (intervention against neglected tropical diseases), SDG 3.5.1 (treatment intervention for substance abuse disorder), WHA 68.7 (antibiotic consumption level) and SDG 3.4.1, SDG 3.9.2 and SDG 3.9.3 (mortality indicators).
Our results also provided information to assess how the 47 countries fared overall in tracking public health trends. Over the period 2015-2020, 19 of the countries collected data on 21 or more of the 46 health outcome indicators and 16 countries had data on 11 to 20 indicators; nine countries collected no data, of which five countries were from WHO African Region, three from Western Pacific Region and one from Eastern Mediterranean Region (Table 2). When we analysed the time period 2018-2020, the number of countries with data on 21 or more indicators dropped to six, and countries with no data increased to 20, across five WHO regions.

Discussion
As WHO Member States are mandated to track and report their progress at country level towards SDGs and triple billion targets, 5 it is essential that they collect and use health data to identify the key priority areas for improvement. Well-functioning health information systems, particularly civil registration and vital statistics and other administrative data systems, were lacking in the 47 least developed countries. 12 WHO's SCORE global report estimated that 40% of annual deaths went unregistered globally, particularly in Africa. 9 Many low-and middle-income countries, particularly the least developed countries, relied on international surveys as the primary source for many essential health data. We were therefore able to assess the key health data gaps in the studied countries by examining international surveys. In conjunction with the 46 health outcome indicators identified by WHO, international surveys can provide a relatively simple and quick way to assess gaps in essential health data in these countries.
We found data gaps in the 47 countries studied. None of the countries had data for all 27 indicators measurable by surveys between 2015 and 2020. The most monitored indicators were measured in about three quarters of countries. The least tracked indicators were those for infectious diseases, particularly tuberculosis and hepatitis B, each of which were reported by 4.3% and 6.4% of the countries, respectively. This result is expected given that the diagno-sis of these health conditions requires laboratory work that was not provided by most surveys. Furthermore, less than one third of the countries had data to monitor child development status and child abuse, a sign that more investment is needed to track children's welfare.
The pandemic of coronavirus disease 2019 (COVID-19) has highlighted the urgency of timely and accurate measurement of population mortality, as well as the need for rapid collection of key data for informing policies and actions. Household surveys are one of the major sources for mortality data in countries that do not have reliable civil registration and vital statistics systems. However, surveys often lack instruments to obtain mortality statistics. Five of the 10 mortality indicators identified by the WHO thirteenth general programme of work are either not measurable by surveys or potentially measurable but not measured by the surveys we studied. Sibling survival history -a method commonly used in surveys to measure mortality -only provides all-cause and pregnancy-related mortality, and tends to produce sparse data among older adults. Researchers have tested additional questions for sibling survival history to determine HIV/AIDS mortality, 15 while others have explored the idea of a social network survival method to capture death data more efficiently. 16 Further research is needed to develop more innovative methods to enhance the capacity of surveys for rapid measurement of population mortality.
The COVID-19 pandemic also showed the need to rapidly assess the local situation during health emergencies. Health emergency readiness is usually measured through reports based on the International Health Regulations, 17,18 which relies on government reporting prepared in advance and does not provide up-to-date information. Surveys can be used to obtain quick results. However, out of the 46 indicators studied, only two indicators were directly related to health emergencies and neither were measurable directly by surveys. Developing suitable indicators for population-based surveys is needed to track and monitor the rapidly changing situation during health emergencies.
Heavy reliance on a few surveys can have negative consequences. Comprehensive population surveys such as DHS and MICS take significant time, manpower and financial invest-Health data gaps for SDGs Luhua Zhao et al.
ment to implement, and are thus usually conducted every 5 years or more. 11 Countries relying on these surveys may find themselves without data to monitor important health trends between survey cycles, which can be affected by unexpected events such as pandemics. In the current study, the number of countries with no data for the studied indicators rose from nine to 20 when the time period studied was shortened from 2015-2020 to 2018-2020, showing the risk of dependence on a small number of surveys. Our analyses revealed that the difference was primarily due to the scheduling of DHS and MICS. Greater efforts are needed to find new ways of filling the data gaps between these surveys. Moreover, most health surveys still rely on face-to-face interviews for data collection. As the COVID-19 pandemic has shown, this method is often not feasible due to safety concerns and travel restrictions imposed by authorities. Many countries have responded by adopting mobile phone surveys and web-based surveys as alternatives, 19 even though guidelines have been developed to restore the capacity to run in-person interviews. Adapting to the challenges posed by COVID-19 would help not only to collect data that is urgently needed for responding to the pandemic but also to prepare surveys for other emergencies and rapid responses. Mobile phone surveys are suitable for this purpose as they can collect data rapidly with lower costs. Such surveys can also be used to quickly fill the specific data gaps in combination with prior health data evaluations, such as the assessment introduced in our study and WHO SCORE technical package. 9,20 International regulations need to be developed to reduce unnecessary ethical review requirements for non-sensitive surveys and minimize restrictions on mobile network access. Such barriers can offset the speediness of mobile phone surveys.
Data that are comparable across countries and consistent over time are important not only for national planning and evaluation of health systems but also for global assessment and progress tracking. Previous studies have indicated poor correlation of data on SDG indicators across various reports due to differences in indicator definitions, data sources, data processing and methods of synthesis. 21,22 International surveys may adopt indicator definitions that are not comparable with those approved  23 whereas measuring progress on SDG 3.3.1 requires HIV indicators for broader populations disaggregated by sex, age group and other categories. This inconsistency would likely represent a challenge for tracking progress towards the SDG and triple billion targets in countries that rely on these data. Despite the efforts among international partners to coordinate the monitoring of health trends, greater collaboration is needed to increase data comparability and close data gaps.
This study has a few limitations. First, we did not include important data sources such as civil registration and vital statistics and administrative data sources. However, the impact on our findings should be minimal as most of the indicators examined rely on surveys as the primary data source, and civil registration and vital statistics systems are mostly too poorly developed in the 47 least developed countries to play a key role. Second, when details were not available online for some countries, we used core questionnaires from the same survey series for assessment. This approach could cause potential discrepancies for countries with tailored questionnaires. Last, low data quality can render some data unusable. However, as data quality varies across surveys and can be influenced by many factors that are difficult to measure, this issue, although important, was not included in the discussion.