Logo of joghLink to Publisher's site
J Glob Health. 2012 Jun; 2(1): 010304.
PMCID: PMC3484764

Setting priorities for development of emerging interventions against childhood pneumonia, meningitis and influenza

Acute lower respiratory infections, which broadly include pneumonia and bronchiolitis, are still the leading cause of childhood mortality. ALRI contributed to 18% of all deaths in children younger than five years of age in 2008 [1], and the main pathogens responsible for high mortality were Streptococcus pneumoniae, Haemophilus influenzae and respiratory syncytial virus [2-4]. In addition, meningitis was estimated to contribute up to 200 000 deaths each year, and influenza anywhere between 25 000 and 110 000 [1,5]. It is widely acknowledged that a major portion of this mortality should be avoidable if universal coverage of all known effective interventions could be achieved. However, some evaluations of the implementation of World Health Organization’s (WHO) Integrated Management of Childhood Illness (IMCI) strategy, which promotes improved access to a trained health provider who can administer “standard case management”, have shown somewhat disappointing results [6-8]. Only a minority of all children with life-threatening episodes of pneumonia, meningitis and influenza in developing countries have access to trained health providers and receive appropriate treatment [6-8]. Thus, novel strategies for control of pneumonia that balance investments in scaling up of existing interventions and the development of novel approaches, technologies and ideas are clearly needed.

Emerging interventions against childhood pneumonia, meningitis and influenza

Several recent studies quantified the burden of child mortality due to childhood infections [1] and sub-divided it further according to the causing infectious pathogens [2-5]. In a series of papers that followed, we systematically reviewed the available information relevant to the emerging interventions against childhood pneumonia, meningitis and influenza [9-14]. We defined the list of emerging interventions of interest as follows: (i) the first set of emerging interventions was suggested by the officers from the Bill and Melinda Gates Foundation (BMGF) and it was based on strategic priorities that were being discussed at the Foundation in the year 2009; (ii) additional ideas were proposed by our team at the University of Edinburgh, after provisionally reviewing the literature on emerging interventions against childhood infections; (iii) the third set of emerging interventions was suggested by the 20 international experts invited to take part in the CHNRI expert panel meeting (see later). We eventually agreed to evaluate 29 emerging interventions that seemed feasible for reaching the implementation within a 10-year period (Table 1). We aimed to be inclusive and open-minded in their selection because some of them may still be far from implementation.

Table 1
The consolidated list of 29 emerging interventions against childhood pneumonia, meningitis and influenza

The expert opinion exercise

The CHNRI methodology for priority setting in health research (and technologies) investments was proposed as a systematic tool that can be used by those who develop research policy and/or invest in health research [15-18]. It should assist them to understand (i) the full spectrum of research investment options; (ii) the potential risks and benefits that can result from investments in different research options; and (iii) the likelihood of achieving reductions of persisting burden of disease and disability through investments in health research and health technologies. The CHNRI methodology has 3 stages: input from investors/policy-makers (who define the context and criteria for priority setting); input from technical experts (who propose, list in a systematic way, and then score different research investment options against a pre-defined set of criteria); and input from other stakeholders (weighing the criteria according to wider societal system of values). The method has been described in detail elsewhere and many examples of its implementation are publically available [19-22].

The expert opinion exercise focused only on emerging interventions and a broad, long-term (downstream) context/vision. We invited 20 leading international experts from international agencies, industry, basic science and public health research to Dubrovnik, Croatia, in September 2009. The invited experts provided opinion on how the 29 chosen emerging interventions satisfy a number of criteria relevant to prioritization of support to emerging interventions against childhood infections. Based on a modified CHNRI’s conceptual framework, 12 criteria for prioritization were developed for emerging interventions: (i) answerability (in an ethical way); (ii) low development cost; (iii) low product cost; (iv) low implementation cost; (v) likelihood of efficacy and effectiveness; (vi) likelihood of deliverability; (vii) likelihood of affordability; (viii) likelihood of sustainability; (ix) maximum potential impact on mortality burden reduction; (x) likelihood of acceptability to health workers; (xi) likelihood of acceptability to end users; (xii) predicted impact on equity. Further details about the modified CHNRI framework with the 12 criteria used for the expert panel meeting in Dubrovnik in 2009, and the process of the expert opinion exercise, are available from the corresponding author upon request.

The first task for the experts was to read the background information assembled about the 29 emerging interventions in a 285-page landscape review, later published as a series of papers [9-14]. The second task was to participate in the expert panel meeting where, over the course of 5 days and a total of 10 discussion sessions, the experts were told why each of the 12 criteria was chosen, and then they discussed how to apply them to each of the 29 emerging interventions. They were free to challenge all information provided to them in a background document and to share further personal knowledge or opinion with the group. Notes of their input were taken and the landscape review was being continuously amended. After each discussion session the experts were invited to score, independently of each other, all emerging interventions according to the 12 agreed CHNRI criteria. For each of the 29 emerging interventions and each criterion, each expert answered questions targeted to assess the likelihood of the proposed emerging interventions to comply with the priority-setting criterion. A summarized version of those questions is presented in Table 2. The full version of questionnaires that were used is available upon request from the corresponding author.

Table 2
A summarized version of questions used to assess whether proposed 29 interventions satisfy the 12 priority-setting criteria

The process of expert assessment (scoring) of emerging interventions was performed as follows: all the experts answered the questionnaire related to each criterion by answering ‘Yes’ (1 point) or ‘No’ (0 points). They were also allowed to declare an informed but undecided answer (0.5 points) or declare themselves insufficiently informed to answer the question (missing input). Thus, the proposed research questions got a score from 20 experts for each of the 12 criteria. This score was “the proportion of maximum possible points scored when an answer was given” (ie, excluding the missing input), and it was a number between 0 and 100%. This number represented a direct measure of “collective optimism” of all the scorers toward each emerging intervention, given the criterion in question. Each of the 29 proposed emerging interventions received 12 criterion-specific scores, each ranging between 0%-100%. The criterion over which the experts were most uncertain was the cost of implementation, which was deemed very difficult to predict by most of them. We agreed that a separate exercise should be conducted in a low-income setting to improve understanding of the factors that affect this cost, and this has been done later [23].

The overall research priority score (RPS) for each intervention was computed as the mean value of 9 intermediate scores for 9 selected criteria. The reason why all 12 criteria weren’t used is because CHNRI exercise requires that the criteria need to be relatively independent of each other (similar to principal component analysis in statistics). In this exercise, we were interested in different components of the cost (development cost, product cost, implementation cost and affordability), but those 4 criteria are in fact a single criterion, and if all 4 were kept in the exercise, this would give an undue 4-fold ‘weight’ to one criterion at the expense of the others. The experts agreed that the most important of the 4 cost-related criteria related to emerging interventions is ‘development cost’, because costs of product and implementation can be met through other mechanisms (such as GAVI, PEPFAR, Global Fund, etc.). Thus, the cost of product, cost of implementation and affordability were kept out of the final score calculation. The exact scores given to all 29 emerging interventions are presented in Table 3. The final report on CHNRI exercise has received the approval of the experts, among whom some (mainly from the industry) wished to remain anonymous.

Table 3
The results of the CHNRI exercise: 29 emerging interventions with 9 intermediate scores and an overall research priority score
figure jogh-02-010304-Fa
Photo: Courtesy of Alasdair Campbell, private collection

The main messages

Table 3 shows that the experts declared most of their collective optimism to improvement of low-cost pneumococcal conjugate vaccines. This was followed by the development of non-liquid and mucosal antibiotic pediatric formulations with improved deliverability and acceptability in low resource settings. The development of common-protein pneumococcal vaccines and multivalent meningococcal vaccines were seen as the third most promising emerging intervention. Following this cluster at the top, the second level of priority was assigned to improvements in existing vaccines (eg, measles or H. influenzae type b) to enable needle-free delivery and heat stability. Similar overall scores were given to evaluations of maternal immunization, improved use of oxygen systems and the development of combination vaccines and vaccines against major viral pathogens. The next level of priority was assigned to various diagnostic tools, the impact of which is currently limited with sub-optimal levels of access to care, care-seeking behavior and the availability of 1st and 2nd line antibiotics. Interventions that proposed passive immunization, action on risk factors such as indoor air pollution or poor sanitation, or development of vaccines against sepsis-causing bacterial pathogens such as S. aureus or E coli received the lowest scores (Table 3).

An extended version of the results of the CHNRI process with the current status of each emerging interventions’ development, the key challenges that remain to be addressed, the visual representation of scores given by the expert panel to each intervention and the assessment of potential effectiveness of each intervention is available in the series of papers published elsewhere [9-14]. It should be noted that the assessment of potential effectiveness (Table 3) can also range from 0%-100%, but its interpretation is different than of the other 11 criteria; rather than measuring collective optimism, it actually predicts the proportion of mortality burden that could be averted through implementation.

Pneumococcal conjugate vaccines, which were treated as emerging interventions back in 2009 because of a very low uptake in low and middle income countries at the time, achieved scores over 80% for all criteria apart from “low product cost” – which indeed ended up being the main point of discussion once they were introduced. In comparison, common protein pneumococcal vaccines are still held back by concerns over answerability (although it is getting closer to 80%), and over all criteria related to their future cost. Other interventions show quite different score profiles. For example, anti-RSV vaccine for use in infants failed on all criteria apart from “acceptance for health workers”, whereas monoclonal antibodies for passive immunization against RSV failed entirely on product cost, affordability and sustainability concerns, although product development cost was considered feasible. The introduction of oxygen systems was considered answerable and did not suffer from major cost concerns, but these systems were not deemed sustainable, sufficiently acceptable and equitable. In comparison, common protein flu vaccines were considered sustainable, acceptable and equitable, but there were still concerns about answerability and costs of development and of the final product.


In accordance with other similar exercises with CHNRI methodology the process showed some clear advantages. The context and the criteria were transparent and the management of the process was overseen by the funding agency (BMGF) over its entire duration. This kind of partnership should result in better understanding and promote ownership and commitment to the main messages of the expert opinion exercise. The scoring process was highly systematic and structured. It was free from undue influence from prominent members within the expert group, because all the experts submitted their opinions and scores independently from each other. The varied mix of the experts from different backgrounds ensured that the scientific assessment of the research priorities is combined with a view of the broader community in which the priorities would be implemented. The entire process from the initial to the final stages was documented and can be viewed and challenged at any point in time. The final result of the process was a simple quantitative outcome (“research priority score”), which measured the “value” of each research option when all the criteria and views were taken into account. This “value” can be combined with the predicted cost of further research and development needs in order to derive an optimal mix of emerging interventions to be funded from a limited budget.


The authors express their gratitude to all the participants of the expert panel meeting in Dubrovnik in September 2009 for their time and valuable input, which allowed conduction of this unique exercise. The authors also thank their project officers for clear guidance and excellent collaboration throughout the course of the project.

Funding: Bill and Melinda Gates Foundation.

Ethical approval: Not required.

Authorship declaration: IR, ET, LZ, HN, KYC, MT, AT, ZB, TH, SEA, MC and HC prepared background reading material (a 285-page “landscape review”) and moderated different stages of the expert meeting. IR, HC, ET, HN, ZB, KYC, AT, MT and MC designed a modified CHNRI process for setting research priorities in emerging health technologies. LZ, ZB and ET conducted the analyses. IR, KYC, HC, ET, HN and LZ drafted the paper with other various authors’ contribution in revision. TH, MC and SEA provided important intellectual input at different stages of the work.

Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare support from Bill and Melinda Gates Foundation for the submitted work. The authors declare no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; and no other relationships or activities that could appear to have influenced the submitted work.


1. Black RE, Cousens S, Johnson HL, Lawn JE, Rudan I, Bassani DG, et al. Global, regional, and national causes of child mortality in 2008: a systematic analysis. Lancet. 2010;375:1969–87. doi: 10.1016/S0140-6736(10)60549-1. [PubMed] [Cross Ref]
2. O'Brien KL, Wolfson LJ, Watt JP, Henkle E, Deloria-Knoll M, McCall N, et al. Burden of disease caused by Streptococcus pneumoniae in children younger than 5 years: global estimates. Lancet. 2009;374:893–902. doi: 10.1016/S0140-6736(09)61204-6. [PubMed] [Cross Ref]
3. Watt JP, Wolfson LJ, O'Brien KL, Henkle E, Deloria-Knoll M, McCall N, et al. Burden of disease caused by Haemophilus influenzae type b in children younger than 5 years: global estimates. Lancet. 2009;374:903–11. doi: 10.1016/S0140-6736(09)61203-4. [PubMed] [Cross Ref]
4. Nair H, Nokes DJ, Gessner BD, Dherani M, Madhi SA, Singleton RJ, et al. Global burden of acute lower respiratory infections due to respiratory syncytial virus in young children: a systematic review and meta-analysis. Lancet. 2010;375:1545–55. doi: 10.1016/S0140-6736(10)60206-1. [PMC free article] [PubMed] [Cross Ref]
5. Nair H, Brooks WA, Katz M, Roca A, Berkley JA, Madhi SA, et al. Global burden of respiratory infections due to seasonal influenza in young children: a systematic review and meta-analysis. Lancet. 2011;378:1917–30. doi: 10.1016/S0140-6736(11)61051-9. [PubMed] [Cross Ref]
6. Horwood C, Vermaak K, Rollins N, Haskins L, Nkosi P, Qazi S. An evaluation of the quality of IMCI assessments among IMCI trained health workers in South Africa. PLoS ONE. 2009;4:e5937. doi: 10.1371/journal.pone.0005937. [PMC free article] [PubMed] [Cross Ref]
7. Osterholt DM, Onikpo F, Lama M, Deming MS, Rowe AK. Improving pneumonia case-management in Benin: a randomized trial of a multi-faceted intervention to support health worker adherence to Integrated Management of Childhood Illness guidelines. Hum Resour Health. 2009;7:77. doi: 10.1186/1478-4491-7-77. [PMC free article] [PubMed] [Cross Ref]
8. Graham WJ, Conombo SG, Zombre DS, Meda N, Byass P, De Brouwere V. Undertaking a complex evaluation of safe motherhood in rural Burkina Faso. Trop Med Int Health. 2008;13(Suppl 1):1–5. doi: 10.1111/j.1365-3156.2008.02081.x. [PubMed] [Cross Ref]
9. Webster J, Theodoratou E, Nair H, Seong AC, Zgaga L, Huda T, et al. An evaluation of emerging vaccines for childhood pneumococcal pneumonia. BMC Public Health. 2011;11(Suppl 3):S26. doi: 10.1186/1471-2458-11-S3-S26. [PMC free article] [PubMed] [Cross Ref]
10. Huda T, Nair H, Theodoratou E, Zgaga L, Fattom A, El Arifeen S, et al. An evaluation of the emerging vaccines and immunotherapy against staphylococcal pneumonia in children. BMC Public Health. 2011;11(Suppl 3):S27. doi: 10.1186/1471-2458-11-S3-S27. [PMC free article] [PubMed] [Cross Ref]
11. Catto AG, Zgaga L, Theodoratou E, Huda T, Nair H, El Arifeen S, et al. An evaluation of oxygen systems for treatment of childhood pneumonia. BMC Public Health. 2011;11(Suppl 3):S28. doi: 10.1186/1471-2458-11-S3-S28. [PMC free article] [PubMed] [Cross Ref]
12. Choudhuri D, Huda T, Theodoratou E, Nair H, Zgaga L, Falconer R, et al. An evaluation of emerging vaccines for childhood meningococcal disease. BMC Public Health. 2011;11(Suppl 3):S29. doi: 10.1186/1471-2458-11-S3-S29. [PMC free article] [PubMed] [Cross Ref]
13. Nair H, Verma VR, Theodoratou E, Zgaga L, Huda T, Simőes EA, et al. An evaluation of the emerging interventions against Respiratory Syncytial Virus (RSV)-associated acute lower respiratory infections in children. BMC Public Health. 2011;11(Suppl 3):S30. doi: 10.1186/1471-2458-11-S3-S30. [PMC free article] [PubMed] [Cross Ref]
14. Higginson D, Theodoratou E, Nair H, Huda T, Zgaga L, Jadhav SS, et al. An evaluation of respiratory administration of measles vaccine for prevention of acute lower respiratory infections in children. BMC Public Health. 2011;11(Suppl 3):S31. doi: 10.1186/1471-2458-11-S3-S31. [PMC free article] [PubMed] [Cross Ref]
15. Rudan I, Chopra M, Kapiriri L, Gibson J, Lansang MA, Carneiro I, et al. Setting priorities in global child health research investments: Universal challenges and conceptual framework. Croat Med J. 2008;49:307–17. doi: 10.3325/cmj.2008.3.307. [PMC free article] [PubMed] [Cross Ref]
16. Rudan I, El Arifeen S, Black RE, Campbell H. Childhood pneumonia and diarrhoea: setting our priorities right. Lancet Infect Dis. 2007;7:56–61. doi: 10.1016/S1473-3099(06)70687-9. [PubMed] [Cross Ref]
17. Rudan I, Chopra M, Kapiriri L, Gibson J, Lansang MA, Carneiro I, et al. Setting priorities in global child health research investments: Universal challenges and conceptual framework. Croat Med J. 2008;49:307–17. doi: 10.3325/cmj.2008.3.307. [PMC free article] [PubMed] [Cross Ref]
18. Rudan I, Gibson JL, Ameratunga S, El Arifeen S, Bhutta ZA, Black M, et al. Setting Priorities in Global Child Health Research Investments: Guidelines for implementation of the CHNRI Method. Croat Med J. 2008;49:720–33. doi: 10.3325/cmj.2008.49.720. [PMC free article] [PubMed] [Cross Ref]
19. Rudan I, El Arifeen S, Bhutta ZA, Black RE, Brooks A, Chan KY, et al. Setting research priorities to reduce global mortality from childhood pneumonia by 2015. PLoS Med. 2011;8:e1001099. doi: 10.1371/journal.pmed.1001099. [PMC free article] [PubMed] [Cross Ref]
20. Tomlinson M, Swartz L, Officer A, Chan KY, Rudan I, Saxena S. Research priorities for health of people with disabilities: an expert opinion exercise. Lancet. 2009;374:1857–62. doi: 10.1016/S0140-6736(09)61910-3. [PubMed] [Cross Ref]
21. Lawn JE, Bahl R, Bergstrom S, Bhutta ZA, Darmstadt GL, Ellis M, et al. Setting research priorities to reduce almost one 409 million deaths from birth asphyxia by 2015. PLoS Med. 2011;8:e1000389. doi: 10.1371/journal.pmed.1000389. [PMC free article] [PubMed] [Cross Ref]
22. George A, Young M, Bang A, Chan KY, Rudan I, Victora CG, et al. GAPPS expert group on community based strategies and constraints. Setting implementation research priorities to reduce preterm births and stillbirths at the community level. PLoS Med. 2011;8:e1000380. doi: 10.1371/journal.pmed.1000380. [PMC free article] [PubMed] [Cross Ref]
23. Shehzad S, Sadruddin S, Bari A, Khan A, Ibaad-ul-Haq M, Qazi SA. Cost of community treatment of severe pneumonia in district Haripur, Pakistan. (submitted for publication)

Articles from Journal of Global Health are provided here courtesy of Edinburgh University Global Health Society
PubReader format: click here to try


Save items

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...