CONTEXT IN RWANDA LEADING UP TO THE PROGRAM

Economic and Financial Context for Health in Rwanda

As Rwanda recovered and rebuilt from the genocide against the Tutsi, its economy eventually also started to experience considerable growth. Figure 3-1 shows the trajectory of growth from the early 2000s, as illustrated by gross domestic product (GDP), the total national expenditure, and the portion of that national expenditure that is government spending. The HRH Program was situated in this period of economic growth.

FIGURE 3-1

The size of the Rwandan economy from 2000–2018 in current U.S. dollars. NOTES: Gross national expenditure is the sum of household final consumption expenditure, general government final consumption expenditure, and gross capital formation. General (more...)

Concurrently, there was growth in spending on health in Rwanda. Figure 3-2 shows the trajectories of growth in both per capita total health expenditures and per capita government spending on health, alongside the growth in per capita GDP.

FIGURE 3-2

Health expenditures in Rwanda from 2000–2016 (in purchasing power parity per capita). NOTES: Current health expenditure is the estimate of all health care goods and services consumed each year. Domestic general government health expenditure is (more...)

In this same time period, as Figure 3-3 shows, close to half of the health expenditure in Rwanda has consistently come from external aid, ranging from 41 percent to 54 percent between 2000 and 2016. Spending specifically for Rwanda's response to HIV increased drastically during this period, and as of 2014 it has been consistently estimated at about $200 million annually. Of this, 80 percent comes from external funders, a larger proportion than for health overall (Nsanzimana et al., 2015).

FIGURE 3-3

Proportion of health expenditure in Rwanda by revenue source. NOTE: Domestic “all sources” comprises out-of-pocket voluntary health insurance and domestic public expenditures. SOURCES: WHO Global Health Expenditure Database; WHO, 2019. (more...)

PEPFAR Funding Context

Before the HRH Program was launched in 2012, the President's Emergency Plan for AIDS Relief (PEPFAR) had an established funding history in Rwanda going back to 2003 (Binagwaho et al., 2016). Figure 3-4 shows PEPFAR total annual planned funding for programs in Rwanda from fiscal year (FY) 2009 to FY 2019, including the years surrounding the HRH Program.¹ Amounts shown in the bar graph were PEPFAR allocations to treatment, testing, prevention, health systems strengthening (HSS), government and systems, care, and applied pipeline. Percentages indicate proportion of annual funding allocated for HSS over time (PEPFAR, 2019c).

FIGURE 3-4

PEPFAR Rwanda planned funding by program area and percent allocation for HSS. NOTES: The bar graph shows amounts allocated to each program area indicated and the percentage of funding dedicated to health systems strengthening (HSS). These activities are (more...)

PEPFAR's total planned funding for its Rwanda program portfolio declined overall from 2009 to 2019. The proportion of funding allocated to HSS increased during the period between FY 2013 and FY 2016 relative to surrounding years, with a peak in FY 2013. This period is concurrent with the timing of financial support for the HRH Program. Before FY 2013, PEPFAR planned funding in Rwanda for HSS had not exceeded $9 million and had remained under 10 percent of total planned funding. Between FY 2013 and FY 2016, PEPFAR funding for HSS activities ranged from $24.7 million in FY 2013 (27 percent of its total portfolio) to $11.8 million (17 percent of its portfolio) in FY 2016. Since FY 2016, funding for HSS activities in the portfolio has not exceeded $1.9 million, or 2 percent of total planned funding. In the same period between FY 2013 and FY 2016, there was a decrease in the proportion of funding allocated to other program areas. In contrast, the proportion of PEPFAR planned funding devoted to HSS activities between FY 2009 and FY 2019 in other East African Community countries has been less variable and has not exceeded 13 percent of the total portfolio.²

PEPFAR Investments in the HRH Program

At the inception and during the first 3 years of its funding, PEPFAR considered the HRH Program to be a strategic or key priority within its overall portfolio (PEPFAR Rwanda, 2012, 2013, 2014). Initially, PEPFAR shifted resources from other activities to “transfer significant support to the area of HRH” (PEPFAR Rwanda, 2012) and thus fund the Program (PEPFAR Rwanda, 2012, 2013). Subsequently, the Program's placement relative to other PEPFAR programmatic priorities decreased over time (PEPFAR Rwanda, 2013, 2014, 2015). The HRH Program was being funded during the window of peak PEPFAR funding for HSS in Rwanda, shown in Figure 3-4.

Between FY 2011 and FY 2013, the Program was funded as a component of an existing U.S. Centers for Disease Control and Prevention (CDC) cooperative agreement with the MOH called “Strengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under PEPFAR” (also known as the Clinical Services Cooperative Agreement) (Mtiro, 2018). Preceding the HRH Program, CDC had already been providing direct support to the MOH for its HIV/AIDS-related programs (CDC, 2009c, 2012):

• The integration of HIV services and programs into the health system at all levels;
• Capacity building for infection control;
• Injection safety;
• Medical waste management;
• Epidemiological investigation, lab management, pharmaceutical management, health communication, electronic medical record use, and data collection and analysis;
• Use of terminology standards and registries; and
• Training for physicians, community health workers, and service providers in clinical handling and management of sexual and gender-based violence.

This funding also provided salary support and benefits for physicians, technicians, and data managers, as well as supporting quality improvement initiatives, performance-based financing evaluations, technical meetings with stakeholders, and management and audits of MOH/PEPFAR-supported facilities and sites (MOH, 2014a).

The United States Agency for International Development (USAID) also provided some funds for the Program, although the amounts could not be confirmed (CDC, 2012; PEPFAR Rwanda, 2013). In addition to PEPFAR funding, the Program had a diversified funding base, evidenced by the engagement of the Global Fund to Fight AIDS, Tuberculosis and Malaria.³

Starting in FY 2014, CDC used a separate cooperative agreement with the MOH, “Strengthening Human Resources for Health Capacity in the Republic of Rwanda under PEPFAR,” to fund the HRH Program (Mtiro, 2018). In 2015, PEPFAR's priorities shifted, and the HRH Program was classified as a “noncore” investment as part of an analysis in alignment with PEPFAR's new 3.0 strategy. The decision was made to end funding for the Program on March 30, 2017, instead of continuing PEPFAR support through June 30, 2019, the Program's official closing date (Mtiro, 2018; PEPFAR Rwanda, 2015).

Public reporting systems provided limited information about the actual amount invested by PEPFAR in the HRH Program. Table 3-1 shows available information regarding the mechanisms PEPFAR used to provide funding for the Program.

TABLE 3-1PEPFAR Funding Sources That Contributed to the HRH Program

View in own window

	FY 2012 (Oct 2011–Sep 2012)	FY 2013 (Oct 2012–Sep 2013)	FY 2014 (Oct 2013–Sep 2014)	FY 2015 (Oct 2014–Sep 2015)	FY 2016 (Oct 2015–Sep 2016)	FY 2017 (Oct 2016–Sep 2017)	FY 2018 (Oct 2017–Sep 2018)
U.S. Centers for Disease Control and Prevention (CDC)*
Strengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda Under PEPFAR	$17,556,432 (total cooperative agreement)	$22,168,552 (total cooperative agreement)	$23,507,981 (total cooperative agreement)	$0	$0	N/A	N/A
(Award No. U2GPS002091; Mechanism No. 10825)
Strengthening Human Resources for Health Capacity in the Republic of Rwanda Under PEPFAR	N/A	N/A	N/A	$11,000,000	$10,500,000	Unknown**	Unknown**
(Award No. U2GGH001614; Mechanism No. 17621)
United States Agency for International Development (USAID)***
Unknown Award(s)/Mechanism(s)	Unknown	Unknown	Unknown	Unknown	Unknown	Unknown	Unknown

NOTES: N/A = not applicable (funding mechanism not used to contribute to the HRH Program that year). All amounts are in U.S. dollars.

*

Annual total disbursed funding by award/mechanism as reported through the U.S. HHS Tracking Accountability in Government Grants System (TAGGS) (accessed July 7, 2019). TAGGS reports the total amount disbursed through such agreements, but does not provide any further breakdown by program or activity. Therefore, the proportion of the amount that was specific to the HRH Program is not known in FY 2012 to FY 2014, when the HRH Program was funded as a component of the cooperative agreement, “Strengthening Capacity to Respond to the HIV/AIDS Epidemic,” that started in FY 2009. Starting in FY 2015, the HRH Program was funded through its own cooperative agreement, “Strengthening Human Resources for Health Capacity in the Republic of Rwanda Under PEPFAR,” and the amounts reported in TAGGS are reflective of the amounts disbursed for the HRH Program: $11 million in FY 2015 and $10.5 million in FY 2016. Any amounts disbursed through CDC after FY 2016 are also not known.

**

Information is not publicly available on whether amounts were disbursed through this or any other CDC award or mechanism after FY 2016.

***

Although there is documentation that some PEPFAR investments in the HRH Program came from other implementing agencies, those amounts and sources are not reported publicly. Approval of USAID maternal and child health funds to be used for implementing the HRH Program is referenced in a 2012 CDC memo requesting a multiyear expansion supplement for the “Strengthening the Ministry of Health's Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda Under PEPFAR” cooperative agreement. However, amounts were redacted, and data on the portion of the award/mechanism allocated to the HRH Program were not publicly available (correspondence provided by CDC Rwanda in August 2018). In addition, Rwanda's Country Operational Plan FY 2013 referenced a planned shift of $2 million from USAID's Family Health Project to the HRH Program, but the actual amount disbursed and allocated for the HRH Program is not publicly available.

SOURCES: CDC, 2012; PEPFAR Rwanda, 2014; TAGGS, 2019.

HRH Program Budget and Ministry of Health Expenditures on Health

The HRH Program budget proposed in 2011 was $151.8 million over 8 years (MOH, 2011b). The Program received external funding from the U.S. government, through PEPFAR, and from the Global Fund. The total funding amount from these external sources, as provided by the MOH, was just under $100 million, with approximately 60 percent coming from the U.S. government (see Table 3-2). This comprised 12 percent to 16 percent of the total annual PEPFAR investment in Rwanda. The amounts disbursed from CDC to the MOH, however, were slightly lower than the budgeted amounts (see Table 3-3). Where there was a balance, CDC approved carrying it over to the next year, with the exception of 2015 to 2016, after PEPFAR had deemed the Program noncore.

The total amount budgeted for the HRH Program comprised about 4 percent of the total annual health budget for the MOH as reported in the Ministry's Health Resource Output Tracking Report for available concurrent years (see Table 3-4). Given that government expenditure comprises less than half of the total expenditure on health in Rwanda (as described in Figure 3-3), the HRH Program likely represented less than 3 percent of total health spending in Rwanda at the time it was implemented.

TABLE 3-2HRH Program Budget by Year (U.S. Dollars)

View in own window

Fiscal Year	U.S. Government	Global Fund	Total Annual Budget
2012–2013	12,300,163	6,775,325	19,075,488
2013–2014	14,971,013	6,775,325	21,746,338
2014–2015	12,976,798	6,775,325	19,752,123
2015–2016	11,000,000	5,729,026	16,729,026
2016–2017	10,500,000	5,847,111	16,347,111
2017–2018	0	5,519,856	5,519,856
2018–2019	0	362,246	362,246
Total HRH Program Budget	61,747,974	37,784,214	99,532,188

NOTES: Amounts and totals are reported as provided by the MOH in current U.S. dollars. On average, inflation was about 5.2 percent in the same period, and the average growth of the exchange rate was about 5.8 percent between 2012 and 2018 (World Bank, 2019f,g). U.S. government and Rwandan FYs are not the same: U.S. FY runs October 1 through September 30; Rwandan FY runs July 1 through June 30.

SOURCE: Financial data provided by the MOH.

Figure 3-5 provides the context of the MOH's broader concurrent workforce-related expenditures before and during the HRH Program. From FY 2010/2011 to FY 2014/2015, as the amount of MOH expenditure going toward health workforce increased there was a small decline in workforce expenditure as a proportion of the total MOH expenditure, from 42 percent to 39 percent. Concurrently, there was an increase in the proportion of investments for drugs and other consumables and for materials and equipment. However, trends in these data are difficult to interpret because of changes from year to year in how expenditures were reported by the MOH. It is also not possible to clearly interpret how the overall expenditures of the MOH intersected with the HRH Program expenditures because these reports do not indicate whether these expenditures include or exclude the funds expended as part of the HRH Program. In addition, there may be other sources of expenditures on HRH in Rwanda.

FIGURE 3-5

MOH expenditures FY 2010/2011–FY 2014/2015 (U.S. dollars). NOTES: USD = U.S. dollars. Amounts shown are the current amounts reported at the time each source report was published. Different input categories were used each year and aggregated according (more...)

TABLE 3-3CDC Disbursements for the HRH Program by Year (U.S. Dollars)

View in own window

Year	Disbursement from CDC	Approved Carryover	Expenses	Balance
2012–2013	12,300,163		6,630,040	5,670,123
2013–2014	12,577,279	5,670,123	4,898,183	13,349,219
2014–2015	12,769,798	13,349,219	26,119,017	—
2015–2016	11,000,000		10,558,378	441,622
2016–2017	10,058,378		10,500,000
Total	58,705,618

SOURCE: Financial data provided by the MOH.

TABLE 3-4Ministry of Health Budget and Expenditures Reported Preceding and During the HRH Program (U.S. Dollars, Rounded to Millions)

View in own window

	FY 2010/2011	FY 2011/2012	FY 2012/2013	FY 2013/2014	FY 2014/2015
Total Budget	514 milliona	539 millionb	533 millionc	*	499 milliond
Total Expenditure	420 milliona	514 millionc	*	481 milliond	480 milliond

NOTES: Amounts shown are the current amounts reported at the time each source report was published. Amounts were provided in Rwandan francs (RWF) after FY 2011/2012 and therefore converted to U.S. dollars (USD) using the following World Bank historical exchange rates, rounded to the nearest dollar: in 2013, 1 USD = 647 RWF; in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g).

*

Expenditure data for FY 2012/13 and budget data for FY 2013/14 were not reported.

SOURCES:

a

MOH, 2012c;

b

MOH, 2012c, 2013b;

c

MOH, 2013b;

d

MOH, 2018b.

TABLE 3-5MOH Health Workforce Expenditures with Category Breakdowns by Year (U.S. Dollars)

View in own window

	FY 2010/2011	FY 2011/2012	FY 2013/2014	FY 2014/2015
Total Expenditures	419,636,014	514,014,611	480,938,416	479,889,043
Workforce (Percent of Expenditures)			40%	39%
Workforce Expenditure Amount	176,132,460	212,751,636
Workforce Category Breakdowns (as Reported)
Direct Salaries/Labor Costs/Remuneration	109,328,045		28% (of total expenditures)	30% (of total expenditures)
Capacity Building	42,369,081		7% (of total expenditures)	5% (of total expenditures)
Incentives	24,435,334
In-service Training and Workshops		40,200,881
Incentives for Community Health Workers		29,381,321
Pre-service Training and Workshops		8,064,948
Salaries (Contracted Government Personnel)		35,049,224
Salaries (Government Personnel)		27,784,464
Salaries (Nongovernment Personnel)		1,475,639
Salaries (Short-Term Consultants)		3,998,622
Service Provision		18,688,571
Technical Assistance		21,554,278
Performance-Based Financing		26,553,688
Social Benefits			5% (of total expenditures)	4% (of total expenditures)

NOTES: Amounts shown are the current amounts reported at the time each source report was published. In FY 2013/2014 and FY 2014/2015 expenditure categories were reported as percentages of the total expenditure. Amounts reported in Rwandan francs (RWF) were converted to U.S. dollars (USD) using the World Bank's historical exchange rates, rounded to the nearest dollar: in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g).

SOURCES: MOH, 2012c, 2013b, 2016a, 2018b.

The categories of expenditures aggregated into the broad category of “workforce,” and how they were reported, varied from year to year (see Table 3-5).

A consistent pattern was that salaries and labor costs represented the largest proportion of expenditures. When reported, categories such as capacity building, pre-service training, and workshops represent a very small amount of the total investment.

HUMAN RESOURCES FOR HEALTH PROGRAM VISION AND DESIGN

DESIGN PROCESS

Figure 3-6 depicts the overall time line of the HRH Program. In 2011, the Government of Rwanda submitted an unsolicited proposal, which was funded in 2012 under the expanded Clinical Services Cooperative Agreement (CDC, 2012). Prior to funding, 18 USIs submitted letters of intent to join the Academic Consortium, although one withdrew before the Program was launched (MOH, 2011c). USI participation in the Program increased until 2015, when institutions began withdrawing. An MOU was established between the MOH and a new USI in 2018, after PEPFAR's investment had ended. Membership in the Academic Consortium provided USIs with a mechanism through which to establish annual MOUs with the MOH. The Consortium was also a mechanism for determining the clinical purview of each USI. For example, Yale University was an obstetrics and gynecology partner, and New York University and Emory University provided support to the nursing programs, both upgrading nurses and the establishment of the MSN program.

FIGURE 3-6

HRH Program time line. NOTE: HRH = human resources for health; MGHD = Master of Global Health Delivery; MSN = Master of Science in Nursing; PEPFAR = President's Emergency Plan for AIDS Relief; USI = U.S. institution.

USIs under the Academic Consortium contracted with individuals to work at the University of Rwanda to “twin” with and mentor existing Rwandan faculty, aid in strengthening existing residency programs and establish new programs, provide direct teaching services to Rwandan students, and, in some cases, provide direct care to patients. With few exceptions, the USIs contracted with U.S. citizens and did not engage regional faculty who could lend practical experience and knowledge from a more locally relevant context to the Program (see Chapter 4 for a more detailed discussion).

FINANCIAL MANAGEMENT

The 2011 HRH Program proposal indicated that a Single Project Implementation Unit would be established “to centralize programmatic, financial, and administrative management functions” (MOH, 2011b). PEPFAR investments in the HRH Program flowed directly to the MOH, although according to one respondent, a “parastatal” had been set up to manage the money, where funds were disbursed by MINECOFIN. One respondent involved in the early phases of the Program noted that this design was intended “to try and eliminate the high cost of an international NGO go-between … and having a government that had demonstrated their ability to manage and oversee, monitor, and evaluate programs” (25, Non-Government of Rwanda HRH Program Administrator).

The Permanent Secretary of the MOH had oversight over the HRH Program budget. The Government of Rwanda employed a Public Financial Management system, in which funds were held in an account separate from the National Treasury. The system facilitated budget oversight, because the University of Rwanda used the same system.

According to MOH data, the bulk of the expenditures from PEPFAR investments in the HRH Program went to USIs, followed by equipment procured for health professional education and clinical training (see Table 3-7).

Table 3-7 includes only funds provided by CDC and does not reflect other investments from sources such as the Global Fund and the MOH. According to the MOH, the $441,661 difference between the budgeted amount and the disbursed amount was not funded by CDC.

The MOH articulated the importance of unmeasurable inputs that were critical for the running of the HRH Program, including overhead costs expended by the University of Rwanda for managing its facilities, unallowable expenses incurred by USIs in recruiting and managing USI faculty, and other essential financial inputs by the MOH, such as annual travel allowances for all residents to move across sites for their rotations and visit their families when they were rotating at a site away from their primary residence.

Financial audits of the Global Fund's investments during periods concurrent with the HRH Program (2014, 2016–2018), and therefore presumably inclusive of funds applied toward the HRH Program, yielded no major concerns. Systems of internal control were typically found to be “generally” or “partially” effective. The 2014 audit identified three areas for improvement: data quality assurance, supporting documentation, and low absorption of funds (Global Fund, 2014). The 2016–2018 Global Fund audit found, however, that grant oversight and assurance mechanisms needed significant improvement (Global Fund, 2019). Audits of other investments in the MOH had similar findings. An audit of the MOH's management of Gavi funds rated the MOH as partially satisfactory (Gavi, 2018), and an assessment of a Swedish International Development Cooperation Agency–supported program that centered on building research and higher education capacity identified three key issues in the MOH's financial management: lack of consolidated financial reporting; delays in disbursements to Swedish partner universities; and slow budget execution that affected implementation (Tvedten et al., 2018). Additionally, a thorough search for a CDC audit of the HRH Program was conducted, and none was located.

Beyond the financial management processes, program administrators expressed some frustration about PEPFAR's processes and restrictions on how funding could be used, especially in light of the HRH Program's holistic and integrated design:

The second [challenge] was how surprisingly hard this was using PEPFAR money … it was a pain in the ass, I'm sorry for my language.... For anything, any comment, any point, you need to ask for authorization…. People from Atlanta will have to sign something and Rwandans as well and … you need to write 200 pages of document. The equipment that was supposed to come in year 1 came in year 3 or 4. It was really not easy to use the funds. (45, Government of Rwanda HRH Program Administrator)

There are too many specialties that these fund were not really able to support and for us, I can see how [HIV] is related to that. A radiologist doesn't just treat a non-HIV patient, it treats both and the most complicated cases come from the people who have HIV. Same with dentistry. (03, Government of Rwanda HRH Program Administrator)

PROGRAMMATIC MANAGEMENT APPROACHES AND CHALLENGES

The HRH Program experienced challenges in its design, launch, and execution that were both internal to Rwanda (between the MOH and the MOE and stakeholders outside the Government of Rwanda) and external to Rwanda (between the MOH and USIs).

MONITORING AND EVALUATION

Reporting requirements for the HRH Program evolved over the years, in part because of the changing funding mechanisms as well as the broader shift in standards for M&E. In April 2012, before the Program was integrated into the MOH's periodic program reports for the Clinical Services Cooperative Agreement, the MOH submitted to CDC a detailed work plan and budget for the Program. The work plan was similar to a logical framework and included goals, objectives, activities, a quarterly time line, and program indicators that were consistent, if not more detailed, than CDC reporting requirements at the time (CDC, 2009a; MOH, 2010, 2012a). Program indicators for the HRH Program and other components of the Clinical Services Cooperative Agreement, however, remained largely output measures (CDC, 2009a; MOH, 2010, 2012e).

In 2013, the work plan for the Clinical Services Cooperative Agreement Continuing Application called for the development of an “M&E framework and indicators” as one of the HRH Program's “major programmatic/technical activities” (MOH, 2013a). An open invitation for Program participants to volunteer to join one of two stakeholder groups—an in-country M&E Working Group (comprising HRH Program members and Rwandan faculty) and a USI M&E advisory group—was also sent in 2013 to support the M&E plan development. As a result, the MOH submitted to CDC the Program's “Human Resources for Health Monitoring & Evaluation Plan” in March 2014 (MOH, 2014b).

In addition to the three main strategic outcomes described previously, the M&E plan provided outputs, detailed process and outcome measures, data sources, annual performance targets, reporting frequency, and baseline and midterm evaluation components; the rigor of this plan reflected PEPFAR's increasing emphasis on M&E (PEPFAR, 2011, 2012) and use of more sophisticated metrics, which had begun in 2009 with the development of PEPFAR's Next Generation Indicators. In contrast to PEPFAR's earlier metrics, which had been largely output measures, the Next Generation Indicators sought to measure program coverage and quality using both process and outcome indicators.

In October 2014, when CDC issued a noncompetitive funding opportunity announcement to continue the Program's funding at a level of $14 million per year for 5 years under its own funding mechanism, PEPFAR's M&E requirements for the Program were both comprehensive and rigorous:

CDC will work with the awardee to implement, using PEPFAR and national indicators as a base, a robust monitoring and evaluation system designed to track the implementation of HRH efforts, preservice training, and maintenance of critical health care equipment that have already begun and will continue after [U.S. government] support for these activities concludes. This system will provide evidence needed to drive a programmatic process for decision making. This will be built on a three-tiered approach that captures data through routine program monitoring, program evaluation, and specific surveys. (CDC, 2014)

To fulfill these requirements, the MOH submitted a project narrative and evaluation and performance measurement plan in its award application (MOH, 2014d). Under the new cooperative agreement, CDC required the MOH to revise and submit a more detailed evaluation and performance measurement plan within the first 6 months of the award (CDC, 2014). In 2015, the MOH submitted a revised programmatic/technical work plan that documented “SMART (specific, measurable, achievable, relevant, and time-based) Objectives” activities, a time line, responsible parties, and “process measures” (MOH, 2015b).

According to the 2014 M&E plan, a point person, trained in the use of data collection tools for routine reporting, would be available for every teaching hospital and health facility (MOH, 2014b). Three main evaluations at three time points would also be planned to measure the program's outcomes and impact, recognizing that a baseline might only be available for some areas because the Program had already started (MOH, 2014b). Most indicator data for program monitoring would be collected semiannually from the following sources: the Health Management Information System, Human Resources Information System, health education institution records, and program records. A Health Education Institution Survey and Hospital Survey were also planned, although no evidence was found to show that either survey occurred (MOH, 2014b).

The HRH Program midterm review reported that quarterly surveys were administered to individuals involved in the Program, although response rates were low, especially among Rwandan faculty (MOH, 2016b). An organizational case study of the Program concluded that its prioritization of implementation costs and underestimate of administrative, monitoring, and evaluation costs—coupled with the substantial size and scope of the Program—created a “mismatch between needs and availability of resources and expertise” and a lack of resources for M&E (Cancedda et al., 2018).

From the perspective of interview respondents, little investment was made in M&E processes, which had also been noted in the Program's midterm review. A USI respondent commented that having a single M&E officer for the Program was insufficient:

One M&E officer, like 100 U.S. faculty on the ground from 21 different institutions across 5 or 6 physical locations in Rwanda and in 4 major disciplines, and an officer barely had the capacity to figure out what kind of questionnaire to write, much less disseminate to collect it, analyze it, and report on it in a way that was receivable by funders, partners, government, and university. Impossible. Totally impossible. Total fail. (83, USI Faculty in Surgery)

Another respondent stated that there was “a high-level M&E document [referring to the 2014 M&E plan], but it's not specific to actual activities on a yearly basis” (22, Non-Government of Rwanda HRH Program Administrator from an International NGO).

SUSTAINABILITY PLANNING

Under PEPFAR's partnership frameworks, an important component of PEPFAR's 2.0 strategy (PEPFAR, 2014, 2019b), sustainability was defined as “supporting the partner government in growing its capacity to lead, manage, and ultimately finance its health system with indigenous resources (including its civil society sector), rather than external resources, to the greatest extent possible” (PEPFAR, 2009). PEPFAR's role was to move from supporting the expansion and provision of direct services to offering technical assistance to partner countries to work toward sustainable programming (PEPFAR, 2019b). This was operationalized through a 5-year Partnership Framework, which aimed to produce results in HIV prevention, care, and treatment, and to position countries “to assume primary responsibility for the national responses to HIV/AIDS in terms of management, strategic direction, performance monitoring, decision making, coordination, and, where possible, financial support and service delivery” (PEPFAR, 2009).

For the HRH Program, the 2011 proposal referred to sustainability in the context of the intention that, after 8 years, the Government of Rwanda would operate the Program under its own budget. The proposal included a rationale for the availability of the financial resources that would be needed to absorb the newly trained health workers and maintain equipment purchased under the HRH Program. It argued that the government's commitment to allocating 15 percent of the total national budget to health, as agreed to in the Abuja Declaration, combined with the projected 5 percent annual increase in the total Rwandan budget due to GDP growth, would generate enough funds to cover the estimated $43 million per year for Rwandan health professionals produced by the HRH Program, $9 million per year for tuition support, $13 million (over several years) for equipment replacement, and $1.5 million per year for equipment maintenance (MOH, 2011b). The MOH's 2014 funding application program narrative in response to the funding opportunity announcement articulated sustainability as the programmatic aim of building “sufficient Rwandan educators, infrastructure and equipment, and domestic financing to support Rwanda's health care and health sciences education without external support” (MOH, 2014d).

Although it appears that some planning for sustainability took place during the proposal phase, interview respondents expressed variation in how to define and measure HRH Program sustainability, which was linked to how they conceptualized the Program's goals and objectives, as articulated by an administrator at the University of Rwanda:

It depends on what you mean by sustainability. If you mean a program that goes on forevermore, having a hundred people coming every year … if that's your version of sustainability we need to sit down and talk about it. If it's about building on the gains, that's another issue and that's something that the university takes very seriously.… In terms of sustainability, do we need the same thing forevermore for the next 20 years? Then, no. We would have failed. Do we want input of experts for the expertise that we don't have? Absolutely, you know! Sustainability is about us, being able to do by ourselves, better than it is done anywhere in the world, and that's what we are striving to do at the University of Rwanda all the time. (02, University of Rwanda Administrator)

Among respondents who believed the intent was to build capacity within the University of Rwanda to continually produce high-quality health workers, sustainability was seen as built into the design:

[T]he goal was to mentor teachers and train faculty. So, in the design, the program is already sustainable. (43, Government of Rwanda HRH Program Administrator)

One respondent explained that the design included a gradual phasing out of USI faculty and a simultaneous increase in Rwandan faculty:

The program was implemented with the target that after 7 years the program will end.… [T]he sustainability plan was kind of [a] mechanism of decreasing gradually the U.S. faculty, while increasing retained and available local faculty. That is one thing. On the other side, the [University of Rwanda] had in their planning to have to increase the college faculty so the local Rwandan faculty in the university funded for 2 years, I think that was the initial plan for the 2 last years of the program, with the program funding while negotiating to be integrated in the structure of the university. I think that is how the sustainability plan was thought. (20, Government of Rwanda HRH Program Administrator)

In contrast, for another respondent, who had expressed that the objective of the HRH Program was to build capacity of Rwandan health education institutions, most directly the University of Rwanda, the Program failed to clearly understand and build on the capacity of local institutions:

[L]ocal institutions were … involved, but there was nothing really done to understand the local institutions' capacity, strength, readiness, levels, interest, priorities, process. And nothing was really done to understand how we could integrate successfully this faculty and effectively in the local institutions, you know, how they were going to support the leadership, how they were going to complement and not replace local faculty in the way that built on synergies and not mix with redundancies which happened a lot, I think. (22, Non-Government of Rwanda HRH Program Administrator)

Respondents external to the government were skeptical of the sustainability and long-term planning that was done in the design phase:

I think the attitude of a lot of Rwandan institutions, departments, was that “Ok, we will take the support and enjoy it while it's there. It is really helpful; it is filling a gap.” And then one day it ends and there's something else coming in. And that's fine, but not really long-term planning. (22, Non-Government of Rwanda HRH Program Administrator)

CONCLUSIONS

The HRH Program represented a confluence of unusual circumstances and an opportunity in foreign assistance. Its design endorsed Rwanda's larger vision of strengthening the country's workforce, including in the health sector, although there were missed opportunities to learn systematically. PEPFAR 2.0 marked “a new era of collaborative planning and health systems strengthening activities with … partner governments” (PEPFAR, 2014). PEPFAR's investments in the HRH Program were a critical contribution to launching an Academic Consortium that leveraged (1) existing and new partnerships with individual USIs, (2) the political commitment of a government with record of successfully managing health systems programs, and (3) a holistic PEPFAR strategy that emphasized HSS.

PEPFAR's investments in the HRH Program in Rwanda were relatively unique, departing from U.S government and many other big donors' models for “business as usual,” but aligned with prevailing principles established by the Busan Partnership for Effective Development Co-operation, in which funding partners would support shared goals according to specific country situations (Fourth High-Level Forum on Aid Effectiveness, 2011). PEPFAR investments have continuously supported programs and activities implemented in partner countries and, consistent with PEPFAR's strategy to move toward sustainability, have directed their funding over time to local prime partners, although the proportion of funding to partner country governments has been relatively stable (IOM, 2013).

From the outset, Rwanda expressed a strong commitment to national ownership, as articulated in Vision 2020, and sought to increase the direct management of externally funded health and HIV programs. To move toward this vision, the Government of Rwanda and its PEPFAR-funded partners promoted a transition plan that shifted HIV program leadership from partners to the host country, including direct PEPFAR financing to the government (Government of Rwanda, 2012). The management transition took place by February 2012, when the MOH became a direct recipient of PEPFAR funds and gained responsibility for coordinating critical service delivery (Binagwaho et al., 2016). These investments challenged the conventional model, and used country ownership and shared responsibility as a new paradigm when the HRH Program commenced in 2012 (Goosby et al., 2012a,b).

The investments in the HRH Program also were an opportunity to add to the evidence around twinning because the Program's Academic Consortium was designed not only as a departure from short-term faculty stints in Rwanda, but also to completely phase out by the end of the Program as the “twins” and new graduates gained skills to serve as local faculty. The twinning arrangements were between individual USI faculty and University of Rwanda faculty, not between the two institutions. The implications of this arrangement, discussed more fully in subsequent chapters, were felt in terms of sustained capacity at the institutional level to provide ongoing health professional education.

There was concurrence among HRH Program participants on a high-level vision and intent, which aligned with broader health-sector goals. However, there was a lack of clarity around the mechanisms and pathway for achieving a world-class health care system, the consequences of which were felt during the implementation of the Program, as discussed throughout subsequent chapters. The relative importance of building health education capacity, including enhancing the skills of Rwandan faculty and the learning environment at the University of Rwanda, was inconsistent during the life of the Program. In particular, it was unclear whether activities for building institutional capacity for health professional education were prioritized simply as a mechanism to produce the desired number of health professionals across cadres, or viewed as complementary goals to having a sustainable, skilled, and specialized workforce, as expressed in the later strategic outcomes (MOH, 2011b, 2014b, 2015b).

The downstream effects of the shifting language in MOH documents was a lack of clarity across stakeholders around the Program's goals and objectives, which had implications for the design and for sustainability planning and were further compounded by differing (and sometimes conflicting or restrictive) administrative practices across participating institutions (Government of Rwanda, USIs, and PEPFAR). Planning for sustainability evolved from accounting for funding of the established training activities under the HRH Program after 2019 to more holistically ensuring the quality of health professional education and service delivery.

Although the Government of Rwanda recognized the unprecedented scale and nature of the HRH Program's strategies, there was insufficient planning to exploit the opportunity to learn systematically from this endeavor by establishing rigorous M&E processes and supportive mechanisms at the outset (MOH, 2011b, 2014b, 2016b). The original program objectives were incorporated into various indicators for the M&E plan and midterm review; however, the development of an M&E plan after implementation had begun resulted in a lack of baseline data for some cadres and program areas, as well as unexplained differences in baseline and target values (MOH, 2011b, 2014b, 2016b). Even as the post hoc M&E plan articulated an intention to “provide a reliable data-driven approach … to ensure accountability, program improvement, impact, and learning,” there was lack of follow-through by the Government of Rwanda in providing dedicated resources to establish an M&E platform (e.g., M&E working groups were composed of volunteer HRH Program participants, as opposed to designated roles), to carry out planned assessments, and to build overall capacity for monitoring, evaluation, and learning (MOH, 2014b).

Overall management of the HRH Program was challenged by the lack of time for operational management, both at the outset of implementation and continuously, as unexpected circumstances arose. More time between the design, launch, and execution phases would have supported stakeholders' ability to better anticipate and develop contingent strategies for issues such as PEPFAR funding processes and restrictions.

REFERENCES

• Binagwaho A, Kyamanywa P, Farmer PE, Nuthulaganti T, Umubyeyi B, Nyemazi JP, Mugeni SD, Asiimwe A, Ndagijimana U, Lamphere McPherson H, Ngirabega Jde D, Sliney A, Uwayezu A, Rusanganwa V, Wagner CM, Nutt CT, EldonEdington M, Cancedda C, Magaziner IC, Goosby E. The Human Resources for Health Program in Rwanda—new partnership. New England Journal of Medicine. 2013;369(21):2054–2059. [PubMed: 24256385]
• Binagwaho A, Farmer PE, Nsanzimana S, Karema C, Gasana M, De Dieu Ngirabega J, Ngabo F, Wagner CM, Nutt CT, Nyatanyi T, Gatera M, Kayiteshonga Y, Mugeni C, Mugwaneza P, Shema J, Uwaliraye P, Gaju E, Muhimpundu MA, Dushime T, Senyana F, Mazarati JB, Gaju CM, Tuyisenge L, Mutabazi V, Kyamanywa P, Rusanganwa V, Nyemazi JP, Umutoni A, Kankindi I, Ntizimira C, Ruton H, Mugume N, Nkunda D, Ndenga E, Mubiligi JM, Kakoma JB, Karita E, Sekabaraga C, Rusingiza E, Rich ML, Mukherjee JS, Rhatigan J, Cancedda C, Bertrand-Farmer D, Bukhman G, Stulac SN, Tapela NM, Van Der Hoof Holstein C, Shulman LN, Habinshuti A, Bonds MH, Wilkes MS, Lu C, Smith-Fawzi MC, Swain JD, Murphy MP, Ricks A, Kerry VB, Bush BP, Siegler RW, Stern CS, Sliney A, Nuthulaganti T, Karangwa I, Pegurri E, Dahl O, Drobac PC. Rwanda 20 years on: Investing in life. Lancet. 2014;384(9940):371–375. [PMC free article: PMC4151975] [PubMed: 24703831]
• Binagwaho A, Kankindi I, Kayirangwa E, Nyemazi JP, Nsanzimana S, Morales F, Kadende-Kaiser R, Scott KW, Mugisha V, Sahabo R, Baribwira C, Isanhart L, Asiimwe A, El-Sadr WM, Raghunathan PL. Transitioning to country ownership of HIV programs in Rwanda. PLoS Medicine. 2016;13(8):e1002075. [PMC free article: PMC4978459] [PubMed: 27505355]
• Cancedda C, Cotton P, Shema J, Rulisa S, Riviello R, Adams LV, Farmer PE, Kagwiza JN, Kyamanywa P, Mukamana D, Mumena C, Tumusiime DK, Mukashyaka L, Ndenga E, Twagirumugabe T, Mukara KB, Dusabejambo V, Walker TD, Nkusi E, Bazzett-Matabele L, Butera A, Rugwizangoga B, Kabayiza JC, Kanyandekwe S, Kalisa L, Ntirenganya F, Dixson J, Rogo T, McCall N, Corden M, Wong R, Mukeshimana M, Gatarayiha A, Ntagungira EK, Yaman A, Musabeyezu J, Sliney A, Nuthulaganti T, Kernan M, Okwi P, Rhatigan J, Barrow J, Wilson K, Levine AC, Reece R, Koster M, Moresky RT, O'Flaherty JE, Palumbo PE, Ginwalla R, Binanay CA, Thielman N, Relf M, Wright R, Hill M, Chyun D, Klar RT, McCreary LL, Hughes TL, Moen M, Meeks V, Barrows B, Durieux ME, McClain CD, Bunts A, Calland FJ, Hedt-Gauthier B, Milner D, Raviola G, Smith SE, Tuteja M, Magriples U, Rastegar A, Arnold L, Magaziner I, Binagwaho A. Health professional training and capacity strengthening through international academic partnerships: The first five years of the Human Resources for Health Program in Rwanda. International Journal of Health Policy and Management. 2018;7(11):1024–1039. [PMC free article: PMC6326644] [PubMed: 30624876]
• CDC (U.S. Centers for Disease Control and Prevention). Funding opportunity announcement: Clinical services 5-year proposed work plan. Atlanta, GA: U.S. Centers for Disease Control and Prevention; 2009a. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• CDC. Funding opportunity announcement: Strengthening the Ministry of Health's capacity to respond to the HIV/AIDS epidemic in the Republic of Rwanda under PEPFAR. Atlanta, GA: U.S. Centers for Disease Control and Prevention; 2009b. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• CDC. U.S. Department of Health and Human Services Centers for Disease Control and Prevention strengthening the Ministry of Health's capacity to respond to the HIV/AIDS epidemic in the Republic of Rwanda under the President's Emergency Plan for AIDS Relief (PEPFAR) funding opportunity. Atlanta, GA: U.S. Centers for Disease Control and Prevention; 2009c.
• CDC. Request approval to award a multi-year expansion supplement exceeding 25% for the cooperative agreement PS09-9118, 3U2GPS2091 “Strengthening the Ministry of Health's Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the President's Emergency Plan for AIDS Relief (PEPFAR)” and approval to waive the requirement of an objective review for the single awardee to this cooperative agreement. Atlanta, GA: U.S. Centers for Disease Control and Prevention; 2012. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• CDC. Funding opportunity announcement: Strengthening human resources for health capacity in the Republic of Rwanda under the President's Emergency Plan for AIDS Relief (PEPFAR) CDC-RFA-GH15-1568. Atlanta, GA: U.S. Centers for Disease Control and Prevention; 2014. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• Flinkenflögel M, Kyamanywa P, Asiimwe-Kateera B, Musafiri S, Kayumba P, Irakoze M, Cotton P, Hibble A. Umuganda for improved health professions education in Rwanda: Past, present and future in the training of health professionals at the University of Rwanda. Rwanda Journal. 2015;2:96.
• Fourth High-Level Forum on Aid Effectiveness. Busan partnership for effective development co-operation: Fourth High-Level Forum on Aid Effectiveness, Busan, Republic of Korea, 29 November–1 December 2011. Paris, France: Organisation for Economic Co-operation and Development; 2011.
• Gavi (Gavi, the Vaccine Alliance). Republic of Rwanda: Programme audit of Gavi support to the Ministry of Health. Geneva, Switzerland: Gavi, the Vaccine Alliance; 2018.
• Global Fund (Global Fund to Fight AIDS, Tuberculosis and Malaria). Audit report: Audit of global fund grants to Rwanda. Geneva, Switzerland: Global Fund; 2014. [PMC free article: PMC3904874] [PubMed: 24489702]
• Global Fund. Audit follow up report: Global fund grants to Rwanda. Geneva, Switzerland: Global Fund; 2019.
• Goosby E, Dybul M, Fauci AS, Fu J, Walsh T, Needle R, Bouey P. The United States President's Emergency Plan for AIDS Relief: A story of partnerships and smart investments to turn the tide of the global AIDS pandemic. Journal of Acquired Immune Deficiency Syndrome. 2012a;60(Suppl 3):S51–S56. [PubMed: 22797740]
• Goosby E, Von Zinkernagel D, Holmes C, Haroz D, Walsh T. Raising the bar: PEPFAR and new paradigms for global health. Journal of Acquired Immune Deficiency Syndrome. 2012b;60(Suppl 3):S158–S162. [PubMed: 22797738]
• Government of Rwanda. Vision 2020, revised 2012. Kigali, Rwanda: Government of Rwanda; 2012.
• IOM (Institute of Medicine). Evaluation of PEPFAR. Washington, DC: The National Academies Press; 2013.
• Kayirangwa E, Hanson J, Munyakazi L, Kabeja A. Current trends in Rwanda's HIV/AIDS epidemic. Sexually Transmitted Infection. 2006;82(Suppl 1):i27–i31. [PMC free article: PMC2593071] [PubMed: 16581756]
• MOH (Ministry of Health). Health sector policy, February 2005. Kigali, Rwanda: Ministry of Health; 2005a.
• MOH. Health Sector Strategic Plan, 2005–2009. Kigali, Rwanda: Ministry of Health; 2005b.
• MOH. Health Sector Strategic Plan, July 2009–June 2012. Kigali, Rwanda: Ministry of Health; 2009a.
• MOH. National Strategic Plan on HIV and AIDS 2009–2012. Kigali, Rwanda: Ministry of Health; 2009b.
• MOH. Clinical services work plan. Kigali, Rwanda: Ministry of Health; 2010. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Human Resources for Health Strategic Plan, 2011–2016. Kigali, Rwanda: Ministry of Health; 2011a.
• MOH. Rwanda Human Resources for Health Program, 2011–2019: Funding proposal. Kigali, Rwanda: Ministry of Health; 2011b.
• MOH. U.S. academic institutions—selection rationale. Kigali, Rwanda: Ministry of Health; 2011c. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. HRH application brief cabinet paper, May 2012–30 Sept 2013. Kigali, Rwanda: Ministry of Health; 2012a. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. HRH U.S. Academic Consortium budget years 1 & 2. Kigali, Rwanda: Ministry of Health; 2012b. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Rwanda health resource tracker output report for FY 2009–10 exp. & FY 2010–11 budget & FY 2010–11 exp. & FY 2011–12 budget. Kigali, Rwanda: Ministry of Health; 2012c.
• MOH. Third Health Sector Strategic Plan, July 2012–June 2018. Kigali, Rwanda: Ministry of Health; 2012d.
• MOH. Work plan. Kigali, Rwanda: Ministry of Health; 2012e. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. CDC-RFA-PS09-9118-05CONT13: Strengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the President's Emergency Plan for AIDS Relief (PEPFAR). Kigali, Rwanda: Ministry of Health; 2013a. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Rwanda health resource tracker output report December 2013: Expenditures FY 2011/12 and budget FY 2012/13. Kigali, Rwanda: Ministry of Health; 2013b. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Annual report on the implementation of CDC-funded projects, period: October 2012-September 2013. Kigali, Rwanda: Ministry of Health; 2014a. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Human Resources for Health monitoring & evaluation plan, March 2014. Kigali, Rwanda: Ministry of Health; 2014b. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. National Human Resources for Health Policy, October 2014. Kigali, Rwanda: Ministry of Health; 2014c.
• MOH. Project narrative. Kigali, Rwanda: Ministry of Health; 2014d. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Health sector policy, January 2015. Kigali, Rwanda: Ministry of Health; 2015a.
• MOH. Programmatic/technical work plan. Kigali, Rwanda: Ministry of Health; 2015b. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Rwanda health resource tracking output report: Expenditure FY 2013/14 and budget FY 2014/15. Kigali, Rwanda: Ministry of Health; 2016a.
• MOH. Rwanda Human Resources for Health Program midterm review report (2012–2016). Kigali, Rwanda: Ministry of Health; 2016b. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• MOH. Fourth Health Sector Strategic Plan, July 2018–June 2024. Kigali, Rwanda: Ministry of Health; 2018a.
• MOH. Rwanda health resource tracking output report: Expenditure FY 2014/15 and budget FY 2015/16. Kigali, Rwanda: Ministry of Health; 2018b.
• MOH. Rwanda HIV and AIDS National Strategic Plan 2013–2018: Extension: 2018–2020. Kigali, Rwanda: Ministry of Health; 2018c.
• Mtiro E. Termination of Rwanda cooperative agreement. Kigali, Rwanda: 2018. (Available by request from the National Academies Public Access Records Office [paro@nas​.edu] or via https://www8​.nationalacademies​.org/pa/managerequest​.aspx?key=HMD-BGH-17-08.
• Nsanzimana S, Prabhu K, McDermott H, Karita E, Forrest JI, Drobac P, Farmer P, Mills EJ, Binagwaho A. Improving health outcomes through concurrent HIV program scale-up and health system development in Rwanda: 20 years of experience. BMC Medicine. 2015;13(1):216. [PMC free article: PMC4564958] [PubMed: 26354601]
• PEPFAR (President's Emergency Plan for AIDS Relief). Guidance for PEPFAR partnership frameworks and partnership framework implementation plans. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2009. [September 14, 2009]. Version 2.0.
• PEFPAR. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2010.
• PEPFAR. FY 2012 country operational plan (COP) guidance. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2011.
• PEPFAR. FY 2013 country operational plan (COP) guidance. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2012. Version 2.
• PEPFAR. PEPFAR 3.0: Controlling the epidemic; delivering on the promise of an AIDS-free generation. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2014.
• PEPFAR. Executive summary of PEPFAR's strategy. 2019a. [accessed February 24, 2019]. https://www​.pepfar.gov​/about/strategy/document/133244.htm.
• PEPFAR. PEPFAR's five-year strategy. 2019b. [February 21, 2019]. https://www​.pepfar.gov​/about/strategy/document/133251.htm.
• PEPFAR. PEPFAR panorama spotlight. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2019c.
• PEPFAR. PEPFAR Panorama spotlight glossary. 2019d. [October 11, 2019]. https://data​.pepfar.gov/glossary.
• PEPFAR Rwanda. Populated printable COP without TBD partners 2008 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2008.
• PEPFAR Rwanda. Populated printable COP 2009 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2009.
• PEPFAR Rwanda. Rwanda operational plan report FY 2010. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2010.
• PEPFAR Rwanda. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2011.
• PEPFAR Rwanda. Rwanda country operational plan FY 2012. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2012.
• PEPFAR Rwanda. Rwanda operational plan report FY 2013. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2013.
• PEPFAR Rwanda. Rwanda operational plan report FY 2014. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2014.
• PEPFAR Rwanda. Rwanda country operational plan (COP) 2015 strategic direction summary. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy; 2015.
• Seymour B, Muhumuza I, Mumena C, Isyagi M, Barrow J, Meeks V. Including oral health training in a health system strengthening program in Rwanda. Global Health Action. 2013;6:1–6. [PMC free article: PMC3593139] [PubMed: 23473054]
• TAGGS (Tracking Accountability in Government Grants System). Ministry of Health Rwanda. 2019. [July 9, 2019]. https://taggs​.hhs.gov.
• Tvedten I, Byabagambi A, Lindstrom J, Tedre M. Evaluation of the SIDAsupported research capacity and higher education development program in Rwanda, 2013–2017. Stockholm, Sweden: Swedish International Development Cooperation Agency; 2018.
• USAID (United States Agency for International Development). Rebuilding postwar Rwanda: The role of the international community. Washington, DC: United States Agency for International Development; 1996.
• Uwizeye G, Mukamana D, Relf M, Rosa W, Kim MJ, Uwimana P, Ewing H, Munyiginya P, Pyburn R, Lubimbi N, Collins A, Soule I, Burke K, Niyokindi J, Moreland P. Building nursing and midwifery capacity through Rwanda's Human Resources for Health Program. Journal of Transcultural Nursing. 2018;29(2):192–201. [PubMed: 28826335]
• WHO (World Health Organization). Health workforce requirements for universal health coverage and the Sustainable Development Goals. Geneva, Switzerland: World Health Organization; 2016. [PMC free article: PMC3853967] [PubMed: 24347717]
• WHO. Global health expenditure database. Geneva, Switzerland: World Health Organization; 2019.
• Willis BM, Levy BS. Recognizing the public health impact of genocide. JAMA. 2000;284(5):612–614. [PubMed: 10918708]
• World Bank. Current health expenditure per capita, PPP (current international $)—Rwanda. 2019a. [November 4, 2019]. https://data​.worldbank​.org/indicator/SH.XPD​.CHEX.PP.CD?locations=RW.
• World Bank. Domestic general government health expenditure per capita, PPP (current international $)—Rwanda. 2019b. [November 4, 2019]. https://data​.worldbank​.org/indicator/SH.XPD​.GHED.PP.CD?locations=RW.
• World Bank. GDP (current US$)—Rwanda. 2019c. [November 4, 2019]. https://data​.worldbank​.org/indicator/NY.GDP​.MKTP.CD?locations=RW.
• World Bank. GDP per capita, PPP (current international $)—Rwanda. 2019d. [November 4, 2019]. https://data​.worldbank​.org/indicator/NY.GDP​.PCAP.PP.CD?locations=RW.
• World Bank. General government final consumption expenditure (current US$). 2019e. [November 4, 2019]. https://data​.worldbank​.org/indicator/NE.CON.GOVT.CD.
• World Bank. Inflation, consumer prices (annual %)—Rwanda. 2019f. [November 4, 2019]. https://data​.worldbank​.org/indicator/FP.CPI​.TOTL.ZG?end=2018&locations​=RW&start=2000.
• World Bank. Official exchange rate (LCU per US$, period average). 2019g. [November 4, 2019]. https://data​.worldbank​.org/indicator/PA.NUS​.FCRF?locations=RW.
• World Bank. PPP conversion factor, GDP (LCU per international $)—Rwanda. 2019h. [November 4, 2019]. https://data​.worldbank​.org/indicator/PA.NUS.PPP?locations=RW.