BACKGROUND

Tanzania has committed to the progressive realization of Universal Health Coverage (UHC) in 2030 stated in their Fifth Health Sector Strategic Plan.1 In 2017 the Tanzanian Ministry of Health, Community Development, Gender, Elders and Children (MOHCDGEC) established an advisory committee - Tanzanian Health Technology Assessment Committee (THTAC) - in recognition of the increasing role of Health Technology Assessment (HTA) to achieve UHC.2–5 HTA is a multidisciplinary process that uses explicit methods to determine the value of health technology at different points in its lifecycle. The purpose is to inform decision-making to promote an equitable, efficient, and high-quality health system.6

An advisory committee is a central element of an HTA process for benefits package design, as recognized by several frameworks7–10 and implemented by many HTA bodies worldwide.7 A typical advisory committee involves members representing specific expertise and/or interests, with the mandate to develop recommendations on technology reimbursement for decision-makers. In that role, committee members need to combine scientific and social judgments vis-a-vis the technology under evaluation.7 It is widely recognized that there are challenges to the technical capacity and human resource availability of HTA worldwide, including decision-making committees.11 Committee members essentially use experience and discretionary judgment favouring scientific evidence to inform decisions.12 International guidance recommends training for committee members in the principles of health research, including evidence-based medicine, economic analysis and health systems research which will improve their decision making.7

The THTAC in Tanzania advises the Ministry on including (or excluding) health technologies in the Essential Package of Health Services (EPHS). It includes approximately 20 committee members appointed by the Permanent Secretary - comprising senior representatives from key departments in health, finance and planning, and representation from academia and health associations. The committee chair is the Chief Medical Officer (CMO), and the secretary is the Chief Pharmacist (CP). The secretariat for the committee sits under the Pharmaceutical Services Unit (PSU) within the MOHCDGEC.

This paper reports on a self-administered survey among committee members to assess their capacity for the tasks at hand. Its results indicate the type of evidence an advisory committee can reasonably handle, can guide future capacity strengthening activities and serve as a baseline for measuring the impact of capacity strengthening programmes.

METHODOLOGY

The Study Design

The survey questions were developed regarding the definition of ‘capacity building in the context of HTA’ provided by the HTAi Scientific Development and Capacity Building (SDCB) Committee:

the process by which individuals and organizations develop or strengthen abilities related to understanding, providing input to, conducting, or utilizing HTA for health policy and decision making, as well as developing awareness and support in the environment within which HTA is being used.13

The survey was piloted by members of the review team and refined based on feedback from the exercise. It included questions on qualifications, experience with HTA methods such as literature reviews and economic analysis, and ratings of confidence, ability or knowledge on technical skills. Since the survey was self-reported, true/false questions were used to verify some understanding of related aspects of HTA.

A leadership and management section assessed confidence in defining tasks, planning and coordinating activities for teams through effective planning and management. Respondents were asked to rate their level of knowledge and confidence on a scale from one to five (low to high).

The assessment in Tanzania was predominately conducted using an online questionnaire however if members were unable to complete the survey online, members completed the questionnaire on a hard copy which was then transcribed by the research team. Information on the survey and a link were sent to all the THTAC members in the second quarter of 2018.

RESULTS

Out of 20 committee members, 15 members completed the survey (response rate 75%). Respondents were primarily representatives from the Ministry of Health but also from the National Health Insurance Fund (NHIF), Commission for Science and Technology (COSTEC), National Institute of Medical Research (NIMR), Tanzania Food and Drugs Authority (TFDA), Pharmacy Council, Tanzania Health Laboratory Services Organisation, Medical Stores Department (MSD), Ministry of Finance and Planning and the Tanzania Diabetes Association.

All committee members had a higher-level qualification with most having experience related to pharmaceuticals. The gender of committee members was equally split between men and women. The majority had previously sat on an expert committee that made decisions concerning medicines (Table 1).

Table 1.Response Rate and Characteristics of Respondents
    N %
Survey responses Distributed 20 100%
Completed 15 75%
Incomplete 1 5%
Sex of respondents Male 8 50%
Female 8 50%
Employment MoHCDGEC 7 44%
Other govt. 8 50%
Non govt. 1 6%
Qualifications Post-graduate diploma/ professional registration 1 6%
Bachelors 3 19%
Masters 8 50%
Doctorate 4 25%
Expertise Pharmaceuticals 9 56%
Other 7 44%
Committee experience Expert committee member 11 69%
Years on an expert committee >10 years 3 20%
7-10 years 0 0%
5-7 years 2 13%
3-5 years 1 7%
1-3 years 5 33%
0 years 4 27%

MoHCDGEC: Ministry of Health, Community Development, Gender, Elders and Children

Approximately half of the respondents had experience with literature reviews and publications however, few had been involved in randomized control trials (RCT) or economic evaluations (Table 2).

Table 2.Experience in conducting research and publications
N %
No. with systematic reviews of clinical/medical literature conducted 1-5 5 33%
5+ 2 13%
No. with peer reviewed publications 1-5 5 33%
5+ 5 33%
Years’ experience in health research 1-5 8 57%
5-10 2 13%
10+ 5 40%
No. with experience in randomized control trials   5 33%
No. with experience in economic evaluation   2 13%

Most respondents expressed a very low to average confidence, ability and knowledge in conducting HTA methods such as systematic literature reviews, cost-effectiveness analysis or health inequity analysis. However, at the same time, most respondents expressed an average to very high confidence in the synthesis of clinical evidence, interpretation and appraisal of studies (Table 3).

Table 3.Confidence, ability or knowledge on technical skills related to HTA
  Level of confidence, ability or knowledge
  1
Very low		 2
Low		 3
Average		 4
High		 5
Very high
conducting systematic literature reviews 20% 20% 27% 20% 13%
determining cost-effectiveness 20% 33% 27% 13% 7%
differences between economic evaluation methods 27% 20% 27% 13% 7%
PICO (patient/population, intervention, comparison) 43% 7% 36% 7% 7%
decisions impact on healthy inequity 21% 21% 43% 7% 7%
ability to synthesize clinical evidence 21% 14% 14% 21% 29%
ability to conduct and interpret literature reviews (scoping, systematic, literature) 7% 21% 21% 14% 36%
ability to appraise the economic evaluation (cost analysis, cost effectiveness, cost utility, cost benefit, budget impact) 29% 14% 21% 21% 14%
ability to conduct critical appraisals of epidemiological studies 21% 14% 14% 29% 21%
ability to extract the PICO 21% 14% 14% 29% 21%
ability to interpret the statistical output such as odds ratios, risk ratios and the related chi-square, p-values, confidence intervals 21% 14% 14% 36% 14%
ability to determine the quality of scientific evidence (e.g. using the SORT tool) 43% 14% 7% 21% 14%

Less than half of the respondents correctly answered the verification questions (Table 4). The majority correctly responded to the cost-effectiveness and systematic reviews questions, but only a small proportion showed a correct understanding of the incremental cost-effectiveness ratio, willingness-to-pay threshold, and social welfare.

Table 4.Verification questions
Correctly answered
When a drug is cost-effective, it basically means it is cheap 86%
Systematic review and meta-analysis are the highest level of evidence one can use to inform decision-making as compared to a randomized clinical trial 71%
If a treatment policy change leaves at least one group better off and no group worse off, then we are happy to say social welfare has increased 43%
A drug is said to be cost-effective when the calculated ICER (incremental cost-effectiveness ratio )is higher than the willingness-to-pay threshold for the country 21%
Cost-effectiveness analysis and cost-benefit analysis are the same except that cost-benefit outcomes are measured in monetary terms (i.e. Dollars or Shillings) 7%

Respondents on average rated their confidence in leadership and management skills (Table 5) as average and above.

DISCUSSION

This study indicates the mixed capacity of the THTAC members to make sound scientific and social judgments on the inclusion of technologies in the HBP. On the one hand, committee members are generally well qualified and well-experienced in clinical medicine. However, on the other hand, they seem less qualified and experienced in HTA and express low confidence in this subject matter. More specifically, there were no members with expertise in economics, and there was only one member with a background in quantitative analyses (epidemiology). Also, only two committee members indicated a high understanding of the concept of equity. Equity is a priority of the Government of Tanzania,14 but it is an inherently complex concept. This indicates the need for capacity strengthening, particularly on economic evaluation and equity.

Our findings are in line with a study in Thailand which found that, among 14 national-level policymakers on coverage decisions, only four could define the concept of economic evaluation correctly.15 Another study showed that decision-makers generally find the concept behind the QALY challenging to understand.16

Low- and Middle-Income Countries (LMICs) face the challenge of building and retaining technical capacity. These are success factors for HTA organizations, and LMICs face the challenge of limited human resources with very few professionally trained individuals competent in HTA in areas such as clinical effectiveness and economic assessments.17

The survey has several limitations. HTA is a relatively new concept in Tanzania, and the questionnaire did not define terms, meaning respondents could have interpreted these differently. In addition, we used a self-reported questionnaire, and respondents may have over or underestimated their capacity and skills. The responses to our verification questions indicate this. Also, this survey was conducted in 2017 and since then, the committee’s membership has changed slightly.

An understanding of the committee’s capacity is useful for providing directions for further capacity strengthening and informing the organization of HTA in the country. The current committee has limited technical capacity on a few key aspects, and the processes should account for this. For example, the committee could benefit from the involvement of external consultants to increase its HTA capacity.

This survey also sheds light on what kind of evidence an advisory committee in a country like Tanzania might be able to absorb. This is relevant as more advanced methods in HTA and economic evaluation is being developed. These methods are only relevant to the extent their results can be correctly interpreted and used by an advisory committee.18

Table 5.Leadership and Management Skills
  Level of confidence, ability or knowledge
  1
Very low		 2
Low		 3
Average		 4
High		 5
Very high
Ethical, honest behaviour and model integrity in personal and professional engagements, in order to promote institutional confidence, trust and value-based practices 0% 0% 0% 23% 77%
Ability to communicate persuasively, exchange information and ideas and influence others to gain cooperation, commitment and support 0% 0% 8% 46% 46%
Ability to work collaboratively to achieve a shared goal 0% 0% 0% 23% 77%
Ability to effectively guide and direct people to achieve a goal or develop certain skills and tactics 0% 0% 8% 31% 62%
Ability to lead, encourage, inspire and support others to develop confidence and capability that will help them realize their full potential 0% 0% 0% 38% 62%
Ability to proactively plan, establish and allocate resources and monitor, evaluate and adjust work to accomplish goals 0% 0% 23% 31% 46%
Ability to work cooperatively with diverse teams and work groups 0% 0% 8% 62% 31%
Ability to define tasks and milestones that will ensure systemic planning and the accomplishment of prioritized objectives and quality results 0% 0% 31% 15% 54%
Ability to create and nurture multidisciplinary, diverse and dynamic teams that are interactive and can achieve and maintain excellence 0% 0% 15% 23% 62%
Ability to provide direction through the effective planning and management of activities, programmes and projects 0% 0% 15% 31% 54%
Ability to build cohesive and productive work and project teams in order to achieve the required outputs 0% 0% 15% 23% 62%
Ability to adapt the way in which ideas and information are communicated to ensure that the message is understood 0% 0% 0% 23% 77%
Ability to adapt to diverse cultural practices, customs, values and norms of individuals and groups 0% 0% 23% 23% 54%
Ability to communicate your position on a given issue in a systematic and convincing manner, to reach an agreement and acceptance by all parties involved 0% 0% 15% 23% 62%
Ability to meet challenging targets, in compliance with quality, time and diversity standards, while delivering the required business results 0% 0% 15% 46% 38%
Average     12% 31% 57%

ETHICAL APPROVAL

This study was conducted at the request of the Permanent Secretary of the Ministry of Health to understand the current capacity for Health Technology Assessment (HTA) among HTA committee members. The survey does not require information that is detrimental or harmful to the respondents. Respondents were informed about the purpose, analysis and dissemination process and were given the opportunity to input into the report based on the survey. For the above reasons, the research team did not request ethics approval for the study. As part of the survey, participants gave written informed consent for the use and publication of the participant data

DATA AVAILABILITY

All data underlying the results are available as part of the article, and no additional source data are required.

FUNDING

The work received funding from Bill & Melinda Gates Foundation (grant OPP1087363, ‘Establishing Priority Setting Institutions in Developing Countries’), the UK Department for International Development, and the Rockefeller Foundation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

ACKNOWLEDGEMENTS

We would like to thank the Tanzania Health Technology Assessment Committee for completing the survey. This research was conducted as part of the International Decision Support Initiative (www.idsihealth.org), a global initiative to support countries to get the most from every dollar they spend on health.

CONFLICT OF INTEREST

The authors completed the ICMJE Unified Competing Interest form at (available upon request from the corresponding author), and declare no conflicts of interest.

CORRESPONDENCE

Gavin Surgey, Radboud University Medical Center, Nijmegen, The Netherlands; gsurgey@gmail.com