What is GEAR and How to use GEAR?

What is GEAR Online Resource?

To start, we introduce you to GEAR online resource’s several features:

  • Mind mapping that provides solutions (long- and short-term) to common and important methodological difficulties
  • Comparisons of recommendations from selected health economics guidelines
  • “Ask an Expert,” a question-and-answer feature that allows users to pose questions to experts on issues that are not dealt with in the other features of the website

Understanding GEAR

Do you want to understand a little more on what GEAR can offer to you? Watch this animation that explains in the simplest way possible the features of GEAR. 

GEAR Project Background & Purpose

You are a researcher who is responsible for conducting an economic evaluation to be used as evidence for healthcare policymaking in your country. After doing some research, you’re sitting in front of your computer, trying to decide what to do now that you’re facing creation of your model.

What type of modelling should you do?

What health outcome should you use?

How do I overcome data limitations in conducting economic evaluation for my country?

How do I correctly present my research results?

These are some of the questions that researchers contend with when they begin working on economic evaluations. To address these concerns, the Guide to Economic Analysis and Research (GEAR) online resource was born.

GEAR provides various ways to solve specific methodological difficulties researchers may encounter in the conduct of their studies. The resource is designed as a global public good dedicated towards helping low- and middle- income countries (LMICs) academics, researchers and economic evaluation practitioners worldwide conduct high quality, policy relevant healthcare research. GEAR compiles and resolves gaps arising from methodological issues in the conduct and the use of economic evaluations. The resource will explore the issues in the conduct and the use of these evidences, potential solutions to the issues and future research questions to address these issues but also will.

The site is based on the working paper of a HITAP International Unit study in 2015 that the International Decision Support Initiative (iDSI) supported. The paper, “Identifying Priority Methodological Issues in Economic Evaluation in Low- and Middle-Income Countries: Finding the Holy Grail,” details the results of a literature review and questionnaire used to survey national and international experts, academics, public health officials, and relevant stakeholders on economic evaluation methodological issues. The respondents ranked the issues by order of priority or importance per their own experiences working in their country and/or abroad as well as propose some solutions that they consider relevant. These results were analysed, triangulated with research questions proposed by the research team based on the solutions nominated, and presented on the database. The prioritization of methodological problems and finding solutions to these issues will lead to methodological research that will yield improved tools for the conduct of economic evaluations. This database complements this work and provides immediate solutions to researchers’ needs.

The GEAR online resource can be a useful guide to researchers. However, after consulting the GEAR and you’re confronting your research again, remember the wise words from the great scientist Albert Einstein:

“After a certain high level of technical skill is achieved, science and art tend to coalesce in aesthetics, plasticity, and form. The greatest scientists are always artists as well.”

The GEAR online resource is supported by the International Decision Support Initiative (iDSI), through the grant from the Bill and Melinda Gates Foundation (BMGF) and the Department of International Development (DFID), UK, and the Thailand Research Fund (TRF).

Purpose 1

To explore the methodological issues that researchers face in low- and middle-income countries

Purpose 2

To link HTA methodological issues to different potential solutions

Purpose 3

To provide the visual mapping of methodological issues that lead to potential methodologies

Purpose 4

To explore guidelines and other governing issues that researchers encounter

Purpose 5

To act as an open discussion forum for methodological issues between experts and researchers globally

Purpose 6

To ensure the continued improvement of economic evaluation methods through surveys and analysis of GEAR content and updates

Purpose 7

To provide an avenue for registered users access to referenced research papers as well as download mind maps, reports, and materials in the GEAR

Purpose 8

To allow registered users to participate in and have information on the surveys (e.g. methodology, demographics, research questions)

Method & Process

    Economic evaluations are increasingly becoming of interest to low- and middle-income (LMIC) governments, especially those keen to make efficient health care systems. However, economic evaluations can be useful for policy decisions only when they are conducted accurately and reported properly. Because economic evaluation is a relatively new discipline in LMICs compared to high-income countries (HICs), it’s important to learn the methodological problems in conducting economic evaluations in LMICs. Information from this study and the GEAR database can help inform scholars to overcome these methodological barriers in both the short (providing short-term advice and solutions) and long term (e.g. better tools based on methodological research).

A. Identifying Methodological Issues in Low- and Middle-Income Countries

The Guide to Economic Evaluation and Research (GEAR) online resource is based on the study entitled “Identifying Methodological Issues in Low- and Middle-Income Countries (LMICs).” This study aims to determine the research questions that arise from their results and can be explored to contribute to better quality economic evaluations evidence for health resource allocation. To achieve this, in January 2015, a team of HITAP’s international unit conducted a comprehensive review of literature identifying issues for conducting economic evaluations. Then, the issues identified from the review were used to inform a survey sent to international scholars working all over the world who have conducted economic evaluation in LMICs to prioritize the issues as they have observed in their work. Results from this survey were analysed and outlined as prioritized gaps which were discussed in a face-to-face consultation meeting with policy-makers and HTA practitioners working in LMIC settings in January 2016. This project has two main outputs which will showcase the results and additional research in different ways: the study and the GEAR. The results after discussions were inputted into an electronic database to inform the global community. The database will be updated biennially to facilitate the accuracy and continued relevance of the methodological issues addressed on the database for LMICs.
Diagram 1: The three-step research study design: from the literature review (1), to the survey (2), and finally, to consultation (3).

Potential methodological issues that might be hindrances to the conduct of economic evaluation in low- and middle-income settings were identified through the review of literature in PubMed and University of York’s Centre for Research and Dissemination (CRD) databases.
Key publications discussing the barriers, flaws or difficulties in the conduct of economic evaluations were retrieved from PubMed and supplemented by reference search. The methodological issues in both high-income and low- and middle-income counties (LMICs) were included since although there were differences in the stage of the research between these settings, the issues in the high-income settings also tended to exist in the LMICs. Context-specific issues were cited as barriers to acceptance of economic evaluations in decision making and were therefore included in the study.
The CRD database was employed owing to its critical appraisal of economic evaluation studies. From the CRD, primary economic evaluations of which a commentary is available were identified through the search of LMICs name based on World Bank classifications as of 2015. The search returned 568 hits of which 180 studies were done in LMICs. Of these, commentaries of 100 studies were randomly selected for review of the criticism on its methodological problems and weaknesses. The issues identified from the review were summarised and categorised based on types of methodological issues: on interventions/comparators (e.g. did not use standard comparator), study population (e.g. no mention of randomisation), effectiveness/benefits (no sensitivity and specificity), costs (e.g. discounting not used even when appropriate), and analysis and results (e.g. no discussion on limitations and/or transferability).

The results of the literature review informed the content of a questionnaire that was disseminated globally to researchers and research assistants that have conducted or currently working on economic evaluations in low- and middle-income countries (LMICs). These were divided into two lists – one on methodological challenges and another on context-specific challenges. For both methodological and context-specific components, the respondents were asked to consider the list of issues provided, add an issue that they thought was relevant but not yet on the list, and rank the top 3 methodological issues and the top context-specific issue that they considered most important in their contexts. They were also asked to propose possible solutions and to choose whether the methodological or the context-specific issues were more important.
The study population comprised researchers who had completed at least one economic evaluation project as the primary investigator or as a part of a team in low- and middle-income settings. The low- and middle-income settings are defined per World Data Bank classifications as of 2015. Both individuals and networks were approached to be respondents. Invitation to participate in the survey was sent to the secretariat of health economic and outcome research networks as well as various health technology assessment networks in different regions for the secretariats to distribute the survey to their members. Individuals were also identified through snowball approach (existing respondents recruit future respondents from their network). In total, there were 927 respondents. However, since the mailing lists of each network are classified and respondents were mostly contacted through the network, there is a possibility of duplication which cannot be eliminated and the exact number of recruited respondents may be less than the figure shown.

Who answered our survey and told us what challenges they face in health economic evaluations? What are they like? We have the following graphs to show you the characteristics of our respondents and understand the background of the results of our study.

The top priority methodological and context-specific issues were analysed separately. Since there is more than one methodological issue, a Likert type of system was applied to rank the methodological issues. For each response, the methodological issue that was ranked first would be assigned the score of 3, 2 for the second rank, and 1 for the third rank. The score for each issue in the components was then ranked in a league table to derive the list of top priorities. Where there was more than one issue that received the same score, their frequencies in ranking amongst the higher levels would be considered. The results of the ranked issues were then analysed in terms of WHO regions, and by respondent affiliation and experience. The solutions that fell within the same concerns or areas or same direction were grouped together.
For more information on the study methodology, results, and discussion, please download the following full file of the original report:

The prioritized list of methodological issues and context-specific issues are found in the GEAR section of the website. This list informed the creation of the mind maps and the potential solutions outlined. Guideline recommendations were taken from the survey results. Further, the list was used to inform the research questions that would be relevant to pursue, and these will also be ranked on the website.
The content in the GEAR will be continuously updated through the interactive discussions in the Ask an Expert page and other online content as well as through the biennial survey that will be based on the original survey.

B. Guide to Economic Evaluation and Research Online Content

The study “Identifying Methodological Issues in Low- and Middle-income Countries (LMICs)” yielded methodological questions to pursue in the future and improve the quality of economic evaluations in LMICs. Through improved tools for economic evaluation, the quality and usability of this kind of evidence will be boosted and the decision-making informed by these evidences will also be better. This will call for sustainable production of evidence and even higher quality and usability evidences. However, the research team foresaw that filling the gaps through research will require substantial amount of time; therefore, providing academics, researchers and economic evaluation practitioners ‘immediate solutions’ which could be applied to tackle methodological difficulties they are currently facing is as important and needed. It aims to be a part of the cycle in Theory of Change of Methodological Research Development outlined below (Diagram 2).
The ultimate goals of this project are to develop a growing, up-to-date database of issues, solutions and possible research questions to fill the gaps, to provide a unique and trustworthy platform for HTA practitioners to overcome methodological challenges that will respond to queries in a timely manner with up-to-date information, to offer neutral and best available advice to practitioners without advocating any particular method and to function as a global public good, and to which the global community can collectively contribute (i.e. a Wiki approach). With more use of this database, the underlying hope is that capacities in economic evaluation, both to generate and to use the evidences, will be improved.
Diagram 2: GEAR Database Theory of Change

There is a wide range of health economic evaluations guidelines available whereas the details of methodological specifications and perspective vary according to the purpose of these guidelines. Several guidelines were selected and compared in terms of the issues mainly related to the identified methodological difficulties from the survey. Comparing similarities and differences between these guidelines allows exploring the acceptability and applicability of existing guidelines in LMICs. Guidelines included on the GEAR database were selected to capture different perspectives from different guidelines (e.g. guidelines aiming to use either/both in LMICs or/and in HICS). More details in the rationale for included guidelines are found here.

Experts are those with considerable contribution and influence in the field of health economics in specific areas, geographic regions, and/or experience. They have published, taught, and researched extensively on one or more subjects of interest and can reliable provide information on such subjects, e.g. cost-effectiveness thresholds, costing, meta-analysis, etc. Accordingly, each expert will be assigned “tags” that denote to their particular expertise and questions from GEAR users that apply to these tags may fall under their remit. The GEAR database experts are required to respond within 2 weeks of the question’s receipt. Experts are recruited on a voluntary basis through networks and recruitment campaigns, though they undergo an internal verification and selection process through the iDSI before they are included on the panel. Non-response or late responses (receipt after exceeding 2 weeks) to questions may result to experts’ removal from the panel.

The GEAR addresses the short-term and long-term problems researchers face when conducting research, and this is visually represented through the mind map. The right-side provides researchers with immediate solutions for their current challenges based on available information. The left-side shows researchers which questions arise from the methodological challenges presented and the hypotheses they are based on.
Diagram 3: The basic set-up of a mind map on the GEAR database

iDSI

The International Decision Support Initiative (iDSI) is a sustainable, adaptable, international mechanism to provide policymakers (at sub-national, national, regional and international levels) with coordinated support in priority-setting as a means to Universal Health Coverage (UHC). The initiative shares experiences, showcases lessons learned, and identifies practical ways to scale technical support for more systematic, fair and evidence informed priority-setting processes. Its interventions help to improve access to effective health interventions and the quality and efficiency of health care delivery, and to help elevate the value of priority setting as essential for attaining and sustaining UHC.

For more information, visit the page: http://www.idsihealth.org

Kalipso Chalkidou
Amanda Glassman
Karen Hofman
Anthony Culyer
Tommy Wilkinson
Ryan Li
Francis Ruiz

HITAP | HIU

The Health Intervention and Technology Assessment Program (HITAP) is a semi-autonomous research unit under Thailand’s Ministry of Public Health. It was established in 2007 as a non-profit organization in order to take responsibility for appraising a wide range of health technologies and programs, including pharmaceuticals, medical devices, interventions, individual and community health promotion, and disease prevention as well as social health policy to inform policy decisions in Thailand. The work of HITAP have been used to informed coverage decisions of those health technologies under the public health plans.

In 2013, HITAP established a HITAP International Unit (HIU), drawing on its experiences locally and internationally, to work at the global level with overseas development aids, international organizations, non-profit organizations, and overseas governments to build capacity for health technology assessment. HIU is currently working with ministries of health in various countries in the region such as Bhutan, India, Indonesia, Myanmar, Nepal, the Philippines, Vietnam.

For more information, visit pages: http://www.hitap.net/en and http://www.globalhitap.net

Francis Carlo Panlilio
Benjarin Santatiwongchai
Serah Clarence
Nann Vongpuapan
Saudamini Dabak
Waranya Rattanavipapong
Yot Teerawattananon

Thailand Research Fund

The Thailand Research Fund (TRF) was established in response to the 1992 Research Endowment Act and although it is part of the government system, it lies outside the government administrative bureaucracy. This freedom allows great efficiency in research support. Since its creation, TRF has remained focused on its main duty: supporting at both local and national levels the creation of a knowledge-base to help those tackling societal problems. TRF’s main role is to assist in the development of researchers and research-based knowledge through making research grants and assisting with research management.

For more information, visit the page: http://www.trf.or.th/eng