Can information contained in Evidence-Based Medicine Summaries Help Answer Health Economic Questions?

Background: High quality meta-analyses, systematic reviews, and structured literature reviews are extremely useful for understanding the quality, and strength of published findings. Many published studies are suboptimal – lacking rigor, statistical power, or sufficiently specified models - a particular concern for cost-related studies. High quality review studies, however, are time consuming. Efficiency, and perhaps quality, gains would be afforded by an electronically assisted approach to data extraction.

Aim: The purpose of this evaluation is to assess the extent to which evidence-based medicine (EBM)-oriented summaries of studies from published literature can be used for answering health economics questions.

Research Questions: 1. Is an electronic tool that was developed to help analyze EBM-related outcomes applicable to the field of health economics? 2. If so, does the tool increase efficiency and quality of the economic literature searches and analyses?

Methods: Multi-step analysis beginning with base-line comparison of clinical outcomes for heart disease identified from summaries of EBM generated by an electronic tool against outcomes identified and extracted via human review. Next the analysis was repeated but expanded to include economic outcomes and the comparison (electronic tool vs. human review). Results of the accuracy/quality were compared across the first and second analyses as well as across electronically assisted vs. manual data extraction.  

Findings: The initial test corpus consists of 117 clinical trial abstracts obtained from PubMed using the query term “myocardial ischemia.” The electronic tool was able to compute a correct absolute risk reduction (ARR) value for 53% (57 of 117) of the abstracts. When applied to economic outcomes, considerably fewer data points as compared to clinical outcomes would be identified by either review means. As with human review, the electronic tool’s accuracy/quality in identifying economic outcomes is contingent upon each manuscript’s content and the authors’ specification of outcomes. 

Conclusion: Automating the process of extracting outcomes from published literature affords positive time efficiencies with only a minor quality-effect. The findings provide insights with which to adapt an electronic tool to better serve health economics research. This initial analysis focused on heart disease, additional evaluation will help determine the applicability of the findings to other disease states and outcomes.