A review of NICE appraisals of pharmaceuticals 2000-2016 found variation in establishing comparative clinical effectiveness

Objective To identify and assess the methods for estimating comparative clinical effectiveness for novel pharmaceutical products licensed on the basis of non-RCT data and to evaluate the corresponding NICE recommendations. Methods Our identification strategy was two-fold. First, we reviewed all NICE appraisals between 2010 and 2016 and identified technologies where comparative clinical effectiveness estimates were calculated using non-RCT data. Second, we checked if NICE appraisals completed from 2000 to 2010 had included pharmaceuticals that were granted EMA marketing authorisation without RCT data between 1999-2014. Information was extracted on the methods used as well as the corresponding NICE recommendations. We also collected. Results Of 489 individual pharmaceutical technologies assessed by NICE, 22 (4%) used non-RCT data to estimate comparative clinical effectiveness. Methods for establishing external controls in such studies varied: 13 (59%) used published trials, 6 (27%) used observational data, 2 (9%) used expert opinion, and 1 (5%) used a responder vs non-responder analysis. Only 5 (23%) used a regression model to adjust for covariates. We did not observe a notable difference in the proportion of pharmaceutical technologies that received a positive recommendation from NICE whether the decision was based on RCT or non-RCT data. (83% vs 86%) Conclusions To date, the small number of appraisals by NICE based on non-RCT data did not result in substantially different treatment decisions. The majority of the technologies appraised on the basis of non-RCT data either received a positive recommendation or a positive recommendation with restrictions. The methods used to calculate comparative clinical effectivenes.