What are the lessons learned?

Lessons related to the benefit-risk methodology

PrOACT-URL and BRAT are comparable-both define decision context into a transparent process and compare alternatives in a series of criteria assessment-although there are also differences between the two frameworks. The difference, however, is not substantial. For example, the comparison group is defined in the first step in BRAT (decision context) and in the third step in PrOACT-URL (alternative). While both frameworks are essential for an effective BR analysis, neither PrOACT-URL nor BRAT is a fully quantitative decision analysis tool.

In this study case, two most suitable decision tools are MCDA and SMAA. Both methods can be used to integrate multiple benefits and risks quantitatively taking the relative important of benefits and risks into account. This is done by scoring data into a utility function and, based on preference from different stakeholders (weight, or relative importance), an average weighted utility score for risks and benefits is then calculated. The main difference between MCDA and SMAA is that, while MCDA does not fully address uncertainty. SMAA handles uncertainty by using distribution of statistics instead of a precise point estimate. Furthermore, MCDA requires explicit preference information from stakeholders upfront, which might slowdown the benefit-risk assessment. SMAA process is more flexible that stakeholder preference can be absent or alternatively be elicited in the form of criteria ranking at start.

Another lesson from this case study is that, even after great deal of effort to quantify benefits and risks of rimonabant when compared to placebo, the benefit-risk balance is still unclear when rimonabant is to be used in a large population. Impact numbers approach to benefit-risk assessment may provide more suitable concept as they directly describe the impact on the populations of interest in terms of number of people affected. The main challenge with this method is its inability to integrate multiple benefits and risks.

Lessons related to the visual representation of benefit-risk assessment results

A good visual representation is one of the keys for effective communication and the content and visualisation methods should be suitable for the intended audience. Although most visualisation tools presented in the rimonabant case study were limited to graphical output from the software used for the analysis, they were clear, simple and easy to understand. New graphical outputs including interactive visualisationswere specifically developed in wave 2 to show how changes in the assumptions and stakeholders’ preference may change the score and conclusion.