What are the key messages?

Planning

The descriptive frameworks PrOACT-URL and BRAT support any benefit-risk assessment and provide a structured approach to assess and manage the problems. It is crucial as the problem must be framed carefully in order to decide which quantitative benefit risk methodology to use

Evidence gathering and data preparation

Data availability influenced the criteria definitions and choice of alternatives (comparator) defined the steps identify outcomes in BRAT and objectives and alternatives in PrOACT-URL. The limitations in data availability were also connected to the problem of double counting. This is, especially the case for risk criteria, where a choice had to be made either to list criteria by organs AEs or the seriousness or include both and then try to take the risk of double counting into account in the analysis.

When aggregating data from different sources it is important to be aware of issues such as different definitions of outcomes and different way of measuring certain effects, and also issue of bias when combining data.

The Benefit Risk Ratio (BBR) method is confined to one benefit criterion compared to one risk criterion. Hence, the four risk criteria were combined by summing the number of adverse events. It was assumed they were independent and severities were similar between them, and there was no risk of double-counting.

Analysis

For BRAT and SBRAM explicit trade-off is not required, and the visual summary of key benefits and risks (KBRS-table, forest plot and SBRAM tornado diagram) are the basis for discussion around overall benefit-risk balance. The quantitative method requires explicit weights, and this information was found difficult to obtain. Generally I can be difficult to distinguish between analysis and exploration.

Exploration

The two main issues in evaluation of uncertainty are uncertainty in data and uncertainty in weights and value function. Especially the quantitative methods proved use full in exploring the uncertainty related to preferences.

Conclusion and dissemination

All methods increase transparency in the decision process with clear audit trails so they can all add to consistency in further decisions.