PrOACT-URL Framework

The PrOACT-URL frameworkprovides a step-by-step guide to approaching decision problems. The framework was tested in the EMA Benefit-Risk Methodology Project Field Tests and consequently adapted for use in the benefit-risk assessment of medicinal products in the EMA Benefit-Risk Tools and Processes. We present the use of this framework in the table below.

In the book "A Practical Guide to making better life decisions",[Hammond JS, 1999] the authors proposed a method of making trade off using even swaps, by increasing the value of an alternative in terms of one criterion while decreasing its value by an equivalent amount in terms of another criterion – a form of bartering, so to allow elimination of criteria. As more criteria are eliminated, the final objective option would become more dominant and clear. We also demonstrate this method in step 7 of PrOACT-URL in the table below.

An "Effects Table" of the treatment effects by criteria follows. We created an effects table for benefit criteria, and another effects table for risk criteria.

Step Description

Problem

1. Determine the nature of the problem and its context.

(Source of information: EPAR)

Should rimonabant be licensed for use in Europe?

1a. The medicinal product

Rimonabant (Acomplia/Zimulti) is a selective antagonist of cannabinoid types I (CB1) receptors. The cannabinoid system has been shown to be involved in the central regulation of food intake and the central nervous system (CNS) reward system. CB1 receptors were first found in the brain, and later in several human tissues, including adipocytes

Rimonabant is a new drug, the first in class, indicated for weight loss in obese or overweight patients with co-morbidities

1b. Indication(s) for use.

Weight lost

1c. The therapeutic area and disease epidemiology

Around two-third of the US population is overweight or obese (Hedley, Ogden et al. 2004). In Europe, there is also a high prevalence of overweight, around 50%, and obesity, up to 30%(James, Rigby et al. 2004; York, Rossner et al. 2004). As obesity is strongly associated with and increased risk of diabetes and cardiovascular diseases (Klein, Burke et al. 2004)and mortality, obesity remains a great public health problem(Solomon and Manson 1997)

1d. The unmet medical need, severity of condition, affected population, patients’ and physicians’ concerns, time frame for health outcomes.

With around 2.5 million deaths worldwide due to obesity, there is a large unmet medical need in obesity(Organisation 2002). The prevalence of obesity has been increasing not only in the US but also in other countries, in adults as well as children(Hedley, Ogden et al. 2004). Because diet and exercise has a limited long-term success in reducing weight(Loveman, Frampton et al. 2011), a more comprehensive approach to treating obesity with a long-term effect is needed.

1e. The decision problem (what is to be decided and by whom, e.g., industry, regulator, prescriber, patient)

Decision problem: Rimonabant should be licensed to be used in Europe.

This decision should be carried out by regulators, physicians and patients.

We have limited input from patient group at this stage of the project. Therefore, we will concentrate on the decision making process as regulators and physicians.

2. Frame the problem.

(Source of information: EPAR, literatures)

2a. Whether this is mainly a problem of uncertainty, or of multiple conflicting objectives, or some combination of the two, or something else (e.g., health states’ time progression).

Problem regarding safety and trade-offs of side effects for benefits in weight lost. We have limited input from patient group into the project at this stage, also with a limitation on time and resources to assess patients prospective. Our group had decided to use in house medical and regulators assessment.

Apart from uncertainties with stakeholder prospective on trade-offs, in order to simplify the model for the trade-off exercise, we had limited the total criteria for decision from 15 benefits and 34 adverse events to the 5 most important criteria. The 5 criteria were selected based on a separate ranking exercise performed for the MCDA model. Some other factors might contribute into the trade-off which could affect the final results, this itself is an important uncertainty we need to consider when interpreting the results from this exercise.

2b. The factors to be considered in solving the problem (e.g., study design, sources and adequacy of data, disease epidemiology, presence of alternative treatments).

There are 5 randomised controlled studies (Desres, 2005; Van Gaal, 2005;Pi-Sunyer, 2006;Scheen, 2006;Rosenstock, 2008) of which 4 have final outcome at 12 months (Desres, 2005; Van Gaal, 2005;Pi-Sunyer, 2006;Scheen, 2006) and the remaining study only presented outcomes at 6 months[14] Samples sizes were adequate in all 5 studies. We were interested in 12 months outcome in this exercise, therefore, we dropped study with only 6 months follow up. In the 4 remaining RCT. There are alternative treatment dosages, but for the purpose of this exercise, we only used rimonabant 20mg compared to placebo.

Objectives

3. Establish objectives that indicate the overall purposes to be achieved.

3. The aim

To assess the treatment related risk/health benefits of Rimonabant compared to placebo in obese people

4. Identify criteria for

a) favourable effects

b) unfavourable effects

(Source of information: EPAR, literatures)

4. In this example, we selected the following,

Favourable effects: Proportion of patient achieved 10% weight lost at 1 year, Waistline changes at 1 year, Proportion reduction in metabolic syndrome

Unfavourable effects: Incidence of psychiatric disorder, Incidence of severe adverse events

All criteria listed in EPAR:

Favourable/beneficial effects: Unfavourable effects/ risk
Weight lost at 12 months:
  • Percentage of patient reached 10% weight lost

Lipid control at 12 months:
  • Total Cholesterol
  • HDL Cholesterol
  • LDL Cholesterol
  • Ratio HDL Cholesterol/Total Cholesterol
  • Triglyceride
Waist Circumference at 12 months
Diabetes control at 12 months
  • Fasting glucose
  • Fasting insulin
  • Insulin resistance
  • HbA1c
  • Glucose intolerance
  • Blood pressure control
  • Systolic Blood Pressure
  • Diastolic Blood Pressure

Metabolic Syndrome at 12 months


Infection and infestation:
  • Upper respiratory tract infection
  • Gastroenteritis viral

Psychiatric disorder:
  • Anxiety
  • Isonmia

Mood alteration with depressive symptoms

Depressive disorders

Irritability

Parasomnia

Nervousness

Sleep disorders

Nervous system disorders:
  • Dizziness
  • Memory loss
  • Hypoesthesia
  • Sciatica
Vascular disorders
  • Hot flushes

Gastrointestinal disorders
  • Nausea
  • Diarrhoea
  • Vomiting

Skin and Subcutaneous Tissue disorder
  • Pruritus
  • Hyperhydrosis

Musculoskeletal and connective tissue disorder
  • Tendonitis
  • Muscle cramp
  • Muscle spasms

General disorder
  • Influenza
  • Asthenia/Fatigue

Injury, Poisoning and Procedural complications
  • Joint sprain
  • Contusion
  • Fall

Severe Adverse Events
  • Death
  • Overall Psychiatric disorder
  • Cardiac disorder
  • Urinary disorder
  • Road traffic accident

Alternatives

5. Identify the options to be evaluated against the criteria.

5a. Pre-approval:

Placebo

5b. Post-approval: N/A

Consequences

6. Describe how the alternatives perform for each of the criteria, i.e., the magnitudes of all effects, and their desirability or severity, and the incidence of all effects.

See effects table.

Trade-offs

7. Assess the balance between favourable and unfavourable effects.

There were 16 key criteria in the assessment of rimonabant. For the purpose of this exercise, we concentrated in the top 5 key criteria, ranking for the listed benefit and risk criteria were collected by previous weighting survey performed for MCDA (Details can be found in MCDA report):

1) Proportion of patient achieved 10% weight lost at 1 year

2) Changes in waist line at 1 year

3) Incidence of psychiatric events

4) Incidence of severe adverse events

5) Percentage reduction in metabolic syndrome

Note: Trade-offs scales between criteria used in this report were factitious, based on a team physician’s prospective. Results from this analysis was intended for illustration of the technique only, formal assessment of benefit and risk of this medication would require established decision meeting with stakeholders to decide on agreeable trade-offs.

Results from these 5 criteria.

ObjectivesRimonabant 20mgPlacebo
10% weight lost at 1 year25%6%
Changes in waist line at 1 year-6.2-1.9
Incidence of psychiatric disorder22.2%11.1%
Incidence of severe adverse events1.7%0.8%
% Reduction in metabolic syndrome42.9%20.7%

Step 1: Assuming we were able to exchange proportion of reduction in metabolic syndrome with adverse events in the following exchange rate:

10% metabolic syndrome = 1% severe adverse events

ObjectivesRimonabantPlacebo
10% weight lost at 1 year25%6%
Changes in waist line at 1 year-6.2-1.9
Incidence of psychiatric disorder22.2%11.1%
Incidence of severe adverse events1.7%0.8% + (42.9%-20.7%)/10%x1%->3.02%

Step 2: Assuming we were able to exchange proportion of psychiatric disorder with severe adverse events with the following exchange rate:

1% severe adverse events= 5% psychiatric disorder

ObjectivesRimonabantPlacebo
10% weight lost at 1 year25%6%
Changes in waist line at 1 year-6.2-1.9
Incidence of psychiatric disorder22.2%11.1%+(3.02%-1.7%)x5% ->17.7%

Step 3: Assuming we were able to exchange proportion of patient with 10% weight lost at 1 year and changes in waistline with the following exchange rate:

1cm difference in waist line = 2% in proportion patient for every

ObjectivesRimonabantPlacebo
10% weight lost at 1 year25%+(6.2-1.9)x2%->33.6%6%
Incidence of psychiatric disorder22.2%17.7%

Final step: Assuming we were able to exchange proportion of patient achieved 10% weight lost with incidence of psychiatric disorder with the following exchange rate:

0.20% psychiatric disorder = 1% of patients achieving 10% weight lost at 1 year

ObjectivesRimonabantPlacebo
10% weight lost at 1 year33.6%6%+(22.2%-17.7%)/0.2%-> 28.5%

Uncertainty

8. Report the uncertainty associated with the favourable and unfavourable effects.

This model has the following limitations/uncertainties

1) Albeit these data were collected from well conducted randomised controlled studies. There were different restrictions in study inclusion criteria. For example, RIO-diabetes limited to participants who were diabetic at start; As well the uncertainties between patient populations in different catchment area. Some of these uncertainties were handled by using meta-analysis to summarise the results before input into this model.

2) Data used in this framework were summary statistics. There are uncertainties in the natural variation in statistic.

3) In terms of nature of adverse events, there were possibilities of under reporting.

9. Consider how the balance between favourable and unfavourable effects is affected by uncertainty.

It is inevitable that the risk benefit balance would be influenced by the uncertainties.

For example, if rimonabant should be more efficacious over patients with diabetes, this would change this balance towards improvement in benefit as the data collected are unbalanced.

Besides, data used in this model were summary statistics. Therefore, this is subjective to random error.

Risk tolerance

10. Judge the relative importance of the decision maker's risk attitude for this product

The main benefit of rimonabant was for weight reduction, which there is a selection of licensed alternatives available; and the key concern with rimonabant was the increased incidence of depression and psychiatric disorder, which is not uncommon and could be severe in some cases.

Therefore, decision maker might have a lower tolerance of risk related to this medication.

11. Report how this affected the balance reported in step 9.

Lower risk tolerance was reflected in the trade-offs. For the purpose of this exercise, trade-offs were factitious and decided by the group in house physician, and have no input from other stakeholders.

Linked decisions

12. Consider the consistency of this decision with similar past decisions, and assess whether taking this decision could impact future decisions.

Decision on this drug may influence future application on medications targets cannabinoid type I (CB1)

Table 1 Effects table for favourable effects (benefit criteria)

Effects Description Fixed Lower Fixed Upper Units Rimonabant 20mg Placebo
Favourable Effect at 12 months Weight control Percentage patients reached 10% weight lost 0 100 % 25 6
Lipid control Total Cholesterol changes -2 2 mmol/L 0.05 0.12
HDL Cholesterol changes -2 2 mmol/L 0.22 0.11
LDL Cholesterol changes -2 2 mmol/L 0.08 0.15
Ratio HDL /Total Cholesterol changes -2 2 -0.65 -0.33
Triglyceride changes -2 2 mmol/L -0.26 0.01
Waist Circumference Waist circumference changes -10 10 cm -6.2 -1.87
Diabetes control Fasting glucose changes -2 2 mmol/dL -0.22 0.05
Fasting insulin changes -5 5

?IU/mL

-1.04 1.08
Insulin resistance changes -5 5 Compared to placebo -0.8
HbA1c changes -5 5 % -0.6 0.1
Glucose intolerance changes -5 5 Compared to placebo -0.89
Blood pressure control Systolic blood pressure changes -10 10 mmHg -1.26 0.48
Diastolic blood pressure changes -10 10 mmHg -1.45 -0.28

Table 2 Effects table for unfavourable effects (risk criteria)

Effects Description Fixed Lower Fixed Upper Units Rimonabant 20mg Placebo
Unfavorable le effect at 12 months Infection and infestation Upper respiratory tract infection 0 15 % 11.4 12.4
Gastroenteritis viral 0 10 % 2.9 3.6
Psychiatric disorder Anxiety 0 10 % 2.4 5.6
Insomnia 0 10 % 3.2 5.4
Mood alteration with depressive symptoms 0 10 % 3.1 4.8
Depressive disorders 0 10 % 1.6 3.2
Irritability 0 10 % 0.6 1.9
Parasomnia 0 10 % 0.2 1.5
Nervousness 0 10 % 0.2 1.2
Sleep disorders 0 10 % 0.4 1
Nervous system disorders Dizziness 0 10 % 4.9 7.5
Memory loss 0 10 % 0.9 1.6
Hypoesthesia 0 10 % 0.6 1.6
Sciatica 0 10 % 0.6 1
Vascular disorders Hot flushes 0 10 % 0.7 1.9
Gastrointestinal disorders Nausea 0 15 % 4.9 11.9
Diarrhoea 0 10 % 4.8 6.3
Vomiting 0 10 % 2.2 4
Skin and Subcutaneous Tissue disorder Pruritus 0 10 % 0.5 1.2
Hyperhydrosis 0 10 % 0.5 1.2
Musculoskeletal and connective tissue disorder Tendonitis 0 10 % 1 2.1
Muscle cramp 0 10 % 1 1.4
Muscle spasms 0 10 % 0.5 1
General disorder Influenza 0 10 % 8.6 8.9
Asthenia/Fatigue 0 10 % 5 6
Severe Adverse Events Death 0 10 % 0.25 0.16
Overall Psychiatric disorder 0 10 % 0.12 0.48
Severe Depressive disorder 0 10 % n/a 0.24
Cardiac disorder 0 10 % 0.25 0.48
Urinary disorder 0 10 % 0.12 0.36
Road traffic accident 0 10 % 0.00 0.24

Lower to Upper Limits define the range of a measurement scale that includes all the data for each criterion and is meaningful for assessing swing weights.