aRMM effectiveness evaluation, with every country’s evidence checked the day it arrives
Every country's guides and patient cards checked the day they go out, gaps closed in days, the evaluation drafted with citations.
See one case, screen by screen ↓An educational programme runs in fourteen countries, each with its own national agreement, its own target prescriber list and its own clocks: a first wave within 30 days, new prescribers within a quarter, materials sent apart from promotional mailings. The evidence that it happened — distribution logs, courier receipts, card text, pharmacy panel extracts — arrives from each affiliate in its own format, by e-mail.
So the checking waits. A superseded patient card goes out to 38 prescribers in the July and August waves, and nobody sees the item code until someone reconciles the log months later. Then the data lock comes, and the risk management lead is pulling survey tables, drug utilisation results and fourteen country records together by hand to write the effectiveness evaluation for the PSUR — with every number to be traced back to its source.
Estimated split for one evaluation round of a multi-country programme, by hand and with the solution.
Four agents check every country's evidence as it arrives and two more write the evaluation and the requests; the risk management lead decides each finding.
At 08:12 on Oct 7, the Poland affiliate uploads its Q3 evidence for the Talmirex educational programme — 24 days before the PSUR data lock. Here is what happens next, screen by screen, in the working solution.
Priya opens the solution: 3 programmes, 23 country commitments, PSUR data lock Oct 31 in 24 days. 9 of 14 countries are on track, prescriber reach is 94.6% and patient card coverage 90.7%. Poland is marked as a gap, with new evidence today.
“Superseded patient card v2.0 sent to 38 prescribers.”
Marek uploads the Q3 distribution log, the courier receipts and a text comparison of patient card v2.0 against v3.0 through the affiliate portal. They sit beside the letter of agreement with the national authority, the target prescriber list and the pharmacy panel extract, ready to be checked.
“3 new files arrived today at 08:12 from the Poland affiliate.”
Evidence intake classifies the three files and attaches them to Poland. The distribution reconciler matches 812 dispatches to the 860 prescribers on the target list and checks item codes against version 3.0. The commitment tracker checks 4 clocks against the Jul 14 agreement, and the survey and utilisation analyst reads card coverage of 87.0% from the pharmacy panel.
6 of 8 checks pass. The high finding: the national authority agreed patient alert card v3.0 on Jul 14, but the Jul 22 and Aug 5 waves list item code TX-PAC-PL-02 — version 2.0 — for 24 and 14 prescribers, and v2.0 lacks the heart-event and shingles wording. The medium finding: the Sep 18 new-prescriber wave to 52 physicians went out with a congress invitation.
Every finding cites the log line and the agreement clause it rests on.
For each finding Priya chooses Ask the affiliate, Accept with a note, or Not an issue. Both distribution findings go to the affiliate. Card coverage at 87.0% is down from 91.2% in Q2 but still above the 85% threshold, so she accepts it with a note on the audit trail.
Agents propose findings; only the risk management lead decides them.
The request writer turns her decisions into one letter to Marek: send card v3.0 to the 38 prescribers with a cover letter asking them to replace v2.0 stock, and resend the guide and card to the 52 new prescribers as a stand-alone educational mailing — both by Nov 6. It is sent only when Priya presses Send.
“The checks will run again as soon as they arrive.”
612 prescribers in 8 countries answered the wave 2 survey (22.0% response). Three of five key messages meet the 80% threshold; the heart-event message is weakest at 74.1%, lowest in Poland at 68%. Drug utilisation data across 9,812 new users agree: 11.8% of patients aged 65 or over with heart risk factors started without a documented reason, against a 10% target.
Each key message is reported separately, with n and a 95% confidence interval.
From the risk management plan v5.0, both survey tables, the drug utilisation study, the outcome analysis and the 14 country records, the agents draft six sections — programme and objectives, reach, knowledge, prescribing behaviour, safety outcomes, conclusion and actions — with 19 citations. 7 of 11 success criteria are met.
The draft suggests Partly effective and proposes actions: a targeted reminder on the heart-event message, a version 3.1 of the guide and card for agreement with the national authorities, and the country actions. Priya chooses the verdict — Effective, Partly effective or Not effective — and sends it to the QPPV.
“Reach, card coverage and outcomes meet their criteria; prescriber knowledge of the heart-event message, patient knowledge of contraception and prescribing in older patients do not.”
Dr. Chen reviews the signer, the verdict and the criteria — 7 of 11 met — enters her password and signs. The meaning is recorded with the signature, the PSUR section and RMP v5.1 Part V are updated, and the country actions go to 4 affiliates.
“I approve this effectiveness evaluation for inclusion in the PSUR and the risk management plan update.”
The same programme, seen by the people who run it — from the affiliate who sends the evidence to the QPPV who signs the evaluation.
Four agents check every country's evidence as it arrives and two more write the evaluation and the requests; the risk management lead decides each finding.
Takes files from the affiliate portal, the risk minimisation mailbox and the survey vendor, decides what each one is — agreement, target list, distribution log, receipts, card text, survey or study table — and attaches it to the right programme and country.
Matches every log line and receipt to the national target list and the versions the authority agreed, checks the channel is separate from promotional material, and explains each mismatch with the passages it rests on.
Keeps a clock for every national commitment — first wave, quarterly new-prescriber waves, survey windows, PSUR data locks and REMS assessment dates — and alerts 7 days before and on the day a clock expires.
Reads survey vendor tables, pharmacy panel extracts, drug utilisation and outcome analyses, and computes every indicator with its confidence interval against the pre-set threshold.
Drafts the effectiveness evaluation — reach, knowledge, behaviour, outcomes, country gaps and actions — for the PSUR and the risk management plan, with a citation for every number.
Turns the confirmed findings for a country into one clear request to the affiliate — what to send, why, and by when, with the due date from the commitment clock.
Decides each finding. The agents propose; a named person decides.
Ask about any country, survey result or deadline — or say what to change. Answers come from the national agreements, distribution logs, receipts, survey results and study tables, with their sources.
Which countries are below the reach threshold?
Three commitments are under 90%: France · Talmirex at 85.9%, after 279 dermatologists were added on Aug 26; Czechia · Talmirex at 85.1%, with the first wave incomplete; and Poland · Corventa at 86.4%. Overall Talmirex reach is still 94.6%.
Why is the heart-event message below threshold?
Prescribers answered the heart-event message (KM3) correctly 74.1% of the time (95% CI 70.4–77.6), under the 80% threshold. It is the weakest message in all 8 countries — lowest in Poland (68%) and France (70%). Prescribing data agree: 11.8% of patients ≥ 65 with heart risk factors started without a documented reason, against a 10% target. The heart wording was only added in v3.0, so some prescribers still hold older guides.
Add a rule: flag evidence older than 45 days
Done. A country whose latest distribution evidence is older than 45 days is now shown as evidence missing. Running it now flags one country — Italy, whose latest log is from Aug 14 (54 days). The rule is on in Settings and recorded on the audit trail.
Draft the request to the Poland affiliate
To Marek Wójcik, two items by Nov 6: send patient card v3.0 to the 38 prescribers who received v2.0 on Jul 22 and Aug 5, asking them to replace old stock; and resend the guide and card to the 52 new prescribers as a stand-alone mailing, because the Sep 18 wave went with a congress invitation, which the agreement does not allow.
13 screens from the working solution, on its sample data. Pick one to see it large.
Thresholds, approvers and where evidence comes from are settings, taken from your risk management plan.
| Setting | Default | Choose from |
|---|---|---|
| Prescriber reach — target list sent the guide within 60 days | 90% | Set from the plan · versioned |
| Patient card coverage — new patients issued the card | 85% | Set from the plan · versioned |
| Prescriber knowledge — correct answers per key message | 80% | Set from the plan · versioned |
| Patient knowledge — correct answers per key message | 70% | Set from the plan · versioned |
| Confirm country gaps | Priya Raman | Priya Raman · Dr. Samuel Okoro |
| Sign the effectiveness evaluation | Dr. Maya Chen | Dr. Maya Chen · Priya Raman |
| Sign the REMS assessment | Sam Patel | Sam Patel · Dr. Maya Chen |
| Remind affiliates before a wave is due | 7 days before | On · off |
Every figure is labelled: a target the solution is built to, an estimate, a typical published result, or a proven one.
“demo” = seen in the working solution, on its sample programme data · “target” = the design goal, measured in the live solution · “estimated” = our estimate · Context: EU good pharmacovigilance practice, Module XVI; FDA REMS assessment guidance (2019). People, companies and products named on this page are fictional — characters and sample data in the working solution.
It shows whether additional risk minimisation measures — such as a prescriber guide and a patient alert card — reached the right people, were understood, changed prescribing and protected patients. The solution measures reach, knowledge, behaviour and outcomes against the success criteria in your risk management plan and drafts the evaluation for the PSUR and the plan, with a citation for every number.
Each affiliate’s distribution log and courier receipts are matched to the national target prescriber list and the versions the authority agreed. It checks the first wave went out within 30 days of the agreement, new prescribers were reached within a quarter, and mailings were sent apart from promotional material.
Yes. The survey and utilisation analyst reads the survey vendor tables, pharmacy panel extracts, drug utilisation and outcome analyses, and reports each key message separately with n and a 95% confidence interval against the threshold. It does not pool countries with different questionnaires.
Yes. In the working solution a REMS sits beside two EU educational programmes, with its survey window and its 3-year FDA assessment date on the commitment clock, and a named REMS programme manager signs before submission.
Yes. The agents propose findings; the risk management lead decides each one and presses Send before any request reaches an affiliate. The lead chooses the evaluation verdict, and the QPPV signs it with the meaning of the signature recorded.
Yes. Affiliate safety officers upload evidence and answer requests for their own country only. Their uploads are checked as soon as they press Submit, and evidence can also arrive through the risk minimisation mailbox.
Yes. Prescriber reach, card coverage, prescriber and patient knowledge thresholds come from your risk management plan and every change is versioned. Who confirms country gaps, who signs the evaluation and who signs the REMS assessment are settings.
The Agentic Solution Engine builds and deploys it from your requirements — your risk management plans, national agreements and a sample of affiliate evidence and survey results — and it goes live once every quality gate has passed. We will walk you through it on your own programme first.
Every side-effect report becomes a cited case in minutes, filed with days to spare.
1-min filmSafety LiteratureEvery journal article screened with a written reason, and reportable cases at intake in about a day.
1-min filmSignal ManagementEvery potential safety signal gets its evidence assembled in days, each number traced to its query.
1-min filmWe’ll run the Risk Minimisation Effectiveness Analyzer on a sample of your own affiliate evidence and survey results.