SystemResearch & Discovery

Evaluate · database or source

Drug pipeline databases and the public registries behind them

A pipeline database is a curated cache of public sources, and what you pay for is normalisation and linkage, not access. That makes the questions worth asking the ones about what a cache cannot hold: the phase 1 trials no rule covers, the version history a current record hides, the same trial sitting in two registries, and the passage behind a date.

Updated · 14 min read

See one case, screen by screen →
Watch with transcript →

A drug pipeline database is a commercial store of what pharmaceutical companies are developing — assets, mechanisms, phases, indications, sponsors, trials and dates — assembled by reading sources that are, almost without exception, already public. That last clause is the one that should shape a renewal decision. You are not buying access to information that is otherwise unavailable; you are buying normalisation, mechanism classification and the linkage of one company's asset to another's trial, done by people whose job it is. Both of those have real value, and neither of them is the same thing as the truth. Botminds' Competitive Pipeline Tracking takes the other route for a watch list you define: it reads the public sources directly each week and keeps the version of each page it read, so every date on a board opens the sentence behind it.

Where the solution appears below, the assets, companies and trial numbers in it are fictional sample data created for the demonstration. The registries, rules and research described are real and linked at each figure.

What the public record covers, and where it stops

Free

the registries themselves: registration, phase, eligibility, facilities, dates and version history, for anyone

Phase 1

is excluded from the United States registration duty, so the earliest competitive signal is the least reliably public

14%

of trials in one cohort had primary-outcome changes visible only between historical versions, with the latest record matching the publication

Coverage and exclusions from the United States registry rule at 42 CFR § 11.22. The 14% figure is from Holst and colleagues' study of 292 assessed results publications from trials at German university medical centres completed between 2009 and 2017.Source: 42 CFR § 11.22; Holst et al., PLoS Medicine, 2023

What is a drug pipeline database?

It is a curated index of development programmes, maintained by analysts who read registries, company disclosures, conference material and regulatory decisions, and who then do three things a raw source does not. They give each asset a stable identity across the companies that license it and the names it changes along the way. They classify it by mechanism and target, so that you can ask a question like "every G12D-selective inhibitor in phase 2 or later" and get an answer. And they link the asset to the trials, indications and dates that belong to it.

The scale of that work is the honest argument for buying it. In one of the largest published analyses of clinical trial success rates, Wong, Siah and Lo observed that the United States registry held over 217,000 trial entries at the time, and that "It is estimated that trained analysts would require tens of thousands of hours of labor to incorporate its full information manually" (Biostatistics, 2019). Nobody reads the whole registry by hand. The question is whether you need the whole registry, or the forty assets that bear on your programmes.

What the public registries already contain, free

More than most buyers assume, and in a structured form. A registered interventional trial publishes its sponsor, its phase, its design, its eligibility criteria, its planned enrolment, its facilities with a per-site status, and its estimated and then actual completion dates. Each of those has an update obligation behind it, and the record keeps its earlier versions, so a change is visible as a change rather than as a new fact.

The international picture is a network rather than a single source. The World Health Organization describes its own role carefully: the platform's mission "is to ensure that a complete view of research is accessible to all those involved in health care decision making", and then, pointedly, "Note: You cannot register a trial with WHO. The WHO ICTRP is not a clinical trials registry" (WHO ICTRP). It is a search layer over primary registries that each meet published criteria. For a competitive analyst that means there is no single authoritative world list; there is a set of national and regional registries with an index across them.

In the European Union the position changed recently and in the buyer's favour. Information submitted through the Union's clinical trials system "is made available on the CTIS public portal, in line with revised CTIS transparency rules applicable as of June 2024", and those "Revised transparency rules apply as of 18 June 2024 for publishing information on clinical trials submitted through CTIS" (EMA, Clinical Trials Information System). A database whose European coverage was specified before that date is working to an older picture of what is public.

Is ClinicalTrials.gov enough for competitive intelligence?

For trial facts, very nearly; for a pipeline, no. The United States registry will tell you what a rival has registered, where, in whom, and when it expects to finish. It will not tell you four things a competitive intelligence function needs every week.

It does not carry the asset. A registry record is about a trial, not about a molecule: the same compound under two names, or licensed to a partner in Asia, appears as unrelated records. It does not carry mechanism. A G12C inhibitor and a G12D inhibitor look identical to a registry field; the distinction that decides whether a rival threatens your programme lives in the pharmacology, not in the registration. It does not carry the sponsor's own account. Guidance about when data will be released, a decision to stop a programme, a partnership — these appear in releases and on calls, not in a registration. And it does not cover everything, which is the subject of the next two sections.

So the practical answer is that the registry is the best single source and a poor only source. What it is unambiguously good for is a baseline: whatever a database tells you about a trial, the registry can be used to check it, free, in a minute.

The four public source families a database curates

Where a pipeline database's facts come from, and what each source family publishes

Source familyWhat is publicThe clock on itWho publishes it
Trial registriesRegistration, phase, design, eligibility, planned enrolment, facilities with per-site status, estimated and actual completion dates, and a version history.Status changes within 30 days; registration information refreshed at least every 12 months.The sponsor, into a national or regional registry in the WHO network.
Company disclosuresPipeline pages, press releases and earnings-call commentary: phase changes, readout guidance, filings, partnerships, discontinuations.None. Published when the company chooses, and amended without notice.The sponsor, on its own site and in its own investor materials.
Congress materialAbstracts, late-breaker schedules and presented data: design, population, endpoints, effect sizes and safety.Fixed by the congress calendar; abstracts typically weeks ahead, late-breakers at the session.The congress organiser and the presenting investigators.
Agency decisionsApprovals, filing acceptances and label changes in each major jurisdiction, with the approved indication in the agency's own words.Published on decision, with the supporting review documents following.The regulator, in its own approval database.
The registry clocks are the United States rule's at 42 CFR § 11.64. A curated database reads all four families; so does an analyst, and so does an agent. The difference is scale on one side and currency on the other.Source: 42 CFR § 11.64

Those four families do not contribute equally, and their mix is specific to the therapy area. On the sample watch lists in the solution, just over half of the changes found in a quarter came from registries and the rest from the other three families — a proportion worth measuring on your own assets before deciding how much of a subscription's coverage you actually use.

Where a quarter's changes came from, across four source families

Changes found across five watch lists, twelve weeks

52%
23%
17%
8%
  • Trial registries 52% · 52%
  • Releases and pipeline pages 23% · 23%
  • Congress abstracts 17% · 17%
  • Agency approvals 8% · 8%
Fictional sample data shown on the solution's own dashboard, across five sample watch lists over twelve weeks. It is a demonstration figure, not a measured customer result, and the mix differs sharply by therapy area — an area with a dense congress calendar shifts weight to the third band.

What a database adds: normalisation, mechanism and linkage

Three things, and they are worth naming precisely because they are what a renewal is actually paying for.

One identity per asset. A compound acquires a development code, a partner's code, an international non-proprietary name and finally a brand name, and it may be licensed by region. Holding those together as one thing is tedious, continuous work, and it is the single most valuable product of curation.

Mechanism and target classification. Registry fields do not say what a drug does. A curated classification is what makes a landscape by mechanism possible, and a landscape by mechanism is how a programme team sees which rivals are actually in its lane rather than merely in its disease.

Linkage. Asset to trials, trials to indications, indications to approvals, approvals to labels. Each link is a small judgement; a few thousand of them is a product.

What the classification is for: fourteen assets placed by mechanism and phase

The landscape screen in Competitive Pipeline Tracking for Oncology KRAS: 14 watched assets, our programme NX-4127 named, and a grid placing each asset by mechanism — KRAS G12C, KRAS G12D, pan-RAS, other KRAS alleles, pathway partners — against phase from Phase 1 to approved, with markers for significant change, changed this week and ours. A side panel reports 11 changes this week, 3 of them significant, listing a United States approval, a Phase 3 primary endpoint met and a Phase 3 registration, plus 8 other registry, date and pipeline updates. Below, a table of all 14 watched assets gives mechanism, phase, lead indication, last change and a threat rating.
Fictional sample data. A landscape like this is exactly what mechanism curation buys; the point of this guide is that it can also be maintained for the forty assets you care about rather than for a whole therapeutic universe.

What a database cannot give you

Four things, and each of them is the reason a serious function keeps a route back to the sources.

Currency beyond its own refresh cycle. A curated record is as fresh as the last time a curator looked at it. The source has its own clock; the cache has another, and the second is never faster than the first.

The version history of the source. This matters more than it sounds. Holst and colleagues, studying changes to prespecified primary outcomes, found that "Changes only within registry records, with no apparent discrepancy between latest registry entry version and publication, were observed in 14% of trials (41 of 292; 95% CI [10%, 19%])" (PLoS Medicine, 2023). In other words, in one trial in seven the change was invisible to anyone reading only the current record. A database that stores the current state of a trial inherits that blindness.

Agreement between the registration and the publication. Pranić and Marušić, comparing registered against published outcomes in a cohort of trials, concluded that "Discrepancies remain relatively high between registered and published outcomes, particularly regarding registered omissions in publications and concomitant reporting, nature of statistical method used, and reporting of AEs" (Journal of Clinical Epidemiology, 2016). A single normalised field cannot represent two sources that disagree; it has to choose, and the choice is usually invisible to you.

The passage behind a date. A date in a cell is an assertion. A date with the sentence it came from is evidence, and the difference shows up the first time a programme lead asks where a number came from in a meeting.

A curated database against reading the public sources directly

A pipeline databaseReading the sources, weekly, for your watch list
Breadth

A pipeline database: Thousands of assets across every therapeutic area, including ones you have never heard of. This is its great advantage.

Reading the sources, weekly, for your watch list: The assets you chose. Discovery of a new entrant depends on your own scanning rules.

Mechanism and identity

A pipeline database: Curated: one asset, one identity, classified by target and mechanism.

Reading the sources, weekly, for your watch list: You maintain it, for a much smaller set. Cheap at forty assets, impossible at four thousand.

Currency

A pipeline database: The curator's refresh cycle, which you generally cannot see.

Reading the sources, weekly, for your watch list: A weekly read of the source, which cannot beat the source's own 30-day posting obligation but will not lag it either.

History

A pipeline database: Usually the current state of a record, sometimes with a change log of the database's own edits.

Reading the sources, weekly, for your watch list: The version of the page as it stood on the day it was read, kept, so a change is a comparison rather than a claim.

Evidence

A pipeline database: A field value, with a source named at the level of the source rather than the sentence.

Reading the sources, weekly, for your watch list: The sentence, highlighted, in the stored copy of the page.

Interpretation

A pipeline database: None, by design. The database is neutral about your programme.

Reading the sources, weekly, for your watch list: The whole point: what the change means for your asset, with an owner for the action.

Failure mode

A pipeline database: A confident cell that is quietly eleven weeks old.

Reading the sources, weekly, for your watch list: A gap, because something outside your watch list moved and nobody was looking.

Neither column is a recommendation on its own. Many functions run both: a database for breadth and discovery, a weekly sourced watch for the assets that bear on a live programme.

Phase 1, and the hole in registry coverage

The earliest competitive signal is the one least reliably public, and the reason is in the rule itself. The United States registration duty applies only to a trial that meets every condition on a short list: it is interventional, its phase is something other than phase 1, it studies a drug product the FDA regulates, and it has a United States connection — a site in the country or its territories, a product made there and exported for study, or a US investigational new drug number (42 CFR § 11.22).

Read condition (ii). Phase 1 trials of drug products are not applicable clinical trials, so there is no federal duty to register them. Many sponsors register them anyway — some journals require prospective registration for publication, and investigators may prefer it — but the coverage is voluntary and therefore uneven, and it is uneven in the direction that hurts: a company that wants to keep an early programme quiet can.

For a pipeline database this is the honest limit of its registry-derived content, and it is also where its company-disclosure reading earns most, because a first-in-human start is often announced even when it is not registered. For a buyer, the right question is not "do you cover phase 1" but "from which sources do you learn about a phase 1 programme, and how long after it starts".

Two registries, one trial: the duplication problem

A multinational trial is registered in more than one place, by design, and the records are not copies of each other. Each has its own identifier, its own required fields, its own update obligations and its own enrolment figure. For anyone counting rival trials in an indication — which is what a landscape is — this is a quiet source of error that easily survives into a slide.

One trial, several records: what links them and what does not

The United States record

An NCT number, assigned on first posting, carrying the sponsor, phase, eligibility, facilities and estimated completion dates, with a version history.

Required where the trial is interventional, beyond phase 1, studies a regulated drug product and has a United States connection.

The European record

An EU trial number from the Union's own system, carrying the trial's therapeutic area, sponsor details and the start and end dates of recruitment and of the trial.

Public on the Union portal under the transparency rules applying since 18 June 2024.

Other primary registries

A national number from any of the registries in the World Health Organization's network, for the countries where the trial also runs.

Same trial, different record, different update culture and often a different stated enrolment.

What links them

Secondary identifiers the sponsor chooses to enter — the protocol number, the other registries' numbers, a universal trial number — plus the trial acronym, the sponsor name and the dates.

None of these is mandatory everywhere, so linkage is a judgement and the judgement is where a curated database earns its fee.

What goes wrong

A trial is counted twice in a landscape; two enrolment figures disagree; a status change posted in one registry and not the other reads as a contradiction.

The symptom a practitioner notices is an asset that appears in two phases at once.

How a single multinational trial appears across the registry network, and what a curated database or a weekly watch has to do about it. The registries and rules named are real; no trial is described.Source: WHO ICTRP; EMA, Clinical Trials Information System

Two practical rules follow. Count trials by the sponsor's own protocol number or acronym rather than by registry identifier, because the first is one per trial and the second is one per registration. And when two records disagree about enrolment or dates, keep both and say which you used, rather than silently taking the more recent — the disagreement is frequently the most interesting thing on the page.

One event, its sources kept side by side, with the changed fields marked

A change feed in Competitive Pipeline Tracking for Oncology KRAS, week 40: 11 changes, 3 significant, 8 other, 2 to confirm. The open item is a United States approval for VNT-224 in previously treated metastatic pancreatic cancer, tagged approval, significant and 99% confident, found by the approval watcher on 2 October. A 'what changed' table compares the last capture a week earlier with now: status from under review, priority, to approved in the United States; indication from blank to second-line metastatic pancreatic cancer with a KRAS G12 mutation; label from blank to test-selected, overall survival 12.4 against 6.8 months. A 'why it matters' panel explains the consequence for our own programme in two sentences with two source markers. On the right, two source panels — the agency approval letter and the sponsor's release — show the exact passages with the relevant sentences highlighted, each with a link to open at that passage. At the foot, a line records that a named analyst confirmed the change and included it in this week's report.
Fictional sample data. This is the shape a stored source version takes in practice: not a link to a page that may since have changed, but the text as it stood, with the field-level difference beside it.

Is a drug pipeline database worth it?

It depends on which of two jobs you are buying it for, and dissatisfaction tends to come from buying it for the second.

For breadth and discovery, usually yes. If the question is "what else is in development against this target, anywhere, including from companies we do not watch", a curated database answers it in minutes and nothing else does. That is a real and recurring need: business development, portfolio strategy and due diligence all live on it.

For the weekly competitive cycle, usually not on its own. If the question is "what changed last week that matters to this programme, and what should we do about it", a database gives you a current state with no change, no significance and no evidence attached. A team that tries to run a weekly report out of a database tends to end up re-reading the sources anyway, to find the sentence.

A sensible split, for a function with both needs, is to keep a subscription for breadth and run a sourced weekly watch over the thirty to fifty assets that bear on live programmes. The second is small enough to do properly and is the half that reaches programme teams. The twelve questions to ask of a tool for that half are in the buyer's guide to competitive intelligence tools for pharma.

One test: ask for the passage behind a single date

Before a renewal, run one test that takes ten minutes and settles more than a month of discussion. Pick a single date in the database — a rival's primary completion date, say, or a filing. Then ask three questions.

Where did this come from? The answer should name the source and, ideally, the sentence, not the source type. When was it last checked against that source? The answer should be a date, and the gap between it and today is the real freshness number for everything else you are looking at. And what did the source say before this value? If the answer is that the previous value is not kept, you now know that every trend you have drawn from this database is a trend through the present tense.

Five tests to run on a pipeline database before you renew

  1. The single-date test

    Take one date, ask for its source sentence, its last check date and its previous value. Three answers, ten minutes, and the freshness claim is settled.

  2. The phase 1 test

    Pick an early programme you know about from a conference or a company announcement. Ask when the database first carried it, and from which source.

  3. The duplication test

    Take a multinational trial and ask how many records the database holds for it, and which identifier it reconciles them by.

  4. The disagreement test

    Find a trial whose registered enrolment differs between two registries, or from the published paper, and ask what the database shows and why.

  5. The usage test

    Ask your own team which questions they actually took to the database last quarter. If the answer is breadth questions only, size the subscription to breadth.

Five tests a buyer can run without the vendor's cooperation, except for the questions that have to be asked. Each is designed to produce a fact rather than an assurance.

The last of those is easy to skip and often decisive. A subscription in this category can be renewed out of habit as much as on usage. A quarter's record of what was actually asked of it is the cheapest piece of evidence in the whole decision.

The alternative, for the assets that matter: the sources read weekly, against your own plan

A race board in Competitive Pipeline Tracking for psoriasis, headed 'Psoriasis — who is ahead of IM-7740', week 40, with 12 rival positions across 2 indications and a note that every date opens its source. Eight assets are placed by phase from preclinical through filed to approved, our IM-7740 highlighted in Phase 2, and the lane header reads 5 ahead of IM-7740, 6 of 7 read out before us. Two assets trail dotted arrows from the phase they left, annotated with the registry evidence for the move. A next-readout column dates and counts down each rival's next readout or decision, tagging those that fall before our own. A side panel lists this week's three moves with their sources and the next readouts on the board.
Fictional sample data. The difference from a database view is the right-hand column: every rival date is counted down against the programme's own planned date, which is the comparison a programme lead actually wants and the one a neutral database cannot make.

If the make-or-buy question is really a build question, the ethics and information-gathering rules that constrain either route are in the competitive intelligence code of ethics and the legal line, and how a rival decision calendar is built works through the four kinds of date neither a database nor a registry supplies on its own. To see the public sources read weekly for a watch list of your own, start from Competitive Pipeline Tracking.

Questions people ask

What are drug pipeline databases in pharma?

Commercial, curated indexes of what companies are developing: assets with a stable identity, classified by target and mechanism, linked to their trials, indications, filings and approvals. They are assembled by analysts reading public sources — registries, company disclosures, congress material and regulatory decisions — so what you pay for is the curation, not the access.

How accurate are pharma pipeline databases?

Accurate about what has been published and uneven about how recently it was checked, which is a different property. Two structural limits apply to any of them: a curated record is only as fresh as the curator's last look at the source, and a normalised field has to choose between sources that disagree. The research literature finds such disagreements are common — between registered and published outcomes, and between successive versions of the same registration.

What are the free alternatives to a drug pipeline database?

The sources themselves. The national and regional trial registries in the World Health Organization's network are free and structured; the European Union's portal has published clinical trial information under revised transparency rules since 18 June 2024; regulators publish their approvals and labels; and sponsors publish their own pipeline pages, releases and calls. What the free route costs you is the curation — identity, mechanism and linkage — and the time to do it.

Does ClinicalTrials.gov cover phase 1 trials?

Not as a duty. The United States rule defines an applicable drug clinical trial as one that is interventional, studies a regulated drug product, has a United States connection and whose "Study Phase is other than phase 1". Many phase 1 studies are registered voluntarily, sometimes because a journal requires prospective registration, but coverage is uneven and cannot be relied on for an early competitive signal.

Why does the same trial appear twice in a landscape?

Because a multinational trial is registered in more than one registry, each with its own identifier and its own required fields. Unless the records are reconciled by something that is genuinely one per trial — the sponsor's protocol number, or the acronym — a count of rival trials in an indication will double-count the multinational ones, which are usually the large and important ones.

Should we keep the subscription and also watch the sources?

For many functions, yes, and for different jobs. Keep the database for breadth: what else exists against this target, anywhere, including from companies nobody is watching. Run a sourced weekly watch over the assets that bear on live programmes, because that is where change, significance and evidence matter and where a current-state record is not enough.

Competitive Pipeline Tracking.
See it on your own files.

An agentic solution from Botminds. A race board per indication: every rival trial on its phase, this week's moves sourced, and each rival's next readout counted down against ours. Book a demo, or explore how it works.