SearchResearch & Discovery

Learn · sources outside the journals

Grey literature search: the results that never became papers

Most of what is known about a question never reached a journal. A grey literature search goes after the rest: conference abstracts, trial registry results, regulatory documents, and the research reports and signed notebook entries your own company wrote. This guide shows where each kind lives, what it is good for, and how Scientific Literature Search answers from all of them at once without opening a report to someone who may not read it.

Updated · 13 min read

See one case, screen by screen →
Watch with transcript →

A grey literature search is the half of a literature search that looks outside the journals: conference abstracts, clinical trial registry results, regulatory documents, theses, and the research reports and signed lab-notebook entries a company wrote for itself. It matters because the journals hold far less of the evidence than most people assume — of 307,028 conference abstracts followed across 425 methodology reports, only 37.3% had been published in full when they were checked — and it is hard because every one of those sources sits somewhere that does not share a search box with the others. Botminds’ Scientific Literature Search indexes published papers, patent families, internal research reports and signed notebook entries together, answers a question asked in plain words as numbered claims, and opens every claim at the exact passage it came from, whether that passage is in a journal or in your own interim report.

The screens on this page come from the solution running on its sample library; the programmes, studies, people, journals and sources named in them are fictional. The research figures are from government rules and peer-reviewed methodology studies, linked where they appear.

What never arrives in a journal

37.3%

of 307,028 conference abstracts were published in full, across 425 methodology reports

46.4%

the proportion estimated to be published in full by ten years, for all studies

1year

after the primary completion date: the deadline for submitting results to the United States trial registry

Abstract figures: Scherer and colleagues, Cochrane Database of Systematic Reviews MR000005.pub4, 20 November 2018; the ten-year figure is from a survival analysis and reached 68.7% for randomised and controlled trials. The deadline is 42 CFR 11.44(a).Source: Cochrane methodology review MR000005.pub4; 42 CFR 11.44(a)

What is grey literature?

Grey literature is research output that was never published by a commercial or academic publisher. The definition is about the route to the reader, not about quality: a conference abstract presented to 4,000 specialists, a trial’s results posted to a public registry, a national regulator’s review document and a company’s own pharmacology report are all grey, and any of them may be the best evidence that exists on a narrow question.

The practical consequence is that grey sources have no shared index. A paper is found through a bibliographic database because a publisher deposited a structured citation for it. An abstract is found by reading a proceedings volume. A registry result is found by querying the registry. An internal report is found by knowing who wrote it. That is why a grey literature search is a list of separate searches rather than one query, and why the hardest of them are the easiest to skip quietly.

Why most results never became papers

The reason is often not that the work was poor. A full publication costs its authors months, and the incentive to spend them tracks the result. Scherer and colleagues measured the bias directly: across 45 reports covering 15,783 abstracts, those with a “positive” result — any significant finding — were more likely to reach full publication, with a risk ratio of 1.31. Oral presentation (1.46) and randomised trial design (1.51) also made full publication more likely, while basic research made it less likely (0.78).

That is publication bias stated as a number, and it is the whole argument for a grey literature search. If the probability that a result reaches a journal depends on which way the result went, a search restricted to journals is a biased sample of what is known. The director and principal deputy director of the National Institutes of Health, Francis Collins and Lawrence Tabak, put the same point in plainer words when they wrote about reproducibility in 2014: “Then there is the problem of what is not published. There are few venues for researchers to publish negative data or papers that point out scientific flaws in previously published work.” Negative data is not missing because nobody produced it. It is missing because nobody had a reason to typeset it.

Two numbers bound the loss. Only 37.3% of abstracts followed in that review had been published in full at the time they were checked; a survival analysis estimated 46.4% by ten years, rising to 68.7% for randomised and controlled trials and falling to 44.9% for everything else. So for a question answered mostly by trials, roughly a third of the abstract evidence is still only an abstract a decade later. For a question answered mostly by other designs, basic and preclinical work among them, it is more than half. Deciding how to build a search you can repeat is where that decision gets made, usually without anyone noticing it was a decision.

Registry results: the record that is public and still unread

Registry results are the strangest category, because they are neither hidden nor read. In the United States the trial registry exists by statute: 42 CFR part 11 “implements section 402(j) of the Public Health Service Act … to be made publicly available via ClinicalTrials.gov, the Internet-accessible clinical trial registry and results data bank established by the National Library of Medicine”. For an applicable clinical trial, results information is due within a year of the primary completion date (42 CFR 11.44(a)).

Read those two rules together and the consequence for a search is sharp. An applicable trial that reached its primary completion date eighteen months ago should, unless a delay was certified, have its results posted whether or not a journal ever accepted the paper, and whether or not the sponsor liked the outcome. A reviewer who searches only journals will see the subset that was written up; a reviewer who searches the registry as well sees the rest, with the primary and secondary outcome measures as they were pre-specified, which is the comparison that catches an outcome quietly swapped between protocol and paper.

Registry records are also the easiest grey source to search well, because they are structured. They are the hardest to read, because the result is a table with no discussion section. That is a reasonable division of labour: the registry tells you what was measured and what came out; a paper, if one exists, tells you what the investigators think it meant.

The grey library you already own

For a company doing its own research, the largest grey library in the world for its own questions is the one it wrote. Translational interim reports, pharmacology study reports, assay qualification memos, discovery screening summaries and signed notebook entries describe the exact model systems and the exact compounds the next study will use — and they are invisible to every external search, because they were never published anywhere.

They are also the sources most likely to change a decision, for the same reason: they are about this programme, not about a neighbouring one. In the solution’s sample answer, four of the twelve sources behind a resistance question are internal: a translational interim report, a pharmacology study report and two notebook entries. One of those four contradicts a published series. A search that reached only journals would have returned a tidy answer with the disagreement missing — and that is the failure mode worth designing against, which is why what to do when your own data contradicts a paper is a question about search before it is a question about statistics.

Five grey sources, and what each one is actually good for

SourceWhat it isGood forWhat to watch
Conference abstractA short summary presented at a meeting, often with no full paper behind itEarly signal, and the only report of most preclinical and basic workNumbers move between abstract and paper; an abstract has no methods section to check them against
Registry record and resultsPre-specified design and posted outcome tables; in the United States, results of an applicable trial are due within a year of primary completionOutcomes as pre-specified, and trials whose results were never written upA posted table without a discussion; registration entries are sometimes updated mid-study
Regulatory documentA reviewer’s own assessment of a submitted dossier, published by the agencyIndependent reading of data you cannot otherwise see, including analyses the sponsor did not featureWritten about one product’s dossier, not as a general finding
Internal research reportYour own translational, pharmacology or discovery report, with its model system and nThe closest evidence to the study you are about to designReadable only by the programme that owns it; the version you found may have been superseded
Signed notebook entryA dated, signed experimental recordThe primary observation, before anyone summarised itAn unsigned entry is work in progress, not evidence
The first three rows are categories any team can search today; the last two exist only inside the company that made them. Scientific Literature Search indexes all five and labels each row with its source type, so a reader can tell at a glance which kind of evidence a claim rests on.

Should grey literature be included in a literature review?

Yes — and in a formal review, searching it is a reporting requirement rather than a courtesy. The PRISMA 2020 statement asks authors to name every database, register, website, organisation and reference list they searched or consulted to find studies (PRISMA 2020, item 6). Registers and websites are named in the same breath as databases; a review that searched only databases has to say so, and a reader may judge it accordingly.

The reporting extension for searches is blunter still. PRISMA-S devotes separate items to each grey route — a dedicated item for trial registries, one for online or print sources purposefully browsed, one for contacting people — and asks that strategies be “copied and pasted exactly as run”. You cannot copy and paste a search you did by asking a colleague, which is the real reason grey searching is reported badly: it resists the format the standard wants. The honest answer is to write down what you did do — which proceedings volumes, which registry queries, which internal stores, on what date — and what the reporting standard makes you publish sets out the rest of that record.

There is a limit worth stating. Including grey sources widens the evidence base; it does not make every grey source equal to a trial report. A conference abstract with no methods section cannot support a claim about how an effect was measured. The useful discipline is to let a grey source carry the weight its own detail allows, and to say on the page which kind of source each number came from.

When does a notebook entry become evidence?

When it is signed. An unsigned entry is a scientist mid-thought; a signed one is an assertion someone put their name to on a date, and that difference is what makes it citable. The rules for electronic signatures in regulated research say what the signature has to carry: a signed electronic record must show the printed name of the signer, the date and time, and what the signature meant — review, approval, responsibility or authorship (21 CFR 11.50(a)).

Electronic signatures and handwritten signatures executed to electronic records shall be linked to their respective electronic records to ensure that the signatures cannot be excised, copied, or otherwise transferred to falsify an electronic record by ordinary means.

21 CFR 11.70, signature/record linking

That requirement — the signature cannot be peeled off the thing it signed — is what lets a search engine treat a notebook entry as a source rather than as a note. In Scientific Literature Search a notebook entry is indexed only after it is signed, so a claim can never rest on an entry its author had not yet stood behind. The same rule has a side effect teams like: the index is a reasonable proxy for what the organisation is prepared to say, which is not the same as everything it has written down.

Before an internal source becomes an evidence row in Scientific Literature Search

  1. Signed

    A notebook entry enters the index only once it is signed off; an unsigned entry is not searchable and cannot support a claim.

  2. Dated, with its author named

    The source viewer names the authors and the signature date; for regulated records, the electronic-records rule also asks the signature to show its meaning — authorship, review or approval.

  3. Owned by a programme

    Who may read the source is written onto it when it is indexed, not decided later by whoever asks the question.

  4. Described well enough to compare

    Target, mechanism, indication, model system and n are recorded with the passage, so a row can be set beside a published one.

  5. Opens at its page

    Every row opens the source at the page it came from with the cited sentences highlighted; a row with no passage behind it is removed before the answer is shown.

How the solution treats an internal source. The first two conditions come from the signed-record rules; the last three are the solution’s own rules, set by the library administrator and versioned on the audit trail.

Who may read what: searching internal sources without breaking the walls

The reason most companies do not index their own reports is not technical. It is that a report belongs to a programme, and a search box that returns a line from it has disclosed something, even if nobody opened the file. Pharmacology data from a partnered asset, a translational interim from a blinded study, a notebook entry from a programme under a separate wall: each is readable by some people and not others, and a search engine that gets this wrong is worse than no search engine at all.

The workable answer is to decide access when the source is indexed rather than when the question is asked. In the solution, programme access is written onto each internal source at index time; published papers and patents are open to everyone in research, while internal reports and notebooks answer only to the programme that owns them. An overnight watch is held to the same rule and never surfaces a source its watcher may not open — the case that is easiest to get wrong, because nobody is in the room to notice.

Inside Scientific Literature Search: papers, patents, reports and signed notebooks in one list

The sources screen: a list of library sources with, for each, its type — paper, patent, internal report or lab notebook — the target and mechanism it covers, the model system and n, the programme that owns it, who may read it, the date it was indexed and how many answers cite it. Internal rows show a programme name such as VLT-302 and a programme-only access marker; papers and patents show open access to research.
Scientific Literature Search on its fictional sample library of 128,412 sources. Each row carries what the source studied and who owns it, so the same list serves a scientist looking for evidence and a library administrator checking who can see what.

Citing something with no journal, volume or page

A grey citation has to answer the same question a journal citation answers — where would a reader go to check this? — using whatever identifiers the source actually has. Each kind has some.

A conference abstract has a meeting, a year and an abstract number, and often a DOI for the supplement it appeared in. A registry entry has a registration identifier, which is more stable than most journal citations, plus the date the results were posted, because the posted table may be updated. A regulatory document has an agency, a document type and a date. An internal report has a report number, a version and a date, and a notebook entry has a notebook identifier, a page and the signature date. In every case the version and the date carry the weight: grey sources are revised more often than papers, and a citation without a version points at a moving target.

The practical test is whether the citation lets a colleague land on the same sentence. That is also the test the solution applies to itself: every evidence row opens its source at the page, with the cited sentences highlighted, so a reader never has to reconstruct where a number came from. Patent text is handled by the same machinery but labelled as a claim rather than a result, which is a distinction papers as prior art, claims as claims works through in detail.

The grey source open at the page the claim came from

The source viewer: an internal translational interim report open at page 18 with the cited sentences highlighted, stating that in three of four cases with MET copy gain a second pathway alteration was present in the same plasma draw. A side panel shows why the passage was returned, what the source studied, its model system and n, the programme that owns it, its named authors and who may read it.
A contradicting evidence row opened at its passage in the fictional sample library. The panel beside the page is what makes a grey source usable as evidence: what it studied, who wrote it, which version this is, and who is allowed to read it.

What changes in an answer once grey sources are in it

Three things change, and only one of them is “more results”.

The first is direction. Adding abstracts and posted registry results pulls an answer towards the full set of attempts rather than the published ones, which usually means a smaller effect and a wider range. If an answer moves when grey sources are added, that movement is the finding, not a nuisance.

The second is specificity. Internal reports carry your model systems, your assay formats and your n. An answer built from them stops being about a mechanism in general and becomes about a mechanism as this company measures it — which is what a study plan needs and what a review article cannot give.

The third is friction, in a good way. Once your own work is in the same answer as the literature, disagreements become visible on the day they arise instead of at a review meeting. In the solution a contradicting internal row turns amber, carries the agreement agent’s note on why the two may differ, and goes to the person who owns that internal source — who alone may explain or settle it. An agent never decides which side is right. To see all five source types answering one question, start from Scientific Literature Search.

Questions people ask

What are examples of grey literature?

Conference abstracts and proceedings, clinical trial registry records and their posted results, regulatory assessment documents, theses, government and institutional reports, preprints, and a company’s own research reports and signed lab-notebook entries. The common feature is that no commercial or academic publisher issued them, so no publisher deposited a citation for them in a bibliographic database.

Where do you search for grey literature?

In several places, one at a time: the relevant societies’ proceedings for abstracts, the trial registries for study records and posted results, the regulators’ own document libraries for assessment reports, institutional repositories for theses, and your own report store and notebook system for internal work. There is no single index, which is why a grey search has to be written down source by source to be repeatable.

Why is grey literature important?

Because whether a result reaches a journal depends partly on what the result was. Abstracts reporting a significant finding were more likely to be published in full, with a risk ratio of 1.31 across 45 reports and 15,783 abstracts. A journals-only search is therefore a sample skewed towards positive results, and the size of the skew is not knowable from inside the journals.

How do you cite grey literature?

With whatever stable identifiers the source has, plus a version and a date. An abstract: meeting, year, abstract number. A registry entry: registration identifier and the date results were posted. A regulatory document: agency, document type, date. An internal report: report number, version, date. A notebook entry: notebook identifier, page and signature date. Version and date matter more than for a paper, because grey sources are revised.

Can a search tool include our internal reports without exposing them?

Only if access is decided when the source is indexed rather than when the question is asked. In Scientific Literature Search, each internal report and notebook entry carries the programme that owns it from the moment it is indexed, and is answered only to that programme; papers and patents are open to everyone in research. The same rule binds the overnight watch, which never surfaces a source its watcher may not open.

Is a preprint grey literature?

By the usual definition, yes: a preprint has not been issued by a publisher and has not been through peer review. In practice it behaves better than most grey sources, because it usually has a stable identifier, a version history and a full methods section, so it can be cited precisely and read critically. Treat it as a full report whose peer review has not happened yet, and say on the page that it is a preprint.

Scientific Literature Search.
See it on your own files.

An agentic solution from Botminds. Ask in plain words; get numbered claims, each backed by evidence rows and the exact passage, checked against your own reports. Book a demo, or explore how it works.