botminds.aiRequest a demo
WorkflowEngineering Data, Tags & Documents

Engineering Records Classification and Deduplication

Engineering document classification and deduplication that sorts file shares into one family per drawing, crowns the signed copy and sends a changed value to an engineer.

See the demoBook a demo
Follow one P&ID from 37 copies to one ↓
The problem

The same drawing in dozens of folders, and one copy with a different value

An engineering estate grows for decades across file shares, contractor drives and the folders of people who left. The same P&ID sits in dozens of places, re-plotted, stamped and scanned, and nobody can say which copy governs.

A duplicate finder compares file hashes, so a stamp and a changed set pressure look equally different. A title block check is worse: it calls two files the same when their content is not. And a hand-marked print looks like a spare copy, though its pencil note may be the only record of a field change.

Built forDocument control leadsRecords analystsDiscipline engineersInformation managersInternal auditors

The Problem Today

The same drawing sits in many folders, and a copy with the right letter can hide a different value.

Every
P&ID and relief basis is listed process safety information published
99
person-years to sort 12.4 million files at a minute each our estimate

Sorted by hand today

  • P&IDs
  • Isometrics
  • Datasheets
  • Manuals
  • Hashes miss the meaningA stamp and a changed value look the same
  • Notes that look like copiesA marked print can hold the only field record

The Solution

  1. Scan without movingShares read with a read-only account; nothing moved or renamed
  2. Sort into familiesType, tag and revision read from the title block, not the file name
  3. Overlay the copiesA stamp is set aside; a changed value goes to an engineer
  4. Retire in signed lotsSampled lots with an owner, a clock and a second signature

Built-in controls

  • Agents only read, and never delete a file
  • Changed values go to an engineer
  • A legal hold beats any retirement

What You Get

Engineers judge only the few copies that matter, and each file carries a plain reason to stay or go.

  • A crown only with a signatureTaken from the signed issue, whatever the newest date says
  • Hand notes kept as evidenceMarked prints on their own branch, each needing a verdict
  • Savings after sign-offOnly files signed twice and moved are counted
One drawing, one ruling

Engineering document classification on one compressor P&ID with 37 copies

The wet gas compressor interstage drawing on an FCC unit has drifted across the refinery's shares for three decades. Twenty-three files say Rev F. One of them shows the drum's relief valve set at 150 psig, where the signed issue says 125.

demoThe refinery, its shares, the people and the counts are sample data.

  1. 01First screen

    The worst family of 142,213 opens first, drawn as a tree

    Document control lead

    Rev 0 sits at the root and the trunk climbs to Rev F. The copies found on the shares hang from the revision they claim. The signed Rev F wears a crown, hand-marked prints have their own violet branch, and one file sits alone in red. A file hash sees 22 different files here, because only 2 match byte for byte.

  2. 02The build

    Copies settle onto their branches as the agents lay them over the signed issue

    Agents, for the lead

    Pressing the build replays how the family was put together. Files gather on the Rev F branch, then sort: plot dates, stamps and scan quality on one side, hand notes on another. Notes with sources run beside the tree, so the lead sees why each copy landed where it did.

  3. 03The odd one

    It says Rev F, and it was copied out of an old capacity study's working folder

    Document control lead

    The red file's card gives its path on the turnaround share and what makes it different: PSV-1322 shows 150 psig. It came from a study folder, and its byte-for-byte twin is still filed under that study. The copy has been opened 41 times by 6 accounts. The agents do not call it a duplicate and do not say which value is right.

  4. 04Overlay

    One tag bubble changed, and the drum label still reads design 125 psig

    Document control lead

    Laid over the signed copy, the overlay shows only what differs. Two differences: the plot date in the margin, harmless, and PSV-1322's set pressure, 125 psig to 150 psig, marked as an engineering change that never goes out as a duplicate. The title block calls the two the same. She sends it to the discipline engineer instead of the retire pile.

  5. 05Evidence

    The bench test opened at 124 psig and was reset at 125

    Discipline engineer

    The ruling page puts the evidence in one place: the drum's manufacturer's data report, the signed valve datasheet, the bench test from the last turnaround, the capacity study and the project decision note that dropped its option. He opens the bench test first. The valve on the drum agrees with the signed issue, so the 150 psig copy describes a study that was never built.

  6. 06Ruling

    Keep the signed Rev F, and mark the study copy not for use

    Discipline engineer

    He picks the first ruling and the reason, test record agrees with the signed issue. The panel above shows what the ruling does to the family: 18 kept, 4 hand-marked prints kept as evidence with their own verdicts, 14 retired with a pointer to the kept copy, and the different content held. Nothing moves yet.

  7. 07Check

    His ruling pulled the study twin in, and the family grew to 38 files

    Document control lead

    As the second person she cannot sign until she has opened the overlay and his evidence. The check panel counts it: overlay opened, evidence opened 2 of 2. Only then does the sign button come alive, offering to retire 14 files, beside a button to disagree and reopen it for the engineer.

  8. 08Signed

    Signed, and the four hand notes still need their own verdicts

    Document control lead

    Signed now, with the steps that got it here listed in order. Yet the hand notes on this drawing, a low level alarm, a moved gauge, a flush line not as drawn and a startup note, still say needs a verdict: a signature on the family does not settle a print. The move itself waits for the information manager.

  9. 09The move

    The 150 psig file leaves the share, and a NOT FOR USE stub takes its place

    Information manager

    She confirms the move as the named person who carries it out. The tree now shows the signed copy kept, 10 that must stay and 8 retired on the Rev F branch, and two files moved to quarantine. A stub stands at both old paths, and the 6 accounts that opened the wrong copy are told. Retired copies are set aside with a pointer.

Where the 38 files of one Rev F family end up after a signed ruling

The ruling on the compressor drawing, file by file, as the ruling page counts it.

Kept: the signed copy, one of each revision and copies that must stay18
Kept as evidence: hand-marked prints, each with its own verdict4
Retired: set aside, with a pointer to the kept copy14
Held: different content, never out as a copy2
Files in the family once the study twin joined38

demoSample data, from the working solution.

A read-only scan of 31 shares turns 12,416,384 files into 2,904,117 families

The estate's ledger as the scan screen shows it. Every count below adds back to the files found.

Line in the ledgerCount
Files found on 31 shares12,416,384
Could not be read, held for a person41,208
Not engineering records1,862,540
Engineering files10,512,636
Families2,904,117
Governing copies, in families that are clear2,761,904
Families waiting for a person142,213

demoSample data, from the working solution.

Around the family

The whole estate, the prints and the lots that retire copies in bulk

What the records team, the information manager and an auditor open around one drawing's ruling.

Families that need a personWorst first, with four reasons: 9,285 same letter with different content, 71,540 with no clear governing copy, 38,177 unclear type or revision, 23,210 with an unknown tag.
The scan, nothing movedEvery share, its owner and size. The weekly change scan flags one share whose read account was refused, so its changes are not yet seen.
Hand-marked printsEach pencil or pen note is read, with how sure the reading is, and waits for a verdict: keep as evidence, send to an engineer, or overlay. Silence retires nothing.
A near-copy lot, before signingThis near-copy lot holds 148,902 files. Its sample of 40 starts with the lowest confidence; 40 clean means 95% sure fewer than 8 in 100 are wrong.
One revision, two contractorsEach contractor issued its own Rev 4 of a demethanizer overhead P&ID: valve 530-FV-2204 is 6 in on one and 8 in on the other. It waits for the engineer.
Savings, counted lateOnly retirements that two people signed and a named person moved: 1,204,374 files, 9.8 TB. Lots still waiting and copies under legal hold sit beside it, uncounted.

demoSample data, from the working solution.

Four copies a hash or a title block would have retired

From the sample estate, each one caught by laying a copy over its signed issue.

A study value under the right letter

PSV-1322 at 150 psig on a copy labeled Rev F, opened 41 times from a turnaround share.

A twin in another folder

The same file, byte for byte, filed under the study. The ruling pulled it into the family and quarantined both.

A pencil note on a spare Rev 0

Of 8 spare older copies, 6 match their kept copy; one carries a vendor stamp and one a pencil note, so both stay.

Two contractors, two valve sizes

One contractor's Rev 4 shows 6 in, the other's shows 8 in. Both stay held, not retired, while the engineer has the family.

demoSample data, from the working solution.

Five people around one family, and none signs their own proposal

A ruling needs a maker and a different checker; an engineering difference needs an engineer first.

Records analyst
Sees
The queue, the read type, tag and revision with confidence
Signs
Drafts rulings and builds lots, which someone else checks
Document control lead
Sees
Copies laid over the signed issue, and only what changed
Signs
Sends changes to an engineer, and checks rulings after opening the evidence
Discipline engineer
Sees
The family's evidence: data report, datasheet, bench test, study
Signs
Rules on engineering differences, with a reason
Information manager
Sees
Lots, holds, retention rules and the move list
Signs
Owns lots and confirms each move as the person who carries it out
Internal auditor
Sees
The trail of agents and people, and the evidence exports
Signs
No signature: reads the trail and downloads the evidence pack
The agents

Seven readers of the shares, and none of them moves a file

They list, read, group and compare. People rule, sign and move.

Share scanner

Lists the files on each share read-only and takes a hash.

  • Never moves, renames or deletes
  • Stops when access is refused
Type and tag reader

Sets type, tag and revision from the title block and the body.

  • Never invents a tag
  • Below the confidence line, hands it to the analyst
Copy finder

Groups files into families, revisions in order.

  • Never joins two families without a reason on record
Overlay checker

Lays copies on the signed issue and sorts a stamp from a change.

  • Never calls a change in a tag bubble harmless
  • Never picks a value
Mark reader

Finds and reads hand notes on prints.

  • A marked print is never a duplicate
Governing copy finder

Puts the signed issue forward as the copy that governs.

  • No signature, no governing copy
  • A newest date proves nothing
Rule checker

Checks retention and holds, routes by discipline and builds lots.

  • Never retires a file
  • A hold beats every rule

What document controllers ask about engineering document classification

What is engineering document classification and deduplication?

Engineering document classification and deduplication finds the copies of each drawing or document across shares, groups them into one family per document, and decides what is kept. Here the copies are laid over the signed issue, so a harmless stamp is told apart from a changed value, and a person rules on anything that differs.

Why does a hash-based duplicate finder miss the dangerous copy?

A hash says two files differ, never how. A re-plot, a stamp and a changed set pressure all look equally different to it. The overlay shows only what changed and labels it harmless or an engineering change, and an engineering change goes to a discipline engineer.

How are type, tag and revision read off a drawing?

From the title block and the body of the drawing, not the file name, with a confidence for each read. Below the confidence line the read goes to the records analyst, and a correction is checked by someone else.

What happens to hand-marked prints?

They are never treated as duplicates. Each note is read, the print sits on its own branch of the family, and each one needs its own verdict: kept as evidence, or sent to an engineer when the note is not on the drawing.

Can we clean up millions of copies without approving each one?

Copies retire in lots. Each lot has an owner, a clock, a sample with a recorded verdict per file, a second signature and a named person who carries out the move. The lot screen states what a clean sample would and would not prove.

Does anything get deleted from our file shares?

No. Agents only read. A retired copy is set aside with a pointer to the kept copy, under your retention schedule, and a file under legal hold cannot move. Savings count only what two people signed and a person moved.

Is this the same as ROT data cleanup?

It covers the same ground for engineering files, redundant, obsolete and trivial copies, but it judges a copy by what is on the drawing. Stamped copies, marked prints and copies in a record set stay; plain copies retire.

Engineering Data, Tags & Documents

More for engineering files, tags and drawings

All 11 solutions →
P&ID Digitization

Plain questions for engineers, and every line followed from sheet to sheet.

1-min demoEquipment Datasheet Extraction

Only the lines that matter kept, each one marked meets, differs or missing.

1-min demoAs-Built Reconciliation

One field walk answers the open questions, and each safety item is read by two people.

1-min demo

Point it at one share
you have been afraid to clean.

Pick a drawing with too many copies, and we’ll show you its family tree, the overlay and the ruling path.

Book a demoBrowse solutions