botminds.aiRequest a demo
WorkflowAsset Management, Maintenance & Reliability

Root Cause Failure Analysis

Root cause failure analysis where an agent argues against the favored cause, people knock branches down on opened records, and the sister machines are found the same day.

See the demoBook a demo
Follow day 13 of a slurry pump shaft failure ↓
The problem

By four in the morning the pump had a story, and the story was a bearing

A slurry pump trips in the night and the standby picks up. Within the hour someone says it looks like a bad bearing, the work order gets a bearing code, and the housing is drained without an oil sample. Raw vibration data starts to roll off its system while the parts sit in a shop cage.

Weeks later the investigation team is arguing with a failure code. The vibration history, the lab, the purchase order and the stores record sit in different systems, and nobody is asking where else the same part is turning today.

Built forReliability engineersRotating equipment engineersMetallurgistsReliability managersProcess safety leads

The Problem Today

After a failure the first story sticks, and the same weak part keeps running in the sister machine.

48 hours
to start an incident investigation, by rule published
5 years
incident investigation reports must be kept published

Gathered by hand today

  • Work orders
  • Vibration
  • Lab reports
  • Purchasing
  • The failure code misleadsCoded bearing, when the shaft broke first
  • Evidence ages outRaw data rolls off and oil gets drained

The Solution

  1. Hold the evidenceRaw data held, physical items tracked, records gathered into one file
  2. Argue with each branchAn agent argues against the favorite and names the test that settles it
  3. People knock branches downThe checker reads the cited records before agreeing
  4. Find the sister machinesSame lot and same frame traced through purchase and work order records

Built-in controls

  • Agents never declare a root cause
  • A second person checks each knock-down
  • Checkers open the records before agreeing

What You Get

The fix lands on what actually broke the machine, and the panel can show why the other causes fell.

  • One evidence fileRecords held before they age out, each with its source
  • Sister machines foundSame lot, same frame, decided while the clock runs
  • Actions that fix a causeDrafted, rated by strength, and closed by people
One failure, day 13 of 45

Root cause failure analysis of a slurry pump shaft that broke at the keyway

A catalytic cracker slurry pump tripped on vibration in the night, and the standby took over in 38 seconds. The teardown found the shaft cracked through at the coupling keyway. Here is one working day of the analysis.

demoPumps, people and readings in this day are sample data, and so is the refinery.

  1. 01Day 13

    Ten branches on the tree, and the first story still says bearing

    Investigation lead

    The cause tree sits under a drawing of the pump train with the break marked in red. The first story, drive-end bearing failed first, has 4 records for it and none against yet. Two branches are already knocked down, each with who proposed and who agreed. The clock says the analysis started 8 hours 53 minutes after the trip.

  2. 02Day 13

    The agent argues against the story everyone believes

    Investigation lead

    Asked to argue against bearing first, the agent lays out its case. The 2× rose 52 days before the trip, the bearing tones came only in the last 31 hours, and both 1× and 2× phase drifted as they grew. It says what it could not check, the oil on the day, and names the test that settles it: how old the break is, from the lab.

  3. 03Timeline

    Phase moved 212° to 251° at steady load, while the other pump held within 4°

    Investigation lead

    A 2× rise alone cannot tell a crack from misalignment, and the pump had just been realigned. Phase separates them. Over the weeks before the trip, 1× phase moved from 212° to 251° and 2× phase from 104° to 168°, while the standby pump stayed within 4°. That points to the rotor changing, and the lab has to close it.

  4. 04Timeline

    Three spectra show the bearing tone arriving last

    Rotating equipment engineer

    Spectra for the three nights before the trip sit one above the other. On the first, 1× and 2× are there with no bearing tone. The inner race tone at 8.4× arrives on the second night with sidebands and grows on the third. On that first night the overall level was still 0.24 in/s, under the 0.30 alarm.

  5. 05Lab

    Beach marks over 78% of the section, and a keyway corner far too sharp

    Metallurgist

    The fracture face shows where the crack started, at the keyway root corner at the hub edge, with ratchet marks from several starts. Beach marks cover about 78% of the section and the final break 22%, square to the axis: rotating bending, not torsion. The drawing asks for a 0.031 in minimum root radius. The lab found 0.005 in.

  6. 06Day 13

    The checker opens three records before she can agree

    Rotating equipment engineer

    The investigation lead proposes knocking down bearing first, as a secondary effect, not a cause. The rotating equipment engineer is the second person. She opens the vibration trend, the drive-end spectra and the lab report herself, and only then do agree and disagree become live. Her opens go on the trail with her name and time.

  7. 07Day 13

    Bearing first falls, and two branches close with it, marked as not checked

    Rotating equipment engineer

    She agrees, and the branch falls with her name on it. The two branches under it, lube oil low and bearing fitted badly, close with their parent and say so: closed with parent, no check of their own. Shaft cracked by fatigue now leads, and the next open question on the tree reads: why did it crack now, and why this pump and not its twin?

  8. 08Purchasing

    A shaft from another supplier, bought to a sample with no change record

    Investigation lead

    The agent traces the shaft through purchasing and stores. Four shafts were bought from a second vendor to a sample, the drawing was never sent, and goods receipt took them as like for like. No change record exists. The agent adds an eleventh branch to the tree, and says plainly where the risk is: the twin pump runs on the same lot today.

  9. 09Day 13

    The same lot runs in the twin pump, so the manager decides today

    Reliability manager

    The lead's proposal: keep the twin pump running on watch until the failed one is back, then swap and pull its shaft, since it shows no 2× rise or phase drift and its alignment is right. A line under the lot table says to swap the third pump on the lot today, still to be proposed. Before he can agree, the manager opens the vibration trend, the stores record and the keyway lab report.

  10. 10Day 13

    Every shaft in the lot accounted for, and three pumps of unknown origin

    Reliability manager

    Decided: running on watch at 0.10 in/s, with its owner named. The lot table shows all four shafts: one broken, two running, one in stores and not yet blocked. Of 23 pumps on the frame, 17 are proposed as clear and need a sampled check by a second person. Three got a new shaft years ago with no supplier recorded, so they stay unknown, never clear.

What the pump said in the weeks before the shaft broke

Lined up on one timeline from vibration, oil, starts and work orders.

WhenWhat the records show
After a seal jobOffset left at 0.000 in, against a datasheet asking for cold offset
52 days beforeFirst 2× rise
Three nights beforeOverall 0.24 in/s, still under the 0.30 alarm
31 hours beforeFirst inner race bearing tone, at 8.4×
The trip0.52 in/s, standby running in 38 seconds

demoSample data, from the working solution. The 2× rise is never called a crack signal: on its own it cannot tell a crack from misalignment.

Around the tree

The evidence file, earlier failures and actions behind the cause tree

Six screens the panel, the action owners and the auditor read alongside it.

The evidence filePhysical items with who holds them, raw data held before it rolls off, and 1,312 records. Links the agent only proposed wait for a sampled check.
Has this failed before?Past pump work orders compared by what actually happened. Of the 11 above the match line, 6 carry the bearing code and only 2 were truly bearing failures.
Actions, rated by strengthNine drafted actions: who owns each, when it is due, how strong a fix it is. A reminder to follow a procedure is rated weak and can be dropped.
All failure analysesOpen analyses by level, lead, clock and stage, and recently closed ones with the root cause each panel signed.
Who opened what, and whenEach proposal, open, agreement and agent step on its own line. Branches closed with a parent are logged as notes, never as checks.
Clocks and limitsMatch line for similar failures, the report clock, the watch limit on a sister pump and how long reports are kept. One person proposes, another signs.

demoSample data, from the working solution.

Calls the investigation panel keeps for itself

Agents gather, line up, argue and trace. The panel keeps these.

Declaring a causeThe agent argues against the favorite and names the settling test. People propose and agree.
Knocking down a branchThe checker opens the cited records first, and a knocked-down branch can be reopened.
Calling a pump clearWith no record, a pump is unknown. It is measured at its next opening.
Stopping or starting equipmentThe reliability manager decides what the sister machines do, with an owner and a limit.
Changing a lab resultThe metallurgist enters results; the lead links them to the tree.
Behind the tree

Eight agents build the file and the timeline, and none can declare a cause

Each has one job and a line it holds.

Evidence keeper

Holds raw data, tracks physical items and proposes record links.

  • Never edits a source record
  • Reports lost evidence
Timeline builder

Lines up vibration, process, oil, starts and work orders.

  • Never fills a gap in the data
Failure matcher

Reads free text and teardowns to sort same mechanism from same words.

  • Never treats a failure code as proof
Devil’s advocate

Argues against the favored cause and names the settling test.

  • Never declares a cause
  • Never knocks down a branch
Lab liaison

Proposes which branch a lab result touches.

  • Never changes a lab result
Sister finder

Finds where the part, lot, vendor or frame runs.

  • No record means unknown
  • Never stops or starts equipment
Action tracker

Drafts, rates and chases actions.

  • Never closes an action
Report drafter

Drafts the report and lesson from decisions.

  • Never writes a cause nobody declared

Eleven systems in one failure file, held before the data ages out

Read only. Raw data is held the day the analysis starts.

Condition monitoringVibration trend, spectra and phase
Plant data historianProcess data around the trip
Operations logbookShift log entries and starts
Maintenance systemWork orders on the pump and its frame
Drawings and datasheetsShaft drawing, keyway radius, cold offset
Purchasing and storesPurchase orders, goods receipts and issues
Lab and inspectionFracture, profile and chemistry reports

What reliability teams ask about root cause failure analysis

What is root cause failure analysis?

Root cause failure analysis is the investigation of why a piece of equipment failed, down to the physical, contributing and system causes. Here agents gather the evidence, line up the timeline and argue against the favored story. People propose and knock down branches on opened records, and a panel signs the cause.

How is RCFA different from coding the failure on the work order?

A failure code is written in the first hour, often from the most visible damage. The analysis tests it. In the sample, the work order said bearing, and the shaft had cracked first.

What does RCFA software need to read?

Vibration trends and spectra, process data, shift logs, work orders, drawings and datasheets, purchasing and stores records, and lab reports. Raw data is held as soon as the analysis starts, before it rolls off its system.

How are sister machines handled during an equipment failure investigation?

The same part, lot, vendor and frame are traced through purchase, stores and work order records while the analysis runs. Where no record exists, a machine stays unknown and is measured at its next opening, and the reliability manager decides what runs.

Can the agent decide the root cause?

No. The agent argues against the leading story and names the test that would settle it. A cause is proposed by the investigation lead and agreed by a second person who has opened the records.

What does an auditor see afterwards?

Each proposal, each record opened and by whom, each agreement and each agent step, with branches closed alongside a parent shown as not checked on their own. The trail downloads, and so does an evidence pack.

Asset Management, Maintenance & Reliability

Other asset integrity and reliability solutions

All 10 solutions →
Maintenance and Inspection Deferral Management

Each deferral judged with everything it leans on, not alone.

1-min demoCritical Spares Analysis

One case both sides sign, and spares you hold stop being bought twice.

1-min demoPreventive Maintenance Optimization

Routines that find nothing relaxed, routines that miss failures tightened, each change signed by an engineer.

1-min demo

Bring one failure
everyone already has a story for.

We’ll take one of your own failures through the timeline, the tree and the search for its sister machines.

Book a demoBrowse solutions