SearchIT/OT Security & Compliance

OT Incident Response

OT incident response for electricity and water: your own playbook step, drawing, contact and reporting clock behind one plain question

The right playbook step at 3 a.m.

See one case, screen by screen ↓
demo39minto spare on the CIP-008 one-hour clock: determined Reportable at 03:41, notice signed and sent at 04:02
demo0customersaffected by the containment rule, with 58,400 customers behind the 7 RTUs reached shown before anything was blocked
target100%of reportable incidents notified inside the one-hour clock, with the determination time on record
target< 10sfrom a plain question to the cited playbook step, against 22 minutes in the demo utility's last tabletop
The problem

Why the right playbook step takes 22 minutes at 3 a.m.

OT Incident Response is software that answers OT responders from their own incident response plan. The night SOC analyst types what is happening in plain words, and the plan's playbook opens as a decision tree at the step they are on, each instruction quoted from its page, with the zone drawing, the vendor's security line and the reporting clock beside it. Today those answers sit in a 64-page plan, nine playbooks, 14 zone and network drawings, 23 vendor contracts and five sets of reporting rules, in binders and on SharePoint, written for the auditor. In the demo utility's July 2026 tabletop the night team took 22 minutes to reach the right playbook step and 31 minutes to find the vendor's security hotline.

Meanwhile the clocks run. Under CIP-008-6 the initial notification to the sector ISAC and the national cyber agency is due one hour after the determination of a Reportable Cyber Security Incident, not after detection, so a team that does not record when it determined cannot prove it met the hour. The drawing that shows which RTUs sit behind which firewall is in a different folder from the playbook, so pulling the wrong conduit can take the control room's view of the grid away. And the timeline is rebuilt afterwards from memory, so the notification, the 7-day update and the 90-day lessons learned are written late.

typical≈ 50%of ICS/OT organisations have no ICS/OT-dedicated incident response plan
typical82%of organisations in Dragos's incident response work lack clear criteria for when an operational anomaly should trigger a cyber investigation
typical73%of Dragos's OT incident response cases, all time, involved compromised VPN or jump-host credentials
Where the first hour of an OT incident goes, in minutesestimated
By hand108 days
With the solution12.7 days
  • Finding the right playbook and the step you are on22 → 0.2 d
  • Finding the vendor's security line and contract clause31 → 1 d
  • Checking the zone drawing for what a block would cut15 → 1 d
  • Working out which reporting clock applies and when it starts10 → 0.5 d
  • Drafting the notification from notes and memory30 → 10 d

Estimated split, in minutes, by hand and with the solution. The first two “before” figures are the demo utility's own July 2026 tabletop (22 and 31 minutes); the rest are our estimate.

How it works

How a question becomes the next step

Six agents find the step, the drawing, the contact and the clock; the incident commander decides every branch and signs.

What comes in
What they answer from5 sources · plan, playbooks, drawings, contracts, rules
Agents at work
Incident classifiertype + playbook
Then
Playbook navigator
Zone and asset resolver
Contact finder
Then
Reporting clock keeperrules choose the clock
Then
Timeline scribecited notice
A person decides
OT incident commanderdecides every branch, signs
What comes out
Cited step
Notice sent
Clock met
One case, step by step

One night at Northgate, from the alert to a notice sent inside the hour

Lumina Utilities runs electricity and water: 41 substations, two control centres and six water works. On Tuesday 13 October 2026 at 02:51 a remote session opens on jump host JH-02 with corv-svc2, a vendor account of Corvane Automation, from an address not on Corvane's approved list. Here is what happens next, screen by screen, in the working solution.

  1. 0103:07

    The alert lands on the Respond screen, with the timeline already started

    Sam Okoro · OT SOC analyst, night shift

    Alert OTS-88213: unusual DNP3 function codes from engineering workstation EWS-14 at Northgate, severity high. The running timeline already holds what the sensor saw: the JH-02 session at 02:51 with no booked window, EWS-14 reading 7 RTUs where its normal set is 2, and two DNP3 select commands to breaker CB-212 at Marlow Cross 33 kV, with no operate. The nine playbooks of plan IRP-OT-01 v6.1 sit above it.

    “No clock running yet. Five rule sets are read for every incident: CIP-008-6, the water licence, the cyber policy, CIRCIA and the plan's own targets.”

  2. 0203:09

    One plain question opens PB-OT-04 at step 1

    Sam Okoro · OT SOC analyst

    Sam types “Suspicious traffic on the SCADA network?”. The incident classifier names playbook PB-OT-04, “Suspicious activity on the control network”, and the navigator opens it as its nine-step decision tree at step 1, quoted from §3.1, page 6. The plan's 60-minute determination target starts from detection: 03:07 to 04:07. Beside the step, the agents have found the session on JH-02, that EWS-14 is approved for 2 RTUs read only, and Corvane's 24/7 security line.

    PB-OT-04 §3.1: “Check the source against the approved connections list (ACL-SCADA). Call the vendor's security line to confirm any vendor session. Do not disconnect the source yet.”

  3. 0303:14 – 03:22

    Not approved. Control not affected. The tree moves

    Elena Varga · OT incident commander, on call

    Sam calls Corvane's security line under contract CV-2291 §9.2: no work is booked tonight and corv-svc2 was not issued. Step 1 is answered “Not approved”, and the contact finder pages Elena at 03:15; she joins the bridge from home at 03:19. Joel Brandt, the Northgate shift supervisor, confirms no uncommanded operations, CB-212 closed and full view of all 41 substations. Branch: contain at the conduit, keep control.

    PB-OT-04 §3.3: “Contain at the conduit, not at the device. … Keep the workstation powered: do not reboot or log off.”

  4. 04Before anything is blocked

    The drawing shows who stays on supply

    Zone and asset resolver

    On zone drawing NET-DWG-0142 rev F, tonight's path is traced from 203.0.113.47 through JH-02 to EWS-14 and the seven RTUs. Two ways to contain sit side by side. Block EWS-14 at firewall FW-L3-02 with pre-staged rule 41: the front-ends keep polling all 41 substations and 0 customers are affected. Pull FW-L3-02 itself: the control room loses its view and 58,400 customers behind the 7 RTUs are out of sight.

    The resolver reads the drawing and never sends a command to a control system or a firewall. Sam enables rule 41 himself.

  5. 0503:33

    Contained, and the determination is hers

    Elena Varga · OT incident commander

    Sam disables the vendor path for corv-svc2 on JH-02 at 03:29 and enables FW-L3-02 rule 41 at 03:31; the resolver checks that FE-01 and FE-02 still poll 41 substations. Elena records the containment, with EWS-14 kept powered for memory capture. Step 4 sets out the facts: JH-02 is an EACMS of the Medium Impact Northgate EMS, EWS-14 is a BES Cyber Asset inside the perimeter, and what each possible answer starts.

    “Reportable: notify within 1 hour. Attempt to compromise: by the end of tomorrow, 14 Oct. Neither: no external notice.”

  6. 0603:41

    Reportable — and the rules swap the clock

    Reporting clock keeper

    Elena determines a Reportable Cyber Security Incident. The clock keeper stamps the time and replaces the plan's determination target, met with 26 minutes to spare, with CIP-008-6 R4.2: notify the sector ISAC and the national cyber agency within one hour, due 04:41. The 7-day update and the cyber insurer's 72-hour notice under policy NMS-CY-22871 clause 6.1 arm with it. The water licence clock stays n/a, and CIRCIA is shown but not counted, because it is not yet in force.

    The clock keeper never moves a start time a person recorded, and never sends a notification.

  7. 07The notice

    Drafted from the record, signed by a person

    Timeline scribe

    The scribe drafts the initial notification from 14 timeline entries, with the functional impact, attack vector and level of intrusion CIP-008-6 Part 4.1 asks for, each cited to its entry: “Reached a BES Cyber Asset (EWS-14) inside the Electronic Security Perimeter through an EACMS (JH-02). Operation of CB-212 was attempted, not achieved.” Elena reads it, edits it and signs as OT incident commander.

    “The signature, the time and the text are kept on the record for three calendar years.”

  8. 0804:02

    Sent inside the hour, 39 minutes to spare

    Elena Varga · OT incident commander

    The notice goes to the sector ISAC portal and the national cyber agency, with a copy to Farida Nasser, the CIP Senior Manager, who was informed of the determination at 03:42. The clock stops and records the minutes to spare. The 7-day update clock under Part 4.3 is armed and runs from the first new or changed fact.

    “Sent at 04:02:00 by Elena Varga … 39 minutes inside the hour.”

  9. 09Afterwards

    The key times the auditor asks for

    Timeline scribe

    Every sensor event, branch answered, call and signature is on the running timeline of INC-2026-0417 with its source. The key times sit beside it: detected 03:07:00, playbook opened 03:09:09, contained 03:33, determined 03:41:00 Reportable, notified 04:02:00, 39 of 60 minutes to spare. The same record feeds the 7-day update and the lessons-learned review, due 11 January 2027, 90 days on.

    The scribe never edits a signed entry, and every line keeps its source.

  10. 10The same morning

    Every open incident on one clock line

    Omar Haddad · Plant manager, Westbrook water treatment works

    Halden Ridge is one of four incidents on the clock line. At Westbrook water works, 41 failed logins on HMI-WB-03 run on the water licence's clock instead: WL-7, 24 hours from detection, due 14 Oct 01:58. Fenby wind farm's 7-day update is due 18 Oct 16:40, and the Ridgeway substation's lessons-learned review is due 31 Dec. Each clock was chosen by the incident's own facts.

    “My clock is not NERC's clock. Show me mine.”

Who it’s for

Built for everyone on the bridge at 3 a.m.

The same incident, seen by the five people who carry it — what that night looked like, and what it looks like now.

EV
Elena VargaOT incident commander, on call 9–16 Oct · decides every branch and signs the notice
OT incident commander
Before
Paged at home at 3 a.m., she has to decide every branch of a 64-page plan and prove when she determined the incident reportable.
Now
Joins the bridge to the playbook open at the right step, the facts for her determination and the clock it starts. “Tell me what step we are on, what it says, and how long I have.”
SO
Sam OkoroOT SOC analyst, night shift · 41 substations, 2 control centres, 6 water works
OT SOC analyst
Before
Watches the OT sensor console from 19:00 to 07:00. “I see the alert in seconds. Finding the right page of the plan takes twenty minutes.”
Now
Asks in plain words and gets the playbook step quoted from its page, the session log, the approved connections and Corvane's security line with its clause.
JB
Joel BrandtShift supervisor, Northgate control centre · runs the grid from the EMS
Control-room shift supervisor
Before
Has to say whether control is affected while someone else decides what to isolate. “Before you isolate anything, tell me which customers are behind it.”
Now
Sees the conduit, the rule and the customers behind every RTU on the zone drawing before anything is pulled — 0 customers affected by rule 41.
FN
Farida NasserCIP Senior Manager · owns the plan, informed of every Reportable determination
CIP Senior Manager
Before
Owns the plan, its tests and its lessons learned. “If we determine it at 03:41, the notice is out by 04:41. No exceptions.”
Now
Approves every change to the rules and targets as a new version, and sees which playbook is due its 15-month test and which drawing is older than the last firewall change.
OH
Omar HaddadPlant manager, Westbrook water treatment works · answers to the water licence
Plant manager
Before
Owns the treatment control system and its licence. “My clock is not NERC's clock. Show me mine.”
Now
Sees a water-side incident on the water licence's 24-hour clock from detection, beside every other open incident's clocks.
Built on the engine

6 agents. Each with one job, and hard limits.

Six agents find the step, the drawing, the contact and the clock; the incident commander decides every branch and signs.

Incident classifier

Reads the responder's question and the open OT sensor alerts, names the incident type in the plan and the playbook to open, and says why.

  • Never declares an incident reportable — the incident commander decides
  • If no playbook matches, says so and opens none
Playbook navigator

Opens the approved playbook as its own decision tree, highlights the current step and quotes it word for word with its section and page.

  • Never skips, reorders or invents a step
  • Answers only from approved versions
  • A branch moves only on a person's answer
Zone and asset resolver

Finds every asset in the incident on the current zone drawings and, for each way to contain, shows the conduit, the firewall rule and how many customers sit behind each RTU.

  • Never sends a command to a control system or a firewall
  • Flags a drawing older than the last firewall change
Reporting clock keeper

Re-reads the reporting rules at every answer, starts each clock at the event its rule names — detection, determination or new information — swaps and stops clocks, and reminds the incident commander before each deadline.

  • Never sends a notification
  • Never moves a start time a person recorded
  • Shows rules not yet in force without counting them
Contact finder

Matches the incident to the vendor contracts, the forensic retainer, the notice clauses and tonight's on-call rota, and names who to call with the clause that says so.

  • Pages only the on-call list in the plan
  • Vendors and regulators are called by a person
Timeline scribe

Writes every event, answer, call and signature on the running timeline with its source, and drafts the initial notification, the 7-day update and the lessons-learned report from it.

  • Never edits a signed entry
  • Every line keeps its source
  • Never sends a notice
OT incident commander

Decides every branch and signs the notice. The agents propose; a named person decides.

Ask in plain words

Ask about the plan, the drawings or the clocks

Ask about the plan, the drawings, the contracts or the clocks. Every answer is quoted from an approved document.

Is this reportable under CIP-008?

That is your determination, at step 4. The facts the agents found: JH-02 is an EACMS of the Medium Impact Northgate EMS, and it was used to reach EWS-14, which armed CB-212 without operating it. A compromised EACMS meets the plan's definition of Reportable. If you determine it Reportable, the notice to the sector ISAC and cyber agency is due one hour after the time you record.

Who do I call at Corvane at this hour?

Corvane's security line runs 24 hours a day on +1 (555) 014-2290; for a Severity 1 security incident they call back within 30 minutes (CV-2291 §9.2). Their only booked session this week was CHG-26-1184 on 12 Oct, so corv-svc2 at 02:51 was not theirs to book.

What happens to customers if we block EWS-14 at FW-L3-02?

Nothing. Rule 41 on FW-L3-02 is pre-staged to deny EWS-14 only; front-ends FE-01 and FE-02 keep polling all 41 substations, so the control room keeps its view. Pulling FW-L3-02 itself would take that view away and put you on PB-OT-07.

Draft the notice for the sector ISAC and cyber agency

The draft is open with the three attributes CIP-008-6 Part 4.1 asks for: functional impact, attack vector and level of intrusion, each from a timeline entry. It needs your determination at step 4 first; then you read it, edit it and sign. I do not send it.

Every screen

The working solution, as it ships.

13 screens from the working solution, on its sample data. Pick one to see it large.

The incident roomPB-OT-04 as its decision tree at step 5, the CIP-008 one-hour clock counting down from the determination, and every rule this incident carries.
Before the questionThe nine playbooks of the plan, the live alert, the running timeline from the sensor, and no clock yet.
Step 1: confirm the sourceThe step quoted from its page, the answer buttons for the branch, and what the agents found: the session, the approved connections and the vendor's line.
Step 3: contain at the conduitTwo branches answered and traced; the containment step with the firewall rule to apply and the one not to pull.
The zone drawingTonight's path through the zones, the conduit's rule, and two ways to contain with the customers each one affects.
Step 4: the determinationThe facts for the incident commander's determination and what each answer starts: one hour, end of next day, or no notice.
Sign and sendThe notification drafted from the timeline, each attribute cited, signed by the incident commander.
Notice sentSent at 04:02, 39 minutes inside the hour, with the 7-day update clock armed.
The running timelineEvery event, answer, call and signature with its source, the key times the auditor asks for and the notice's attributes so far.
Reporting clocksEvery open incident and every clock its rules started, on one time line, with what is met and what is due.
Contacts and escalationTonight's rota, the escalation order from the plan, the vendors and retainer with their clauses, and the notices the rules require.
Plans, drawings, contracts and rulesEverything the answers come from, with version, owner and last test, and what is due or out of date.
Your rulesWhich clock starts from which event and goes to whom, the plan's targets and reminders, and who decides and signs.
Governance

Built for OT: read-only, cited, signed by a person.

Quoted from the approved planEvery instruction is quoted from an approved document with its version, section and page — PB-OT-04 v3.2 §3.3, page 8. Superseded versions are never answered from, and the library flags a drawing older than the last firewall change.
Read-only towards OTNo agent can send a command to a control system, a firewall or an account. Containment is carried out by a person and recorded on the timeline, as Sam did with FW-L3-02 rule 41.
A person decides and signsThe incident commander answers every branch, makes the determination and signs the notification and its updates. The settings mark each of these “person only”.
The clock starts at the event the rule namesThe determination time is stamped when the incident commander records it, and no agent can move it. Rules not yet in force, such as CIRCIA, are shown and never counted.
One timeline, kept three yearsEach sensor event, branch, call and signature is written with its source and time. Signed entries cannot be edited, and the record is kept three calendar years — the evidence for the dated notices and the lessons learned.
Rule changes are versions the CIP Senior Manager approvesEvery change to a reporting rule, target or reminder is a new version, approved by the CIP Senior Manager, who is also informed automatically of every Reportable determination.
Configuration

Your plan's clocks and targets, not ours

Which clock, from which event, to whom — and who decides — are settings, approved by the CIP Senior Manager.

SettingDefaultChoose from
CIP-008-6 R4.2 · Reportable1 hour from determinationCounted on or off · to the sector ISAC and the national cyber agency
Water licence WL-724 hours from detectionCounted on or off · to the Calder Valley Water Commission, for water incidents
CIRCIA · covered incidentOff72 hours from reasonable belief, counted once the final rule is in force
Determination target60 minSet in minutes from detection, per IRP-OT-01 §2.3
Page the incident commander if not escalated10 minSet in minutes, per IRP-OT-01 §4.1
Reminders before a deadline20 min, then 10 minSet in minutes · first to the incident commander and deputy, second also to the CIP Senior Manager
Offline copy on the control-room terminalOnOn · Off — plan, playbooks and drawings, refreshed at every approval
Agents may send commands to OT systemsNeverRead-only by design; containment is done by a person
Connections

Works with the documents your response already depends on

Incident response plan and playbooksthe OT cyber incident response plan and each playbook's steps, approved versions only
Zone and network drawingszone drawings, the approved connections list and the connectivity model's customer counts
Vendor support contracts and retainersecurity lines, call-back times, remote-access terms and the forensic retainer
Regulatory reporting rulesCIP-008-6, the water licence, the cyber insurance policy's notice clause and CIRCIA
OT sensor alerts and session logsthe alert that starts the incident and the jump-host session log
On-call rotathis week's response team and the plan's escalation order
What it changes

The difference, in numbers.

Every figure is labelled: a target the solution is built to, an estimate, a typical published result, or a proven one.

target
100%
of reportable incidents notified inside CIP-008's 1-hour clock, the determination time on record
Notified at minute 21
39 minutes to spare in the hero case
target
0
customer outages caused by the response: the customers behind every conduit shown before it is pulled
target
< 10s
from a plain question to the cited playbook step, against 22 minutes in the last tabletop
Binder and SharePoint22 min
With agents< 10 s

“demo” = seen in the working solution, on its sample data · “target” = the design goal, measured in the live solution · “estimated” = our estimate, except the 22 and 31 minutes, which are the demo utility's own tabletop figures · “typical” = published figures (SANS OT response planning guidance — about 50 % without an ICS/OT-dedicated plan; Dragos 2026 OT Cybersecurity Year in Review — 82 % and 73 %). Regulatory context: NERC CIP-008-6, Table R4; CIRCIA is not yet in force. Lumina Utilities and the people, sites, vendors, systems and incidents named on this page are fictional — characters and sample data in the working solution.

Questions

What OT security teams ask us.

What is OT incident response software for utilities?

Software that puts a utility's OT incident response plan, playbooks, zone drawings, vendor contracts and reporting rules behind one question at the moment of an incident. In OT Incident Response a responder types what is happening, the right playbook opens at the right step with every instruction quoted, and the reporting clock the rules choose runs beside it. The incident commander decides every branch and signs.

How does an ICS incident response playbook work in it?

The playbook navigator shows your approved playbook as its own decision tree, with the current step lit and quoted word for word from its page. A branch moves only when a person answers it, and each answer writes the timeline. In the sample case PB-OT-04 has nine steps, from confirming the source to lessons learned.

How does it handle the CIP-008 one-hour reporting clock?

CIP-008-6 sets the initial notification at one hour after the determination of a Reportable Cyber Security Incident, by the end of the next calendar day for an attempt to compromise, and an update within 7 calendar days of new information. The clock keeper starts each clock at the event its rule names, from the time the incident commander records, and the notice is drafted from the timeline with the three attributes Part 4.1 asks for.

Does it isolate systems or send commands to SCADA?

No. The agents are read-only towards OT: none can send a command to a control system, a firewall or an account. The zone and asset resolver shows the conduit, the rule and the customers behind each RTU, and a person carries out the containment and records it.

Can it cover water treatment incidents as well as the grid?

Yes. Each incident's facts choose its clocks. In the sample, failed logins on a Westbrook water works HMI run on the water licence's 24-hour clock from detection under playbook PB-OT-08, while the Northgate incident runs on CIP-008-6.

Do people stay in control of the response?

Yes. The incident commander answers every branch, makes the determination and signs every notification; the settings mark these “person only”. Agents never send a notice, never page anyone outside the plan's on-call list, and never edit a signed timeline entry.

Can we use our own response plan, playbooks and reporting rules?

Yes — the answers come only from your approved documents. Which rules apply, when each clock starts, the determination target, escalation and reminder times, the offline copy and who decides and signs are all settings, and every change is a new version approved by your CIP Senior Manager.

How long does it take to go live?

The Agentic Solution Engine builds and deploys it from your requirements and documents — your OT incident response plan, playbooks, zone drawings, vendor contracts and reporting rules — and it goes live once every quality gate has passed. We will walk you through it first on a tabletop scenario of your own.

See it on
your own playbooks.

We'll run OT Incident Response on your own response plan, playbooks and drawings, against a tabletop scenario you choose.