Practising the decision, not only the plan
Crisis exercises for OT environments, at the table or on real controllers in our model city YekCity. We build the scenario from your plant, not from a catalogue.
The one decision no plan makes for you
Contingency plans describe procedures. They get written in calm conditions. The situation in which they matter looks different: at three in the morning it is unclear whether the ransomware has already reached the control level, and somebody has to decide whether the plant comes off the network.
That decision is more expensive in OT than in IT. A data centre can be shut down and brought back up. A furnace, a substation or an interlocking cannot. The damage from a hasty shutdown can exceed the damage from the attack. On top of that comes the order that applies in OT: safety before availability, availability before confidentiality. A shutdown that leaves the process in an unsafe state is not an option, even where it would stop the attack.
That judgement can be practised before it arises.
Deciding without knowing all the facts
A tabletop exercise runs at the table, with the people who would decide on the day: operations management, maintenance, IT, security, communications and the management board. We set a scenario and inject a new situation every ten minutes.
The situations are deliberately incomplete. An engineer reports that an operator station has stopped responding. Ten minutes later a second site fails. After that a journalist calls. The point of the exercise is not to find the right answer but to notice which information would have been missing in order to find it.
What regularly stands out: the technology is rarely the problem. What is missing is the authority to stop a plant, or it is unclear when the reporting clock starts.
From the table to real controllers
An exercise does not have to end on paper. We have rigs on which an attack really runs, without your operation noticing any of it.
| Tabletop | Exercise on YekCity | Exercise in the plant | |
|---|---|---|---|
| What gets tested | Decision routes, reporting chains, responsibilities | Detection and response on real technology | Detection and response on your own technology |
| Length | half a day to a day | on request | on request |
| Who takes part | Operations, maintenance, IT, security, communications, management | SOC, control room, security staff | as for YekCity, plus plant responsibility |
| Technology involved | none | real controllers in our model city | spare or lab rigs of your technology |
| Risk to operation | none | none, the attack runs at our site | limited, only in an agreed window with abort criteria |
| Preparation | A preliminary conversation and a scenario draft | plus mapping your topology onto the rig | plus approvals, time windows, abort criteria |
| When it makes sense | the usual way in, because it costs least and uncovers most | when the attack really has to run, and the detection with it, without anybody losing production | only where the other two formats are not enough |
Who has to be at the table
The exercises that pay off do not have only IT at the table. The gaps sit at the interfaces. Behind each of these roles there has to be a person; a department on its own is not enough.
- Operations and control room. Knows the process and what a shutdown does to it. Without this role the exercise stays theoretical.
- Maintenance. Knows which maintenance accesses exist and who uses them. Often the source of the most surprising finding of the day.
- IT and information security. Brings the view of the office network and the crossings into the control level.
- Communications. Press, customers, staff. The calls arrive before the clarity does.
- Legal and compliance. Reporting deadlines, the regulator, documentation duties.
- Somebody with authority to decide. The role missing most often. Without it every exercise ends at the same point. That is then the finding.
Where the exercise turns up gaps in knowledge rather than gaps in authority, training is the next step.
How a crisis exercise is prepared
The preparation decides the value of the exercise, not the day itself.
A scenario from your plant
Not a standard case. We take your topology, your protocols and an attack route that would actually be possible at your site. A preliminary conversation and a look at the transitions is enough for that.
Roles and observers
Who sits at the table, who plays the outside world, who records. Observers do not intervene; they note when which decision was made and on what basis.
The day itself
The situation develops on a beat, with injects from the press, the regulator and suppliers. No winning, no losing, and expressly no assessment of individuals.
Write-up
Verbally the same day, in writing afterwards: which decision hung on which missing information, and what should change in the contingency plan as a result.
What the record says
The exercise record is the document you work with afterwards. It rates no individuals and names no names.
| Time | Decision | Basis | What was missing | Measure |
|---|---|---|---|---|
| 09:15 | no shutdown, keep observing | the assumption that the control level is unaffected | evidence for it; there is no capture at the crossing | set up passive recording at the IT/OT crossing |
| 09:20 | escalation to management | contingency plan, section 4 | the role named there no longer exists after the reorganisation | bring the contingency plan onto the current organisation |
| 09:40 | report to the regulator prepared | the communications team's assessment | the start of the deadline was disputed | define the start of the deadline in the reporting process |
The rows come from typical exercise runs and are anonymised.
A documented exercise record is used in practice as evidence of regular review, towards the regulator, for the exercise programme under ISO 22301 and towards cyber insurers. Whether it suffices in a given case is for the assessing body to decide. How the evidence is built up as a whole is on the compliance page.
Taking the substation off the network
A scenario we run often, because it happens. An alert comes up in the SOC pointing at access on the process network of a substation. The response that would be right in the office network suggests itself. The affected segment is isolated, which means cutting the telecontrol link. Technically a single person in the SOC can do that, because on the network that path looks like any other.
In OT the measure inverts its own effect. The substation keeps running, protection equipment and controllers work locally. What is gone is the control centre: no readings, no alarms, no remote switching. Whoever is already inside the substation stays there. The cut takes away your own team's ability to respond and takes nothing from the attacker.
In the exercise this surfaces at a particular point. Somebody asks who is actually allowed to approve that intervention. An answer to it is rarely written down anywhere, and that is the finding. It is worth a line in the record.
What usually changes afterwards: the telecontrol link gets an approval rule of its own, separate from the other network paths. The SOC playbook then states who has to be reached before a cut. And the control centre gets a defined fallback route for as long as the link is down.
From research, demonstrators and real plants
The scenarios come out of the same work as our OT monitoring: substations, control rooms and interlocking landscapes, and the analysis of documented attacks on supply networks and rail systems.
The rigs come out of our security lab. YekCity was built from the analysis of real attacks on power and utility networks and represents a model city with real controllers. For crisis exercises at utilities and municipalities it is the rig on which a scenario can be played through completely: from the first telegram that stands out to the question of whether to shut down.
The scenario itself we build for you. An exercise that works with somebody else's topology and somebody else's protocols tests nothing that could happen at your site. We take your plant as the template and put the attack route on it, even when the exercise runs on YekCity.
Common questions about OT exercises
A moderated exercise at the table, with no technology. A facilitator sets a scenario and, at a fixed rhythm, new situations. Participants decide in their real roles. What gets tested are decision routes, reporting chains and responsibilities. In German it also goes by Krisensimulation; in English it is a TTX.
The tabletop exercise yes, it touches no plant. Exercises with a real attack run on our demonstrators or on your spare rigs. In the production plant only in an agreed window and with abort criteria fixed in writing beforehand.
Once a year as a tabletop, plus a technical exercise when something substantial has changed: a new plant, a new service provider, a new control system. Regulation requires regular review but names no fixed interval.
The terms overlap. A tabletop is usually shorter, focused on one scenario and driven by discussion. A staff exercise works more with real communication channels and with the crisis team in its actual composition, over a longer period. We facilitate both and follow what you want to examine.
Then the exercise is the better way in than the writing. A plan that comes out of a situation played through is shorter in the end, and it gets used.
Certifications and memberships.
Talk to us.
A first conversation usually takes 30 minutes. We look at where you stand and tell you frankly whether we are the right partner.